Pipe-It Command Line Interface

Pipe-Itc

Pipt-Itc is the command line interface of Pipe-It. This allows the execution of a Pipe-It project from the command line of the operating system without opening the graphical interface.

Pipe-It is built with model-view paradigm. The model part of a project is disconnected from the view part and they are stored separately. The model part consists of actual files, their locations on disks, their relationships with the launching of programs and the mechanisms of launching these programs. This information is stored in the Pipe-It project model file (*.ppm). This it the only information needed to execute a project.

The view part to the project defines the location, size and colour of the graphical elements. This is only used to display the project graphically onto a screen. This information is stored in a Pipe-It project view file (*.ppv). This information is not required to execute a project.

Pipe-Itc, in its simplest form, takes the ppm file as an argument and runs the project. This command line would look like:

Pipe-Itc myproject.ppm

This command line can be executed from a Windows cmd window, a Mac OSX terminal or a Linux terminal.

The command line can be placed in a Windows batch file and executed in a loop, with or without some logic. And similarly in a shell script on Mac or Linux.

The command line can be invoked from a scripting or programming environment, for example from an Excel VBA macro.

Any application or environment can has the ability to launch command lines on any operating system can launch and execute Pipe-It projects in this manner.

When the project executes, any Pipe-It internal or external applications that are setup to be launched via Processes will be launched. In the proper sequence or parallel as specified. If an error is encountered in Pipe-It, or any of the launched applications, it will cause the Pipe-Itc instance to be halted and report a failure to the operating system. Or to the environment that launched it in the first place.

Since Pipe-It has the ability to launch any application, it can launch Pipe-Itc too! This is an unique capability allowing design of modular solutions and nested optimisations as we will describe soon. # Command line options The first argument to Pipe-Itc is the name of the model (*.ppm) file. To get a comprehensive list of the command line options you can instead specify the —help option. The following output will result:

      **************************************************************
      *                                                            *
      *                      ~~~ Pipe-Itc ~~~                      *
      *             Command line interface for Pipe-It             *
      *                                                            *
      *              Version 1.4.0 Compiled: 20131112              *
      *                                                            *
      *     Petrostreamz AS, Trondheim, Norway                     *
      *     Phone: +47 7384-8080, Web: www.petrostreamz.com        *
      *     (c) Copyright Petrostreamz AS 2006-2013                *
      *     All Rights Reserved                                    *
      *                                                            *
      **************************************************************
1: Pipe-Itc.exe --help
2: Pipe-Itc.exe --solver-list
3: Pipe-Itc.exe Project.ppm --list
4: Pipe-Itc.exe Project.ppm [--pc composite_name] [--quiet]
5: Pipe-Itc.exe Project.ppm [--pc composite_name] [--quiet] --po input.ppo [--direction dir] 
                            [--solver solver] [--case-matrix cases.tab] [--save-ppo result.ppo] [--save-history hist.tab]

--help prints the help message.

--solver-list prints available optimization solvers.

Project.ppm argument is always the Pipe-It project file to be used.

--list prints list of all available composites.

--pc comp_name (optional) is the desired composite to be executed.
	If not supplied on the command line the top-level composite is used.

--po input.ppo (optional) is the name of the desired 'input' Optimization Ppo file 
	and also the flag to launch the Optimizer to control the Runner.

--quiet (optional) not to print Runner messages, only Optimizer and warnings

--nobanner (optional) not to display the banner at startup.

--direction dir (optional) allows to change direction of solver, available: Min, Max and Feasible

--solver solver (optional) allows to use another solver, see list of available with --solver-list

--case-matrix cases.tab (optional) allows to use load data from 
	tab delimited file into Case Matrix solver. Also if 
	the option is applied, it automatically applies Case Matrix solver.
	CSV files are also supported. Type of file is determined by extension '.tab' or '.csv'
	First row should contain variable names to have columns assigned to them.

--save-ppo result.ppo (optional) allows to save resulting Optimization as 'output' Ppo file.
	Will contain latest values of all variables and complete optimization history.

--save-history hist.tab (optional) allows to save optimization 
	history into tab delimited file or CSV file. Type of file is  
	determined by extension '.tab' or '.csv'. First row contains variable names.

Return codes:
	 5 BROKEN
	 4 INCOMPLETE
	 3 UNKNOWN
	 2 RUNNING
	 1 PENDING
	 0 UP_TO_DATE
	-1 INACTIVE

While some options are self-explanatory in the above output, some may need additional explanation:

(--list) Prints a list of all available composites. In Pipe-It the user can navigate to any composite in a large project and execute it. Runner will limit execution of the project to within that composite (and nested sub-composite). To achieve the same effect in Pipe-Itc the user will use the

(-—pc) option with the exact _path_ of the composite. Instead of memorising this path and pasting it into the command line, this option allows the user to list the names, that can later be used with the (--pc) option.

(--pc) As explained above, this option will equivalent to the user graphically navigating the the specified composite in the chosen project and executing it.

(--po) The Pipe-It Optimizer allows the user to perform multiple runs of their projects. The variables to change and their relationships to each other it contained in the optimzation case file (*.ppo). This option will allow such multiple launches to be invoked using the command line interface. Pipe-Itc will load the project, load the optimization case and run it as many times as specified in the associated ppo file. This may be as simple as a case matrix, a random set of iterations or a complete optimization using the solver mentioned in the ppo file (the last used solver before the ppo was saved).

(--direction) This allows the direction of the optmization to be set irrespective of what was saved in the ppo file.

(--solver) This allows the the user to override the solver that was set when the ppo file was last saved. This might be used when multiple solvers need to be tested for the same optimization case.

Various uses of Pipe-Itc

The ability of Pipe-It to launch Pipe-Itc as and the command line options of Pipe-Itc allowing choice of optimization gives enormous possibilities:

Modular design of Pipe-It projects: Some functionality of a Pipe-It project might be needed at multiple points in the project. Rather than duplicate that part of the project it would make more sense in using a Pipe-Itc instance to run that external project from a master project. This ensures that the exact same workflow is executed at different parts of a large project. Any changes to the workflow can be localised to this single sub-project and updated efficiently. It is analogous to calling a subroutine or function in programming.
Nested Optimization : Normally a Pipe-It project used with optimization will use a single solver specified in the ppo file. With large integrated projects the same solver may not be ideal for each physical system modelled in the the various parts. Also the sub-parts of an integrated model may need local optimization to be occurring within a single iteration of a "global" optimization. Such scenarios are easily handle by use of Pipe-Itc and an associated optimization using a --po option.
Efficiency of project runs: Some overhead is used while executing Pipe-It projects from the graphical interface. Projects requiring tens of thousands of cases can benefit from running Pipe-its in batch files.
Using Pipe-Itc allows other programs to control Pipe-It projects.
If a Pipe-It project is tried-and-tested, it is safer to use Pipe-Itc since the user is sure nothing is modified by mistake.
A Pipe-It project can use Pipe-Itc to alter the execution sequence of contained Composites. The functional Composites are laid out in any manner and order but should be both disabled & unconnected. The functional branch of Pipe-It uses a series of Pipe-Itc processes, each containing the appropriate Composite using the --pc option, with dummy intermediate resources, in the desired order.
An instance of Pipe-Itc with the appropriate ppm could be registered on the schedular (cron on Linux or Mac) to be launch at specified intervals. This might, for example, be used to update data from a database once every day.
A Pipe-Itc instance could be registered as a Windows service (daemon in Linux) doing nothing, since it runs nothing if nothing has changed, but triggering a run whenever a particular file has been updated. The file update could be triggered by a copy command from another user-controlled project running on the same or another computer on the network.

Guidelines on use of Pipe-Itc

If a Pipe-Itc Process is used to run a Composite in the same project, the said Composite should be disabled. Else it will be run twice.
A Pipe-Itc Process referring to the same project as the one it resides in will do nothing unless it refers to a Composite too using the --pc option. This is to prevent indefinite recursive runs.
A Script Assistant Short-cut (Utility) is available on the toolbar for dropping instances of Pipe-Itc on the canvas.
If in doubt use the --help option of Pipe-Itc to get a description and syntax (discussed above).