When you run a Quality project or export a Quality project to batch, you can configure your project or processes to run using pipes. Using pipes in a project moves data from process to process without creating interim output entities. By eliminating these entities, you can reduce disk space usage and improve performance. Running pipes also allow large data files to run with lower risk of running out of disk space.
Because there are no interim output entities in a piped process flow, you cannot review the results of intermediate processes for debugging or troubleshooting. Therefore, piping is only recommended after the process has been successfully tested and tuned.
When you are ready to run the full data set, export the project as a piped process and run it as a batch process. The piped process will use less space as it will only write the final output file and some temporary files to the disk. The process should run faster as a batch process.
To run projects or processes using pipes
-
Select the project or processes and select Run. The Execute Process window opens.
-
Select Use pipes.
-
Select Run or Background to run the process.
To export a project using pipes
-
Right-click the project and select Export. The Export Project Options window opens.
-
In the Script Options section, select the Use pipes check box.
-
Click Export.