Required options within a task definition file - Connect_ETL - 9.13

Connect ETL Data Transformation Language (DTL) Guide
Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect (ETL, Sort, AppMod, Big Data)
Version
9.13
Language
English
Product name
Connect ETL
Title
Connect ETL Data Transformation Language (DTL) Guide
Copyright
2023
First publish date
2003
Last updated
2023-09-11
Published on
2023-09-11T19:01:45.019000
The /DTL and /END task definition command options must be included within each task definition file. In addition to these minimum option requirements, the task should include a /TASKTYPE option.
Note: If a source or target definition or both are not specified with a task definition file, Connect ETL reads from stdin or writes to stdout or both. If under these circumstances data is not directed into the command via a pipe or redirection, Connect ETL may appear to be hung.

DTL and /END

At a minimum, every task definition file must include the /DTL and /END task definition command options:

  • If the /DTL command option is not included in the task definition file, the task is run using the Software Development Kit (SDK) language.

SDK is an older text-based command language that is the precursor to DTL. SDK does not offer full compatibility with the Connect ETL IDE, whereas DTL offers full compatibility with the Connect ETL IDE. If a DTL task is run without the /DTL option and interpreted as an SDK task, unexpected results and errors could be generated.

  • If the /END command option is not included in the task definition file, the following unexpected behavior could occur on Windows: Connect ETL could abort when run under the Windows command prompt or Connect ETL could hang as it waits for input when run under Cygwin.

/TASKTYPE

To define one of the following Connect ETL task types, the /TASKTYPE task definition command option must be included in the task definition file:

  • Aggregate - Aggregate tasks create sets of records, which are grouped based on equal values within specified fields or values, and generate summaries based on these sets of data.
  • Copy - Copy tasks extract records from each input source and send the extracted records to the output target. Copy tasks can be used to concatenate input sources without changing the order of source records. They also allow record filtering and reformatting when no sorting, aggregation, or joining is required.
  • Join - Join tasks evaluate data from two data source sides, a left side and a right side, each of which can contain multiple sources, and join these sources based on specified left side and right side fields and values to create targets that contain fields from both data source sides.
  • Merge and sort - Merge and sort tasks sequence source records based on specified key fields and output these records to target in ascending or descending order.

One /TASKTYPE option is required per task definition.

For examples of task definition files that include each of the task types specified within /TASKTYPE options, see Task Definition File Examples.