/DEFAULTFLOW - Connect_ETL - 9.13

Connect ETL Data Transformation Language (DTL) Guide
Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect (ETL, Sort, AppMod, Big Data)
Version
9.13
Language
English
Product name
Connect ETL
Title
Connect ETL Data Transformation Language (DTL) Guide
Copyright
2023
First publish date
2003
Last updated
2023-09-11
Published on
2023-09-11T19:01:45.019000

Purpose

To indicate that all applicable data flows connecting a single file target to single file source should be automatically treated as direct data flows.

Format

/DEFAULTFLOW dataflow_optimization

where

dataflow_optimization = {DIRECT [VERIFY]} {NOTDIRECT }

Location

The option may appear anywhere in the job definition.

Defaults

When the DIRECT or NOTDIRECT clauses are not specified, the default flow is NOTDIRECT. When the data flow is NOTDIRECT, data flows connecting a single file target to single file source are not treated as direct data flows.

Notes

When the DIRECT keyword is specified, all data flows connecting a single file target to a single file source are treated as direct data flows. Direct data-flows bypass writing the intermediate file to disk for better performance. This attribute is applicable only for data flows connecting a single file target to a single file source.

When the VERIFY keyword is specified and direct data flow optimization is enabled, Connect ETL generates a warning that identifies the data flows connecting a single file target to a single file source that cannot be treated as direct data flows and provides the reasons.
Note: Direct data flows apply to input and output files only. When using the VERIFY keyword, no warning is generated when 2 tasks in a flow contain non-file input and output. For example, when the VERIFY keyword is specified and task1 has DB output only and task2 has DB input, no warning is generated.

Data-flows cannot be optimized into direct data-flows in the following cases:

  • The source or target of the data flow has more than one connection. If a task has two data-flow connections to two other tasks from its single target file, for example, neither of these data flows can be optimized into direct data-flows.
  • The source of the data-flow contains a header layout.
  • The file name of the data flow target file contains a wildcard pattern.
  • The DTL job is customized with a third-party language.
  • Optimizing the data-flow into a direct data-flow creates a job cycle, which causes an infinite loop in the run sequence.

Example

/DEFAULTFLOW DIRECT VERIFY
Data flows are optimized to direct data flows whenever possible.
/DEFAULTFLOW $USEDIRECTFLOWS VERIFY

The data-flow option can be specified via an environment variable.