RunMultiFileLoader Job initiator implementation strategy - Connect_ETL - 9.13

Connect ETL Vertica Load Guide

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect (ETL, Sort, AppMod, Big Data)
Version
9.13
Language
English
Product name
Connect ETL
Title
Connect ETL Vertica Load Guide
Copyright
2023
First publish date
2003
ft:lastEdition
2023-09-11
ft:lastPublication
2023-09-11T19:09:11.694437

When implementing the runMultiFileLoader job initiator script strategy, the transformation and load workflow is as follows:

  1. The runMultiFileLoader job initiator script, runs the configuration file, creates the required named pipes based upon the number of standard input files, and submits an instance of the Connect ETL transformation and load job for each named pipe.

    Each parallel transformation and load job reads the named pipe and calls the Connect ETL transformation task and the custom load task

  2. Using the named pipes as input, the Connect ETL task transforms the data.

    The following example reformats the EBCDIC-encoded data on the named pipe into ASCII-encoded standard input.

  3. Using the standard input file as input, the Connect ETL custom load task, calls the database load script.
  4. Using environment variables, the database load script builds the vsql command that loads the data from the standard input file directly into initiator nodes of the Vertica database.