Prepare special load and refresh apply engine - connect_cdc_sqdata - Latest

Connect CDC (SQData) Change Data Capture

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect CDC (SQData)
Version
Latest
Language
English
Product name
Connect CDC (SQData)
Title
Connect CDC (SQData) Change Data Capture
Copyright
2024
First publish date
2000
Last edition
2024-09-05
Last publish date
2024-09-05T15:00:09.754973
When using a cloned version of the CDC Apply Engine for the Load/Refresh process only a few modifications are required to the Apply Engine script:
  • The cdc:// connect string must be modified, generally using parameters specified when parsed, to specify the "Unload" Publisher.
  • Comment out the primary CASE statement WHEN clause for all but the required segments.
  • If the target is an RDBMS but the CDC Apply Engine target DATASTORE is configured with the FOR INSERT then each row loaded or refreshed will be inserted or added to any existing rows in those tables. Precisely recommends identifying the refresh/initial load target rows with a column containing an "L" rather than the usual CDCOP code (I,U,D) if that column exists in the target.
  • If the target is Kafka, the best solution is to indicate with a metadata object that the topics contain load/refresh data.
Note:
  • If the Load/Refresh process will be a regular occurrence consideration should be given to the creation of more specialized Load/Refresh Engines as well as the use of the Engine Controller SQDAEMON for remotely starting the Engines from zOS as part of an Unload/refresh JOB stream.
  • If the target is an RDBMS and it is a Refresh and the CDC Apply Engine target DATASTORE is configured with the FOR CHANGE then you must drop and recreate the existing target table(s). That will allow compensation to fully resynchronize the Source and Target.