DB2 to Kafka AVRO with Confluent - connect_cdc_sqdata - Latest

Connect CDC (SQData) Replicator Engine

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect CDC (SQData)
Version
Latest
Language
English
Product name
Connect CDC (SQData)
Title
Connect CDC (SQData) Replicator Engine
Copyright
2024
First publish date
2000
ft:lastEdition
2024-08-01
ft:lastPublication
2024-08-01T16:58:30.842568

Replicate DB2 changed data (CDC) for the IVP_HR database, EMPLOYEE and DEPARTMENT tables into unique AVRO formatted Kafka Topics with default partitioning based on table keys. All changes to the source tables are Replicated and no filtering is supported. The values for TOPIC and SUBJECT used here are arbitrary but were selected based on the source Table Names, source application and schema source, in this example, the EMPLOYEE and DEPARTMENT Tables, a DB2 "IVP_HR" Database and SQData respectively. Operation of the Producer will be optimized using one worker thread.

Example
----------------------------------------------------------------------
-- Name: DB2TOKAF:  Z/OS DB2 To Kafka AVRO on Linux
-- Client/Project: client/project
----------------------------------------------------------------------
--       Change Log:
----------------------------------------------------------------------
-- 2019-07-01 INITIAL RELEASE using Kafka Replicator Engine
--
----------------------------------------------------------------------
--       Replicate Source/Target
----------------------------------------------------------------------
REPLICATE
   DB2 cdc://<src_host_name>:<src_sqdaemon_port>/<publisher_name>/<replicator_engine>
   TO
  AVRO CONFLUENT 'kafka:///<prefix>_*_<suffix>/key'
  WITH 1 WORKER
;
----------------------------------------------------------------------
--       Processing Option References
----------------------------------------------------------------------
OPTIONS
  STRIP TRAILING SPACES
  CONFLUENT REPOSITORY 'http://confluent.repo.<domain>.com:8081'
;
----------------------------------------------------------------------
--       Source References
----------------------------------------------------------------------
MAPPINGS
   SOURCE 'IVP_HR.EMPLOYEE'
          TOPIC 'EMPLOYEE'
          SUBJECT 'SQDATA_DB2_HR_EMPLOYEE'
          ALIAS 'EMP'
  ,SOURCE 'IVP_HR.DEPARTMENT'
          TOPIC 'DEPARTMENT'
          SUBJECT 'SQDATA_DB2_HR_DEPARTMENT';
          ALIAS 'DEPT'
;