Determine Kafka topic data requirements - connect_cdc_sqdata - Latest

Connect CDC (SQData) Kafka Quickstart

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect CDC (SQData)
Version
Latest
Language
English
Product name
Connect CDC (SQData)
Title
Connect CDC (SQData) Kafka Quickstart
Copyright
2024
First publish date
2000
Last edition
2024-07-30
Last publish date
2024-07-30T20:00:09.892433

While your Kafka Topic set may eventually contain lots of different types of data, Precisely recommends that you start with only a few sources. That usually means a small number of Relational database tables, a subset of segments in a legacy IMS database or a few VSAM file Copybooks. Since your data may eventually come from different platforms, pick just one to get started.

When implementing Connect CDC (SQData)'s change data capture to collect the data sent to Kafka, it is easy to forget that downstream Kafka Consumers may need access to data that has not changed in some time and therefore has never been published as a Kafka topic. There are several methods for performing an "Initial Load" and they vary depending on the original source of data, be it hosted on the Mainframe, Linux or Windows.