Replication pipeline - connect_cdc_sqdata - 4.1

Connect CDC (SQData) Architecture

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect CDC (SQData)
Version
4.1
Language
English
Product name
Connect CDC (SQData)
Title
Connect CDC (SQData) Architecture
Topic type
Installation
How Do I
Copyright
2024
First publish date
2000
ft:lastEdition
2024-02-07
ft:lastPublication
2024-02-07T13:24:57.122078

The Pipeline refers to the flow of data from Source to Target platform and is essentially identical regardless of the platform operating systems and type of Source and Target Datastore.

Apply Engine and Replicator Engines

While there are some platform specific options the picture remains the same Publish and Subscribe model sequence of Capture, Publisher, Subscribing Apply or Replicator Engine components. There are really only two other variations.

  • Kafka (and Kafka compatible) Targets and HDFS Targets require the platform running the Engine to be Linux as it is the only operating system supported by the Open Source librdkafka C Language API.
  • When a Replicator Engine operating in Parallel IMS Distributor mode writes to Kafka with downstream Apply Engine Kafka consumers both the Replicator and the Apply Engines must run on Linux due to the requirement for librdkafka.

Replicator Engine