Connect CDC (SQData) provides an efficient and effective mechanism for propagation and/or replication of data utilizing two components; Data Capture Agents and what we refer to as an Engine. Connect CDC SQData supports both homogeneous and heterogeneous environments including zOS (mainframe), Linux and Windows operating systems. Source and target data may utilize different data structures, database management systems and operating systems. Supported sources including Db2/z, IMS, VSAM, Db2/LUW (UDB) and Oracle. Targets include all supported sources as well as any relational database supported by ODBC and streaming Kafka and HDFS targets. Relational databases can be running on premise or in the cloud such as AWS RDS. Engines need not run on the same platform as the target and are most commonly run on Linux.
While the configuration and operation of the Apply Engine is virtually identical on all platforms, the Connect CDC SQData Capture Agents operates under constraints that vary from one platform to another. To accommodate those constraints Capture utilizes various forms of Log Readers and Exits.
Near Real-Time data capture is accomplished on most platforms using the transaction log, maintained by the database manager for the purpose of database recovery. Historically, these logs have been the source of data for other purposes, including audit or replication once the database manager has been configured to include additional information required by the non-recovery related activities. The impact of larger logs is a minor inconvenience compared to the additional processing required by database trigger capture methods.