Connect CDC (SQData) provides many options to control the capture process. Some of the more relevant features include the following:
- Changed data routing is the ability to specify that changes to a specific object such as a particular IMS segment, relational table or file will be published to a specific Engine via TCP/IP. This permits both processing and network transport workload balancing.
- Auto-disable allows critical transactions/programs to continue processing in the event an error occurs during the data capture process. Typically, these errors involve one or more of the transient datastores used to manage captured data before it is published.Note: When auto-disable is initiated, messages are written to both the operator’s console and the program job log indicating that the auto-disable event occurred and the associated error code. Optionally, a user specific WTO message may be written when auto-disable is triggered.
- Auto-Commit applies to batch ONLY IMS/DLI programs where a single unit-of-work may consist of millions of updates before a commit is performed. This feature is used to prevent large units-of-work from overloading a system LogStream by specifying the optional auto-commit frequency (i.e. how many changed data records to process between auto-commit events).Important: The auto-commit feature is only intended to provide a temporary solution. Data integrity can be compromised if changed data is auto-committed and a DL/I batch program subsequently abends or rolls back uncommitted updates. In other words, there is no coordination between the IMS and the LogStream as far as the commit scope is concerned. Precisely recommends using the IMS Log Reader Capture for changes made by long running DLI batch programs. Contact Precisely https://www.precisely.com/support before using the Auto-Commit option.
- Program and transaction Include/Exclude provides for the inclusion or exclusion of specific programs or transactions from the data capture process. For example, eliminating the capture of deletes performed by purge or data cleanup programs.
- Other Database specific Include/Exclude options are also supported such as Db2 Correlation ID, Plan and Authorization ID.
- Multi-Target Assignment allows data captured for a single change to drive multiple downstream processes running on one or more target platforms. This can be accomplished using a variety of techniques involving both capture agents, downstream Engines with multiple target datastores, multiple Engine subscriptions and intermediate splitter/distribution engines. Visit Precisely https://www.precisely.com/support for assistance in determining how best to implement Multi-Target processing.
- Changes to source data structures are inevitable. When new columns are added to tables or a new fields added to IMS segments or Files already configured for captured the additional data is automatically included in the captured data. The Engine will ignore the new column/field until the source and target Descriptions referenced by the Engine are revised and implemented. The impact of other types of changes in Source data structures are summarized in the following table:
Change Impact Required for Capture Required for Engine New Table None ALTER table for supplemental logging Source DDL*, Re-Parse New Index None None None unless key change New Segment None None if capture is specified at DBD level Updated DBD Source; IMS Segment description (copybook) IMS DBD hierarchy change None if change at bottom of hierarchy None if capture is specified at DBD level Updated DBD Source; IMS Segment description (copybook)
* See Schema Evolution