One additional activity often overlooked when discussing Change Data Capture is the initial load of the Target datastores and the methods employed to achieve full synchronization of the source and target. Modifications to Source datastores, overlooked requirements, business rule changes affecting filters or transformations or even operational issues may also surface the need to Refresh all or a subset of the CDC/Apply targets. Various methods that support Initial Load and Refresh should be considered based on all applicable factors including performance, ease of configuration and operational impact:
- Native database unload/reload utilities may be available to unload the source datastore and load the target datastore, they are however generally restricted to the same type (RDBMS, IMS, etc) source and target datastore.
- Connect CDC (SQData) IMS Unload Utility . When the source and target datastores are not identical, Precisely recommends that a special version of the already tested CDC Apply Engine script be used for the initial load of the target datastore. Contact Precisely https://www.precisely.com/support to discuss both the benefits and techniques for implementing and perhaps more importantly maintaining, a load/refresh Engine solution.
- A special Connect CDC (SQData) Unload engine that reads the source datastore locally and writes records to be loaded by a database utility.
- Third party remote disk mirroring, often the only practical solution when large scale disaster type replication systems are being implemented.Note: The method selected for the initial load of the target datastore must also consider concurrent source database activity. The source capture and target apply process must ensure that source and target synchronization is achieved, often with a "catch-up" phase during which Connect CDC (SQData) will perform compensation or in the case of streaming targets clear identification of Load/Refresh data payloads for consumer side processing.