Once an Apply Engine script has been through the normal development and testing cycle, no target Exceptions should be encountered. When one occurs it is almost certainly due to a change in the nature of a target datastore or the environment that will require modification of the script. The recommended the approach is to allow the engine to simply fail. While this will ensure that no data is lost, its can, depending on the volume of data involved, require immediate attention. If the problem is environmental, such as a lack of space, allowing the engine to fail is usually the best approach. Once the problem is resolved, the Engine can be restarted and it will automatically resume from the point of failure.
Other target exceptions, particularly those involving relational target datastores can be accommodated in other ways:
Changes to target structure - for example a new column is added and no default value is specified.
Changes to target keys - for example a unique index is added that results in duplicate keys.
These types of exceptions can be handled by allowing the Apply Engine to create an Exception Datastore containing the SQL statements that failed when they were attempted by the Engine. This makes it possible to make the necessary modifications to the target datastore or even to the SQL data itself and then simply process the SQL statements directly outside of Connect CDC SQData.
The generated SQL will contain a where clause containing keys from the database catalog ensuring that all necessary keys. Data columns in the SQL will come from the Target DESCRIPTION specified in the script.