Dynamic mappings - connect_cdc_sqdata

Dynamic mappings can provide runtime invocation of a user process that provides the SOURCE MAPPINGS parameter values in JSON format for each fully qualified SOURCE object (table) name not specified by Static MAPPINGS.

Syntax

MAPPINGS EXIT
/+
<mapping_exit_name> [<parm1> [, <parm2>] ...]
+/

Note:

One or more Static MAPPINGS Source(s) may precede the MAPPINGS EXIT statement and will take precedence over the Exit.
The content between /+ and +/ will be taken verbatim, if necessary a space will be added at the end and the quoted fully qualified source object (table) name will be appended. The resulting string will be used to invoke the specified mapping exit process. The stdout of that process will be parsed by the Replicator Engine as JSON.
While typically used to accommodate "new" source objects for which return JSON object pairs values have not been specified by Static MAPPINGS, the exit may also perform exception processing, that terminates execution of the Replicator.
Kafka may or may not be configured to allow a PRODUCER to create unregistered Topics.

Example 1

The Replicator Engine will call the exit "mysqdata_mappings" along with a parameter specifying the environment. This will occur both at startup and again each time it receives a metadata record from the Capture indicating a new or altered Source object not accompanied by STATIC MAPPINGS.

MAPPINGS EXIT
/+
/var/sqdata/app/mysqdata_mappings --env=prod
+/
;

The Replicator Engine will then call the Exit one or more times for each Source object published and when the metadata for any Source object changes.

/usr/local/bin/mysqdata_mappings --env=prod "IVP_HR.EMPLOYEE"

The Exit might, for example then return:

{ "subject":"SQDATA_DB2_HR_EMPLOYEE", "namespace":"sqdata", "alias":"EMP", "topic":"HR" }

The “subject”, “namespace”, "alias" and “topic” in the root of the returned JSON object are recognized. If present they must be of type “string”. Any other fields are silently ignored and the result will essentially identical to a Static Mappings statement, had it been specified:

MAPPINGS
  SOURCE 'IVP_HR.EMPLOYEE'
          TOPIC 'HR'
          SUBJECT 'SQDATA_DB2_HR_EMPLOYEE'
          ALIAS 'EMP'
          NAMESPACE 'sqdata'
;

Example 2:

By default the Replicator Engine will automatically generate a new Kafka message Topic when it encounters a source object not accommodated by a Static MAPPINGS or or by logic within a Dynamic Mappings exit. The MAPPINGS EXIT can be used to terminate the Replicator rather than generate the Topic.

MAPPINGS EXIT
/+
/var/sqdata/app/terminate_on_unknown_source.sh
+/

The Exit might, for example then return:

{ "error":"SQData Replicator has encountered unknown source table: <table_name>" }

Dynamic mappings - connect_cdc_sqdata - Latest

Connect CDC (SQData) Replicator Engine