This section describes how you modify or update a kafka target to support connections to Apache Kafka distributed streaming target.
-
Select the EDMM and expand Hosts, Servers, Tables node.
-
Choose a predefined Kafka server and expand the node.
- Right click any available topic, and select Properties.
The Target Kafka Properties dialog opens.
-
Enter information for each field on the Target Kafka Properties dialog. Refer to Target Kakfa Properties table for field information.
Table 1. Target Kafka Properties Parameter Description Topic A Kafka topic name to where Kafka cluster stores streams of records. Message format: Sets the format for messages passed to Kafka: - CSV (No Schema): Comma Separated Values
- Schema Registry (Avro): encodes messages in Apache Avro format, which conforms to field names and types defined in the schema registry.
Images for update Sets how Connect CDC writes Before and After values for all columns to the Kafka Topic when it captures an update statement:
- After image only: Default. Connect CDC only writes After values to the record.
-
Both in the same record: Connect CDC writes both Before and After values to the same record. Only available for targets set to use the Json message format.
-
Each in a separate record: Connect CDC writes both Before and After values to different records. Connect CDC sets the before value entry in the sv_manip_type column to B.
Always send LOB Always writes a Large Object (LOB) after image row whether or not an update causes a change. If the LOB column value contains a change, the value in the before image row contains an empty string.-
No change value: Sets the value written to the LOB column value in the before image row when the LOB column value has not changed. This value only applies when you select Always send LOB. For After image only option, the no change value is not used. Connect CDC ignores this setting when Images for update is set to After image only.
Max Lob size Sets the maximum size in bytes of Large object (LOB) values written to Kafka, which has a default message size of 1MB. The default value of this parameter is 50000 bytes (5 KB), and valid values range from 100 bytes to 2GB. If a LOB column value is larger than the Max Lob size value, Connect CDC truncates the value to the maximum LOB size before committing it Kafka and issues a warning message to the kernel log.
Character set (required) Null string Connect CDC differentiates between null and empty strings and identifies an empty string as no character. Quote character The default is a double quote ("). The quote character has the same requirements as the delimiter character. Delimiter - Field (required) - The default file delimiter character is comma. You can specify multiple printable characters as delimiters. The default size value of the field delimiter is 10 and 20 in the case where escaped characters are used. At least one printable character and two for the escaped character should be entered as a delimiter.
- Record (required) - Sets the trailing character used to separate each row data set. The default is backward slash n (\n). Supports a maximum of 10 characters (20 characters if escape characters are used).
- Click OK or Apply.