Select the EDMM and select New Server from the context menu.
On the Servers Properties dialog select Kafka from the dropdown list in the DBMS type field.
Enter information for each field on the Server Properties dialog. Refer to Server Properties dialog for Kafka for field information.
Parameter |
Description |
---|---|
Message Format |
Sets the format for messages passed to Kafka:
|
Omit null columns |
Discards data resulting in a null value from the message. Only available for targets set to use the Json message format. |
Multiple Tables to 1 Topic |
If selected, Connect CDC maps all tables to the specified name in the topic field. Available when messages are sent in CSV or Json format. |
Topic |
A Kafka topic name to which Connect CDC writes changes. |
Key |
Key determines the Topic partition behavior:
|
Prefix Table Name |
If selected, prepends the source table name as a prefix to the Kafka message. |
1 Topic Per Table |
If selected, Connect CDC only maps one table to any single topic. Available when messages are sent in Json or Schema Registry (Avro) format. |
Brokers |
Comma separated list of names of Kafka nodes with active ports in the Kafka cluster. For example: staging1-master.hostfarm.net:6667 |
Config File |
Path to the local Kafka configuration file that contains Schema Registry URL. Connect CDC typically detects the Schema Registry vendor based on the installed client jar libraries. To set a specific vendor for schema registry, add the following parameters to the Kafka configuration file. For more information see, Sample kafka configuration.
|
Use Kerberos |
Enables Kerberos authentication support. Kerberos settings include:
|
More options ... tab |
|
Images for update |
Sets how Connect CDC writes Before and After values for all columns
to the Kafka Topic when it captures an update statement:
|
Always send LOB |
Always writes a Large Object (LOB) after image row whether or not an update causes a change. If the LOB column value contains a change, the value in the before image row contains an empty string.
|
Max Lob size |
Sets the maximum size in bytes of Large object (LOB) values written to Kafka, which has a default message size of 1MB. The default value of this parameter is 50000 bytes (5 KB), and valid values range from 100 bytes to 2GB. If a LOB column value is larger than the Max Lob size value, Connect CDC truncates the value to the maximum LOB size before committing it Kafka and issues a warning message to the kernel log. |
Asynchronous Options |
Enables Connect CDC to apply row changes to a Kafka target asynchronously, which means Connect CDC submits changes to Kafka without waiting for the execution results for each row. This also means Connect CDC does not calculate the execution results for each change until either (a) the number of rows changed reaches the number of rows set by the Cache Size Limit, or (b) there are no more rows to apply. This approach has been shown to significantly increase performance, but Connect CDC does not apply rows in the batch resulting in an error and reports the error when it checks execution results, not when it submits the change to Kafka. Note: Do not use asynchronous apply for data connections that require
immediate error handling or reporting; Connect CDC does not log
errors or resubmit data if a row change fails due to errors other
than a lost connection.
|
Cache Size Limit |
Sets a limit on the number of rows Connect CDC buffers before calculating the execution results when applying changes asynchronously. The default value is 1000. |
Use Azure Keyvault | Enable this option to connect to external servers like Kafka, Event hub, and Source data systems. Communication between Connect and Azure Key Vault is secured by using TLS.
|