External library requirements - connect_cdc_sqdata - Latest

Connect CDC (SQData) Kafka Quickstart

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect CDC (SQData)
Version
Latest
Language
English
Product name
Connect CDC (SQData)
Title
Connect CDC (SQData) Kafka Quickstart
Copyright
2024
First publish date
2000
Last edition
2024-07-30
Last publish date
2024-07-30T20:00:09.892433

In addition to the basic librdkafka client one or more additional libraries may be required. Those requirements come from various sources including system and network level security, target datastore security and connectivity, etc.

When the Apply or Replicator Engine starts it begins by loading or attempting to load a variety of external libraries. Some like libicuuc.so also known as the "ICU" library should be part of every base linux system installation while others like libasan.so the "AddressSanitizer" may be installation standards. The librdkafka library is loaded during Engine startup when the Engine processes the kafka:// target datastore URL. If your installation utilizes a librdkafka build containing site specific dependencies you may encounter the following error:
SQD0136E (1) Error encountered in function REPLICATE, parameter 
This could be caused by a variety of factors however the first thing to look at should be the engine diagnostic log, named depending on what you are executing sqdeng_nnnn.log, sqdata_nnnn.log or sqdrpl_nnnn.log where the nnnn represents the process-id of the failed engine. In that log you may find something like the following, indicating which library is missing:
00000410 2022-11-18 08:57:32 4138159 45caf4c0 [   KAFKA] :kafka_open_producer -->
00000411 2022-11-18 08:57:32 4138159 45caf4c0 [DLLKAFKA] :dllkafka_load -->
00000412 2022-11-18 08:57:32 4138159 45caf4c0 [DLLKAFKA] :Try loading /hosting/lib/lib/librdkafka.so
00000413 2022-11-18 08:57:32 4138159 45caf4c0 [DLLKAFKA] :could not load /hosting/lib/lib/librdkafka.so, (libasan.so.0: cannot open shared object file: No such file or directory)
In this example, libasn.so was not found. Before proceeding further run the Linux ldd command to identify all the dependant libraries and which ones are missing:
ldd <path_to>/librdkafka.so
Output from ldd will contain several lines referencing dependent objects and most likely one or more will be "not found".
libasan.so.0 > not found
libibm.so => /lib64/libm.so.6 (0x0000acb80c000)
libssl.so.10 => not found

Consult your Linux sysadmin group for assistance installing the required libraries since you will likely find your account does not have permission to do so.