Protobuf type datastores - connect_cdc_sqdata - Latest

Protobuf (Protocol Buffers) formatted datastores use a compact binary serialization format developed by Google. Like AVRO, Protobuf separates schema from data and integrates with a Confluent Schema Registry. Protobuf offers two advantages over AVRO and JSON:

1. Protobuf payloads are smaller than AVRO payloads and significantly smaller than JSON payloads. Binary encoding produces a more compact representation than either text-based JSON or the schema-and-data structure of AVRO.
2. Protobuf serialization and deserialization is faster than both JSON and AVRO, improving throughput in high-volume streaming environments.

Schemas are automatically generated from source record definitions and registered in the Confluent Schema Registry at parse time. A Confluent Schema Registry is always required for Protobuf targets; there is no schema-registry-free Protobuf option.

Protobuf field names automatically preserve their original case from the source definition. The USE AVRO COMPATIBLE NAMES option is specific to AVRO and JSON targets and has no effect on Protobuf datastores.

------------------------------------------------------------
--       DATASTORE SECTION
------------------------------------------------------------
-- SOURCE DATASTORE
DATASTORE cdc://server:port/capture/target
          OF UTSCDC
          AS CDCIN
          DESCRIBED BY GROUP SOURCE_TABLES;

-- TARGET DATASTORE
DATASTORE kafka://[<hostname>[:<port_number>]] / [<kafka_topic_id>][/ | /<partition> | /key | /root_key]
          OF PROTOBUF FORMAT CONFLUENT [TOMBSTONE]
          AS TARGET
          KEY IS DEPTNO, MGRNO
          KEY SUBJECT <kafka_topic_id>-key
          DESCRIBED BY GROUP SOURCE_TABLES;