Replication to AVRO containers in plain file can be accomplished with minimal configuration and provide excellent performance.
Syntax
OPTIONS [AVRO COMPATIBLE NAMES]
,[CONFLUENT REPOSITORY <registry_url>]
,[MAX IN FLIGHT <number>]
,[ISO8601 PRECISION <number>]
,[METADATA (<metadata_name> [AS <metadata_alias> ] [,<metadata_name> [AS <metadata_alias> ]] [, ...])]
,[STRIP TRAILING SPACES]
,[ROTATE DELAY <duration>]
,[ROTATE SIZE <size> [K|M|G]]
,[NAMESPACE '<namespace>']
, [FORMAT TOMBSTONE]
Example
REPLICATE DB2 file:./data/G1A02_001.DAT
TO AVRO CONFLUENT 'kafka:///*/key'
WITH 1 WORKER
;
OPTIONS
AVRO COMPATIBLE NAMES,
CONFLUENT REPOSITORY "https://XXXX:XXXX@XXXX.aws.confluent.cloud",
LOGICAL DECIMAL TYPE,
STRIP TRAILING SPACES,
FORMAT TOMBSTONE,
IGNORE NOMAPPING,
MAX IN FLIGHT 100,
UOW CACHE MEMORY OF 512M
;
;
MAPPINGS
SOURCE SQDATA.TBLT6
TOPIC SQD_RPL_DEPT6_AVRO
SUBJECT 'SQD_RPL_DEPT6_AVRO-value'
KEY SUBJECT 'SQD_RPL_DEPT6_AVRO-key'
,SOURCE SQDATA.TBLT6
TOPIC SQD_RPL_DEPT2_AVRO
SUBJECT 'SQD_RPL_DEPT2_AVRO-value'
KEY SUBJECT 'SQD_RPL_DEPT2_AVRO-key'
;