The TRXMETA option in the SQData Apply Engine allows the engine to write detailed transaction metadata to a text file during replication. This metadata is used for observability, latency analysis, and debugging.
Important: TRXMETA output is generated only when the Apply Engine is started with
the
--enable-trxmeta option. For more information, see Command line
execution.The transaction meta data will be written in JSON format to the file added in OPTIONS
statement as below:
OPTIONS
,TRXMETA 'file:./trxmeta_file_name.txt' -- file name where trxmeta data will be written
,TRXMETA_ROTATE SIZE nG -- size GB
,TRXMETA_ROTATE RECORDS n -- number of records
,TRXMETA_ROTATE DELAY n -- delay minutes- In addition to specifying the file in the OPTIONS statement,
file rotation support has been added for the TRXMETA file. This
is enabled through additional optional statements, as described below:
- TRXMETA_ROTATE SIZE nG – Rotates the TRXMETA file when it reaches n gigabytes in size.
- TRXMETA_ROTATE RECORDS n – Rotates the TRXMETA file after n records have been written.
- TRXMETA_ROTATE DELAY n – Rotates the TRXMETA file after n minutes have passed.
- The order of precedence for these TRXMETA file rotation
options is as follows:
- TRXMETA_ROTATE SIZE nG
- TRXMETA_ROTATE RECORDS n
- TRXMETA_ROTATE DELAY n
- The default values for the file rotation options are:
- TRXMETA_ROTATE SIZE – 1 GB (the TRXMETA file is rotated after reaching 1 GB).
- TRXMETA_ROTATE RECORDS – 0 (by default, the TRXMETA file is not rotated based on the number of records).
- TRXMETA_ROTATE DELAY – 120 minutes (the TRXMETA file is rotated after 2 hours).
Sample TRXMETA file data for Db2z Source for Apply Engine
{"trx_id":"000000000058647E148B000000000000","trx_begin":"E01949A8B56
5914C","trx_commit":"E01949A8B59D9822","trx_capture":"E01949A90967D710",
"trx_pub_read":"E01949A90967D710","trx_pub_push":"E01949A90971856C",
"trx_beginur_recv":"E01949D5C21EE260","trx_ack_recv":"E01949D5C22DB260",
"trx_msg_write":"E01949D5C22C9260","trx_ack_write":"E01949D6B6A1C264",
"trx_bytes":"202","trx_records":"1"}Latency Reporting
The sample latency report gives an overview of how efficiently the Apply Engine processes transactions from the source to the target. It shows both individual transaction performance and overall system responsiveness by tracking important timing and volume metrics.
Low Transaction Latency...............: 239 ms
TrxId.................................: 000000000058647E148B000000000000
TrxBytes..............................: 202
TrxRcds...............................: 1
TrxCommit.............................: 2024-12-05 17:07:13.560537 +05:30 (e01949a8b59d9822)
High Transaction Latency..............: 365 ms
TrxId.................................: 000000000058647CBBAA000000000000
TrxBytes..............................: 202
TrxRcds...............................: 1
TrxCommit.............................: 2024-12-05 17:05:15.240467 +05:30 (e0194937dee13b1a)
Avg Transaction Latency...............: 328 ms Kafka topic support
The TRXMETA options statement for Apply Engine can support Kafka topic also in addition to a file.
OPTIONS
,TRXMETA 'kafka://[<hostname>[:<port_number>]]/[<kafka_topic>]' -- kafka topic where trxmeta data will be written in JSON formatNote: Generic URL for TRXMETA Kafka topic also and [/<partition> | /key | /root_key | / ] for TRXMETA Kafka Topic are not supported.