Connect CDC (SQData) Release 4.0 - connect_cdc_sqdata - Latest

• Backport from V4.1.25. Corrected AVRO encoding of Double and Single precision floating point numbers.
• Backport from V4.1.20. Default the conversion of source code page to UTF-8 for special characters (specifically bad string conversion from IMS to Kafka).

lrwxrwxrwx  1 root root       20 Jul  8  2020 libgnutls.so -> libgnutls.so.28.43.3
lrwxrwxrwx  1 root root       20 Sep 11  2019 libgnutls.so.28 -> libgnutls.so.28.43.3
-rwxr-xr-x  1 root root  1300504 Mar 14  2019 libgnutls.so.28.43.3 

In this case the libgnutls.so link is installed in a location included in the 'default library path' : /usr/lib64. If the library has been installed in the wrong location, the following environmental variable can be used but Precisely only recommends doing so in a test environment:SQDATA_GNUTLS_LIBRARY=<path_To_softlink/>libgnutls.so

Previously the default was 10 seconds which came into use when the Oracle LogMiner continuous mine option was lost due to the requirements of both Oracle RAC and Oracle 19.

The new parms may only be used with sqdconf modify and become effective immediately following sqdconf stop and sqdconf start sequence. NO apply is required.

The --small-stall parm value applies to the first "wait" after capture is caught up. If after the --small-stall wait, the next request also returns no changed data the --large-stall parm value is used for subsequent "waits" until logminer once again returns data to the capture.

The effectiveness or efficiency of the wait interval is solely dependent on transaction patterns which are hard if not impossible to analyze. These two parameters are "tuning" values. Precisely recommends all implementations begin with the defaults and then tune for your environment. If --small-stall is too short there is an increased potential for wasting CPU performing logminer calls, current LSN queries and other factors.

The sqdconf display shows current values ONLY if you have overridden the defaults!

SQDF803I Instance (Version) : orcl19c (19.0.0.0.0)
SQDF800I Display as of: 2021-01-04 15:11:03 -00:00
SQDF901I Configuration file : /home/sqdata/oracdc/oracdc.cab
SQDF902I Status : MOUNTED,STARTED,STALLED
SQDF903I Configuration key : cab_033B41F7447D5EDA
SQDF904I Allocated entries : 31
SQDF905I Used entries : 5
SQDF906I Active Database : localhost/orcl19c
SQDF907I Start Log Point : 25915451
SQDF908I Last Log Point : 26667017
SQDF940I Last Log Timestamp : 2021-01-04-15.11.03
SQDF981I Safe Restart Point : 26667014
SQDF986I Safe Remine Point : 26637169
SQDF910I Active User : SQDATA
SQDF913I Fix Flags : RETRY,DICT_FROM_LOG
SQDF914I Retry Interval : 30
SQDF919I Active Flags : CDCSTORE,RAW LOG
SQDF980I Pending Flags : CDCSTORE,RAW LOG
SQDF915I Active store name : /home/sqdata/oracdc/oracdc_store.cab
SQDF916I Active store id : cab_061EA9E047296655
SQDF957I small stall (ms) : 50
SQDF958I large stall (ms) : 150
SQDF920I Source Entry : # 0
SQDF930I Key : SQD.ADDRESS
SQDF923I Active Flags : ACTIVE
SQDF928I Last Log Point : 22388290
SQDF950I session # insert : 0
SQDF951I session # delete : 0
SQDF952I session # update : 0
SQDF960I cumul # of insert : 1389
SQDF961I cumul # of delete : 1200
SQDF962I cumul # of update : 72
SQDF925I Active Datastore : cdc:///ORATOKAF

• Support for logical decimal type with precision <= 18 : New OPTIONS block item:

  ,LOGICAL DECIMAL TYPE (Will produce schemas using LOGICAL DECIMAL TYPE for Decimal source data <= 18 digits otherwise will still be written as a string)

• Make sure backward compatibility in schema generation, a change was made to add a default value to each field in the schema.

• Allow precision for iso8601 timestamp - New OPTIONS block item: ,ISO8601 PRECISION <n> (where <n> represents the number of digits of precision requested, 6 for example specifies microseconds. The actual result is dependent on source data precision)

• For DL/I Batch, get the log data set name from the batch feed zlog.
• Fix DL/I Batch log restart error after abnormal termination.

• Oracle RAC for Oracle 11, 12 and 19 - Oracle 11 and 12 captures must have the --disable-continuous-mining parameter added to their Capture Configuration (.cab) files:
  Syntax

  $ sqdconf modify <cab_file_name>
  --disable-continuous-mining

  Keyword and Parameter Descriptions

  <cab_file_name>  - Path and name of the Capture/Publisher Configuration (.cab) file

  --disable-continuous-mining - Oracle RAC capture only. This option instructs the Capture Agent not to use the Oracle LogMiner CONTINUOUS_MINE option which is not supported by Oracle 11 or 12 on a RAC system. The --disable-continuous-mining option is the default for Oracle version 19 on both RAC and non-RAC systems.

• Oracle Capture now requires SELECT authority on V$THREAD in addition to the other existing LogMiner requirements.
• Additional Oracle 19 stability improvements including tolerating ORA-1013 generated by user cancellation of the Capture agent.

When the source DATASTORE is an RDBMS and it is being read remotely for an initial load or refresh process, a Select statement and WHERE clause can now be specified to limit the source data being selected. There are no other changes required.

DESCRIPTION SQLDDL /home/sqdata/employee.ddl AS S_EMPLOYEE;
DATASTORE RDBMS
          QUERY
          /+
          SELECT EMPNO, LAST_NAME, FIRST_NAME,... FROM PROD.EMP
          WHERE EMPNO = 12345 OR
                EMPNO = 12346
          +/  

         OF RELATIONAL
         AS CDCIN
         DESCRIBED BY S_EMPLOYEE;