Configure utility - connect_cdc_sqdata - Latest

Connect CDC (SQData) Change Data Capture

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect CDC (SQData)
Version
Latest
Language
English
Product name
Connect CDC (SQData)
Title
Connect CDC (SQData) Change Data Capture
Copyright
2024
First publish date
2000
Last updated
2024-11-25
Published on
2024-11-25T15:05:48.570404

The SQDIMSU Unload utility can be configured as either an IMS Batch or BMP process. JCL similar to sample member SQDIMSU included in the distribution should be used with the appropriate parameter values.

When CF Structures are used for the z/OS System Logstreams, the IMS Unload Utility uses a throttling technique to avoid offloads and keep data in the CF structure until it is Published and consumed by the downstream Engine(s). This provides better performance and more efficient use of CPU instead of having the publisher retrieve data from offloaded datasets. Throttling is controlled by the specification of High and Low Offload percentages for structure utilization. It is important to size CF structures to avoid throttling if possible while at the same time to avoid offloads.

Throttling is turned ON by default but can be disabled by using --ptk=14. Throttling slows down the LogStream (zlog) put process used by the Capture to give publishers time to catch up and hopefully avoid offload. It is a crude control, and sometimes not very effective because it can cause capture to slow down by elongating the runtime of the batch process. The Capture and Publisher do not communicate with each other to refine it. Some customers do however choose to disable throttling with the PTK because they would rather have offloads occur than slow down change data capture.
//SQDIMSU JOB 1,MSGLEVEL=(1,1),MSGCLASS=H,NOTIFY=&SYSUID 
//* 
//*--------------------------------------------------------------------
//* Extract IMS Data to a Publisher Monitored Logstream in CDC Format
//*--------------------------------------------------------------------
//* Required DDNAMEs: 
//* DBDSRC - Fully Qualified Name of the IMS DBD Source Member 
//* SQDPARMS - Parameters for IMS Database and Target Logstream 
//* 
//* Required parameters: 
//* --database - The Name of the IMS Database being Extracted 
//* --dbdsrc - The Fully Qualified Name of the DBD Source Member
//* --zlog - The Target Logstream where IMS Extract Written 
// Optional parameter: 
//* --ptk=14 - Disables throttling 
//*--------------------------------------------------------------------
//* 
// SET DBDSRC=SCOTTQ.DBDSRC 
// SET DBDMEM=IVPDB2 
// SET IMSPROC=USER.PROCLIB 
// SET SSID=IVP1 
// SET PSB=SQDIVP2 
//* 
//IMSPROC JCLLIB ORDER=&IMSPROC 
//* 
//SQDIMSU EXEC PROC=DLIBMPD,SOUT='*',MBR=SQDIMSU,PSB=&PSB,IMSID=&SSID
//DFSSTAT DD SYSOUT=* 
//PRINTDD DD SYSOUT=* 
//SYSPRINT DD SYSOUT=* 
//SYSOUT DD SYSOUT=* 
//SQDLOG DD SYSOUT=* 
//*SQDLOG DD DUMMY 
//* 
//DBDSRC DD DISP=SHR,DSN=&DBDSRC(&DBDMEM) 
//SQDPARMS DD * 
--database=IVPDB2 
--dbd=DD:DBDSRC 
--zlog=SQDATA.IMSCDC.LOG1 
/* 
--ptk=14
Note:
  • While the sample JCL specifies that the utility will be run as an IMS BMP, the PROCLIB may be changed to run as an IMS Batch Job.
  • When CF Structures are used for the z/OS System Logstreams, the IMS Log Capture and IMS Unload Utility components use a throttling technique to avoid offloads and keep data in the CF structure until it is Published and consumed by the downstream Engine(s). This provides better performance and more efficient use of CPU vs having the publisher retrieve data from offloaded datasets. Throttling is controlled by the specification of High and Low Offload percentages for structure utilization. It is important to size CF structures to avoid throttling if possible while at the same time to avoid offloads.