DIF has an interface to the IBM MQSeries commercial messaging facility. This facility allows the products to establish communication links with other computers and move information between the machines. Message queues can also be used for communication between the DIF started task and the products, and even provide communication between DIF started tasks in a Sysplex environment.
MQSeries was developed to guarantee message delivery, and is heavily used in the banking and financial industries. The product holds data in the queues until the information is read from the message queue. If a machine goes offline, the queue will hold the records until the receiving machine becomes available.
Messages and records written by the rules language can be placed in an
MQSeries message queue using the DEFCONN rule
statements. A DEFCONN statement maps a log file DDname requested by the
WRITEREC and WRITEMSG statements to an
MQSeries message queue. Local and remote computers that have access
to the message queue can retrieve messages.
Performance vs. Data Integrity
If you activate the DIF MQSeries interface, you need to decide how you want to use the interface. This is really a decision about how important you consider the data you are writing to MQSeries message queues. There are two general models you can use: a performance model and a data integrity model. You can also mix the models as required by your installation.
In the performance model, DIF writes data to the message queue on behalf of the product. The product schedules the message and continues execution; DIF buffers and later writes the data to the queue.
If you want the best possible performance, messages can be scheduled for execution and the
DIF started task will be responsible for writing the message or record to the queue. The
WRITEREC and WRITEMSG statements buffer the records in
the DIF address space. Your installation has control over the size of the record buffer.
For more information, examine LOGBUF in the Product
Installation Guide.
If you select the performance model, the product’s rules language should NOT use the
DEFCONN statements to redirect information to
MQSeries. Messages and records are still passed to DIF, and DIF will
perform the I/O to the DD statement. You can then use the DIF command
EXEC(CONNECT(member)) to redirect the data to
MQSeries. In other words, the product rules produce messages or
records and pass the information to DIF. The DIF started task writes the data to the
MQSeries message queue.
There is no measurable performance penalty when passing a record to DIF; the product rule requesting the I/O schedules the request and immediately continues execution. DIF has a subtask that periodically gathers the buffered I/O requests and writes the information to the message queue. The overhead of writing the data to the message queue will be allocated to the DIF started task.
To use the performance model, specify the parameter
EXEC(CONNECT(member)) on DIF. The
member coded on the parameter specifies a member name in the DIF
parameter library. The member contains the DEFCONN commands that redirect
I/O to an MQSeries queue name.
If you have multiple DIF address spaces running in a Sysplex, you can consolidate messages so one DIF address space funnels data to the message queue. For more information, examine Log Record Consolidation in the next section.
Data Integrity Model
If you never want to lose records, the product rules can write messages directly to the MQSeries message queue — even from environments where I/O is normally restricted. The obvious advantage is you will never lose data, and DIF does not have to be active to write the message to the queue. The disadvantage is MQSeries must be active, and jobs in execution will wait for MQSeries to respond before continuing. There is also a relatively small performance penalty for writing data to a message queue, and the penalty will be reflected in a job that writes messages.
If you select the data integrity model, the product’s rules language should use the
DEFCONN statements to redirect the information directly to a message
queue. Records that use the DEFCONN interface in the rules language do
not use the MQSeries interface provided by the DIF started task, so
the DIF EXEC(CONNECT(member)) parameter is not required.
Implementation - The MQSeries interface can be used by the products in several ways: data transport for SCC Monitor/Realtime, DIF log consolidation, and a remote DIF command interface.
Data Transport for Monitor
Whenever data sets are created, expanded, or deleted, SCC Monitor/Realtime captures the event. MQSeries message queues or TCP/IP can be used to pump records to a Java-based client/server storage management database. This information is available for real-time and historical reporting.
Using MQSeries to Consolidate Log Records
When multiple DIF address spaces are active in your Sysplex, MQSeries can be used to consolidate log records to a single data set. Multiple systems can be writing messages to a log queue, but one DIF should be assigned to read the queue and write the records to the log data set.
ABC provides a perfect example. If ABC is running on multiple systems in your Sysplex to track data set usage for ABARS backups, records updates can be routed to the log server DIF which has control of the ABC VSAM database. Without consolidated updates, each image in the Sysplex would require an associated ABC VSAM database to avoid VSAM share problems, and each ABC utility would have to merge input data from multiple databases.
In the following example, MQSeries is used to perform log record
consolidation. The DIF commands would be placed in the DIFINIT member
in DTS.R71.PARMLIB. In the example, log records are routed through an
MQSeries message queue to SYSA which will be the
log server used for record consolidation.
SYSID(EXCLUDE(SYSA))
INITOPT EXEC(CONNECT(MQINIT)) SYSID(INCLUDE(SYSA))
INITOPT LOGSERVER(MQSERIES(DTS.LOG.QUEUE)) SYSID()
The first SYSID section executes on all systems except
SYSA and would execute the DEFCONN commands in
member MQINIT to establish connectors which route records to the system
logger. The MQINIT contains the DEFCONN commands to
capture an I/O request made to a DDname and reroute records to a message queue. For more
information, examine EXEC in the Product Installation
Guide.
DEFCONN statements.
A sample DTS.R71.PARMLIB(MQINIT) member is shown below.
DEFCONN ACCLOG MQSERIES(DTS.LOG.QUEUE)
DEFCONN ABCDB MQSERIES(DTS.LOG.QUEUE)
DEFCONN RTMDB MQSERIES(DTS.LOG.QUEUE)
The second SYSID section would only execute on SYSA
and activates the log server to gather records from the system logger and write them to
the target logs. For more information, examine LOGSERVER in the
Product Installation Guide. The last SYSID()
terminates conditional command processing.
The DD statements for the log files must be allocated to the LOGSERVER DIF. However, if you wish to maintain only one DIF PROC, the DD statements can be added to the DIF PROC but will only be opened by the LOGSERVER.
Rerouting Consolidated Log Records to another Message Queue
If you want to consolidate the log records and then write the records to another
MQSeries message queue, you can add another
EXEC(CONNECT) statement to the DIFINIT member in
the log server DIF section. For example, the LOGSERVER statement
activates inter-DIF communication to consolidate the messages, and EXEC
CONNECT broadcasts the messages to the queues controlled by the
DEFCONN commands found in
DTS.R71.PARMLIB(MQOUTPUT):
SYSID(EXCLUDE(SYSA))
INITOPT EXEC(CONNECT(MQINIT)) SYSID(INCLUDE(SYSA))
INITOPT LOGSERVER(MQSERIES(DTS.LOG.QUEUE))
INITOPT EXEC(CONNECT(MQOUTPUT))
SYSID()
If the records are being consolidated and written to a PC application, the
CONVERT parameter can be specified to translate the record to ASCII
before placing the message on the queue. A sample
DTS.R71.PARMLIB(MQOUTPUT) member:
DEFCONN RTMDB MQSERIES(DTS.RTMDB.QUEUE CONVERT)
Message Scheduling
When the rules language writes a message from an environment that does not allow I/O,
the message is scheduled for I/O in the DIF address space. The products have two methods
available for scheduling a message: a cross-memory move or an
MQSeries message queue. By using the DEFCONN
rules statements and the LOGSERVER DIF initialization parameter, DIF
can use an MQSeries message queue to schedule messages or records to
be written to a log file. For more information, examine LOGSERVER in
the Product Installation Guide.
DIF Command Interface
Normally, DIF commands can only be requested on the same machine executing the DIF
started task. The SERVER DIF parameter can establish an
MQSeries queue for remote DIF command requests. Any machine
authorized to use the MQSeries message queue can issue a DIF command
and receive the results of the command. For more information, examine
SERVER in the Product Installation Guide.
Installing MQSeries on your Host
If you do not have MQSeries installed on your host machine, the product should be ordered and installed by your systems programming staff. The MQSeries for MVS/ESA Program Directory, provided with the IBM installation materials, lists the contents of the MQSeries installation tape, the program and service level information for MQSeries, and describes how to install MQSeries under MVS using SMP/E.
After performing the installation, you must carry out a number of tasks before you can make it available to users. This process is documented in the MQSeries System Management Guide (SC33-0806) in Part 2.
Customizing MQSeries after installation
Sample configuration members provided by the software can be found in the DIF sample
distribution library: DTS.R71.SLyyddd.SAMPLIB. The configuration
members have names starting with CSQ.