Using a Header to Eliminate Duplicate Information within a Section - mfx - 3.1
Syncsort™ MFX Programmers Guide
Product type
Software
Portfolio
Integrate
Product family
Syncsort™ Software
Product
Syncsort™ MFX > MFX
Version
3.1
Language
English
Content type
Programmer’s Guide
Product name
Syncsort™ MFX
Title
Syncsort™ MFX Programmers Guide
Copyright
2024
First publish date
2010
Last updated
2024-11-06
Published on
2024-11-06T17:38:26.716054
Introduction
An Introduction to Syncsort™ MFX
Basic Functions of MFX
Data Utility and SortWriter Features of MFX
MFX Operational Features
Syncsort™ MFX Value-Added Products
Structure of the Programmers Guide
Related Reading
Online Message Help
MFX Control Statements
Control Statement Summary Chart
Disk Sort, MAXSORT, and PARASORT Control Statement Requirements
Data Utility Processing Sequence
Control Statement Examples
Rules for Control Statements
ALTSEQ Control Statement
DUPKEYS Control Statement
END Control Statement
INCLUDE/OMIT Control Statement
Rules for Multiple Comparisons
Simplified Expression of EQ/OR and NE/AND Conditions
Using Data in a File as Comparison Constants
Specifying Field-to-Field Standard Comparisons for Non-date Fields
Padding of Compared Fields
Specifying Field-to-Field Comparisons for Unicode Fields
Specifying Field-to-Field Standard Comparisons for Year Fields
Specifying Field-to-Constant Standard Comparisons
Alphanumeric Comparisons
INREC Control Statement
JOIN Control Statement
JOINKEYS Control Statement
MERGE Control Statement
Valid Formats for Merge Control Fields
Rules for Specifying Merge Control Fields
Comparing PD and ZD Control Fields
FIELDS=COPY (Required for a Copy)
CENTWIN Parameter (Optional)
Using Full-Date Formats with CENTWIN
CKPT/CHKPT Parameter (Optional)
EQUALS/NOEQUALS Parameter (Optional)
FILES Parameter (Optional)
SKIPREC Parameter (Optional)
STOPAFT Parameter (Optional)
Sample MERGE Control Statements
MODS Control Statement
MODS Control Statement Format
Exit-Name Parameter (Required)
Sample MODS Control Statement
OMIT Control Statement
OUTFIL Control Statement
The Multiple Output Capability
The SortWriter Capability
The PDF, HTML, RTF and E-mail Capability
The POST-E35 Reformatting Capability
OUTFIL Control Statement Format
FILES Parameter (Optional)
FNAMES Parameter (Optional)
INCLUDE/OMIT Parameter (Optional)
REPEAT Parameter (Optional)
ACCEPT Parameter (Optional)
STARTREC Parameter (Optional)
ENDREC Parameter (Optional)
SAMPLE Parameter (Optional)
SAVE Parameter (Optional)
SPLIT Parameter (Optional)
SPLITBY Parameter (Optional)
SPLIT1R Parameter (Optional)
OUTREC/BUILD Parameter (Optional)
IFTHEN Parameter (Optional)
PARSE Parameter (Optional)
IFOUTLEN Parameter (Optional)
FINDREP Parameter (Optional)
OVERLAY Parameter (Optional)
VLFILL Parameter (Optional)
CONVERT Parameter (Optional)
VTOF Parameter (Optional)
FTOV Parameter (Optional)
VLTRAIL Parameter (Optional)
VLTRIM Parameter (Optional)
BLKCCH1 Parameter (Optional)
BLKCCH2 Parameter (Optional)
BLKCCT1 Parameter (Optional)
HEADER1/HEADER2 Parameters (Optional)
Rules for Specifying HEADER Subparameters
TRAILER Parameters (Optional)
Rules for Specifying TRAILER Subparameters
SECTIONS Parameter (Optional)
LINES Parameter (Optional)
Valid ANSI Control Characters
IFTRAIL Parameter (Optional)
NODETAIL Parameter (Optional)
REMOVECC Parameter (Optional)
NOTMTOFL Parameter (Optional)
NULLOFL Parameter (Optional)
OUTPUT Parameter (Optional)
Sample OUTFIL Control Statements
OUTREC Control Statement
OUTREC Control Statement Format
PARSE Parameter
IFTHEN Parameter
OVERLAY Parameter
FINDREP Parameter
FIELDS/BUILD Parameter
Data Fields (p,l) or (%pp) Subparameters
Specifying the FIELDS Parameter for Variable-Length Records
How to Convert Numeric Data
Converting Year Data with Century Window Processing on INREC, OUTREC, or OUTFIL OUTREC
Converting SMF Date and Time Formats
Time of Day Formats (DCn, TCn, DEn, and TEn)
Literal Fields Subparameters
Generating Run-time Date and Time Constants
Function Field Subparameters
Sample DATEADD Specifications
Sample DATEDIFF Specifications
EDIT Subparameter
LENGTH=n Subparameter
Mm Subparameter (Editing Masks)
SIGNS Subparameter
CHANGE Subparameter
JFY Subparameter
SQZ Subparameter
CONVERT Parameter (Optional)
VTOF Parameter (Optional)
FINDREP Parameter (Optional)
IFTHEN Parameter (Optional)
IFOUTLEN Parameter (Optional)
OVERLAY Parameter (Optional)
PARSE Parameter (Optional)
Sample Statements Using PARSE
Sample OUTREC Control Statements
Sample OUTREC Control Statements with CENTWIN Processing
RECORD Control Statement
RECORD Control Statement Format
TYPE Parameter (Optional)
LENGTH Parameter (Conditionally Required)
Rules for Specifying the Length Parameter
Sample RECORD Control Statements
REFORMAT Control Statement
REFORMAT Control Statement Format
FIELDS Parameter (Required)
Specifying the FIELDS Parameter for Variable-Length Records
FILL Parameter (Optional)
Sample REFORMAT Control Statement
SORT Control Statement
Cultural Environment Support
SORT Control Statement Format
FIELDS Parameter (Required)
Valid Formats for Sort Control Fields
Rules for Specifying Sort Control Fields
Comparing PD and ZD Control Fields
CENTWIN Parameter (Optional)
Using Full-Date Formats with CENTWIN
Examples Using Full-Date Formats
FIELDS=COPY (Required for a Copy)
CKPT/CHKPT Parameter (Optional)
DYNALLOC Parameter (Optional)
EQUALS/NOEQUALS Parameter (Optional)
FILSZ Parameter (Optional)
SIZE Parameter (Optional)
SKIPREC Parameter (Optional)
STOPAFT Parameter (Optional)
Sample SORT Control Statements
SUM Control Statement
SUM Control Statement Format
FIELDS Parameter (Required)
XSUM Parameter (Optional)
General Considerations for SUM
Sample SUM Control Statements
How to Use the MFX Data Utility Features
Using the MFX Data Utility Features
Selecting Input Records
Including Relevant Records
Omitting Irrelevant Records
Selecting Relevant Fields from the Input Records
Selecting a Number of Fields from Longer Records
Eliminating Irrelevant Data Field(s)
Selecting Fields from Variable-Length Records
Combining Records within a File
Combining Records and Summing Numeric Data Fields
Eliminating Duplicate Records
Joining Records from Multiple Files
Joining Records
Retaining Unpaired Records from One of the Join Files
Retaining Unpaired Records from Both Join Files
Using Join Processing To Copy a Large Number of Master File Records
Making Output Records Printable and Easy to Read
Reordering the Positions of Record Fields
Inserting Blanks and Repositioning Record Fields
Inserting Binary Zeros
Converting Unprintable Data to Readable Form
Converting Unprintable Data to Hexadecimal Format
Converting and Editing Unprintable Data
Putting a Data Field in Standard Format
Converting from Variable to Fixed-Length Format
Printing Input Records on Multiple Output Lines
Dividing a Report into Sections
Dividing Output into Sections
Writing Headers and Trailers for a Report
Writing a Title Page for a Report
Writing a Page Header
Writing a Section Header
Using a Header to Eliminate Duplicate Information within a Section
Writing a Report Trailer or Summary
Writing a Page Trailer
Totaling and Subtotaling Data
Totaling Data at the End of a Report
Subtotaling Data at the End of a Page
Totaling Data at the End of a Section
Obtaining Maximum, Minimum and Average Data
Printing Maximum, Minimum and Average Data in Section Trailers
Counting Data Records
Obtaining a Count of Data Records
Obtaining a Cumulative (Running) Count of Data Records
Creating Multiple Output Files
Generating Several Output Files with Different Information
Writing Identical Output Files to Different Devices
E-mailing a Report in PDF Format
JCL and Sample JCL/Control Statement Streams
EXEC Statement
For MAXSORT, PARASORT, DB2 Query and MULTIIN Support
Coding Conventions for DD Statements
STEPLIB/JOBLIB DD Statement
SYSOUT DD Statement
SORTIN DD Statement
Concatenating Input Data Sets
Sorting Large Input Data Sets
Reducing Elapsed Time for SORTS with Multi-volume or Concatenated Tape SORTIN
SORTINnn or SORTINn DD Statement
SORTJNF1 and SORTJNF2 DD Statements
SORTOUT, SORTOFxx, SORTOFx, SORTXSUM, and SORTXDUP DD Statements
Secondary Allocation
SORTWKxx or SORTWKx DD Statement
Secondary Allocation
Are SORTWKxx DD Statements Necessary?
SYSIN DD Statement
$ORTPARM DD Statement
$ORTPARM Processing for Century Window COBOL Applications
$ORTPARM DD Processing for Multiple Sort Invocations
SORTCKPT DD Statement
For Exit Routines that Require Link-editing at Execution Time
SORTMODS DD Statement
SYSLIN DD Statement
SYSLMOD DD Statement
SYSPRINT DD Statement
STDOUT DD Statement
STDERR DD Statement
DD Statements for MAXSORT, PARASORT, DB2 Query andMULTIIN Support
Sample JCL/Control Statement Streams
Sorts without Exit Routines
A Merge without Exit Routines
A Sort with an Exit Routine Already Link-edited
A Sort with an Exit Routine to be Link-edited
Multiple Output Files
PARM Options
Precedence Rules
Additional PARMs
MFX PARM Options
BMSG
CENTWIN
CMP
COLLKEY
COMMAREA
CORE
CPU
DEBUG
DIAG
DYNALLOC
E15
E35
ELAP
EQUALS
EXTCOUNT
FILSZ
FLAG
HBSI
HBSO
IO
IOERR
JPn
L6
L7
LIST
LOCALE
MSG
MSGDD
NOTMTOUT
NULLOUT
OPTELAP
OVFLO
PAD
PRINT121
RC16
RELEASE
RESERVE
RESERVEX
RESERVEZ
RESET
RLSOUT
SDB
SKIPREC
STOPAFT
SWCRYPT
SZERO
TRUNC
UNINTDS
VLTEST
VLTESTI
VSAMEMT
ZDPRINT
ZIIPPCT
Invoking MFX from a Program
Programming Flexibility vs. Performance
DD Statements
Invoking the Sort/Merge from an Assembler Program
Macro Instructions
Coding the Sort/Merge Control Statements
The 24-Bit Parameter List
Return Codes
Sample Assembler Invocation Using 24-Bit Parameter List
The 31-Bit Extended Parameter List
Return Codes
Sample Assembler Invocation Using 31-Bit Parameter List
The 64-Bit Parameter List
Syncsort Java Class
Constructor Detail
Method Summary
Methods Inherited from Class java.lang.Object
Method Detail
Java Wrapper Example
The Coding and Use of Exit Programs
What Is an Exit?
Loading the Exit Routines into Main Storage
Exit Conventions
Register Conventions
The Exit Communication Area
Exits E11, E21, and E31 - Preparing for Other Exit Routines
Exit E32 - Invoked Merge Only: Creating Input Records
Return Codes
Exit E14 - Deleting, Summing, Changing Records
Return Codes
Exit E15 - Creating, Revising, or Analyzing the Input File
Coding the E15 Exit Routine for an Invoked Sort or Copy
Return Codes
Coding a COBOL E15 Exit Routine
The LINKAGE SECTION
Example 1: Fixed-Length Records
Example 2: Variable-Length Records
The IDENTIFICATION, ENVIRONMENT, and DATA Divisions
The WORKING-STORAGE SECTION
The PROCEDURE DIVISION
EXIT-STATUS Codes (Fixed and Variable-Length Records)
RETURN-CODE Codes (Fixed and Variable-Length Records)
To Change a Record
Sample COBOL E15, Fixed-Length Records
Sample COBOL E15, Variable-Length Records
Coding a C E15 Exit Routine
Exit Communication
Fixed-Length Records - Function Definition
Variable-Length Records - Function Definition
RETURN-CODE Codes (Fixed and Variable-Length Records)
How to Change a Record
Sample C E15, Fixed-Length Records
Sample C E15, Variable-Length Records
Exit E25 - Deleting, Changing, and Summing Records
Return Codes
Exit E35 - Adding, Deleting, and Changing Records
Coding the E35 Exit Routine for an Invoked Sort/Merge/Copy
The E35 Parameter List
Return Codes
Coding a COBOL E35 Exit Routine
The LINKAGE SECTION
Example 1: Fixed-Length Records
Example 2: Variable-Length Records
The IDENTIFICATION, ENVIRONMENT, and DATA Divisions
The WORKING-STORAGE SECTION
The PROCEDURE DIVISION
EXIT-STATUS Codes (Fixed and Variable-Length Records)
RETURN-CODE Codes (Fixed and Variable-Length Records)
To Change a Record
Sample COBOL E35, Fixed-Length Records
Sample COBOL E35, Variable-Length Records
Coding a C E35 Exit Routine
Exit Communication
Fixed-Length Records - Function Definition
Variable-Length Records - Function Definition
RETURN-CODE Codes (Fixed and Variable-Length Records)
Change a Record
Sample C E35, Fixed-Length Records
Sample C E35, Variable-Length Records
Exit E16-Taking Action on Insufficient Intermediate Storage
Return Codes
Exits E17, E27, and E37 - Closing Data Sets
Exits E18, E38, and E39 - Checking Labels, Processing Read or Write Errors, End-of-File Routines, Special VSAM Processing
Exit E18 and E38 Programs
VSAM Input to E18 and E38
Exit E39 Programs
Exit E61 - Modifying the Collating Process
The Parameter List
Lengthening a Control Field Image
Shortening a Control Field Image
Reversing a Collating Sequence
Coding REXX Exits
REXX Variables Provided by MFX
Sample REXX Exit
The Flow of the Sort
MAXSORT
MAXSORT: A Maximum Capacity Sort
Advantages of MAXSORT
Job Control Language
Sample EXEC Statement
SORTBKPT DD Statement
Allocating Disk Space for the Breakpoint Data Set
Sample Allocation of the Breakpoint Data Set
Sample SORTBKPT DD Statement
SORTOU00 DD Statement
Sample SORTOU00 DD Statement
SORTOUnn DD Statements
Sample SORTOUnn DD Statements
Using Disk for Intermediate Output
SORTCKPT DD Statement
Control Statements
PARM Options
BKPTDSN
DYNATAPE
MAXSORT
MAXWKSP
MINWKSP
RESTART
SORTSIZE
SORTTIME
TAPENAME
Exit Programs
Invoking MAXSORT from a Program
Restarting MAXSORT
Restarting MAXSORT with Exit Routines or an Invoked MAXSORT
Operator Interface of MAXSORT
Sample MAXSORT JCL/Control Streams
Example 1: A 100 Gigabyte MAXSORT with only Minimal Disk Space Available
Tuning MAXSORT
PARASORT
PARASORT: Parallel Input Processing for Elapsed TimeImprovement
PARASORT Applicability
Job Control Language
Sample EXEC Statement
SORTIN DD Statement with PARASORT
SORTPARn DD Statements
Special Channel Separated Esoteric Names
Sortwork Considerations
Operations Notes
MFX DB2 Query Support
Restrictions
Job Control Language
Sample EXEC Statement
DD Statements
SORTDBIN DD Statement
PARM Options
DB2
MULTIFETCH
Operation
Record Description
Record Description: Trial Mode Execution
Sample MFX DB2 Query Application
Multiple Input Files
Restrictions
Job Control Language
EXEC Statement
DD Statements
SORTMInn DD Statements
Operation
Record Length Determination
Notes
Sample Multiple Input File JCL and Control Statements
The Dictionary Feature
Building a Dictionary
Dictionary Statement Format
The Constant_name Statement: Rules and Syntax
Specifying System Symbols
The Field_name Statement: Rules and Syntax
Specifying Position (p) in Field_name Dictionary Statements
Specifying Length (l) in Field_name Dictionary Statements
Specifying Format (f) in Field_name Dictionary Statements
Specifying a Parsed Field (%pp) in Field_name Dictionary Statements
The Operator Statement: Rules and Syntax
Using POSITION in Operator Statements
Using SKIP in Operator Statements
Using ALIGN in Operator Statements
Activating the Dictionary Feature
Using Dictionary_names in MFX Control Statements
Notes on Format Substitution
Specifying Field_Names after Record Reformatting
INREC, OUTREC, OUTFIL TO Subparameter
Specifying Dictionary Listing Output
Error Handling for Dictionary Statements
Sample Dictionary Statements
Using JCL SET and PROC Symbols to Create Dictionary_Names
Performance Considerations
Disk Sort? MAXSORT? PARASORT?
JCL Sorts vs. Program-Invoked Sorts
Control Statement Issues
The Efficient Use of PARMs
Optimizing System Resources
The Incore Sort
Disk Space Considerations
The Impact of Disk Space on the Work Data Sets on MFX
Disk Sort Intermediate Storage Calculation Formulas
Special Considerations Concerning MFX’s Disk Space Management on Work Data Sets
The Coding and Use of Checkpoint-Restart
The DISP Parameter for SORTCKPT, SORTWKxx and SORTOUT Data Sets
The SORTCKPT Data Set
The SORTWKxx Data Set(s)
Automatic Checkpoint-Restart
Deferred Checkpoint-Restart
Optimizing Data Set Placement
Device Type Considerations
The HISTOGRM Utility Program
What Is HISTOGRM?
Using HISTOGRM to Determine L6 and L7 Values for MFX
Control Parameters for HISTOGRM
NRECS
WIDTH
DEVWK
KEYL
BIGREC
BLOCK
REC
BIGSTOP
Job Control Language
Executing HISTOGRM through an E15 Exit
HISTOGRM Messages
The SYNCMEML Analysis Program
What Is SYNCMEML?
Intended Users
Run-Time Environment
JCL
Value-Added Products
Syncsort PROCSort – An Accelerator for SAS® Sorting
Syncsort PipeSort
Syncsort ZPSaver
Syncsort ZPCopy
Syncsort ZPCompress
Syncsort ZPSort
Syncsort ZPEncrypt
Messages
Syncsort™ MFX Messages
Syncsort ZPCopy Messages
Syncsort ZPCompress Messages
Syncsort ZPSort Messages
Syncsort ZPEncrypt Messages
Syncsort PROCSort Messages
License Key Messages
Diagnostics and Technical Support
Troubleshooting Abends
MFX Internal Abend
U-Type Abend Codes
Before Contacting Precisely Support
Searching the Precisely Knowledge Base
Notices