Overview - syncsort_scc_monitor - Latest

Controlling the use of direct-access space

Controlling space utilization The advantages of using DASD as a medium for storing information are so overwhelming that almost all important corporate information is today kept on disk. Existing applications, online systems, data warehouses, decision support systems, and new applications all compete for DASD space. Corporate mergers and data center consolidations add their requirements to the insatiable demand for space. With almost every data processing function dependent upon the availability of disk storage, it is not uncommon for DASD growth rates to exceed 50% annually. Since no company has an unlimited budget, containing the runaway growth of DASD is an important priority.

Traditional methods Data centers have relied on several methods to limit the growth of DASD space. Manual methods include assigning particular volumes or storage groups to different work groups, such as devoting volumes ‘PAY01’ through ‘PAY22’ to the payroll application and volumes ‘HR001’ through ‘HR005’ to the human resources application. In data centers which have implemented an allocation control product such as Allocation Control Center (ACC), the task of ensuring that only datasets belonging to a particular application reside on specified volumes can be automated. Much additional effort is often devoted to ensuring that only the most recently used data remains on “primary” DASD. Sophisticated backup and recovery systems such as IBM’s DFSMShsm and Innovation Data Processing’s FDRABR are used to migrate older data to compressed files on DASD or to tape, reducing the amount of disk space required to hold ‘active’ data. Proper use of such archival utilities requires strict attention to the migration criteria, recall rate, and tracking of backups.

System-managed storage considerations

Devoting entire disk volumes to particular applications is wasteful. If an application group requires only a few cylinders of DASD space, the rest of the volume may remain unused. More importantly, storage use across the system may be unbalanced, with some volumes almost empty while others are nearly full. This can result in failed allocations, out-of-space abends, and ultimately, the need to purchase more DASD than would otherwise be required. Furthermore, dependence on particular volume serial numbers makes it very difficult to create and maintain application JCL and imposes a heavy maintenance burden on the storage administrator who attempts to add or remove disk volumes on the system.

For these reasons, most installations have moved to a system-managed storage environment, in which datasets are allocated to volume pools based upon the service requirements of the dataset. For datasets managed by DFSMS, the Automatic Class Selection (ACS) routines derive a data class, storage class, and management class for each dataset and assign the dataset to an SMS storage group. For non-SMS datasets, a similar volume pooling function is performed by products such as the Allocation Control Center (ACC). With system management of DASD storage, datasets are no longer restricted to the particular volumes specified in the JCL, but may be placed anywhere in the storage group or volume pools according to the policies implemented by the storage administrator.

Runaway space use

System-managed storage removes the burden of volume-specific allocation and allows datasets to be placed on the volume where they are best suited. With system-managed storage (whether the dataset is managed by SMS or whether it is simply a “pooled” non-SMS dataset), space utilization across the system is more even, and allocation failures and out-of-space conditions are less frequent. However, this benefit comes with a cost.

With system-managed storage, users are no longer restricted to allocating only to their “own” volumes. An application’s datasets may be spread out among any of the volumes in a pool, or among several pools. There is no mechanism within the operating system to limit the amount of space available to an application. In a non-system-managed environment, allocation of datasets is limited to those volumes that belong to the application. In a system-managed environment, this limitation is lifted. The possibility then exists for ill-behaved applications to occupy vast amounts of space. Without any basis for preventing excessive space use, or even of tracking the amount of space used by an application, data centers are open to the possibility of runaway DASD growth.

Newer “instantaneous-copy” hardware technologies increase the potential for out-of-control allocation of space still further, as it becomes very easy for application users or storage administrators to make many duplicate copies of large datasets.

Using DLimit to measure and control utilization

DLimit addresses the problems of unlimited DASD use and lack of accountability that plague even the most aggressively tuned system-managed storage environments. With DLimit, the storage administrator can establish a space “account” for each application, and can ensure that applications stay within their assigned space limits.

To use DLimit, the storage administrator establishes a set of rules that specify the characteristics of datasets that are to be assigned to the various accounts. As datasets are created, extended, renamed, deleted, and released, DLimit examines the rules and updates the appropriate accounts, so that a constant running total of the amount of disk space used by each application is available.

Track space utilization

DLimit tracks the current amount of space occupied by each application as well as the largest amount ever used by the application (the high-water mark). Each application is assigned an “account” in the DLimit database, and the space amounts recorded in the account are updated as requests for space are made.

Requests for space can be assigned to one or more accounts that describe the type of space requested. The currently used space and high-water mark are recorded for each account. Other information, such as the date and time of the most recent update, are also recorded in the account record. Since DLimit receives control in real time at the moment a request for space is made, the values in the account record are always up-to-date.

Establish space limits

Accounts may be explicitly established by the storage administrator (via the DLADD command) or automatically by DLimit rules when a non-existent account name is assigned to a request for space. In either case, a maximum value for the amount of space allowed can be set for the account. Space amounts for each account may be recorded in:

• tracks
• kilobytes (KB)
• megabytes (MB)
• gigabytes (GB)

Warn or deny allocation

If the amount of space which an application is allowed to use is exceeded, DLimit can perform a number of actions. The allocation may be denied (if MODE(ACTIVE) is specified for the account), or a warning message may be issued (if MODE(WARN) is specified). Warning messages are written to the JES log for a batch job, or to the TSO session for a TSO user. If desired, additional warning messages may be written to a log file, to SMF, or to a storage administrator’s TSO userid. Users can also be warned if the amount of space used by an application is approaching the maximum limit for one of the space accounts but has not yet exceeded it.

How DLimit interfaces with DADSM

DLimit uses the operating system IGGPRE00 and IGGPOST0 exits to obtain control whenever a request is made within the system for direct-access space. During IGGPRE00 time, the DLimit rules are executed. If the request for space is assigned to an account, DLimit checks to see whether the maximum space for the category is exceeded. If so, DLimit may prevent the allocation, warn the user, or take no action, depending on the setting of the MODE value for the account. Once the dataset has been allocated, DLimit updates the account with the new space values during IGGPOST0 processing. If dataset allocation fails, either because DLimit denied the request or for some other reason, the account is not updated and DLimit does not change the current space amount in the account.

DLimit gets control for each type of request for space the system may make. These request types include:

• create
• delete
• extend
• release
• rename

During dataset creation or extend processing, space will be added to the specified accounts. During delete or release processing, space amounts will be subtracted from the accounts specified by the rules. For this reason, it is important that the rules use durable variables whenever possible, and that the rules assign the same account name to a dataset during both creation and deletion processing. A durable variable is one that does not change depending upon how the dataset is referenced. For example, high-level qualifier and RACF owner id are durable, but DD name is not.

During rename processing, space is subtracted from the account for the old name and added to the account for the new name. The particular type of space request being processed is available within the DLimit rules by examining the &ENVIRONMENT variable.

Many installations may already have IGGPRE00 and IGGPOST0 exits in use. DLimit can coexist with existing exits and will call these exits before doing any of its own processing.

The following figure shows how DLimit uses the DADSM exits to assign datasets to accounts, determine if the space limit for the account has been exceeded, and take appropriate action.

Figure 2-1. DLimit Space Request Processing

The DLimit database

A repository for space information DLimit keeps up-to-the-minute information about the amount of space assigned to each account in the DLimit database. The DLimit database is a BDAM file, specified to act as the DLimit information repository.

Only one BDAM file needs to be used system wide as the active DLimit database. All requests that the system makes for disk space are intercepted by DLimit, and if the DLimit rules assign the request for space to a DLimit account, then the entry representing the account in the currently active database is updated. Additional databases may be defined.

Other DLimit databases

The DLMAINT utility program provided with DLimit can extract the contents of the BDAM file used as a database and place it in a key-sequenced VSAM file. This VSAM file can then be used by the DLMAINT program instead of the active database, if desired. DLMAINT allows the storage administrator to examine, change, and report on the contents of DLimit databases. The database used by DLMAINT can be the active DLimit database, or a VSAM file specified by the SET DATABASE statement.

The $CONTROL record and model accounts

The $CONTROL record

The $CONTROL record specifies defaults that apply to DLimit processing. These defaults apply unless they are overridden by specifications within the DLimit rules or on the DLimit commands. The DLSYS command is used to display and change the contents of the $CONTROL record.

Model accounts

Accounts within the DLimit database may be designated as a model for use when creating new accounts. New accounts may be created either dynamically or explicitly via the DLADD command.

Once an account is designated as a MODEL, it is used as a model for the creation of new accounts. The MODEL operand of the DLADD command is used to specify the name of the model account to be used, or model accounts can be set in the DLimit rules.

Most installations will have procedures already in place for the backup of system datasets. If a copy of the DLimit database is required, it can be obtained with the BACKUP and RESTORE statements of the DLMAINT utility program, or via the EXPORT and IMPORT commands of IDCAMS.

DLimit accounts

What is an account? DLimit tracks space by assigning requests for space to accounts. An account represents the space assigned to all datasets that meet some particular criteria, as specified in the DLimit selection rules. Within the DLimit selection rules, the SET statement is used to assign a request for space to an account, for example: SET &DACCT = PAYROLL

In this example, the current request for space (which may be due to allocation of a new dataset, or extension of an existing one) is assigned to the “PAYROLL” account. Account names may be up to 30 characters in length and may be any combination of valid alphanumeric characters. Embedded blanks are not allowed in account names.

Establishing characteristics for an account

The DLADD command is used to explicitly create an account. Account names may be from 1 to 30 alphanumeric characters. Once established via the DLADD command, the DLALTER command may be used to change the characteristics for the account. The syntax and operands of the DLADD and DLALTER commands, as well as various other TSO commands for DLimit, are described in Chapter 6, "TSO Command Interface". These TSO commands can also be specified in the SYSIN input to the DLMAINT Utility, which is described in Chapter 7. DLMAINT can be used to back up or recover the DLimit database, to make changes to individual accounts, or to make mass changes to many accounts by processing each account in the database against a set of rules.

Accounts may also be created dynamically whenever a SET &DACCT= statement is encountered in the DLimit rules. In order for accounts to be created dynamically, the $CONTROL record in the DLimit database must indicate that dynamic account creation is allowed. The DLSYS command can be used to change the contents of the $CONTROL record. The syntax of the DLSYS command can be found in Chapter 6, "TSO Command Interface".

The following characteristics may be set or changed in an account, using the DLADD or DLALTER commands:

Warning Percentage: Specifies the percentage of the total amount of space that the account is allowed to use before warning messages are issued (if the account is in WARN or ACTIVE mode).

Space Tracking Units: Specifies the type of units (TRACKS, KB, MB, or GB) in which space is to be accounted. If the space unit is TRACKS, the maximum track size of the device is used. If the space unit is KB, MB, or GB, then the number of tracks requested is converted to the desired unit.

Current Space Amount: The amount of space currently occupied by datasets assigned to the specified account. The date and time of the most recent request for space in the account are also recorded by DLimit.

Maximum Space Amount: The maximum amount of space which may be occupied by datasets assigned to the account. If a request for space is received which will cause the current space amount in the account to exceed this value, warning messages may be issued or the request for space may be rejected by DLimit, depending upon the account mode setting.

Maximum Space Amount: The maximum amount of space which may be occupied by datasets assigned to the account. If a request for space is received which will cause the current space amount in the account to exceed this value, warning messages may be issued or the request for space may be rejected by DLimit, depending upon the account mode setting.

High-water-mark

High-Water-Mark: The largest amount of space which has ever been requested for the account. The date and time of the occurrence of the high-water mark are also recorded by DLimit.

DLADD EAST23 MODEL(MODLEAST)
DLALTER EAST23 WARN(80) MAX(5000M)   

How SCC-DLimit calculates space amounts

SCC-DLimit receives control in two system exits, IGGPRE00 and IGGPOST0, which are entered during the allocation of a new dataset or when an existing dataset is extended, renamed, deleted, or has space released. SCC-DLimit determines the size of the space request from the parameter list passed to these exits.

The size of the space request is sometimes presented in units of tracks, which SCC-DLimit then converts to kilobytes, megabytes, or gigabytes, depending upon characteristics of the DLimit account to which the space is assigned. The track size used by SCC-DLimit in its calculations is the maximum track capacity according to the LSPACE SVC:

• 3380-type devices: 47,968 bytes
• 3390-type devices: 58,786 bytes
• Unknown device type: 57,664 bytes (maximum blocksize of a 3390-type device)

Because tracks are not always filled to capacity, and not all allocated space is always used by applications, the space amounts calculated by SCC-DLimit may be slightly different than the values obtained by other methods.

Using the rules language to assign accounts

The primary function of the DLimit rules is to assign requests for space to an account. The DLimit rules are executed whenever the system makes a request for space on a volume. When coding the DLimit rules, it is advisable to use durable variables when determining what account to assign. That is, the rules should test and use variables which will not change between executions. It is better to use &QUAL1 than &STEPNAME or &DDNAME, for example, since the high-level qualifier of a dataset is unlikely to change, whereas the stepname or DD name assigned to a dataset may differ from one job to the next.

The following example assigns a request for space to one of several different accounts, depending upon the value of the high-level qualifiers of the dataset name and the type (permanent or temporary) of the space required:

DEFRULE SETACCT
  CONTINUE(NEXTIF)
  IF &QUAL1 EQ (PROD,TEST)
    THEN SET &DACCT1 = &QUAL1
         SET &MODEL = MODEL
         DO
           IF &TEMP = YES
             THEN SET &DACCT2 = "&DACCT1:TEMP"
         END
  IF &QUAL1 EQ P* &QUAL3 EQ WORK
    THEN SET &DACCT1 = 'PERMWORK'
         SET &DACCT2 = 'GROUP14'
      

If the account name does not already exist, default account values will be obtained from the MODEL account.

The name of the account(s) assigned to a dataset may be a fixed character string (for example, PAYROLL) or may be a symbolic variable such as &QUAL1 or &RACF_OWNER. A request for space may be assigned to up to 10 accounts (&DACCT1 through &DACCT10).

Automatically creating new accounts

DLimit assigns space to accounts when a SET &DACCT= statement is encountered in the rules while processing a request for space. If the account specified does not exist in the DLimit database, and dynamic creation of accounts is allowed, then the account will be automatically created. The AUTOCREATE(YES/NO) operand of the DLSYS command can be used to enable or disable automatic account creation.

An existing account in the DLimit database can be used as a MODEL account to be used as a basis for creating new accounts. When creating new accounts using a MODEL account, all of the values associated with the model account are copied into the new account.

When creating new accounts, the symbolic variables prefixed with ACCT_ can be used to override the defaults set by the MODEL account. However, for an existing account, these variables are ignored and the settings currently assigned to the account are used.

Do not create an excessive number of accounts

If you use low-level qualifier or some other characteristic of the dataset name to assign the account name, you should be careful not to create an excessive number of accounts. For example, if the rules specify

SET&DACCT = &DSNAME

then an account will be created for every dataset. This defeats the purpose of DLimit, which is to track the space assigned to applications or other logical groups of datasets. Similarly, account names shouldn't be created based on temporary dataset names, to avoid creating accounts such as SYS97321 or SYS98224, etc.

Each request for space that's examined by DLimit is assigned to an account by the DLimit rules when a SET &DACCT=statement is encountered.

SCC-DLimit reporting

Information can be obtained directly from the SCC-DLimit database via the ISPF interface, or via the DLMAINT batch utility. SCC-DLimit can also use the flexible reporting facilities of the Storage Control Center Monitor via program MONRUN for customized reporting on information contained in the SCC-DLimit database. The SCC-DLimit Sample Library contains several examples of reports that can be used to show the status of SCC-DLimit accounts, including those which exceed specified warning levels.