Testing with Data in a Batch File - trillium_quality - 17.2

You can use predefined data to test cleansing rules, testing some or all of the records contained in a batch file. The batch file can contain a maximum of 5,000 records; additional records will be ignored. The input data can be fixed-format or delimited. Supported delimiters are comma, semicolon, tab, pipe, or a user defined delimiter.

To test using data in a batch file

1. Under Cleanse Batch Data, select the Delimited or Fixed tab, depending on the format of your data.

   Note: If using the sample initialization file, input and output information is already entered. Skip to Step 7.
   Figure 1. Batch Input File

2. Identify the characteristics of the input file.

   For delimited input data:

   1. Enter the name of the input file, or click Browse to select the input file.

   2. Select the delimiter from the drop-down list or enter a delimiter character.

   3. Select the encoding used in the input file. For example, if your input file contains Japanese data, select CP932. Note that the CRA supports all of the encoding schemes supported by TS Quality, but only the most frequently used encodings are listed. You can enter any supported encoding scheme.

   4. If your data includes text qualifiers, enter or select the quote character.

   5. Check Has Header if the input file includes a header line.

      For fixed format input data:

   1. Enter the name of the input file, or click Browse to select the input file.

   2. Enter the name of the ddx file that defines the data structure, or click Browse to select the ddx file.

3. If the field names in your batch file do not match the fields used by the cleanser, enable the With Map option and click the Map Cleanser Fields button. The Map Cleanser Fields window opens.

   Figure 2. Map Cleanser Fields

   In the Map Cleanser Fields window, you can:

   • Browse to open an existing map file.

   • Use the window to create a map file. Click in the User Input Field Names column and use the pull-down list to select a field to map to each of the input field names.

   • Click OK.

4. Identify the characteristics of the output file to be created.

   Figure 3. Batch Output File

5. In the Input Field Names text boxes, select repository fields. All of the fields in the DDL repository of the first enabled process in this project are available through the drop-down lists.

6. In the Output Field Names text boxes, select the repository fields that will contain cleansed text.

7. Click Cleanse to begin processing records. Notice that the Cleanse button label changes to Stop here. As your batch file is processing records, you can stop it at any time by clicking Stop Here; click Resume to continue processing.When processing has finished or you click Stop here, the first cleansed record is displayed in the CRA.

You can review batch data cleansing results in the CRA or in a report file.

To review batch data results

1. Use the Review buttons to review records forward and backward. To go directly to a specific record, enter the record number in the Record text box and click Go.

   Figure 4. Review Buttons

2. To review all cleansed data, click the View Report button , located next to the Review buttons. When batch data is cleansed, the cleansed data is written to a report file and to an output file. While the report file contains all of the data displayed on the screen, the output file contains only the output fields and output results.

Figure 5. Batch Report File