Business Data Parser - trillium_discovery - trillium_quality - 17.1

Business Data Parser Process

Step 1: The BDP identifies each word and phrase from the input data and compares them to word and phrase definitions in the Customized Definitions table. If a word or phrase is not specified in the table, the BDP assigns it an intrinsic attribute.

Step 2: When the BDP finds defined words and phrases in the table, it assigns them to output categories that you define in the Customized Definitions table (you can define up to 500 categories). Patterns are defined based on the sequence of categories on a row of data. The BDP looks up word combinations in the table and matches them to the patterns and substring patterns. If no match exists, the BDP considers this a bad pattern and writes the details to the exceptions file.

Example:

Words/Phrases	1995	Toyota	Camry
Categories	YEAR	MAKE	MODEL

Step 3: Standardized data is assigned up to 500 output attributes with the default names BP_USER1 through BP_USER500. These attributes correspond to the output categories you define in the Customized Definitions table. You can rename these output attributes to be more meaningful and easier to work with; for example, if the attribute BP_USER4 contains product name recodes, you could change it to Recode_Product_Name.

For each record, the BDP generates the following three output attributes:

• BP_USERxx. The USERxx attributes contain the standardized data.
• BP_USERxx_DISPLAY. The DISPLAY attributes contain the original data standardized in BP_USERxx.
• BP_USERxx_ORIGINAL. The ORIGINAL attributes contain the original data in mixed case used in the input data.

Data that the parser does not recognize is assigned to the bp_misc_data (miscellaneous) attribute.

Step 4: The customizable output schema determines which of the attributes are returned to the output. BDP output includes original data and recoded (standardized) data. The BDP passes output data to the Business Data Parser Repository (BPREPOS). The BPREPOS consists of fixed-field character data results including error codes and identification indicators. By default, the BDP output schema contains all of the attributes from the BPREPOS. Use the BDP Schema Editor to remove any unwanted attributes from the output. For information about how to view parser results, see Viewing Business Data Results.

Guidelines

Note the following when working with business data in the Control Center:

• The BDP process is available in Business Data Projects only. For information about projects, see Creating a Business Data Project.
• Run the Transformer process first. The output attributes from the Transformer are used as input to the BDP. If necessary, make modifications to Transformer attributes and rules. For example, you can remove unnecessary data from the attribute to be parsed, combine multiple data sources into one attribute name, and standardize some of the data to make it more uniform.
• Use the BDP Schema Editor to make changes to your attributes. You can also use the Transformer Schema Editor to make changes to your input attributes.
• Use the BDP Parser Tuner to review problems reported by the BDP and update the Customized Definitions word and pattern tables to identify input data elements.
• Business data cannot be tuned using the external Parser Tuner.
• If you are importing a BDP project created in the earlier Trillium version, you must run Parser Customization to regenerate the Customized Definitions table before running the BDP because the numeric codes for the user attributes have changed.