Spectrum Data Normalization examines terms in a record and determines if the term is in the preferred form.
Components
Spectrum Data Normalization consists of the following stages:
- Advanced Transformer—This stage scans and splits strings of data into multiple fields, placing the extracted and non-extracted data into a new or existing field.
- Open Parser—This stage parses your input data from many cultures of the world using a simple, but powerful parsing grammar. Using this grammar, you can define a sequence of expressions that represent domain patterns for parsing your input data. Open Parser also collects statistical data and scores the parsing matches to help you determine the effectiveness of your parsing grammars.
- Table Lookup—This stage evaluates a term and compares it to a previously validated form of that term. If the term is not in the proper form, then the standard version replaces the term. Table Lookup can change full words to abbreviations, change abbreviations to full words, change nicknames to full names, and can correct misspellings.
- Transliterator—Transliterator converts a string from one script to another script, such as Latin to Arabic or Arabic to Latin.