Configuring scorecard - discovery - 23.1

Spectrum Discovery Guide
Product type
Software
Portfolio
Verify
Product family
Spectrum
Product
Spectrum > Discovery
Version
23.1
Language
English
Product name
Spectrum Discovery
Title
Spectrum Discovery Guide
First publish date
2007
Last updated
2024-02-07
Published on
2024-02-07T17:21:58.768552

After selecting the source files of data to be scored, you need to configure the score card. Configuration involves:
  • Adding data quality dimensions on the basis of which scoring is to be performed
  • Adding and configuring rules for every DQ metric
  • Defining the threshold percentages for categorizing the data as bad, acceptable, or good
  1. On the Create Scorecard page, from the Entities list, select the entity for which you want to define the DQ dimensions and perform one of these steps, based on whether you want to select a DQ metric from the list or define a new one.
    1. Click the Add DQ Metric icon, and select the required parameter from Accuracy, Consistency, and Completeness.
    2. To define a new DQ metric, click Add DQ Metric, write your parameter where the cursor is blinking, and press the Enter button on your system.
    The selected DQ metric is displayed as a tab adjacent to the Add DQ Metric tab.
  2. Click the Add rule button to create a rule for the DQ metric.
    The Rule Configuration page is displayed.
    Note: On the Rule Configuration page, you can view the profile statistics for your entity by clicking on the PROFILE STATISTICS palette on the right side of the page. For more information, see Viewing Profile Statistics.
  3. Click the Create New Rule button to create a new rule for the DQ metric, or select an existing business rule from the Rule Name drop-down using the Select a Business Rule button.
  4. In case of a new rule, enter the name of the rule in the Rule Name field.
    Note: If you create any rule with the same name that exists in Glossary > Business Rule, you can choose to overwrite the existing rule.
  5. Enter the description for the business rule in the Business Rule field.
  6. In the Rule Builder, build your rule by using the Select Column and Select Operator drop down lists.
    1. The Select Column drop-down lists all the columns in the selected data source along with the data type. For example, PartyID (string).
      Note: For models built upon the Business Glossary entities, the properties of the entities are displayed here. For example, in case of a Party Business Glossary entity, the property can be Party ID (string).
      Note: If you have specified a business rule for the properties of the business glossary entities, it will be displayed here as suggestion.
    2. The Select Operator drop-down lists the operators. These are the operators that display based on the data type of the column you selected: exists in, equal, greater than or equal, less than, is between, ends with, contains, is not null, not equal, less than or equal, greater than, is not empty, starts with, matches, is null, is empty, matches semantic type, Does not contain, Does not end with, and Does not start with.
      • To make use of the custom semantic types defined in the Glossary, use the matches semantic type opeator. When you select it, the custom semantic types defined for the selected data type are displayed in the adjoining drop-down list. For example, if you have defined BusinessParty and PassportNumber as two custom semantic types for the string data type, these two options will be shown to you when you select PartyID (string) as the column and matches semantic type as the operator.
      • Use the Is between operator to fetch the values lying in between two primitive integer and primitive date type data.
    3. Use the text box after Select Operator to enter values for the expression.
      Note: Avoid using anchor operators, such as ^ (beginning of line) and $ (end of line) in the regular expression, as these are not supported by Lucene’s regular expression engine.
    4. To build multiple conditions, click the Condition button, and use the AND OR slider to join the multiple conditions.
    5. To create another group of conditions, click the Group button.
    6. Drag the conditions set up and down and across the groups to build the needed expression.
    The expression you built is displayed as Rule Expression.
  7. Optional: In case you want to save this newly built rule as a business rule in the business glossary section, click Save as Business Rule.
    Note:
    • You can edit and save an existing business rule too. The changes you make to an existing business rule reflects in the business glossary section.
    • Only rule expressions can be edited and saved to glossary rules. An edited business rule cannot be saved to business rules in the glossary.
  8. Click OK.
    You are navigated back to the Create Scorecard page, with the rule details displayed in a grid below the Add Rule button.
  9. To edit rule details, click the Edit rule icon in the Actions column.
  10. To delete the rule, click the Delete rule icon in the Actions column.
  11. Adjust the Threshold slider to define your score card as Needs Attention, Marginal, or Acceptable.
    Note: Color coding for the slider:
    • Needs Attention: Red
    • Marginal: Yellow
    • Acceptable: Green
  12. Click the forward arrow on the top right of the page to move to the next stage in creating scorecard.
    Note: At any point, to go back and edit details in the previous stage, click the backward arrow on the top right of the page.