Execute Rule Library - Data360_DQ+ - Latest

Data360 DQ+ Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 DQ+
Version
Latest
Language
English
Product name
Data360 DQ+
Title
Data360 DQ+ Help
Copyright
2024
First publish date
2016
ft:lastEdition
2024-07-09
ft:lastPublication
2024-07-09T15:09:58.774265

Use the Execute Rule Library node to apply the reusable rules you have created within a Rule Library to a data set you are working on in an analysis. The output of this node is a data set containing the fields under evaluation, a result field, and an error field.

For more information, see Rule Library.

Properties

Display Name

Specify a name for the node.

The default value is Execute Rule Library.

Apply System Data Quality Checks

Data360 DQ+ can apply predefined data quality checks to fields that have a system defined semantic type. The semantic type is taken from the field definition in the data store the field is defined in. You can also add your own Data Quality Checks for fields that have a custom-defined semantic type.

If Apply System Data Quality Checks is selected, the Execute Rule Library node output includes a boolean field displaying the result of the check for each value in the input data set for fields that have a system defined semantic type. The name of each field uses the format <FIELD_NAME>SysCheck, for example citySysCheck for a field called "city".

Tokenized and encrypted fields

If a tokenized or encrypted field is used in a rule, it will be detokenized or decrypted before the rule is executed. The resultant data:

  • Is only stored in memory, never on disk.
  • Never leaves the data quality rule operation, or is passed to nodes downstream.

Data Quality Checks

  1. To add a new data quality check, click Add. The Add Data Quality Check dialog is displayed.
  2. Enter a name for your new data quality check.
  3. Complete the tabs.

Details tab

  1. Select the Rule Library that contains the Rule Group you want to apply to your data set.
  2. From the available Rule Groups in the selected Rule Library, select the Rule Group that contains the rules that you want to execute.
  3. Enter a name for the Result Field. The Result Field is a boolean field that contains the result for each evaluated record. The result is True if all the rules that are applied to the record produce a True result.
  4. Enter a name for the Error Reason Field. If a record's Result Field evaluates to False, the Error Reason field contains the list of reusable rules that produced a False result. If a record's Result Field evaluates to True, the Error Reason field is empty.
  5. Tick the box for Execute All Rules in a Group, or select individual rules in the Rules to Execute panel.

Placeholder Mapping tab

The Placeholder Mapping tab is used to map incoming fields to the Placeholder Fields within the selected Data Quality Rule Group. Incoming fields that are mapped to Placeholder Fields will be evaluated in the selected reusable rules wherever the Placeholder Fields are referenced, in the rules' definitions.

For each of the Placeholder Fields applicable to the rule group, click Edit to open the Edit Placeholder Mapping panel. The Placeholder Data Type is automatically filled in based on the rule.

Choose a Field Selection Type to determine which fields in the Data Store will be tested against the rule group.
  • Specified Field - select one field to be tested from the Select Incoming Fields dropdown.
  • Specified Fields - select multiple fields to be tested from the Select Incoming Fields dropdown.
  • All Fields based on Semantic Type - checks all fields that have been defined with the Semantic Type that the Rule Library has been configured to test against.
  • All Fields - checks all fields.