Execute Rule Library - Data360_DQ+ - Latest

Properties

Display Name

Specify a name for the node.

The default value is Execute Rule Library.

Apply System Data Quality Checks

Data360 DQ+ can apply predefined data quality checks to fields that have a system defined semantic type. The semantic type is taken from the field definition in the data store the field is defined in. You can also add your own Data Quality Checks for fields that have a custom-defined semantic type.

If Apply System Data Quality Checks is selected, the Execute Rule Library node output includes a boolean field displaying the result of the check for each value in the input data set for fields that have a system defined semantic type. The name of each field uses the format <FIELD_NAME>SysCheck, for example citySysCheck for a field called "city".

Tokenized and encrypted fields

If a tokenized or encrypted field is used in a rule, it will be detokenized or decrypted before the rule is executed. The resultant data:

• Is only stored in memory, never on disk.
• Never leaves the data quality rule operation, or is passed to nodes downstream.

Data Quality Checks

1. To add a new data quality check, click Add. The Add Data Quality Check dialog is displayed.
2. Enter a name for your new data quality check.
3. Complete the tabs.

Details tab

1. Select the Rule Library that contains the Rule Group you want to apply to your data set.
2. From the available Rule Groups in the selected Rule Library, select the Rule Group that contains the rules that you want to execute.
3. Enter a name for the Result Field. The Result Field is a boolean field that contains the result for each evaluated record. The result is True if all the rules that are applied to the record produce a True result.
4. Enter a name for the Error Reason Field. If a record's Result Field evaluates to False, the Error Reason field contains the list of reusable rules that produced a False result. If a record's Result Field evaluates to True, the Error Reason field is empty.
5. Optionally, enter a name for the Pass Reason Field. When specified, the system will output information for all passed rules performed.
6. Optionally, select the checkbox Output Error Reason and Pass Reason values in JSON format. You can easily parse this data in JSON format using a JSON Parser node. This will generate a collection of passed checks, each containing the rule name, the mapped field name, and any additional rule attributes and values defined in the rule library. Note that the new rule attributes are not included in the traditional reason output formatted string.

Rules to Evaluate tab

Select the checkbox for Execute All Rules in a Group, or select individual rules in the Rules to Execute panel.

Placeholder Mapping tab

The Placeholder Mapping tab is used to map incoming fields to the Placeholder Fields within the selected Data Quality Rule Group. Incoming fields that are mapped to Placeholder Fields will be evaluated in the selected reusable rules wherever the Placeholder Fields are referenced, in the rules' definitions.

For each of the Placeholder Fields applicable to the rule group, click Edit to open the Edit Placeholder Mapping panel. The Placeholder Data Type is automatically filled in based on the rule.