Configuring Advanced Options When Creating Entity Business Rules - trillium_discovery - trillium_quality - Latest

Trillium Control Center

Product type
Software
Portfolio
Verify
Product family
Trillium
Product
Trillium > Trillium Discovery
Version
Latest
Language
English
Product name
Trillium Quality and Discovery
Title
Trillium Control Center
Copyright
2024
First publish date
2008
Last updated
2024-10-18
Published on
2024-10-18T15:02:04.502478

When you create an entity business rule (EBR), you can include a Filter By, Group By, and Aggregate By option. These options define criteria that refine your profiling results. Use the advanced options to help track data quality compliance by highlighting errors in your data.

This topic describes how to use each option, and includes guidelines and examples.

Guidelines

Note the following when working with EBR options:

  • You define the three EBR advanced options by adding expression, attribute, and function parameters. Use multiple values to further refine the results.
  • Any filter, group, or aggregate expression added to a rule remains with the rule when it is copied to another entity, promoted to the Entity Business Rules Library, and exported from the Library. When a library rule is re-used, the associated rule inherits the option along with the rule's expression.
Note: If you use an EBR that includes the Filter By option in a Quality project, that option is appended to the rule's expression by an AND operator. The Group By and Aggregate By options are not carried over for use in Quality projects.
  • Each option generates information about errors that occurred during analysis, such as the number of rows that caused the error, the value of the first error row, and the corresponding error message. For more information, see Business Rules Analysis Details.
  • EBR options are not supported for dynamic entities. When you create an EBR for a dynamic entity, the options are grayed out.
  • The Group By option has no maximum limit.

Using EBR Options

You can enable these options in the Expression Builder when you first build a rule or when you edit an existing rule. Combine options when necessary to further enhance your results.

Filter By

Runs the business rule test against a specified set of rows and values.

Note: You filter rules using expressions that return a Boolean (true or false) result only.

Group By

Splits the results of a business rule expression into multiple categories based on parameters you define. You then drill down from the analyzed rule to view and compare the passing, failing, and total number of rows in each group.

Right-click the entity, select Business Rules > Drill down to Entity Business Rules, and then right-click the business rule and select Drill down to Group to display each group.

Note: You group rules using expressions and functions that can return any result.

Note about ulimits and nofiles

When using the Groups By option with a very large number of rules, say for example 300 rules, for a single entity, you will be opening 300 files at once (one for each group). However, if your UNIX ulimits only allows you fewer file handles than 300, you will receive an error.

In order to avoid this situation, it is recommended that you set the nofiles value to unlimited. Alternatively, assess the number of rules with groups you need to create and double the number for ulimits.

Aggregate By

Totals numeric values for business rule passing and failing results. You aggregate results using one of five mathematical operators: sum, average, maximum, minimum, and count. When used with the Filter By or Group By options, aggregation is run on the filtered or grouped rows. Aggregated group results show the aggregate value results over passing and failing rows for each group.

Note: You aggregate rules using expressions and functions that can return any result.