Using Custom Expressions in Exception Monitor - dataflow_designer - spectrum_quality_1 - 23.1

Spectrum Data Quality Guide

Product type
Software
Portfolio
Verify
Product family
Spectrum
Product
Spectrum > Quality > Spectrum Quality
Version
23.1
Language
English
Product name
Spectrum Data Quality
Title
Spectrum Data Quality Guide
First publish date
2007
Last updated
2024-03-04
Published on
2024-03-04T22:52:13.486265

You can write your own custom expressions to control how Exception Monitor routes records using the Groovy scripting language to create an expression.

Using Groovy Scripting

For information on Groovy, see groovy-lang.org.

Groovy expressions used in the Exception Monitor stage must evaluate to a Boolean value (true or false) that indicates whether the record is considered an exception and should be routed for manual review. Exception records are routed to the exception port.

For example, if you need to review records with a validation confidence level of <85, your script would look like:

data['Confidence']<85

The monitor would evaluate the value of the Confidence field against your criteria to determine which output port to send it to.

Example: Checking a Field for a Single Value

This example evaluates to true if the Status field has 'F' in it. This would have to be an exact match, so 'f' would not evaluate to true.

return data['Status'] == 'F'; 

Example: Checking a Field for Multiple Values

This example evaluates to true if the Status field has 'F' or 'f' in it.

boolean returnValue = false;
if (data['Status'] == 'F' || data['Status'] == 'f')
{
 returnValue = true;
}
return returnValue; 

Example: Evaluating Field Length

This example evaluates to true if the PostalCode field has more than 5 characters.

return data['PostalCode'].length() > 5; 

Example: Checking for a Character Within a Field Value

This example evaluates to true if the PostalCode field has a dash in it.

boolean returnValue = false; 
if (data['PostalCode'].indexOf('-') != -1) 
{ 
	returnValue = true; 
} 
return returnValue; 

Common Mistakes

The following illustrate common mistakes when using scripting.

The following is incorrect because PostalCode (the column name) must be in single or double quotes

return data[PostalCode]; 

The following is incorrect because no column is specified

return data[]; 

The following is incorrect because row.set() does not return a Boolean value. It will always evaluate to false as well as change the PostalCode field to 88989.

return row.set('PostalCode', '88989'); 

Use a single equals sign to set the value of a field, and a double equals sign to check the value of a field.