Detects duplicate data within specified fields, segregating the data into two outputs.
The first output contains all rows that have no duplicate collision, the second contains all those rows which do contain duplicates.
To remove duplicates, you can use the Remove Duplicates node.
Properties
InputExpr
Specify the expression to test for duplicates.
Detecting rows that have duplicate values in a single field:
id
Detecting rows that have duplicate values across multiple fields:
id, 'type', status
Note that you may need to surround a field name in single quotes if it is also a reserved keyword in Data360 Analyze Script.
A value is required for this property.
ErrorIfDuplicates
Optionally specify whether to generate an error if any duplicates are detected.
The default value is True.
Inputs and outputs
Inputs: Input to Validate.
Outputs: single occurrence, multiple occurrence.