If you have records that you want to match and you want to use more than one matching operation, you can create a dataflow that uses more than one match key then combines the results to effectively match on multiple separate criteria. For example, say you want to create a dataflow that matches records where:
The name and address match
OR
The date of birth and government ID match
To perform matching using this logic, you create a dataflow that performs name and address matching in one stage, and date of birth and government ID matching in another stage, then combine the matching records into a single collection.
This topic provides a general procedure for setting up a dataflow where matching occurs over the course of two matching stages. For purposes of illustration this procedure uses Intraflow Match stages. However, you can use this technique with Interflow Match as well.
- In Enterprise Designer, create a new dataflow.
- Drag a source stage onto the canvas.
- Double-click the source stage and configure it. See the Dataflow Designer's Guide for instructions on configuring source stages.
-
Define the first matching pass. The results of this first matching pass will be
collections of records that match on your first set of matching criteria, for
example records that match on name and address.
-
Drag a Match Key Generator and Intraflow Match stage to the canvas and
connect them so that you have a dataflow that looks like this:
-
Drag a Match Key Generator and Intraflow Match stage to the canvas and
connect them so that you have a dataflow that looks like this:
-
Save the collection numbers from the first matching pass to another field. This
is necessary because the CollectionNumber field will be overwritten during the
second matching pass. It is necessary to rename the CollectionNumber field in
order to preserve the results of the first matching pass.
-
Drag a Transformer stage to the canvas and connect it to the Intraflow
Match stage so that you have a dataflow that looks like this:
-
Drag a Transformer stage to the canvas and connect it to the Intraflow
Match stage so that you have a dataflow that looks like this:
-
Define the second matching pass. The results of this second matching pass will
be collections of records that match on your second set of matching criteria,
for example records that date of birth and government ID.
-
Drag a Match Key Generator and Intraflow Match stage to the canvas and
connect them so that you have a dataflow that looks like this:
-
Drag a Match Key Generator and Intraflow Match stage to the canvas and
connect them so that you have a dataflow that looks like this:
-
Determine if any of the duplicate records identified by the second matching
pass were also identified as duplicates in the first matching pass.
-
Create the dataflow snippet shown below following the second Intraflow
Match stage:
-
Configure the Duplicate Synchronization stage to group records by the
CollectionNumer field (this is the collection number from the second
matching pass). Then within each collection, identify whether any of the
records in the collection were also identified as duplicates in the
first matching pass. If they were, copy the collection number from the
first pass to a new field called CollectionNumberConsolidated. To
accomplish this, configure Duplicate Synchronization as shown
here:
-
Create the dataflow snippet shown below following the second Intraflow
Match stage:
- After the Stream Combiner you will have collections of records that match in either of the matching passes. The CollectionNumberConsolidated field indicates the matching records. You can add a sink or any additional processing you wish to perform after the Stream Combiner stage.