Outputs a filtered subset of rows from an input data set based on one or more groups of filter criteria that you define.
The simplest way to filter your data is by using the Filter grid in the Filter property. Alternatively, for more advanced techniques, you can use the Advanced tab.
You can also filter your data in the data viewer then apply the filter to your data flow, see Adding filters from the data viewer to your data flow.
Specify one or more filter criteria groups, which consist of one or more rows of filter criteria.
In the simplest case, you may wish to add just a single filter criterion to the default group provided. However, if you specify multiple criteria, you can choose to Match all (AND in Boolean logic terms), Match any (OR) or Match none (NAND) of these criteria in order to output a row of data.
By default, a single filter criteria group is created for you to populate. You can create further groups by clicking Add new criteria group. When you have multiple groups, you can then choose whether to Satisfy all groups or Satisfy any group in order to output a row from the input data set. This allows you to specify filter logic such as:
- Match all in group 1 AND Match any in group 2
- Match all in group 1 OR Match any in group 2 ORMatch any in group 3
The easiest way to configure the Filter node is to ensure that the input node has been run first. You can then simply choose a field to filter on from the drop-down list available for the Field column in the table. You can then choose an operator and specify a value.
If the input node has not been run, you can still configure your Filter node. In this case you must type in the name of the Field to filter on and then choose the data Type, before going on to choose an operator and specify a value. When the Filter node is executed, an error will be generated if this field does not exist, or exists as a different data type. Any rows in error will be highlighted in red in the filter criteria group table.
To remove a row from the criteria grid, click the menu on the far right of the row that you want to remove and select Delete.
Alternatively, select the Advanced tab if you wish to create more complex filter expressions that require parentheses and specific Boolean logic, such as:
Match none in group 1 OR (Match all in group 2 ANDMatch any in group 3)
If you have set your criteria using the basic filter grid, you can click the Advanced tab to view the underlying Python script.
Example data flows
A number of sample Data Flows are available from the Samples workspace, found in the Analyze Directory page.
In the Directory under the
/Data360 Samples/Node Examples/ folder, you will find "Filtering Splitting and Concatenating Data", which shows examples of how to use this node.
Inputs and outputs
Outputs: out1, multiple optional.