Streaming SQL - Latest

Data360 DQ+ Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 DQ+
Version
Latest
Language
English
Product name
Data360 DQ+
Title
Data360 DQ+ Help
Copyright
2024
First publish date
2016
ft:lastEdition
2024-07-09
ft:lastPublication
2024-07-09T15:09:58.774265

The Streaming SQL node streams input using SQL statements.

  1. Specify a Schema Name for each node that enters the SQL node by clicking the Edit icon in the Input Mapping table. This allows you to treat incoming nodes like tables, to which you can apply SQL statements.

    Optionally, select a datetime field in the Watermark Time Field, then enter an integer in the Watermark Window Threshold to specify the number of seconds to wait for late events. Note that if you specify a value for one of the watermark properties, the other is also required.

  2. On the Outputs tab, specify which fields you want to work with. You can work with existing input fields, or add new fields. Click Add Field, then add Input Fields or create a New Field.
  3. On the SQL tab, click Add to open the SQL Editor. Type your SQL query into the Editor.

    You will need to use the schema names that you defined in step one to refer to different input sources as if they were tables. For example:

    SELECT * FROM schema_A

    Tip: Type CTRL+SPACE for a full list of SQL keywords and functions. Typing CTRL+SPACE after you have already typed letters can also function as autocomplete.
    Note: Avoid putting a semicolon (;) at the end of statements, it is not required.

    You can add multiple SQL statements. For the first SQL statement, the SQL Editor will display the sources and their fields for autocomplete. In subsequent SQL statements, you can refer to the results of the prior SQL statement by using the name ${ResultTable}, but no autocomplete will be available in this case.

    To edit or delete an existing SQL statement, select the relevant row in the SQL table, then click Edit or Delete.

Properties

Input Mapping

Click the Edit icon in the Input Mapping table to open the Schema configuration dialog.

Enter a Schema Name which you can then reference from a SQL statement.

Configure the Watermark settings as required, then click Save.

Outputs

Click Add Field to specify the fields that you want to work with. You can add Input Fields, or create a New Field.

To remove a field from the list, select the relevant row and click Remove Field.

SQL

Click Add to open the SQL Editor. Type your SQL query into the Editor.