Variables are part of every process model node. You can define variables to be passed between nodes and to be used in logical expressions which determine how the process model executes.
Pass variables between nodes
When passing a variable from one node to the next, your selection will be limited to "one node prior".
To pass a variable through multiple nodes, you can use Property Reference variables.
System defined variables
When a node is used to select an executable stage, system defined variables come attached to that stage. These variables are created by Data360 DQ+ when the stage executes or loads.
You cannot change system defined variables, but they can be passed from node to node and used in logical expressions.
Define a new variable
You can define new Output Variables on a node within your process model for custom use.
- Select the node on which you want to define the output variable.
- From the Properties panel on the right of the screen, click the Add button on the Output Variables property.
- Enter a Name for the new variable.
- Choose from these property types (the property types that are available vary according to the node that is selected):
- Literal - Creates a variable with a fixed value. Literals may then be used by other parts of a process model, such as an Expression variable, or in other nodes. Available data types include String, Datetime, Number and Boolean.
- Expression - Applies logic to other variables. May use variables from an incoming node, from the node on which the expression is being built, or from the process model itself. When writing expressions, you can use the following syntax to denote the source of the variable:
input
- Incoming noderesult
- Current nodeprocess
- Entire process model
Since multiple variable sources can be used in a single Expression, variables must be prepended by their source using curly braces and a period. For example:
{input}.resultCode
- Refers to the system defined result code variable from an incoming node.{result}.literalX
- Refers to a user defined literal variable calledliteralX
within the current node.{process}.refEndTime
- Refers to the system defined end time variable that exists for the entire process model. - Property Reference - References variables from other nodes.
For example, if there were a variable (X) in TaskA that you'd like to use in TaskB, you could create a Property Reference variable (Y). Doing so would effectively pass variable X to TaskB, making it available for use in logical expressions.
Note: Property References are restricted to only "one node upstream". To pass a variable through multiple nodes, you can define multiple Property Reference variables.For example, if there were a variable (X) in TaskA that you'd like to use in TaskC, you would first create a Property Reference variable (Y) in TaskB. Then, you'd need to create another Property Reference variable (Z) in TaskC, referencing Y.
- External - Available in the External Start Event node. References system level variables that enable specific functionality. External type variables:
Name
Purpose
commentLink
Used to access a Dashboard Control's system generated comment link. Useful when constructing Custom Notification Processes, with a Notify Task.
thresholdResults
Used to access a JSON object representing a Dashboard Control's threshold violations. Useful when constructing Custom Notification Processes, with a Notify Task.
- Choose whether to encrypt the variable. Encrypted variable values are not shown to a user viewing executions of stages where the variable is provided as an execution parameter.
- Select a Data Type and enter a Value for the new variable.
- Click OK.
Create variables for use in an analysis
If a process model is to be executing an Analysis, you can access variables that exist in the process model within the Analysis by using the RUNTIME function.
For example, if you wanted access to the workId of the Analysis, you could create a new column within the Analysis using the expression RUNTIME('workId')
. Provided the Analysis was being run by a process model, the new column would then populate with the workId of the Analysis at runtime. See Runtime Properties.
Execution result codes
The following numeric codes represent the execution status of a Data Stage. They can be used in process model expressions, to create the Model's logic:
Result code |
Status |
---|---|
0 |
Succeeded - Indicates that the Data Stage has executed without error. |
1 |
Failed - Indicates that something has caused the execution to fail. More details should be available in the Result Detail field. |
9 |
Lease Expired - Data360 DQ+'s execution engine features an Expired Lease Job, which checks for inactivity due to random error from machine or process crashes. If an execution has crashed, it will be marked with a Status of "Lease Expired" and a Result Code of 9. |
8 |
Terminated - Applied to an execution that either crashed or was manually terminated by a user. If the Expired Lease Job finds a crashed Data Stage execution and that Data Stage is part of a process model, the Process Model will be Terminated. |
Execution result types
Alternatively, result type strings can be used in Process Model expressions. The following result types are available for use:
Result Type |
Description |
---|---|
'SUCCESS' |
Indicates that the Data Stage has executed without error. |
'COMPLETED_WITH_ERRORS' |
Indicates that the Data Stage has executed successfully but that minor errors occurred. This code can usually be safely ignored. |
'FAIL' |
Something has caused the execution to fail. More details should be available in the Result Detail field. |
'TERMINATE' |
Applied to an execution that either crashed or was manually terminated by a user. |