Advanced Java node configuration - 3 - 3.12

Data360 Analyze Server Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 Analyze
Version
3.12
Language
English
Product name
Data360 Analyze
Title
Data360 Analyze Server Help
Copyright
2023
First publish date
2016

For information on the Java API, see the "laeJavaApi" Zip file which you will find in the following location: <Data360Analyze installation directory>/docs/lae

Classpath modifications

You may want to modify the classpath on the Java node for inheritance and interface purposes, or to reference external classes and jars that are not included with your installation. If external classes are required for the Java node to run, you can add these to the Classpaths property.

For example, if you had written some utility Java code packaged into a .jar file called "myUtilityClasses.jar", which you wanted to reference in your Java node, the .jar file needs to be placed in a location where the server can access it.

If the .jar file was in the following location: /usr/home/myUtilityClasses.jar then you would need to add /usr/home/myUtilityClasses.jar to the Classpaths property. Note that if the path that you are referencing contains spaces, you do not need to enclose these in quotes in the Classpaths property.

Controlling downstream processing

Note: Broadcasting output and reschedule signals may not work correctly when the nodes are run in streaming mode. This is because in streaming mode, the downstream nodes start execution as soon as possible, rather than waiting for the predecessor nodes to complete. This means that by the time the signal has been set by a node, the downstream node whose execution status it is attempting to control will have already started execution.

To prevent this from happening, when constructing a Java node which signals such events, set that node to never be run in streaming mode:

  • Create a new boolean parameter on the node (name it "Streamable").
  • Set the Run Time Property Name to ls.brain.node.streamable.
  • Set the property to False.
  • If you are creating a library node, this property should then be hidden to prevent end users of the node from modifying it.

The ErrorAndSignallingExample node (Advanced composite > Controlling Downstream Processing) has five properties, in addition to those inherited from the Java library node, as follows:

Property Options Description
ErrorPosition
  • NONE
  • DURING_SETUP
  • DURING_PROCESS_ALL
  • DURING_CLEANUP
Tells the code where to fail the node. "NONE" implies that the node will not fail.
OutputNodeSignal
  • NONE
  • DIRECT_DEPENDENTS
Specifies the DependencySignalMode to use for the OutputDependencyEvent to broadcast.
ClockedNodeSignal
  • NONE
  • DIRECT_DEPENDENTS
  • ALL_DEPENDENTS
Specifies the DependencySignalMode to use for the ClockDependencyEvent to broadcast.
RescheduleSignal
  • True
  • False
Specifies whether or not a RescheduleEvent will be broadcast.
PreserveOutput
  • NONE
  • 1-16
If RescheduleSignal is set to True, specifies which (if any) of the outputs are to be preserved during the reschedule.

The ErrorAndSignallingExample node shows what occurs when the node fails at different stages of processing. If the node runs successfully, it shows how the different return states can be used to control downstream processing and re-run dependencies, see Creating run dependencies.

The mechanism within the Java node API for signalling the events is to call the broadcast method defined on AbstractNode:

protected void broadcast(NodeEvent event)

The events are not broadcast immediately, they are used to determine the return code of the node which is then sent back to the controller which determines how nodes should execute. While it is possible to call broadcast with a RescheduleEvent, ClockDependencyEvent and a OutputDependencyEvent, if a RescheduleEvent is broadcast, the ClockDependencyEvent and OutputDependencyEvent will have no effect.

If the node status is set to failed (i.e. ErrorPosition != NONE), none of the downstream nodes will be able to be correctly processed and no reschedule events will be sent. The signals are only ever broadcast in the case where the node finishes running successfully.

An interesting case is where the ErrorPosition property is set to NONE, and the OutputNodeSignal and ClockedNodeSignal properties or the RescheduleSignal and PreserveOutput properties are modified. Within the JavaCode property of the ErrorAndSignallingExample node, the code which uses these properties to modify the downstream processing is contained in the following lines:

broadcast(new OutputDependencyEvent(this, m_outputNodeSignal));

broadcast(new ClockDependencyEvent(this, m_clockNodeSignal));

if (m_reschedule) {

broadcast(new RescheduleEvent(this));

if (m_preserveOutput != -1) {

broadcast(new PreserveOutputEvent(this, m_preserveOutput));

}

}

The OutputNodeSignal and ClockedNodeSignal properties have already been read into the variables m_outputNodeSignal and m_clockNodeSignal respectively by this stage of processing. Similarly, the RescheduleSignal has been read into the m_reschedule variable, and the PreserveOutput property has been read into the m_preserveOutput variable (set to -1 if the property is unset or set to NONE).

By modifying these properties in the example data flow, you can investigate the different effects on downstream processing.

The following table details what will occur with each combination of OutputNodeSignal and ClockedNodeSignal properties. The corresponding returnCode which could be provided as an argument to the Script function setSuccessReturnCode (see setSuccessReturnCode) is shown in the left column:

returnCode ClockedNodeSignal OutputNodeSignal Behavior
200 NONE NONE Nothing connected (output clock or output) to this node can run. Normal behavior seen on failing nodes.
202 NONE DIRECT_DEPENDENTS Nodes connected to the output of this node can run. Nodes connected to the output clock of this node cannot run.
0 DIRECT_DEPENDENTS DIRECT_DEPENDENTS Nodes connected to the output clock of this node can run. Nodes connected to the outputs of this node can run. Normal node success behavior.
201 DIRECT_DEPENDENTS NONE Only nodes connected to the output clock of this node can run. Nodes connected to outputs of this node cannot run.
203 ALL_DEPENDENTS NONE Nodes connected to the output clock of this node can run. Nodes connected to the output clock of nodes connected to the output of this node can run. Nodes connected to the output of this node cannot run.
N/A ALL_DEPENDENTS DIRECT_DEPENDENTS Error, not allowed.
N/A * ALL_DEPENDENTS Error, not allowed.

Adding extra code blocks

In some circumstances, it may be impractical to write all of the Java code required into the one code block provided in the Java node, for example if a large amount of code is required and it may make more sense to have an object model, as opposed to one node class. Or, if you are creating a Java library node that you want to have some common base code, however each instance of this node may need to add or modify a small section of code.

Therefore, multiple Java code blocks are sometimes required. This section describes how this can be achieved using the Java node.

The same run time property name format used by the Java node is also required for additional code blocks. For each additional code block that you implement, two new property file properties are required:

  • ls.brain.node.java.classes.<something>.code (Java code to compile)
  • ls.brain.node.java.classes.<something>.class (name of the class to compile)

The class defining the node to run must be referenced in the NodeClass property.

For example, you have a node called AbstractMetadataPassThrough which provides the code required to open all of the input streams, setup the output metadata on the corresponding input metadata, and clean up after itself. For this to work, the node must have the same number of inputs as outputs. This node, however, cannot be run, and serves only as a library node which is to be inherited by other nodes.

The JavaCode defines the abstract base class, which defines the setup and cleanup methods required. The node introduces an additional code block, with the property name ImplCode. This property provides the skeleton for defining the required processAll and propertyBase methods when inherited by a node instance. (See Composition and inheritance).

The ImplCode property contains the following line:

public class {{^ImplClass^}} extends {{^JavaClass^}}

This states that value of the ImplClass property is to be taken as the name of the class defined within the code block. Further, this class is to extend the JavaClass class defined in the JavaCode property. If you use an instance of this library node, you can configure the ImplClass property.

The NodeClass property contains the following text:

{{^JavaPackage^}}.{{^ImplClass^}}

As the JavaClass property refers to an abstract class, this cannot be run as the node. Therefore, the node class which is to be run is declared in the ImplCode property, and is referenced by the ImplClass property.

Tip: You can view the run time property name for each property in the Edit Property Definition dialog from the Define tab of the Properties panel, see Property types.
Tip: The JavaCode and JavaClass properties have been hidden. You can view them from the Define tab on the Properties panel.

You have a second node called MultiFilter. The MultiFilter node inherits from the AbstractMetadataPassThrough node.

The code within the processAll method shown below, shows that the node continues reading records from each input until the inputs have no more records to read. On each record, a method processRecord is called on a class {{^ProcessRecordClass^}}.

@Override

public void processAll() throws NodeFailedException

{

boolean allDone = false;

int recNum=1;

while (!allDone) {

allDone = true;

for (int i=0; i<numInputs();i++) {

Record record = read(i);

if (record != RecordInput.EOF) {

allDone = false;

try {

{{^ProcessRecordClass^}}.processRecord(i, record, recNum, input(i), output(i));

}

catch (IOException ioe) {

logger().error(ioe, "Error Processing record on input: "+i);

throw fail();

}

}

}

}

recNum++;

}

The MultiFilter node also introduces an additional Java code block called ProcessRecordCode, which has the class name ProcessRecord, as specified in the ProcessRecordClass property. The node is a utility class and does not implement the Node interface. Therefore, unlike the AbstractMetadataPassThrough node, the NodeClass property does not need to be modified, as the actual Node to be run is still defined in the ImplCode property.

The code in the ProcessRecordCode property defines the class structure and then references other properties.