Java node process flow - Latest

Data360 Analyze Server Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 Analyze
Version
Latest
Language
English
Product name
Data360 Analyze
Title
Data360 Analyze Server Help
Copyright
2024
First publish date
2016
Last updated
2024-11-28
Published on
2024-11-28T15:26:57.181000

For information on the Java API, see the "laeJavaApi" Zip file which you will find in the following location: <Data360Analyze installation directory>/docs/lae

The code within the JavaCode property should generally be a class which extends the com.lavastorm.brain.node.SimplifiedNode class.

In general, this class should have no constructor. If there is a reason to have a constructor (rather than simply performing initialization in setup as recommended), then there must be a no-argument constructor. The code skeleton that is provided by default with the Java node is already defined to extend this SimplifiedNode class.

The server then knows that it has a node to execute, and will call the methods defined on the node interface to execute the node.

For example, the JavaCode property contains the following line which ensures that this class inherits from the SimplifiedNode:

public class {{^JavaClass^}} extends SimplifiedNode

The Node interface is provided in the "laeJavaApi" Zip file which you will find in the following location: <Data360Analyze installation directory>/docs/lae. The process flow of the node is outlined in this topic and follows the path:

create > setup > processAll > cleanup > destroy

This path is always followed, assuming that no exceptions are thrown from the code within these methods, and assuming that the node status is never set to failed in any of these methods.

Create

The create method defined on the Node interface does not need to be implemented in the user-defined class in the JavaCode property.

The create method guarantees that by the time setup is called, all inputs have been opened.

Setup

The setup method is where you should perform the required steps to setup the node. This generally involves:

  • Reading any required properties.
  • Setting up output metadata (if that can be performed without needing to read input records).
  • Open outputs (if the output metadata can be setup).
  • Performing any other initialization that may be required.

As stated in the "laeJavaApi" Zip file (see <Data360Analyze installation directory>/docs/lae), if any errors occur during setup, the user defined code should catch these errors, log them appropriately, indicate that the node has failed via calling the fail() method, then throw a NodeFailedException, or return from the method normally.

If this method does not throw an exception, if the node fail() method was not called, then:

  • processAll, cleanup and destroy will all be called.
  • The cleanup method should be valid to be called to cleanup any resources allocated during the setup method, regardless of whether or not processAll fails.

Else:

  • Execution of the node will be terminated, processAll and cleanup will not be called, destroy will be called.

Otherwise:

If the method throws a RuntimeException:

  • The node controller will mark the node as failed, regardless of whether or not fail() has been called within the node code.
  • Node execution will be terminated, processAll and cleanup will not be called, destroy will be called.

Else (if the method throws a NodeFailedException)

  • The node status will not be changed by the node controller. If this has not been set to fail, then the node will not be marked as failed.
  • Execution of the node will be terminated, proceessAll and cleanup will not be called, destroy will be called.
Note: importantly, this means that the NodeFailedException does not signal to the node controller that the node status should be set to failed. Rather, the node controller assumes that the node has correctly set its own status. The NodeFailedException does signal to the node controller that further processing should be aborted.

In general, when an error occurs within the node, and is caught by the node code, the node code should first log any appropriate error messages then call:

throw fail();

The fail method correctly sets the node status to failed and returns a NodeFailedException. The throw call then throws this returned exception.

When fail is called within setup, or whenever exceptions are thrown from setup it is important to realize that cleanup will not be called. Therefore, prior to throwing an exception from setup, or exiting out of the setup method after fail has been called, it is the responsibility of the node developer to ensure that all I/O is cleaned up.

ProcessAll

The processAll method contains the business logic of the node. This will involve the reading and writing of records, and any business logic or data transformations that need to be applied to these records.

As stated in the "laeJavaApi" Zip file (see <Data360Analyze installation directory>/docs/lae), if any errors occur during processAll, the user defined code should catch these errors, log them appropriately, indicate that the node has failed via calling the fail() method, then throw a NodeFailedException, or return from the method normally.

If this method returns normally, and the node fail() method was called, then the node will be marked as failed.

Otherwise, if the method throws a RuntimeException, the node will be marked as failed.

In all cases, if processAll is called, cleanup and destroy will be called.

This means that if a NodeFailedException is thrown, but fail() is not called, then the node will not be marked as failed.

Cleanup

The cleanup method finishes execution of the node and cleans up internal resources allocated during the setup and processAll methods.

This method will always be called if setup completed successfully.

After this method is called, calls to input, output, properties, and log accessors should still be valid. The destroy method will always be called after this method.

While there may be some additional code required in the cleanup method to handle the closing of external files, database connections, sockets and so on, the following code should almost always be placed into the cleanup method to ensure that the node's inputs and outputs are cleaned up correctly:

cleanupIo();

Destroy

The destroy method defined on the Node interface does not need to be implemented in the user-defined class in the JavaCode property.