Data360 Analyze what's new - Data360_Analyze - Latest

Run multiple sets of nodes simultaneously

You can now run multiple sets of nodes simultaneously. This means that you can start running a set of nodes while other nodes are already running. Each time you do this, the nodes selected to run will be in a separate "execution". The number of concurrent executions that can be ongoing per user per data flow is controlled by this property: ls.lae.execution.concurrentExecutionsPerSession

This is a server side property that will affect all runs by all users. The default is 4.

Share run data

You can share run data with other users to work collaboratively by saving run states and temporary execution data in a shared folder in the Directory. This is possible even when execution sandboxing is enabled. If the temporary execution data is saved in a location that is accessible to all users in your team, then all users will be able to view the run data associated with the data flow, without needing to re-run the data flow themselves.

On the Designer page Run tab, you can specify where to store temporary run data and you can edit the temporary run data location for an existing run.

All users (except reviewer only users) can set their default temporary execution data location from the Directory on the Settings > Designer Settings page.

Data viewer enhancements

A number of usability enhancements have been made to the data viewer, including:

• Sort multiple columns: A new Sort Multiple Columns option has been added to the tab menu, allowing you to edit the column sort order. You can hold down the Ctrl button and left click a column header. Each time you click, the column is added to the sort. The sort order is given by the number you see on the sort indicator.
• Freeze rows and columns: You can freeze rows by right-clicking a data viewer cell and selecting Freeze Rows Top. This will freeze the currently selected row and all rows above it. Frozen rows remain in place as you scroll vertically. You can freeze individual columns by selecting the Freeze option from the column header menu. Frozen columns will remain in place when you scroll horizontally.
• Search columns: A new Search Columns option has been added to the tab menu. Selecting this option opens a dialog that lists all columns in the data viewer. You can use the search bar to filter for a specific column. Choosing a column from the list will scroll the corresponding column into view in the data viewer.
• Hide columns: A new Hide option has been added to the data viewer column menu, allowing you to hide the current column, or hide several selected columns at once. Hidden columns can be shown again using the column menu or the tab menu.
• Search tabs: A new Search Tabs option has been added to the tab menu allowing you to filter tabs using the search field. Selecting a tab will select it in the data viewer.
• Select node: A new Select Node option has been added to the tab menu. Selecting this option will select and center the node that the tab was opened from. If the node is at a different composite level, the current composite level will change to that of the node.
• Open from canvas: You can now open the data viewer pop-out window directly from the canvas by holding down the Shift key when clicking on a pin.
• Go to column: When the horizontal scroll bar is active, the tab menu lists the names of the data viewer columns. Choosing an option from the dropdown will scroll the corresponding column into view.
• Filters and sorts: When filters and sorts are applied to an open data viewer, if the node is re-run and there are no changes to the output metadata then the data viewer stays open, the data is refreshed and the filters and sorts are re-applied.

Stop a running schedule

You can now stop a currently running schedule. This allows you to edit the schedule before re-planning. You can do this from the Schedules page, the Runs page or the details panel in the Directory when the schedule is selected.

SAML SSO group membership

When SAML SSO users login to Data360 Analyze, group membership can be configured to automatically update based on the information passed in via the SAML assertion.

On the Groups page in the Directory, you can filter the group list to show only:

• Local groups
• SSO managed groups
• LDAP imported groups

On the Users page in the Directory, you can filter the user list to show only:

• Local users
• LDAP imported users

Default roles for imported users and groups can be set independently. Previously the default role was shared.

Performance improvements

The performance of the application has been improved, particularly when running data flows:

• Reduced the time taken from a run being initialized to when the nodes start executing when the entire data flow is being run and there is no existing state (for example, for scheduled runs).
• Reduced the amount of parameter evaluation that needs to occur on all Java nodes.
• Improved the performance of the initial time for an ad hoc run to start node execution when running a subset of nodes in complex data flows where upstream nodes may need rerunning as a result of the call to run.

Breaking changes

• Data flows built in 3.16.0 will not be backwards compatible with earlier versions.
• RedHat Enterprise 7.X 64-bit is no longer supported.
• Due to Java 17 incompatibilities in the Snowflake database driver, for the database nodes (JDBC Store, JDBC Query, JDBC Execute, Database Profiler and Database Metadata) if a DbType of Snowflake is specified and no DbUrl property is provided, the nodes will be automatically configured to communicate with the Snowflake database using the "JDBC_RESULT_QUERY_FORMAT" set to JSON. When forming connections to a Snowflake database using the shipped driver, if you do not specify a DbType or if the DbUrl is set explicitly, you will need to manually configure the node to use the JDBC_RESULT_QUERY_FORMAT option in the DbUrl or DbOptions property. Note that this is a breaking change if you have configured this node in Data360 Analyze prior to the 3.16.0 release. If you have existing nodes configured to connect to Snowflake using our shipped driver, where the DbUrl property has been set explicitly or if you have not set the DbType property, you must add the JDBC_RESULT_QUERY_FORMAT option to prevent the node from failing.

JDBC driver updates

Third party library updates

Other changes

• Execution sandboxing: When using a Docker l2bridge network with execution sandboxing, the trusted host configuration is setup by default for the l2bridge scenario where communication from the web application will appear to the containers to come over the host's IP instead of the gateway IP. Added configuration in the executionContainers enable script to be able to override this behavior.
• Notification dialog: The notification dialog can now be dismissed by clicking outside of it.
• Tabular XML File node: A new SubstituteInvalidCharacters property has been added to the Tabular XML File node. If this property is set to True, the node will substitute both characters that are reserved characters in the metadata (for example, newline, ':') and characters which are not in the Data360 Analyze server's character set into a valid alternative such that the field names in the metadata are valid.

Corrected issues