What's new
Automatic Cross view filtering on fields with same name
It is now possible to apply a filter to the dashlets that has a field by the same name with the same type across sources. By default, and for all existing dashboards, this setting will be unchecked.
AWS Implementation for Kubernetes
Added support for running the app server in Kubernetes i.e whenData360 DQ+ is deployed to Kubernetes a Tomcat application server is initiated on each Kubernetes node and deploys Data360 DQ+ application into each of them. Data360 DQ+ is then connected to the Postgres database, S3 etc running outside of Kubernetes cluster.
Big Data Warehouse Cost Avoidance
Added the ability to disable Redshift , Synapse, and Vertica from being used as the data store for data views.
Added new 'Low Volume' option in storage type for data view
A Low volume data view is backed by the standard Postgres database (the one used for case stores, for example). Switching a data view storage type to Low Volume from High or Regular performance incurs a migration process upon save that moves the data from Redshift/Vertica/Synapse to Postgres and vice-versa.
S3 bucket access via STS Assume Role call
Data360 DQ+ is enabled with S3 cross account bucket access via STS Assume Role call. This configuration returns a set of temporary security credentials that can be used to access AWS resources. These temporary credentials consist of an access key ID, a secret access key, and a security token. AssumeRole allows cross-account access.
S3 bucket access in different regions
Enhanced Data Stores for external S3 data to be able to access S3 buckets in different AWS regions.
Ability to customize Home screen Resources Section
It is now possible to manage the resource section on theData360 DQ+ home page. There is also an option to publish your own content and training material within Data360 DQ+ homepage.
Larger box to edit SQL statements on Data Stores
Data360 DQ+ has option to edit the SQL statements in database data stores using SQL editor.
Copy and Paste Functionality in Process Model
Copy and Paste functionality has been implemented in all the Process Model nodes such as external stage event, Stage Intermediate Event, Execute Stage/ Process Task, etc. This functionality works similar to how copy and paste works in the Analysis Chain.
Other Changes
- Updated spring framework and spring-security components to 5.3.22 and 5.7.3 respectively.
- Added search field in analysis and process model to filter the available nodes for selection.
- Added rate limiter to prevent password reset from being called too often.
- Allow admin to change internal user's authentication type to OIDC or SAML in addition to AUTH0 in the Users screen.
Corrected issues
| Issue Summary | Issue Number |
|---|---|
| Increased the maximum expression length from 500 to 1500 in consistency, completeness, regular expression and general expression checks of Rule Library. | ISX-11365 |
| Added backupservernode parameter in Vertica jdbc url. This is parameter is used to failover to secondary Vertica nodes if master Vertica node is down. | ISX-11161 |
| Fixed an error where DQ+ Expression errors were not visible until the expression window was closed | ISX-11361 |
| Fixed caching issue where data store queries were not picking up the latest data. | ISX-11289 |
| Prevent deletion of rule group or rule if it is being used in analysis or data store. | ISX-11213 |
| Show Current Filters menu item on dashlet has been renamed to Show Current Dashboard Filters. | ISX-11369 |
| Fixed an unexpected error that appears in dashboard when filter dashlet is copied from one sheet to another sheet using a different data source. | ISX-11368 |
| Fixed error when copy-and-paste collapsible group dashlet involving different data sources. | ISX-11364 |
| Fixed an error that happens occasionally when using the back button in browser after switching environment in view or edit screen of dashboard. | ISX-11150 |
| Added validation of supported image file types when embedding image in noticeboard content. | ISX-11173 |
| Prevented profile fields with invalid field categories from being saved in the system. | ISX-11301 |
| Fixed incorrect time zone being used for displaying time series data values. | ISX-11214 |
| Fixed bad URL link in email sent out for comment update. | ISX-11227 |
| Fixed error in mutation API updateTenantInfo() when retentionSchedule is not provided. Also allow tenant domain to be used as the tenantId in the tenantInfo() query API. | ISX-11178 |
| Fixed x-axis display issue in bar chart when there's only one data point. | ISX-11284 |
| Fixed an issue where the UI may not be displaying the correct information for the "Mapped Groups", "Mapped Admin Groups", or "Initial Group Assignment" in the Users Settings dialog if these values are updated using API with certain values. | ISX-11278 |
| Fixed broken function help links due to CSP violations. | ISX-11215 |
| Fixed occasional errors that appear when more than one stage is being used in dashboard. | ISX-11235 |
| Fixed display issues of the favorites or recently updated menu items in the top toolbar when the paths are long. | ISX-11234 |
| Changed GraphQL APIs /tenantInfo and /updateTenantInfo end points to use TeantPreference instead of UserPreference. | ISX-11125 |
| Introduced security enhancements to prevent HTML injection in emails. | ISX-11141 |
| Fixed issue where date fields are having time-related values when data are processed in analysis. | ISX-11384 |
| Fixed issue of not able to save the data view for enterprise deployment if Big Data is not configured. | ISX-11399 |
| Fixed an issue where internal users are still able to login using username/password after the application is switched to use SSO authentication. | ISX-11391 |
| The email used by the users for registration or authentication is now case insensitive. Because of this change, the system will disallow adding users with the same email with different letter cases. | ISX-11142 |
| Fixed ordering issue of arguments in Sqoop In/Out analysis nodes. | ISX-10130 |
| Fixed the color issue of the header check box in data grid, and the layout issue of the Edit Data Quality Check dialog in analysis. | ISX-11413 |
| Fixed an issue where dashboard UI might be unresponsive under certain circumstances. | ISX-11409 |
| Fixed an error in database data store where the "nullable" setting for a field is incorrect after it has been removed as identity field. | ISX-11444 |
| Added validation to createDataView() and updateDataView() GraphQL APIs to make sure primary data store name is specified in the data view definition. | ISX-11439 |
| Improved performance of retention processing for internal data stores which contain many part files (e.g. about 100 for each execution) and with retention period set long enough so that the total number of files is in the tens of thousands. Also, improved viewing of this data store listing these files. | ISX-11451 |
| Fixed error in pipeline tree if a node is selected together with one of its child nodes for export. | ISX-11432 |
| Fixed the issue of not able to cancel out of the "Unsaved Changes" dialog when adding a new computed field in data view. | ISX-11438 |
| Fixed color issues in dashboard's filter fields and data grids. | ISX-11336 |
| Fixed layout and cosmetic issues in the Analysis Settings dialog. | ISX-11466 |
| Fixed execution error in Analysis edit mode involving Union node with empty input data. | ISX-11457 |
| Preserve 3 output fields in REST API node as input fields into subsequent REST API node during node saves. | ISX-11452 |
| Fixed the issue where rollback command asynchronously so UI stays responsive. | ISX-11470 |