Other - Data360_DQ+ - Latest

Data360 DQ+ Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 DQ+
Version
Latest
Language
English
Product name
Data360 DQ+
Title
Data360 DQ+ Help
Copyright
2024
First publish date
2016
ft:lastEdition
2024-07-09
ft:lastPublication
2024-07-09T15:09:58.774265

The Other tab enables you to define or select the following, as appropriate:

  • Dashboards.

    Add or remove dashboards as required.

  • Case Selection Search Screen.

    The search screen displayed when the case store is selected.

  • My Cases Search Screen.

    The search screen displayed when My Cases is clicked.

  • All Cases Search Screen.

    The search screen displayed when All Cases is clicked.

  • Environment Groups with Access to All Cases.

    Define the names of the environment groups, as appropriate.

Defaults

The Defaults panel enables you to define which of the following are selected or displayed by default:

  • Default Dashboard.

  • Default Search Screen.

  • Default Queue.

  • Initial View.

    Select the initial view to display, from the following, provided they have been defined previously:

    • Default Dashboard.
    • Default Queue.
    • Default Search Screen.
    • None.
Note: Previously defined case stores have a blank Initial View, and display the same view that they did before.

Section Title Overrides

Use this panel to customize the titles of the three main sections of the navigation panel, positioned to the left of the data store screen:

  • Dashboards.
  • Queues.
  • Search.

Enter the customized title in the appropriate field. If left blank, the titles remain at the default setting. They can also be returned to the default, by removing the customized title.

Data retention settings

You can use the Data Retention settings to set a time limit on how long a data store will store data.

  • Retention Period - Define the length of time for which a data store will store data. Data retention applied at the data store level will override retention period settings at the environment level made by your administrator.
  • Period Precision - Define the precision at which a retention period is applied.

For example, if a data store's Retention Period was set to 1 year and its Period Precision was set to Month, the retention period would span one year back from the current day, plus the remaining days to get to the beginning of the month.

Retention example one

6/5/2016 6/5/2015 1 year Retention Period 6/1/2015 ...with Month Precision

  • The current day is 6/5/2016.
  • The Case Store Retention Period is set to 1 year.
  • The Period Precision is set to Month.

In this example, the system would first look back one year to 6/5/2015. Then, since Precision is set to Month, the system would go to the beginning of the month, to 6/1/2015. At this point, any data that was loaded into the Data Store before 6/1/2015 would be removed.

Retention example two

  • The current day is 6/5/2016.
  • The Environment Retention Period is set to 1 year.
  • The Period Precision is set to Quarter.

In this example, the system would first look back one year to 6/5/2015. Then, since Precision is set to Quarter, the system would go to the beginning of the quarter, to 4/1/2015. At this point, any data that was loaded into the Data Store before 4/1/2015 would be removed.

Retention period over time

Tip: Data retention is applied once per day, and is calculated from the end of the most recent successful load. If a load fails, data retention will be calculated from the failed load's start time.

Retention period data flush

Over time, data will be flushed from data stores based on the specified data retention settings. For instance, in the example above, data from Load 1 on 4/29/2015 is flushed on 5/1/2016. This is because a 1 year retention period with Month precision is used, so, on 5/1/2016, the 4/29/2015 Load falls outside of the calculated Retention period (which spans from 5/1/2015 to 5/1/2016).

Note: This policy applies only to Internal data stores of type database, which are not owned by (associated with) a case store. For data stores which are owned by a case store, the settings on the data store Other tab are ignored, and the associated case store retention settings are applied. When a case reaches a terminal state and goes outside of the retention period, the records in a data store which are associated with this case will also be deleted.
Note: During retention processing, any records in an internal database based data store that have a SysDeleted field value of true will be deleted from the data store.
Note: Due to the fact that Data360 DQ+ cannot delete the contents of an external location, data retention is not permitted with non-default external data stores. Data retention is permitted on external data stores of the default type and internal data stores.

Partition by fields setting

This option is available for external (non-default) data stores of type HDFS or S3. For such data stores, if the partition fields are specified, then the organization of files in the data store will be in the Hadoop format with folders of the form <fieldname>=<fieldvalue> for each of the partition fields, in the order they are specified in the UI. If no partition fields are specified, the files should be directly under the path specified for the data store.