Dimensions serve as top level containers for categorically similar data. For example, you might create a Product dimension for product data and a Customer dimension for data related to customers.
When exploring data, Dimensions will contain Fields that can be dragged-and-dropped onto the x or y-axis. Dimensions can contain the following field types.
Fields
Data view fields are built using fields from data stores. When a data view is executed, its fields will populate with the information held in the selected data store field.
In general, fields contain information describing one particular aspect of a dimension. For example, a Customer dimension might have a field called Customer Name, which will populate with all customer names when the data view is run.
If a field contains numeric type data, Data360 DQ+ will automatically make it available as a Measure that you can perform computations on in Dashboards and the Visualizer.
Data View field settings
Many of these settings work similarly to those in Data Store fields, see Fields. However, there some notable differences highlighted in this topic.
Description
This optional setting is a description of the field. This displays as a tooltip when you hover over the relevant field column header in the dashboard/s.
Display Format Pattern
This optional setting determines how numbers will be displayed in the Dashboards and Visualizations that use a Data View. In Data Views, Display Format Patterns work just as they do in Data Stores. In fact, if a field's Display Format Pattern has been configured in its Data Store, these settings will simply carry over to the Data View where the field is used.
In Data View fields, Display Format Pattern can be kept the same as the fields' settings in its Data Store, made unique, or simply left blank.
Text Alignment
Select an option to specify how to align text in grid and pivot dashlets. If the text alignment value for a field is not set, then the text for the field is aligned based on its data type. Text alignment for computed and other derived fields created in a dashboard is based on data type. Choose from Left, Center and Right.
By default, this property is not set.
Scale
This setting controls the number of digits to the right of the decimal point that will be displayed in Dashboards and Visualizations that use a Data View's field. For example, if value = 1.234 and Scale is set to 2, value will be displayed as 1.23 in Dashboards and Visualizations. By default, Scale settings made in Data Store fields are passed on to Data View fields.
Data Store field Scale can be overridden in a Data View field, however it is important to note that this can cause the values in each Data Stage to be different. For example, in a Data Store, if value = 1.999 and Scale is set to 3, value would be displayed as 1.999 in the Data Store UI or Analysis Sheets. If Scale were set to 2 for the same value field in a Data View, however, value would be displayed in Dashboards/Visualizations that use that Data View as 1.99.
This functionality can have an impact on Aggregate functions.
For example, consider a situation where you had 1000 records in the value field, all with the same id field value of 1, with Scale set to 3 for the value field in the Data Store. If you were to use value in an Analysis, Group By id, and then take the SUM of value, you would get 1999. If Scale were set to 2 for the same value field in a Data View, however, the same type of SUM aggregation would yield 1990 - a difference of 9.
Rounding Type
Rounding Up and Truncating in a Data View work just like Rounding up and Truncating in Data Stores.
It is important to note, however, that when Rounding Up or Truncating values in a Data View, you will be doing this to values that may have already been Rounded or Truncated in a Data Store.
By default, a field's Rounding Type is passed from Data Store to Data View automatically, but it can be changed if needed.
Precision
Precision controls the number of digits to include in an integer value.
A Data View field's Precision must always be greater than or equal to that of the value with the highest precision in a data set. This can be different than the Precision of a Data Store field.
For example, imagine you have a data set where the largest value for field1 is 1,000,000,000,000. This value has a Precision of 13; so, to properly execute the Data View, field1's Precision in the Data View must be 13 or greater.
Where you can run into issues is if Precision for field1 has been set to less than 13 in its Data Store - which may lead you to believe that Precision for the data set is less than 13. Doing so will not cause any issues in the Data Store or Analyses, but it will cause issues in the Data View execution. Namely, a field with too low of a Precision will throw the following Error in the Executions Log:
Unable to copy data into data store; nested exception is java.sql.SQLException: [Amazon](500310)[Amazon](500310)
If you encounter this error, try raising Precision in applicable fields and Rebuilding the Data View. Since Data Store field Precision may actually be lower than true data set precision, you may not be able to use this as a reference. If in doubt, there is no harm in setting Precision to something very high and seeing if this will solve the issue.
Currency Code
Currency Code must be set to display currency signs in Dashboards and Visualizations that use a Data View.
You can apply Currency Codes to both values that do or don't have Currency Format Patterning in the Data Store they come from. Note however that if the field values have a currency format, this will need to be specified in the Data Store's Format Pattern to be properly parsed.
Location Category
If a String field contains values that represent locations, you can choose a category to specify whether values represent Countries, Regions, State/Provinces, or Counties. Doing this will allow you to create maps in Dashboards and Visualizations.
For example:
Drill Field
Country field
State/Province field
County field
Map Field
When a Location Category is selected, you will also need to specify a Map Field option that matches how the String is formatted.
In order to account for the multiple formatting styles that exist, a number of Map Field options are available.
|
Country |
Region |
State/Province |
County Division |
|---|---|---|---|
|
2 Letter Country Code*
|
|
2 Letter Code* |
2 Letter Code* |
|
ISO 2 Letter Country Code |
|
FIPS Code |
FIPS Code* |
|
ISO 3 Letter Country Code |
|
HASC Code* |
County/Division Name (Standard names, e.g. ‘Cook') |
|
Country Name(Standard names, e.g. ‘Spain') |
Full nameFor example:Northwest |
Postal Code* |
|
|
Abbreviated Country Name* |
Abbreviated For example: NW |
State/Province Name (Standard names, e.g. ‘Illinois') |
|
Map Field Formatting Options
*Currently the best way to determine if your data conforms to one of these formats is to browse through the High Charts JavaScript files available here.
Simply find a file for a geographic region that is included in your dataset and then perform a Ctrl-F on the Map Field type of interest to see how values of that format are written.
Data Format for US Regional Map
Full names: Northwest, Southwest, Central, Northeast, Southeast
Abbreviated names: NW, SW, CR, NE, SE
Max Length
The Max Length property controls the maximum number of characters allowed in values for string fields.
The Max Length of a Data View field must always be greater than, or equal to, that of the value with the maximum length in a data set. This can be different than the Max Length of a Data Store field.
For example, imagine you have a data set where the longest String in field1 is 'ReallyBigData.' This String has a length of 13; so, to properly execute the Data View, field1's Max Length in the Data View must be 13 or greater.
Like with Precision, you can run into issues if Max Length for field1 has been set to less than 13 in its Data Store - which may lead you to believe that Max Length for the data set is less than 13. Doing so will not cause any issues in the Data Store or an Analyses, but it will cause issues in Data View execution. Namely, a field with too low of a Max Length will throw the following Error in the Executions Log:
Unable to copy data into data store; nested exception is java.sql.SQLException: [Amazon](500310)[Amazon](500310)
If you encounter this error, try raising Max Length in applicable fields and Rebuilding the Data View. Since Data Store field Max Length may actually be lower than true data set Max Length, you may not be able to use this as a reference. If in doubt, there is no harm in setting Max Length to something very high and seeing if this will solve the issue.
Semantic Type
Applying a Semantic Type to any type of field can help to further categorize the field's content and enable additional functionality throughout Data360 DQ+.
For example, adding the Airport semantic type to a string field containing airport codes could help to categorize a dataset containing unlabeled airport codes.
For more information on semantic types, see DQ+ Semantic Types
Displaying URLs as Hyperlinks in Dashboards with Semantic Type
As another specific example, the URL Semantic Type can be applied to String fields that contain String values which should be treated as URLs. Doing so will actually cause the URL field's Strings to be displayed as clickable hyperlinks in Data Grid dashlets. If the URLs are valid, clicking on these links will open the associated website in another browser tab.
Encrypted Fields
Specifying a Data View field to be encrypted will cause the field's records to be encrypted in storage.
Permissions Required to Modify Encryption
Any user with access to a Data View can encrypt a Data View field if it hasn't yet been encrypted.
Once a field has been encrypted, only Administrators and Users with Administer or Manage Security permission to the Data View will be able to modify the field's encryption settings.
An encrypted field that is being visualized may only be used as a Dimension or as a Measure with the None operation.
Encrypting a Data View field will not cause it to be masked when visualized. For masking, you will need to make the field secure.
If changing field encryption, a migration job will be performed upon save to encrypt/decrypt the data view's pre-existing data.
Tokenize fields
When tokenization is enabled and the Secure check box is displayed, a Tokenize check box and Tokenize Data Element Name text field are available.
When Tokenize is selected:
-
The Tokenize Data Element Name text field is enabled.
Enter the appropriate data element name.
-
The Secure check box is selected and disabled from further changes.
The remaining secure fields are still enabled.
Tokenize is selected and disabled from further changes, if the corresponding data store has it selected. Furthermore, if the data store field is not tokenized, then the data view cannot be tokenized either.
Secure Fields
Specifying a data view field to be a secure field will cause the field to be masked throughout Data360 DQ+. Masking is defined through use of the properties described below.
Secure field properties
- Hide this field when user does not have access - Users who have access to a secure field are those who belong to the environment groups chosen in the secure field's Environment Groups property. If this option is selected and a user who is not part of the secure field's Environment Groups uses the data view, they will not see the secure field at all. If this option is not selected, all users will see the secure field, but its contents will always be masked.
- Mask - Select how the secure field will be masked.
- Mask Pattern - If the User Defined Mask option is chosen, you can use Mask Pattern to define a preset or custom mask for the content of your secure field.
- Display Mask Pattern - If the User Defined Mask option is chosen, you can use Display Mask Pattern to define a preset or custom display mask for the content of your secure field.
- Environment Groups - Users in a secure field's selected environment groups will be able to see the field with its contents initially masked, as per Mask settings. Unlike other users, however, users within the selected environment groups have the option to unmask secure fields and see their actual contents. Unmasking can be performed throughout the product, at the individual record level or in bulk via Unmask All.If a user is not in a secure field's selected environment groups, they will either see the field as always masked or they will not see the field at all. This will depend on whether Hide this field when user does not have access is selected.
Inheriting settings from data stores
If a field is set as secure, all the security settings will carry over when the field is used in a data view.
When the secure flag's setting is inherited, it will not be able to be changed by anyone, unless it is changed in the data store.
Data360 DQ+ Administrators and users with Administer or Manage Security permission to the data view will be able to modify settings besides the secure flag. Other users will not be able to modify any of these settings, though.
You will see the effects of secure fields in dashboards and the visualizer.
Computed fields
Computed fields give you the ability to write expressions to manipulate the information held in selected data store fields.
For example, you could create a computed field called Customer Age, which extracts a customer's birth year from a birth date field and then subtracts that year from the current year.
Data360 DQ+ allows for a wide variety of computed fields combining fields, operators, functions and/or manually entered values. For additional help on the available functions, see Building expressions.
Syntax when using fields
Fields used within computed fields should be referenced as follows:
{dataStoreName}.fieldName
Where dataStoreName refers to the data store that the field comes from and fieldName refers to the field itself.
Drill fields
Drill fields are simply compound fields, built using multiple data store fields, where each field contains more granular information than the field above it. For example, you might have a drill field called Customer location. Customer location could contain a Customer Country field, a Customer State/Territory field, and a Customer City field.
When this Customer location drill field is used in a dashboard or visualization, users will be able to "drill in" and "drill out," visualizing data at different levels of granularity.
Group fields
Group fields are essentially folders, used to group related fields that don't have a granular relationship. A group field cannot be used as a field in a dashboard or a visualization; only the fields contained within the group may be used.
Group fields: Applying location category and map fields
If you have multiple fields that share the same Location Category and Map Field type, you can use a Group Field to group them together and set these options in bulk. For example, suppose you had three fields that represented countries using country names: vendor_country, customer_country, and salesrep_country. Applying a Country Location Category and Country Name Map Field to a Group field would mean that if you placed all three of these fields into the Group field, they would all inherit its location settings. Location Category and Map field would not need to be explicitly set for each _country field.
Geo fields
Geo fields allow you to group a Latitude and a Longitude field. Latitude and Longitude fields must be of the decimal type.
When building dashboards or visualizations, Latitude and Longitude Geo Fields can be visualized with the Map charting type as bubbles or as markers.
Date fields
Date fields are simply data view fields that populate with date-formatted data. For example, Customer Birth Date.