Publish to DQ+ - Data360_Analyze - 3 - 3.12

Data360 Analyze Server Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 Analyze
Version
3.12
Language
English
Product name
Data360 Analyze
Title
Data360 Analyze Server Help
Copyright
2023
First publish date
2016

This node publishes data from Data360 Analyze to a data store within Data360 DQ+.

To establish a connection between Data360 Analyze and Data360 DQ+ configure the following node properties:

  1. EnvironmentName - The name of the environment within Data360 DQ+ that will contain your data store. This is the same as the environment value in the top right corner of Data360 DQ+.
  2. DataStoreName and ItemAlternateId - The name and ID of the data store to create in Data360 DQ+. You can select any name and ID.
  3. DeploymentName and DeploymentId - The name and ID of the path that contains your data store in Data360 DQ+. This value does not need to match your Tenant ID.
  4. RequestMode - Select how to handle the creation of the data store. The recommended setting is Create or Update.
  5. Url - The full URL of Data360 DQ+, including tenantId, if applicable. For example: https://sampledqplus.precisely.com
  6. Username and Password - The credentials of a Data360 DQ+ user. You must create an Application User for this setting. A standard user will result in a HTTP 403 Forbidden error.
  7. Connect a node that contains your data to the Publish to DQ+ node and run the node.

Within Data360 DQ+, the data store will be created in the specified location.

Note: In Data360 DQ+, you must create a new pipeline and path and move the data store out of the incoming pipeline before it can be used. All data stores within the incoming pipeline are read-only.

Properties

EnvironmentName

Specify the name of the environment.

A value is required for this property.

DataStoreName

Specify the name of the Data Store in Data360 DQ+.

A value is required for this property.

ItemAlternateId

The UUID identifying the data store.

It is recommended that it is prefixed with a constant unique to the client for easy identification. The same id must be used for this item across all environments: dev, test, production etc.

A value is required for this property. The value should not contain spaces.

DeploymentName

Specify the name of the Deployment.

A value is required for this property.

DeploymentId

Specify the ID of the Deployment.

A value is required for this property.

RequestMode

Optionally specify the mode with which to use the node. Choose from:

  • Create - Creates a new Data Store definition sending through the data on the input. If one already exists, the node will fail.
  • Update - Updates an existing Data Store definition, adding the data on the input. If no definition already exists, the node will fail.
  • Validate - Uses an existing Data Store definition adding the data on the input. If no definition already exists, the node will fail. The definition will not be updated, however the metadata will be validated and data appended.
  • Create or Update - If no Data Store definition exists, one will be created. Otherwise, Update mode is used.
  • Create or Validate - If no Data Store definition exists, one will be created. Otherwise, Validate mode is used.

The default value is Create or Validate.

Url

Specify the URL of the Data360 DQ+ instance to connect to. Example: http://cafe-modeloffice.example.com

Username

Specify the username to connect with.

Password

Specify the password to connect with.

ProductName

Optionally specify the ProductName.

The default value is "Data360 Analyze".

ProductId

Optionally specify the ProductId.

The default value is "Data360Analyze".

AdditionalFieldBehavior

Optionally specify the behavior when there is an existing Data Store on the Data360 DQ+ instance and there is a field on the input to the node which does not exist in the existing record definition. Choose from:

  • Error - The node will fail.
  • Ignore - The additional field will not be sent to Data360 DQ+.
  • Add - The record definition on the Data Store in Data360 DQ+ will be updated with the new field and the data sent through for the new field.

The default value is Error.

MissingFieldBehavior

Optionally specify the behavior when there is an existing Data Store on the Data360 DQ+ instance and there is a field in that existing record definition which does not exist on the input. Choose from:

  • Error - The node will fail.
  • Log - A warning will be logged about the missing field but the node will continue. The record definition will be updated to remove that field.
  • Ignore - As per Log, but no warning will be logged.

The default value is Error.

InvalidCharactersInFieldBehavior

Certain characters, such as spaces, are not allowed as field names in Data360 DQ+. Data360 Analyze will detect invalid field names, before attempting to publish to Data360 DQ+.

Optionally specify the behavior, when invalid characters are detected in field names.

Choose from:

  • Error - The node will fail.
  • Replace - The invalid characters will be replace with . No warning will be logged.
  • Replace and Log - The invalid characters will be replace with an underscore ('_' ). A warning will be logged.

ProxyUrl

Optionally specify the URL of the proxy server to use for the HTTP requests. The URL should contain the protocol (http or https), the host, and the port. All other parts of the URL will be ignored.

The URL should be in a format similar to the following:

HTTP proxy: http://proxy.example.com:1234

HTTPS proxy: https://proxy.example.com:443

The node supports automatic detection of system proxy settings. To override system proxy settings see the IgnoreSystemProxy property.

ProxyUsername

Optionally specify the proxy username, where the proxy requires authentication.

This property should be left blank if ProxyUrl is not set.

ProxyPassword

Optionally specify the proxy password, where the proxy requires authentication.

This property should be left blank if ProxyUrl is not set.

ProxyAuthenticationMethod

Optionally specify the authentication method used to verify the user of the proxy server. Choose from:
  • Basic
  • Digest
  • NTLM

If this property is not set, the system will try to automatically detect the best option. By default, this property is not set.

RetryCondition

Optionally specify under what conditions to retry a request.

Choose from:
  • Never - Never retry a request.
  • Http Status Error - Retry the request if the response status code is in the 4xx client error or 5xx server error categories.
  • Network Error- Retry the request if a network error occurs. A network error is any low level network error that might occur. This could be a socket or connection timeout, TLS error and so on.
  • Any Error - Retry the request if the response contains a http status error or a network error.

The default value is Never.

MaxRetries

Optionally specify the number of times to attempt an unsuccessful retry before error.

The default value is unlimited.

RetryTimeout

Optionally specify the maximum total amount of time to attempt to retry a failed request regardless of the number of attempts.

The default value is unlimited, the node will not timeout.

Inputs and outputs

Inputs: Input.

Outputs: RecordDefinition, Fields.

Creating and automating Publish to DQ+

To create and automate the Publish to DQ+ node process, do the following:

  1. In Data360 DQ+, create an application user credential.

    Note: A user credential does not work.
  2. In Data360 Analyze, populate the Publish to DQ+ node, using the information given above.
  3. Run the Publish to DQ+ node.

    The next steps are carried out in Data360 DQ+:

  4. Move the incoming data store to an appropriate location.

    See Data360 DQ+ User guide > Data stores > Creating an internal data store, for more details.

  5. Create a data view.

    See Data360 DQ+ User guide > Data views > Configuring data views, for more details.

  6. Create a dashboard.

    See Data360 DQ+ User guide > Dashboards > Building a dashboard, for more details.

  7. Create a process model to automate the data view and dashboard.

    See Data360 DQ+ User guide > Process models > Creating process models, for more details.

Next time you run the Data360 Analyze Publish to DQ+ node, it will automatically update your Data360 DQ+ data.