Maintenance tasks - Data360_DQ+ - 11.X

Data360 DQ+ Enterprise Installation

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 DQ+
Version
11.X
Language
English
Product name
Data360 DQ+
Title
Data360 DQ+ Enterprise Installation
Copyright
2024
First publish date
2016
ft:lastEdition
2024-06-06
ft:lastPublication
2024-06-06T12:37:34.761477

Below is a list of commands that can be used to stop and start the entire system or individual system components.

Stop or start the entire application

To stop and start the entire system, use these commands:

stop

start

In most cases, a full stop and start of the entire system is recommended, rather than stopping and starting components individually.

Stop or start individual components

To stop and start the individual system components, use these commands:

Application

stopApplication

startApplication

ApplicationDB

stopApplicationDB

startApplicationDB

ComputeDB

stopComputeDB

startComputeDB

Load Balancer

stopLoadBalancer

startLoadBalancer

Switch from your standby server to your primary server

If you need to switch from your standby server back to your primary server, perform these steps:

  1. Run the backupStandbyApplicationDb script from the application’s bin folder on the maintenance machine. This will initiate a backup process for the ApplicationDB standby node.
  2. Run the stopApplicationDb script from the application's bin folder on the maintenance machine. This will stop both the ApplicationDB primary and standby nodes.
  3. Run the startApplicationDb script from the application’s bin folder on the maintenance machine. This will start both the ApplicationDB primary and standby nodes.
  4. Restore the standby ApplicationDB's backup, that was created in Step 1, into the primary ApplicationDB, using the standard restore procedure.

Gather logs

To generate logs to send to support, use this command:

./createSupportPackage <filepath>

The generated log files can then be found in the specified file path.

Remove old backups

To remove old backups, use the rm -rf command.

Schema changes when upgrading

When upgrading, you will need to run a number of SQL statements on ApplicationDb (PostgreSQL). The statements that you need to run depend on your currently installed version.

Upgrading from 7.1

If you are upgrading to the latest release from 7.1, install trigram extension. Trigram database index support case insensitive search feature and trigram extension needs to be installed prior to creating trigram indexes for Postgresql database.

  1. Login to DQ+ Postgresql database as user with superuser or CREATE privilege on current database.
  2. Run the query to install extension:

    CREATE EXTENSION IF NOT EXISTS pg_trgm;

  3. Run the query to verify that the extension is installed, it should return results without error:

    SELECT show_trgm('hello');

For additional help, please contact Support at support.precisely.com.

Upgrading from 5.2 or 6.2

If you are upgrading to the latest release from 5.2 or 6.2, please run the following SQL statement, which has to be run in the igxsagacity database, by a sagacity user:

  • ALTER TABLE caf_verdefcontentdep ALTER COLUMN fromfieldpath TYPE text, ALTER COLUMN tofieldpath TYPE text;

These are the commands used to connect postgres with a sagacity user:

  • docker exec -it dqplusApplicationDb bash
  • psql -U sagacity -d igxsagacity
Note: After applying schema changes, restart the application server.

Change ComputeDB’s MAXMEMORYSIZE property after installation

To change ComputeDB’s MAXMEMORYSIZE after installation, use this command, where x is a number between 0-100:

configureComputeDb MAXMEMORYSIZE <x%>

ComputeDB compliance

By default, the verifyComputeDbCompliance task is scheduled in the crontab file, to verify that VERTICA_MAX_DATA_UTILIZATION_PERCENTAGE does not surpass the maximum amount specified in the install.properties file.

If verifyComputeDbCompliance finds that data utilization has surpassed the specified threshold, an email will be sent to the email address assigned to the sagacityAdministratorEmail property in the install.properties file.

verifyComputeDbCompliance can also be manually run at any time.

Change properties after initial installation

If properties need to be changed after installation, please contact Support at support.precisely.com.

Patch the product

  1. Download the patch into the /tmp folder.
  2. Give execute permission to the downloaded patch, for example where X.X is the product version:

    chmod 777 dqplus_IS-X.X-202004151330-2-fixpack.sh

  3. Extract the downloaded patch by running its name as a command. For example:

    ./dqplus_IS-X.X-202004151330-2-fixpack.sh

    Extracting a patch for the first time will create a directory at opt/infogix/dqplus/maintenance

  4. Run the apply-patch command from within opt/infogix/dqplus/maintenance/<patchName>/bin

To verify that the fix pack has been applied, navigate to the Help menu in the product and select About. The product's Updates information should be updated.

Revert a patch

To revert a patch that has been applied, run the revert-patch.sh command from within the patch’s bin that was created when the patch was extracted. For example:

cd opt/infogix/dqplus/maintenance/dqplus_IS-X.X-202004151330-2-fixpack/bin

revert-patch.sh

Reinstall and preserve data

This section describes how to reinstall the application while preserving user data from a previous installation.

Note: This is not a recommended upgrade process. These steps should only be used when recommended by Support. For information on upgrading, see Performing an upgrade.
Note: Before reinstalling, please note that you should not delete the following files:
  • sagacityDataFolder=${sagacityExclusiveMountPoint}/application
  • sagacityFolderCompute=${sagacityExclusiveMountPoint}/runtime/hadoop
  • APP_KEYSTORE_FILE=${sagacityExclusiveMountPoint}/security/sagacity.keystore
  • DATA_HOME=${sagacitySharedMountPoint}/data• externalDataFolder=/opt/share/testdata
  • externalDataFolderMountDestination=/opt/share/testdata

Where <version> is used within file paths, this refers to a numeric version of the product, for example, dqplus-10.2

  1. Backup the install.properties file from the following location:

    /opt/infogix/dqplus-<version>/properties

  2. Backup the id_rsa file from the following location:

    /opt/infogix/dqplus-<version>/ssh-config/

  3. Run backup. It is recommended that you backup ApplicationDB, ComputeDb, and Configurations.
  4. Stop
  5. Uninstall
  6. Remove dqplus-<version> at /opt/infogix on all nodes
  7. Unpack the new build
  8. Update the install.properties file or copy the install.properties file from the backup created in Step 1 to the following location:

    /opt/infogix/dqplus-<version>/properties

  9. Copy the id_rsa file from the backup created in Step 2 to the following location:

    /opt/infogix/dqplus-<version>/ssh-config/

  10. Copy the known_hosts file to the following location:

    /opt/infogix/dqplus-<version>/ssh-config/

    See Performing installation for more information on the known_hosts file.

  11. verifyEnvironment
  12. encryptProperties
  13. install
  14. initialize
  15. stop (optional)
  16. start (optional)

Optional steps

  1. Remove old ApplicationDB backups. ApplicationDB backups will be located under an applicationdb folder, within the folder pointed to by the sagacityBackupFolder in the install.properties file.
    Note: Do not delete the computedDb or configrations folders at this location.
  2. Remove old logs. Refer to the sagacityLogsFolder property in the install.properties file to find the location for old logs.

Add JDBC drivers

To enable the functionality of some third-party products, you may need to add the products' JDBC driver JARs to your installation:

  1. Add the required JAR files to the following folders, where <version> refers to a numeric version of the product, for example dqplus-10.2:
    • /opt/infogix/dqplus-<version>/overrides/spark/jars
    • /opt/infogix/dqplus-<version>/overrides/lib
  2. Run the deploy command to push the files to the Application Server Docker containers.

Deploy node plugins

To deploy custom node plugins, add any custom node JARs and dependent third-party JARs to your installation:

  1. Add the required JAR files to the following folders, where <version> refers to a numeric version of the product, for example dqplus-10.2:
    • /opt/infogix/dqplus-<version>/overrides/spark/jars
    • /opt/infogix/dqplus-<version>/overrides/lib
  2. Run the deploy command to push the files to the Application Server Docker containers.

Override the default Hive jar files

Complete these steps to override the default Hive jar files and use your own custom Hive jar files:

  1. Create a folder in the overrides folder to store the Hive jar files:/opt/infogix/dqplus-<version>/overrides/hive

  2. Add the Hive jar file to the folder /opt/infogix/dqplus-<version>/overrides/hive

  3. Run the deploy command to push these files to the Application Server Docker containers.

Complete these steps to remove the custom Hive jar files and use the default Hive jar files:

  1. Remove the Hive jar files from /opt/infogix/dqplus-<version>/overrides/hive
  2. Run the deploy command to push the update to the Application Server Docker containers.

Add custom runtime files

To enable the functionality of some third party products, you might need to add custom runtime files, such as additional key tab files or trust store files, to this folder:

/opt/infogix/dqplus-<version>/runtime

Complete these steps to add custom runtime files:

  1. Add "cm-auto-global_truststore.jks" to /opt/infogix/dqplus-<version>/runtime
  2. Update DEPLOY_JAVA_OPTS by adding -Digx.runtime.dir=/opt/cafe/runtime.
    DEPLOY_JAVA_OPTS=-Dfile.encoding=UTF-8 -Duser.timezone=America/Chicago -Digx.runtime.dir=/opt/cafe/runtime
  3. Update the ssl-client.xml file in hdfs-client.zip and yarn-client.zip, by changing the value of ssl.client.truststore.location to ${igx.runtime.dir}/cm-auto-global_truststore.jks
    <property>
    	<name>ssl.client.truststore.location</name>
    	<value>${igx.runtime.dir}/cm-auto-global_truststore.jks</value>
    </property> 
  4. Run the deploy command to push these files to the Application Server Docker containers.

Re-encrypt data keys

  1. Run the stopWebApplication command to stop the application servers in all nodes.
  2. Run the backupApplicationDb command to back up the application database.
  3. Make a copy of the key store file that is being configured in the install.properties file.

    To show the keys currently in the key store, use this command:

    keytool -list -keystore ./sagacity.keystore

  4. Add a new key to the key store. Make sure you use the same passwords as before. For example:

    keytool -genkeypair -keystore ./sagacity.keystore -alias sagacityKey2 -keyalg RSA -validity 36500 –v -dname CN=Sagacity,OU=Dev,O=Infogix,L=Naperville,ST=IL,C=US

  5. Confirm that the key has been added by listing the keys in the key store:

    keytool -list -keystore ./sagacity.keystore

  6. Update the install.properties file to change the master key alias to point to the new key:

    APP_MASTER_KEY_ALIAS=sagacityKey2

  7. Run the reencrypt command to re-encrypt the data keys.
  8. Run the deploy command to deploy the application.
  9. Verify that access to encrypted data is available before making the application available to users.