Setup requirements - Data360_DQ+ - 12.3

Operating system requirements

• RedHat Enterprise Linux 8.9
• ext3, ext4 or XFS file system

Docker support

The bundled Docker packages have been updated to version 25.0.5.

Vertica prerequisites

Please consider the following prerequisites which are required to run Vertica, the component that is responsible for running analytics for Data360 DQ+.

Support for Vertica has been updated to version 25.3.0.

Vertica file system requirements

Vertica requires that you have one of the following file systems:

• ext3
• ext4
• XFS

Vertica package dependencies

Vertica requires the following packages be installed on the system:

• dialog
• mcelog
• sysstat
• gdb
• perl
• gcc-c++

Setting Vertica swap space

"Swap space" is a way to obtain more memory resources when RAM is full. It allows you to reserve space on a hard disk that can be used like RAM. To function properly, Data360 DQ+ needs a minimum of 2 GB swap space.

1. Check if you have swap space already set up by running the following command:

   free -m

2. If you do not have swap space set up, you can use the following commands to create new swap space:

          sudo mkdir -p /var/swap
   	sudo touch /var/swap/swap.1
   	sudo /bin/dd if=/dev/zero of=/var/swap/swap.1 bs=1M count=2050
   	sudo /sbin/mkswap /var/swap/swap.1
   	sudo chmod 600 /var/swap/swap.1
   	sudo /sbin/swapon /var/swap/swap.1
   	sudo /bin/sh -c "echo '/var/swap/swap.1 swap   swap defaults 0 0' 		>> /etc/fstab"
   

Setting read-ahead for Vertica

1. Run the following commands, inserting the disk path where indicated:

   sudo /sbin/blockdev --setra 2050 {disk path here}

   sudo /bin/sh -c "echo '/sbin/blockdev --setra 2050 {disk path here}' >> / etc/rc.local"

Setting SELinux to permissive mode

For Vertica to function properly, SELinux must be set to Permissive mode, as follows:

1. Edit the /etc/selinux/config file to set SELINUX=permissive. This ensures that SELinux remains set to permissive after a reboot.

If you want to disable SELinux immediately, within the current session, type the following command:

sudo setenforce permissive

Enabling or disabling transparent hugepages for Vertica

1. Determine if transparent hugepages is enabled by running this command:

   cat /sys/kernel/mm/transparent_hugepage/enabled [always] madvise never

   The setting returned in brackets is your current setting. For systems that do not support /etc/rc.local, use the equivalent startup script that is run after the destination runlevel has been reached. For example SuSE uses /etc/init.d/after.local.

2. You can enable transparent hugepages by editing /etc/rc.local and adding this script:

   if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo always > /sys/kernel/mm/transparent_hugepage/enabled fi

3. Reboot your system for the setting to take effect.
4. To disable transparent hugepages, edit your boot loader (for example /etc/grub.conf). Typically, you add the following to the end of the kernel line. However, consult the documentation for your system before editing your bootloader configuration:

   transparent_hugepage=never

   Alternatively, edit /etc/rc.local (on systems that support rc.local) and add this script:

   if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi

   For systems that do not support /etc/rc.local, use the equivalent startup script that is run after the destination runlevel has been reached. For example SuSE uses /etc/init.d/after.local.

5. Reboot your system for the setting to take effect.

For more information, see the Vertica documentation here.

Hadoop prerequisites

Data360 DQ+ can connect to an existing Cloudera Data Platform (CDP) installation with Spark 3.5.x support configured. The Hadoop cluster can fulfill the role of Analysis processing. It is referred to elsewhere in this guide as, for example, "Compute primary and Compute secondary", and in the install.properties file as "Compute Master" and "Compute Slave" (or the "Compute Cluster" collectively).

To connect to your Hadoop cluster, there are a number of Hadoop related properties in the install.properites file for which you will need to provide values, see Hadoop connectivity properties (Compute properties).

The following steps must be completed in order to connect to an existing CDP cluster:

1. Configure the keytab directories. For each Application Server node in your setup, the installation process expects a directory with a name matching the IP address of the Application Server node, and, the Sagacity system user keytab needs to be in each one of these directories. For example, the directory structure may look like this:

   <keytabsDir>/[app sever_ ip_address1]/sagacity.keytab
               /[app sever_ ip_address2]/sagacity.keytab

2. Download the Hadoop Yarn configuration file must and save the file at an accessible location. This will be checked by the verifyEnvironment script during installation.

Other prerequisites

Before installing Data360 DQ+ you must ensure that you have completed the following steps:

1. Run the following command on every machine that Data360 DQ+ will be installed on:

   sudo systemctl start chronyd

   Installation requires the chronyd service to be running on every node.

2. Run the following command on each machine:

   sudo yum install -y libtool-ltdl

   Installation requires that libltdl is installed on every node.

3. Set up a shared file system. This file system will be shared across all machines in your Data360 DQ+ cluster and it will be used to hold data, logs, and backup content. During installation, you will need to point to the path of this shared file system using the sagacitySharedMountPoint property. You can use any type of shared file system.