Upgrade the infrastructure - Data360_DQ+

Prerequisite: The same prerequisites that are required to deploy the infrastructure are also needed when upgrading, see Setup requirements.

Download the latest installer zip file.
Extract the files in the location where you want to run the upgrade, and locate the infra/aws/config folder.
Copy the following files from your existing installation directory into the new infra/aws/config folder:
- terraform.tfstate
- dqplus.tfvars
- dqplus.properties
- pw.properties
If you want to change any existing settings as part of the upgrade, such as instance sizes, or the max count of an instances for a component, edit the appropriate line in the dqplus.tfvars file.
If you are upgrading the infrastructure from any previous version directly to the latest version, execute these Terraform commands to change the state file for the Redshift Cluster, S3 bucket module:
terraform init

terraform state mv module.redshift module.redshift[0]

terraform state mv module.buckets module.buckets[0]

terraform state mv module.eb module.eb[0]
From the infra/aws/config directory run the following command:
python3 ./setup.py

The script checks for existing files and uses any existing values. You will only be asked to provide a value if it does not already exist.
After the infrastructure upgrade is complete, you can install the application using the dqplus.properties and pw.properties files, see Installing the application (Elastic Beanstalk or Kuberntes Deployment).

Upgrading From Data360 DQ+ 6.x

Upgrading the RDS instance

The Aurora RDS instance must be upgraded through the AWS Console, rather than through Terraform. Do the following:

When initially installing the infra, Terraform created two RDS Aurora nodes. For example, {appname}-dqplusaurora-0 and {appname}-dqplusaurora-1. As there is a limitation in AWS with regard to what instance types can be used for an upgrade, the instance types need to be temporarily changed:
1. Select the first instance ({appname}-dqplusaurora-0), then click Modify.
2. Look for "DB Instance class", and update it from db.r5.xlarge to db.r6g.large.
3. Click Continue followed by the Apply Immediately option.
4. Repeat steps 1.a to 1.c for instances of the other database ({appname}-dqplusaurora-1) to change the class from db.r5.xlarge to db.r6g.large.
When you have successfully updated the instance classes, navigate to your RDS and select the {appname}-dqplusaurora-cluster database cluster, in the AWS Console, then click Modify.
Select the DB engine version 13.13 to use to upgrade.
Select Continue.
Click to modify and select Apply Immediately.
Modify the cluster and wait.
Revert the instance classes back to db.r5.xlarge, using steps 1.a to 1.d, but changing the instance class this time, from db.r6g.large to db.r5.xlarge.

You now need to ensure that the Terraform code matches the new state of the Aurora cluster, as follows:

Unzip the installation package for the latest version and copy your existing Terraform State and Property files into the infra/aws/config folder.

See Upgrading the infrastructure above, for more details.
Run terraform init.
Run terraform show and you should see the following entry:
output: engine_version = "12.4"
Run terraform plan --target=module.aurora --var-file=dqplus.tfvars --out=rdsplan and you should see the following:
output: ~ engine_version = “13.13” -> “12.4” # forces replacement`
Run terraform refresh --var-file=dqplus.tfvars
Run terraform show.
Run terraform plan --target=module.aurora --var-file=dqplus.tfvars --out=rdsplan and you should see the following:
No changes. Infrastructure is up-to-date.

You can now continue with the upgrade as detailed in Upgrading the infrastructure above.