Upgrade from Polarity Server v4

This page will walk an administrator through upgrading from Polarity Server version 4 to version 5.

Prepare Your System

Confirm Operating System 

Check your operating system to ensure you follow the correct process: 

cat /etc/os-release

Ensure Adequate Storage

You must ensure your system has adequate storage to accommodate the upgraded Server components.  To find out, you can run the following command:

df -h

Look for the volumes containing the /app and the /var/lib directories.  You will need at least 2 GB of space in /app and at least 10 GB of space in /var/lib to proceed.

Database Backup 

Create database backup:

su - postgres -c '/app/polarity-server/data/backups/db-backup.sh'

If you see cannot access errors related to the logs, or .git directory of integrations these errors can be safely ignored.  The backup script does not copy any logs you may have from your integrations, though the integrations themselves are backed up.

Convert TLS Key & Retain CA/Intermediate Certificate Chain

Navigate to the key location: 

cd /etc/pki/tls/private

Make a backup copy of your existing key:

cp server.key server.key.orig

Convert the key:

openssl rsa -in server.key.orig -outform pem > server.key

Ensure you have a copy of the CA and intermediate certificates required to validate the TLS certificate used on the v5 Polarity Server.

Edit DB Maximum Connection

The POLARITY_DB_CONNECTION_POOL_SIZE needs to be added or edited in the Polarity /app/polarity-server/.env file. 

Open the /app/polarity-server/.env for editing, and check to see if there is the line "POLARITY_DB_CONNECTION_POOL_SIZE".  If the line exists, edit it to be the following; if it doesn't exist please add the line to the .env file: 

POLARITY_DB_CONNECTION_POOL_SIZE=50

Save and close the file.

Install and Check Dependencies  

CentOS 7 / Amazon Linux 2 

JQ / Docker

yum install jq docker -y

yum --disablerepo=* -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm

Docker Compose 

wget https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -O /usr/local/bin/docker-compose
chmod -v +x /usr/local/bin/docker-compose

With CentOS 7 / AL2, you will need to ensure that docker-compose is added to the PATH to ensure the commands will work in the future.  To do this, open the ~/.bashrc file for editing and add the following line to the bottom of the file:

export PATH="$PATH:/usr/local/bin"

Enforce the change immediately:

source ~/.bashrc

CentOS Stream 8 / Rocky Linux 8 / RHEL 8

JQ

sudo dnf install jq -y

Docker 

dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
dnf install docker-ce -y --allowerasing

Docker Compose 

wget https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -O /usr/local/bin/docker-compose
chmod -v +x /usr/local/bin/docker-compose

Check the Docker version commands to ensure Docker and Docker Compose is accessible:

docker --version
docker-compose --version

Download the Upgrade Script 

Navigate to the /root directory or a directory such as /tmp and make it executable:

cd /root
wget {{URL Provided by Polarity Rep}}
chmod u+x upgrade_v4_to_v5-al2-offline-selfextract.sh

Upgrade Steps

Execute the Upgrade Script 

./upgrade_v4_to_v5-al2-offline-selfextract.sh

Script Prompts 

First Prompt: 

  "Press any Key to Continue"

The initial prompt to let the admin know the script is running and give a final chance to abort before any changes are made to the existing Server.  Press any key on the keyboard to continue. 

Second prompt: 

  "Which directory do you wish to install Polarity V5 into? [/app]:" 

The default is /app, which is the recommended location.  To accept the default, just hit [Enter].

Third Prompt: 

  "Do you want to create a backup of the existing Polarity Platform v4 install directory (y/n) [y]:" 

Here we recommend not creating a backup of the v4 install directory as there is a database backup already confirmed in an early step above. 

We recommend "n" or "no" and the [Enter] key to confirm. 

Fourth Prompt: 

  "Do you want to create a backup of the existing .env file (y/n) [yes]:" 

This prompt is asking if you want to ensure a backup of the .env file is taken.

We recommend the default of "y", "yes", or [Enter].

Fifth Prompt: 

  "Do you want to using the values from the existing .env file (y/n) [yes]:" 

This prompt will utilize the values that were setup in the .env file originally. 

We recommend keeping these values and selecting the default of "y", "yes", or [Enter].

Sixth Prompt:  

  "Do you want to be prompted for each environment variable (y/n) [yes]:" 

This prompt is to allow the admin to review and or change each of the values in the .env file. 

We recommend "n" or "no" and the [Enter] key to confirm.

Integration Prompts: 

The script will upgrade each of the integrations that are currently installed on the Polarity v4 Server to ensure the integrations will work correctly.  The script will check and try and match the names of the integrations to supported integrations.  If there are any edited integrations or custom integrations present, the script will prompt the admin to ask if they want to map the integration to a supported integration.

We recommend mapping integrations if they have been renamed to something else (for example "Splunk Firewall" if it was a renamed "Splunk" integration).  We recommend mapping it to our Splunk integration then changing the name after the upgrade is complete to ensure the values are saved in the database. 

There are two options to map an installed custom integration to a supported one: 

1. Name of the Integration
   1. Enter the name of the integration as it appears on the integrations GitHub page, which can be found here: https://github.com/polarityio
2. Polarity Unique Identifier
   1. Enter the integration's Unique ID, which can be found in the integration's config/config.json file on GitHub.
      1. Example location to find the Splunk UID:  https://github.com/polarityio/splunk/config/config.json 
   2. The unique ID will look something like this:  "13aab2c0-a435-11ee-b809-03732f21d597"

Seventh Prompt: 

  "Do you want to remove Polarity v4 packages (y/n) [no]:" 

This prompt asks if you want to remove the Polarity Version 4 packages from the server. 

We recommend keeping the packages on the server until the v5 server is up and running without issues.  The packages can be removed at a later date. 

We recommend "n" or "no" and the [Enter] key to confirm.

Final Prompt: 

  "Do you want to remove Polarity v4 directories (y/n) [no]:" 

This is the final prompt which asks if admins want to remove the Version 4 directories from the server. 

We recommend to keep the directories on the server until the V5 server is up and running without issues. The directories can be removed at a later date. 

We recommend "n" or "no" and the [Enter] key to confirm.

Script is complete when you see: 

  All done. Good luck! 

Post-Upgrade Steps 

When finishing the v4 to v5 upgrade process, Docker will be running 2 containers (polarity_platform and polarity_web).  The PostgreSQL, Redis Integration Cache, and Redis Metrics Cache services will continue running on the host, not in containers.

In the event your server reboots, it is important that the Redis services and PostgreSQL service start up after the Docker service starts.  This is to ensure that the Docker network (which is required for PostgreSQL and the Redis caches [polarity-integration-cache and polarity-metrics-cache]) is available for use.  

To do this, you will override the systemd unit file for each service using the systemctl edit command.  

Starting with the PostgreSQL service run the following command:

systemctl edit postgresql-15

If you are running PostgreSQL 13, replace "postgresql-15" with "postgresql-13".

The above command will open the default text editor, where you should enter the following content:

[Unit]
After=docker.service

The default text editor is most likely vi in which case you will need to press the "i" key to enter "insert" mode before typing or pasting the above content.

If the editor is vi you can save the file by pressing [ESC] and then ":wq".

Next you will want to modify the Integration Cache service:

systemctl edit polarity-integration-cache

Add the same text as you did for the PostgreSQL service. 

Finally, repeat the process for the Metrics Cache service:

systemctl edit polarity-metrics-cache

After making the necessary unit file changes, you will need to restart the services for the changes to take effect:

systemctl daemon-reload 
systemctl restart postgresql-15
systemctl restart polarity-integration-cache 
systemctl restart polarity-metrics-cache

Finally, we recommend ensuring that the services are all enabled to start on boot with the following command:

systemctl enable postgresql-15
systemctl enable polarity-integration-cache 
systemctl enable polarity-metrics-cache

Again, if you are running PostgreSQL 13, replace "postgresql-15" with "postgresql-13" in the above commands.