Polarity Server v5 (Platform)

v5.4.1

WARNING!

Prior to updating to any version of Polarity Server 5.4 please ensure that all of your integrations are running the latest version.  

Integrations that have not been updated may not be compatible with Polarity Server 5.4+.  

When updating integrations please keep in mind that any custom types that were manually added to the integration will be lost.  Prior to updating, be sure to have a copy of any custom type regular expressions so they can be re-added after the update.

Polarity 5.4+ allows adding these custom types directly through the Data Types page on the integration settings page.  Once on Polarity 5.4+, future integration updates will not reset your custom types.

About Latest Version

Note

If you have not yet updated to v5.4.x of Polarity Server, please review the new features and updates available in v5.4.x of the Server as well as updates available in v5.4.x of Web.

Polarity Server 5.4.1 is a minor patch release that addresses several issues in v5.4.0.  This is a recommended upgrade for all customers.

• Fix - Improve upgrade migration path from server versions prior to v5.4.0 to preserve whether data types are enabled or disabled when transitioning to new custom type implementation.

• Fix - Ensure proper typeflag on custom entity object values that are passed to Integrations.

• Fix - Fix loading of certain values for the new  dataTypes  setting within an Integration's config.json

Troubleshooting Upgrade to Latest 5.4 Version

Permission Issues

After upgrading to V5.4 and the server will not start you might see an error like the following :

:cacertfile either does not exist, or the application does not have permission to access it

This is due to a permission not getting set during the upgrade process.  The fix is fairly straight forward as all you will need to do is run the following script to force set the permissions:

cd /app
bin/polarity docker-set-permissions

Improperly Referenced tls-ca-bundle.pem file

If you notice the following error in your server logs then you may need to update your docker-compose.ymlfile to remove the POLARITY_TLS_CA_PATH environment variable:

Kernel pid terminated (application_controller) 
({application_start_failure,polarity,{{shutdown,{failed_to_start_child,'Elixir.PolarityWeb.Endpoint',
{#{'__exception__' => true,'__struct__' => 'Elixir.ArgumentError',message => 
<<"could not start Cowboy2 adapter, the file /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem required by SSL's :cacertfile either does not exist, 
or the application does not have permission to access it">>},

If you see the above error, open the /app/docker-compose.yml file and look for the following line within the app service:

environment:
  - POLARITY_TLS_CA_PATH=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

If you see this line, delete the environment and POLARITY_TLS_CA_PATH lines, then restart your containers.

cd /app && ./down.sh && ./up.sh

v5.4.0

The latest 5.4.0 release adds in some great new features and functionality for users!  We have added the ability to consolidate Errors into a single page for admins and users, as well as added the ability for admins to add in custom entity types from the UI versus having to update them on the backend. Enabling a much more seamless administrative experience for integrations.  As always for more information on Errors and Custom Types, please see the Polarity Web release docs.

Single Pane for Error Management

The server added the ability for users to be able to see their errors as well as for there to be a single view for errors. Allowing admins to be able to quickly take action on errors and clear them out if needed. It also allows for users for the first time to see their errors.

Adding Custom Types through UI

Adding custom types to integrations is one of the biggest abilities our integration framework enables. It allows for any type of information to be looked up in different tools.  However with the enablement of our integration store adding a custom type has become a little bit harder since the work takes place on the backend server, causing friction when Polarity publishes a new version of an integration which will lead to the over writing of an integration.

However we have now added the ability for admins to quickly add a custom type to integrations that support it right through the UI. Allowing for quick setup and time to value for different tools.

Server User Permissions Update

As a part of the new 5.4.0 release we have added in a new user that will own all of the polarity files on the server.  This change was added in to address the issue of customers not being able to run docker as root.  As a part of the upgrade the permissions will attempt to get set. If you run into issues during the upgrade please see the troubleshooting section below or reach out to us at support@polarity.io.

Issues Addressed

• Fixed issue when looking up more than 2 entities with spaces on web search

• Addresses issue with web search parser when looking up defanged entities

• Added ability to rate limit emails

• Increased integration timeouts to 60 seconds versus 30 seconds

• Added environment variable for admins to disable integration store

• Added ability to change and edit email templates

• Fixed issue with lookup pagination

Troubleshooting Upgrade to Latest 5.4 Version

Permission Issues

After upgrading to V5.4 and the server will not start you might see an error like the following:

:cacertfile either does not exist, or the application does not have permission to access it

This is due to a permission not getting set during the upgrade process. The fix is fairly straight forward as all you will need to do is run the following script to force set the permissions:

cd /app
bin/polarity docker-set-permissions

Improperly Referenced tls-ca-bundle.pem file

If you notice the following error in your server logs, then you may need to update your docker-compose.yml file to remove the POLARITY_TLS_CA_PATH environment variable:

Kernel pid terminated (application_controller) 
({application_start_failure,polarity,{{shutdown,{failed_to_start_child,'Elixir.PolarityWeb.Endpoint',
{#{'__exception__' => true,'__struct__' => 'Elixir.ArgumentError',message => 
<<"could not start Cowboy2 adapter, the file /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem required by SSL's :cacertfile either does not exist, 
or the application does not have permission to access it">>},

If you see the above error, open the /app/docker-compose.yml file and look for the following line within the app service:

environment:
  - POLARITY_TLS_CA_PATH=/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

If you see this line, you should delete the environment and POLARITY_TLS_CA_PATH line and then restart your containers.

cd /app && ./down.sh && ./up.sh

WARNING!

Prior to updating to any version of Polarity Server 5.4 please ensure that all of your integrations are running the latest version.

Integrations that have not been updated may not be compatible with Polarity Server 5.4+.  

When updating integrations please keep in mind that any custom types that were manually added to the integration will be lost.  Prior to updating, be sure to have a copy of any custom type regular expressions so they can be re-added after the update.

Polarity 5.4+ allows adding these custom types directly through the Data Types page on the integration settings page.  Once on Polarity 5.4+, future integration updates will not reset your custom types.

About Latest Version

The latest 5.4.0 release adds in some great new features and functionality for users!  We have added the ability to consolidate Errors into a single page for admins and users, as well as added the ability for admins to add in custom entity types from the UI versus having to update them on the backend.  Enabling a much more seamless administrative experince for integrations.  As always for more information on Errors and Custom Types, please see the Polarity Web release docs.

v5.3.4

The latest release of the Polarity Server is a server-side patch that addresses some identified vulnerabilities, adds in support for TLS connections to Redis and PostgreSQL, fixed an issue with integration functions being called out of order, added the ability to configure SMTP without having to set a username.

Issues Addressed

• Addressed issue with integration functions sometimes being called before the startup function causing integrations to stop.

• Addressed vulnerabilities identified

• Added ability for the server to connect to Redis and PostgreSQLinstances over TLS connections

• Added ability to use SMTP without needing to specify a username

v5.3.1

The 5.3.1 release of the Polarity Server is a server-side patch that addresses the issue with some of the desktop client lookups not working for users leading to lookups returning no information back to users.

Issues Addressed

• Addressed issue with server caching certain entities, not allowing desktop client lookups to go through

v5.3.0

New Functionality

• Team Subscriptions

  • Admins can now quickly subscribes all users that have access to an integration right form the UI, as long as that integration is configured. Enabling analysts to quickly get the full use out of Polarity without needing to think about subscribing to data.

• Global Proxy

  • Instead of having to add in a global proxy to the server in the environment file, admins can now easily add the global proxy right to the UI, making configuration that much easier.

Issues Addressed

• Addressed issue with newly installed integrations not showing up

• Addressed issue where full text was getting sent to some integrations instead of just entities that match on regexes.

• Added functionality to ensure user emails and user fullnames are apart of the integration options payload for the dolookup, onMessage and onDetails functions for integration developers.

v5.2.2

Our latest V5.2.2 version of the Polarity server adds the ability for user emails to be added to and adds additional logging for LDAP.

Improvements:

• Update:  Added the ability to pass through user emails to integration options.

• Update: Added additional logging for LDAP troubleshooting.

v5.2.1

V5.2.1 of the Polarity Server patches up a few issues with the server.

Improvements:

• Update: Moved caddy configurations into its own file for easier configuration with global proxies.

• Patch: Resolved issue with caching layer preventing options from being saved.

• Patch: Resolved issue with caching and searching integrations that have been un subscribed.

• Patch: Resolved issue with incorrect hostname getting set in SMTP emails

• Patch: Resolved issue with incorrect DL links for client

• Patch: Resolved inconsistencies with global versus integration proxies

v5.1.1

We are super excited to be rolling out our newest Polarity Server Version, version 5!  This is a massive update for us here at Polarity and includes some great new functionality.

Complete re-write

• This update is a complete re-write of our original NodeJS server to Elixir. Which allows us to gain massive improvements as an engineering team and allowed us to look at our deployment architecture and support deployments with Docker natively.

• This update also enables us as a team to deliver features and functionality at a higher rate!

New Features and Functionality

• Integration Store

  • We now have a new integration store! Which pulls in all of the new integrations and updates so admins no longer will need to go to the server to run updates or install new integrations!

• Integration Proxy Support

  • Polarity admins can now easily set and manage integration specific proxies right from the UI instead of making changes on the server!

• Deleting Integrations

  • Polarity admins can now easily completely delete integrations right from the UI instead of just removing the directories on the server like before.

• Updating Integration name and descriptions

  • Polarity admins can now also easily update the name and descriptions of integrations without having to update the configs on the server.

• Docker Deployments

  • Our architecture for v5 is now very different from that of v4 as we now deploy Docker containers for our server and web components. Upgrading to Polarity V5 will install polarity_platform and polarity_web Docker containers.