Logging on the Polarity Server

Polarity produces logs for Server processes, and each integration produces their own individual logs. While the Server logs are consistent, the verbosity of the individual integration logs can vary based on the amount of information offered by the resource(s) contacted for data.

Server Process Logs

Each container produces its own logs. These can be viewed using the docker logs <container_name> syntax.  The output is standard ASCII-based text, and may be viewed and parsed on the command line with standard POSIX tools.

Example:

docker logs polarity_platform will display all logs for the polarity_platform container since it was last instantiated.

Log Level

By default, the Polarity Server processes will log individual user queries to integrations. This is reflected in the Server’s .env file with this line:

POLARITY_LOG_INTEGRATION_LOOKUPS=verbose

Other logging levels available include (in order of verbosity, from least to greatest):

off
minimal
info
verbose

Each logging level is additive, data-wise.  For example, info contains all of the data in minimal.

Docker Logging Drivers

There are a number of logging drivers supported by Docker by default. Polarity is shipped with the json-file logging driver set up by default, with log rotation set with 6 log files of 10MB each, as seen in the Server’s docker-compose.yml file.

Depending on Server use, this may not retain enough log history to be useful. However, the settings are adjustable.

NOTE

Since containers are ephemeral, the logs associated with them last only as long as the instance does.

To retain logs longer, a log aggregator (such as Splunk or Elasticsearch) is needed.

See Polarity Source Analytics for additional information on forwarding logs.

Integration Logs

Each integration produces logs.  These can be found in the individual integration’s directory, under ./logs/integration.log.  These are ASCII-based text files, and may be viewed and parsed on the command line with standard POSIX tools.

Example:

The integration.log file for a Splunk integration might be found here:  /app/integrations/splunk_3_6_7_node_18_27043de191_1726234895/logs/integration.log.

Log Level

By default, integrations will log at the info level.  Since each data source produces different responses, the amount and usefulness of this log level may vary from integration to integration.  

The log level for a given integration may be set in the individual integration’s ./config/config.json file.  

Example:

For the Splunk integration mentioned above, this file would need to be edited:  /app/integrations/splunk_3_6_7_node_18_27043de191_1726234895/config/config.json

Find and edit the level displayed in a block that appears like this (truncated for clarity):

...
  "logging": {
    "level": "info"
  },
...

Other logging levels available include (in order of verbosity, from least to greatest):

fatal
error
warn
info
debug
trace

Each logging level is additive, data-wise.  For example, info contains all of the data in warn.