Polarity Server Logs

Polarity produces logs for Server processes, and each integration produces their own individual logs. While the Server logs are consistent, the verbosity of the individual integration logs can vary based on the amount of information offered by the resource(s) contacted for data.

Server Process Logs

Each container produces its own logs. These can be viewed using the docker logs <container_name> syntax. The output is standard ASCII-based text, and may be viewed and parsed on the command line with standard POSIX tools.

Example:

docker logs polarity_platform will display all logs for the polarity_platform container since it was last instantiated.

Log Level

By default, the Polarity Server processes will log individual user queries to integrations. This is reflected in the Server’s .env file with this line:

POLARITY_LOG_INTEGRATION_LOOKUPS=verbose

Other logging levels available include (in order of verbosity, from least to greatest):

off
minimal
info
verbose

Each logging level is additive, data-wise. For example, info contains all of the data in minimal.

What is Logged

Platform Container

Sample logs for the polarity_platform container are shown below:

Creating /tmp/tmp.5LU5faizLx/ca-certificates.crt
Adding /app/certs/ca/LocalCA-pub.crt to CA store
Running /app/bin/entrypoint-start-server.sh
Testing if Postgres is accepting connections. polarity_platform_local_postgres 5432 polarity
Database postgres exists, running migrations...
{"level":"info","message":"Migrations already up","metadata":{},"timestamp":"2025-04-15T21:10:06.105Z"}
Migrations finished.
{"level":"info","message":"Running PolarityWeb.Endpoint with cowboy 2.12.0 at 0.0.0.0:4001 (https)","metadata":{},"timestamp":"2025-04-15T21:10:09.068Z"}
{"level":"info","message":"Access PolarityWeb.Endpoint at https://server.polarity.io:4000","metadata":{},"timestamp":"2025-04-15T21:10:09.068Z"}
{"req":{"headers":{"accept":"application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml","accept-encoding":"gzip, deflate","accept-language":"en-US,*","authorization":"...","content-type":"application/vnd.api+json","cookie":"...","user-agent":"PolarityX/5.2.2","x-forwarded-for":"192.168.0.105","x-forwarded-host":"server.polarity.io:443","x-forwarded-proto":"https"},"method":"GET","url":"/api/users/refresh"},"response_time":"16ms","status":200,"timestamp":"2025-04-15T21:10:09Z"}
{"req":{"headers":{"accept":"application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml","accept-encoding":"gzip, deflate","accept-language":"en-US,*","authorization":"...","content-type":"application/vnd.api+json","cookie":"...","user-agent":"PolarityX/5.2.2","x-forwarded-for":"192.168.0.221","x-forwarded-host":"server.polarity.io:443","x-forwarded-proto":"https"},"method":"GET","url":"/api/users/refresh"},"response_time":"15ms","status":200,"timestamp":"2025-04-15T21:10:09Z"}
{"req":{"headers":{"accept":"application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml","accept-encoding":"gzip, deflate","accept-language":"en-US,*","authorization":"...","content-type":"application/vnd.api+json","cookie":"...","user-agent":"PolarityX/5.2.2","x-forwarded-for":"192.168.0.221","x-forwarded-host":"server.polarity.io:443","x-forwarded-proto":"https"},"method":"GET","url":"/api/channel-filters/"},"response_time":"176µs","status":200,"timestamp":"2025-04-15T21:10:09Z"}
{"req":{"headers":{"accept":"application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml","accept-encoding":"gzip, deflate","accept-language":"en-US,*","authorization":"...","content-type":"application/vnd.api+json","cookie":"...","user-agent":"PolarityX/5.2.2","x-forwarded-for":"192.168.0.221","x-forwarded-host":"server.polarity.io:443","x-forwarded-proto":"https"},"method":"GET","url":"/api/features"},"response_time":"555µs","status":200,"timestamp":"2025-04-15T21:10:09Z"}
{"req":{"headers":{"accept":"application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml","accept-encoding":"gzip, deflate","accept-language":"en-US,*","authorization":"...","content-type":"application/vnd.api+json","cookie":"...","user-agent":"PolarityX/5.2.2","x-forwarded-for":"192.168.0.221","x-forwarded-host":"server.polarity.io:443","x-forwarded-proto":"https"},"method":"GET","url":"/api/integration-types"},"response_time":"10ms","status":200,"timestamp":"2025-04-15T21:10:09Z"}

Web Container

Sample logs for the polarity_web container are shown below:

Creating /tmp/tmp.BdnKnA/ca-certificates.crt
Adding /app/certs/ca/LocalCA-pub.crt to CA store
Running /srv/entrypoint-start-caddy.sh
No Caddyfile found at /config/Caddyfile; copying from /etc/caddy/Caddyfile
{"level":"info","ts":1744751405.5055804,"msg":"using config from file","file":"/config/Caddyfile"}
{"level":"warn","ts":1744751405.5071347,"logger":"caddyfile","msg":"Unnecessary header_up X-Forwarded-For: the reverse proxy's default behavior is to pass headers to the upstream"}
{"level":"info","ts":1744751405.5082066,"msg":"adapted config to JSON","adapter":"caddyfile"}
{"level":"warn","ts":1744751405.5082276,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/config/Caddyfile","line":5}
{"level":"info","ts":1744751405.5104153,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1744751405.5107226,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000599180"}
{"level":"warn","ts":1744751405.5113587,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [polarity-server server.polarity.io]: no OCSP server specified in certificate"}
{"level":"info","ts":1744751405.512001,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","ts":1744751405.5121455,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv1"}
{"level":"warn","ts":1744751405.5152223,"logger":"tls","msg":"unable to get instance ID; storage clean stamps will be incomplete","error":"open /data/caddy/instance.uuid: permission denied"}
{"level":"warn","ts":1744751405.5156195,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":8080"}
{"level":"warn","ts":1744751405.5157907,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":8080"}
{"level":"info","ts":1744751405.5158887,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"error","ts":1744751405.5158823,"logger":"tls","msg":"could not clean default/global storage","error":"unable to acquire storage_clean lock: creating lock file: open /data/caddy/locks/storage_clean.lock: permission denied"}
{"level":"info","ts":1744751405.5159512,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1744751405.5161426,"logger":"http","msg":"enabling HTTP/3 listener","addr":":8443"}
{"level":"info","ts":1744751405.5163088,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
{"level":"info","ts":1744751405.5164497,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
{"level":"info","ts":1744751405.5167122,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1744751405.5167303,"msg":"serving initial configuration"}
{"level":"error","ts":1744751405.5637834,"logger":"http.log.error.log0","msg":"dial tcp 172.18.0.5:4001: connect: connection refused","request":{"remote_ip":"192.168.0.105","remote_port":"44772","client_ip":"192.168.0.105","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/api/servers","headers":{"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/vnd.api+json"],"Accept":["application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml"],"Cookie":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"duration":0.00111198,"status":502,"err_id":"mwv21ef93","err_trace":"reverseproxy.statusError (reverseproxy.go:1373)"}
{"level":"error","ts":1744751405.5638123,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.0.105","remote_port":"44772","client_ip":"192.168.0.105","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/api/servers","headers":{"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/vnd.api+json"],"Accept":["application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"bytes_read":0,"user_id":"","duration":0.00111198,"size":0,"status":502,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":8443\"; ma=2592000"]}}
{"level":"info","ts":1744751405.5800755,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.0.105","remote_port":"44772","client_ip":"192.168.0.105","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/v2/servers","headers":{"Accept":["application/json, application/xml, text/json, text/x-json, text/javascript, text/xml"],"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/json"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"bytes_read":0,"user_id":"","duration":0.008323238,"size":3901,"status":200,"resp_headers":{"X-Content-Type-Options":["nosniff"],"Alt-Svc":["h3=\":8443\"; ma=2592000"],"Etag":["\"d79r6x66j4zk30d\""],"Content-Type":["text/html; charset=utf-8"],"Accept-Ranges":["bytes"],"Content-Length":["3901"],"Strict-Transport-Security":["max-age=31536000"],"Vary":["Accept-Encoding"],"Last-Modified":["Thu, 23 Jan 2025 21:00:11 GMT"],"X-Frame-Options":["DENY"],"X-Xss-Protection":["1; mode=block"],"Content-Security-Policy":["default-src 'self' 'unsafe-inline' 'unsafe-eval' data: blob: wss: https:;"]}}
{"level":"error","ts":1744751407.5488908,"logger":"http.log.error.log0","msg":"dial tcp 172.18.0.5:4001: connect: connection refused","request":{"remote_ip":"192.168.0.105","remote_port":"44772","client_ip":"192.168.0.105","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/api/servers","headers":{"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/vnd.api+json"],"Accept":["application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"duration":0.001183062,"status":502,"err_id":"jemgifmqv","err_trace":"reverseproxy.statusError (reverseproxy.go:1373)"}
{"level":"info","ts":1744751405.7011292,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.0.221","remote_port":"51139","client_ip":"192.168.0.221","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/v2/servers","headers":{"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/json"],"Accept":["application/json, application/xml, text/json, text/x-json, text/javascript, text/xml"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"bytes_read":0,"user_id":"","duration":0.000310876,"size":3901,"status":200,"resp_headers":{"Content-Type":["text/html; charset=utf-8"],"Accept-Ranges":["bytes"],"Content-Length":["3901"],"Strict-Transport-Security":["max-age=31536000"],"X-Frame-Options":["DENY"],"X-Xss-Protection":["1; mode=block"],"Alt-Svc":["h3=\":8443\"; ma=2592000"],"Etag":["\"d79r6x66j4zk30d\""],"Last-Modified":["Thu, 23 Jan 2025 21:00:11 GMT"],"Content-Security-Policy":["default-src 'self' 'unsafe-inline' 'unsafe-eval' data: blob: wss: https:;"],"X-Content-Type-Options":["nosniff"],"Vary":["Accept-Encoding"]}}
{"level":"error","ts":1744751407.7086473,"logger":"http.log.error.log0","msg":"dial tcp 172.18.0.5:4001: connect: connection refused","request":{"remote_ip":"192.168.0.221","remote_port":"51139","client_ip":"192.168.0.221","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/api/servers","headers":{"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/vnd.api+json"],"Accept":["application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml"],"Cookie":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"duration":0.001105958,"status":502,"err_id":"cn2usd51e","err_trace":"reverseproxy.statusError (reverseproxy.go:1373)"}
{"level":"error","ts":1744751407.5489545,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.0.105","remote_port":"44772","client_ip":"192.168.0.105","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/api/servers","headers":{"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/vnd.api+json"],"Accept":["application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"bytes_read":0,"user_id":"","duration":0.001183062,"size":0,"status":502,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":8443\"; ma=2592000"]}}
{"level":"info","ts":1744751407.5548131,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.0.105","remote_port":"44772","client_ip":"192.168.0.105","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/v2/servers","headers":{"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/json"],"Accept":["application/json, application/xml, text/json, text/x-json, text/javascript, text/xml"],"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"bytes_read":0,"user_id":"","duration":0.000349444,"size":3901,"status":200,"resp_headers":{"Alt-Svc":["h3=\":8443\"; ma=2592000"],"Etag":["\"d79r6x66j4zk30d\""],"Last-Modified":["Thu, 23 Jan 2025 21:00:11 GMT"],"Accept-Ranges":["bytes"],"Strict-Transport-Security":["max-age=31536000"],"X-Content-Type-Options":["nosniff"],"Vary":["Accept-Encoding"],"Content-Type":["text/html; charset=utf-8"],"Content-Length":["3901"],"Content-Security-Policy":["default-src 'self' 'unsafe-inline' 'unsafe-eval' data: blob: wss: https:;"],"X-Frame-Options":["DENY"],"X-Xss-Protection":["1; mode=block"]}}
{"level":"error","ts":1744751407.7086742,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.0.221","remote_port":"51139","client_ip":"192.168.0.221","proto":"HTTP/2.0","method":"GET","host":"server.polarity.io:443","uri":"/api/servers","headers":{"Accept":["application/json, application/xml, application/vnd.api+json, text/json, text/x-json, text/javascript, text/xml"],"Cookie":["REDACTED"],"Accept-Encoding":["gzip, deflate"],"Accept-Language":["en-US,*"],"User-Agent":["PolarityX/5.2.2"],"Content-Type":["application/vnd.api+json"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"server.polarity.io"}},"bytes_read":0,"user_id":"","duration":0.001105958,"size":0,"status":502,"resp_headers":{"Server":["Caddy"],"Alt-Svc":["h3=\":8443\"; ma=2592000"]}}

Docker Logging Drivers

There are a number of logging drivers supported by Docker by default. Polarity is shipped with the json-file logging driver set up by default, with log rotation set with 6 log files of 10MB each, as seen in the Server’s docker-compose.yml file.

Depending on Server use, this may not retain enough log history to be useful. However, the settings are adjustable.

NOTE

Since containers are ephemeral, the logs associated with them last only as long as the instance does.

To retain logs longer, a log aggregator (such as Splunk or Elasticsearch) is needed.

See Polarity Source Analytics for additional information on forwarding logs.

Integration Logs

Each integration produces logs. These can be found in the individual integration’s directory, under ./logs/integration.log. These are ASCII-based text files, and may be viewed and parsed on the command line with standard POSIX tools.

Example:

The integration.log file for a Splunk integration might be found here: /app/integrations/splunk_3_6_7_node_18_27043de191_1726234895/logs/integration.log.

Log Level

By default, integrations will log at the info level. Since each data source produces different responses, the amount and usefulness of this log level may vary from integration to integration.

The log level for a given integration may be set in the individual integration’s ./config/config.json file.

Example:

For the Splunk integration mentioned above, this file would need to be edited: /app/integrations/splunk_3_6_7_node_18_27043de191_1726234895/config/config.json

Find and edit the level displayed in a block that appears like this (truncated for clarity):

...
  "logging": {
    "level": "info"
  },
...

Other logging levels available include (in order of verbosity, from least to greatest):

fatal
error
warn
info
debug
trace

Each logging level is additive, data-wise. For example, info contains all of the data in warn.