Dataminr Pulse Alerts Engine Integration User Guide

Overview

The ThreatConnect^® integration with Dataminr^® Pulse unlocks the power of real-time alerting in ThreatConnect by ingesting Dataminr Pulse Alerts into ThreatConnect and converting them into actionable threat intelligence data. Specifically, the integration will convert Alerts from Dataminr Pulse Alert Lists into Event Groups in ThreatConnect that include associated URLs, ASNs, Addresses, and File Indicators. New Alerts will be ingested into a Dataminr Pulse Alerts Source in ThreatConnect every 10 minutes starting from the time at which the integration is deployed.

Dependencies

ThreatConnect Dependencies

• Active ThreatConnect Application Programming Interface (API) key
• ThreatConnect instance with version 7.2.0 or newer installed

Dataminr Dependencies

• Dataminr Pulse API public key
• Dataminr Pulse API secret (no permissions, roles, or additional steps required)

Application Setup and Configuration

1. Log into ThreatConnect with a System Administrator account.
2. Install the Dataminr Pulse Alerts Engine App via TC Exchange™.
3. Use the ThreatConnect Feed Deployer to set up and configure the Dataminr Pulse Alerts Engine App.

Configuration Parameters

Parameter Definitions

The parameters defined in Table 1 apply to the configuration parameters available when using the Feed Deployer to configure the Dataminr Pulse Alerts Engine App.

Dataminr Pulse Alerts Engine

After successfully configuring and activating the Feed API Service for the Dataminr Pulse Alerts Engine App, you can access the Dataminr Pulse Alerts Engine user interface (UI). This UI allows you to interact with and manage the Dataminr integration.

Follow these steps to access the Dataminr Pulse Alerts Engine UI:

1. Log into ThreatConnect with a System Administrator account.
2. Hover over Playbooks on the top navigation bar and select Services.
3. Locate the Dataminr Pulse Alerts Engine Feed API Service on the Services screen, and then click the link in the Service’s API Path field to open the DASHBOARD screen of the Dataminr Pulse Alerts Engine UI.

The following screens are available in the Dataminr Pulse Alerts Engine UI:

• DASHBOARD
• JOBS
• TASKS
• DOWNLOADS
• REPORTS

DASHBOARD

The DASHBOARD screen (Figure 1) provides an overview of the total number of Alerts that ThreatConnect has ingested from Dataminr Pulse.

JOBS

The JOBS screen (Figure 2) breaks down the ingestion of Dataminr Pulse data into manageable Job-like tasks.

• Job Type: (Optional) Select a Job type by which to filter Jobs. Available types include ad-hoc and scheduled.
• Status:(Optional) Select a Job status by which to filter Jobs. Available statuses include the following:
  1. Download In Progress
  2. Download Complete
  3. Convert In Progress
  4. Convert Complete
  5. Upload In Progress
  6. Upload Complete
• Request ID: (Optional) Enter text into this box to search for a specific Job by its request ID.

To open a Job’s Request Details drawer (Figure 3) and view more details about the Job, select the Job's entry in the table on the JOBS screen.

TASKS

The TASKS screen (Figure 4) is where you can view and manage the Tasks for each Job. Most users will not need to perform any actions on this screen, as it is primarily for advanced users.

DOWNLOADS

The DOWNLOADS screen (Figure 5) is where you can download individual Alerts from Dataminr Pulse using the Dataminr Alert ID.

• ID(s): Enter the Dataminr Alert ID(s) for the Alert(s) to download. Data will be retrieved in JavaScript^® Object Notation (JSON) format.
• Convert: Select this checkbox to convert the data to ThreatConnect batch format, or leave this checkbox cleared if you need the raw data from Dataminr.
• Enrich: Select this checkbox to submit the data to the ThreatConnect Batch API.

REPORTS

The REPORTS screen provides a BATCH ERRORS view (Figure 6) that displays batch errors for each request in a tabular format. Details provided for each error include the error’s code, message, and reason.

Data Mappings

The data mappings in Table 2 illustrate how data are mapped from Dataminr API endpoints into the ThreatConnect data model.

Alert

ThreatConnect object type: Event Group

Frequently Asked Questions (FAQ)

When does ThreatConnect start ingesting data from Dataminr Pulse? 

ThreatConnect ingests Dataminr Pulse data as soon as the Dataminr Pulse Alerts Engine App is deployed successfully, with the time of deployment being the starting time from which to retrieve Alerts. This means that ThreatConnect will ingest only Alerts created at the time of deployment or later; it will not ingest Alerts created before the time of deployment.

Does ThreatConnect ingest Alerts from all Dataminr Pulse Alert Lists? 

Yes, ThreatConnect will ingest Alerts from any Dataminr Pulse Alert Lists you have provisioned.

In ThreatConnect, how can I view certain Alert categories that contain malicious Indicators?

Use a ThreatConnect Query Language (TQL) query to filter the ingested Alerts based on their category. The following categories will always contain malicious Indicators from Alerts:

• Category: Network Scans
• Category: Phishing
• Category: Malware
• Category: Domain Impersonation

Other categories may also be used when filtering Indicators with TQL; however those categories may not contain malicious Indicators.

For example, the following query will return only Alerts that belong to the Network Scans, Phishing, Malware, or Domain Impersonation categories and contain Indicators (that is, the query will return only Event Groups that are associated to one or more Indicators and have a Category: Network Scans, Category: Phishing, Category: Malware, or Category: Domain Impersonation Tag in ThreatConnect):

typeName in ("Event") and hasIndicator() and tag in("Category: Network Scans", "Category: Phishing", "Category: Malware", "Category: Domain Impersonation")

In ThreatConnect, how can I view only Alerts from select Alert Lists?

Use a TQL query to filter the ingested Alerts based on the Alert Lists to which they belong. For example, the following query will return only Alerts that belong to the Cyber Alerts or Ransomware Alert Lists (that is, the query will return only Event Groups that have an Alert List Attribute with a value of Cyber Alerts or Ransomware in ThreatConnect):

typeName in ("Event") and attributeAlert_List in ("Cyber Alerts", "Ransomware")

In ThreatConnect, how can I view only Alerts assigned a certain priority?

Use a TQL query to filter the ingested Alerts based on their priority. For example, the following query will return only Alerts whose priority is either Urgent or Flash (that is, the query will return only Event Groups that have a Priority Attribute with a value of Urgent or Flash in ThreatConnect):

typeName in ("Event") and attributePriority in ("Urgent", "Flash")

ThreatConnect^® is a registered trademark, and TC Exchange™ is a trademark, of ThreatConnect, Inc.
Dataminr^® is a registered trademark of Dataminr, Inc.
JavaScript^® is a registered trademark of Oracle Corporation.

30087-01 EN Rev. C