Overview

The ThreatConnect^® integration with Recorded Future^® ingests Domain, Hash, IP, URL, and Vulnerability Risk List entities, as well as Analyst Notes, from Recorded Future. After ingesting these data, the integration creates corresponding objects with select Recorded Future metadata in ThreatConnect.

Dependencies

ThreatConnect Dependencies

• Active ThreatConnect Application Programming Interface (API) key
• ThreatConnect instance with version 7.4.0 or newer installed

Recorded Future Dependencies

• Active Recorded Future API token
• Active Recorded Future module subscriptions: SecOps Intelligence, Threat Intelligence, and/or Vulnerability Intelligence (see Table 1 for more information)

Application Setup and Configuration

1. Log into ThreatConnect with a System Administrator account.
2. Install the Recorded Future Intelligence Engine App via TC Exchange™.
3. Use the ThreatConnect Feed Deployer to set up and configure the Recorded Future IntelligenceEngine App.

Configuration Parameters

Parameter Definitions

The parameters defined in Table 1 apply to the configuration parameters available when using the Feed Deployer to configure the App.

Recorded Future Intelligence Engine

After successfully configuring and activating the Feed API Service, you can access the Recorded Future Intelligence Engine user interface (UI). This UI allows you to interact with and manage the Recorded Future integration.

Follow these steps to access the UI:

1. Log into ThreatConnect with a System Administrator account.
2. On the top navigation bar, hover over Playbooks and select Services. The Services tab of the Playbooks screen will be displayed. 
3. Locate the Recorded Future Intelligence Engine Feed API Service and then click the link in the Service’s API Path field. The DASHBOARD screen of the Recorded Future Intelligence Engine UI will open in a new browser tab.

The following screens are available in the Recorded Future Intelligence Engine UI:

• DASHBOARD
• JOBS
• TASKS
• DOWNLOAD
• REPORT

DASHBOARD

The DASHBOARD screen (Figure 1) provides an overview of the total number of Risk List entities (Domain, Hash, IP, URL, and Vulnerability) and Analyst Notes retrieved from Recorded Future. Depending on the data available to you, cards representing all or a subset of these object types will be displayed on the DASHBOARD screen.

JOBS

The JOBS screen breaks down the ingestion of Recorded Future data into manageable Job-like tasks. There are two views available for the JOBS screen: Risk Lists and Analyst Notes.

Risk Lists

Hover over the JOBS tab and select Risk Lists to display the Download Risk Lists Jobs screen (Figure 2). This screen breaks down the ingestion of Risk List entities into manageable Job-like tasks.

• Job Type: If desired, select a Job type by which to filter Jobs. Available types include ad-hoc and scheduled. 
• Status:If desired, select a Job status by which to filter Jobs. Available statuses include the following:
  1. Convert Risk Lists Complete
  2. Convert Risk Lists In Progress
  3. Download Risk Lists Complete
  4. Download Risk Lists In Progress
  5. Enrich Risk Lists Complete
  6. Enrich Risk Lists In Progress
  7. Failed
  8. Pending
  9. Upload Risk Lists Complete
  10. Upload Risk Lists In Progress
• Request ID: If desired, enter text into this box to search for a specific Job by its request ID.
• + Add Request: Click this button to display the ADD REQUEST window (Figure 3). On this window, you can specify the object types for an ad-hoc Job request. After a Job request is added, it will be displayed in the table on the JOBS screen (Figure 2), and its Job type will be listed as ad-hoc.
  
  Note

  Recorded Future provides a single Risk List file for each entity type that contains the most current data available. Using the + Add Request feature will always download the most recent Risk List data.

  

   

Analyst Notes

Hover over the JOBS tab and select Analyst Notes to display the Download Analyst Notes Jobs screen (Figure 4). This screen breaks down the ingestion of Analyst Notes into manageable Job-like tasks.

• Status:If desired, select a Job status by which to filter Jobs. Available statuses include the following:
  1. Convert Analyst Notes Complete
  2. Convert Analyst Notes In Progress
  3. Download Analyst Notes Complete
  4. Download Analyst Notes In Progress
  5. Failed
  6. Pending
  7. Upload Analyst Notes Complete
  8. Upload Analyst Notes In Progress
• Request ID: If desired, enter text into this box to search for a specific Job by its request ID.

TASKS

The TASKS screen (Figure 5) is where you can view and manage the Tasks for each Job. Tasks are divided into separate Analyst Notes and Risk Lists workstreams for the Recorded Future integration.

DOWNLOAD

The DOWNLOAD RISK LIST ITEMS screen (Figure 6) is where you can download data for Risk List entities. Data for Analyst Notes cannot be downloaded.

• ID(s): Enter the Recorded Future Risk List ID(s) for the object(s) to download. Data will be retrieved in JavaScript^® Object Notation (JSON) format. When entering the IDs for Domain, Hash, IP, and URL Risk List entities, prepend idn:, hash:, ip:, and url:, respectively, to the entity's ID. For Vulnerability Risk List entities, use the CVE ID or Recorded Future ID. The following examples demonstrate the ID format that should be used for each Risk List entity type:
  • Domain: idn:efavengh.com
  • Hash: hash:092910024190a2521f21658be849c4ac9ae6fa4d5f2ecd44c9055cc353a26875
  • IP: ip:124.71.84.65
  • URL: url:https://send.exploit.in/
  • Vulnerability: CVE-2019-0841 or ZgFn9x
• Convert: Select this checkbox to convert the threat intelligence data to ThreatConnect batch format.
• Enrich: Select this checkbox to submit the threat intelligence data to the ThreatConnect Batch API.

REPORTS

The REPORTS screen provides two views: BATCH ERRORS and REPORT UPLOAD TRACKER. The BATCH ERRORS screen (Figure 7) displays batch errors for each request in a tabular format. Details provided for each error include the error’s code, message, and reason.

The PDF TRACKER screen (Figure 8) is where you can view ThreatConnect's attempts to download PDFs from Recorded Future. The table on this screen displays the most recent date on which ThreatConnect attempted to download a PDF, the number of times an attempt to download the PDF was made, and whether the PDF was downloaded successfully. You can also search for PDFs by ID on this screen, which can be useful if you do not see a Recorded Future PDF in ThreatConnect as expected.

Data Mappings

The data mappings in Table 2 through Table 8 illustrate how data are mapped from Recorded Future Intelligence API endpoints into the ThreatConnect data model.

Domain

ThreatConnect object type: Host Indicator

Hash

ThreatConnect object type: File Indicator

IP

ThreatConnect object type: Address Indicator

URL

ThreatConnect object type: URL Indicator

Vulnerability

ThreatConnect object type: Vulnerability Group

Analyst Note

ThreatConnect object type: Report Group

Note Entity

Risk Score Mappings

ThreatConnect follows the Criticality mapping in Recorded Future when assigning a Threat Rating to data ingested from Recorded Future; however, because the Recorded Future Criticality rating only goes from 0–4, it has been augmented by 1 in ThreatConnect to fit the 0–5 scale for Threat Rating.

Frequently Asked Questions (FAQ)

When configuring the Recorded Future Intelligence Engine App, I do not see Analyst Notes offered as a downloadable data type. How do I ensure my Recorded Future Analyst Notes are imported into ThreatConnect?

Analyst Notes added to the selected Risk List entity type(s), as well as those created in your Recorded Future modules within the last 24 hours, will be imported into ThreatConnect. See the “Analyst Note” section for more information on how Analyst Notes are mapped to the ThreatConnect data model.

Why are there several URL errors in the batch errors report? (e.g., [xyz.com] could not be processed as a valid URL due to missing or invalid data (summary is invalid for the given type))

The ThreatConnect Recorded Future Intelligence Engine App may run into instances where URLs coming from Recorded Future use an invalid URL format. Some examples of this behavior include the following:

• ww3.xyz.com: This URL is missing the protocol, such as http://.
• http:ww2.xyz.com/page#: This URL is terminated with a special character.

URL objects with an invalid URL format will not be imported into ThreatConnect. Note that this issue occurs rarely.

Why are Indicators with risk scores that are less than the Minimum Risk Score being ingested into ThreatConnect?

These Indicators are ingested because they exist as links from other Risk List entities. To prevent Indicators with a risk score less than the minimum risk score (i.e., the value for the App's Minimum Risk Score parameter) from being ingested, clear the Collect Indicators Linked in Recorded Future Below the Minimum Risk Score checkbox in the Feed Deployer when configuring and deploying the App.

How does the Recorded Future Intelligence Engine Feed API Service App differ from the Recorded Future Risk List Job App?

The Recorded Future Risk List Job App allows users to do the following:

• collect data from Risk List entities
• create Indicators with evidence details
• map evidence details to a Description Attribute and risk rules to Tags

The Recorded Future Intelligence Engine Feed API Service App collects data in the following ways:

• ingests Risk List entities with several of their Attributes
• ingests all of the Analyst Notes and attached PDFs associated with Risk List entities
• ingests associated Risk List entities and allows users to view the first-level associations created between the entities and Analyst Notes (i.e., the actual link)
• obtains a link to each Risk List entity's Recorded Future Intelligence Card
• ingests Analyst Notes that have been published in the last 24 hours on a daily basis

For more information on how Feed API Service Apps function in ThreatConnect, see Feed API Services.

How long does the Recorded Future Intelligence Engine Feed API Service App take to ingest a complete set of data on its initial run?

In most cases, the Recorded Future Intelligence Engine App takes 2–4 days to complete the initial data ingestion, depending on how the customer configured risk score and which Risk List entity types they selected to ingest in the Feed Deployer.

In which order are Risk List entities ingested on the initial run?

On the initial run of the Recorded Future Intelligence Engine App, Risk List entities will be ingested in the following order:

1. Domain
2. Hash
3. IP
4. URL
5. Vulnerability

Users will likely see clusters of the Risk List entities ingested in chunks along with the associated or linked entities. Note that there may be delays between the creation of the Risk List source entities and the Risk List link entities in ThreatConnect.

How often does the integration ingest each Risk List entity type after the initial run?

See the following table for each Risk List entity type's download frequency. Analyst Notes are downloaded daily.

How can I identify where the integration collected Indicators and Groups from in Recorded Future?

The following Source: Tags, which are applied to Indicators and Groups in ThreatConnect that were ingested from Recorded Future, indicate where the integration collected them from in Recorded Future:

• Source: Risk List: The entity was ingested from a Risk List.
• Source: Risk List Link: The entity was ingested as a linked entity from a Risk List entity.
• Source: Analyst Note: The entity was ingested from an Analyst Note.
• Source: Analyst Note Link: The entity was ingested as a linked entity from an Analyst Note.

Note that an Indicator or Group may have more than one of these Tags applied to them, as the entities could be associated to one another.

Why are associations to Malware and Adversaries in Recorded Future missing from Indicators and Groups in ThreatConnect?

Currently, Indicators and Groups created by the Recorded Future Intelligence Engine App will have Tags representing associated Malware and Adversaries applied to them, allowing users to pivot on and explore these relationships in ThreatConnect. Support for direct associations to Malware and Adversaries will be available in a future version of the Recorded Future Intelligence Engine App.

ThreatConnect^® is a registered trademark, and TC Exchange™ is a trademark, of ThreatConnect, Inc.
JavaScript^® is a registered trademark of Oracle Corporation.
Recorded Future^® is a registered trademark of Recorded Future, Inc.
MITRE ATT&CK^® and ATT&CK^® are registered trademarks of The MITRE Corporation.

30083-01 EN Rev. A