Feed Metrics and Report Card

Overview

The TC Exchange™ Settings screen includes a Feeds tab that lists all feeds available to a ThreatConnect^® instance. System Administrators can use the metrics provided for the feeds, including a report card for each feed, to determine which feeds they want to provide to their ThreatConnect instance. The metrics are derived from information gathered from CAL™.

See the ThreatConnect blog post “Introducing ThreatConnect’s Intel Report Cards” for more information on feed report cards and The Feed Explorer for more information on open-source feeds accessible by all ThreatConnect users.

Before You Start

Enabling CAL

CAL must be enabled in two places in order to view report card data. First, it must be enabled in the System Settings screen in ThreatConnect. Second, the System Organization must be given permission to enable CAL data from the Account Settings screen.

System Settings

1. Log into ThreatConnect with a System Administrator account.
2. On the top navigation bar, hover the cursor over Settingsand select System Settings. The Settings tab of the System Settings screen will be displayed.
3. Select Data from the menu on the left side of the System Settings screen, scroll down to the CAL section, and select the CALEnabled checkbox (Figure 1).

    

Account Settings

1. Log into ThreatConnect with a System Administrator account.
2. On the top navigation bar, hover the cursor over Settingsand select Account Settings. The Organizations tab of the Account Settings screen will be displayed.
3. Use the search box to locate the System Organization (Figure 2). 

    

4. Click Editin the Options column. The Standard Options tab of the Organization Information window will be displayed (Figure 3). 

    

5. Click the Permissions tab (Figure 4). 

    

6. Select the Enable CAL Data checkbox, and then click the SAVE button. If this checkbox is already selected, click the CANCEL button.

Feed Metrics

1. Log into ThreatConnect with a System Administrator account.
2. On the top navigation bar, hover the cursor over Settingsand select TC Exchange Settings. The Installed tab of the TC Exchange Settings screen will be displayed.
3. Click the Feeds tab. The Feeds tab of the TC Exchange Settings screen will be displayed (Figure 5). 

    

   • The Feeds tab provides information and metrics about the data provided by a feed and allows you to activate and deactivate feeds.
   • Name: This column provides the name of the feed.
   • Description: This column provides a description of the feed.
   • Reliability Rating: This column provides a letter grade indicating how reliable the feed is, from F (worst) to A+ (best). It is derived from the number of false positives found in the feed, among other things, and is a measure of how likely a feed is to yield large numbers of negatively impactful false positives.
     
     Important

     If a value of “--” is displayed in the Reliability Rating column for a feed, then the feed is either retired or a new feed that does not have data populated yet.
   • Unique Indicators: This column provides the number, in thousands, of Indicators in this feed that are unique. For example, a value of <1k indicates that fewer than 1,000 Indicators in the feed are unique, whereas a value of 3k+ indicates that over 3,000 Indicators in the feed are unique.
   • Report Card: Click the graphicon to display a graphic showing metrics from the other columns and how they compare with aggregated metrics from other feeds. See the “Report Card” section for more information.
     
     Important

     If a message stating “No data is currently available for this feed.” is displayed when you click the graphicon for a feed, then the feed is either retired or a new feed that does not have data populated yet.
   • Active: Toggle the slider on or off to activate or deactivate the feed, respectively.

Report Card

Figure 6 shows the report card for the CINS Army IP List feed.

A list of Common Classifiers from CAL that apply to the feed is displayed on the right side of the report card. The six bullet graphs displayed in the middle of the report card provide data on the following metrics:

• Reliability Rating: This metric is the same as Reliability Rating on the Feeds tab.
• Unique Indicators: This metric is the same as Unique Indicators on the Feeds tab, except the number of unique Indicators is represented as a percentage. For example, a value of 75% means that 75% of the Indicators in the feed are not found in other feeds (and, consequently, that 25% of the Indicators appear in other feeds).
• First Reported: This metric is a measure of how often a feed is the first feed to report a particular Indicator when that Indicator is observed in other feeds as well.
• Scoring Disposition: Like ThreatAssess, CAL produces a score for each Indicator that measures how dangerous the Indicator is, on a scale of 0 (benign) to 1000 (very dangerous). The Scoring Disposition metric is a weighted average of the CAL scores for the Indicators in the feed.
• Classifier Coverage: This metric indicates the percentage of Indicators in the feed that have at least one Classifier applied by CAL’s analytics. It is a measure of how well existing analytics can qualitatively understand the data from the feed.
• Indicator Status Coverage: This metric indicates the percentage of Indicators in the feed that have a definitive Indicator Status set by CAL. It is a measure of how conclusively CAL’s analytics can provide quantitative statements of the data from the feed.

Each bullet graph uses four visual elements to put the data for the feed in context with the other feeds:

• Horizontal Black Line: The horizontal black line represents the value of the metric for the particular feed. For example, in Figure 6, for Unique Indicators, the black line represents a value of 75%, where the left side of the chart is a value of 0% and the right side of the chart is a value of 100%.
• Vertical Orange Line: The vertical orange line represents the target value of the metric across all feeds. The target value is computed by CAL and ThreatConnect analysts to help determine which feeds have the most impact. In this example, the 75% value for Unique Indicators for the CINS Army IP List feed is less than the target value of this metric across all feeds, indicating that, on average, this feed provides a lesser number of unique Indicators than other feeds do.
• Colored Bands: The red, yellow, and green bands represent algorithmically derived segments of quality, where red is a “bad” range, yellow is a “medium” range, and green is a “good” range. For the CINS Army IP List feed, its Unique Indicators value, at 75%, falls in the “medium” range, while its First Reported value, at 45%, falls in the “good” range.
• Value: The value for each metric is given to the right of its respective bullet graph.

The Daily Indicators graph, which appears below the six bullet graphs, is a sparkline depiction of the number of Indicators the feed is bringing in per day over the last 30 days. The value to the right of the graph indicates the total number of Indicators in the feed added in the last 30 days.

ThreatConnect^® is a registered trademark, and CAL™ and TC Exchange™ are trademarks, of ThreatConnect, Inc.

20070-01 v.05.A