Feed Metrics and Report Card
  • 08 Sep 2022
  • 5 Minutes to read
  • Dark
    Light

Feed Metrics and Report Card

  • Dark
    Light

Article Summary

Minimum Role: System role of Administrator

Prerequisites: System settings configured to populate the Feeds tab of the TC Exchange Settings screen (see the “Apps and Jobs” section of ThreatConnect System Administration Guide for more information)

Overview

The TC Exchange™ Settings screen includes a Feeds tab that lists all feeds available to a ThreatConnect® instance. System Administrators can use the metrics provided for the feeds, including a report card for each feed, to determine which feeds they want to provide to their ThreatConnect instance. The metrics are derived from information gathered from ThreatConnect’s Collective Analytics Layer (CAL™).

See the ThreatConnect blog post “Introducing ThreatConnect’s Intel Report Cards” for more information on feed report cards and The Feed Explorer for more information on open-source feeds accessible by all ThreatConnect users.

Note
Report cards are available only for select feeds on which CAL collects data. CAL does not currently collect data on premium feeds, custom feeds, and certain open-source feeds, so report cards for these feeds are not provided.

Enabling CAL

CAL must be enabled in two places in order to view report card data. First, it must be enabled in the System Settings screen in ThreatConnect. Second, the System Organization must be given permission to enable CAL data from the Account Settings screen.

System Settings

  1. Log into ThreatConnect with a System Administrator account.
  2. On the top navigation bar, hover the cursor over Settings A picture containing text, clipart, light  Description automatically generated and select System Settings. The Settings tab of the System Settings screen will be displayed.
  3. Select Data from the menu on the left side of the System Settings screen, scroll down to the CAL section, and select the CALEnabled checkbox (Figure 1). Graphical user interface, text, application, email  Description automatically generated

     

Account Settings

  1. Log into ThreatConnect with a System Administrator account.
  2. On the top navigation bar, hover the cursor over Settings A picture containing text, clipart, light  Description automatically generated and select Account Settings. The Organizations tab of the Account Settings screen will be displayed.
  3. Use the search box to locate the System Organization (Figure 2). Graphical user interface, application, Teams  Description automatically generated

     

  4. Click Edit Icon  Description automatically generated in the Options column. The Standard Options tab of the Organization Information window will be displayed (Figure 3). Graphical user interface, application  Description automatically generated

     

  5. Click the Permissions tab (Figure 4). Graphical user interface, text, application  Description automatically generated

     

  6. Select the Enable CAL Data checkbox, and then click the SAVE button. If this checkbox is already selected, click the CANCEL button.

Feed Metrics

  1. Log into ThreatConnect with a System Administrator account.
  2. On the top navigation bar, hover the cursor over Settings A picture containing text, clipart, light  Description automatically generated and select TC Exchange Settings. The Installed tab of the TC Exchange Settings screen will be displayed.
  3. Click the Feeds tab. The Feeds tab of the TC Exchange Settings screen will be displayed (Figure 5). Graphical user interface, text, application, email  Description automatically generated

     

    • The Feeds tab provides information and metrics about the data provided by a feed and allows you to activate and deactivate feeds.
    • Name: This column provides the name of the feed.
    • Description: This column provides a description of the feed.
    • Reliability Rating: This column provides a letter grade indicating how reliable the feed is, from F (worst) to A+ (best). It is derived from the number of false positives found in the feed, among other things, and is a measure of how likely a feed is to yield large numbers of negatively impactful false positives.
    • Unique Indicators: This column provides the number, in thousands, of Indicators in this feed that are unique. For example, a value of <1k indicates that fewer than 1,000 Indicators in the feed are unique, whereas a value of 3k+ indicates that over 3,000 Indicators in the feed are unique.
    • Report Card: Click the graph icon to display a graphic showing metrics from the other columns and how they compare with aggregated metrics from other feeds. See the “Report Card” section for more information.
    • Active: Toggle the slider on or off to activate or deactivate the feed, respectively.
Note
System Administrators can still manage their own feeds. However, the Feeds tab allows feed ingestion to be set up by toggling the slider in the Active column and produces historical data curated by the ThreatConnect Analytics team.

Report Card

Figure 6 shows the report card for the Malware Domain Blocklist feed.

 

A list of Common Classifiers from CAL that apply to the feed is displayed on the right side of the report card. The six bullet graphs displayed in the middle of the report card provide data on the following metrics:

  • Reliability Rating: This metric is the same as Reliability Rating on the Feeds tab.
  • Unique Indicators: This metric is the same as Unique Indicators on the Feeds tab, except the number of unique Indicators is represented as a percentage. For example, a value of 44% means that 44% of the Indicators in the feed are not found in other feeds (and, consequently, that 56% of the Indicators appear in other feeds).
  • First Reported: This metric is a measure of how often a feed is the first feed to report a particular Indicator when that Indicator is observed in other feeds as well.
  • Scoring Disposition: Like ThreatAssess, CAL produces a score for each Indicator that measures how dangerous the Indicator is, on a scale of 0 (benign) to 1000 (very dangerous). The Scoring Disposition metric is a weighted average of the CAL scores for the Indicators in the feed.
  • Classifier Coverage: This metric indicates the percentage of Indicators in the feed that have at least one Classifier applied by CAL’s analytics. It is a measure of how well existing analytics can qualitatively understand the data from the feed.
  • Indicator Status Coverage: This metric indicates the percentage of Indicators in the feed that have a definitive Indicator Status set by CAL. It is a measure of how conclusively CAL’s analytics can provide quantitative statements of the data from the feed.

Each bullet graph uses four visual elements to put the data for the feed in context with the other feeds:

  • Horizontal Black Line: The horizontal black line represents the value of the metric for the particular feed. For example, in Figure 6, for Unique Indicators, the black line represents a value of 44%, where the left side of the chart is a value of 0% and the right side of the chart is a value of 100%.
  • Vertical Orange Line: The vertical orange line represents the target value of the metric across all feeds. The target value is computed by CAL and ThreatConnect analysts to help determine which feeds have the most impact. In this example, the 44% value for Unique Indicators for the Malware Domain Blocklist is a lot less than the target value of this metric across all feeds, indicating that, on average, this feed provides a lesser number of unique Indicators than other feeds do.
  • Colored Bands: The red, yellow, and green bands represent algorithmically derived segments of quality, where red is a “bad” range, yellow is a “medium” range, and green is a “good” range. For the Malware Domain Blocklist, its Unique Indicators value, at 44%, falls in the “bad” range, while its First Reported value, at 92%, falls in the “good” range.
  • Value: The value for each metric is given to the right of its respective bullet graph.

The Daily Indicators graph, which appears below the six bullet graphs, is a sparkline depiction of the number of Indicators the feed is bringing in per day over the last 30 days. The value to the right of the graph indicates the total number of Indicators in the feed added in the last 30 days.


ThreatConnect® is a registered trademark of ThreatConnect, Inc.
 CAL™ and TC Exchange™ are trademarks of ThreatConnect, Inc.

20070-01 v.04.C


Was this article helpful?