urlscan.io Enrichment

Enabling the urlscan.io Enrichment

Before you can retrieve data from urlscan.io for URL Indicators, a System Administrator must first enable and configure the urlscan.io enrichment in ThreatConnect.

1. Log into ThreatConnect with a System Administrator account.
2. On the top navigation bar, hover over Settingsand select System Settings. The System Settings screen will be displayed with the Settings tab selected.
3. Select the Indicators tab. The Indicators screen will be displayed.
4. Click Enrichment Tools in the menu on the left side of the Indicators screen. The Enrichment Tools screen will be displayed.
5. Click Editin the Options column for URLScan. The Edit Vendor window will be displayed (Figure 1).
   

    

   • Enable Vendor: Select this checkbox to enable urlscan.io.
   • Enable Automatic Retrieval: Select this checkbox to enable automatic data retrieval for urlscan.io. If automatic data retrieval is enabled, urlscan.io data will automatically populate when a user clicks on an Indicator’s Enrichment tab for the first time. This checkbox is selected by default.
   • API Key: Enter the API key that will be used to retrieve data from urlscan.io.
   • VALIDATE: After entering the urlscan.io API key, click this button to validate it. If the API key is accepted, the VALIDATE button’s label will change to VALID, indicating that a valid API key has been entered. If the API key is not accepted, a message stating “API Key is invalid.” will be displayed at the top of the Edit Vendor window.
   • Query Visibility: Select the visibility level for queries sent to urlscan.io. Available choices include Private, Public, and Unlisted.
   • Tag Settings: Select the Automatically import URLScan tags checkbox to import tags applied to a URL in urlscan.io into ThreatConnect as Tag objects and apply them to the corresponding URL Indicator.
     Note

     If the Automatically import URLScan tags checkbox is selected and a tag applied to a URL in urlscan.io exactly matches an ATT&CK^® Tag in ThreatConnect, then that ATT&CK Tag will be applied to the URL Indicator in ThreatConnect; otherwise, a new standard Tag will be created and applied to the URL Indicator in ThreatConnect.
   • Lookup/Retrieve: Select the Indicator type(s) for which to retrieve data from urlscan.io. The only available Indicator type is URL.
   • Click the SAVE button.

When urlscan.io is enabled, a value of true will be displayed in the Enabled column for its entry on the Enrichment Tools screen.

Data Overview

The Overview section of the URLScan card (Figure 2) provides a summary of data retrieved from urlscan.io for a URL Indicator and the date and time the data were last retrieved.

• Detection: This field specifies whether the scanned URL is malicious.
• Domain: The primary domain or subdomain contacted by the URL.
• IP: The primary IP address contacted by the URL.
• Submitted URL: The original URL that was submitted to urlscan.io.
• Effective URL: The URL of the primary webpage after redirection.
• Country: The country associated with the primary IP address contacted by the URL.
• ASN name: The primary autonomous system (AS) number and name contacted by the URL.
• Website Contacted: The number of unique IP addresses, countries, and domains or subdomains the URL contacted in order to perform the specified number of HTTP transactions.
• Tags: The tag(s) applied to the URL in urlscan.io.
• Targeted Brands/Vertical: The name and industry vertical of the brand(s) targeted by the URL.

URLScan Detailed View

Click the Open Detailed View link on the URLScan card to display the URLScan Detailed View drawer (Figure 3). This drawer displays cards with additional data retrieved from urlscan.io.

The cards displayed on the URLScan Detailed View drawer are collapsed by default. Click on a card to expand it and view its data. To collapse or expand all cards, click the Collapse All or Expand All button, respectively, at the top right of the drawer. Figure 4 shows the URLScan Detailed View drawer in Figure 3 with all available cards expanded.

See Table 1 for a list of cards that may be displayed on the URLScan Detailed View drawer for a URL Indicator.

Importing Indicators From urlscan.io Into ThreatConnect

You may import all or a subset of the Indicators displayed on the IP and ASN Details and Links to Domains cards into ThreatConnect and associate them to a new or existing Group. You may also import Indicators displayed on these cards into ThreatConnect and associate them directly to the enriched Indicator (i.e., the URL Indicator whose Details screen you are viewing) via a custom association.

1. On the URLScan Detailed View drawer, expand one of the following cards to display a table containing Indicators retrieved from urlscan.io that are related to the enriched URL Indicator (Figure 4):
   • IP and ASN Details: The Indicators on this card will be imported as Address Indicators.
   • Links to Domains: The Indicators on this card will be imported as Host Indicators.
2. On the expanded card, select the checkbox for each Indicator you want to import into ThreatConnect. To select all Indicators displayed on the current page in the table, select the checkbox in the table’s header.
   Important

   If a selected Indicator already exists in the ThreatConnect owner into which you are importing data, that copy of the Indicator will be updated based on the information entered and options configured during the import.
3. Click the Import dropdown at the top left of the expanded card and select one of the following import options (Figure 5). Further instruction on each import option is available in the following subsections.

   

    

   • To New Group: Select this option to import the selected Indicators and associate them to a new Group created during the import.
   • To Existing Group: Select this option to import the selected Indicators and associate them to an existing Group.
   • As an Indicator: Select this option to import the selected Indicators and associate them directly to the enriched Indicator via a custom association.

Importing Indicators Into a New Group

1. Follow Steps 1–3 in the “Importing Indicators From urlscan.io Into ThreatConnect” section and select To New Group from the Import dropdown. The Details section of the Create screen will be displayed (Figure 6).

   

    

   • Type: By default, Event is selected. If desired, select another Group type from the dropdown.
   • Fill out all remaining fields on the Details section. For descriptions of each field available on this screen, see the “Creating a Group” section of Create.
     Note

     By default, a UrlScan Enrichment Tag will be applied in the Tags field. You can remove this Tag, if desired.
     Important

     If you select the Apply Tags to Associations checkbox, it is recommended that you remove the UrlScan Enrichment Tag so that this Tag does not get applied to the enriched Indicator (that is, the Indicator whose Enrichment tab from which you are importing urlscan.io data), as the enriched Indicator will be added as an association to the new Group. Alternatively, if you want the UrlScan Enrichment Tag to be applied to all associations except the enriched Indicator, you can select the Apply Tags to Associations checkbox, retain the UrlScan Enrichment Tag, and then, after completing the import, navigate to the Overview tab of the Details screen for the enriched Indicator and remove the Tag manually.
   • Click the Next button.
     Note

     The Save button is available only on the Associations and Attachments sections.
2. The Associations section will be displayed (Figure 7).

   

    

   Note

   A checkmark in the Known column indicates that the corresponding Indicator exists in at least one owner to which you have access.
   • Associations: The selected Indicator(s), as well as the enriched URL Indicator and its corresponding Host Indicator, will be displayed in the Associations card. In this card, you can complete the following actions:
     • Private: To mark an Indicator as private, select the corresponding checkbox in the Private column. This column will be displayed only if your System Administrator has enabled private Indicators.
     • Actions: To remove an Indicator from the list of Indicators being imported, click Delete.
   • Association Details: In the Associations Details card, you can fill out the following information, which will be applied to all Indicators being associated to the Group:
     Important

     All information added in this section will be applied to the enriched Indicator (that is, the Indicator whose Enrichment tab from which you are importing urlscan.io data), because the enriched Indicator is always added as an association to the new Group, along with the Indicators selected on the card on the URLScan Detailed View (Figure 5). If the enriched Indicator has a default Description, a Threat Rating, or a Confidence Rating and you enter a value for one of these fields, then that value will replace the existing value for the enriched Indicator. Tags entered in this section will be applied in addition to the enriched Indicator’s existing Tags.
     • Description: Enter a default Description for the Indicator(s).
     • Tags: Enter one or more Tags to apply to the Indicator(s).
     • Threat Rating: Use the skull icons to set the Threat Rating for the Indicator(s).
     • Confidence Rating: Use the slider to set the Confidence Rating for the Indicator(s).
   • Click the Next button.
3. The Attachments section will be displayed (Figure 8). Attachments is an optional section where you can attach related files to the Group.

   

    

   • Upload files for which Document Groups will be created and associated to the Group being created, if desired. After each file is uploaded, the filename will be displayed below the upload area, along with a checkbox labeled Add to Malware Vault. Leave this checkbox cleared unless you are uploading a malware file.
   • Click the Save button.

The selected Indicator(s) will be imported into ThreatConnect and associated to the newly created Group, and the Group’s Details screen will be displayed. These Indicators will be displayed on the Indicators card of the Group’s Associations tab. You may also view these associations on the Associations card of the Group’s legacy Details screen, under the Associated Indicators section when the card is in table view.

Importing Indicators Into an Existing Group

1. Follow Steps 1–3 in the “Importing Indicators From urlscan.io Into ThreatConnect” section and select To Existing Group from the Import dropdown. The Select Group section of the Import to Existing Group screen will be displayed (Figure 9).

   

    

   • Select the Group to which the selected Indicator(s), as well as the enriched Indicator and its corresponding Host Indicator, will be associated. To search for a Group, enter its name in the search bar above the table containing all Groups.
   • To view a Group’s Details screen , click the Open in New Tabicon to the right of the Owner column.
   • Click the Next button.
2. The Associations section will be displayed (Figure 10).

   

    

   Note

   A checkmark in the Known column indicates that the corresponding Indicator exists in at least one owner to which you have access.
   • Associations: The selected Indicator(s), as well as the enriched Indicator and its corresponding Host Indicator, will be displayed in the Associations card. In this card, you can complete the following actions:
     • Private: To mark an Indicator as private, select the corresponding checkbox in the Private column. This column will be displayed only if your System Administrator has enabled private Indicators.
     • Actions: To remove an Indicator from the list of Indicators being imported, click Delete.
   • Association Details: In the Associations Details card, you can fill out the following information, which will be applied to all Indicators being associated to the Group:
     Important

     All information added in this section will be applied to the enriched Indicator (that is, the Indicator whose Enrichment tab from which you are importing urlscan.io data), because the enriched Indicator is always added as an association to the new Group, along with the Indicators selected on the card on the URLScan Detailed View (Figure 5). If the enriched Indicator has a default Description, a Threat Rating, or a Confidence Rating and you enter a value for one of these fields, then that value will replace the existing value for the enriched Indicator. Tags entered in this section will be applied in addition to the enriched Indicator’s existing Tags.
     • Description: Enter a default Description for the Indicator(s).
     • Tags: Enter one or more Tags to apply applied to the Indicator(s).
     • Threat Rating: Use the skull icons to set the Threat Rating for the Indicator(s).
     • Confidence Rating: Use the slider to set the Confidence Rating for the Indicator(s).
   • Click the Save button.

The selected Indicator(s) will be imported into ThreatConnect and associated to the existing Group, and the Group’s Details screen will be displayed. These Indicators will be displayed on the Indicators card of the Group’s Associations tab. You may also view these associations on the Associations card of the Group’s legacy Details screen, under the Associated Indicators section when the card is in table view.

Importing Indicators as Indicators

Follow Steps 1–3 in the “Importing Indicators From urlscan.io Into ThreatConnect” section and select As an Indicator from the Import dropdown. The Import Indicators window will be displayed (Figure 11).

• New Indicators to be Imported & Associated: This section displays the Indicators that will be imported into ThreatConnect and associated directly to the enriched Indicator. To remove an Indicator from this list, click Delete.
• Click the Import Indicators button.

The selected Indicator(s) will be imported into ThreatConnect and associated to the enriched Indicator, and the Associations tab of the enriched Indicator’s Details screen will be displayed. The associated Indicators will be displayed on the Indicators card of this tab. You may also view these associations on the Associations card of the enriched Indicator’s legacy Details screen, under the Associated Indicators section when the card is in table view.

Viewing Screenshots of URLs

To view a PNG screenshot of the webpage associated with a URL Indicator, click the View Screenshot URL link on the URLScan card (Figure 2). The screenshot will be displayed in a new browser tab.

Retrieving Data Manually

When you click on a URL Indicator’s Enrichment tab for the first time, data will be retrieved from urlscan.io automatically if your System Administrator has enabled automatic data retrieval for urlscan.io. Otherwise, a message stating that “Automatic Data Retrieval has been disabled by the System Administrator” will be displayed on the card, and you will need to click the Retrieve Data button to populate the card with data. Once data have been retrieved, they will be cached for a period of time configured by your System Administrator. Each time you revisit that Indicator’s Enrichment tab, the cached urlscan.io data will be displayed until this period of time has passed.

To retrieve the latest urlscan.io data for the Indicator manually, click the Retrieve Data button on the URLScan card (Figure 2).

Enriching Indicators Using the ThreatConnect API

You can also use the ThreatConnect v3 API to enrich URL Indicators with data from urlscan.io. For instructions on using the ThreatConnect v3 API to enrich Indicators, see Indicator Enrichment Overview.

ThreatConnect^® is a registered trademark of ThreatConnect, Inc.
MITRE ATT&CK^® and ATT&CK^® are registered trademarks of The MITRE Corporation.

20146-05 v.03.A