Parsing for Indicators

ThreatQ gives you the option to import a file, parse it for indicators, and add those indicators to your Threat Library.  During the import process you can assign a source, tag, and a relationship to the imported indicators.

See the Importing Indicators via CSV topic for specific instructions and examples on parsing indicators from a .csv file.  

Indicator Parsers and Permissions

In addition to permission to Perform Bulk Manual Import, users with custom roles must have permissions for the indicator context included in the files they parse. For example, if you use the ThreatQ CSV file parser to parse a file that includes indicator statuses, your role must have Perform Bulk Manual Import permission and Edit/Delete permissions for Status.

These permission requirements apply to all indicator parsers except ThreatQ’s Generic Text/PDF parser. For this parser, ThreatQuotient requires that the parsed file only includes the indicator and its value. As such, no additional object context permissions are required.

Selecting a File to Parse

1. Click the Create button, located at the top-right of the menu bar, and select the Indicator Parser option. 

   You can also click on Create > Indicator and then select the Parse for Indicators option at the top of the Add Indicators dialog box.

   The Add Indicators dialog box will open.
   
   

2.  Select the file to upload by either:
   • Dragging and dropping the file into the dialog window
   • Clicking on the Click to Browse option and locating the file on your local device
   • Copying and pasting the file's contents in the text field provided.  
3. Select the type of parser to use.  Options include
   • Cuckoo
   • Trellix Analysis
   • Generic Text / PDF - These files must contain only Indicators and their values.
   • Palo Alto Networks WildFire XML
   • ThreatAnalyzer Analysis
   • ThreatQ CSV File - see the Importing Indicators via CSV topic for specific instructions on using this parser.
4. Use the checkboxes to select your parsing options:

   Option	Description
   Normalize URL Indicators	When checked, parsed URLs will have ports and leading protocol adjusted, as well as unneeded quotes and spaces removed.  Normalization also adds attributes for protocol and query string.   See the Indicator URL Normalization topic for more details.
   Parse FQDNs	When checked, the Indicator Parser will parse FQDNs from the text and derive FQDN indicators from URLs in the text. Example (checked): URL: https://tqexample.com/table.jspa?query_string_example Indicators created: tqexample.com/table.jspa (the URL) tqexample.com (the derived FQDN from the URL) When unchecked, the Indicator Parser will not generate FQDN indicators from the parsed text. Example (unchecked): URL: https://tqexample.com/table.jspa?query_string_example Indicator created: tqexample.com/table.jspa (the URL)

   Administrators can configure the default setting for these options under the General Tab on the System Configurations page. See the Indicator Parsing Presets topic for more details.

5. Click on Next Step.
   
   The Step 1 - Import Indicators form will load.  
   
   

Step 1 - Import Indicators Settings

6. Select whether or not to save the import file.  Saving the file will result in all extracted indicators being linked to the file for reference.  If you select Yes, review the File Title and File Category.  You can also add an option File Description.
   
   
7. Select a Source for the extracted indicators.
   You can also click the Add a New Source option if the desired source is not listed in the dropdown list . If administrators have enabled TLP view settings, you can select a TLP label for the new source in the dropdown list provided. See the Traffic Light Protocol (TLP) topic for more information on TLP schema.
   
8. Select a Status for the extracted indicators.   
   
   
9. Enter any Tags to apply to the extracted indicators.  This field is optional.  
10. Select any attribute, attribute value, and attribute source to apply to the extracted indicators.
11. Add Relationships for the extracted indicators.

    If you enter an object name that is not found, you can click the Create link to add the new object. If you limited your search to a specific object type, you are linked to the corresponding form. For example, if you limit your search to Adversaries, the Create link opens the Add An Adversary form. If you leave the Limit search to field set to All Objects, you can select the object type you want to create from a drop-down list.

12. Click on Next Step.
    
    The Organize and Classify form will load.  
    
    

Step 2 - Organize and Classify

13. You can review the original content of the file and the extracted indicators' information. 
    
    Filtering Extracted Indicators List
    • The top tabs allow you to filter the list of indicators by New and Pre-existing.  This allows you to isolate any indicators that already exist in the platform.
      
      
      
      Pre-existing indicators will also be marked with a Pre-existing label in the list.  You can click this label to view the preview panel for the object.
      
      
    • You can click on the Select dropdown to automatically select indicators by sub-type.
      
      

      Selecting All will select all extracted indicators, not just the ones in your current filtered view (New, Pre-Existing).

    Adding Indicators
    • You can click on the Add Indicator option, located to the top-right of the list, to add an indicator to the extracted list.  You can add further context to the new indicator using the Editing Extracted Indicators actions listed below.  
      
      
    Editing Extracted Indicators
    • Clicking on the  icon will show you where the indicator appeared in the Original Content window. 
    • Clicking on the icon will open the Edit Indicator dialog box and allow you to edit the indicator value and indicator sub-type. 
      
      
    • Selecting one or more indicators and clicking on Add Info option allows you to perform the following actions:

      Action	Details
      Add Attribute	You can add an attribute to one or more extracted indicators.  Select the checkbox next to the indicator(s) to update and then click on the Add Info option.  The Add Attributes tab will be selected by default.  Select an Attribute Name, Value, and Source to apply to the selected indicator(s).
      Set/Update  Status	You can update the status of one or more extracted indicators.  Select the checkbox next to the indicator(s) to update and then click on the Add Info option.  Click on the Set Status tab and select your new status.
      Create Relationship	You can link one or more extracted indicators to another system object.  Select the checkbox next to the indicator(s) to update and then click on the Add Info option.  Click on the Set Relationship tab and set the relationship. When you add a relationship, it is displayed in the indicator list and you can click it to view its details in a preview panel. If the object you want to link is not found, you can click the Create link to add the new object. If you limited your search to a specific object type, you are linked to the corresponding form. For example, if you limited your search to Adversaries, the Create link opens the Add An Adversary form. If you leave the Limit search to field set to All Objects, you can select the object type you want to create from a drop-down list. In the Add form, the indicators you selected in the second step of the import process are listed in the Create Relationship section.
      Add Comment	You can add a comment to one or more extracted indicators.  Select the checkbox next to the indicator(s) to update and then click on the Add Info option.  Click on the Set Comment tab and enter your comment.

      
      
    Removing Extracted Indicators
    • You can delete one or more extracted indicators.  Select the checkbox next to the indicator(s) to delete and then click on the Remove icon.

      Selecting All from the Select dropdown will select all extracted indicators, not just the ones in your current filtered view (New, Pre-Existing).

14. When finished editing the extracted indicators list, click Finish Export to complete the process.