Integration Details

ThreatQuotient provides the following details for this integration:

Introduction

ThreatQ v6 Users – while the connector’s overall operation remains the same, installation and usage commands for ThreatQ v6 users will differ from what is listed in this guide. See the Installing Custom Connectors in ThreatQ v6 and Installing Custom Connectors on Another Instance topics for additional information.

The Events Builder Connector accepts a provided payload, parses it, and creates an event object in ThreatQ with attributes and related indicators.   The events ingested into ThreatQ can include related objects such as Sources, Targets, Identities, and Assets.

The payload is defined in the command used to execute the driver.  See the Usage chapter for more information regarding the payload flag.   

You can identify your private IP Addresses in the connector's UI configuration page in ThreatQ to prevent them from being added as Indicators.   

Prerequisites

Review and confirm that you have fulfilled the connectors prerequisites before proceeding with installation/upgrading.  

Time Zone

The time zone steps are for ThreatQ v5 only.  ThreatQ v6 users should skip these steps.  

You should ensure all ThreatQ devices are set to the correct time, time zone, and date (UTC is recommended), and using a clock source available to all.

To identify which time zone is closest to your present location, use the timedatectl command with the list-timezones command line option.

For example, enter the following command to list all available time zones in Europe:

Enter the following command, as root, to change the time zone to UTC:

Integration Dependencies

The integration must be installed in a python 3.6 environment.

The following is a list of required dependencies for the integration.  These dependencies are downloaded and installed during the installation process.  If you are an Air Gapped Data Sync (AGDS) user, or run an instance that cannot connect to network services outside of your infrastructure, you will need to download and install these dependencies separately as the integration will not be able to download them during the install process.

Items listed in bold are pinned to a specific version.  In these cases, you should download the version specified to ensure proper function of the integration.

Installation

The following provides you with steps on installing a Python 3 Virtual Environment and installing the connector.

Creating a Python 3.6 Virtual Environment  

Run the following commands to create the virtual environment:

Proceed to Installing the Connector.

Installing the Connector

Upgrading Users - Review the Change Log for updates to configuration parameters before updating.  If there are changes to the configuration file (new/removed parameters), you must first delete the previous version's configuration file before proceeding with the install steps listed below.  Failure to delete the previous configuration file will result in the connector failing.

1. Navigate to the ThreatQ Marketplace and download the .whl file for the integration.
2. Activate the virtual environment if you haven't already:
   source /opt/tqvenv/<environment_name>/bin/activate
3. Transfer the whl file to the /tmp directory on your ThreatQ instance.  
4. Install the connector on your ThreatQ instance:
   pip install /tmp/tq_conn_events_builder-<version>-py3-none-any.whl 

   A driver called tq-conn-events-builder will be installed.   After installing, a script stub will appear in /opt/tqvenv/<environment_name>/bin/tq-conn-events-builder.

5. Once the application has been installed, a directory structure must be created for all configuration, logs and files, using the mkdir -p command. Use the commands below to create the required directories:
   mkdir -p /etc/tq_labs/
   mkdir -p /var/log/tq_labs/
6. Perform an initial run using the following command:
   /opt/tqvenv/<environment_name>/bin/tq-conn-events-builder -ll /var/log/tq_labs/ -c /etc/tq_labs/ -v3
7. Enter the following parameters when prompted:

   Parameter	Description
   ThreatQ Host	This is the host of the ThreatQ instance, either the IP Address or Hostname as resolvable by ThreatQ.
   ThreatQ Client ID	This is the OAuth id that can be found at Settings Gear → User Management → API details within the user’s details.
   ThreatQ Username	This is the Email Address of the user in the ThreatQ System for integrations.
   ThreatQ Password	The password for the above ThreatQ account.
   Status	This is the default status for objects that are created by this Integration.

   Example Output

   /opt/tqvenv/<environment_name>/bin/tq-conn-events-builder -ll /var/log/tq_labs/ -c /etc/tq_labs/ -v3
   ThreatQ Host: <ThreatQ Host IP or Hostname> 
   ThreatQ Client ID: <ClientID> 
   ThreatQ Username: <EMAIL ADDRESS> 
   ThreatQ Password: <PASSWORD>
   Status: Review 
   Connector configured. Set information in UI

You will still need to configure and then enable the connector.

Configuration

ThreatQuotient does not issue API keys for third-party vendors. Contact the specific vendor to obtain API keys and other integration-related credentials.

To configure the integration:

1. Navigate to your integrations management page in ThreatQ.
2. Select the Labs option from the Category dropdown (optional).
3. Click on the integration entry to open its details page.
4. Enter the following parameters under the Configuration tab:

   Parameter	Description
   Private IPs	Enter your private IPs so that IP Addresses will not be added as Indicators if they are private.  Both, a comma-delimited list and a range using a dash, formats are accepted.
   Filter Out Events	You can filter out certain events based on their attribute names.  This value should be in following format:  --- <Syslog source 1>: <tq_object>: <Attribute Name>: <Attribute Value> <Attribute Name>: <Attribute Value> <Syslog source 2>: <tq_object>: <Attribute Name>: <Attribute Value> In the example above, the <tq_object> is one of the following - attributes, adversaries, asset, attack_pattern, campaign, corporate_email, identity, incident, indicators, malware, uri_source, uri_target, or vulnerability.  The <Attribute Name> would either be exactly an attribute name, a TQ object type (as the URI type) - ie. Identity - or an Indicator Type, ie. IP Address; <Attribute Value> would simply be the value. If a TQ object has no type, <Attribute Name>: <Attribute Value> would simply be replaced by the value - see malware below below. If you wish to pass Event Time as a filtered attribute, make sure to format the date as a datetime as demonstrated below.   Additionally, the values can be comma-delimited as seen in the example below.   Example with Sources: --- McAfee EPO: uri_target: Identity: SYSTEM attributes: Source Process: java.exe malware: RAT McAfee EPO: attributes: Event Time: 2020-10-01 12:57:18 Workstation Name: 123AEP, 436asd, sdfs345
   Events Aggregation Rules by Source	List the aggregation rules by Event source. These are the aggregation rules the integration will use to decide when to create a new event and when to keep adding to an existing event. Do not include a source, if there are no aggregation rules. The format is as follows: --- <Syslog source 1>: relationships: <Object Type>,<Object Type> <Syslog source 2>: relationships: <Object Type>,<Object Type> Example: --- Fidelis Extended: relationships: URI Sources,URI Targets QRadar: relationships: URI Sources,URI Targets
   Update Event Attributes by Source	List the names of the attribute for an Event that need to be updated. Note that only the Event attributes can be updated. Do not include a source, if there is no need to update any attributes from that source. The format is as follows: --- <Syslog source 1>: attributes: <Attribute Name>,<Attribute Name>,<Attribute Name>,<Attribute Name> Syslog source 2>: attributes: <Attribute Name>,<Attribute Name>,<Attribute Name>,<Attribute Name> Example: --- Fidelis Extended: attributes: Severity,Status Code,Threat Score,Alert ID QRadar: attributes: Country Code,Threat Score,Offense ID

   
   
5. Review any additional settings, make any changes if needed, and click on Save.
6. Click on the toggle switch, located above the Additional Information section, to enable it.

Usage

Use the following command to execute the driver:

Command Line Arguments

This connector supports the following custom command line arguments:

{
    "event_type":"Sighting",
    "malware":null,
    "identity":"user@co.com",
    "uri_target":null,
    "uri_target_attributes":null,
    "event_name":"AWS User Add Detection: CreateUser from 1.2.3.4 to random.random.com",
    "event_source":"AWS User Add Detection",
    "event_attributes":"Event Domain<|random.random.com~|Target Domain<|rando-rando-1~|User-Agent<|console.random.com",
    "uri_source":"IP Address:1.2.3.4",
    "syslog_message":{
        "eventVersion":"1.08",
        "userIdentity":{
            "type":"AssumedRole",
            "principalId":"ABCDE6FGHI7JKLM:user@co.com",
            "arn":"arn:aws:sts::123456789:test-role/UserName/user@co.com",
            "accountId":"123456789",
            "accessKeyId":"ZYXWVUT",
            "sessionContext":{
                "sessionIssuer":{
                    "type":"Role",
                    "principalId":"ABCDE6FGHI7JKLM",
                    "arn":"arn:aws:iam::123456789:role/UserName",
                    "accountId":"123456789",
                    "userName":"UserName"
                },
                "webIdFederationData":{},
                "attributes":{
                    "creationDate":"2021-10-05T18:32:56Z",
                    "mfaAuthenticated":"false"
                }
            }
        },
        "eventTime":"2021-10-05T19:01:02Z",
        "eventSource":"random.random.com",
        "eventName":"CreateUser",
        "awsRegion":"rando-rando-1",
        "sourceIPAddress":"1.2.3.4",
        "userAgent":"console.random.com",
        "requestParameters":{
            "userName":"this_username",
            "tags":[
                {
                    "key":"ProductName",
                    "value":"this_username"
                },
                {
                    "key":"Owner",
                    "value":"user@co.com"
                },
                {
                    "key":"Team",
                    "value":"My_Team"
                },
                {
                    "key":"BusinessUnit",
                    "value":"IT"
                }
            ]
        },
        "responseElements":{
            "user":{
                "path":"/",
                "userName":"this_username",
                "userId":"AAABBBCCCDDDEEE1223",
                "arn":"arn:aws:iam::123456789:user/this_username",
                "createDate":"Oct 5, 2021 7:01:02 PM",
                "tags":[
                    {
                        "key":"ProductName",
                        "value":"this_username"
                    },
                    {
                        "key":"Owner",
                        "value":"user@co.com"
                    },
                    {
                        "key":"Team",
                        "value":"My_Team"
                    },
                    {
                        "key":"BusinessUnit",
                        "value":"IT"
                    }
                ]
            }
        },
        "requestID":"111222-a333444-ds555-ab667899",
        "eventID":"1a2b3c-456f7g-890j1",
        "readOnly":false,
        "eventType":"AwsApiCall",
        "managementEvent":true,
        "recipientAccountId":"123456789",
        "eventCategory":"Management",
        "sessionCredentialFromConsole":"true"
    },
    "uri_source_attributes":null,
    "event_happened_at":"2021-10-05T19:01:02Z"
}

Change Log

• Version 1.2.0
  • Added the following new configuration parameters:
    • Events Aggregation Rules by Source - specify aggregation rules for events.  
    • Update Event Attributes by Source - specify which event attributes to update.
  • Updated the logic to check if the Point of Contact for existing events is assigned every time the integration is executed.
  • Added the Event Occurrence attribute to every event.  This attribute will count the number of times an event occurs.  
• Version 1.1.0
  • Updated delimiters to allow parsing of lists.
  • Added clean up logic for emails.
  • The integration now handles cases when the syslog messages from Fidelis provides comma-delimited lists of MITRE Techniques and Names.
• Version 1.0.3
  • Added new configuration parameter, Filter Out Events, to filter out events based on certain attributes/objects.
  • Added event_happened_at as "Event Time" event attribute.
  • Added all event related-object attributes to the main event.
  • Fixed a ThreatQ Adversary upload issue.  
• Version 1.0.2
  • Initial Release