Integration Details

ThreatQuotient provides the following details for this integration:

Introduction

ThreatQ v6 Users – while the connector’s overall operation remains the same, installation and usage commands for ThreatQ v6 users will differ from what is listed in this guide. See the Installing Custom Connectors in ThreatQ v6 and Installing Custom Connectors on Another Instance topics for additional information.

The LogRhythm integration is a single install Windows based python integration between ThreatQ and the LogRhythm Platform Manager (PM).

The integration offers the following capabilities:

• Export indicators from ThreatQ to LogRhythm lists: Exports data from a data collection created via the Threat Library in ThreatQ to LogRhythm for use in correlations
• Full synchronization of ThreatQ data collections with LogRhythm lists: Functionality that exports indicators from ThreatQ data collections to LogRhythm lists and removes indicators from those lists in order to fully synchronize ThreatQ and LogRhythm
• Smart Response Plugins: Several Smart Response Plugins have been created to allow for the context of an IOC within ThreatQ to be exported to LogRhythm, and to allow LogRhythm to export context around detections back to ThreatQ

The required version of LogRhythm for this integration to work is 7.4.x. This is due to the reliance on the Admin REST API.

Prerequisites

Review the following requirements before attempting to install the integration.

Firewall Rules

Configure your firewall rules to allow the following traffic:

Creating an API User and JW token for LogRhythm

This section describes how to create an API user and obtain the JW token to use with the ThreatQ integration. Make sure you have the LogRhythm Administrator account credentials to create the new user and token. The account should be created by the LogRhythm Administrator not an admin user. Even if the user is admin, the integration won’t work. When the integration executes, LogRhythm checks who owns the created user, and if it’s not the LogRhythm Administrator account it will return error code 400 with the message: “Invalid owner id available in request body”.

1. Log into the VM where your LogRhythm instance is installed.
2. Open your LogRhythm management console.
3. Go to the Deployment Manager tab and click on People.
4. Click on New to create a dedicated ThreatQ integration account. The account should be a Role with the following permissions:

   Permission	Setting
   Read Access	Public All Users
   Write Access	Public Global Administrator
   Owner	LogRhythmAdmin

5. Enter the LogRhythm Administrator account credentials when prompted for credentials.
6. Go to the Third Party Applications tab to create the JW token (JWT).
7. Right click on the blank field and select New.
8. Enter the Application Name and Description.
9. Enter the LogRhythm Administrator account credentials when prompted for the username/password.
10. Copy the token for use in ThreatQ and hit Apply and then click OK.

    The JWT will be needed when configuring the integration in the ThreatQ UI.

Installing Python 3

Use the following steps to install Python 3.6:

1. Navigate to The Python Downloads Page - https://www.python.org/downloads/.
2. Select Download Python 3.6.X.
3. Click on the Downloaded Python MSI.
4. Select Install for all users and click Next.
5. Select the installation location.

   It is suggested that the default C:\Python36 is used.

6. Verify that Add python.exe to Path is selected and click Next.
7. Click on the Finish button after the installation is complete.
8. Once complete, you should be able to navigate to C:\Python36 and execute the python program.
9. Type exit() to exit the shell.

Integration Dependencies

The following is a list of required dependencies for the integration.  These dependencies are downloaded and installed during the installation process.  If you are an Air Gapped Data Sync (AGDS) user, or run an instance that cannot connect to network services outside of your infrastructure, you will need to download and install these dependencies separately as the integration will not be able to download them during the install process.

Items listed in bold are pinned to a specific version.  In these cases, you should download the version specified to ensure proper function of the integration.

Installation

Upgrading Users: If you are upgrading from a previous version of the integration, review the Change Log for any changes regarding configuration parameters. If there are any parameter changes listed, then you must first delete the configuration file from the pervious version before proceeding with the steps below. Failure to delete this file will result in the integration failing.

The command pip is used to install the ThreatQ Integration on the LogRhythm server.

To install this, open a command prompt Windows Key + R: cmd as an administrator.

The instructions below assume that Python 3.6 or newer has been installed on the system. If it has not, refer to the Installing Python3 section of the Prerequisites chapter.

Python must be installed in the Python3.6 directory for this to operate correctly. If another installation directory is used, replace C:\Python36 with the correct directory. A specific SRP bundle will have to be generated for this configuration. Contact support@threatq.com.

Do not continue past this point until you have confirmed that LogRhythm can reach ThreatQ.

1. Log into the VM where LogRhythm is installed.
2. Open the command line from the start menu.
3. Navigate to the Python Directory C:\Python36\Scripts.
4. Execute the following command:

   With Internet Access

   If your LogRhythm instance has access to the internet:

   pip install -i
   
   https://<USERNAME>:<PASSWORD>@extensions.threatq.com/threatq/integrations tq_conn_logrhythm

   <USERNAME> and <PASSWORD> are the username and password used to get updates from ThreatQ on the main appliance (The ThreatQ Credentials entered during setup).

   This will install several command line tools.

   Without Internet Access

   Download the installable with its dependencies on an instance with access to the internet, transfer all the files to LogRhythm, and run pip install:

   mkdir /tmp/logrhythm
   
   pip download tq_conn_logrhythm -d /tmp/logrhythm

   Transfer the tq_conn_logrhythm-<version>-py3-none-any.whl, and its dependencies to the Downloads folder on the LogRhythm instance.

   pip install C:\Downloads\tq_conn_logrhythm-<version>-py3-none-any.whl --no-index --find-links C:\Downloads
5. Optional - Add C:\Python36 to your Windows Path. This will allow you to execute commands without having to specify the directory.
6. Create new folders in your Log Files disk to store the integration’s logs and config. For example:
   
   Config: L:\ThreatQ\config
   Logs: L:\ThreatQ\logs
7. Check if there is a route to ThreatQ from LogRhythm.
   C:\ping <ThreatQ Host>
8. Execute the following command. If you are using different paths for the config and logs folder, please change the respective values below:
   C:\Python36\Scripts\tq-conn-logrhythm.exe -c L:\ThreatQ\config\ -ll L:\ThreatQ\logs\ -v3
9. Complete the following fields:

   Parameter	Description
   ThreatQ Host	This is the hostname or IP Address of the ThreatQ appliance
   Client ID	This is the OAuth Client ID, found by going to Gear → OAuth Credentials in the ThreatQ Appliance
   Email Address	The email address of the LogRhythm ThreatQ User
   Password	The password of the LogRhythm ThreatQ User
   Status	The status of newly created IOCs. It is recommended to select the Active status

   Once complete, the log and config paths, and the verbosity level will be stored in HKEY_LOCAL_MACHINE\Software\ThreatQuotient\LogRhythm and will be available to this system.

   After the integration is installed on the LogRhythm Platform Manager, the user must configure the integration in the ThreatQ UI.

Configuration

ThreatQuotient does not issue API keys for third-party vendors. Contact the specific vendor to obtain API keys and other integration-related credentials.

To configure the integration:

1. Navigate to your integrations management page in ThreatQ.
2. Select the Labs option from the Category dropdown (optional).
3. Click on the integration entry to open its details page.
4. Enter the following parameters under the Configuration tab:

   Parameter	Description
   LogRhythm Host	The Hostname of IP Address of the LogRhythm Platform Manager
   LogRhythm API Port	This is port 8501 by default and should not have to change. This is the port of the REST API not the SOAP API
   Check this box if LogRhythm is version 7.5.1 or higher	Due to a change in the LogRhythm API, this box needs to be checked if your version is 7.5.1 or above
   Name of the LogRhythm user owning the Threat Lists in LogRhythm NOT the username, but the name of the user as it appears in LogRhythm	Name of the LogRhythm user who will own the threat list. Use the name for the dedicated ThreatQ integration user as described in the Prerequisites section. The name can be found by navigating to LogRhythm Console -> Deployment Manager -> People. Enter the name as see in the first column, NOT the username.
   JW Token	This is the token that was generated during the creation of the LogRhythm API User as described in the Prerequisites section.
   A ThreatQ data collection with IOCs to add to LogRhythm	This is the mapping of data collections to LogRhythm lists, explained in detail in the ThreatQ Data Collection section of this document
   Select the sync method between ThreatQ and LogRhythm	Add indicators from ThreatQ data collections to LogRhythm lists Fully sync (add and delete) ThreatQ data collections with LogRhythm lists
   Maximum number of LogRhythm indicators to compare against during the sync	The maximum number of indicators is used for managing compute resources on LogRhythm. Changing this value will change the maximum memory and CPU utilization of the LogRhythm host. The. default value is 1000000.

5. Review any additional settings, make any changes if needed, and click on Save.
6. Click on the toggle switch, located above the Additional Information section, to enable it.

ThreatQ Data Collection

This section explains how to configure a Data Collection in ThreatQ. Go to the ThreatQ Library in ThreatQ and enter the search parameters for the indicators you would like to send to LogRhythm. The IOCs should be of the following types – IP Address, FQDN, MD5, SHA-1, and SHA-256.

This is an example of a data collection called “LogRhythm-IOCList” that includes IP Address, FQDN, SHA-256 and MD5 created before 06/01/2020 at 2:20pm:

Once the search is configured, from the top menu navigate to Integrations -> Feeds & Connectors and click on LogRhythm. Open the card for LogRhythm and enable it by moving the slider to the right.

Fill out the required information for host, LogRhythm user who will own the lists, and the JW Token.

Select the required synchronization method between ThreatQ and LogRhythm. Selecting Add Indicators from ThreatQ data collections to LogRhythm lists is additive and will result in only exporting indicators from ThreatQ data collections to LogRhythm lists. Each consecutive run of the integration will add more indicators to the LogRhythm lists.

Selecting Fully sync (add and delete) ThreatQ data collections with LogRhythm lists does a complete synchronization between ThreatQ data collection and LogRhythm lists. The synchronization executes two actions. First, all the indicators found in the ThreatQ data collection, but not in LogRhythm are added to LogRhythm. Next, the indicators that are in the LogRhythm lists but not found in the ThreatQ data collection are deleted from LogRhythm.

Each list can only have up to 1 Million IOCs in it. Anymore than this and the system will be unable to synchronize correctly.

In the Data Collection box, enter the name of the data collection search followed by the name of the threat intel list you would like to create in LogRhythm similar to the syntax below, including the three dashes on the first line:

In the line above, “LogRhythm-IOCList” is the name of the Data Collection in ThreatQ. The phrase "ThreatQ-all: " is the basename of a family of lists in LogRhythm. All of the lists will have the prefix “ThreatQ-all:” and then have the specific indicator type. For instance, FQDNs in the above list will be put into the ThreatQ: FQDN list, IP Addresses will be in the ThreatQ: IP Address list, and so on.

The splitting of IoCs into their specific indicator types is a best practice for LogRhythm and is enforced in this integration.

Further splitting of lists is allowed by specifying a splitBy parameter, as below:

This will have a similar affect as above, except that it will create one more level of list. For instance, FQDNs in the above data collection, “LogRhythm-IOCList”, will be saved by score range in the following lists:

• ThreatQ: FQDN: Very High
• ThreatQ: FQDN: High
• ThreatQ: FQDN: Medium
• ThreatQ: FQDN: Low
• ThreatQ: FQDN: Very Low
• ThreatQ: FQDN: Not Scored

The above lists will only be created if an FQDN of that level is in the data collection. For instance, if no Medium FQDNs are in the data collection, there will be no ThreatQ: FQDN: Medium list. At any point in the future, these will be added as necessary.

The following are the currently available splitBy parameters:

Recurring Execution

One of the commands provided during the installation is the tq-conn-logrhythm.exe command. This command is what is used to read the configuration set in the UI of the ThreatQ appliance, parse the indicators from a data collection in ThreatQ and upload those indicators to the LogRhythm lists.

During the execution of tq-conn-logrhythm.exe, the UI fields are read, lists are created, and all lists are compared against the content of the data collections in the ThreatQ’s Threat Library.

Each Indicator of Compromise listed in the data collection is compared to the contents of the associated list. If the IOC is not in the list, it is added during the synchronization step.

When each IOC has been compared to the list, those IOCs which are no longer in the data collection, but are in the list in LogRhythm, are tagged for deletion during the synchronization step. Finally, each list is synchronized.

This must be setup on a recurring basis to keep the data in the lists synchronized to the data in the data collections. To do this, use the Task Scheduler, and configure it as below:

The frequency of synchronization between ThreatQ and LogRhythm is at the discretion of the user but it should be no more than once an hour.

Other schedulers can be used, the syntax of the command above should be used in whatever scheduler is provided.

Smart Response Plugin

LogRhythm allows for tools to be added to the platform to enrich data within LogRhythm. Using these tools, IOCs can also be enriched within ThreatQ. The following SRPs are provided with the installation. All of these can be found in the C:\Python36\Scripts directory and can also be executed via the CLI on the LogRhythm host.

Installation of SRPs

The SRP executables are installed during the original installation of the integration. However, to enable these within LogRhythm to be used in the Client or Web Consoles, the SRP bundle has to be added. The bundle is presented as an all in one.

To install the SRP bundle:

1. Login to the LogRhythm console
2. Open the Deployment Manager

   

3. Go to Tools → Administration → SmartResponse Plugin Manager

   

4. Click on the Actions button

   

5. Navigate to the ThreatQ SRP bundle and click Open.
   
   Once complete, the ThreatQ Smart Response Plugins list found in Smart Response Plugins.
   
   

Using a Smart Response Plugin

There are several ways in which a Smart Response Plugin can be used. The first is to con- figure it as part of an AI Engine Rule. This will not be covered in this documentation as it is covered in depth in the LogRhythm Client Console Reference in the LogRhythm Administration → AI Engine Section. This section will focus on using the SRPs in the Web Console.

1. Log into the Web Console.

   

2. Open up the "Logs" pane and Navigate to a log entry that is interesting.
3. Open the "Inspector" pane if it is not already open.
4. In the Inspector Pane, Navigate to "Smart Response."
   1. Choose "Plugin: ThreatQ"
   2. Choose the appropriate Action (in this example it will be the ThreatQ Add IOC Action).
5. Fill in the appropriate Values (here an IP address is used). Depending on the action executed, different values would need to be entered in the UI
6. Ensure "Execute from: Platform Manager" is selected and click Run.
   
   
   At this point LogRhythm will execute action in a separate window, retuning the results.
   
   
   
   If a case is currently selected, this can be added to the case by clicking the Add to Case button. If it is added to a case, it will appear in that case as Evidence.

The ThreatQ Add Sighting action is accessible through this interface but is not usable here. This action is to be used only in the Client Console when configuring AI Engine Rules. This is used to bring over a Case as a "Sighting" type Event in ThreatQ, including the observed IOCs.

Using the ThreatQ Add Sighting

Tracking the sighting of Indicators of Compromise is an integral piece in the Threat Intelligence feedback loop. This feedback loop alerts Threat Analysts to existence of known Indicators of Compromise that have been observed in LogRhythm. Unlike the other SRPs provided above, this SRP is designed to solely be used as part of an AI Engine rule. When this rule is triggered, it creates an Event within ThreatQ with a type of "Sighting". These sightings have the AlarmID, Alarm Name, and other pieces of information configurable in the AI Engine in the LogRhythm Client Console.

To provide the most flexible solution to capture all possible Sightings, the ThreatQ Add Sighting SRP has options for all known IOC fields.

To add a rule that uses this SRP:

1. Open Deployment Manager and go to the "AI Engine" tab.
2. Click on the Green "+" Button in the toolbar or navigate to the rule you wish to edit, select it, right click and select edit.
3. Configure the rule as needed.  When complete select the actions tab.
   
   

   Only the fields listed in the "Group By" section will be available in the Actions Menu.

4. On the Actions screen, use the Set Action drop down and select the ThreatQ: ThreatQ Add Sighting Action.
5. Make sure the fields that are present in the Group by have the correct value. For instance, the Destination IP Address should have <IP Address (Impacted)> selected.
6. For the Alarm Date value, modify the Time Format to yyyy-MM-dd HH:mm:ss
7. For all other fields, change the Type from Alarm Field to Constant Value. Click the Save Action button.
8. You will be prompted to restart the AI Engine.
   
   Once completed, when that Rule is executed, a sighting should be added that looks similar to the following:
   
   
   
   Currently, this only records the actual IOCs associated with the sighting itself. In the future, more attributes and details will be added.‎

Setup Alarm Rules

LogRhythm Lists are the primary interface for which Cyber Threat Intelligence data is stored and used in LogRhythm. These lists can come from many different sources, and can store many different data types. For the purpose of ThreatQ, lists will be dynamically generated at execution time based on configurations given on the My Integrations page of ThreatQ.

These lists can be used in conjunction with the LogRhythm AI Engine to determine correlations against incoming log data.

To configure the list to be used in the AI, do the following:

1. Open Deployment Manager and go to the AI Engine tab.
2. Click on the green + button in the toolbar.

   

3. When the AI Engine Rule Wizard opens, select the Observed block on the Log Section.

   

4. Configure it as needed for detection of Cyber Threat Events. In the example given here, the Host (Origin or Impacted) Field must be in the ThreatQ: FQDN list. The list names and fields will vary based on the purpose of the rule.

   

5. Go through the rest of the Wizard and select the required settings based on your Incident Response Manifesto.
   
   Once saved and the AI Engine Service restarted, any Log that has a host field with an Origin or Impacted Host in the ThreatQ: FQDN list will have the associated log alerted on.
   
   For more information on the AI Engine and its possibilities, please consult the LogRhythm Client Console Reference Guide in the LogRhythm Administration → AI Engine Section.

Testing the Integration

Use the following sections to test the integration.  

Prerequisites

• LogRhythm is installed
• RDP is enabled for LogRhythm
• ThreatQ integration is installed
• Custom connector
• SRP (Smart Response Plugin) Actions

Testing Custom Connector (Sync)

This part of the integration downloads a ThreatQ data collection and uploads the intelligence to lists in LogRhythm. Here is how to test the integration.

1. RDP into the LogRhythm server
2. Install the custom connector. If you don’t yet have, please see the Installation section
3. Run the custom connector for the first time to install it into ThreatQ
   1. If it's already installed, the custom connector connection settings (paths to the config and log folders and the verbosity level) are stored in the Windows Registry.
   2. To reset the connected ThreatQ host, you will need to remove the registry keys. See step 5 in the Uninstallation section.
4. Configure the connector using this guide: see the Configuration section.
5. Run the connector using this command
6. Here is a quick checklist on what to look out for:
   1. The connector runs with no errors.
   2. The connector creates LogRhythm lists to store the indicators (if it hasn't already).

      The lists created will be determined by your YAML configuration.

   3. Make sure each list contains the same number of indicators as in the data collection

Testing the SRP Actions (Contextual Actions)

This part of the integration allows you to execute specific actions on alarms or individual indicators within the LogRhythm platform. They can also be used in the "AI Engine" to perform automatic actions when an alarm is triggered. This is usually the case for the "ThreatQ: Add Sighting" Action. Since logs are slightly complicated to get into the LogRhythm platform, we can test the actual script directly (as if LogRhythm was calling it).

1. RDP into the LogRhythm server.
2. Install the custom connector. If you don’t yet have, please see the Installation section
3. Run the custom connector for the first time to authenticate and install it into ThreatQ.
4. Test each of the SRP actions by running the action command (with the required parameters). Here are some examples (replace <> text):

Add Sighting to ThreatQ

Add IOC to ThreatQ

Lookup IOC from ThreatQ

Mark IOC as False Positive in ThreatQ

‎Mark IOC as True Positive in ThreatQ

Change the Status of IOC to “Whitelist” in ThreatQ

Uninstall Integration

If it is decided that the integration needs to be removed, there are three main steps to uninstalling the integration. Due to audit requirements, however, it is not possible to delete the created Lists. Lists can only be disabled.

1. Stop the Windows Job for synchronization.
2. Deactivate all of the Lists created by ThreatQ.
   1. Open the List Manager.
   2. Check All Lists created by ThreatQ.
      
   3. Right Click → Actions → Retire.
   4. Confirm Retirement.
3. Uninstall the SRPs.
   1. Open the Deployment Manager.
   2. Navigate to Tools → Administration → SmartResponse Plugin Manager.
   3. Select the ThreatQ SRP and click Actions.
   4. Click Retire.
4. Uninstall the integration.
   1. Open an Administrative cmd session.
   2. Navigate to C:\Python36\Scripts.
   3. Execute pip.exe uninstall tq-conn-logrhythm.
   4. Confirm removal.
5. Clean up Windows Registry.
   1. Click Windows Key + R and type regedit.
   2. Navigate to HKEY_LOCAL_MACHINE → SOFTWARE.
   3. Delete the ThreatQuotient key and all sub keys.

Change Log

• Version 1.5.2
  • Updated ruamel.yaml dependency requirement to < 0.18.5.
  • Updated the minimum ThreatQ version to 5.6.0.  
• Version 1.5.1 rev-b (Guide Update)
  • Updated ThreatQSDK and ThreatQCC dependencies to threatqsdk1.8.6 and threatqcc1.4.2.  
• Version 1.5.1 rev-a (Guide Update)
  • Updated Python3-related steps and examples.
  • Updated Prerequisites chapter:
    • Removed section: Creating an API User for LogRhythm.
    • Add new section: Creating an API User and JW token for LogRhythm.
• Version 1.5.1
  • Added six package to integration dependencies for the python 3.6 version.  
• Version 1.5.0
  • Added the capability to delete indicators from LogRhyhtm lists.
  • Added a radio button to the config UI in ThreatQ for users to select the synchronization method - only add IOCs to LogRhythm or full synchronization (i.e. add and delete IOCs to/from LogRhythm lists). 
• Version 1.4.0
  • Added backward compatibility for the integration with LogRhythm versions prior to version 7.5.1.
• Version 1.3.0
  • Updated the integration for LogRhythm v7.6.
  • Added a parameter to the ThreatQ UI Configuration for the LogRhythm user that will own the threat lists
• Version 1.2.0
  • Released a version for Python3.
  • Migrated the integration to use the latest Threat Library in the ThreatQ SDK
  • Updated the examples in guide.
• Version 1.1.1
  • Changed the search from the Advanced Search to the Threat Library.
  • Updated the Smart Response Plugin
• Version 1.0.1
  • Initial Release