Intelligence Mailbox Reports Connector
The web format of this guide reflects the most current release. Guides for older iterations are available in PDF format.
Integration Details
ThreatQuotient provides the following details for this integration:
Current Integration Version | 1.0.0 |
Compatible with ThreatQ Versions | >= 4.34.0 |
Python Version | 3.6 |
Support Tier | ThreatQ Supported |
Introduction
ThreatQ v6 Users – while the connector’s overall operation remains the same, installation and usage commands for ThreatQ v6 users will differ from what is listed in this guide. See the Installing Custom Connectors in ThreatQ v6 and Installing Custom Connectors on Another Instance topics for additional information.
The Intelligence Mailbox Reports Connector for ThreatQ enables analysts to use an email inbox for spearphish submissions as well as intelligence sharing. The email itself will be added as a File in ThreatQ, with its articles added as related Reports and all IoCs or other attachments found in the email related correspondingly.
This integration is based on the IMAP Connector for ThreatQ integration with the following updated processes:
- Modifies to upload a report (with corresponding attributes) to TQ for each article in the email instead of one event for the whole email.
- Adds TLP as attributes.
- Adds the entire email as a related Attachment.
- Relates IoCs in articles to article Reports.
- Removes forwarded email spearphish functionality.
Prerequisites
Review the following requirements before attempting to install the connector.
Time Zone
The time zone steps are for ThreatQ v5 only. ThreatQ v6 users should skip these steps.
You should ensure all ThreatQ devices are set to the correct time, time zone, and date (UTC is recommended), and using a clock source available to all.
To identify which time zone is closest to your present location, use the timedatectl
command with the list-timezones
command line option.
For example, enter the following command to list all available time zones in Europe:
Europe/Amsterdam
Europe/Athens
Europe/Belgrade
Europe/Berlin
Enter the following command, as root, to change the time zone to UTC:
Integration Dependencies
The integration must be installed in a python 3.6 environment.
The following is a list of required dependencies for the integration. These dependencies are downloaded and installed during the installation process. If you are an Air Gapped Data Sync (AGDS) user, or run an instance that cannot connect to network services outside of your infrastructure, you will need to download and install these dependencies separately as the integration will not be able to download them during the install process.
Items listed in bold are pinned to a specific version. In these cases, you should download the version specified to ensure proper function of the integration.
Dependency | Version | Notes |
---|---|---|
threatqsdk | >=1.8.2 | N/A |
threatqcc | >=1.4.1 | N/A |
PySocks | >=1.6.7 | N/A |
beautifulsoup4 | 4.9.3 | Pinned |
Installation
The following provides you with steps on installing a Python 3 Virtual Environment and installing the connector.
Creating a Python 3.6 Virtual Environment
Run the following commands to create the virtual environment:
sudo yum install -y python36 python36-libs python36-devel python36-pip
python3.6 -m venv /opt/tqvenv/<environment_name>
source /opt/tqvenv/<environment_name>/bin/activate
pip install --upgrade pip
pip install threatqsdk threatqcc setuptools==59.6.0
Proceed to Installing the Connector.
Installing the Connector
Upgrading Users - Review the Change Log for updates to configuration parameters before updating. If there are changes to the configuration file (new/removed parameters), you must first delete the previous version's configuration file before proceeding with the install steps listed below. Failure to delete the previous configuration file will result in the connector failing.
- Navigate to the ThreatQ Marketplace and download the .whl file for the integration.
- Activate the virtual environment if you haven't already:
source /opt/tqvenv/<environment_name>/bin/activate
- Transfer the whl file to the
/tmp
directory on your ThreatQ instance. - Install the connector on your ThreatQ instance:
pip install /tmp/tq_conn_intel_mailbox_reports-<version>-py3-none-any.whl
A driver called
tq-conn-intel-mailbox-reports
will be installed. After installing, a script stub will appear in/opt/tqvenv/<environment_name>/bin/tq-conn-intel-mailbox-reports
. - Once the application has been installed, a directory structure must be created for all configuration, logs and files, using the
mkdir -p
command. Use the commands below to create the required directories:mkdir -p /etc/tq_labs/
mkdir -p /var/log/tq_labs/ - Perform an initial run using the following command:
/opt/tqvenv/<environment_name>/bin/tq-conn-intel-mailbox-reports -ll /var/log/tq_labs/ -c /etc/tq_labs/ -v3
- Enter the following parameters when prompted:
Parameter Description ThreatQ Host This is the host of the ThreatQ instance, either the IP Address or Hostname as resolvable by ThreatQ. ThreatQ Client ID This is the OAuth id that can be found at Settings Gear → User Management → API details within the user’s details. ThreatQ Username This is the Email Address of the user in the ThreatQ System for integrations. ThreatQ Password The password for the above ThreatQ account. Status This is the default status for objects that are created by this Integration. Example Output
/opt/tqvenv/<environment_name>/bin/tq-conn-intel-mailbox-reports -ll /var/log/tq_labs/ -c /etc/tq_labs/ -v3 ThreatQ Host: <ThreatQ Host IP or Hostname> ThreatQ Client ID: <ClientID> ThreatQ Username: <EMAIL ADDRESS> ThreatQ Password: <PASSWORD> Status: Review Connector configured. Set information in UI
You will still need to configure and then enable the connector.
Configuration
ThreatQuotient does not issue API keys for third-party vendors. Contact the specific vendor to obtain API keys and other integration-related credentials.
To configure the integration:
- Navigate to your integrations management page in ThreatQ.
- Select the Labs option from the Category dropdown (optional).
- Click on the integration entry to open its details page.
- Enter the following parameters under the Configuration tab:
Parameter Description Intelligence Mailbox Server The Intelligence Mailbox server associated with the email provider you want to connect. Intelligence Mailbox Username The username/email to log into. Intelligence Mailbox Password The password associated with the account above. Proxy Address Optional - Socks proxy address used to connect to the Intelligence Mailbox server. This cannot be a HTTP/HTTPS proxy, that is why the proxy settings within ThreatQ will not be used. Proxy Port Optional - The port associated with the proxy address field. Proxy Port Optional - The type of proxy that will be used. The options are socks4 and socks5.
- Review any additional settings, make any changes if needed, and click on Save.
- Click on the toggle switch, located above the Additional Information section, to enable it.
Usage
Use the following command to execute the driver:
Command Line Arguments
This connector supports the following custom command line arguments:
Argument | Description |
---|---|
-h , --help |
Review all additional options and their descriptions. |
-ll LOGLOCATION , --loglocation LOGLOCATION |
Sets the logging location for the connector. The location should exist and be writable by the current. A special value of 'stdout' means to log to the console (this happens by default). |
-c CONFIG , --config CONFIG |
This is the location of the configuration file for the connector. This location must be readable and writable by the current user. If no config file path is given, the current directory will be used. This file is also where some information from each run of the connector may be put (last run time, private oauth, etc.) |
-v {1,2,3} , --verbosity {1,2,3} |
This is the logging verbosity level where 3 means everything. |
-n, --name |
Optional - Name of the connector (Option used in order to allow users to configure multiple Intelligence Mailbox connector instances on the same TQ box). |
|
This enables a proxy to be used to contact the internet for the data required by this connector. This specifies an internet facing proxy, NOT a proxy to the TQ instance. |
-d, --no-differential |
If exports are used in this connector, this will turn 'off' the differential flag for the execution. This allows debugging and testing to be done on export endpoints without having to rebuild the exports after the test. THIS SHOULD NEVER BE USED IN PRODUCTION. |
-f FILES, --files FILES |
Directory where downloaded files are kept. |
-ds, --disable_ssl |
Enables the ability to specify port 143 instead of 993. |
CRON
Automatic CRON configuration has been removed from this script. To run this script on a recurring basis, use CRON or some other jobs scheduler. The argument in the CRON script must specify the config and log locations.
Add an entry to your Linux crontab to execute the connector at a recurring interval. Depending on how quickly you need updates, this can be run multiple times a day (no more than once an hour) or a few times a week.
In the example below, the command will execute the connector every two hours.
- Log into your ThreatQ host via a CLI terminal session.
- Enter the following command:
crontab -e
This will enable the editing of the crontab, using vi. Depending on how often you wish the cronjob to run, you will need to adjust the time to suit the environment.
- Enter the commands below:
Every 2 Hours Example
0 */2 * * * /opt/tqvenv/<environment_name>/bin/tq-conn-intel-mailbox-reports -c /etc/tq_labs/ -ll /var/log/tq_labs/ -v3 - Save and exit CRON.
Change Log
- Version 1.0.0
- Initial release
PDF Guides
Document | ThreatQ Version |
---|---|
Intelligence Mailbox Reports Connector Guide v1.0.0 | 4.34.0 or Greater |