Indicator Expiration Policies

Current ThreatQ Version Filter

Indicator Expiration Policies

THREATQ REQUIRED PERMISSIONS

Default ThreatQ Role: Administrative, Maintenance, or Primary Contributor
Custom Role - Action Permissions: Data Controls - Edit Indicator Expiration

Automatic expiration allows you to deprecate stale intelligence based on a set of defined criteria. As the data becomes less relevant, ThreatQ sets the status to Expired, which relieves the data burden on your team or infrastructure.

Accessing the Indicator Expiration Page

From the navigation menu, click on Threat Library and select Indicator Expiration under the Data Controls heading.
The Data Controls page displays with the Indicator Expiration tab selected by default.

How ThreatQ Calculates Expiration Dates

Scenario	Description
Indicator Reported by Source with an Expiration Policy	If an indicator has an expiration date and it's reported by a new source that has an expiration policy, ThreatQ will set the expiration date using the policy with the greater expiration date.
Indicator Report by a Source with an Expiration Policy of Never Expire	If an indicator has an expiration date and it's reported by a new source that has an expiration policy of Never Expire, ThreatQ sets that indicator to Never Expire.
Indicator Reported by a Source with an Exception for that Indicator	If an indicator is reported by a source that has an exception for the indicator, the exception expiration date will be used regardless of the greater expiration date. An exception takes precedence over the source’s expire policy.
Indicator Reported by Two Different Sources	If an indicator is reported by a source with an Expiration Policy and then reported by a second source with another Expiration Policy, the greatest expiration date is selected to set the expiration date. The expiration date will be set based on the date the second source reported the indicator.
Indicator Reported by Two Different Sources, one with an Exception	If an indicator is reported by a source that has an exception for the indicator and then reported by a second source, the greatest expiration date is selected despite the exception. The expiration date will be set based on the date the second source reported the indicator.

Selecting an Expiration Policy per Feed

You can choose from three options when configuring an expiration policy for a source of intelligence:

Option	Description
Don't automatically expire (No policy set)	ThreatQ sets all feeds to Don't Automatically Expire until an analyst decides otherwise. When set, indicators reported from this specific feed do not have an expiration date automatically applied to them. If an indicator is reported by Source A (an intelligence feed without an expiration policy), and is later reported by Source B (an intelligence feed that expires data in 7 days), ThreatQ sets the indicators to automatically expire in 7 days.
Automatically Expire Indicators	When setting a specific intelligence feed to Automatically Expire Indicators, ThreatQ requires you to provide a specific number of days. After you configure this setting, it applies to all intelligence currently in the system, as well as new intelligence as it is ingested. ThreatQ calculates the appropriate expiration date based on the number of days from ingestion. Once an indicator's expiration date is met, its status changes to Expired.
Never Expire	Using this setting ensures that all intelligence reported by a specific feed is protected from automatic expiration, regardless of scenario.

Option

Description

Don't automatically expire (No policy set)

ThreatQ sets all feeds to Don't Automatically Expire until an analyst decides otherwise. When set, indicators reported from this specific feed do not have an expiration date automatically applied to them.

If an indicator is reported by Source A (an intelligence feed without an expiration policy), and is later reported by Source B (an intelligence feed that expires data in 7 days), ThreatQ sets the indicators to automatically expire in 7 days.

Automatically Expire Indicators

When setting a specific intelligence feed to Automatically Expire Indicators, ThreatQ requires you to provide a specific number of days. After you configure this setting, it applies to all intelligence currently in the system, as well as new intelligence as it is ingested. ThreatQ calculates the appropriate expiration date based on the number of days from ingestion. Once an indicator's expiration date is met, its status changes to Expired.

Automatic Expiration Exception Example

Never Expire

Using this setting ensures that all intelligence reported by a specific feed is protected from automatic expiration, regardless of scenario.

Adding Exceptions

ThreatQ allows you to add exceptions based on specific indicator types within a feed in addition to setting an expiration policy at a global level for all intelligence ingested by a specific feed.

From the navigation menu, click on Threat Library and select Indicator Expiration under the Data Controls heading.
Locate the source.
Click Exceptions to expand the option.
The Exceptions option menu opens.

The number of existing exceptions for a source will be listed next to its Exceptions link.
Click Add Exception.
Select the Indicator Type from the dropdown.
Enter the number of days after the item has been ingested before expiring.
Repeat steps 4-6 to add multiple
Click on Delete next to the row to delete an exception.
Click on Save.

Applying Expiration Policy Changes to Data

When updating an expiration policy, the system now applies the update to all selected existing data in the platform to honor the new policy. This process can take a while based on system resources and the number of indicators in the system.

Refer to the following table for estimates on the total time required for the system to apply the selected policy to existing data, based on the following criteria:

Dataset: 6 Million Indicators
System Specifications: 32GB VM 4 vCPU

Indicators to reset expiration out of 6m total indicators	Reset and Recalculate Expiration	Expire Indicators	Total Time for Reset
50,000	3 hours and 30 minutes	53 seconds	3 hours 31 minutes
100,000	4 hours and 51 minutes	1.8 minutes	4 hours 53 minutes
200,000	10 hours 20 minutes	3.5 minutes	10 hours 24 minutes
1.2 million	2 days 7 hours 4 minutes	35 minutes	2 days 7 hours 40 minutes
3.1 million	3 days 16 hours 42 minutes	3.5 hours	3 days 20 hours
5.3 million	4 days 7 hours 17 minutes	4.7 hours	4 days 12 hours

Common Expiration Policy Scenarios

Scenario	Description
An indicator is reported by a single source (with an expiration policy)	On 10/1, Source A reports the indicator and the expiration date is set to 10/8. When the date switches from 10/7 to 10/8, this indicator is queued to have its status changed to Expired.
An indicator is reported by Source A (with an expiration policy of 7 days) and 3 days later is reported by Source B (with an expiration policy of 10 days).	On 10/1, Source A reports the indicator and the expiration date is set to 10/8. Source B reports the same indicator 3 days later (10/4). The indicator's expiration date is set using the greatest expiration date between the two sources. In this example, the new expiration date will be 10/14 (10 days from when it was reported by Source B). When the date switches from 10/14 to 10/15, this indicator is queued to have its status changed to Expired.
An indicator is reported by Source A (with an expiration policy of 7 days) and is later reported by Source B (with an expiration policy of Never Expire).	On 10/1, Source A reports the indicator and the expiration date is set to 7 days. Source B reports the same indicator 3 days later with a policy of Never Expire. The indicator’s expiration date is removed and the indicator is now set to Protect from auto-expiration.
A FQDN indicator is reported by Source A (with an expiration policy of 10 days with an exception for 5 days for FQDN indicators) and is later reported by Source B (with an expiration policy of 15 days).	On 10/1, Source A reports the FQDN indicator and the expiration date is set to 10/6. An exception takes precedence over the source’s expire policy. Source B reports the same indicator 1 day later (10/2). The indicator's expiration date is set using the greatest expiration date between the two sources. In this example, the new expiration date will be 10/17 (15 days from when it was reported by Source B). When the date switches from 10/17 to 10/18, this indicator is queued to have its status changed to Expired.

Common Expiration Policy Scenarios - Feed Updates of Indicator Statuses

Scenario	Description
Default Indicator Status Behavior:
An indicator is currently set to Expired and is reported by Source A (with an expiration policy of 7 days).	On 10/1, an indicator is in ThreatQ with a status of Expired. On 10/1, Source A reports the indicator. The status of the indicator does not change.
An indicator is currently set to Expired and is reported by Source A (with an expiration policy of Never Expire).	An indicator is in ThreatQ with a status of Expired. Source A, with an expiration policy of Never Expire, reports the indicator. The status of the indicator does not change
Status Behavior with Indicator/Signature Status Overrides Enabled:
An indicator is currently set to Expired and is reported by Source A (with an expiration policy of 7 days).	On 10/1, an indicator is in ThreatQ with a status of Expired. On 10/1, Source A reports the indicator. The status of the indicator changes to whatever the default status is for Source A and the expiration date is set to 10/8. When the date switches from 10/7 to 10/8, this indicator is queued to have its status changed to Expired.
An indicator is currently set to Expired and is reported by Source A (with an expiration policy of Never Expire).	An indicator is in ThreatQ with a status of Expired. Source A, with an expiration policy of Never Expire, reports the indicator. The expiration of that indicator changes to Protect from auto-expiration.