Current ThreatQ Version Filter

Executing Exports

AGDS manual exports are on-demand executions of the threatq:sync-export command used to create an Air Gapped Data Sync export from a source ThreatQ system without relying on automation or scheduling. They are typically used for initial data seeding, one-time transfers, testing, or controlled operational changes where administrators want full visibility into the export process and its results.

The following steps are for manually creating AGDS exports 

  1. Confirm that the Source and Target ThreatQ instances are running the same version of ThreatQ.
  2. Determine what data do you intend to export. At a minimum, decide on the following:
    • Will this be a full export or incremental?
    • Will the export include Investigations?
    • Will the export include Soft Deleted Objects?
    • Will the export exclude large files such as malware samples?
    • How far back do you want export to go or is there a date range?

    See the Export Command Flags topic for a complete list of flags for the AGDS export command. 

  3. SSH into the network-connected (Source) ThreatQ instance.
  4. Run the following command to check the Kubernetes pods' health:
    kubectl get pods -n threatq

  5. Verify that you have enough disk space for the export in the /var/lib/threatq/agds_transfer directory.
  6. Create the export file using the following command and Export Flags:
    kubectl exec --namespace threatq --stdin --tty deployment/api-schedule-run -- \
    ./artisan threatq:sync-export --target=<relative_path>

    It is highly recommended that you use the --start-date flag to determine how far back the export should go. Not using the --start-date flag will result in the export using the "Beginning of Time" date of 1970-01-01.

  7. Navigate to the export file once the process has been completed and locate the export tarball file. The file naming convention will be: tqSync_<timestamp>.tar.gz.

    This directory was specified in the export command via the --target flag and is a relative path to the default ADGS directory: /var/lib/threatq/agds_transfer.

  8. Open the sync report and confirm that there are no errors, the object count looks correct, and that any other flag additions have been included.
    tar -xzf tqSync_<timestamp>.tar.gz
    cat sync/sync-export.txt

  9. Transfer the export tarball file to the air-gapped target system using your organization’s approved offline method.
  1. Confirm that the Source and Target ThreatQ instances are running the same version of ThreatQ.
  2. Determine what data do you intend to export. At a minimum, decide on the following:
    • Will this be a full export or incremental?
    • Will the export include Investigations?
    • Will the export include Soft Deleted Objects?
    • Will the export exclude large files such as malware samples?
    • How far back do you want export to go or is there a date range?

    See the Export Command Flags topic for a complete list of flags for the AGDS export command. 

  3. SSH into the network-connected (Source) ThreatQ instance.
  4. Navigate to the api directory using the following command: 
    cd /var/www/api

  5. Create the export file using the following command while replacing the placeholder flag values with your own:
    sudo ./artisan threatq:sync-export --target=<your_directory> --start-date="<YYYY-MM-DD HH:MM:SS>" --end-date="<YYYY-MM-DD HH:MM:SS>"

    See the Export Flags topic for details on available export command flags.

  6. Navigate to the export file once the process has been completed and locate the export tarball file. The file naming convention will be: tqSync_<timestamp>.tar.gz.

    This target directory was specified in the export command via the --target flag.

  7. Open the sync report and confirm that there are no errors, the object count looks correct, and that any other flag additions have been included.
    tar -xzf tqSync_<timestamp>.tar.gz
    cat sync/sync-export.txt

  8. Transfer the export tarball file to the air-gapped target system using your organization’s approved offline method.