Playbook Fridays: Query Hashes via Email Submission

We were asked by a customer  to extend the analysis functionality of ThreatConnect to other SOC personnel that didn’t have direct access to the Platform. So we did. This Playbook creates a new process in which non-ThreatConnect users can get on-the-fly analysis and context into potential hash IOCs they’ve encountered, and simplifies the process of using the Threat Intelligence team as a conduit for another team to do analysis.

With this Playbook:

• No extra user accounts are needed
• Automated hash analysis occurs
• The Threat Intelligence team’s value across the enterprise is extended

And it solves:

• Lack of user accounts to give access to non-essential SOC staff.
• Uses the API to iterate over any potential owner within the local ThreatConnect instance

The Playbook is triggered when an email with a list of file hashes in the body is sent to an inbox. The mailbox trigger outputs the body of the email to a regex extract step, which pulls all of the included files hashes out and stores them within an array output variable. That array is passed through the main iterator, where the majority of the Playbook logic takes place. The larger concept here is the use of an iterator operator within another iterator. This is because there is a list of file hashes submitted for analysis, and each individual hash could have its own list of owners within the Platform; thus creating a potential for an  array within an array situation.

The main iterator will perform the following steps on each file hash within the array extracted from the email: First an API call is done to the /owners endpoint to get a list of owners within the ThreatConnect instance that contains the singular hash as an IOC. Once that list is compiled, a JMESPath query will extract all of the owners and pass that as an array to a secondary iterator. This iterator is more simple in nature, as all it is doing to looping through the list of owners and substituting each one within separate API calls to the hash value, but now using the ‘?owner=’ query parameter to gather all relevant information on the hash from each source that owns it within the Platform. The secondary iterator is then closed when this is finished, and the result is an analysis listing for each owner for the single hash value that is an output array from the iterator. This array goes through a quick JSON formatting step, and then to an if/else statement. If the data contains a failure message (meaning no results were found in the previous steps) then it takes the ‘false’ path out of the if/else statement and simply logs that no details were found for that hash. If there is no failure message, it will then take the ‘true’ path out of the if/else to a component.

This component is essentially a series of if/else steps that are checking for the IOC types that we define within its input configuration. Once the respective if/else statement hits a true statement, it will exit to its respective JMESPath query and extract the appropriate data relative to the indicator type. In this case for file hashes, there is subsequent checks to determine what type of hash it is. This is then returned to the component trigger, and outputted as a variable for use downstream of the component in the larger workflow.

In this Playbook’s current configuration this step isn’t necessarily needed, as since we are solely dealing with file hashes – we could have hard coded the hash related JMESPath query instead, but having it does offer the flexibility in the future to leverage other IOC types in this workflow without having to drastically change or add to the logic.

This final result is passed back to the original iterator, thus closing the loop on this first pass on our list of hashes where the result is stored at index 0 of the iterator output, and the entire process repeats again for the second hash in our original email list.

Once this array of results is completed, the main iterator will output the results to a series of formatting steps, and then finally out to a Send Email Playbook app that will send an email containing the results of what the customer’s ThreatConnect instance knows, back to the original sender.

How to Set Up this Playbook:

• Mailbox Trigger: Nothing special with this. The mailbox name can be changed to something other than the randomly generated name upon creation.
• Parse Hashes: The is a regex extract application. This outputs arrays of MD5, SHA1, and SHA256 hash values from the body of the email.
• Union MD5 and SHA1: This is an array operations app utilizing the union function. This is compiling a single array between the MD5 and SHA1 hashes.
• Union Built Array with SHA256: Identical to the previous step, only this is performing a union between the newly created array and the SHA256 values to produce a single array of hashes.
• Deduplicate: This is also an array operations step, however this is utilizing the unique function. The idea here is to remove any duplicates in the array so the Playbook doesn’t process the same data more than once to improve speed and efficiency.
• Iterator Hashes: This iterator operator has the deduplicated string array as the input, and will loop through each hash value.
• Get Owners List: This step is using the ThreatConnect API app to make a direct call (GET) to the /v2/indicators/files/{hashVariable}/owners endpoint to get a list of all the owners in the Platform that include the specific hash within their source.
• Extract Owners Name: This JMESPath app takes the owners list generated in the previous step and isolates just the owner names. ‘Data.owner[].name’ is the expression I am using for this.
• Iterate Over Owners: With the newly compiled owners list, this iterator takes that array and will make another API call to the same hash as before, but this time with the owner defined as well. This looks like /v2/indicators/files/{hashVariable}?owner={ownerVariable}. This owner can either be hardcoded into the URL as we just did here, or can be setup within the API app as a key-value pair with owner being the key, and the {ownerVariable} being the value. The outputs from this step are either straight to a merge for success, or to a logger for a failure. The logger is basically there for error checking later on if something where to go wrong. The merge is simply there to reform both outputs from the logger and the API app into one output to close the loop on the owner iterator.
• Change Formatting: This encapsulates the iterator result in square brackets if more than one owner is found for any of the given hash values.
• Check for IOC IF/ELSE: This is taking the formatted results from the owner iterator and checking if it contains a message from the API that an IOC was not found. If it does, it will exit out the failure path to log that it found no data for that specific hash. If that API error message isn’t found the success path will be taken to the next step of the playbook.
• Extract Results JMES Component: This component is a series of IF/ELSE statements that match the IOC type defined in its configuration. Once it matches, it will redirect the input data to the appropriate JMESPath app that will strip out the relevant data we want to return as the output of the component.
• Merge Results: This is another simple merge to redirect the output of both the component and the no IOC found logger to one output that is sent back to the first iterator, closing the loop.
• To string: The output of the iterator by design is a stringArray, but for the next few steps I needed to work with the data as a string data type. This step is doing just that, an array to string conversion.
• Format JSON (both)/Flatten JSON: The are just steps to clean up the data and reformat to create valid JSON. The iterator encapsulates each string output it is building into the overall array within double quotes. The steps are removing those quotes, and then flattening the overall object with a JMESPath ‘[]’ expression.
• Back to string: The send email app accepts string type data as an input for the body of the email. Much like before, we are simply converting the array into a string so it can be inserted into the response email.
• Send Results Email: This is the final step of the Playbook, as we are sure you are eagerly awaiting the results of your submission. The results are inserted into the body of the email, and the ‘To’ line is simply pulled from the initial trigger utilizing the trg.mbox.from output variable.

See this Playbook on Github.