Playbook Fridays: Human in the Loop Playbook Systems

When automating processes, you may run into situations where you need a human to be part of an automated process

The whole goal of Playbooks is to make it easy to automate tasks so you can be faster, more effective, and more secure. When automating processes, you may run into situations where you need a human to be part of an automated process. This blog post introduces a Playbook system you can use to create processes which include human review or validation in the middle of the process. If you just want to find more information about the system, you can find links to the system’s documentation here: https://github.com/ThreatConnect-Inc/threatconnect-Playbooks/tree/master/components/TCPBC-CT-Human%20in%20the%20Loop . If you just want to import the Playbooks and start exploring, you can jump to the “Setup Instructions” section near the end of the blog post.

The Problem

If you’ve made a habit of automating processes, you will soon realize that a number of processes require some human involvement. For example, let’s say you have an incoming stream of phishing URLs reported by your employees or customers. Each reported URL is sent to a Playbook that creates an Indicator in ThreatConnect. For each reported URL, you want to create a task for an analyst to investigate the URL. If the analyst thinks that the URL is not malicious, the process can stop. If, however, the analyst determines that the URL is likely malicious, the threat and confidence ratings for the URL should be increased and the URL should be sent via Playbooks to a firewall. The difficulty is that you need to have a human in the middle of the automated processes. You want a workflow that looks something  like:

What would it look like to design a system for handling this type of process with Playbooks in ThreatConnect?

The Solution – Playbooks Style

To understand how to solve the problem, take another look at the diagram above. The difficulty with a process that sandwiches human interaction between automated processes is not really in part 1 or part 2 of the automated process (building automated processes is easy with Playbooks) nor is it in the human review/validation step. The challenge is the arrows that connect each step together. We need to:

1. Let a human know when he/she needs to do something
2. Kick off another automated process once the human is done (without adding more work for the human)

In a sense, we are automating (as much as possible) the connection between human and automated processes. So what does this look like in a Playbook? To demonstrate how this works in a Playbook, we’ll walk through the example Playbook here. There are instructions for installing and setting this system up later in this blog post; for now, you can just follow along.

This Playbook starts with an HTTP trigger followed by a “Value Lookup” app (which looks through the headers of the incoming request) and an “If” operator. The “If” operator checks if the value retrieved by the “Value Lookup” app is not equal to ‘true’.

 

Closeup of the “Value Lookup” App

Closeup of “If” Operator

This creates two different paths of execution. If the request to the Playbook does not contain a certain header with a certain value, we are going to go in one direction; if the request does contain the header with the proper value, we go a different direction. In the example Playbook (shown below), the upper path (which leaves the “If” operator on the blue line) is executed if the header is not present and the lower path (which leaves the “If” operator on the orange line) is executed if the header is present.

As you can see, at the end of the upper line there is a component named “[Human-in-the-Loop] Create task 1”. This component will create a task in ThreatConnect and record the link to the current Playbook in the datastore (which will be important later). As such, this component takes two arguments: the name of the task which will be created and the link to the current Playbook. So the upper line is executed if a certain header is not present and will run the “Logger 1” app and create a task using the “[Human-in-the-Loop] Create task 1” component.

The lower line in the example only contains an app to log a value, but in a real-world scenario, you would build out the process which should be executed once the human interaction is done. The basic structure of the playbook is something like this:

The upper line contains the action performed before human involvement; the lower line contains the process to be performed after human involvement.

Let’s remind ourselves of what we are trying to accomplish. We are trying to build a system that allows us to sandwich human interaction between automated processes.

To do this, we determined that we need to:

1. Let a human know when he/she needs to do something
2. Kick off another automated process once the human is done (without adding more work for the human)

The upper row of the example playbook we walked through achieves the first requirement: letting a human know when he/she needs to do something (the component on the end of the last line will create a task).

Now we need to consider the second part: how can we kick off another automated process (which would exist in the lower line of the Playbook above) once the human interaction is done? This capability is enabled by the “[Human-in-the-loop] Complete task.pbx” Playbook. Once imported and activated, this Playbook provides a user-action trigger on tasks that lets a user complete the task and kick off the next step in the process. If you recall how the “[Human-in-the-Loop] Create task 1” component works, it creates a task and stores, in the datastore, the link to the Playbook that should be called once the task is finished. Now, when the user completes the task using the “[Human-in-the-loop] Complete task.pbx” Playbook, this Playbook completes the task, finds the correct link in the datastore, and requests the link. When making this request, it includes the special header that executes the lower path of the Playbook. All said and done, the human-in-the-loop system can be summarized as in the image below.

Setup Instructions

So how do I use this system? There are links to the human-in-the-loop system’s documentation here. To setup the human in the loop system:

1. Download all four of  the “.pbx” files here.
2. Import and activate the “[Human-in-the-Loop] Create task” component and the “[Human-in-the-loop] Complete task.pbx” Playbook.
   • There will likely be errors when you import the Playbooks and try to activate them. You may have to open and save the datastore apps (you don’t have to change anything in them) to be able to turn the Playbook/Component on.
3. Import the “[Human-in-the-loop] Example.pbx” playbook.
   • This Playbook is the one we walked through earlier in this blog post. Once you have imported it, double-click on the “[Human-in-the-Loop] Create task 1” component to edit it. You can change the task name to anything you choose. To set the “Target Playbook URL” value (this needs to be the link to the playbook we are currently editing which will be called once the task is completed by the user), type something in the field (you can type anything) and activate the playbook. Copy the link to the Playbook, deactivate it, reopen the “[Human-in-the-Loop] Create task 1” component, and paste the copied link in the “Target Playbook URL” field of the “[Human-in-the-Loop] Create task 1” component.
   • Activate the Playbook and make a request to the Playbook’s trigger link.
     • This should create a task in ThreatConnect (assuming your account has permissions to create a task).
     • Open up the page for the task.
     • There should be a Playbook Action button on the right side of the task’s page that says “Human-in-the-Loop Complete Task”. Once you click this, the task will be completed and the “[Human-in-the-loop] Example.pbx” Playbook will be executed (and will follow the lower path).
     • If this doesn’t show up, go back to step 2 and make sure you have imported and activated the “[Human-in-the-loop] Complete task.pbx” Playbook
4. To make it easy to make Playbooks that include humans in automated processes, import the “[Human-in-the-loop] Template.pbx” Playbook. This provides the basic template for a human-in-the-loop playbook system which you can clone to create a human-in-the-loop system quickly.

As always, if you have any questions, please raise an issue and thanks to all of those who have contributed to our Playbook repository!