Azure Sentinel is a Microsoft flagship security product: a cloud-native Security Information & Event Management (SIEM) tool that has enjoyed great marketplace acceptance in the last year. Azure Sentinel includes Security Orchestration, Automation, & Response (SOAR) capabilities that increase SOC effectiveness and save you time and resources. SOAR takes your SIEM to the next level by proactively staging automated response workflows–that take deny, block, isolate, and shutdown actions in real time–which can potentially stop a hostile incident in its tracks.

Two new Microsoft innovations: Azure Arc and Azure Automation User hybrid runbook worker V2 combine to significantly extend and enhance Azure Sentinel’s SOAR features. This article delivers an end-to-end scenario: How to deploy a user hybrid runbook worker V2 to a server that is not in Azure (an Azure Arc server) and use it for real-time SOAR actions with an on-premises hardware firewall. Figure 1 illustrates the workflow when installing a V2 hybrid worker using an Azure VM Extension, and when running a SOAR automation worker task (a PowerShell script on-prem) in response to a specific Azure Sentinel incident.

Figure 1 – Achieving a simple and effective SOAR capability using an on-prem Azure Arc server to launch SOAR automations.

Azure Sentinel Automation Rules and Playbooks

The SOAR functions of Azure Sentinel come into play after a security incident is created by Azure Sentinel. When a new incident is created, automated responses can take the form of either Automation Rules (an internal feature of Azure Sentinel) or Playbooks, which are Azure Logic Apps with Azure Sentinel triggers.

Lead with Automation rules for simple actions

An automation rule can fully process a new incident, as well as extend processing to one or more playbooks. A best practice in Azure Sentinel incident response is to use the built-in Automation Rules when sufficient, and use playbooks as needed for more complex workflow. The built-in Automation rule actions are:

Change status of the incident
Change severity of the incident
Assign owner to the incident
Add tags to the incident

A simple and common use case for an automation rule is to reduce alert fatigue by auto-closing incidents when known ‘benign positive’ entities are involved. That is, an automation rule that changes the status of an incident to “Closed” when certain override criteria are met. For this simple automation, no playbook (Logic app) is required.

Playbooks are Azure Sentinel’s primary automation tool

Often, Automation rules call one or more playbooks, which as mentioned, are Azure Logic apps. Playbooks are a popular and powerful tool in the Azure Sentinel SIEM/SOAR solution because Logic apps can perform an infinite number of functions that are quickly assembled to meet business and technical needs. Here are examples of playbooks available in Azure Sentinel Playbooks templates (also seen in Figure 2):

Block AAD user – Incident
Block IP – Cisco ASA
Crowdstrike API authentication
Enrich multiple entities – AlienVault-OTX
Post Message Teams

Figure 2 – Playbook templates (Preview) in the Azure Sentinel console.

Hundreds of playbook templates to create or augment your SOAR capabilities may be found in the Azure Sentinel GitHub repository: https://github.com/Azure/Azure-Sentinel/tree/master/Playbooks. You will discover a dynamic and prolific global community of security researchers and DevOps engineers actively contributing to the repo and growing community support for the platform.

Azure Arc + Azure Automation hybrid user runbook worker V2

Remote instrumentality: Extending the reach of SOAR tools to remote environments–is an engineering challenge that requires excellent security. Effective real-world SOAR solutions require ‘reach-back’ into the environment where the trouble is to perform LAN-based control actions against the devices and services on the local network. The new support for extension-based deployment of Azure Automation user hybrid runbook worker V2 is great news for Microsoft Sentinel customers because the hybrid worker represents a remote instrumentality you can add to your SOAR toolkit.

Hybrid runbook workers run your scripts

A user hybrid runbook worker can run scripts (PowerShell or Python)–locally on Windows and Linux servers–in Azure and not in Azure. With extension-based hybrid worker (V2), both Azure machines and non-Azure machines (through Arc enabled server) can be managed identically and synchronously through ARM templates and policies. Every server in your enterprise becomes a potential SOAR utility worker!

While it’s been possible for some years to deploy a user hybrid runbook worker V1 to an on-prem server using an agent-based script, the new extension-based V2 model is a big improvement in usability and marks a renewed investment by Microsoft in the Azure Automation service. Using on-prem Azure Arc servers’ hybrid runbook workers to launch SOAR actions that need to occur on-prem is an efficient and logical fit. Table 1 summarizes the differences between the V1 and the V2 runbook workers.

AUTOMATION HYBRID RUNBOOK WORKER OVERVIEW
Platform	Description
Agent based (V1)	Installed after the Log Analytics agent reporting to an Azure Monitor Log Analytics workspace is completed.
Extension based (V2)	Installed using the Hybrid Runbook Worker VM extension, without any dependency on the Log Analytics agent reporting to an Azure Monitor Log Analytics workspace. This is the recommended platform.

Table 1 – Differences between V1 and V2 Automation hybrid runbook workers.

Comparing V1 and V2 runbook workers

If you have already deployed Automation hybrid runbook workers with the V1 agent-based method, everything in this article about Azure Sentinel SOAR integration applies and you are ‘ready to automate’. How V1 and V2 runbook workers execute runbooks is identical. V2 runbook workers have the edge in the ease of installation and management categories.

Figure 3 illustrates another difference between V1 and V2 runbook workers when installed on a non-Azure VM, i.e., an Azure Arc server: While the V1 worker presents itself to Azure automation with only an (unclickable) server name, the V2 worker is clickable and links to the Azure Arc server resource.

Figure 3 – Contrasting the agent based (V1) and extension based (V2) hybrid workers.

One-click deployment of V2 hybrid worker

If you have not yet explored the usefulness of Automation hybrid runbook workers in your environment, the new V2 extension-based installation deploys with one click! This is also an opportunity to try again with the hybrid runbook worker if you passed on it previously due to complexity of V1 agent-based deployment. After you create a Hybrid worker group in Azure automation, all your Windows and Linux servers–both Azure VMs and all your Azure Arc servers–will appear together in the list seen in Figure 4. This is the list of servers in your estate that are eligible for installation of a user hybrid runbook worker.

Figure 4 – One-click deployment of hybrid worker to Azure VMs and Azure Arc servers.

Summary of steps (and tips!) to deploy hybrid workers on Azure Arc servers

To summarize, here are the high-level steps to deploy Azure Arc server hybrid workers:

1 – Deploy an Azure Automation account to your Azure Subscription.
Tip: Be sure and locate your automation account(s) in the correct region to map to your log analytics workspace(s): https://docs.microsoft.com/en-us/azure/automation/how-to/region-mappings)

2 – Create an Azure Automation Run As credential (username and password) that grants access to run scripts locally on Azure Arc servers.
Tip: The password is not viewable in the portal after you create and save it.

3 – Define an Azure Automation User hybrid worker group and choose the Run As credential appropriate for that group.
Tip: The account should not be a Domain Admin. The account needs rights to log on locally, access the file system, and run scripts on the Windows or Linux computers in the worker group.

4 – Deploy one or more Azure Arc servers.
Tip: if you are just deploying a few servers, use the Interactive script method. If you want to deploy to dozens or hundreds of servers, use the at-scale method.

5 – Add Azure Arc servers to the hybrid worker group.
Tip: Two or more servers in the same worker group achieve high availability of the worker group.

Figure 5 shows the new V2 extension-based user hybrid worker installed on an Azure Arc server:

Figure 5 – Azure automation deploys a V2 worker role to an Azure Arc server using VM Extensions.

Running a SOAR automation on an Azure Arc server hybrid worker

Once you have deployed Azure Automation and Azure Arc assets as described in this article, you have in place a mechanism and platform (SOAR worker roles) to deliver real-time management code to your on-prem devices in response to security operational needs. Using Azure Arc servers as SOAR workers to launch scripts is just one of several ways to ‘reach back’ into managed environments from Azure in order to run code locally against on-prem devices.

Opportunities to leverage SOAR

All SOAR actions that touch hardware devices are highly tailored to the environment they are created for. For example, if you have Cisco ASA firewalls as endpoints, some of your SOAR actions might take the form of Cisco ASA instructions. These instructions could be communicated to your Cisco devices directly or indirectly via SSH, API, or another network or device management platform.

A common and valuable use of SOAR techniques is to actively add entities (like IP addresses or DNS domain names) to firewall block lists when threats from identifiable entities are detected. So ‘step 1’ in really using SOAR in this manner is to identify relevant SIEM incidents that make sense to pre-stage automation workflows for. Here are some sample Azure Sentinel incident types to consider staging IP address blocking automations for:

Azure Security Center incident: Traffic detected from IP addresses recommended for blocking
Azure Active Directory Identity Protection incident: Malware linked IP address
Azure Sentinel incident: Brute-Force Detection

What these alerts have in common is they may contain one more clearly identified IP addresses, like that seen in the Failed SSH brute force attack incident (Figure 6 below), that you might want to add to a firewall block list. Creating an Azure Sentinel playbook to do that for you automatically, or instantly on-command, can materially increase your organization’s cyberdefense.

Figure 6 – An example Azure Sentinel incident containing IPs you want to block (Azure Defender view).

Choosing the right platform for your SOAR automations

Before you proceed to author your own automation using a hybrid runbook worker, see what’s already available–that applies to your technical use case–from Microsoft and the technical community already. The specific scenario covered in this article has a Fortinet firewall endpoint. A search uncovered two Fortinet-specific solutions in the Azure Sentinel GitHub repo of possible interest:

Both of these solutions leverage an Azure Function app, which might also require Azure Private Link, in order to send SOAR-driven code in the form of REST API calls, to an on-prem device. This is a classic serverless approach and one designed for the larger organization requiring additional functionality.

For our simpler scenario, the on-prem hybrid runbook worker on the Azure Arc server is not ‘serverless’, in fact, it leverages the fact that we have a server in the right place to deliver the SOAR task code securely to the endpoint. The hybrid runbook worker gives you an asset that’s not part of the published solutions. Hybrid runbook worker solutions run scripts you author and understand–there is no costly or complex back-end.

Hybrid runbook worker in a SOAR role with firewall devices

Once you have decided that using a hybrid runbook worker is the right platform to launch a SOAR task against your firewall, you next select a Windows or Linux server hybrid worker group to run the task. If you are using a Windows hybrid worker, PowerShell is probably your script format. If you are running a Linux hybrid worker, Python2 or Python3 are likely formats.

PowerShell to create an SSH input file

For this scenario, we are using a Windows hybrid worker, and the firewall we want to control (Fortinet) accepts both SSH commands and REST API calls. One possible script solution is to deploy an Azure Automation runbook that creates a text-file based input file on the hybrid runbook worker’s file system. The last line in the PowerShell script calls an SSH command line client with the input file. See Figure 7 for a concept script using this method, the exact commands shown are specific to a Fortinet firewall.

Figure 7 – One possible script solution concept: creating an SSH script input file one line at time, then running an SSH command line client with the input file.

Tips when using the SSH input file method

1 – One line of script per line of PowerShell.

2 – If you need an actual quote ” to appear in the output file, escape it with the ` (back-tick) character, that is, `”.

3 – Put the PowerShell command in double quotes “” followed by pipe | and Out-File (see Figure 7). All Out-File lines after first line have the -Append switch.

4 – Use the -Encoding ascii switch on every Out-file line.

5 – Install PUTTY on the hybrid worker server(s) (https://www.puttygen.com/download-putty) this includes the PLINK.EXE SSH command line utility. Make sure the PUTTY program files folder is in the system PATH.

6 – Log onto the Windows server(s) as the user identity that is the Run As credential. Run the PLINK command line manually once to accept the certificate key of the device.

Use a PowerShell module if available

Further research uncovered a community solution that adds a Fortinet module to PowerShell: PowerFGT: https://www.powershellgallery.com/packages/PowerFGT/0.5.0. The PowerShell module uses the Fortinet REST API.

This solution proved to be the best: You have the convenience of familiar PowerShell commands with little development work. Using PowerFGT, it wasn’t necessary to convert the native firewall FortiOS command line task into an intermediate SSH script. Figure 8 shows the final PowerShell runbook script authored for this scenario. (This runbook can be downloaded from: https://github.com/john-joyner/AzureAutomationConfigurations/blob/master/FGT-add-IP-to-group.PS1.)

Figure 8 – PowerShell runbook to add an IP address to the Blocked Address Group of a Fortinet firewall using PowerFGT module.

Avoid plain-text credentials in runbooks

To prepare to use this PowerShell runbook, an Azure Automation credential named “PowerFGT Credential” was created. The runbook calls the username and password out of the stored credential at runtime using the Get-AutomationPSCredential command, providing a secure means of storing, delivering, and using the firewall login identity.

Invoking a SOAR playbook from Azure Sentinel

Once you have authored the runbook to perform your desired SOAR action, you need to make some decisions on how to ‘package’ the runbook for either automatic or manual activation (or both). And since the only supported automated response mechanism with Azure Sentinel (besides the few basic Automation rule functions) is the playbook (Azure Logic app), you will be authoring one or more playbooks—that call your PowerShell runbook–to get results.

A key feature of Azure Sentinel playbooks is that they are Azure logic apps that are anchored with either the Incident trigger or the Alert trigger. Table 2 summarizes the applicable scenarios for each trigger type.

	Automation rule	Playbook
Automatic activation: Alert	N/A	Playbook has Alert trigger, assign to individual analytic rules
Automatic activation: Incident	Playbook has Incident trigger, assign to Automation rules	N/A
On-demand activation: Alert	N/A	Playbook has Alert trigger, don’t assign to automation rule
On-demand activation: Incident	N/A	Not yet available

Table 2 – Alert automation vs. Incident automation options

So based on how you want to employ the SOAR automation, use either the Incident or the Alert trigger as the first task in your playbook (Logic app). Figure 9 is the playbook authored for this scenario. The playbook starts with the Alert trigger, gets the IP entities in the alert, and calls the Azure Automation runbook for each IP entity.

Figure 9 – The playbook “FGT-Add-IP-to-Block-List” gets the IP entities from an alert and calls the automation runbook.

If you used the Alert trigger, you will have the option to perform an on-command activation of the playbook from the Alerts blade of an Azure Sentinel Incident view. Put another way, you will only see playbooks available to run on the Alerts blade when those playbooks include the Alert trigger.

The envisioned use of the playbook in this environment is that Azure Sentinel operators investigating an incident will make a decision: That all the IP entities in any given alert are to be blocked. When the alerts have applicable IP addresses to block, the operator just needs to click on the View playbooks link beside the alert as seen in Figure 10.

Figure 10 – If an Azure Sentinel alert has one or more IP entities and you want to block all of them, follow the View playbooks link to run the SOAR playbook.

The View playbooks link in Figure 9 will open the Alert playbooks view seen in Figure 11, where you have one-click activation of the desired playbook.

Figure 11 – Push the run button to immediately execute the runbook on the hybrid worker of the Azure Arc server.

Within a few minutes of pushing the Run button seen in Figure 10, the PowerShell code will have run on the Azure Arc server in a logical sandbox and the desired firewall IP block will be in place. Figure 12 shows the final step complete: The IP entity in the Azure Sentinel incident is programmatically blocked in the on-prem firewall.

Figure 12 – End-to-end scenario: IP address added to the firewall block list.

In this manually-activated playbook scenario, the SOAR action can be commanded by your security staff from anywhere they can access the Azure portal after reviewing the particulars of the Azure Sentinel incident. This solution allows you to act on security intelligence from Azure Sentinel and immediately turn it into defensive changes on your network firewall, accurately and faster than was possible without SOAR.

Azure Sentinel SOAR worker: Azure Arc + Azure Automation