Migrating from Azure Arc server to Azure VM

It’s becoming more common that on-prem and non-Azure cloud servers are in situations where they are Azure Arc servers that are migrating to Azure. After the migration their server resources are represented in Azure as Azure VMs. There is a recently documented process from Microsoft, that if not followed when you are in this migration scenario, can require a bit of unwinding after migration to restore monitoring services.

Why Azure Arc server before Azure VM migration?

There are a number of reasons why you might have deployed Azure Arc server to your non-Azure VMs on prem, in AWS, or in Google Cloud Platform (GCP):

  • Azure Monitor features (Microsoft Monitoring Agent (MMA) and Dependency Agent) with Azure Policy and VM extensions on non-Azure VMs as well as Azure VMs.
  • Additional extension-enabled governance such as downloading and executing scripts using the Custom Script Extension and updating certificates in Azure Key Vault.
  • Deploy and use Azure Automation solutions like Update Management with Tag-based update groups and Hybrid Runbook Worker V2 on an Azure Arc server.
  • Extend Microsoft Defender for Cloud protections and coverage such as the integrated vulnerability scanner from Qualys to Azure Arc servers.
  • Early adoption of Azure Monitor Agent (AMA) which installs as an Azure Connected Machine Extension on Azure Arc-enabled servers.

In all these scenarios, your non-Azure machine is dependent on some Azure-based services using the Azure Arc server agent–the Azure Connected Machine Agent installed with either the interactive or the at-scale versions of OnboardingScript.ps1 when you enabled servers for Azure Arc.

Monitoring continuity during migration phases

A desirable, but not mandatory, quality of VM migration projects is that ‘apples-to-apples’ performance data is available for before-and-after migration analysis. This technique can help satisfy application stakeholders that business processes are running as well–or better–in Azure than before migration. This analysis is easier when the same tools are used for the performance assessment before and after migration.

Azure Arc server with VM Insights monitoring includes a high-value map of server-to-endpoint network connectivity like that shown in Figure 1, which is identical in form and function to the same mapping feature on Azure VMs.

Figure 1 – VM Insights Service Map showing network connections (in an Azure resource group) between on-prem computers (Azure Arc servers).

Customers performing VM migrations to Azure have reported great results from the VM Insights ServiceMap solution, in terms of understanding the relationships between servers in domain and distributed application environments. Having no mystery when it comes to service dependencies is key to a seamless migration experience. Examine the same ServiceMap view with your server after migration to Azure to validate expected dependencies in the new setting.

Migrate your on-premises or other cloud Azure Arc-enabled servers to Azure

Microsoft published an article intended to help you plan and successfully migrate your on-premises, AWS, and GCP computer assets to Azure. By following the steps in the article, you can transition management from Azure Arc-enabled servers to Azure VMs: https://docs.microsoft.com/en-us/azure/azure-arc/servers/scenario-migrate-to-azure. There are six (6) high-level steps to execute for a successful migration:

Step 1: Inventory and remove VM extensions

Step 2: Review access rights (if you’re using a managed identity for an application or process running on an Azure Arc-enabled server)

Step 3: Disconnect from Azure Arc and uninstall agent

Step 4: Install the Azure Guest Agent

Step 5: Migrate server or machine to Azure (using Azure Migrate, Azure Site Recovery (ASR), or VM image upload)

Step 6: Deploy Azure VM extensions (can occur hands-off if Azure Policy so assigned)

Figure 2 represents the VM extension lifecycle as a server migrates from Azure Arc to Azure VM. As a best practice in Azure governance, the same Azure policies that deploy management services to computers, such as the policy initiative Enable Azure Monitor for VMs, are assigned to both Azure Arc servers and Azure VMs.

Figure 2 – As a best practice in Azure governance, leverage the same Azure Policy for both Azure Arc servers and Azure VMs.

Removing VM extensions on the Azure Arc server side

Essentially, if you migrate an Azure Arc server to Azure–while VM extensions installed by the Azure Arc server Azure Connected Machine Agent are present–you ‘break’ the migrated Azure VM from a management perspective. (Later in this this article, we will cover the repair steps if you need them.)

To get your Azure Arc server to Azure VM migration started smoothly, just uninstall all the extensions from the Azure Arc server using the Azure portal, Azure PowerShell, or ARM templates. Figure 3 shows what the installed extensions on a representative Azure Arc server might look like before migration. This server has the MMA, the Dependency Agent, a Hybrid Runbook Worker, and the Qualys vulnerability scanner agent installed.

Figure 3 – Typical VM Extensions installed on an Azure Arc server before migration to Azure.

Using the Azure portal, click into each extension one at a time and push the Uninstall button. If you have many computers to migrate in a short time frame, consider the PowerShell command Remove-AzVMExtension: 

Remove-AzVMExtension -ResourceGroupName "ResourceGroup11" -Name "MicrosoftMonitoringAgent" -VMName "VirtualMachine22"

Whatever method you use to uninstall the extensions from your Azure Arc server, wait until the extensions list for the Azure Arc server is empty before proceeding.

Disconnect from Azure Arc and uninstall agent, install the Azure Guest Agent

Disconnecting from Azure Arc and uninstalling the Azure Arc agent (the Azure Connected Machine Agent) are separate activities.

Notice: Understand that the most important part of the sequence of events is that you must properly uninstall the VM extensions before uninstalling the agent. If you delete the resource representing the Azure Arc-enabled server in your resource group, but you don’t uninstall the VM extensions, when you re-register the machine, you won’t be able to manage the installed VM extensions.

After you are sure all VM extensions have been uninstalled from an Azure Arc server, then disconnect the machine from Azure Arc. You can do this by running the azcmagent disconnect command on the machine or server, or the easiest way: locate the Azure Arc server in the Azure portal and push the Delete button.

Next, uninstall the Azure Connected Machine Agent from the Azure Arc server. Use Control Panel -> Programs and Features -> Azure Connected Machine Agent -> Uninstall. At scale, you can push out this bit of PowerShell script that uninstalls the Azure Arc agent: 

Get-ChildItem -Path HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall | `
Get-ItemProperty | `
Where-Object {$_.DisplayName -eq "Azure Connected Machine Agent"} | `
ForEach-Object {MsiExec.exe /x "$($_.PsChildName)" /qn}

Finally, prepare the server to migrate to Azure by installing the Azure Guest Agent. For more information about how to install the VM Agent, see Azure Virtual Machine Windows Agent Overview or Azure Virtual Machine Linux Agent Overview. The Azure Guest Agent is functionally identical to the Azure Connected Machine Agent when it comes to Azure VMs and Azure Arc servers’ ability to interact with VM extensions. It’s just that one works only inside Azure (the Azure Guest Agent) and the other only works outside Azure (the Azure Connected Machine Agent, or Azure Arc agent).

Analysis: While the Azure Guest Agent and the Azure Connected Machine Agent perform the same work, their behind-the-scenes operations are incompatible with one other. It’s necessary to unwind everything one agent does before migrating the computer between non-Azure and Azure clouds.

Perform the computer migration, let Azure Policy do the work

The good news is that, with the proper pre-work performed as described in this article, your migration will proceed effortlessly on the management front. Having the Azure Guest Agent pre-installed enables the computer to immediately communicate with Azure Resource Manager (ARM) when it boots up in the Azure cloud.

  • If you have, as a governance practice, assigned the same Azure policies to all resource groups containing Azure Arc servers and Azure VMs, your server will see the same policies in effect in Azure as it saw on-prem.
  • Policies to deploy management agents will automatically fire during ARM deployment of the Azure VM to re-deploy the same agents in the migrated VMs. (This is illustrated by the right-hand side of Figure 2.)

Recovering migrations with broken management stack

If you have, for whatever reason, migrated an Azure Arc server to Azure without performing the proper steps, the remainder of this article covers how to ‘clean up’ from that situation and resume full monitoring with the Microsoft toolset. Symptoms of having migrated an Azure Arc server to Azure VM ‘intact’ include:

  • The Agent status of your VM from its Overview -> Properties -> Virtual machine page is not “Ready” and no Agent version is listed.
  • No extensions are listed on your VMs Settings -> Extensions page nor can you successfully add an extension.
  • You can’t enable Microsoft management features like Backup, Insights, and Logs.

Make sure the Azure VM is running the Azure Guest Agent

If it’s still installed, the first thing you need to do is uninstall the Azure Connected Machine Agent (the Azure Arc agent) from the Azure VM. Do this using Control Panel -> Programs and Features -> Azure Connected Machine Agent -> Uninstall. The Azure Guest Agent won’t install when the Azure Connected Machine Agent is already installed.

Then install the Azure Guest Agent. See Azure Virtual Machine Windows Agent Overview or Azure Virtual Machine Linux Agent Overview.

Uninstall other agents, assess policy compliance

It is necessary to restore full functionality of VM extensions, and of the agents that were deployed by VM extensions. Generally, more reliable/predicable results are obtained by manually uninstalling agents from the Azure VM that had been installed by the Azure Arc agent, such as MMA, Dependency Agent, or Qualys agent, and allowing the same agents to be re-installed by Azure VM extensions on the Azure VM side.

Here is a protocol to follow to restore monitoring service with the MMA and Dependency agents, which includes functionality of the ServiceMap network connection mapping feature. This protocol assumes you use Azure Policy to control management agent installation and configuration.

  • Uninstall the MMA and Dependency agent using Control Panel -> Programs and Features -> MicrosoftMonitoringAgent (and -> DependencyAgentWindows) -> Uninstall.
  • Assess compliance status of migrated Azure VM to pre-existing policies. Launch remediation tasks Deploy – Configure Log Analytics extension to be enabled on Windows virtual machines and Deploy – Configure Dependency agent to be enabled on Windows virtual machines scoped to the migrated VM and with the Re-evaluate resource compliance setting enabled.
  • If remediation tasks to existing policies fail, temporarily create a scoped assignment of necessary policies, assign to the VM, and try again. (A freshly assigned policy can do the trick, if that works for you, after successfully remediation, delete the temporary policy assignment.)

Recovering from a failed Dependency Agent status

You may encounter the “Reset state failed” status (seen in Figure 4) with the Dependency Agent after attempting Azure policy deployment or remediation tasks:

Figure 4 – Unusual failed plugin state arising from migrating to Azure with an installed Dependency Agent extension.

The remedy here is to (1) Uninstall the failed extension, then (2) manually deploy the VM Insights Dependency agent (https://docs.microsoft.com/en-us/azure/azure-monitor/vm/vminsights-dependency-agent-maintenance) as seen in Figure 5.

Figure 5 – Manually installing the Dependency Agent extension on-command with Azure PowerShell.

Here is the PowerShell seen in Figure 5 that manually deploys the Dependency agent: 

Set-AzVMExtension -ExtensionName "Microsoft.Azure.Monitoring.DependencyAgent" `
    -ResourceGroupName "<MyResourceGroup>" `
    -VMName "<MyVM>" `
    -Publisher "Microsoft.Azure.Monitoring.DependencyAgent" `
    -ExtensionType "DependencyAgentWindows" `
    -TypeHandlerVersion 9.5 `
    -Location "<MyRegion>"

Immediately after getting the MMA and Dependency Agent properly deployed via VM Extension, data types like VMComputer, VMConnection, and VMProcess will start arriving in your Log Analytics workspace. All the useful VM Insights monitoring and other management features you were using with Azure Arc will now be lit up on the Azure VM side.

 

Want to learn more about Azure Arc server?

This article is a complement to my new book co-authored with Azure MVP Steve Buchanan with forward by Microsoft’s Sr. Cloud Advocate Thomas Maurer: Azure Arc-Enabled Kubernetes and Servers: Extending Hyperscale Cloud Management to Your Datacenter.

https://www.amazon.com/gp/product/1484277678

“Azure experts and MVPs Buchanan and Joyner provide just the right amount of context so you can grasp important concepts, and get right to the business of using and gaining value from Azure Arc. If your organization has resources across hybrid cloud, multi-cloud, and edge environments, then this book is for you. You will learn how to configure and use Azure Arc to uniformly manage workloads across all of these environments.”

 

Tags: #MVPBuzz #AzureArc #AzureSentinel #MicrosoftSentinel #AzureAutomation #AzureMigrate #AzureKeyVault #MicrosoftDefender #HybridCloud #AzurePolicy #Governance

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.