ASM to ARM migration and fun with Key Vault

Update: This is now possible, yet billing issues might occur. See this post: https://cloudpuzzles.net/2016/12/update-asm-arm-migration-experience/

TL;DR: ASM to ARM to new subscription, is currently not possible without downtime.

The rant:
As you might know, a lot of stuff is going on in Azure. Not only technical stuff, but also from a payment/partner perspective. A short history:

Azure started out as a Pay-As-You-Go (PAYG) service, and later on introduced Enterprise Agreements (EA) and Open Licensing. Those deals bypassed the partner channels (somewhat at least), which obviously wasn’t satisfying for those partners. Enter Cloud Solution Providers (CSP) where partners can resell Azure, O365 etc., directly to customers. Over the past year I’ve been working with companies (license houses, consultants etc) to teach them how to get started on CSP and onboard customers.

So the scenario is: Customer is running Azure, using Azure Service Management (ASM) which is also known as “Classic”. Now they want to buy Azure through a reseller: CSP. Buying Azure through CSP, you can’t use ASM. It’s simply not available (more info: https://cloudpuzzles.net/2016/02/buying-azure-through-cloud-solution-providers-csp/), so we have to migrate to Azure Resource Manager (ARM) first, and then move resources to the new subscription afterwards.

From an infrastructure point of view, this has been a huge PITA. We’ve used tools like Double Take Move from Vision Solutions to avoid too much downtime. Those license cost money though, so other customers was migrated offline using PowerShell. More info on that part here: https://cloudpuzzles.net/2016/07/notes-field-migrating-azure-classic-azure-resource-manager/

I’ve used the Platform Supported Migration Tool from Microsoft the last couple of months, to migrate from ASM to ARM. Mainly to prepare customers to for CSP migrations later on. The big deal here, is that there is no downtime involved. Everything is migrated on the fly. That was the easy part, ASM to ARM… Next step is moving to CSP subscriptions for those customers. Ready for a headache?

There is a built in tool in Azure, to move resources between resource groups (more info here: https://cloudpuzzles.net/2016/07/move-azure-vms-vnets-storage-accounts-resource-groups/). Again, no downtime is involved, it’s perfect! Using PowerShell we could also move to new subscriptions (this has since been added to the portal too), but there is a blocker: Key Vault
“Why do I all of sudden have a Key Vault?” You may ask. The thing is, Cloud Services (an ASM concept) have certificates assigned to enable WinRM etc. on public IPs. Those certificates (1 per Cloud Service) are moved into Key Vaults when migrating to ARM. From there, the VMs depends on the Key Vault – see the highlighted part here, which is the osProfile part of my VM configuration:

This means, if you want to move to a new subscription, you have to move the Key Vault with the VMs.. No problem, right? One should think so, since Key Vaults are supported to move between subscriptions: https://azure.microsoft.com/en-us/documentation/articles/resource-group-move-resources/#services-that-enable-move

Well, that’s only half the truth. You can’t move the Key Vault when it’s connected to the VMs. Yes, that is true. Okay, I’ll just delete the certificate then, right? No, you can’t remove the connection between VMs and Key Vaults at the moment. It doesn’t matter if you use the certificate – or WinRM – or not.

If you try to delete the Key Vault, you will get this error message, when you try to move the resources:

[code]
Move-AzureRmResource : {"error":{"code":"ResourceMoveProviderValidationFailed","message":
"Resource move validation failed. Please see details. Diagnostic information: timestamp ‘20160923T124509Z’,
subscription id ‘7bce381f-79b3-4521-b36e-138932735300’, tracking id ‘631f93e8-9065-490d-91cf-26925d6b1bf5’,
request correlation id ‘4850e0c7-5527-43e0-6bc-8d1a90fcd175’.","details":[{"target":"Microsoft.Compute/availabilitySets",
"message":"{"error":{"details":[{"code":"MissingMoveDependentResources",
"target":"/subscriptions/7bce381f-79b3-4521-b36e-138932735300/resourceGroups/puzzlesmove-Migrated/providers/Microsoft.KeyVault/vaults
/puzzlesmove","message":"The move resources request does not contain all the dependent resources.
Please check error details for missing resource ids."}],"code":"MissingMoveDependentResources","message":
"The move resources request does not contain all the dependent resources. Please check error details for missing resource ids."}}"}]}}
At line:1 char:1
+ Move-AzureRmResource -DestinationResourceGroupName move-subscription …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Move-AzureRmResource], ErrorResponseMessageException
+ FullyQualifiedErrorId : Conflict,Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.MoveAzureResourceCommand
[/code]

Thinking I was smart, I instead went back and tried from the start again: ASM
Deploying a new vnet, storage account, and VM, I got a certificate on my Cloud Service, which I tried to delete. No cookie, we can’t even migrate to ARM…

Testing with exact same setup, but not deleting my certificate:

The good news is: Microsoft is aware of the Key Vault dependency issue, and they’re working on a fix. I hope to have it available very soon! And I guess the technician who got my case in Premier Support, is also hoping for a quick fix 😉