Migrating virtual machines without shared storage in Windows Server 2012 Hyper-V.
The ability to move a virtual machine without downtime put Hyper-V on the same footing with other hypervisors and provided organizations with greater flexibility in hosting and optimizing resources. For most organizations using Hyper-V, live migration has become so important that one of the criteria for developing new features of Windows Server 2012 was the integrity of its mechanism.
n most issues related to computing, the main task is to consolidate shared resources and functions in a single module. In such a consolidation is the main purpose of virtualization technology. The Windows Server 2008 R2 Hyper-V hypervisor had similar priorities, in which it became possible to share NTFS volumes based on SAN storage by all nodes in the cluster. With this approach, all volumes were granted simultaneous access using Cluster Shared Volumes (CSV) technology. Server 2008 R2 has a live migration migration mechanism that allows you to move virtual machines between nodes in a cluster without downtime. Live migration was performed by copying the contents of the RAM and the state of the virtual machine in the process. .
The Server 2012 Hyper-V hypervisor implements several new features, some of which relate to the operation of fault-tolerant clusters. Let’s highlight two important changes in the live migration migration mechanism in a failover cluster.
-In Server 2012, live migration supports multiple simultaneous migration processes between a pair of host servers. In the previous version only one process was supported.
– The mechanism of fault-tolerant servers supports the use of up to 64 nodes and 4000 virtual machines in one fault-tolerant Server 2012 cluster – the figures increased by 400% compared to the fault-tolerant Server 2008 R2 clusters (although this article is not about this).
Server 2012 has a new type of live migration: migration without shared storage. Yes, I did not make a reservation. No shared storage, no shared cluster — all you need is just a Gigabit Ethernet connection between the Server 2012 Hyper-V hosts. Through this connection, you can transfer the virtual machines between the Hyper-V host servers, while moving the virtual disks of the VHD virtual machines, the contents of the memory, the state of the processor and the system without idleness of the virtual machine. If we consider the most extreme scenario, then a virtual machine running on a laptop with VHDs located on a local shared drive can be moved to another laptop connected via a single Gigabit Ethernet network cable.
A word of caution: do not think that using live migration without shared storage means that failover clusters are no longer needed. A failover cluster is a high-availability solution, while hot migration without shared storage is a mobile solution that gives you additional flexibility when you plan to move virtual machines between Hyper-V hosts in your environment. In addition, hot migration can be an addition to a failover cluster. Imagine the ability to move virtual machines without idle work inside the cluster, from the cluster to the outside, between the clusters and between individual host systems. With live migration without a shared repository, you no longer depend on repositories.
Requirements for using live migration without shared storage
The requirements for activating the live migration mechanism are quite simple.
-You need two (at a minimum) instances of a Server 2012 system with the Hyper-V role installed or the free version of Microsoft Hyper-V Server 2012 installed.
-Each server must have access to its own storage for storing virtual machines. This role can be played by local storage, a dedicated SAN storage, or a Server Message Block (SMB) 3.0 share.
– Servers must have processors of the same type or family (that is, Intel or AMD) if you use the Processor Compatibility feature for virtual machines.
Servers must be members of one Active Directory (AD) domain.
– Between servers, a connection of at least 1 Gb / s should be established (it is recommended to organize a separate dedicated network segment for live migration traffic, but this configuration is not mandatory), through which two servers can exchange data. For the network adapter you use, you must activate the Client for Microsoft Networks and File and Printer Sharing services for Microsoft Networks, as they are used for all types of migration between repositories.
– On each Hyper-V server, the same virtual switches should be assigned the same name to avoid errors and manual operations during the migration process. If a virtual switch with the same name as the switch used in the settings of the portable virtual machine is not defined on the target Hyper-V server, an error message will appear and the administrator performing the migration will have to choose which switch on the target server to connect to virtual machine adapter
-The virtual machines for which you plan to perform the migration should not use transit storage.
When your environment meets the listed requirements, you can proceed to the next step — allow outgoing and incoming migrations for Hyper-V hosts.
Host level permissions
To enable live migration on Hyper-V servers, you must enable the Enable incoming and outgoing live migrations checkbox in the Hyper-V hypervisor settings. These settings are available in the Hyper-V Manager wizard. Figure 1 shows the basic settings required to allow migration outside the cluster.
|Screen 1. Hyper-V server live migration settings|
In the simplest environments, selecting the option Enable incoming and outgoing live migrations, accepting the default setting of Use Credential Security Support Provider (CredSSP) for authentication and using any available network for live migration should ensure that migration is performed without shared storage. The unobvious moment: the Hyper-V integrated exception (MIG-TCP-In) creates a firewall resolution rule for TCP port 6600. If you use an alternate local firewall on the servers or if firewalls are used to filter traffic between servers, manually.
Keep in mind that on the Hyper-V Settings screen you can also set the maximum number of simultaneously running migration processes. Server 2012 removes the limit to one live migration process between any two hosts at the same time, instead the number of simultaneous migrations is now determined by the network transmittance. However, if you want to limit the number of simultaneous migrations to a certain number, set this limit in the Simultaneous live migrations field. You can configure this value using the MaximumVirtualMachineMigrations parameter of the Set-VMHost command in Windows PowerShell.
Although the default settings can work in simple environments or in basic testing, most environments will need to switch to Kerberos authentication and use a dedicated network for live migration traffic, which will include copies of the virtual machine memory and its storage devices. Using the Kerberos mechanism allows administrators to initiate migration processes remotely. Using a dedicated network helps to manage network traffic and ensure that there is sufficient network bandwidth to perform the migration. Let’s look at the authentication process and see why it becomes a source of problems for live migration in non-clustered environments.
Live migration authentication
In clustered environments where the Hyper-V host servers are part of a failover cluster, all the hosts share a common cluster account. This account is used to exchange messages between hosts in the authentication process, simplifying (from the point of view of authentication) operations such as migration within the cluster. Beyond the cluster boundaries, each Hyper-V server has its own computer account — there are no shared credentials. When performing operations, the account of the user who initiates the action is usually used.
During live migration, operations are performed on the source server, on the target server (and on file servers, if the virtual machine is stored in the SMB shared folder), and authentication is required in each case. If the administrator performing the migration connects to the source server or target server and starts the live migration process without shared storage through the local Hyper-V Manager snap-in, then the administrator credentials are used for both local operations and for executing commands on the target Hyper-V V. In such a scenario, CredSSP works correctly, allowing administrator credentials set on the client side to be used on a remote server — usually at a distance of one authentication step between the local machine with which the operation is performed and the remote server.
However, the general ideology of the Server 2012 system (and the general management approach) implies remote control and automation. The need to constantly log in to the source and target Hyper-V servers every time you need to migrate outside the cluster is a huge drawback when remotely managed. If the user is logged on to the local computer running the Hyper-V Manager snap-in and tries to migrate between the Hyper-V A and B hosts, this attempt will fail. The user credentials will be used on host A (which is in one authentication step from the client system), but host A cannot use these credentials on host B to complete the migration. The problem is that the CredSSP protocol does not allow the transfer of credentials to a system that is in more than one step. In this situation, full remote control will ensure the use of the Kerberos protocol: Kerberos supports limited delegation of authentication. Thus, when a user performs an operation on a remote server, this remote server can use the user credentials to authenticate to the second remote server.
Does this mean that the server to which I connect remotely can simply take my credentials and use them on another server without my knowledge? This is where the limiting part of delegation comes into play, despite the fact that you will need to perform certain configuration before you can use the Kerberos protocol as an authentication protocol during migration. You need to configure delegation for each computer account that will be allowed to perform operations on a different server on behalf of the user. To configure delegation, use the Active Directory Users and Computer Management tool and the properties of the computer account for the server that will be given the right to delegate. As Figure 2 shows, the Delegation tab contains settings for the allowed delegation level.
|Screen 2. Delegation settings to perform remote start live migration|
The only service that requires delegation is the Microsoft Virtual Migration Service, which must be activated on the target Hyper-V server. You need to select only one authentication mode – Use Kerberos. I have two servers SERVERA and SERVERB; the screen shows that I am changing the delegation settings for server SERVERA and setting up Kerberos delegation on my other server SERVERB for the Microsoft Virtual System Migration Service. I will repeat the configuration process for the SERVERB computer account, allowing you to delegate it to the SERVERA server. Also keep in mind that I configured delegation for the Common Internet File System (CIFS) service, which will be needed later when virtual machines hosted in SMB public folders are moved between hosts.
Remember that all hosts participating in the migration must have the same authentication settings. Figures 1 and 2 show the main differences between the CredSSP and Kerberos protocols and the settings required to use each of the protocols. Figure 1 shows the use of the CredSSP protocol; its operation requires that hot migration be started from one of the Hyper-V servers. Figure 2 illustrates the use of Kerberos authentication and the remote launch of a process that requires additional restricted Kerberos delegation. Although the use of Kerberos authentication requires more time-consuming configuration, the additional flexibility obtained justifies these efforts and translates this configuration into the recommended one.
|Figure 2. Migrating using the Kerberos protocol|
Authentication was a difficult step. You must now configure the network to use for inbound migrations (that is, the network on which the host will wait and receive live migrations). By default, live migration is allowed from any network. However, I recommend, if possible, to use a closed network dedicated to migration processes to ensure guaranteed bandwidth and separate migrations from the rest of the network traffic. You can add multiple networks and set the order in which they are used: just enter the address of the corresponding subnet in the network prefix notation, also known as the Classless Inter-Domain Routing (CIDR) notation. For example, to specify a network adapter with an IP address of 10.1.2.1 and a subnet mask of 255.255.255.0, I used the notation 10.1.2.0/24. Alternatively, specify the full IP address with a mask of 32 (for example, 10.1.2.2/32), but in this case, you will need to change the settings every time you change the IP address. Make sure that the source and target servers can communicate with each other using the IP addresses that you specified to use during the live migration process, otherwise the migration cannot be performed. To modify these settings using PowerShell, use the Add-VMMigrationNetwork and Set-VMMigrationNetwork commands.
Apply migration without shared storage
After setting up the Hyper-V host servers, the migration itself is easy. For a virtual machine, select the Move action, and then select Move the virtual machine as the move type. Enter the name of the target Hyper-V server to which you want to move the virtual machine, and to top it up, specify how the elements of the virtual machine, such as VHDs, will be moved to the destination. Screen 3 shows the final motion settings. Since we are interested in a scenario without shared storage, in which there is no shared storage and no shared SMB file folders are used, you need to choose one of two options: items. The first option allows you to specify a single place on the target storage, which will contain the virtual machine configuration, hard drives, and snapshots. In addition to the selection of relocatable elements, the second option allows you to specify the storage location for each virtual machine element.
|Screen 3. Selection of a variant of the move operation|
After you make your selection, specify the folder on the target server. The move operation will start. The execution time depends on the size of the VHD disks, the amount of memory being moved, and the frequency of changes. However, the operation will be performed without downtime and loss of connection with the virtual machine. You can also initiate a move using the Move-VM PowerShell command.
Troubleshooting Migration Errors
The following steps should help you deal with any obstacles that may arise in the process.
- Make sure your infrastructure meets the requirements described at the beginning of this article.
- Check the Event Viewer (Applications and Services Logs> Microsoft> Windows> Hyper-V-VMMW> Admin) log for messages describing the problem.
- Verify that the IP addressing settings between the source and destination servers are correct. Servers should be able to exchange messages. Check with the ping command for the availability of the destination IP address of the migration from the source server.
- Run the following PowerShell command from the session with elevated privileges to view the IP addresses used for the server: gwmi -n root \ virtualization \ v2 Msvm_VirtualSystemMigrationService | select MigrationServiceListenerIPAddressList
- Ensure that the target server has a firewall exception enabled for Hyper-V services (MIG-TCP-In).
- The name of the target server must be resolvable through DNS. Try running the Nslookup command on the target server. Also run the command
ipconfig / registerdns
on the target server and command
ipconfig / flushdns
on the original.
7. On the target server, use the following command to clear the Address Resolution Protocol (ARP) protocol cache:
command arp -d *
8. To test connectivity, try remotely sending a Windows Management Instrumentation (WMI) request to the target server (the WMI-In firewall exception must be enabled)
gwmi -computer -n root \ virtualization \ v2 Msvm_VirtualSystemMigrationService
9. Change the IP address used for live migration. For example, if you are using a 10.1.2.0/24 subnet, try replacing it with a specific IP address 10.1.2.1/32. Also, check the IPSec encryption or firewall settings between the source and destination servers. Find out if several network cards are connected to the same subnet – this can cause problems. If you encounter this situation, try disconnecting one of the adapters.
10. Configure authentication via CredSSP and start the process locally from the Hyper-V server. If after this error is resolved, then the root of the problem in the delegation of Kerberos.
The most common problems among those I have encountered are related to errors in configuring the Kerberos protocol or IP addresses. Failure to resolve the target server name through DNS service will also cause problems.
Summing up and next steps
Live migration without shared storage is the most radical type of migration with zero downtime. However, there are other types of migration without downtime, such as storing virtual machines in a shared SMB file folder that both Hyper-V hosts have access to. With this approach, the memory and state of the device are moved across the network, and the storage remains in its place. And there is still live migration within a failover cluster — a process that can use shared SAN storage via the CSV file system (CSVFS).
If you move a virtual machine between failover clusters or between a failover cluster and a standalone Hyper-V host, you will have to remove the virtual machine from the cluster before migrating. It is very convenient that in Server 2012 you can add and remove virtual machines from a failover cluster without stopping them — that is, without idle virtual machines, even if migration affects a failover cluster.
Of course, even in a scenario without shared storage, there is a need for a common physical network infrastructure and dependence on the settings of the IP address of the virtual machine during the move. That is why another possibility of Server 2012, Network Virtualization, opens up a new world: now virtual machines can be moved between any hosts without changing the network settings in their operating systems.
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Recent Posts: TechnoBlogy