7 Apr 2013

Managing Hyper-V with PowerShell in Windows Server 2012

Manage the Hyper-V role in Windows Server 2012 using PowerShell 3.0. This feature is new in Windows Server 2012, as the Windows Server 2008 R2 version of Hyper-V didn't have an integrated PowerShell module.

In Windows Server 2012 there has been extensive development in the PowerShell area. With all modules in the operating system there are now about 2,400+ cmdlets available to manage everything. The Hyper-V module alone consists of 164 cmdlets, but you will also use other cmdlets when automating your Hyper-V virtualization platform.

If you are totally new to Hyper-V, I would recommend that you fire up the Hyper-V Manager and look around in that and see what you can do and what the default settings look like.  A bit later in this post we will look at how we can automate some of these features. Everything in the Hyper-V Manager can be accessed and managed from the PowerShell console, but not everything fits into this article.



To show what PowerShell Hyper-V cmdlets are available you can use the following command:

Get-Command –Module Hyper-V

And if you want to count them you can use the following:

(Get-Command –Module Hyper-V).Count

PowerShell help is your friend! With it you can find what parameters can be used and see examples of the cmdlet usage:

Get-Help Get-VM –Full

Activating Hyper-V on Windows 2012

In order to use the Hyper-V module in PowerShell we need to enable that feature. Note that you do not need to activate the Hyper-V role in order to use the PowerShell module, but you need to use it against a remote Hyper-V host:

Add-WindowsFeature Hyper-V –IncludeManagementTools

This requires a restart because you have enabled the Hyper-V role. If you want to check what Hyper-V features are enabled on the server. simply type the following: Get-WindowsFeature Hyper*



After we have enabled the role we need to do some initial configuration. To be able to see and manage Hyper-V, we have to run the PowerShell Console as Administrator. Star by right-clicking on the PowerShell icon in the start menu.



If you do not use an elevated console you should see the following:



If you do have an elevated console we can actually see some values, as shown below.


Now, there are two locations where the virtual machines and virtual hard disks are stored in Hyper-V that we want to change from the original location. These locations are:

C:\ProgramData\Microsoft\Windows\Hyper-V

C:\Users\Public\Documents\Virtual Hard Disks

This only comes into action if you create VMs and do not specify a particular path. But it is quite important: Say, for instance, you don't have a huge system drive. If you create large virtual machines the disk space will run out, the virtual machines will pause, and the Hyper-V host will not be able to write any more files, which can lead to an outage. Using snapshots can also fill up your system drive if you do not configure the virtual machine path properly, as the snapshots are stored in the aforementioned path by default. (In a future article we'll play with virtual machine snapshots.)

So let's configure the paths to a non-system drive using PowerShell:

PS C:\Windows\system32> Set-VMHost -VirtualHardDiskPath D:\VMs -VirtualMachinePath D:\VMs

See Also >> Microsoft Private Cloud Ready Assessment Tool

If you want to configure five hosts at the same time you can do this:

PS C:\Windows\system32> "HV01","HV02","HV03","HV04","HV05" | %{ Set-VMHost –ComputerName $_ -VirtualHardDiskPath D:\VMs -VirtualMachinePath D:\VMs}

Let's configure networking with PowerShell next. There are three different switch types; typically you want the external one. It is only with the external switch type that you can have virtual machines reaching other networks and machines on remote hosts.

The parameter –AllowManagemetOS creates a virtual switch on the physical NIC called "PhysicalNic1Internal," on which a virtual network adapter is created for the Hyper-V host to use for management traffic.

PS C:\Windows\system32> New-VMSwitch -Name Internal -AllowManagementOS $true -NetAdapterName "PhysicalNic1Internal"

Creating and Removing Virtual Machines

After these essential things has been configured, we can continue to create virtual machines and start them. Now we want to create five test virtual machines with the same configuration but with different virtual hard drives and names.

The first example creates five test virtual machines in the above default path without any virtual hard drive:

PS C:\Windows\system32> 1..5 | %{New-VM -Name "TestVM$_" -NoVHD -MemoryStartupBytes 256MB -SwitchName Lan}

If we add the parameter –Path and instead of –NoVHD and use –NewVHDPath and –NewVHDSizeBytes we will get virtual machines in nice folders with a system.vhdx virtual hard disk that is dynamic and can grow to 10 GB.

PS C:\Windows\system32> 1..5 | %{New-VM -Name "TestVM$_" -Path d:\VMs -NewVHDPath "d:\VMs\TestVM$_\System.vhdx" -NewVHDSizeBytes 10GB -MemoryStartupBytes 512MB -SwitchName Lan}

Perhaps you noticed that I did not specify more than just the parent path in the virtual machine path, but it still created those folders and a virtual machine subfolder where the configuration xml file lies.

To start these VMs after they've been created, we can use PowerShell pipelining:

PS C:\Windows\system32> 1..5 | %{New-VM -Name "TestVM$_" -Path d:\VMs -NewVHDPath "d:\VMs\TestVM$_\System.vhdx" -NewVHDSizeBytes 10GB –MemoryStartupBytes 512MB -SwitchName Lan} | Start-VM

As you can see, it is quite easy to create just a few or hundreds of virtual machines. And you can create them – using PowerShell -- without having go through a wizard for each and every one of them!

It is almost as easy to remove the virtual machines as it is to create them. The following command requires that the virtual machines are turned off: Get-VM TestVM* | Remove-VM -Force

The only problem with this cleaning is that it leaves folders and virtual hard disks behind. If I am using a cmdlet that says Remove-VM, I want it to actually remove everything. So, with a small addition we can achieve this anyway: If we omit the TestVM* portion, it will clean out the whole host of virtual machines registered there. (Hopefully you have a backup somewhere!) See the command below:

Get-VM TestVM* | %{ Stop-VM -VM $_ -Force; Remove-VM -vm $_ -Force ; Remove-Item -Path $_.Path -Recurse -Force}

5 Apr 2013

Live Migration in Windows Server 2008 R2 Hyper-V

Live Migration feature that is included in Windows Server 2008 R2 Hyper-V. The Live Migration feature allows you to move a virtual machine between two Hyper-V server failover cluster nodes without data loss and without service interruption.

Live Migration Feature in Microsoft Windows Server 2008 R2 Hyper-V

Microsoft Windows Server 2008 R2 Hyper-V is the much awaited follow-up to the Microsoft Windows Server 2008 Hyper-V release. In this latest version of the Windows Server 2008 operating system, several new features have been added to the Hyper-V technology, but the most anticipated is the Live Migration feature.

The Windows Server 2008 R2 Hyper-V Live Migration feature provides you with two main benefits. First, Live Migration allows you to implement a high-availability solution based on Failover Clustering for mission-critical applications running in virtual machines, without the requirement for either a cluster-aware guest operating system or application. In addition, Live Migration provides the ability for System Center Virtual Machine Manager 2008 R2 (a Beta version is available for download from Microsoft) to provide dynamic load balancing of virtual machines, also without data loss or service interruption.

Hyper-V Live Migration Feature Requirements

In order to use Live Migration successfully, you must ensure that your Windows Server 2008 R2 Hyper-V failover cluster meets the following requirements:
  • Hyper-V failover cluster nodes must use processors from the same manufacturer and of the same type. Therefore, you cannot use Live Migration to move a virtual machine running on an AMD-V platform to an Intel VT platform.
  • Hyper-V failover cluster nodes must be configured on the same TCP/IP subnet. Although the Windows Server 2008 and Windows Server 2008 R2 Failover Cluster feature does not require that cluster nodes are configured on the same TCP/IP subnet, this is a required configuration to use Live Migration.
  • Hyper-V failover cluster nodes must have access to shared storage. The shared storage options are the same ones that are supported in Windows Server 2008 Hyper-V, and include iSCSI-based devices as well as storage area network (SAN) devices.
A Live Migration of a virtual machine can be started from the Windows Server 2008 R2 Failover Cluster Manager or through System Center Virtual Machine Manager 2008 R2 if it is used to manage the failover cluster nodes. You can also initiate a Live Migration using a Windows Management Instrumentation (WMI) or PowerShell script.

Live Migration Process

The process that takes place during a Live Migration starts with a TCP connection from the Hyper-V failover cluster source node (where the target virtual machine is initially executing) to a selected destination node in the same Hyper-V failover cluster. The TCP connection is established to transfer the virtual machine configuration data from the source node to the destination node, and the data is used to create a new virtual machine with identical settings on the destination node. The virtual machine configuration data includes the number and type of virtual storage adapters, virtual network adapters, virtual processor and memory allocations, and other required virtual machine configuration parameters.

The next step after the virtual machine is created on the destination node is the transfer of virtual machine memory pages from the source node to the destination node. This is an iterative process since the virtual machine continues to execute on the source node as the memory data is transferred to the destination node. In order to ensure that data is not lost, Hyper-V tracks all modifications to the memory pages on the source node and continues to transfer the memory pages to the destination node until an iteration threshold is reached or all modified memory pages are copied successfully to the destination node.

Then, the virtual machine is paused on the source node, remaining modified memory pages are copied to the destination node, and the processor register and device state is transferred from the source node to the destination node. At this stage, the Live Migration process can no longer be cancelled.

During the short period of time when the virtual machine is paused, the virtual machine storage control, including virtual hard disks and pass-through disks, is also assigned to the destination node. At this point, the virtual machine is in a consistent state and resumes execution on the destination node.

The next step in the process is to force the physical network switches to update their tables with the new port to direct the network traffic for the virtual machine. The new port is the physical network switch port that connects the destination node to the physical network. Finally, the virtual machine information is removed from the source node.

Live Migration Considerations

The time frame that the virtual machine pauses during Live Migration is minimized to avoid exceeding application and TCP network connection timeout values that could result in a service interruption. Therefore, it is crucial to fully understand and test the configuration of any Hyper-V failover cluster that will support Live Migration. In particular, you should use high-speed network connection links, with a minimum of 1 Gbps Ethernet, to optimize the memory and state data transfers between the source and destination nodes. Additionally, you must be aware that a Live Migration is not an instantaneous event, and that the amount of time needed to complete the migration process is dependent on several parameters:
  • Large virtual machine memory size increases the time it takes to complete a live migration, since more active memory pages have to be transferred between the source and destination cluster nodes.
  • High virtual machine activity level increases the potential for more memory page modifications that result in additional iterations to create a consistent memory state on the destination node.
  • High utilization rates on the source and destination nodes extend the time frame required to complete the live migration process.
  • High-speed network bandwidth and throughput decrease the amount of time required to transfer the virtual machine memory page and state information between the source node and the destination node.
  • The transfer of a logical unit number (LUN) between the source and destination node also affects the migration timeline. It is possible to reduce the impact of this process by using the new Cluster Shared Volumes (CSV) feature in Windows Server 2008 R2.
Although not a requirement, if you want to optimize the shared storage configuration of a Hyper-V Failover Cluster to support Live Migration, you should implement Cluster Shared Volumes. You should also take into consideration that each Hyper-V failover cluster node can take part only as the source or destination in a single live migration at a time. Therefore, in a fully expanded, 16-node failover cluster, you can only perform a maximum of eight concurrent Live Migrations.

Cluster Shared Volumes

Up until Windows Server 2008 R2, only a single failover cluster node was allowed to have ownership of a LUN and access to the data stored on it. In contrast, the new Cluster Shared Volumes feature in Windows Server 2008 R2 allows multiple cluster nodes to concurrently access a LUN on a shared storage system while providing a consistent file namespace to all cluster nodes. Therefore, to a virtual machine, attached VHDs appear as if stored on an individual LUN. However, all virtual machine VHDs can reside on a single LUN, and every cluster node can have access to the volumes using the same fully qualified path. By default, Cluster Shared Volumes are created as directories beneath a root folder named ClusterStorage. However, you can modify the root folder name as required in your environment.

Some of the other benefits that Cluster Shared Volumes provide are: NTFS compatibility (no requirement to reformat disk media), support of SAN or iSCSI-connected storage systems, and elimination of drive letter limitations that are encountered with Windows Server 2008 Hyper-V.

Working With Replicas in Hyper-V 3.0

It's no big secret that Hyper-V 3.0 is loaded with new features, but the feature that I am probably the most excited about has got to be Hyper-V replica. In case you aren't familiar with the new Hyper-V replica feature, it is a disaster recovery feature that gives smaller organizations the ability to protect themselves in a way that is similar to what large organization might use. In this article, I will explain what you can expect from this new feature, and how to use it.

Clearing the Confusion

One of the things that I have noticed ever since the Hyper-V replica feature was first announced is that there seems to be a lot of misconceptions about what the Hyper-V replica feature is and what it does. That being the case, I want to start out by trying to clear up some of the confusion.

One of the big problems with server virtualization is that a server crash can result in a major outage. In a physical data center if a server were to fail then the workloads that were running on that server would cease to be available. Obviously this would be an inconvenience, but it probably would not be a major catastrophe. That isn't necessarily the case in a virtual data center however. In a virtual data center a single physical machine often hosts multiple virtualized workloads. If a host server fails then all of the virtual machines that were running on the server also fail. That's why the failure of a single physical server can result in a major outage in a virtual data center.

The only way to prevent this type of outage from occurring is to use clustering. In a clustered environment a group of host servers work together to provide redundancy. If a host server were to fail then the virtual machines that were running on that host server can failover to another server within the cluster. That way the virtual machines can keep running in spite of the hardware failure.

Failover clustering is essential to keeping virtual machines online in a production environment. In fact, Microsoft has supported the use of failover clustering for virtual machines ever since Hyper-V was first introduced with Windows Server 2008.

Failover clustering still exists with Windows Server 2012 and Hyper-V 3.0, but it has evolved significantly since its first incarnation several years ago. It is this evolution that seems to be contributing too much of the confusion around the Hyper-V replica feature. Hyper-V 3.0 contains a number of different redundancy features. The reason why Microsoft has introduced so many different features for protecting virtualized workloads is that some features are better suited to larger environments, while others are better suited to smaller organizations.

With that said, failover clustering and the Hyper-V replica feature appear very similar on the surface, but have completely different purposes. Failover clustering is designed to do two different things. First, it protects the virtual machines against hardware failures. As I previously explained, if a clustered node fails then the virtual servers can be failed over to another node in the cluster where they can continue to run.

The other thing that failover clustering is good for is facilitating system maintenance. Just like any other type of server, virtualization hosts need maintenance from time to time. For example, you might occasionally need to install a service pack or upgrade a host server's memory. Failover clustering allows maintenance to be performed on the host servers in a nondisruptive manner. Workloads can be live migrated to another node in the cluster so that the target node can be taken off-line for maintenance.

These are the two things that failover clustering is really good for. It has been possible to perform failovers ever since the days of Windows Server 2008. The ability to live migrated virtual machines for maintenance or load balancing purposes was introduced with Windows Server 2008 R2 and Hyper-V 2.0.

I mentioned that failover clustering has been significantly enhanced in Hyper-V 3.0. The enhancement that has undoubtedly received the most attention is that unlike previous versions of failover clustering, Hyper-V 3.0 does not require the use of shared storage. Previously all of the nodes in a failover cluster had to be tied to a centralized storage pool so that all of the host servers in the cluster would have access to a common set of virtual hard disk files. With Hyper-V 3.0, this requirement goes away. You can still use shared storage if you want to, but it is now possible to build a failover clusters without shared storage. This greatly reduces the cost of failover clustering and puts it within reach of smaller organizations.

One of the other big enhancements that Microsoft made to failover clustering is that you can now create much larger clusters than were previously possible. Now a cluster can contain up to 63 nodes. Because clusters can now be so large and because there is no longer a requirement for shared storage, some organizations are beginning to look at the idea of geographically dispersed clusters. In other words, some of the cluster nodes can reside at an alternate data center so that the virtual machines can remain online even if the primary data center is destroyed.

At first this sounds a lot like the Hyper-V replica feature. The Hyper-V replica feature allows virtual machines to be replicated to a remote host. That way, up-to-date copies of the virtual machines exist in a safe place. As much as this sounds like a geographically dispersed cluster though, Hyper-V replicas do not use clustering.

As previously mentioned, one of the big reasons for creating a cluster is so that virtual machines can keep running even if a host server fails. The Hyper-V replica feature does not allow virtual machines to be automatically failed over to a remote data center as is possible with a true clustering solution. Instead, the remote site is only used for safe keeping of a copy of the virtual machines. If necessary the replica can be mounted and used, but doing so requires manual intervention. It is not an automatic process.

So why would anyone use the Hyper-V replica feature as opposed to building a failover cluster that spans multiple data centers? For one thing, using the replica feature is less expensive than building a distance cluster. When you build a cluster that spans multiple data centers you typically need multiple cluster nodes in each location in order to facilitate data center level failovers and to prevent split brain syndrome across the data center boundaries. In contrast, the Hyper-V replica feature can be used with only a single server in the remote site.

Another important distinction is that the Hyper-V replica feature uses asynchronous replication. This makes it ideal for low bandwidth environments or for use across an unreliable Internet connection. In contrast, failover clustering requires that heartbeat information be exchanged among cluster nodes in a timely manner. This is the only way that the nodes in the cluster are able to determine which cluster nodes are up and running at any given moment. Clustering is sensitive to latency, whereas the Hyper-V replica feature is much more forgiving in high latency environments.

Solving Active Directory replication failure

Not long ago a systems administrator approached me about a situation where replication had been reported in DCDiag as disabled. He resolved it by using RepAdmin to enable replication again: 

Repadmin /options –DISABLE_INBOUND_REPL
Repadmin /options –DISABLE_OUTBOUND_REPL

Despite the fix, however, the admin still wanted to know how replication became disabled in the first place. He believed something -- rather than someone -- was responsible and wanted to get to the bottom of it. In this article, we'll look at ways administrators can find the cause of disabled replication.

Using RepAdmin /Options

The obvious reason for replication being disabled is that someone used the RepAdmin command to disable inbound and/or outbound replication (and that person may not want to admit to the oversight).

Replication is controlled by the Options attribute on the NTDS Settings object as shown in the following table. The Options attribute value is found in ADSIEdit by browsing to Configuration → Sites → <Site Name> → Servers - <Server Name> → NTDS Settings.

  RepAdmin Option         NTDS Settings/Options attribute value
  Enable Inbound and Outbound 1
  Enable Inbound, Disable Outbound 5
  Enable Outbound, Disable Inbound 3
  Disable Inbound and Outbound 7

Here is what it looks like when you disable or enable replication via RepAdmin using the /Options switch. Note that the minus (-) character in front of the option indicates a negative disable or enable. To disable these values, use the plus (+) sign: +DISABLE_INBOUND_REPLICATION, for example.

Enable both inbound and outbound replication:

C:\>repadmin /options wtec-dc1
Current DC Options: IS_GC

Disable replication, outbound only:

C:\>repadmin /options wtec-dc1 -disable_outbound_repl
Current DC Options: IS_GC DISABLE_OUTBOUND_REPL
New DC Options: IS_GC

Disable replication, inbound only:

C:\>repadmin /options wtec-dc1 +disable_inbound_repl
Current DC Options: IS_GC
New DC Options: IS_GC DISABLE_INBOUND_REPL

Disable replication, inbound and outbound:

C:\>repadmin /options wtec-dc1 +disable_outbound_repl +disable_inbound_repl
Current DC Options: IS_GC
New DC Options: IS_GC DISABLE_INBOUND_REPL DISABLE_OUTBOUND_REPL

These commands remain in effect until changed. That is, if you turn on the Disable inbound repl feature, it will remain on (i.e., inbound replication is disabled) until you enable it again using the –disable_inbound_repl command).
There are several reasons why you would want to do this. One reason would be if a report such as RepAdmin /replsum /bysrc /bydest /sort:delta shows that replication has not happened in the past 60 days (tombstone lifetime). Then you would want to disable outbound replication. Of course, if you have strict replication enabled you will be OK, but it's better to be safe than sorry in this instance.

The RepAdmin command is quick and easy. Note that in this case there is really no need to disable inbound replication since the danger is in replicating outbound. Still, I suggest that you play it safe and do both until you determine the existence of lingering objects.

In addition, if you suspect corruption or issues with a domain controller that you don't want replicated, this command is an easy way to prevent replication from that source. Remember, you can remotely execute RepAdmin. And the DCList option in RepAdmin can be used to specify a single DC, or an asterisk (*) can be used to specify all DCs.

Authoritative Restore and lag sites

Authoritative restoration is used to move the Active Directory back in time by taking a singe system state backup from an earlier date, stopping replication on a DC, then restoring the backup using NTDSUtil's Authoritative Restore feature.

When it boots into normal mode and replication is enabled (using RepAdmin), this copy of the Active Directory is pushed out as authoritative and all DCs get a copy. Since you typically want to at least disable inbound replication before starting this -- and then enable it again -- it's easy to forget after the restore that you need to re-enable replication.

Lag sites can be another cause for replication failure. Lag sites are scheduled to replicate only once or twice per week to provide a sort of online backup for a quick authoritative restore.

In case of a disaster recovery situation, such as deleting an OU, it's important to disable replication on the lag site DC(s). Some prefer to simply keep replication disabled on the lag DC(s) and manually re-enable it when they want replication. Again, it's easy to forget that it was purposely disabled. You can view the status of the replication options for all DCs in the forest via RepAdmin as shown below. Note that DC01 and DC04 show only the IS_GC option set. However, DC03 shows that DISABLE_INBOUND_REPL is also set.

Repadmin /Options *

repadmin running command /options against server hpqbox-
Current DC Options: IS_GC

repadmin running command /options against server HPQBOX-

Current DC Options: IS_GC DISABLE_INBOUND_REPL

repadmin running command /options against server HPQBOX-

Current DC Options: IS_GC

RepAdmin /Options command will list all the options set, and among those options are whether DISABLE_INBOUND_REPL and DISABLE_OUTBOUND_REPL are enabled. If those options do not show up, then they are enabled. If you suspect that replication has been disabled, or if you see events or DCDiag reports stating it's disabled, then produce a quick report using RepAdmin /Options * to determine if any other DCs have replication purposely disabled.

In the case of the admin who claimed that replication stopped without someone specifically disabling it, an updated sequence number (USN) rollback could cause replication to be disabled on a DC if it is done improperly.