Archive for the ‘Live Migration’ Tag

Real world scenario issues with VMQ   Leave a comment


Last week Microsoft published part 1 in an article series about VMQ, detailing how VMQ works and trying to clear up some misconceptions about the technology.

It’s well worth the read but the main reason I mention it is because a colleague of mine ran into an issue that’s very much related to VMQ.

The customer is a large hosting provider and they experienced poor network performance when doing backups and live migrations over their 10 Gbit infrastructure. Very important to know though is that they use a virtual switch in Hyper-V to provide vNICs  for backup and LM.

As the Microsofts article states, you won’t get 10 Gbit out of a Hyper-V switch:

Many people have reported that with the creation of a vSwitch they experience a drop in networking traffic drop from line rate on a 10Gbps card to ~3.5Gbps. This is by design. With RSS you have the benefit of using multiple queues for a single host so you can interrupt multiple processors. The downside of VMQ is that the host and every guest on that system is now limited to a single queue and therefore one CPU to do their network processing in the host. On server-grade systems today, about 3.5Gbps is amount of traffic a single core can handle.

Their bandwidth was somewhat lower, around 3 Gbps, but that’s most likely due to having older hardware.

I’m not sure how they’re going to resolve this but my suggestion was to use separate, physical, NICs for backup and, if needed, for LM.

As I don’t have enough information about how they’ve designed their Hyper-V environment I’m not sure if they’ve scaled up or out. If they scale out, bandwidth for LM should be less of an issue as fewer VMs lives on each host but at a certain point you’re still going to need bandwidth (unless you plan on patching your hosts continuously).

The takeaway from this is that when designing high-end environments, it pays to know the nuts and bolts of the technology you’re using.

Posted 20 September, 2013 by martinnr5 in FYI, Technical

Tagged with , , , , , ,

TechEd Europe 2013 – Hyper-V   Leave a comment


I just counted and I have over 11 pages of handwritten notes1 from the sessions I went to so it’ll take me some time to compile them all into something coherent. This, in addition to the fact that my current theme of the blog doesn’t lend itself very well to long blog posts (though I normally try to go for quality over quantity), means that I’ll chunk the posts into a number of categories; Hyper-V, Networking and Storage as these are the main areas I focused on. Most likely I’ll end up with a “catch-all” post as well.

As the title of the post implies, let’s start with Hyper-V. Now, I know that there are numerous blog posts that cover what I’m about to cover but I’m summarizing the event for both colleagues and customers that weren’t able to attend so bear with me.

Hyper-V Replica

In 2012 R2 Hyper-V Replica has support for a third step in the replication process. The official name is Extended Hyper-V Replica and according to Ben Armstrong it was mainly service providers who asked for this to be implemented although I can see a number of my customers benefitting from this as well.

In order to manage large environments that implement Hyper-V Replica Microsoft developed Hyper-V Replica Manager (HRM), an Azure service that connects to your VMM servers and then provides Disaster Recovery protection through Hyper-V Replica on a VMM cloud level.

This requires a small agent to be installed on all VMM servers that are to be managed by the service. The VMM servers then configure the hosts, including adding the Replica functionality if needed (even the Replica Broker on clusters). After adding this agent you can add DR functionality in VM Templates in VMM.

Using HRM you can easily configure your DR protection and also orchestrate a failover including, among other things, the order you start-up your VMs and manual steps if needed. The service can be managed by your smart phone and there are no plans to allow organizations to deploy HRM internally.

Only the metadata used to coordinate the protection is ever communicated to the cloud, using certificates. All actual replication of VMs are strictly site to site, between your data centers.

If you manage to screw up your Hyper-V Replica configuration on the host level you need to manually sync with HRM to restore the settings. At least for now, R2 isn’t released yet so who knows what’ll change until then.

Finally; you are now allowed a bit more freedom when it comes to replication intervals; 30 seconds, 5 minutes and 15 minutes. Since there hasn’t been enough time to test, Microsoft won’t allow arbitrary replication intervals.

Linux

Linux guests now enjoy the same Dynamic Memory as Windows guests. Linux is now backed up using a file system freeze that gives a VSS alike functionality. Finally; the video driver when using VM connect to a Linux guest is new and vastly better than the old one.

I never got any information on what distros that’d be supported but my guess is that all guests with the “R2” integration components should be good to go.

Live Migration

One big thing in 2012 R2 is that Live Migration has seen some major performance improvements. By using compression (leveraging spare host CPU cycles) or SMB/RDMA Microsoft have seen consistent performance improvements of at least 40%, in some cases 200 to 300%.

The general rule of thumb is to activate compression for networks up to 10 Gbit and use SMB/RDMA on anything faster.

Speaking of Live Migration I’d like to mention a chat I had with Ben Armstrong about Live Storage Migration performance as I have a customer who sees issues with this. When doing a LSM of a VM you should expect 90% of the performance you get when doing a unbuffered copy to/from the same storage your VMs reside on. I might do a separate post on this just to elaborate.

Storage

In 2012 R2 you can now set Quality of Service for IOPS on a per VM level. Why no QoS for bandwidth? R2 isn’t finished so there’s still a possibility that it might show up.

One big feature, that should have been in 2012 RTM if you ask me (and numerous others), is that you can now expand a VHDX when the VM is online.

Another big feature (properly big this time) is guest clustering through a shared VHDX, in effect acting as virtual SAS storage inside your VM (using the latest, R2, integration services in your VM). More on this in my storage post though.

One more highly anticipated feature is active de-duplication of VHD/VHDX files when used in a VDI scenario. Why only VDI? Because Microsoft haven’t done enough testing. Feel free to de-dupe any VHDX you like but if things break and it’s not a VDI deployment, don’t call Microsoft. More on this in my storage post as well.

The rest

Ben Armstrong opened his session with the reflection that almost no customer is using all the features that Hyper-V 2012 offers. To me, that says a lot about the rich feature set of Hyper-V, a feature set that only gets richer in 2012 R2.

One really neat feature that shows how beneficial it is to own the entire ecosystem the way that Microsoft does is that all Windows Server 2012 R2 guests will automatically be activated if the host is running an activated data center edition of Windows Server 2012 R2. There are no plans to port this functionality to older OS, neither for guests nor for hosts.

In R2 you now get the full RDP experience, including USB redirection and audio/video, over SMBus. This means that you can connect to a VM and copy files, text, etc. even if you do not have network connectivity to the VM.

2012 R2 supports generation 2 VMs. Essentially a much slicker, simpler and more secure VM that has a couple of quirks. One being that only Windows Server 2012/2012 R2 and Windows 8/8.1 are supported as guest OSes as of now.

The seamless migration from 2012 to 2012 R2 is simply a Live Migration. The only other option is the Copy Cluster Roles wizard (new name in R2) which incurs downtime.

You can export or clone a VM while it’s running in 2012 R2.

Ben was asked the question if there’d ever be the option to hot-add a vCPU to a VM and the reply was that there really is no need for that feature. The reason being that Hyper-V has a very small penalty for having multiple vCPUs assigned to a VM. This is different from the best practices given by VMware where additional vCPUs does incur a penalty if they’re not used. The take-away is that you should alwasy deploy Hyper-V VMs with more than one vCPU.

During the “Meet the Experts” session I had the chance to sit down with Ben Armstrong and wax philosophically about Hyper-V. When I asked him about the continued innovation of Hyper-V he said that there’s another decades worth of features to implement. Makes me feel all warm inside.

Conclusion

As there haven’t been a lot of time between Windows Server 2012 and the upcoming release of 2012 R2 (roughly a year) the new or improved features might not be all that impressive when it comes to Hyper-V but I honestly think that they’re very useful and I already have a number of customer scenarios in mind where they’ll be of great use.

Not to mention that Hyper-V is only a small slice of the pie of new features in R2. I’ll be writing about some of the other slices tomorrow.

Most importantly though, and this was stressed by a fair number of speakers, is that Windows Server and System Center finally, for the first time ever, have a properly synchronized release schedule, allowing Microsoft to deliver on the promise of a fully functional solution for Private Clouds.

I agree with this notion as it’s been quite frustrating have to wait for System Center to play catch up with the OS, or vice versa.

With that, I bid you adieu. See you tomorrow.

1. Yeah, I prefer taking notes by hand as it allows me to be a lot more flexible, not to mention that I’m faster this way. I took advantage of the offer at this TechEd and bought both the Surface RT and Pro though so who knows, the next time I might use one of those devices.
%d bloggers like this: