Tag Archives: vmware

P2V migration servers with “small files syndrome”

The past weekend I was helping a customer perform several P2V of physical servers to VMware virtual machines. Those were critical applications with small downtime window.

Generally P2V are pretty straightforward and easy to do. Given enough time, any x86 physical workloads can be migrated. In the ones I just did, all I had was 4 hours each. On the surface, 4 hours for about 120GB worth of data was achievable over 1Gbps networks. For these, we needed a lot more time. We did a migration on one of the machines which had no issues with a tight window, it took 16hrs!

The cause, nearly 1 million small files!! Small files are the bane of storage, which are a pain to copy.

We had to use different techniques for the P2V. Block based copying was not an option, and so it would have to be just regular file based copying.

The solution? P2V the OS disk only, then copy the other drives using FastCopy.

FastCopy was much more efficient that standard file copying utilities. It is known to be very successful in copying small files, and it’s true! Even for copying over the network! Though it’s still far from being as fast as block copy, but it was certainly faster than leaving it to regular copy/migration means.

the beauty of virtualization – part 1 – introduction

many people in the IT industry should have heard about virtualization, and for the rest of the world… the closest thing they can think of is probably the movie – Avatar. uhm… not the same…

I’m talking about computer virtualization here. And so what the heck is it really about? To senior management, it’s a promise of lowering cost, faster ROI, better flexibility of the infrastructure, simpler management of IT infrastructure.

To the guys on the ground, it means, faster build and release to production, quick and easy back out plan for patch maintenance, fast cloning of production data for beta testing, no more hair ripping experience due to hardware refresh, and much much more.

Right, so these are some of the benefits that I just touched on. So, still what really is virtualization?

To virtualize a computer/server is simply just an extraction of the running system from a physical box, into just a bunch of files and processes. In a traditional approach of having 1 OS instance per physical box, we move into the realm of having multiple OS instances per physical box. No, this is not multiboot, this is having more than 1 OS instance running simultaneously in each physical box. Imagine, one box, be it a regular desktop or a high end server, running multiple OS at the same time. Each OS instance runs unaware of the other in the same box. They are independent of each other, they can work together, they can do anything just like how they will be as in the traditional model.

For example, you can have a Redhat Linux instance, Windows 2008,  Windows 2000 Server, all running as virtual machines (VM) in the same physical box. You can ssh into the Linux VM and do anything as you may as a regular Linux machine. You can RDP into the Windows VMs, check on running services, eventlogs ,etc… just like any other Windows servers.

To the un-initiated who logs into a VM, they will not know that it is a VM. I behaves just like any regular server.

So, since the VM does the same things as the physical counterparts… what’s the point?

Now, think about this for a moment. In nearly all data centers today, going with the 80-20 rule, we can easily find 80% of the servers running with very low utilization most of the time. Some probably only spike up once a week or month even. So, the rest of the days, it’s just there doing nothing much, and just sucking up power.

So, let’s say we take 10 of such servers, virtualized them and consolidate them all into just 1 box. If done correctly, you can expect the performance of these 10 servers to remain the same. And there you have it, 90% reduction!! Instead of buying 10 servers, you just need 1. Just 1 physical server to run, maintain and suck less power and utility. Lower up front cost, certainly less ongoing utility bills, and real estate cost. In real life examples, we can see conservative consolidations of over 20 servers into just 1 box!!

This is only the tip of the ice berg we have touched on here. Let me go on a little more before this introduction becomes too much.

One very powerful aspect of virtualization that some may take for granted, but makes all these wonderful concepts possible, is that virtualization creates a whole new hardware platform. I see it as a virtual hardware platform. The virtualization layer basically creates a whole new infrastructure, which only exists logically. What this means is that the hardware that the VM OS actually thinks it sees, are virtual. They do not exist, physically.The result is that we have essentially de-coupled VM from the real server it resides on. We can then take a copy of that VM, and allow it to run on any physical box that has the virtualization layer/engine/OS installed.

Such is not possible with OS running on physical machines. The closest thing we can probably do is pull out the harddisk from one server, transfer it to another server with the exact same hardware, CPU, RAM, NIC, HBA, etc… and then boot up. Even then, there are some differences, the NIC MAC and HBA addresses are different. If you transfer the harddisk to other boxes, but with different config. You’ll end up with some nightmare to reconfigure all the drivers, which most of the time is a very very risky maneuver.

If you are following my thoughts so far, you should now realize that virtualization transforms servers to be something portable. With the right setup, you can easily move VM from one physical box to another, even in different locations, and geography. Think server migration… you can even just copy a VM on to a 2.5″ portable harddisk, take it with you on the plane, fly half way round the planet, and load it up on to another box, and it’s up and running in minutes!!

Again, we are still only scratching the surface. The promise of what else can be done is tremendous, and I will write about them in my future posts.

For those who I’ve lost since “Avatar”, I’ll close off with a little tidbit to help you appreciate virtualization a little more. Some of you may have played old Nintento consoles, the Famicom, Super Famicom, Gameboy Advance, etc… There are emulators out there, which allows you to play these old school games on your computer, be it a Windows, Mac or even the iPhone. This in a way, is virtualization, the consoles have been virtualized so that you can play the games on other type of hardware. Though not exactly the same as server virtualization, I’m hoping it’s something more people can relate to.