The new book VMware Infrastructure 3: Advanced Technical Design Guide and Advanced Operations Guide details both the design environments and operational processes of VMware Infrastructure 3. Written from the practical experience of proven VMware engineers, this book provides IT consultants and system engineers with the insight needed to tackle tough issues in server virtualization such as virtual machine technologies, storage infrastructure, and designing clustered environments. InfoQ is proud to provide both a book excerpt and an interview with the authors Ron Oglesby, Scott Herold, and Mike Laverick. Ron Oglesby is the director of virtualization and x86 services at GlassHouse Technologies and is one of the top experts in the U.S. for the design and implementation of virtualized server environments. He lives in Chicago. Scott Herold is the lead architect of virtualization solutions for Quest Software and has been a pioneer in architecting advanced virtualization solutions for many Fortune 100 organizations in R&D and implementation roles. He lives in Chicago. Mike Laverick is a professional instructor in technologies such as Novell, Windows, Citrix, and VMware, and the sole author of the popular virtualization blog "RTFM Education" (www.rtfm-ed.co.uk), where he publishes a range of free guides and utilities aimed specifically at VMware ESX/VirtualCenter users:
1. What is the biggest thing that has changed in the virtualization market in the last year?
[Scott] - The drive to introduce mission critical applications is finally settling in for many organizations. With increased capabilities provided by a combination of hardware and hypervisor vendors, it is now possible to run larger and more critical applications inside virtualization. This is also driving the need for more advanced and intelligent ways to monitor and manage the infrastructure. We are seeing an increase in third party software vendors trying to rise to the occasion and take advantage of the virtualization wave. Some are successfully acquired, while many silently disappear trying to capture what is quickly becoming a saturated market.[Ron] - While I see a little of what Scott has mentioned, I am seeing the biggest challenge as being the actual virtualization of the corporate environment. There is a ton of promise for virtualization technologies, but even early adopters are having a hard time getting to 50,60 or 70% virtualized in their x86 environments. The biggest reason is that while P2Ving a server is fairly simple and the technology is very simple to use, finding time for existing staff to virtualize hundreds of servers plus getting application owners and server owners to agree to be migrated is still a problem for some reason. So while adoption in small circles is moving to bigger more utilized servers most large organizations are still struggling to virtualize even their underutilized servers.
[Mike] - The biggest change I have seen in the virtualization market is in the people who now attend my training courses. Back in 2003/4 we saw lots of "proof of concept" types and early adopters. In the last 18 months I've seen a huge rise in Windows Admin types getting their hands dirty with VMware for the first time. This is great, it shows that virtualization heading in the right direction - saturation. Where every person in IT has to get to grips with the revolution that began some years ago.
2. How do you see the arrival Microsoft's Hyper-V on the virtualization scene?
[Scott] - Microsoft is going to continue to do what they have historically done best. They have shown that they have a lot of money and just as much patience. They have won a lot of battles that seemed insurmountable such as displacing or disrupting Novell, Lotus/Word Perfect, Domino/Notes, and Oracle. Virtualization is the next logical step for them and they will make a large splash in the small to mid-market spaces, just as they did with their other core technologies and will push deeper into the enterprise over the coming years.[Ron] - If all technology decisions were made purely on the which technology is better, we would all still be using Novell file servers. So while hyper-V is still behind in features and some performance aspects, it is still going to make a splash. Organizations would be negligent if they didn't at least look at and make an informed decision on their short term direction.
[Mike] - Stifles a yawn. The hypervisor war is over, and the virtualization is truly commoditized since VMware made their hypervisor - ESX3i for free some weeks ago. I've seen a rising tide of FUD and misinformation about Hyper-V about what is, and mainly isn't capable of doing. I can see that MS will make headway in the SMB market with Hyper-V, for many it will be the first virtualization they have seen. But I don't see those big corporate accounts that VMware have jumping shop just yet. That said, message to VMware: Ignore Microsoft at your peril, and don't take your customers for granted.
3. RedHat has stated that they foresee virtualization as becoming an included "feature" of server OS's etc. Do you see the industry continuing to move towards bare metal hypervisors or OS's hosted solutions?
[Scott] - That statement seems rather obvious to me, as the Linux kernel has been including it for some time and Microsoft will be including it with Windows Server 2008. As long as VMware maintains a solid market share we are likely to continue seeing both bare metal hypervisors and hypervisors that are included with a base operating system. Heterogeneous management of the multiple hypervisors will be required within the next 2-3 years as organizations try different technologies for different aspects of their virtualization strategy.[Ron] - I think it will be a mix for the foreseeable future. The reality is that different use cases will require different technologies in this space. I could see a world with hosted solutions for maybe small offices, and desktop uses. With bare metal being the choice for the datacenter.
[Mike] - Bare-metal all the way, the fact you have apply a patch to Hyper-V to put "Obama" in the dictionary is an excellent example while general purpose OS and hosted-solutions are not the way to go. That said, it doesn't the case for bare-metal hypervisors - when someone leaves a "licensing bomb" in live code!
4. Virtualization as an identifying term has grown to be used beyond hardware recently with storage, applications, services etc. How do you feel about that? Is everyone jumping on the bandwagon? Do you think some definable infrastructure units such as databases will evolve to a service structure instead of being though of as deployed on individual virtualized machines?
[Scott] - As we all know, virtualization is not new and has been around for longer than 40 years. It quickly became one of the top IT buzzwords, and by definition, many of the ways people are applying it today technically work. It's used as a marketing tool more than anything else. There are many old or even stale technologies using the word "virtual". One of the worst I've heard was in an article I recently read that claimed "IT Department Virtualization." After reading the article I couldn't help but think "Yeah, I called that 'outsourcing' and 'telecommuting' 10 years ago."We are already seeing measurable infrastructure units such as databases, messaging, and web services evolve into service structures through the introduction of "Cloud Computing", which is another buzzword to keep our eyes on over the next 3-4 years. While the use of external cloud computing resources is a bit of a stretch for true enterprise workloads, we are seeing a strong demand for organizations to build internal compute clouds as a "Capacity on Demand" model that uses virtualization as its core operating and management technology.
[Ron] - Everyone is jumping on the bandwagon. I would bet you could toss the word "Virtualization" in front of some of these failing mortgage company names and their stock would rise 10% overnight. The reality is that there is virtualization in application space and server hardware space. But in areas like storage the word is completely overused and unlike server hardware, it has no real definition beyond what any particular vendor wants to assign to it today. For the near future I see no change in this. The reality is that server applications are designed for a single server, and generally are designed in a vacuum where the assumption is that they will run by themselves. Until that changes you will continue to deal with virtualized individual servers.
[Mike] - As Ron and Scott say virtualization isn't new. It's more like "new to you" to a lot of x86 guys. As Ron says this "Cloud Computing" thing is the next big buzz word, whether it turns out to be the next big thing is quite another. Cloud computing is just-another-concept, what we as technologists will be expected to do is to deliver that. It could be that virtualization of servers, desktops and application (ThinApp, SoftGrid) are the very technologies that we will need to deliver this environment. That said, every 5-6 years it seems that our industry needs the "big idea" to sell to C-Class executives. Anyone here remember how "web portal" was the panacea for our ills?
5. Transitioning to VMware Infrastructure specifically, what is the one thing novices think will be easy/solved when implementing the platform but find out work is still involved?
[Scott] - Somewhere over the past few years organizations started to get this idea that virtualization solves every problem in modern IT. While there are significant benefits to using virtualization, just as many problems can be created as solved when a detailed plan isn't put in place and followed as part of a virtualization project. Anyone can build a few ESX servers and run virtual machines; very few can effectively design and build a true virtual infrastructure. Planning is everything when it comes to the proper implementation of virtualization in any environment.[Ron] - If I could point out TWO things that would be great. The first is getting a VM cost model integrated into the procurement process. In big organizations this is a must and always a problem. IT projects are funded, they get consulting dollars, software dollars, hardware dollars, etc. But how will that project 'pay' for a VM when the capacity is already available and there is no asset tag or PO to cut. They need to work out a way to charge for the VMs to replenish used capacity as projects and departments use it. The second thing would (again) be the migrations themselves. Too many times people build their business case on doing X number of servers year 1, Y year two, etc. Then they come to find out that pushback and scheduling problems keep them from reaching those goals and thus the ROI their bosses expected.
[Mike] - Politics. I've never come across a more political technology than virtualization. I touches so many parts of your infrastructure. It also becomes the infrastructure. Heck, perhaps I license that to VMware. VMware is the infrastructure. As recent problems surfaced with VMware's Update 2, people are finally (at last!) beginning to understand the vulnerabilities in the virtual infrastructure. It's a good as the code behind the hypervisor and the management tools, if that crumbles you can be left in a very unpleasant position. The term "virtual infrastructure" was a bit of a throw away term up until August 12th, when people realize how dependent they had become on it.
6. What is the one thing you wished you knew about VMware Infrastructure before you started using it?
[Scott] - I would have liked to have known how addictive it was going to be. I knew I was onto something unique when I first started implementing systems nearly 7 years ago. Once I started, I never looked back. The opportunities that presented themselves due to my involvement with virtualization definitely changed my life, nearly entirely for the better.[Ron] - I wish I would have known how fast the bandwagon was going to fill up. Today I spent a lot of time trying to keep up with who's in the market, what they are doing, what they offer.
[Mike] - Like Scott I wish I'd realize what an enormous impact it was going to have on my career - which have all been positive. Virtualization and VMware especially has been a fantasic escalator career-wise. All I had to do was get on it, work hard - and it has taken me - and is taking me place I never thought I would get to do. Lesson learned: Get into products/concepts early, you have the ability to strike pay dirt. Virtualization doubled my personal income in year. Thank you virtualization!
7. What chapter of the book are you proudest of and why?
[Scott] - I would have to say "Book 1, Chapter 10 - Recovery and Business Continuity". It's a fairly well-known fact that I work for software vendor that develops backup and disaster recovery products, among other things, around virtualization, yet I was still able to take a completely independent approach to writing this chapter. I had the opportunity to share this chapter with individuals who have different takes on how virtualization recovery could be achieved and was able to properly capture the advantages and disadvantages of each methodology. My goal was to truly enable the reader to determine their own unique requirements and make the most informed decision they could. Feedback has proven I was quite successful in my approach.[Ron] - I am a big fan of the Virtual Center and Cluster Design chapter (Book 1- Chapter 4). One of the biggest things I deal with is educating people on their options when it comes to cluster design. Being able to put a real world example in for cluster design options and explaining all the options people have, the pros and cons to all the different configurations really should help out people trying to make this technology scalable in their environment.
[Mike] - Pride comes before a fall they say. I would have to say I am most proud of the performance chapter. Not least because it was the hardest to write. I'm so used to the spoken word; demonstrations and diagrams - that it was really tricky to transpose the fluidity of my explanation from my training courses - into the written word. Part of me still feels dissatisfied with my efforts, and I'm thinking of doing a couple webcasts on my website RTFM Education - to add the live element. I'm also extremely proud to be associated with Ron and Scott in the book - together we have create an almost definitive tome on Vi3.
8. What was the hardest chapter to write?
[Scott] - For me, it was "Book 1, Chapter 9 - Intro to security". There are a ton of whitepapers on this topic so I needed to really think creatively and present this material in a way that was unique, yet still provided a lot of value to the reader. In addition, this was also the last chapter I had written. The amount of time spent writing this book was immense, and I simply started to run out of gas at the end. After a combined total of 816 pages across the entire book, it started to get very difficult finding the time to sit down and write those last few pages.[Ron] - Book 1 Chapter 8 - Managing the Environment. The struggle here was not to overload people with information A: is general IT management and should be in every environment anyway, and b: was still useful for their project. So trying to boil the relevant information down without creating a 150 page dissertation on IT management was pain. I scrapped that chapter 3 times before coming up with the one that eventually wound up in the book.
[Mike] - That darn performance chapter!
9. Going forward what features would you like to see added to VMI in its next version?
[Scott] - Working for an ISV that is heavily involved in VMware integration, I'd like to see a more robust API. VMware requests that all 3rd party vendors use their VMware SDK, but has extremely limited capabilities for network and storage management that force vendors to use alternate methods and external integration components to capture and manually tie data back to the virtual infrastructure. In addition, new capabilities often do not show up in the SDK at all. If VMware wants to maintain its dominance in the virtualization space, they need to provide access to their partners to fully enable them to extend their platform, not limit them with a partially functional API set.[Ron] - personally I would like to see a more modular management framework that new tools can easily snap into. Because of all the acquisitions and the need to get new features into the product quickly VMware has moved into what I call the "launch from" vs the "snap in" management style with VirtualCenter. Essentially they are developing new tools and instead of building them directly into VirtualCenter (the center of the VMware universe) they are either launching these new tools from Virtual Center in their own window, or creating a completely new management interface outside of VirtualCenter. If you are going to tout a management tool, build everything into one place.
[Mike] - Oh heck, my wish list is as long as my arm - where to begin. I know what's coming in the Vi4 - I've seen the roadmaps and signed the NDA's. But generally I would like VMware to pay more attention to the finer detail of VirtualCenter. As a management tools its sometimes felt a bit lacking in this release. So I would like VMware to focus on the small stuff big time - to the degree of every-little-thing-you-ever-wanted-to-as-an-admin has been thought off.