Complexity really is a growing problem for people. Every day we are hit with “more of everything” - more software choice, more versions, more systems and more business change. And soon - with more virtualization - more VMs. How many VMs will there be in your organization in 2010? How will they be manufactured and provisioned and will this be done in the same way as it is today? InfoQ spoke with Alexis Richardson, a founder of startup CohesiveFT, about their "Elastic Server On-Demand" product, which aims to simplify working with virtualized application stacks.
InfoQ: Please provide an overview of Elastic Server On-Demand for InfoQ readers.
CohesiveFT: Elastic Server addresses the problem of virtual machine management using self-service and automation. The Elastic Server On-Demand service delivers ready-to-run application stacks to order, for virtual server and cloud formats. It takes the complexity and uncertainty out of assembly and deployment lifecycles by replacing multiple manual steps and scripts, with a single automated process that is both quick and easy to use. We give you a place to create and share servers, or reproduce them from recommended patterns, with speed and quality.
InfoQ. How do I use Elastic Server?
CohesiveFT: Say you have an application, written for Ruby on Rails, and you want to run test builds on your laptop but move to a data centre in production - for instance on Amazon EC2. No problem. On the Elastic Server site you’ll see a collection of portals each catering to a different technology community or product pattern. Choose the Ruby on Rails portal and customize the components you want in your base stack - e.g. Rails 2.0.2, Mongrel 1.1.4, PostgreSQL libraries, and some RubyGems. Now press ‘build’. That’s it - in a few minutes you can be running your Rails server on EC2.
Choices can be saved as a template, so you can reuse the same configuration and run it on your own hardware in other formats, currently VMware, Parallels, or Xen. This is why we call servers “Elastic” - you have one server recipe, but keep the flexibility to reuse it across multiple business cases and technology platforms. Every aspect of this is managed in a consistent way in one place. Additionally, every server has a web console and API for organizing maintenance functions such as network settings, security, audit, and lifecycle.
InfoQ. How does this simplify the inherent complexity in managing virtualized server environments?
CohesiveFT: This example shows a couple of important things. First, the Elastic Server On-Demand service gives you a single point of control and management of an ever increasing number of fiddly and annoying activities that you would rather someone else took care of. These are replaced with one simpler process that always works predictably, enabling the exact virtual servers you want, to order, in a fraction of the time it otherwise takes to provision a server. Secondly, Elastic Server On-Demand is a very flexible system which means it scales with your business need and is not confined to one-offs and niche cases. We all know that automation can deliver better quality control, but can come at the cost of flexibility and user choice. With Elastic Servers, we don’t take away your choice and we don’t lock you in to any virtualization or OS platform.
Fundamentally Elastic Server On-Demand represents a way to build, organize and share a catalog of content that matters in both development and operations. What is this content? Software. Components. Stacks. It’s the file systems and metadata, or as we like to call them ‘magic files’, that enable any given server to perform as required in a given use case. You can think of the Elastic Server portals as a Content Management System (CMS) for software stacks. Like any CMS we let you create your own content and organise it into your own portals. New portals are being created all the time, e.g. for Shindig, SOA stacks, RabbitMQ messaging stacks, Google App Engine tools, LAMP stacks, and people are building and sharing their own servers in the community.
InfoQ. Can you describe how the product is licensed?
CohesiveFT: We offer licenses on a subscription basis - Elastic Server On-Demand is primarily a SaaS business model. We have a free Community Edition with the core 'build, save and share' features. There are commercial Personal and Teams editions which allow private workspaces and provide lifecycle support and VM management. The licenses for these will be based on metred provisioning - so you only pay for what you use, and you can always get capacity that you need. For really large scale provisioning across Enterprise teams and clouds, we have a runbook based approach - again offering customers flexibility.
The Community is really important to us. Community means no matter who you are, you can access our platform for free - but we encourage you to share. You can take your stack 'recipes', capture them, reproduce them as virtual servers automatically, and share servers with the community. You can also load your own components. Then, the commercial editions support people working together. Features include portal authoring tools, security, multi-user access control, and tracking the life of a server image. It's about having one place to manage information for development, testing and operations teams.
We have also open-sourced some software, which is described here.
InfoQ: Can you elaborate on how defining and managing your own stack will help your customers?
CohesiveFT: Customers told us they wanted the ability to create new stacks, make unique copies of them on demand, track them, and to manage changes over time, such as patches. So - whichever virtualization technology or cloud people use, they will use our 'elastic' customization to manage and support business change.
But we also ask: "What is a 'stack'?" People like to talk about 'the stack' or 'the LAMP stack' or 'the SASH stack' as if each of these picks out a unique entity. We think stacks are really a combination of design patterns and operating system builds. There are as many stacks as there are use cases for software. Every business has its own set of use cases - and each may require a custom application stack. Of course any two stacks may 'look alike' - for example two stacks using Spring and Hibernate and MySQL on Linux are likely to be quite similar. But each stack is specific to a business use. Given this, the ability of customers to define their own use cases by loading their own components and defining their own stacks, is essential.
Capturing a stack means you can manage information that can otherwise get lost when components change. While initially project budgets may enable investment of thought and energy into defining stacks to solve a problem, later on people may be too busy to reinforce this work by sharing the knowledge and code properly. This loss of critical information can cause maintenance and lock-in costs to climb over time, and the ability to modify or reuse code to new purposes to decrease. So - by capturing knowledge in Elastic Server, that problem is solved.
InfoQ: How do you deploy a server "recipe"?
CohesiveFT: With Elastic Server, before it is made ready for download, each stack is auto-injected with identity and entitlement metadata, application, O/S and middleware configuration files, and management tooling for setting up the network address, firewalls, logging, and more. In summary, each server is hardened for use.
The Elastic Server On-Demand service will then make the server available for download or deployment it to a staging area - possibly a disk share or mounted SAN cache, or perhaps VMware or Opsware, or for an automated test process to take over. Or the user can tell the service to autodeploy it to the cloud and start it running. As much as possible this is automatic, and only takes a few minutes. Try it out to see for yourself! Once a server is running you may wish to change its instantiation. For example, you may wish to change a port in an XML file. People may do this in three ways: through the web admin tool, through a web API programmatically, or through SSH as root.
InfoQ: Is there any way to manage changes, or re-provision, deployed instances?
CohesiveFT: Sophisticated change management support will be introduced with future editions. Everything is based around the notion that every server has a manifest including unique identifiers suitable for tracking all relevant change. In addition the administration functionality is identity-aware which is a prerequisite for patching.
How does this work? Logically our build system is a giant acyclic dependency (rules) graph. Changes to components (build inputs) ripple through the dependency graph to affect servers (built outputs). Obviously we automate this and in the future it will be used to alert people that servers can be patched.
Reprovisioning is currently naive but sufficient for real cases today. Many companies have sophisticated provisioning, orchestration, workflow and management systems that we expect to integrate with. Obviously this applies to data migration too, in cases where the data is closely coupled to the stack server.
InfoQ: What challenges does Elastic Server On-Demand help solve from an architectural point of view?
CohesiveFT: Elastic Server On-Demand is all about delivering quality results on time, in the presence of constant systemic change. And we think virtualization and the cloud change everything, not just architecture! They will have an impact on many more aspects of life than we can know about today.
That said, there are some architectural points to mention. First off - deploying to the cloud, or getting value from your virtual data centre now - today - immediately. These are issues. We enable portability between the two and across multiple virtualization formats, without pain. That’s an issue too.
Second, from a technical or product architecture point of view, there is deep technology behind Elastic Server’s build system, for addressing dependency management for a really wide class of artifacts. This is highly non trivial when you consider how many more stacks, operating systems and virtualization technologies there are today.
Last but not least, it needs saying that “architects are people too”. They share a desire to make work life easier, to be able to have enough influence over development teams, and develop and enhance their reputation as an architect and technology professional. Elastic Servers are all about getting things done more quickly whether trying out new stack configurations, or provisioning an enterprise data centre. By centralising communication and management of information about stacks, the product cuts out overheads. Using Elastic Server portals leads to reduced response time; improved change quality; lower costs for managing quality control of stacks; and better communication between project leaders and developers e.g. across distributed teams.