In this interview from QCon London 2008, Amazon Web Services (AWS) Evangelist Jeff Barr discusses SimpleDB, S3, EC2, SQS, cloud computing, how the different Amazon services interact within an application, the origins of AWS, SimpleDB and Microsoft SQL Server Data Services, globalization of the AWS cloud, the March AWS outage, SimpleDB Stored Procedures and converting between AMIs and VMWare.
Watch Jeff Barr Discusses Amazon Web Services (14 minutes).
From the interview:
Interestingly enough, what happens is that developers start with one and then they realize that these services, although you can get totally good value out of an individual one, you can start stacking them up and having them play off against each other. And now we're starting to see that neat cloud level architectures happen, where we hear from entrepreneurs that they've built an entire website in the cloud where they'll literally say: "We've got this very very complex system put together and we don't own any of our own servers." We love to hear that from developers where their capital costs are essentially zero and everything that they build is in the cloud and scalable. If their business takes off and they need lots of servers they are going to pay accordingly; if their business stays at a moderate level then they don't incur any cost for services they reserve but didn't use.
On how the different AWS components interact:
A couple of different ways. First, there is a common authentication service that goes across all the different services. So once you create your Amazon developer account, you use the same private key, public key mechanism to access those services. The services are running inside the same datacenter so there is no charge for bandwidth between services inside the datacenter. So a good example is if you have data stored in S3, you're going to pull over to EC2 for processing, do all that processing, send it back to S3 -- that bandwidth back and forth doesn't cost us, so we don't charge developers for that bandwidth. So then a common payment mechanism is another common aspect of the services, and then finally what happens is developers will then put these together.
A very common architecture is to use the Simple Queuing Service as the messaging between different parts of a scalable app. So a very common one I talk about all the time is Podango and their podcast processing. So they have a number of different functional units running on different EC2 instances and then each different kind of functional unit, be it a transcoding or assembly or different kinds of processing, each of those is driven by a separate queue and there is one or more EC2 instances pulling off of each queue. If a queue is getting too busy, if it's taking too much time to do work they can simply ramp up the number of EC2s working on a given queue, makes it very easy to scale. They can be functional at a very very low level of processing -- if they only have a few podcasts per hour they can process that; if they get thousands or tens of thousands, the queues are going to get a little bit bigger, the system automatically senses how busy the queues are and then scales up in response.