In his new book "Amazon Web Services: Migrating your .NET Enterprise Application", Rob Linton provides a comprehensive guide to Amazon Web Services (AWS) in a form of a practical step-by-step guide to migrating existing .NET (or three tiered applications) to AWS. It covers a wide range of topics from what AWS is and is not and legal issues of AWS usage to description of AWS components, to impact on existing applications architecture when moving them to AWS.
The book is organized in 10 chapters:
Understanding Amazon Web Services, describes what AWS is and is not, including brief overview of main AWS components, and examines the drivers for AWS adoption. It also discusses AWS impact on applications and points out which application architectures can benefit most from AWS. Finally, the book considers some of the questions and issues that surround cloud-based offerings such as AWS.
Mapping Your Enterprise Requirements against Amazon's Offerings provides additional details about each of the services in AWS, including Simple Storage Service (S3), Elastic Block Store (EBS), Elastic Compute (EC2), Virtual Private Cloud (VPC), CloudWatch, Elastic Load Balancer (ELB), and Relational Data Store (RDS) and shows the role each can play in the application architecture stack.
Getting Started with AWS and Amazon EC2 provides step-by-step instructions on how to set up an actual AWS account and start EC2 instance using both - AWS Web Console and AWS command line tools.
How Storage Works on Amazon discusses how Amazon AWS handles storage and outlines differences between EBS and S3. It also provides step-by-step instructions on how to manage storage from the command line and configure the storage for the sample application.
Amazon's Approach to Networking covers the way Amazon manages networking as well as how to configure a network from the ground up on Amazon AWS. It also provides details on capabilities of Elastic IPs, ELBs, VPCs, and CloudFront components of AWS.
Migrating your Data and Deploying your Code ties it all together by giving a detailed description on how to move both data and code of the sample application to AWS
Amazon's Queuing and Notification Options introduces Amazon's Simple Queue Service (SQS) and Simple Notification Service (SNS) services and shows how its usage can improve a typical enterprise application.
Monitoring and Scaling My Application introduces AWS CloudWatch and Amazon Autoscaling services and shows how to combine them to make our application automatically scale to handle spikes in traffic and withstand failures of individual components. It also outlines architectural changes that might be necessary for application to be able to leverage these capabilities.
Testing and Maintaining the Application describes some approaches to supporting an application once it is up and running in AWS. It also describes features and usage of Grinder - AWS standard AMI - to help implement and manage load testing. Finally it outlines some of the options for patch management, and production releases of new versions.
PACKT provided Infoq readers with an excerpt from chapter nine of the book - "Monitoring and Scaling My Application". InfoQ had a chance to interview Rob.
Infoq: In your book you are calling AWS an IAAS and then explaining that "this means that they provide basic services such as networking and storage, but don't provide higher-level services". Although this is true for such components as EC2 or Elastic Block store, is it really true for such components as S3, Amazon Relational Database Service, or CloudWatch?
Rob Linton: You’re probably right, there is definitely some overlap of these services into the PAAS space. When I think of PAAS I like to think of it in terms of Azure, Amazon Beanstalk and Google Apps. Deploying applications into these environments really constitutes a re-write of key components. While there is certainly a component of this when migrating applications to AWS, there is more of an emphasis on re-architecting how the existing components fit together rather than a re-write of each component. An example of this is RDS, while it is certainly a value add service, if your application was using MySQL or Oracle in your enterprise, then other than re-architecting how your other components connect to your RDS instance, there is virtually no change to the application on AWS.
Infoq: You define very well advantages of IAAS vs PAAS "In my opinion, the biggest reasons are flexibility and control. PaaS services effectively 'lock' you into their platform. Applications designed for a particular PaaS vendor are not transparently transportable to other PaaS vendors." But avoid talking about PAAS and/or SAAS advantages. Do you see any?
Rob Linton: Another good point, you’re right in that I tend not to see the advantages of PAAS over IAAS. I personally am much more in favor of an environment that allows me to switch vendors and allows the control and flexibility to architect solutions in the cloud. An architected solution always requires something ‘out of the box’ or unique, and the thought of locking myself into a particular PAAS vendor with a fixed set of services would be difficult for me. However I can see how a green field solution that fits a PAAS offering could benefit from the ease of deployment and managed services that PAAS/SAAS provide. I just worry that these architectures will hit a wall at some point in their lifecycle with nowhere to go.
Infoq: You provide a great discussion on which application architectures are well suited for AWS and which are not. Any advice for people that have applications not well suited? Refactoring strategies, that worked for you?
Rob Linton: My advice to people with architectures that are not well suited to AWS is not to try to put them into AWS in the first place. There is a lot of hype surrounding the ‘cloud’ at the moment and it is easy to get sucked into the ‘silver bullet’ syndrome where you think the cloud will fix everything. However, that being said, the biggest issue at the moment is latency. If you can re-architect the front-end of legacy applications into a web experience (which is designed to handle network and latency issues natively) then by all means go for it. There are also some use cases such as batch processing and test/dev environments which are particularly suited to AWS.
Infoq: You supply a wealth of information on AWS legalities. One question, that does not seem to be answered although is the issue of sensitive company’s data. Can, in your opinion, it be kept at AWS?
Rob Linton: That is actually a really good question, and one that hits at the core of cloud computing. One way of answering that would be to look at how amazon.com use AWS to run their bookstore. Amazon.com now run 100% of their infrastructure in AWS, however they do not store anything in S3 unless it is encrypted, which is an interesting story. S3 is a highly secure and available object store, yet even amazon.com, the makers and designers of AWS, encrypt their data before it ever touches an S3 bucket. So in answer to your question, yes I do believe sensitive data can be kept in AWS, however it is up to each individual user of AWS to put in place procedures to secure that data, including encryption.
Infoq: In your book you are separating S3 from Elastic Block storage (EBS) functionality by recommending usage of S3 for data backup and recovery and EBS for the actual disk access. At the same time S3 provides file system used in Amazon Elastic Map Reduce. Any suggestions on usage of S3 file system as a file system for EC2?
Rob Linton: S3 should never be used as a filesystem for EC2 instances. S3 is designed as an object store and lacks the locking and integrity features of a true filesystem. While there is some software out there that allows you to emulate S3 as a filesystem, I have heard horror stories of corrupted data and poor performance when used this way. Think of EBS as real time storage and S3 as near line storage.
Infoq: In chapter 5 you have a great explanation on how AWS networking work, where you are explaining that "it is possible that there may be 1+n routers between all of the servers within your application". In your experience what impact does it have on network communications performance?
Rob Linton: Typically performance within the same availability zone is quite good, however communication between availability zones within the same region can introduce small latencies. I would always recommend however that any architecture should always include redundancy between multiple availability zones. The recent outage of an availability zone in the EU region was a good reminder that AWS do not guarantee the 100% of a single availability zone in their SLA.
Infoq: Can you elaborate on main differences between AWS SQS and ,NET MSMQ? When would I leverage which in the case of AWS deployment?
Rob Linton: Interesting question. In reality while both of these a queue/messaging systems, they are significantly different in both their implementation and usage. MSMQ relies on a fixed messaging number of core servers with a guaranteed delivery with strict SLA’s associated with each message, while SQS has a distributed design and a much more ‘fluid’ set of SLA’s. For example, when getting messages off a queue in AWS SQS, you will not be guaranteed to get all of the messages, in fact what SQS states is that ‘over time’ all messages will be retrieved, but never all at the same time. If I had to sum them up, SQS scales whereas MSMQ does not, however SQS sacrifices some features to achieve this.
Infoq: In Chapter 9 you describing usage of Grinder for load testing applications. What if a company already using other load testing tools, for example, HP load runner? Do they have to switch to Grinder or they can still use the tool of their choice? How can they bring this tool into AWS?
Rob Linton: Actually this is easier than you think. I used Grinder in the book because of its ease of use and its pre-configured Amazon Machine Image (AMI), however any load testing tool could be used. Because AWS is an IAAS stack, implementing a load testing application is as easy as firing up the operating system of choice and installing your load-testing tool. However, because you are in the cloud you now have access to the wealth of other load testing sites like blitz.io, so maybe you might think about moving on from tools like HP load runner.
About the Book Author
Rob Linton is the CTO and founder of Jasondb, a cloud based NoSQL social database. He has been a database professional for the past 15 years, and for the 5 years before that a spatial information systems professional, making him a data specialist for over 20 years. He has just published his first book "Amazon Web Services: Migrate your .NET Enterprise Application to the Amazon Cloud" which is available on Amazon.com. His first company Logicaltech Systalk has received numerous awards and commendations for product excellence, and was the winner of the Australian 2010 iAwards as well as winner of the Asia Pacific APICTA awards. He is a certified Security Systems ISO 27001 auditor, and one of the few people to receive a perfect score for his SQL Server 2008 certification.
He is the co-founder of the Amazon Web Services (AWS) user group in Australia and is a past Vice President of the Australia Oracle Users group. In his spare time he enjoys coding in C++ on his Macbook Pro and chasing his kids away from things that break relatively easily.