Why the Enterprise Isn't Ready for EC2. And Why it Never Should Be.

I’ll say from the outset that this post is not an attack against Amazon EC2, nor against IaaS in general. It is merely a consideration of use cases, and end games. Specifically, the Enterprise use case for EC2. I personally use EC2, and for my use case it’s bloody great. For startups, it must be a god send. Same for more mature businesses who are built around 1 or 2 internally developed applications, especially if they are web related. It must be a massive boon for students. But for the Enterprise? No.

Let’s start by looking at some application level requirements to get the most from EC2. First, your application needs to be designed for failure – that means scaling out rather than up, loose coupling of components, and statelessness of the endpoint. Second, you don’t want to run an EBS backed instance (and cop all the IO charges), you want to run the operating system on an S3 backed AMI – again that means statelessness within the OS, and complete automation on this layer from instance creation to destruction. Third, you want management and control over the base image itself to ensure your instances come up pre-patched, secure and ready for action.

The problem is very few Enterprise applications are built this way, operating systems as deployed in the Enterprise certainly are not. Why? Are all Enterprise IT employees stupid? All too often the blame for this is misdirected – it’s not the fault of the application owners, the in-house developers, nor the IT infrastructure people. Ladies and Gentlemen, I introduce you to… the accountants!

Ok so the accountants aren’t totally to blame, but they are a big part it because they are the ones who dictate how internal IT groups are run from a financial perspective. Internal IT is generally run as a cost center, which means it cannot make profit or loss – the books have to be zero’ed out. But in this world you can’t exactly offset the cost of one thing against the “profit” of another, so all costs are “washed” across everything.

This results in a per server charging model where you are paying for a _lot_ more than just some tin and rack space, with humans being the biggest cost – this usually manifests by way of a “hosting charge” or something similar. You can see where I’m going with this… it promotes the monolithic app paradigm. The business gets charged per server by IT, therefore it’s better for them to have a monster app on a single 32 core box rather than a distributed app running on 8 x 4 core boxes. That being the case, that hardware needs to be bloody reliable. And since the cost of the hardware is dwarfed by the “hosting charge”, why wouldn’t you buy n+1 everything.

Hence internal developers are not encouraged to build loosely coupled, scale out applications that are designed for failure. And nor are the vendors who get their requirements from Enterprise application owners. Instead, both internal developers and many application vendors design for scale up scenarios. I have sat on internal infrastructure governance boards for a long time, if I had a dollar for every time I saw hardware requests come through for massive boxes that were justified with “future growth” type statements when I questioned why the need for such power given their historical performance data, I could take the rest of the year off. There is not a mentality of start with what you need now, and scale _out_ later, because the internal IT charge back makes this cost prohibitive. It’s not because these people don’t know better. It’s just the economics of the situation.

But the thing is, why would you bother redesigning and recoding all these apps to work optimally in EC2 when other options are upon us? If application re-working of any scale is going to be undertaken, it should be done so with a firm view towards PaaS – not IaaS as Amazon provides it (note: EC2 is _vastly_ different from the likes of the Savvis IaaS, which much more closely resembles what Enterprise hardware looks like today).

Why bother coding your app to handle failure of an instance, why bother automating all the OS layer configuration and instance creation / destruction, why bother with all the management overhead of the OS… when you could instead target a platform that takes care of all that stuff for you?

Microsoft won’t have done too badly in the end, at least they had the foresight to concentrate on the end game with Azure rather than limp into the game late with an EC2 clone. VMware obviously saw this coming and took the necessary steps that lead to VMforce. Google have only ever had App Engine, the RoR crowd have had Engine Yard for years now. Think PaaS hasn’t arrived yet? Think again!

So back to the title of this post. All the consultants and Web 2.0 developers in the world can laugh at the unreadiness of the Enterprise for EC2. Noses in the air, fingers waving, they don’t fully appreciate the reasons why things are as they are. But it doesn’t make an iota of sense for the Enterprise to start reworking everything to fit into EC2’s model. Rework things, we should. But for the platform, not for the infrastructure.


Tags: , , ,

One Response to “Why the Enterprise Isn't Ready for EC2. And Why it Never Should Be.”

  1. Tweets that mention Why the Enterprise Isn’t Ready for EC2. And Why it Never Should Be. -- Topsy.com Says:

    […] This post was mentioned on Twitter by Steve Chambers, Dwayne Melancon, vinternals, ewindisch, Jon Greaves and others. Jon Greaves said: fantastic post at vinternals on "why the enterprise isn't ready for EC@, and why it never should be" http://is.gd/cdDOo very well done!! […]

Comments are closed.

%d bloggers like this: