Terraform vs. CloudFormation on AWS
SO. If Terraform and CloudFormation were both UFC fighters competing in an AWS Championship Fight...who would win? Okay, okay, I know that's kind of goofy, but you and I both know that X vs. Y conversations between developers may as well be full-blown fights. Granted the fighting is usually nothing more than passive jabs, sarcasm, and snark but they're still fights nonetheless.
Well, one of the big headlining fights of the past couple of years has been Terraform vs. INSERT_IAC_TOOL_HERE
. HashiCorp's young prizefighter came out of nowhere in 2014 ready to duke it out. But, as with all new kids on the block (in technology), it was faced with the usual mix of love and resistance. Early adopters loved it, claiming it to be the savior of our time. The late majority and laggards saw it as another flash in the pan to be ignored.
"Why use this wonky new language (a.k.a HCL) when CloudFormation and other tools get the job done just fine?"
And so the usual back and forth of "10 things I hate about Terraform" and "10 reasons we moved to Terraform" posts began dominating the search engine. Curiosity became love. Caution became hate. Ya know...the usual way of things in technology adoption.
Fast forward to 2018. CloudFormation releases a bunch of jabs in the form of adding more than 80+ updates and new resources to the tried and true framework. They also launched their AWS CDK developer preview to increase its flexibility, just in case you'd rather deal with constructors, types, and OOP over JSON/YAML syntax. By the way, the CDK just generates CloudFormation in the background. (We'll leave the CDK alone for now since it's still new to the scene in terms of adoption and production use).
And then on November 1st, 2018, HashiCorp and Terraform stole the bright lights and crowd's attention with a $100 million dollar investment on a $1.9 billion dollar valuation. Suddenly, it's near impossible to count Terraform, let alone any of HashiCorp's tools, out of the conversation. It's no longer an up-and-coming amateur...it's a true contender ready to compete for the win.
"Well Cole...which one??"
Before we hop into that, let's set something straight here in terms of mindset. One of the biggest barriers to accurately evaluating technology is generally one issue with two faces. First, over-investment in current technology. Second, the threatened job security that new technologies impose. After all, if the young buck comes in and wins it all, a lot of folks are going to wind up retraining. Even though I think most developers are aware on some level that this happens...not everyone clears those biases out before making their choice.
Alright, let's introduce the fighters!
On the left side of the ring we have CloudFormation! The tried and true, battle-tested champion that's been around since 2010. It's stayed the same SO much that it if it ran for politics, it'd probably win the popular vote and the electoral college off of "consistency of character."
Say what you will about it, but even though learning it takes a bit of upfront investment...once you DO...AWS becomes almost an entirely different paradigm. Everything becomes a potential target for automation, convenience, and sanity (since, if everything goes to craps, you can relaunch it in minutes). Additionally, because it's such a seasoned veteran, if you have a problem you can probably find the answer in online communities and forums very easily.
Yes, it's verbose to almost the point of ridiculousness, but at the same time, that creates a massive amount of clarity. Yes, it's all JSON/YAML, but that means you can take anything you create with it and easily write parsers/compilers to meet your needs. Yes, the validation, error catching, and rollback system can be a bit odd...but at least it puts things back in the right place if some half-asleep developer pushes a faulty template update.
On the other hand though, the "haters" have a lot of good reasons. Templates can become long and unwieldy. Managing multiple stacks created from similar templates can become a torturous game of solitaire. Oh, and the feedback loop of waiting 15+ minutes on a deploy, only to see that there was a problem, followed by waiting another 15 minutes for it to rollback, followed by another 15 minutes to redeploy...is painful.
By the way, we're going to skip "vendor lock-in" because basing your decision off of the very slim chance that your company pops up one day and says, "Oh hey Bob, ya know how we have that massive server fleet hosting our high traffic application and services?"
"Yeah Jerry, I know it, I'm working on it right now."
"Great, well guess what? We've decided to migrate everything to Google Cloud Platform LOL."
...isn't A+ decision making.
And on the right side of the ring we have Terraform! The agile, lightning bolt of a fighter that can stun the paparazzi, win the hearts of youth, and knock unprepared opponents out with ONE PUNCH that it refers to as terraform apply
. Between the snazzy branding, powerful name, and open source soul, I'm surprised it's not in tabloids. Oh wait, it is. Just ones for developers.
If you have yet to write anything with Terraform...you should. For all the folks who would crap on HCL, Terraform's configuration language, take a moment and consider the fact that it works so well because...
...it was MADE to write configurations! Sure, sure, sure, I'm glad that I can write some deeply nested prototype chain or OOP pattern to spin up an instance in the CDK, but it's a bit superfluous depending upon your goal. But on the other hand, HCL removes all of the crap and filler from defining infrastructure and lets you get to work. It's simple. It's powerful. And, in my humble opinion, it's far better suited for creating infrastructures over JSON, YAML, and really any programming language out there. Again, if Terraform was a fighter, folks would probably tell it that it was "born to do this."
Unlike a lot of the tools out there, that are maybe an afterthought or some ivory tower abomination, it's very clear that Terraform was made with the developers in mind. Yes, the concept of modules can be a bit wonky to wrap your head around but once you get it, it saves sooooo much time that's otherwise spent on thinking up folder structures. Yes, the concept of state is really weird to get used to if you're used to working with CloudFormation templates...HOWEVER....once you realize that Terraform keeps track of EVERYTHING you deploy and that you can just manage everything from your code editor? HAPPY DAYS. And yeah, while vendor lock-in might be an exaggerated fear, the vendor compatibility is what makes Terraform quite formidable. Being able to provision things like github repos, AWS resources, and connect them in a singular code base and language is incredible.
BUT. On the other hand. Since 2017 there's been 4 major releases, a progression of HCL 1 to version 2, a revamp of how they manage state, and we could keep going here. It's still in version .12 which is fitting because there are and will be breaking changes in the future. Plugins and 3rd party addons will continue to get disrupted because they'll have to account for all of these things. This makes the development ecosystem, beyond HashiCorp official tools, very volatile.
And then there's Terraform's documentation. As is fitting of the rising star, it's the strong silent type. Except, when that's the case for documentation, it's not a good thing. It's extremely lacking in terms of practical examples, thought processes, and the like. Parts of it aren't updated for the most recent changes. Oh, and if you're new, you probably laughed at the installation instructions pointing to a Stack Overflow answer. Granted, it's not hard to install if you're slightly familiar with Linux/Bash...but it's still pretty funny that they did it anyway. It almost fits the modern development ecosystem too well, considering that pointing to Stack Overflow is a cliche.
Alright. So they start duking it out in the AWS Championship Ring. Who wins?
Well, quite frankly, if we base it off pros and cons of the tools themselves? Honestly? I'd say it's a draw. A tie game. An iconic, "Apollo and Rocky knock each other out" scenario.
And why? Well, simply put: they can both get the job done at any level. In terms of working with AWS, they're both more than capable of doing whatever it is you need to do. Yes, Terraform will always lag a bit in terms of adding on new things when AWS adds them, but it's not as if CloudFormation is always ready to go with new services. Yes, managing CloudFormation templates is a task in-and-of-itself, but when something's been around for 10 years, most of the big problems are solved.
"Cole, that's a cheat answer."
Yeah yeah yeah. I know. So, in order to arrive at an answer, I think we have to go beyond qualities of the tools themselves. Because there is a particular referee that can call the right shots every time:
Market Demand
And if we bring in the market demand referee, who's going to win? Terraform will. If you look at any technology job site and compare the listings of Terraform to CloudFormation, well, it's pretty clear. No, it's not a defacto statistic by any means, but on StackOverflow and Indeed.com, there's more than double the listings for Terraform jobs. Go check it out if you want. Do any methodology you'd like. But there's more job openings for Terraform by and far in comparison to CloudFormation.
But...will that be the case 2-5 years from now? Sure, Terraform is in the limelight right now...but you know what they say about how money changes folks. Furthermore, there's plenty of "auto-magical" frameworks out there that were open source darlings that have all but disappeared (Meteorjs anyone?). But then again, on that train of thought, I never thought I'd see Sears go down, Blackberry evaporate, or Yahoo fall from grace.
And so that brings us to...
My Suggestion
I could never understand why bloggers and youtubers would preface their content so much with what I'm about to say...but, I get it now, and I'm going to say it: what I'm about to say is an opinion, not a declaration of war. And here it is:
Learn both, but use Terraform if opportunity and demand allow it.
Why both? Well, if you learn one, learning the other isn't that difficult. Furthermore, it's just basic diversification. Develop with all the latest and greatest aspects of Terraform that will save your wrists and fingers from carpel tunnel. Manage and deploy with Terraform so that you can stop double-double checking your template updates and stacks since "conditional" is often CloudFormation's sarcastic reply to whether or not it will replace something.
BUT. If Terraform does wind up burning out and become the has-been in the gym? Well you'll have good ol' CloudFormation in your back pocket to fall back on.
J Cole Morrison
http://start.jcolemorrison.comDeveloper Advocate @HashiCorp, DevOps Enthusiast, Startup Lover, Teaching at awsdevops.io