Table of Contents

1. Introduction
2. What Are We Going to Do in This Series?
3. The History of CloudFormation
4. So, Why CloudFormation Then?
5. Next Steps
6. Watch The Series Here
7. The Complete CloudFormation Guide Index

Introduction

CloudFormation truly is that game-changing if you're working with AWS because think about it: how do you normally work with AWS? Well, you either go to the console and click tons of buttons and boxes that always seem to be in a different place than last time because their UI team can't seem to settle on a look they like. Or, you're using the command line interface (CLI) which is essential and incredible for scripting but for day-to-day management tasks, it's really the last thing you'd want to use.

So how does CloudFormation differ from those two options? In a nutshell, it lets you do everything in AWS by writing code. All we do is make a single YAML or JSON file, we list all of the things that we want in our infrastructure, and we give it to AWS. From there, CloudFormation takes our list of to-do's and makes all of the things we asked for. It really is that simple.

Now I'm not going to bother rattling off all the benefits, time savings, and all the other reasons to use CloudFormation (let alone infrastructure is code). You've heard about all that more than enough already and, quite frankly, once you get the main concepts down it's pretty self-evident as to why it's so useful. The feeling you get the first time you launch a template and have everything go up, with you doing very little, is pretty amazing. You get all of the same benefits that you get with code but you can apply them to your infrastructure so things like collaboration, version-control, ease of scripting, are all there. The list goes on but I'll stop there because the AWS marketing team does more than enough of this on their own.

What Are We Going to Do in This Series?

In this series, everything we do is geared toward giving you a core understanding that will let you go out into the world and start using CloudFormation for yourself. Everything we talk about and do will be in the same way you'd do it in real life. No 'hello world' examples here!

First, later on in this post, we will go through the history of CloudFormation. It's easy to neglect the origins of a technology because, as technologists, our minds gravitate toward looking to the future. The thing is though, if you know the beginnings of something as important and pivotal as CloudFormation, it's easier to predict where it's going to go next. Then, we'll get into the main concepts of CloudFormation. It's easy to just dive right into step-by-steps and maybe those work sometimes but to truly understand something you need that 50,000 foot view of it. That viewpoint will give you a really solid compass to operate on when the step-by-steps fail you and you'll be able to build things without being dependent on them.

For most of the series though we'll be covering all of the essential nuts and bolts of CloudFormation through the main nine parts of template anatomy including Resources, Parameters, Refs, Functions, Pseudo Parameters, Conditions, Mappings, Transforms, and Outputs. We'll actually put our learning into practice too as we build out a reuseable template for ourselves and complete multiple launches of it so you can see the process the whole way through. Granted it won't be anything big (just a run-of-the-mill security group with some basic rules) but it's the perfect scope to learn all of the crazy syntax semantics, functions, and the like. If we chose something more complicated, we'd end up having to focus on the complications rather than on CloudFormation itself. Instead, with something small like a security group, you'll be able to take the small step towards infrastructure as code in the best, most digestible way.

Like I said at the beginning, all you do with CloudFormation is make a list of everything you want in a JSON or YAML file, and that file is called a template. Even though it's just a simple file, we can do a lot more than just basic JSON or YAML. There's all sorts of special keywords and properties we can use that will be interpreted as more than just plain-text key value pairs. We can write functions, make conditions, set parameters, all sorts of stuff. It'll be a reuseable template too, useful enough that if you're on AWS it's something you can take and use in your own or your company's projects.

One note here: to take on this series on in the most prepared way, you should have a decent grasp on AWS EC2 (Elastic Cloud Compute). We aren't going to walk through what EC2 is or how it works but, if you want to get up to speed on it, you can watch my AWS EC2 Fundamentals Series on YouTube to get up to speed. It has all the knowledge you'll need to get through this series, and anything EC2.

The History of CloudFormation

So let's talk about the history and the origins of CloudFormation. If you've gone through my EC2 series that I mentioned above, you know that it was in August 2006 that Amazon launched EC2. Even though it was the very beginnings of the industry-wide move into the Cloud Age, most enterprises in this time still wanted to keep things on premises and the idea of placing business-critical information on someone else's machine (because, by the way, that's all you're really doing when you think about it) seemed absurd. EC2 was still untested and Amazon wasn't nearly as gargantuan as it is now so poor Werner Vogels was still having to convince companies in-person to give it a shot. It wasn't even called AWS at this point, just Amazon.com Retail Web Services.

All that being said, we were indeed on the verge of leaving the Iron Age when everyone was running their own bare metal and the true catalyst for this change was, of course, virtual machines (VMs). Using VMs in the cloud as a server didn't just push virtual machines to the forefront but also it encouraged the usage of automation configuration tools like Chef and Puppet. These two tools allowed you to automate the setup of your servers.

Let's say you had five servers you wanted load balanced. To deal with massive amounts of traffic, you'd need to have five identical servers that you split the traffic between so you can keep the user experience smooth. But how do you make sure all five of them have the exact same software, and keep that same software each time? And not just on the first deploy but also when you have updates and patches later on? Well, before all of these wonderful automation tools came out, you'd have a lot of home-baked scripts and manual set-up ahead of you, wasting a ton of time on something that's pretty repeatable when you think about it. That's why tools like Chef and Puppet came in and said, "Nope, let's automate this and allow people to do it with code." And so, using one of these tools, you could automate the setup of your servers with nothing but code...you see where I'm going with this?

This is one of the first modern adoptions of infrastructure as code in general. Chef and Puppet started showing up in businesses left and right, both startups and enterprises. So why hadn't this happened yet? Well because virtual machines were just taking off in a serious way, VMWare was getting really hot alongside the whole concept in general. If configuring your servers is now just configuring virtual machines, well, suddenly scripting them made a lot more sense compared to having to individually set up bare metal servers.

So time continues on with this trend growing as more and more companies begin adopting this idea of running their business on VMs. Servers in the cloud where it's just a VM and on AWS EC2 (since it's the one driving this move for VMs) led to AWS going from being considered bleeding edge to an absolute requirement. Chef and Puppet also became the backbone of managing these infrastructures as well. The Cloud Age was beginning to take off.

In 2010, Microsoft finally realized they're behind the growing trend and in February that year they officially launched Azure. Now, I know this series is geared toward AWS but this story is too funny to pass up. Originally, Azure was known as Red Dog and was meant to be a platform as a service for Windows servers. Now, you know it's a good thing they didn't stick with that strategy. And the origin of the name Red Dog is actually pretty weird too: it was inspired by a seedy strip joint in San Jose called The Pink Poodle. So good job keeping it classy, Microsoft.

Anyhow, with Azure coming out in 2010 and Microsoft acknowledging the importance and growth of the space with its launch, the Cloud Age fully arrives because now there are two major vendors in the space. And, just when Microsoft thought they had a leg up in this area, AWS launched a whole army of things in that same year: their simple notification service (SNS) for push messages, a whole new region ap-southeast-1 in Singapore, and they went from being just a segment of Amazon.com into the AWS we know today. Then, as if that wasn't enough, they also launched their Route 53 domain name system and, last but not least, CloudFormation. The poor Red Dog team must have been terrorized by this blitz of launches by their competitor.

So, Why CloudFormation Then?

Now, why did this (CloudFormation that is) become a thing? In the face of new competitors, in the midst of AWS rebranding and all the new demand and market validation, why did AWS shift focus to such a seemingly non-critical product? After all, you don't need infrastructure as code to write infrastructure. Well, their launch post at the time actually summed up their motivations pretty well.

For one, AWS was already programmable so they knew it was possible to build things in it using repeatable processes and, in this case, that repeatable process was going to be code. Second, and very simply, they just listened to their customers needs, and one of those needs was to precisely replicate any infrastructure that you built. Think about it: if you need a different development environment, like a staging one maybe for beta users, or if you just need to make your infrastructure available in a different region...how are you going to do that without CloudFormation? Well, you're gonna end up redoing it all over again and hoping human error doesn't get in the way. But it always does.

There's great analogy Jeff Barr once used in a post that is perfect for this. What customers were doing up to this point was like cooking: every time you go to make something, it's going to be a little different than last time. That's the nature of cooking, it's more freeform and less bounded. But baking? That's a whole other ball game where you want things precise and measured the exact same (or as close as possible) each time if you want it to turn out consistently. And yes, I didn't know that until Barr's post came out because I know nothing about cooking or baking. But, it's with CloudFormation that developers could step away from being cooks and become bakers.

And so, here we are in the future. Just think about how much of a buzzword infrastructure as code has become...which is pretty funny actually since it's three words. But still, it's become one nonetheless with tons of tools and add-ons that people have created around it. Think of all the crazy forum wars and arguments developers have had over it. It's gone from being this nice little feature to a must-have if you want to compete in the DevOps endgame.

Now, there is one thing though about CloudFormation that people tend to underestimate. There's a lot of great things about it so this fact tends to be overshadowed by all the rest. But, this thing makes it essential and the sole reason that I suggest learning it over all of the other tooling out there. I don't care how cool whatever framework or library you've selected is. The biggest reason to learn CloudFormation is because it's incredible stable. If you look at the template format version that tells CloudFormation how to parse the template, the date will change when the rules around how you write templates changes. But guess what? This hasn't changed. At all. It's remained the same, meaning that even templates that you wrote almost ten years ago are still valid today.

The implications of this are that you learn CloudFormation and you don't have to relearn it. You don't have to come back in six months worried about the developers having an identity crisis and deciding to overhaul everything. This is one of those rare instances in our constantly changing field where you can learn a gem of a technology and not have to completely relearn it all again later.

Next Steps

The Next Post - The Main Concepts of CloudFormation

The Previous Post - The Complete CloudFormation Guide

Watch The Series Here

If you'd like to watch instead of read, the AWS CloudFormation Fundamentals series is also on YouTube! Check out the links below for the sections covered in this blog post.

Video 1 - Introduction to the Series (for sections 1-2)

Video 2 - The History of AWS CloudFormation (for sections 3-4)

The Complete CloudFormation Guide Index

If you're enjoying this series and finding it useful, be sure to check out the rest of the blog posts in it! The links below will take you to the other posts in The Complete CloudFormation Guide here on Tech Guides and Thoughts so you can continue building your CloudFormation template along with me.

1. The Complete CloudFormation Guide
2. An Introduction to and History of CloudFormation
3. The Main Concepts of CloudFormation
4. How CloudFormation Does Updates and Deletes
5. Our Project Setup
6. Resources
7. Parameters and Refs
8. Our First Time Launch
9. Functions, Pseudo Parameters, and Conditions Part 1
10. Functions, Pseudo Parameters, and Conditions Part 2
11. Mappings
12. Transforms
13. Outputs
14. Relaunch!
15. The Best Next Steps to Take from Here

Enjoy Posts Like These? Sign up to my mailing list!

My Tech Guides and Thoughts Mailing List