Should you use Lambda or should you use ?? I hear it can replace the world?

Okay, obviously I'm being dramatic. But listen (well, I suppose read): This picture is only partially representative of questions I get surrounding AWS Lambda. There's all sorts of crazy misconceptions. "Can I use it instead of Rails?"; "I don't need to use servers at all now right?"; "I can create my entire architecture out of Lambdas and save all the money right!?";

The really skewed misconceptions are understandable. Somehow, somewhere, Lambda began to equal "serverless." And "serverless" would obviously provoke ideas of some ethereal plane where code just magically runs with zero server configuration.

And then "serverless" started become "everything." Queues and Messaging? LAMBDA. Recurring job? LAMBDA. API? LAMBDA. Website? LAMBDA. Application? LAMBDAAAAA. Containers?? Put 'em in Lambda!!

Some of those can fit the use cases... but it just keeps on going. And that's the problem because the continuum seems to have no boundary. It seemingly does anything for everything.

Therefore it's no surprise to me that folks getting started in AWS or are inexperienced with DevOps in general would tumble down a rabbit hole of use cases.

Putting Lambda in the right context

I'm not going to deep dive into what Lambda is. Because there's so many articles and definitions on it, you'd have to actively NOT search the AWS space in order to avoid getting a basic idea.

What I want to do in this post is lay a ground work for how to "think" about it using an analogy.

The reason is that when folks get a technical definition, usually it's either too specific (code run in containers, managed by AWS) or too general (code that runs). And then they go and Google search it without getting a proper mental framework, and confusion ensues.

So instead, let's start with Larry.

(Beware. This analogy is not one to one and contains sarcasm. Someone somewhere will likely be offended by that.)

Larry had a little Lambda...

Larry is dealing with 2 dilemmas in his life right now:

1) He needs to find a place to live
2) He needs to deploy his company's application to the Cloud. Specifically AWS.

Little does he know... these two things are comparable enough for someone to blog about.

The search for living

First off, Larry wants to Rent. You can either take that as him applying particular financial strategies or just being another developer livin' life in SF.

He's got two solid options here:

a) Rent a House
b) Rent an Apartment

Renting a house is going to give Gary more space, some more control, and probably more peace of mind (no wall sharing). Although he's renting, he'll likely have more responsibilities: possibly need his own appliances, tend to the yard, participate in Halloween candy runs, etc. There's also the chance he doesn't need all of the space a house provides, meaning he's paying for unused square footage.

On the other hand, renting an apartment will be far more economical. The buildings that he's looking at come with additional services like a gym, pool area, etc. It's also close to other amenities. But, if he wants more control, it'll be a bit more tedious since the building space is shared with other tenants.

Key takeaways:

House = More space, more control. Requires more responsibility and costs more.

Apartment = More economical, easy access to other amenities / services. Less space, less control.

Now let's cross over to the upside down - deploying his company's application.

The dilemma of deployment

First off, Larry wants to use a cloud provider (rent). He doesn't want to buy servers or set up his own. And he's going to use AWS. Why? Because otherwise this post won't make any sense. Remember, we're keeping things simple here.

So two options of deployment here:

a) Deploy directly to an EC2 Instance (or set of them)
b) Deploy using Docker containers that are on EC2 Instances

Deploying directly to instances is going to give way more control. He can configure them exactly how he'd like, finagle with the OS, work with the application directly on the OS, and, to an extent, have more simplicity.

But in doing so, he HAS to configure the instances quite a bit. He has to deal with legacy deploys when he redeploys to instances (i.e. updating software on long running instances). He has to pay for not leverage all the space / compute on any given instance. And if he wants to deploy another app to take advantage of extra, unused resources, he's going to have a fun time (not really).

Deploying leveraging Docker is going to make configuration cake. If an instance is set up to run Docker, then as long as his app can run in a container... he's good to go. He just hands the image to the instances running Docker, and up they go.

He can configure the containers however he'd like without gravely messing up the host instances. Since Docker is so portable, he can easily re-deploy his "Dockerized" application to anything that leverages Docker.

Also, he can deploy multiple apps, in containers to the same instance and fully leverage resources.

The downsides? Although Docker deploys, once they're up and running, are pretty easy-going... ramping up to understand them can be quite steep. Without a good understanding of both Docker and deployment solutions, managing something like this takes a learning investment.

Unless it's not already obvious, the loose analogy here is that Instances are like renting houses, where as docker containers are like renting apartments. Now for the less aware (and more picky), docker containers would be more like renting an entire building in which you can add and remove apartments at any given time. And you can also dynamically size them. And you wouldn't be able to hear your neighbors unless you set it up to do so. In fact, I wish apartments were more like containers, because upstairs neighbors gain a +10 in noise...anyhow...

Yambda - A New Living Start-Up Emerges

Larry's friend, Barry, introduces him to this brand new Start-Up called YAMBDA. Yambda is revolutionizing the concept of using building space!

In select regions, Yambda has campuses filled with all sorts of facilities and rooms that can be customized and used ON DEMAND. Kitchens. Backyards.. in a room. Bedrooms. Garages. Disco Floors. And yes Winston, there IS an app for it!

And you only pay for the time you use in the room.

What's the catch? Well you do have to rent each room independently, and they aren't immediately connected. Also, unless you've had the room already in usage, you have to give Yambda some time to put your room together. There's also limits to how much time you can have one AND how much stuff you can put in them.

BUT. If you only need a quick room for a few hours, it's a great fit. Howabout a personal gym for a few hours? Schedule it to be used. A sauna? Absolutely.

AWS Lambda

Larry's DevOps adversary, Harry, introduces him to Lambda. Okay if you're here you know what it is so I won't go into it. More or less, run your code on demand. It should be no surprise that the comparison is to Yambda from above.

But, what generally are the main cited advantages? Code execution on demand. Only pay for what you use. "Eliminates scaling problems." It's cheap. Web developers believe that they are now e-gods amongst men.

What about the disadvantages? These don't get talked about to often. But.. they can only be so big. They can only be running for so long. A Lambda can only be handle so many concurrent executions. Your VPC needs enough ENIs or Subnet IPs to handle scaling them out - if you want them in a VPC. Configuring, managing, and debugging for development hasn't matured very far (and due to their nature, how does it need to?)

(See AWS Lambda Limits and Configuring a Lambda Function to Access Resources in an Amazon VPC. Also take some time to just look up some of the problems folks have when over-dosing on Lambda.)

Now. For what it's good at, it's damn amazing at. Helping to manage actions and alarms within AWS? Great. Extending AWS capabilities through custom functionality? Amazing. Quick and dirty microservices with other APIs? Holy crap amazing.

Meanwhile, the search for housing continues...

Introducing "Houseless"

Lately, Larry has noticed a lot of his developer buddies emerging at dawn from Yambda campuses.. and disappearing to the campuses at dusk. Finally, he stops his friend Barry (with two Rs), and asks what's going on.

Larry: "I thought Yambda was just for temporary room and facility usage? Are you... sleeping there?"

Barry: "Yeah man, I'm actually living there entirely! I pay like 1/2 of what I would pay in rent in a real apartment or house! It's called going Houseless!"

Larry: "Wat? How does that work? I thought you could only be in them for so long? And have so much stuff?"

Barry: "Well, since you can only have rooms for a few hours, I have to make sure that I have other rooms pre-scheduled in advanced"

Barry then goes on to explain how he pre-schedules all of the rooms he's going to need everyday, in advanced. Yes, if he wants a party room and dining room at once he has to schedule them. No they're not always next together. But yes he has a Segway. Since he has to wake up and use the bathroom at 3 am every morning, he just makes it so that he switches rooms at that time every night as well.

Barry: "It's a tad complicated I'll admit. For example, if I leave and come back I'm never quite sure if the room I was just in will still be available! So I can never leave my furniture in there. Good thing the campuses have Simple Furniture Storage."

Barry: "Keeping the contracts for all the different rooms can be a bit to watch. Oh and consistency in feng shui every time I schedule one of my living room layouts to be used is kind of a hassle."

Larry: "So you just hop between rooms on campus in order to replicate an Apartment / House experience?"

Barry: "Yep! And there's even some new services popping up that help manage all of this complexity. To where it's almost exactly like living in a house or apartment!"

Larry: "Couldn't you just get a house then? I mean.. surely the amount of time saved from just renting one place would make up for the saved cost..."

Barry: "That's not the point! This is Houseless!"

The next day at work...

Lambda-All-The-Things

Larry continues to ponder his dilemma of deploying his application. "Server... or containers??" However, since he's in an open-office co-working space his adversary Harry overhears him.

Harry: "Why aren't you using Lambda for all of it?"

Larry: "I thought Lambda was just for microservices and small executions? Not long running code?"

Harry: "Pssh. Oh little Larry, how narrow your world is. Didn't know you can configure a series of Lambda functions to completely replace your need for any servers or containers at all?"

Harry then goes on to explain how one can split all of your code up into chunks that run in Lambda functions, and leveraging an army of configuration, arrive at the same result. He shows Larry a diagram that includes running traffic through an alphabet soup of AWS and 3rd party management tools and services.

Harry: "I'll admit, it's a bit tricky. We had to rewrite quite a bit of the existing applications in order get them to fit the model. We even wound up migrating to DynamoDB since both RDS and other 3rd party DBs are difficult to play with in a VPC. There's also so limits on concurrency and VPC usage, but we have so many CloudWatch alarms set up to watch those things that we know exactly when it happens."

Harry: "And guess what responds to those alarms?"

Larry: "What?"

Harry: "Even more Lambdas!!"

Larry: "So, your team's app, an Uber for street corner sign spinners, that's always has persistent traffic, is hopping between temporary functions to replicate the functionality of a server?"

Harry: "Yep! We do it at like 1/3 to 1/2 of the cost that normals servers with auto scaling built in would cost! Sure, testing and debugging are still kind of hazy, but our team prides itself in complexity and configuration! We see lack of clarity in our set up as a sort of trial-by-fire for potential new developers."

Larry: "But Harry, surely the time you'd save from just spinning up a server..."

Harry: (with furrowed brow) "Pssh. Larry Larry Larry. I knew you wouldn't get it. It's over your head."

One Week Later

Larry passes away from paralysis by analysis.

In the development community, many of the technologies that snowball into popularity are often the result of clever marketing, a few easy-to-read posts/guides, developer fervor, and Github stardom. Of course this type of pattern common in every product and/or service in any industry.

But, good at one thing does not mean great at many things. And great at many things doesn't mean incredible at everything. And incredible at everything doesn't mean "Great Scott! We should migrate our entire system now Marty! The future is here!"

The point I'm trying to make is this:

Lambda isn't some magical solution that should be used to replace every aspect of your architecture. In fact, half the things you think you'd like to stuff into Lambda, you could either just do right in your current service, hand off to another long running compute service, or just have temporary container spin up your self. And if what you're looking to do is build a typical web application... just use a typical server. If you're really fancying lambda, you can add it on later once you figure out where it can actually help your app.

(Disclaimer: I do actually like Lambda)