Do Lambdas(AWS) stop making sense after a certain number of functions?

Question

I am working on a project and created multiple lambda functions and now I am thinking about whether I made the correct decision to go this route.

Reasons I chose lambda:

• The App will have spiky traffic and almost no traffic at some times, so it is cost-effective
• It seemed like it would be less stressful to update particular functions because I know other functions are not touched and would not break the entire app (creates a setup similar to microservices)
• All APIs on my app are under timeout limits of lambdas so no long-running functions

Reasons I am doubting it now:

• It's difficult to share logic between lambdas like common utility functions and constants, I am using layers but that works fine if I don't have to update the layers too often as I have to upload it manually
• Version control : There is an option to create versions(it's like build tagging) but no easy way to actually create a repository and deploy on changes, this makes it difficult to check what could have gone wrong during a release. I could use AWS CodePipeline, CodeBuild and a trigger using EventBridge that produces SAM template and deploys to lambda using CloudFormation, this solution would work but I don't like how many applications it uses in between to achieve this. And not having version control will make the project unmanageable.

All this has made me think whether lambdas are not great if you have lots of functions and would be better off with a server that always runs and is tempting me to extract all function code and put it on a server. Lambdas are a no-brainer if you have few functions that would rarely run but is it practical to create full applications with multiple APIs?

Update: This question has received multiple good answers and it's either of these two

• Lambdas are not that great as people make it seem : I do agree with this now, when I was starting out it felt great as to how fast I was able to prototype and run things that can scale reasonably.
• Your CI/CD is not properly configured or use the serverless framework and all the problems could be solved : Well, this is also true as I also mentioned in my question on one of the possible ways to achieve this. But this does mean that Lambdas do not work for larger projects out of the box and you would need significant abstraction on top of it. Now that I think of it, I don't know why it didn't hit me earlier because it's so obvious now.

For now, I have decided to use lambdas initially and will eventually move the code to servers(because initially, lambdas won't even cost as much as a load balancer). For larger projects, I would say that lambdas are great for prototyping quickly or event-based background processes like step functions but if you have worked with servers before then you will be always questioning yourself when you start adding functions after another.

Answer 1

Working with Lambdas really requires some deployment automation tooling to manage all of these things for you. At my last job, we used Serverless Framework, but after a year I was starting to feel like it was holding back my productivity.

If I had to choose tooling today, I'd probably go for Terraform or the AWS CDK.

Serverless was interesting. I learned a lot, especially about DevOps. But I don't miss that codebase. There were a number of reasons it was becoming a dumpster fire, and Lambda played a minor part in several of them.

Answer 2

AWS Lambdas never made any sense.

Or perhaps I should say "serverless applications" never made any sense. Because it seems to me that's what you are doing, moving application code to lambdas for cost reasons.

As you note, actually the lambdas all run on instances that get billed by the hour, so unless you have long periods of nothing running at all you hardly save any money over always having a tiny instance up.

You also have the various complexities you outline, essentially you are forced into a nano-service++ architecture with complex orchestration problems.

Now you can in theory imagine an application where this architecture is ideal, lots of queues, events and mini functions all working together, but in practice its a bit like functional programming. Nice in theory, but difficult to apply to standard "line of business" applications.

In summary, only use Lambdas where you have a specific need for them. Don't use them because the billing model appears different.

Answer 3

It's true that lambdas can save hosting costs for low- and medium-traffic apps.

You don't have to have a zillion separate lambdas and a deployment nightmare for your little app, though. There is no technical restriction that prevents you from putting an entire monolithic app, which branches to different code depending on the type of request, into a single lambda. Don't be fooled by the marketing. Of course, it might make sense to split up large apps or apps with lots of traffic.

You also don't need to version-control your deployed lambdas, only your source code. If you use EC2 do you version-control your EC2 instances? Of course not. You may version-control the code which creates and configures them - but not the instances themselves, because that makes no sense.

If you only have one lambda, or a few lambdas, you can make a short script (on your own computer) that uses the AWS CLI to deploy them all.

Don't try to make a serious app using the online editor in the AWS console - since (as you pointed out) you don't get version control, or any of the other features you want when programming source code (such as auto-complete).

Answer 4

whether lambdas are not great if you have lots of functions

The reasons you state are not linked to number of functions or code size as elaborated below.

It's difficult to share logic between lambdas

Not a Serverless specific problem. Micro-services too have the same issue. Multi-module monoliths too have shared libraries. When you break logic into pieces, common logic has to live somewhere. One pattern would be to refactor out shared logic in a library/package. Then base each function/service upon it.

Whether it is 20 functions or 200, shared logic has to be refactored out.

Version control...but no easy way to actually create a repository and deploy on changes

Again, a Dev-ops problem. Pick any competent CI-CD tool and automate it to spit out packages/images for ALL functions/service (changed or not) per change committed. All tagged with the same build number and deployed together. The environment should represent a single snapshot of the code base, not a mix of selectively updated and deployed components.

The automated script that can build 20 functions can also build and deploy 200.

In the end, it is not code base size that presents a true hindrance. In fact you are not limited to Lambda and its tools for server-less computing. Stacks like Knative exist that allow shared base images, service versioning etc.