Secure serverless with Lambda breaks the rules?
AWS Lambda lets you just run code without provisioning or managing servers. Its execution is triggered by an event. An example of the supported AWS services that you can configure as an event source is API Gateway. This allows for triggering a Lambda function over HTTPS by configuring an endpoint in the API Gateway. When a HTTPS request is sent to this endpoint, the service translates the request to an invocation of the configured Lambda function. This provides an interesting serverless alternative to implementing backend services running on (virtual) servers which is scalable and cost-efficient, or is it? Well, maybe, but it depends on whether your backend service requires internet access.
Continuing on the scenario of creating a API, at some point the HTTP request would hit the stateless backend service which stores persistent data elsewhere, in a database. And because the backend service is ‘in front’ of it, we need not expose it to the internet directly. When a database does not really (really, really!) need to be exposed to the internet, it is best to ensure that it is not. This limits your system’s attack surface and, well, if you ever hooked any database, e.g. mysql, listening on its default port, e.g. 3306, to the internet you have probably seen the hostile login attempts appear in your logfile almost immediately. Therefore, we rather have the database tucked away safely so nobody can ever login by guessing our credentials or exploiting a vulnerability.
Server-based Architecture
Using AWS, a solution with servers would be comprised of at least one EC2 instance. If you want a relational database, then Amazon RDS with your database engine of your choice, e.g. mysql or postgresql, is a sensible choice.
To ensure that the RDS instance is not accessible via the internet we can create a Virtual Private Cloud (VPC) with public and private subnets. A private subnet is different from a public subnet in that it has no route to the internet. The Amazon RDS DB instance is attached to the private subnet. This is fine, because it is maintained by Amazon (it is a fully managed service) so we don’t need internet access to update or patch it. The EC2 instance is in the public subnet and gets assigned a public IPv4 address that can be used to access the backend service it hosts from the internet.
In this solution, both the EC2 instance and the RDS DB instance have their own security group that enables the control of inbound and outbound traffic. The security group in the public subnet is configured to allow access from the internet. The security group in the private subnet is configured to only accept traffic originating from the security group in the public subnet. Hereby, the backend service in the public subnet is able to connect to the Amazon RDS DB instance, while the Amazon RDS DB instance is not addressable from the public internet.
It’s all good. Even if our backend service requires internet access, which is a requirement that happens faster than you may think. For example, when you want to access services exposed via the API Gateway (which are always routed via the public internet), or use the AWS CLI (likewise), or, not surprisingly, when you need to access a service on the internet 😉
Serverless architecture
Looking at the serverless alternative we start of with the same architecture where the RDS DB instance is safe and well in our VPC’s private subnet. By default, Lambda functions cannot access resources that reside in a private VPC. To enable this, we must provide additional VPC-specific configuration information that includes the subnet and security group the Lambda is attached to. So, keep in mind that we require internet access, our first attempt was to configure the Lambda to use the same public subnet and security group like the one that we used to attach the EC2 instance to. However, it turns out that will not work because Lambda function is only assigned a private IP and without a public IP we cannot access the internet.
Instead, we need configure the Lambda’s to attach to the private subnet and if we require to create outbound connections to the internet we need to use network address translation (NAT). NAT maps private IP addresses to a single public IP address. In AWS a NAT Gateway can be provisioned in a public subnet with an Elastic IP address which connects it to the internet and configure route tables to route traffic from whatever runs in a private subnet to the internet and back.
It’s all good, again. We just created a serverless solution which is more secure because we don’t expose the RDS instance to the internet.
But there is a catch. We need to pay for the NAT Gateway by the hour! At the time of writing this blogpost that price ranges between $0.045 and $0.093 p/hour depending on the region. So we are looking at 30-ish dollars at least, per month, just for enabling internet access from our Lambda’s running in our VPC. (Imagine if we want to increase availability by having subnets in multiple availability zones, then we need multiple NAT Gateways too!) And it is not that I don’t understand that we need to pay. There are additional costs for the bytes of data processed by the NAT Gateway as well. Sure, no problem. Money makes the world go round and I think AWS is overall very reasonably priced and Lambda’s in particular are very, very cost-efficient. It is just that I was a bit disappointed to learn that AWS Lambda, the technology that pioneered “serverless” and probably started it all, breaks one of the defining rules of the Serverless Manifesto, which states that serverless is about only paying for what we use and not for idle. And we can debate that this is the case for the Lambda’s itself, but I IMHO to create a more secure solution we need the NAT Gateway so it is part of this “serverless architecture”.
There is a discussion on the AWS Forum asking for a free-tier, or cheaper pricing for NAT. I like cheaper, but like I said, I don’t mind paying for what we use and maybe whatever Amazon needs to do to run the NAT Gateway behind the curtains is expensive and thus they need to pass that costs on. It’s just the per hour pricing model that is so unfortunate. Alternatively, if Amazon would allow to have public IPs assigned to Lambda functions (or actually their hosts) attached to a VPC (just like we can with EC2 instances) that would be a solution, I think. But I may be missing something.