HTTP and connection pooling
HTTP is used heavily in many microservices architectures. While there are other options to increase the performance, if you conclude HTTP is the way to go, looking into using persistent connections might save your day.
If the server supports persistent connections, HTTP connection pooling can provide performance benefits when multiple HTTP requests target the same host and port. With connection pooling, instead of closing the client HTTP connection after use the connection is kept open and returned for reuse. When a pooled connection is reused, the overhead of creating new TCP connections is avoided.
HTTP/1.1 supports adding “Connection: keep-alive” headers to negotiate a persistent connection with the server that can be pooled an reused.
HTTP/2, introduced in 2015 and already widely adopted1, changed the matter completely since it multiplexes numerous streams over a single TCP connection. There is no need to negotiate the ability to reuse a connection, it is a given.
Looking at AWS API Gateway
I’ve deployed a simple serverless API where we use a Lambda function fronted by API Gateway. This API serves a JSON payload as a response to a GET request on /products.
Giving it a try with curl we see:
curl -v https://xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com/Prod/products [..] * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x7f8e0280a600) > GET /Prod/products HTTP/2 > Host: xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com > User-Agent: curl/7.54.0 > Accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS updated)! < HTTP/2 200 < content-type: application/json < content-length: 82 < date: Wed, 20 Feb 2019 09:32:55 GMT < x-amzn-requestid: 807a4fa3-34f2-11e9-8f55-edd3cf2990b1 < x-amz-apigw-id: VZG_JE1gFiAFQuA= < x-amzn-trace-id: Root=1-5c6d1ec7-395c543f250d2fb1583e0278;Sampled=0 < x-cache: Miss from cloudfront < via: 1.1 80d6ceec7d3cd9fa88dfa92002c593ab.cloudfront.net (CloudFront) < x-amz-cf-id: gsY0vLoFAhYm4IYVaGRc7mNG21kLh7l6WwfmW4Ijf01Bg4aYizQjRA== < * Connection #0 to host xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com left intact
So it uses HTTP/2 and hints that connection #0 is left in tact. What if we call the same host twice in a single curl command?
curl -v https://xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com/Prod/products https://xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com/Prod/products
[..]
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f932a805800)
> GET /Prod/products HTTP/2
> Host: xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com
> User-Agent: curl/7.54.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< content-type: application/json
< content-length: 82
< date: Wed, 20 Feb 2019 09:33:21 GMT
< x-amzn-requestid: 8fe8cf8b-34f2-11e9-b1c3-d19ca18e3629
< x-amz-apigw-id: VZHDMEW-liAFeJA=
< x-amzn-trace-id: Root=1-5c6d1ee1-fb1a1da69a76231287d37fea;Sampled=0
< x-cache: Miss from cloudfront
< via: 1.1 9f51d6a2a4451a14c099e82bc4356b20.cloudfront.net (CloudFront)
< x-amz-cf-id: hZvMSgM5GS80sC1KUuFTkF3DtJ14f04i34Z7zxwRXqTpH2vg2GlvnA==
<
* Connection #0 to host xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com left intact [{"product_id":123,"name":"some name","description":"some description","price":0}]
* Found bundle for host xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com: 0x7f9329d1f820 [can multiplex]
* Re-using existing connection! (#0) with host xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com
* Connected to xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com (52.85.255.216) port 443 (#0)
* Using Stream ID: 3 (easy handle 0x7f932a805800) > GET /Prod/products HTTP/2
> Host: xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/2 200
< content-type: application/json
< content-length: 82
< date: Wed, 20 Feb 2019 09:33:21 GMT
< x-amzn-requestid: 8ff10d4e-34f2-11e9-9e4b-3bf51c0f3b20
< x-amz-apigw-id: VZHDNGOlliAFqEw=
< x-amzn-trace-id: Root=1-5c6d1ee1-b1a7d59a3c72b552236b42e2;Sampled=0
< x-cache: Miss from cloudfront
< via: 1.1 9f51d6a2a4451a14c099e82bc4356b20.cloudfront.net (CloudFront)
< x-amz-cf-id: hli3QP5u2f9BcIRq9whaRezhA8hw2vyuR5yL1WYomfpkANkKY6z_Rg==
<
* Connection #0 to host xxxxxxxxxx.execute-api.eu-central-1.amazonaws.com left intact
[{"product_id":123,"name":"some name","description":"some description","price":0}]
So here we go. The connection can be reused, if the client plays ball. Curl does, but do your microservices calling each other do it too?
[1] W3Techs measured an adoption rate of 33.4% in February 2019 (using the top 10 million websites)