Updating go1.x Runtime in AWS Lambdas

Learn how to update the Go runtime for a serverless application running on AWS.

Juampy NR

December 4, 2024

We've created a Git repository containing all the old and new code blocks mentioned in this article. The main branch uses the latest runtime, while the go1.x uses the old runtime. There is a pull request with the set of required changes.

For many of our projects, Drupal is just one piece of a large and complex puzzle of applications. Some of these applications follow the serverless architecture and such applications:

Don't require a quick response time.
Usually run asynchronously.
Can scale up fast in the cloud.
Don't require provisioning or managing servers.

Many refer to serverless applications as lambdas (a function usually written in Node.js, Go, or Python). However, on top of the lambda, permissions, networking, databases, messaging, logging, and further plumbing are usually needed so the lambda can run successfully.

There are also multiple meanings for "serverless":

There is serverless computing as a cloud computing model.
There is serverless on AWS, which is the service AWS offers for serverless applications.
There is the Serverless framework as an abstraction layer for AWS Serverless, which we use in the sample application below.

While deploying updates to a serverless application on AWS, we encountered an issue with the Go runtime. This article covers how we resolved the problem.

A sample serverless application

You can browse the application at GitHub, but we'll review the main files.

A Go lambda

Below is a simple lambda function that prints the incoming payload:

# hello/main.go
package main
 
import (
    "fmt"
 
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
)
 
func Handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
        fmt.Println("Received body: ", request.Body)
        return events.APIGatewayProxyResponse{Body: request.Body, StatusCode: 200}, nil
}
 
func main() {
        lambda.Start(Handler)
}

To run the above in AWS, we need a serverless.yml file defining what this lambda needs to run.

A serverless.yml file

The below file defines how to deploy the above lambda in AWS. It defines the Go runtime and a triggering event via an HTTP POST request to /hello.

# serverless.yml 
frameworkVersion: ^3.39.0
service: hello
provider:
  name: aws
  runtime: go1.x
package:
  individually: true
functions:
  hello:
    handler: hello
    package:
      artifact: bin/hello.zip
    events:
      - http:
          path: hello
          method: post

Notice that Serverless framework is an abstraction layer for AWS Cloud Formation. Upon deployment via sls deploy, it will build a CloudFormation stack that defines all the resources it needs to run, such as permissions, roles, networking, gateways, etc.

Build and deploy

In this case, we are using GNU make to build and deploy the application:

# Makefile
build:
  env GOOS=linux go build -ldflags="-s -w" -o bin/hello hello/main.go
  cd hello; zip hello.zip hello
deploy:
  sls deploy

However, AWS won't let us deploy this application since it uses the go1.x runtime. Let's update it.

Deployment error

After authenticating in the command line with AWS via aws configure, we can deploy our application by running make build && make deploy. However, we recently ran into the following error:

✖ Stack some-stack-example failed to deploy (93s)
Environment: darwin, node 18.17.1, framework 3.31.0 (local) 3.38.0v (global), plugin 6.2.3, SDK 4.3.2
Credentials: Local, "default" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
CREATE_FAILED: SomeStackLambdaFunction (AWS::Lambda::Function)
Resource handler returned message: "The runtime parameter of go1.x is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (provided.al2023) while creating or updating functions. (Service: Lambda, Status Code: 400, Request ID: redacted)" (RequestToken: redacted, HandlerErrorCode: InvalidRequest)

View the full error: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stack/detail?stackId=redacted
make: *** [deploy] Error 1

The go1.x runtime used for Go applications at AWS is no longer supported since December 31st, 2023. The new runtime is provided.al2023. Full details and migration docs are available at Migrating AWS Lambda functions from the Go1.x runtime to the custom runtime on Amazon Linux 2.

It took us some reading to figure out what we needed to change in the above serverless.yml and why, so we will summarize the changes in the following section.

Fixing the application

Here is the list of changes required to successfully deploy a Go lambda to AWS. You can see a pull request with the list of changes at https://github.com/juampynr/aws-runtime-go-update/pull/1/files.

In the Go main file

Nothing. This stays as-is, but you can take the opportunity to update to the latest version of AWS Lambda for Go.

In the serverless.yml file

Here are the changes we made:

A screenshot of the changes made to serverless.yml

And here is the resulting file:

# serverless.yml 
service: myservice
provider:
  name: aws
  runtime: provided.al2023
  architecture: arm64
functions:
  hello:
    handler: bin/bootstrap
    package:
      artifact: bin/hello.zip
    events:
      - http:
          path: hello
          method: post

In the makefile

Here are the changes:

And here is the resulting makefile:

# Makefile
build:
  env GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -ldflags="-s -w" -o bin/bootstrap hello/main.go
  cd ./bin; zip hello.zip bootstrap
deploy:
  sls deploy

Explanation of the changes

Changed the architecture to AWS Graviton2 Processor. This processor offers better performance and pricing. See Migrating AWS Lambda functions to Arm-based AWS Graviton2 processors for details. This is an optional but recommended change.
Added the -tags lambda.norpc option for even better performance and cold starts. This is only available in combination with GOARCH=arm64. Full details at Migrating AWS Lambda functions to Arm-based AWS Graviton2 processors for details. If your application uses the RPC protocol, do not use this tag when building the application.
Changed the name of the resulting binary from hello to bootstrap. This is a requirement by AWS: the Go executable that AWS Lambda uses has to be called bootstrap. If you have multiple lambdas, we recommend compiling each lambda in its own directory under bin and then zip them independently. For example, hello/main.go could go into /bin/hello/bootstrap and then we would zip /bin/hello/bootstrap into /bin/hello.zip.

Deployment

After making the above changes, it's time to build and deploy the stack to AWS again.

First, we build the application:

$ make build
env GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -ldflags="-s -w" -o bin/bootstrap hello/main.go
cd ./bin; zip hello.zip bootstrap
  adding: bootstrap (deflated 59%)

Then we deploy it to AWS:

$ make deploy
sls deploy --verbose

Deploying hello to stage dev (us-east-1)

Packaging
Retrieving CloudFormation stack
Uploading
Uploading CloudFormation file to S3
Uploading State file to S3
Uploading service hello.zip file to S3 (2.05 MB)
Updating CloudFormation stack
Creating new change set
Waiting for new change set to be created
Change Set did not reach desired state, retrying
Executing created change set
  UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - hello-dev
  CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
  CREATE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
  CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
  CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
  CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
  CREATE_IN_PROGRESS - AWS::ApiGateway::RestApi - ApiGatewayRestApi
  CREATE_COMPLETE - AWS::ApiGateway::RestApi - ApiGatewayRestApi
  CREATE_IN_PROGRESS - AWS::ApiGateway::Resource - ApiGatewayResourceHello
  CREATE_IN_PROGRESS - AWS::ApiGateway::Resource - ApiGatewayResourceHello
  CREATE_COMPLETE - AWS::ApiGateway::Resource - ApiGatewayResourceHello
  CREATE_COMPLETE - AWS::Logs::LogGroup - HelloLogGroup
  CREATE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
  CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
  CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
  CREATE_COMPLETE - AWS::Lambda::Function - HelloLambdaFunction
  CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersion4V0GocYWXLNAdFoBlPWROv3uiByixVfGmcK0lztT3c
  CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionApiGateway
  CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionApiGateway
  CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersion4V0GocYWXLNAdFoBlPWROv3uiByixVfGmcK0lztT3c
  CREATE_COMPLETE - AWS::Lambda::Permission - HelloLambdaPermissionApiGateway
  CREATE_COMPLETE - AWS::Lambda::Version - HelloLambdaVersion4V0GocYWXLNAdFoBlPWROv3uiByixVfGmcK0lztT3c
  CREATE_IN_PROGRESS - AWS::ApiGateway::Method - ApiGatewayMethodHelloPost
  CREATE_IN_PROGRESS - AWS::ApiGateway::Method - ApiGatewayMethodHelloPost
  CREATE_COMPLETE - AWS::ApiGateway::Method - ApiGatewayMethodHelloPost
  CREATE_IN_PROGRESS - AWS::ApiGateway::Deployment - ApiGatewayDeployment1730185335043
  CREATE_IN_PROGRESS - AWS::ApiGateway::Deployment - ApiGatewayDeployment1730185335043
  CREATE_COMPLETE - AWS::ApiGateway::Deployment - ApiGatewayDeployment1730185335043
  UPDATE_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - hello-dev
  UPDATE_COMPLETE - AWS::CloudFormation::Stack - hello-dev
Retrieving CloudFormation stack
Removing old service artifacts from S3

✔ Service deployed to stack hello-dev (67s)

endpoint: POST - https://redacted.execute-api.us-east-1.amazonaws.com/dev/hello
functions:
  hello: hello-dev-hello (2 MB)

Stack Outputs:
  HelloLambdaFunctionQualifiedArn: arn:aws:lambda:us-east-1:redacted:function:hello-dev-hello:1
  ServiceEndpoint: https://redacted.execute-api.us-east-1.amazonaws.com/dev
  ServerlessDeploymentBucketName: hello-dev-serverlessdeploymentbucket-redacted

You might get this warning when deploying via sls deploy:

Warning: Invalid configuration encountered
  at 'provider.runtime': must be equal to one of the allowed values [dotnetcore3.1, dotnet5.0, dotnet6, dotnet7, go1.x, java17, java11, java8, java8.al2, nodejs12.x, nodejs14.x, nodejs16.x, nodejs18.x, provided, provided.al2, python3.7, python3.8, python3.9, python3.10, ruby2.7]

Learn more about configuration validation here: http://slss.io/configuration-validation

Serverless 3 does not recognize the runtime provided.al2023. The closest it knows about is provided.al2 but it's not worth it to use provided.al2 since its EOL is June 2025. Therefore, you can keep using Serverless 3 and ignore that warning. Alternatively, you can upgrade to Serverless 4, but that requires a user subscription, so it's up to you.

Conclusion

The AWS document Migrating AWS Lambda functions from the Go1.x runtime to the custom runtime on Amazon Linux 2 is excellent. However, the documentation covers many scenarios, so figuring out what to change in a serverless.yml file can be challenging. On top of that, there is the chance to update the architecture via Migrating AWS Lambda functions to Arm-based AWS Graviton2 processors for details, which requires further reading and filtering to apply it to a specific case.

Hopefully, this article helped you cut through the noise.

Acknowledgments

This post from the Serverless website had a very simple Go lambda that we used for our example.

Published in:

System Administration

Juampy NR

Loves optimizing development workflows. Publishes articles, books, and code.

More about Juampy

More Resources

See all