Functions

Definitions

There are 4 kinds of function definitions:

1. Dir functions
2. Interned functions
3. Nano functions
4. Standalone functions

Dir functions

tc discovers functions in the current directory. A function is any directory that contains a

1. handler.{py,rb,clj,js} file and/or
2. function.yml file

At it’s simplest, a function directory (say foo) looks as follows:

foo/
 - handler.{py, rb, js, clj}

tc infers the kind of function, runtime and build instructions. However, we can be more specific as follows in a function.yml file

name: foo
runtime:
  lang: python3.11
  handler: handler.handler

Custom function dirs

At times, we like to organize our functions in logical partitioned directories. Say something like the following:

├── bar
│   ├── f3
│   │   └── handler.clj
│   └── f4
│       └── handler.janet
└── foo
    ├── f1
    │   └── handler.rb
    └── f2
        └── handler.py

By default, tc does not recursively discover functions. It looks for functions (dirs) in the current directory of the topology. In the above example, bar and foo are ignored and not discovered.

To intern those functions, we can set the list of dirs to scan in the topology.yml file

name: t1

function_dirs:
  - foo
  - bar

tc compose -f tree to visualize the tree.

Interned functions

Interned functions are those that are explicitly defined in topology.yml

functions:
  fun1:
    uri: ../my-fun1
  fun2:
    uri: ../my-fun2

Nano functions

Nano functions are functions with tiny snippets of code embedded in the topology spec.

Note

Available in upcoming tc 0.10.x

name: example-nano

events:
  ConcatStrings:
    function: f1

functions:
  f1:
    runtime:
      lang: python3.12
      code: |
        def handler(event, context):
          return {'input': ['a', 'b']}
    function: f2

  f2:
    runtime:
      lang: clojure1.10
      code: |
        (defn handler [event context]
           (clojure.string/join (:input event) ","))

Nano functions are useful where

1. functions have no external IO or need to access AWS resources. The IO is offloaded to other entity nodes in the graph.
2. functions have no additional library dependencies.
3. functions are tiny.

Standalone functions

A function that does belong to a topology or has no topology defined is a standalone function. We can set a namespace explicitly in the function spec (function.yml).

name: foo
namespace: bar
runtime:
  lang: python3.11
  handler: handler.handler

Components

Key	Default	Optional?	Comments
lang	Inferred	yes
handler	handler.handler
package_type	zip		possible values: zip, image
uri	file:./lambda.zip
mount_fs	false	yes
snapstart	false	yes
memory	128	yes
timeout	30	yes
provisioned_concurrency	0	yes
reserved_concurrency	0	yes
layers	[]	yes
extensions	[]	yes
environment	{}	yes	Environment variables

Permissions

By default, tc infers permissions and sets the right boundaries in the sandbox. However, you may need to override the permissions by specifying a custom roles file in $INFRA_ROOT (say $GIT_ROOT/infrastructure/tc/TOPOLOGY-NAME/roles/FUNCTION-NAME.json. The contents of the roles file is typically the IAM policy.

Environment variables

Specify env variables in $GIT_ROOT/infrastructure/tc/TOPOLOGY-NAME/vars/FUNCTION-NAME.json

{
  "default": {
    "timeout": 120,
    "memory_size": 128,
    "environment": {
      "API_KEY": "ssm:/path/to/api-key",
      "DB_HOST": "dev.db.net",
      "API_GATEWAY_URL": "{{API_GATEWAY_URL}}"
    }
  },
  "prod": {
    "timeout": 60,
    "memory_size": 1024,
    "environment": {
      "API_KEY": "ssm:/path/to/api-key",
      "DB_HOST": "prod.db.net"
    }
  },
  "SANDBOX-NAME": {
    "timeout": 60,
    "environment": {
      "API_KEY": "ssm:/path/to/api-key"
    }
  }
}

The vars or runtime file is map of default and sandbox-specific overrides. Environment variables in the runtime file can be either an URI or plain text. Supported URIs are ssm:/ , s3:/ and file:/. If an URI is specified, tc resolves the values and injects them as actual values when creating the lambda. Decryption using extensions is also available. See Extensions.

We can also discover Endpoints for routes and mutations that are sandbox-specific. tc does a topological sort and gets the URLs ahead of time before rendering the vars.json file.

Network

To configure a function in a VPC, we can do:

name: my-fn
runtime:
  lang: Go
  handler: bootstrap
  network: true

And in it’s corresponding infrastructure file, we can set the network. For example, in infrastructure/tc/<topology-name>/vars/my-fn.json

{
    "default": {
        "timeout": 300,
        "environment": {
            "DYNAMODB_TABLE_NAME": "foo-bar",
            "REDIS_LOOKUP_ENABLED": "false",
        }
    },
    "dev": {
        "environment": {
            "REDIS_LOOKUP_ENABLED": "false"
        },
        "network": {
            "security_groups": [
                "sg-12345"
            ],
            "subnets": [
                "subnet-1234",
                "subnet-5678",
                "subnet-9876"
            ]
        }
    },

And then tc create --sandbox dev1 --profile dev will configure the function with the given network

Update components

tc provides mechanisms to update specific component of entities or topology in a given sandbox. This is incredibly useful when developing your core topology.

tc update -s sandbox -e env -c functions/layers
tc update -s sandbox -e env -c functions/vars
tc update -s sandbox -e env -c functions/concurrency
tc update -s sandbox -e env -c functions/runtime
tc update -s sandbox -e env -c functions/tags
tc update -s sandbox -e env -c functions/roles
tc update -s sandnox -e env -c functions/function-name

Dependencies

tc has a sophisticated function builder that can build different kinds of artifacts with various language runtimes (Clojure, Janet, Rust, Ruby, Python, Node)

In the simplest case, when there are no dependencies in a function, we can specify how the code is packed (zipped) as follows in function.yml:

name: simple-function
runtime:
  lang: python3.10
  handler: handler.handler

Example

The above is a pretty trivial example and it gets complicated as we start adding more dependencies. We can specify how the function needs to be built. For example:

name: funciton-with-deps-example
runtime:
  lang: python3.10 | python3.11 | python3.12 | ruby3.2
  handler: handler.handler
build:
  kind: Code |Inline |Image | Layer
  pre: [String]
  post: [<String>]
  command: <String>

Attribute	Description
kind	Specifies how the dependencies are packaged Available options are Code, Inline, Image, Layer
pre	Array of commands to run before the dependencies are installed. Has shared build context, Host ssh-agent access. Typically useful to install system dependencies (yum) or private packages (ssh://github etc)
post	Array of commands to run after the dependencies are installed. Has shared build context, Host ssh-agent access and AWS access for given sandbox or centralized repo. Typically useful to pull models, CSV files etc from S3 or object stores and package them in the build artifact
command	Command to pack the code. Typically it is the zip command (zip -9 -q lambda.zip *)
share_context	Default: true. If true, copied current git repository for referencing any shared relative paths in the build container
skip_dev_deps	Default: false. Skips dev dependencies when building deps

Note

The build kinds are interchangeable. All of the build kinds take the same options. We can replace Image with Inline at any point if the size is reasonable (< 50MB) and it will just work.

and then tc create -s <sandbox> -e <env> builds this function using the given command and creates it in the given sandbox and env.

Inline

If the dependencies are reasonably small (< 50MB), we can inline those in the code’s artifact (lambda.zip).

name: python-inline-example
runtime:
  lang: python3.12
  package_type: zip
  handler: handler.handler
build:
  kind: Inline
  command: zip -9 -q lambda.zip *.py

Example

tc create -s <sandbox> -e <env> will implicitly build the artifact with inlined deps and create the function in the given sandbox and env. The dependencies are typically in lib/ including shared objects (.so files).

Note

tc builds the inlined zip using docker and the builder image that is compatible with the lambda runtime image.

Layer

If inline build is heavy, we can try to layer the dependencies:

name: ppd
runtime:
  lang: python3.10
  handler: handler.handler
  layers:
   - ppd-layer
build:
  kind: Layer
  pre:
   - yum install -y git
   - yum install -y gcc gcc-c++

Note that we have specified the list of layers the function uses. The layer itself can be built independent of the function, unlike Inline build kind.

tc build --kind layer
tc publish --name ppd-layer

We can then create or update the function with this layer. At times, we may want to update just the layers in an existing sandboxed function

tc update -s <sandbox> -e <env> -c layers

Note

AWS has a limit on the number of layers and size of each zipped layer. tc automatically splits the layer into chunks if it exceeds the size limit (and still within the upper total limit of 256MB)

Layer lifecycle management

On AWS, the layer versions are global and are not sandbox-aware. When associating layers with a function, we can pin the layers with monotonic versions as follows:

name: ppd
runtime:
  lang: python3.10
  package_type: zip
  handler: handler.handler
  layers:
   - ppd-layer:3

However, this may not be practical when dealing with a large number of functions. Additionally, there is no way to tag layers and annotate if the version is stable or unusable. To solve this problem, tc provides a simple mechanism to differentiate between stable and dev layers. When creating a layer by default using tc build --layer <layer-name>, the layer’s name is suffixed with the string -dev. On updating the sandbox with the layers or when creating/updating functions, tc will bump the functions to the latest “dev” layer versions. When a specific dev layer is ready to be promoted as stable, we do

tc build --promote --layer NAME [--version 123]

If the version is not specified, tc will promote the latest dev layer to stable.

If a sandbox is named stable, it will use the stable layers. This is configurable in TC_CONFIG.

To use latest stable layers in your dev sandboxes, do:

TC_USE_STABLE_LAYERS=1 tc update -s SANDBOX -e PROFILE -c functions/layers

Image

While Layer and Inline build kind should suffice to pack most dependencies, there are cases where 250MB is not good enough. Container Image kind is a good option. For example:

name: python-image-tree-example
runtime:
  lang: python3.10
  package_type: image
  handler: handler.handler
build:
  kind: Image

tc build --publish

Note that the child image uses the parent’s version of the image as specified in the parent’s block.

Syncing base images

While we can docker pull the base and code images locally, it is cumbersome to do it for all functions recursively by resolving their versions. tc build --sync pulls the base and code images based on current function checksums. Having a copy the base or parent code images allows us to do incremental updates much faster.

Inspecting the images

We can run tc build --shell in the function directory and access the bash shell. The shell is always on the code image of the current function checksum. Note that the code image using the Lambda Runtime Image as the source image.

Tip

It is recommended that the ECR repo has a / format. The label can be the image labels specified in function.yml:build (base, code etc)

Library

A library is a special kind of layer where there are no transitive dependencies packed into the layer artifact. This is useful if we have a directory of utilities.

lib/foo
   - bar
   - baz

tc build --kind library --name foo --publish -e <env>

foo now can be used a regular layer in function.yml:runtime:layers

Filesystems

We can mount a filesystem by specifying a runtime attribute mount_fs:

name: fn-with-fs
runtime:
  lang: python3.11
  package_type: image
  mount_fs: true
  handler: handler.handler

Extensions

We can also build extensions (which are technically layers).

tc build --kind extension

see Example

Using AWS-managed extensions

We can use an URI for specifing extensions, particularly AWS-managed extensions. For example:

name: example-ext
description: basic example
runtime:
  lang: python3.12
  package_type: zip
  handler: handler.handler
  extensions:
    - ssm:/aws/service/aws-parameters-and-secrets-lambda-extension/arm64/latest

Providers

Lambda

lambda is the default provider. The rest of the documentation in this page is specific to Lambda provider.

MicroVM

MicroVms provide the flexibility of EC2 instances, isolation of containers and performance of lambdas.

tc makes it simple to create and manage MicroVM Images and MicroVms, while still retaining the ergonomics of managing sandboxes. For example, let’s say we have a function directory with a function spec (function.yml) and the handler code (main.py).

tree .
.
├── function.yml
└── main.py

function.yml:

name: py-mvm
runtime:
  provider: MicroVm
  handler: 'python3 main.py'

build:
  kind: MicroVmImage
  bucket: {{ASSET_BUCKET}}

This, along with main.py that serves a HTTP server on specified port is all that is needed. tc will use the sane defaults to build and create the sandboxed microvm.

We could override the defaults in function.yml or {INFRA_DIR}/functions.json. handler is a command to run as the entry point instead of a function name. We can override bucket with TC_ASSET_BUCKET environment variable.

name: py-mvm
name: pyex
runtime:
  provider: MicroVm
  handler: 'python3 main.py'
  lang: python3.12
  role_name: tc-base-microvm-{{sandbox}}
  port: 8080

build:
  kind: MicroVmImage
  base_image_arn: arn:aws:lambda:us-west-2:aws:microvm-image:al2023-1
  build_role_arn: arn:aws:iam::{{account}}:role/tc-base-microvm-{{sandbox}}
  bucket: my-microvm-bucket

main.py looks something like this.

from http.server import HTTPServer, BaseHTTPRequestHandler
import json, time, os

class Handler(BaseHTTPRequestHandler):
    start_time = time.time()
    request_count = 0

    def do_GET(self):
        Handler.request_count += 1
        body = json.dumps({
            "message": "Hello from Lambda MicroVM!",
            "uptime_seconds": round(time.time() - Handler.start_time, 2),
            "requests_served": Handler.request_count,
            "pid": os.getpid()
        })
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.end_headers()
        self.wfile.write(body.encode())

HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()

cd py-mvm
tc create -s yoda -e dev

Microvm-specific attributes in function.yml. All of these are optional and tc tries to infer the defaults.

Attribute	Description
runtime.port	Port that the application listen on. Default 8080
runtime.role_name	Set a fixed role name
runtime.handler	Command to run the app in the microvm
runtime.mem	Minumum memory to use
runtime.microvm.ingress_network_connectors	ARN for ingress
runtime.microvm.egress_network_connectors	ARN for egress
runtime.microvm.max_duration	Max duration in secs before the microvm suspends (3600)
build.kind	MicroVm
build.version	Override the image version
build.base_image_arn	Base image arn to use to build the microvm image
build.build_role_arn	Role to use when building the microvm image
build.bucket	S3 bucket to store the code artifact
build.pre	List of commands to run locally before the codeartifact is generated
build.post	List of commands that run on the base image when the microvm image is generated

Note

If you already have an existing Dockerfile that you’d like to use in the microvm, place it in the function dir and tc will use it to build the microvm image. Make sure to EXPOSE ports and set the CMD to run. The advantage of specifying the build steps in function.yml instead of a custom Dockerfile is that tc can set up the right egress/ingress network connectors, allow the right ports and manage the versions.

Invoking

To invoke the microvm with a payload run the following. tc gets the Auth token and invokes the right sandboxed microvm.

tc invoke -s yoda -e dev -p '{"data": [1, 2]}'

While we can invoke the microvm via tc, we can also get the endpoint URL and token to use it outside of tc. To get the token and the endpoint to call:

tc list -s yoda -e dev

microvm_id: microvm-123

To invoke:
curl https://123.lambda-microvm.us-west-2.on.aws -X 'x-aws-proxy-auth: <token>'

Infra override

To override the execution role of the microvm (not the microvm Image), we can specify the IAM policy much like we do with other functions [See Configuration > Permissions]

name: pyvm
infra_dir: './infra'
runtime:
  provider: MicroVm
  handler: 'python app.py'
  port: 8080

build:
  kind: MicroVmImage

And we can override the function-specific permissions in ./infra/roles/pyvm.json

See Example

Lifecycle of Microvm Images

To delete the microvm, which actually suspends the microvm instead of deleting it.

tc delete -s yoda -e dev

To delete the microvm image, we can do a force delete.

tc delete -s yoda -e dev --force

There is an issue with creating a MicroVm Image with the same name. To circumvent it, we can bump the version of the MicroVm Image and recreate it.

build:
  kind: MicroVmImage
  version: 0.1.4

and tc create -s yoda -e dev

Testing

Invoke

By default, tc picks up a payload.json file in the current directory. You could optionally specify a payload file

tc invoke --sandbox main --env dev --payload payload.json

or via stdin

cat payload.json | tc invoke --sandbox main --env dev

or as a param

tc invoke --sandbox main --env dev --payload '{"data": "foo"}'

Tasks

We can define tasks in function spec. For example, consider this spec:

name: f1
runtime:
  lang: python3.10
tasks:
  clean: rm -f *.zip Dockerfile
  lint: pylint handler.py
  test: python run-test.py

cd f1
tc run --task :clean
tc run --task :lint [--trace]
tc run --task :test

Note

Note that the : in the task name specifies pre-defined tasks. The tasks are also templated.

We can also run abitrary tasks instead of predefined task:

tc run --task "git log ." --trace

Running tasks in a topology

Let’s say we have functions f1, f2, f3 in a topology. We can now run tasks recursively in all functions:

tc run --task :clean
Running [f1] rm -rf *.zip pkg
Skipping [f2]
Running [f3] rm -f *.zip

task is either predefined (with a : prefix) or arbitrary.