Progress Tracker

https://github.com/tc-functors/tc/examples/progress-tracker

The goal of this example is for us to learn about entity composition and Mutation entity. Additionally, we will attempt to embrace bottom-up design - start with datastructures and entity constraints, evolve the design or architecture based on the constraints and not think about infrastructure.

Preamble

In this example, we will try to start an arbitrary asynchronous job and capture it’s completion state, without any database. We also would like to have a HTML page that subscribes to these status changes preferably through websocket.

The kneejerk reaction, as developers, is to draw an architecture diagram (or ask ChatGPT you say?) with a zoo of AWS services. With functors, we start with datastructures and the entity composition constraints.

Rather than think of it like connecting random services in the cloud, we think of them as composition of entities without consideration of underlying services.

From the entity definitions, we know that the Mutation entity has a unique characteric of having abstract types and subscriptions to it’s change. What’s the the problem statement again ? We need to track the status changes in an arbitrary job. Perhaps we can try to model it as a Mutation.

Types

Let’s say we define a type called Job which captures the status of a job or task.

Job:
  id: String!
  status: String
  message: String

For the sake of this disucssion, assume that when this type’s values are changed, another entity gets triggered. It is as if the composition was dynamically late bound. A mutation represents a change in the values of an abstract type. This change results in triggering other entities and notifying subscribers to it. A mutation has 3 parts - the abstract type, resolver and subscriber. The resolver encapsulates the change.

Mutation

A mutation’s abstract type has 2 key rules:

1. It can be mutated or composed only by a route or event entity. See Composition Matrix
2. It can invoke or compose only with a function, table or event.

With this in mind, let’s draw a topology that adheres to these rules:

How did we arrive at this diagram ? We started by choosing a Mutation entity and identifying it’s composition constraints - A mutation is triggered by only event or a route. The route is typically in the boundaries.

In this diagram, a route mutates the abstract type which in turn triggers a function named starter. The starter function then runs the arbitrary job and triggers a CompleteTask event. This event mutates the abstract type (remember the first rule - type can be mutated only by event or route). In both the mutations, the subscribers were notified.

We had mentioned briefly about resolvers. A resolver is any backing entity (function, table etc) that returns data suitable for subscribers. For example, this is how we define startJob.

startJob:
  function: starter
  input: JobInput
  output: Job
  subscribe: true

startJob resolver has a backing entity function named starter that takes in data with input Type (JobInput) and returns data with output Type (Job). The subscriber, HTML page in this case, gets the value of Job as payload. The backing entity, however, is optional. If no resolver function or table is specified, the type’s value is returned as is with no augmentation. In this example however, we have the functions return data with the Job schema.

How do we write the infrastructure code for this ? duh, we don’t. We define the topology in abstract terms just like the diagram and let tc figure it out.

topology.yml

name: progress-tracker

routes:
  /startJob:
    method: POST
    mutation: startJob

events:
  CompleteTask:
    producer: adHoc
    mutation: completeJob

mutations:
  types:
    JobInput:
      id: String!
    Job:
      id: String!
      status: String
      message: String

  resolvers:
    startJob:
      function: starter
      input: JobInput
      output: Job
      subscribe: true

    completeJob:
      function: completer
      input: Event
      output: Job
      subscribe: true

This mutation behavior is provided by Graphql (Appync on AWS). tc provides a graphql transpiler that generates provider-specific graphql boilerplate from the above definition.

tc compose -c mutations -f graphql

type JobInput @aws_lambda @aws_iam {
  id: String!
  createdAt: AWSDateTime
  updatedAt: AWSDateTime
}

type Query { getJobInput(id: String!): JobInput
getJob(id: String!): Job
getEvent(id: String!): Event
 }

type Event @aws_lambda @aws_iam {
  detail: String
  createdAt: AWSDateTime
  updatedAt: AWSDateTime
}

type Mutation {
completeJob(detail: String ): Job
@aws_lambda @aws_iam

startJob(id: String! ): Job
@aws_lambda @aws_iam
 }
type Subscription { subscribeCompleteJob(id: String!): Job
   @aws_subscribe(mutations: ["completeJob"])
   @aws_lambda @aws_iam
subscribeStartJob(id: String!): Job
   @aws_subscribe(mutations: ["startJob"])
   @aws_lambda @aws_iam
 }
type Job @aws_lambda @aws_iam {
  id: String! status: String message: String
  createdAt: AWSDateTime
  updatedAt: AWSDateTime
}

This boilerplate is head-spinning to say the least. This is just for one Type and a couple of resolvers. With any reasonably sophisticated reactive system, we end up having several types and an unmanageable sphagetti of low-level code. Thankfully we don’t need to write it. tc generates it for us just by defining the type and it’s resolver(s).

Note

We don’t need to explicitly generate the provider-specific graphql code. tc does that internally when rendering the topology onto a sandbox.

Now that we have defined a Type that can be mutated and have set resolvers and subscribers to it. Let’s try and wire the mutations up.

Oh did I say, AWS Appsync also provides a routing endpoint to invoke any arbitrary resolver ? Let’s remove the routes block and just use the default routes.

topology.yml

name: progress-tracker

events:
  CompleteTask:
    producer: adHoc
    mutation: completeJob

mutations:
  types:
    JobInput:
      id: String!
    Job:
      id: String!
      status: String
      message: String

  resolvers:
    startJob:
      function: starter
      input: JobInput
      output: Job
      subscribe: true

    completeJob:
      function: completer
      input: Event
      output: Job
      subscribe: true

The output of these resolvers is the type holding the changed values. Anyone subscribing to this change, gets the status for the given id.

Subscriptions

We mentioned something about subscription earlier. This typically is a websocket subscription. Let’s assume we have a webapp that subscribes to this websocket and initiates startJob via graphql. See example

We can define the page/app to deploy in the topology.

topology.yml

name: progress-tracker

events:
  CompleteTask:
    producer: adHoc
    mutation: completeJob

mutations:
  types:
    JobInput:
      id: String!
    Job:
      id: String!
      status: String
      message: String

  resolvers:
    startJob:
      function: starter
      input: JobInput
      output: Job
      subscribe: true

    completeJob:
      function: completer
      input: Event
      output: Job
      subscribe: true

pages:
  app:
    dist: app/dist
    dir: app
    build:
       - npm install --quiet --no-audit
       - npm run build

tc create -s yoda -e dev

How does this look on AWS you ask ? Here is how:

To summarize, here is what we learned:

1. We can build sophisticated topologies using abstract mutation types. This is well suited for reactive webapps that need to display lot of events.
2. Any event entity can trigger a mutation and thus propagate that change to the subscriber, without an external database.
3. Every entity has a set of composition rules and properties. These rules magically map to constraints in the cloud (AWS et al). These entity rules help us evolve a bottom-up design or architecture.
4. We can define topologies in the abstract and have tc generate the underlying boilerplate - graphql, permissions, integrations, mapping templates etc.