Sometimes the file must be generated on demand. Maybe it’s a report for a user-selected date range, a data export based on filters, or something that only makes sense at that specific moment.

You might be tempted to generate that file directly within the AppSync resolver (e.g. a Lambda function), store it on S3 and then return the pre-signed URL; but this would be an anti-pattern.

• AppSync resolvers should stay thin
  Resolvers are not meant to perform heavy processing. Long-running or compute-heavy tasks don’t belong there.

• Hard time limits
  AppSync operations have a 30-second timeout. File generation, especially for large datasets, can easily exceed that.

• Reliability & retries
  File generation can fail. You may want retries, dead-letter queues, monitoring, and proper error handling — all of which are far easier to implement asynchronously.

In this article, I’ll show you how to properly decouple file generation from the AppSync request using Asynchronous Lambda invocation, Amazon EventBridge, and AppSync Subscriptions.

Architecture Overview

AWS AppSync supports long-running operations by allowing asynchronous Lambda function invocations as a data source (resolver). When the Lambda function is invoked asynchronously, the AppSync resolver returns immediately, while the Lambda function is allowed to run for up to 15 minutes.

We can leverage this capability to trigger our file-generation process without blocking the user's request.

Once the Lambda function finishes generating the file, it can store it in an Amazon S3 bucket, generate a pre-signed download URL, and send it back to the user through a Subscription using the AppSync–EventBridge integration.

Source

High-level flow

We use three GraphQL operations:

generateReport
This is the Mutation the user calls to express the intent to generate a file. It receives, for example, a start date and an end date and returns a requestId that uniquely identifies the process.

onReportGenerated
This is a Subscription that the client uses to subscribe to updates. It takes a requestId as input — the same requestId returned by the generateReport Mutation.
It is essentially a way of saying:

“Notify me once this specific report has been generated.”

reportGenerated
This is another Mutation that the backend invokes to trigger the above Subscription.

Workflow

1. The client invokes the generateReport Mutation.

2. The AppSync JS resolver generates a random requestId, invokes the GenerateReport Lambda function asynchronously, and passes it the requestId along with the user inputs.

3. AppSync immediately returns the requestId to the client.

4. The client receives the requestId and uses it to subscribe via onReportGenerated.

5. Once the Lambda function finishes generating the file, it stores it in an Amazon S3 bucket.

6. The Lambda function publishes a reportGenerated event to the event bus, including the requestId and the pre-signed download URL.

7. The AppSync reportGenerated Mutation target is triggered through the EventBridge integration.

8. AppSync pushes the event back to the client via the active Subscription.

9. The client receives the download URL and initiates the file download in the browser.

Error Handling and Other Considerations

When using asynchronous Lambda invocations, error handling behaves differently from synchronous calls.

By default, if an asynchronous Lambda execution fails, AWS automatically retries it twice after the initial failure (three attempts in total). If all retry attempts fail and no additional configuration is in place, the event is discarded.

If you need more control over failure handling, retries, and observability, there are several options to consider:

Lambda Destinations

You can configure Lambda function destinations for asynchronous invocations. This allows you to route failed events to another target — such as:

• An SQS queue (acting as a DLQ)

• An SNS topic

• EventBridge

• Another Lambda function

This gives you visibility into failures and the ability to reprocess or inspect failed events.

Introduce an SQS Queue

Instead of invoking the Lambda function asynchronously and directly from AppSync, you can place the event onto an Amazon SQS queue.

The Lambda function then consumes messages from the queue.

This approach provides:

• A configurable retry policy

• Built-in dead-letter queue support with redrive capability

• Better control over failure handling

• Buffering and backpressure management

Conclusion

In this article, we explored how to handle on-demand file generation in a clean, event-driven way using asynchronous Lambda invocations, EventBridge, and AppSync Subscriptions. Instead of blocking a GraphQL request, we decouple processing from the client interaction, allowing long-running tasks to execute reliably while still delivering real-time feedback when the file is ready.

The short answer is: you can’t — at least not directly.

While GraphQL is built on top of HTTP, it is not designed for transferring large binary payloads. GraphQL excels as a control plane, not as a data transport layer.

Fortunately, when working with AWS AppSync and the AWS ecosystem, file storage almost always means Amazon S3. And that opens the door to a much cleaner and more scalable approach: S3 pre-signed URLs.

In this article, I’ll walk through how to use AWS AppSync as the orchestrator for secure file uploads and downloads, while letting Amazon S3 handle the heavy lifting.

🏗️ High-Level Architecture

A pre-signed URL is a secure, time-limited URL that grants permission to upload or download an object from an Amazon S3 bucket. Instead of exposing AWS credentials or making your bucket public, the URL embeds a temporary signature that authorizes a specific operation (GET or PUT) on a specific object.

Once generated, the URL can be used directly by the client to interact with S3 in a secure and efficient way — without routing file data through your API.

Under the hood, a pre-signed URL is created using the AWS credentials of the entity that generates it. In the context of AWS AppSync, this is typically a Lambda function configured as an AppSync data source and resolver.

Those credentials define:

• Which bucket and object key can be accessed

• Which operation is allowed (upload or download)

• What type(s) of files can be uploaded (image, pdf, etc.)

• The maximum file size permitted

• How long the URL remains valid

• Metadata on the object (For example, the user id of the file uploader)

The overall flow looks like this:

1. The client requests an upload or download URL via AppSync

2. AppSync validates the request (authentication, authorization, file constraints, etc.)

3. AppSync generates a pre-signed URL using its execution role

4. The client uploads or downloads the file directly from S3

Source code

⬆️ Generate a pre-signed upload URL

The core idea is simple: the client first asks AppSync for permission to upload a file, and AppSync responds with everything the client needs to upload directly to S3.

First, we need a mutation that lets the client declare its intent to upload a file.

Typically, the mutation needs:

• The file name

• The file type (mime type)

and it returns:

• the pre-signed upload URL

• the fields that the client must include in the upload request

Those fields usually include things like:

• content type constraints

• metadata

• the upload policy

• the AWS SigV4 credentials

type Mutation {
  generateUploadUrl(input: GenerateUploadUrlInput!): UploadUrlResponse!
}

input GenerateUploadUrlInput {
  fileName: String!
  contentType: String!
}

type UploadUrlResponse {
  url: AWSURL!
  fields: AWSJSON!
}

We also need an AWS Lambda resolver. First, it validates that the user is trying to upload a file type that is accepted. Then, it adds a few additional constraints that the client must respect in order to upload the file:

• Min/max file size: e.g. 5 MB

• Validates the file type to prevent the user to upload unwanted files and enforces it into the policy

• Makes the acl private for the uploaded object

• Attaches metadata that tracks the ownership of the file on S3

This is also where you would do any sort of validation like:

• Does the user have the right role/permission to upload a file?

• Do they have access to uploads for this resource (project/group/etc.)?

• etc.

import { createPresignedPost } from '@aws-sdk/s3-presigned-post';
import { S3Client } from '@aws-sdk/client-s3';
import { AppSyncResolverEvent } from 'aws-lambda';
import { get } from 'env-var';

type GenerateUploadUrlInput = {
  input: {
    fileName: string;
    contentType: string;
  };
};

const s3Client = new S3Client({
  region: process.env.AWS_REGION,
});

const BUCKET_NAME = get('BUCKET_NAME').required().asString();

const ALLOWED_CONTENT_TYPES = ['image/jpeg', 'image/png', 'application/pdf'];

export const handler = async (
  event: AppSyncResolverEvent<GenerateUploadUrlInput>,
) => {
  const { fileName, contentType } = event.arguments.input;

  // Validate allowed content types
  if (!ALLOWED_CONTENT_TYPES.includes(contentType)) {
    return {
      errorMessage: 'Invalid content type',
      errorType: 'InvalidContentType',
    };
  }

  // TODO: Ensure the user is allowed to upload a file

  const userId = 'user123';

  const presignedPost = await createPresignedPost(s3Client, {
    Bucket: BUCKET_NAME,
    Key: `uploads/${userId}/${fileName}`,
    Conditions: [
      // Limit file size to 5 MB
      ['content-length-range', 0, 5 * 1024 * 1024],
      // Ensure the uploaded file type matches the onerequested by the user
      ['eq', '$Content-Type', contentType],
    ],
    Fields: {
      // Make the file private
      acl: 'private',
      // Add custom metadata
      [`x-amz-meta-user-id`]: userId,
    },

    // Set expiration time to 5 minutes
    Expires: 60 * 5,
  });

  return {
    data: {
      url: presignedPost.url,
      fields: presignedPost.fields,
    },
  };
};

The response looks like this

{
  "data": {
    "generateUploadUrl": {
      "fields": {
        "acl": "private",
        "x-amz-meta-user-id": "user123",
        "bucket": "appsyncs3presignedurlstac-attachmentsbucket8e14a36-hckzhkpulj39",
        "X-Amz-Algorithm": "AWS4-HMAC-SHA256",
        "X-Amz-Credential": "ASIAWMFUPLSIV2CJPSQ6/20251221/us-east-1/s3/aws4_request",
        "X-Amz-Date": "20251221T165443Z",
        "X-Amz-Security-Token": "IQoJb3JpZ2luX2VjEBk...",
        "key": "uploads/user123/picture.jpg",
        "Policy": "eyJleHBpcmF0aW....",
        "X-Amz-Signature": "0ba25cc..."
      },
      "url": "https://appsyncs3presignedurlstac-attachmentsbucket8e14a36-hckzhkpulj39.s3.us-east-1.amazonaws.com/"
    }
  }
}

The url is your S3 bucket url (The one the client can send the POST request to).

The fields must be passed in the form-data upload request performed by the client.

Use the returned info to upload your file from the client:

async function uploadFile({
  file,
  url,
  fields,
}: {
  file: File;
  url: string;
  fields: Record<string, string>;
}) {
  console.log("Uploading file...");

  const request = new XMLHttpRequest();

  const formData = new FormData();

  // Add all pre-signed fields
  Object.entries(fields).forEach(([key, value]) => {
    formData.append(key, value);
  });
  // Add the content-type
  formData.append("Content-type", file.type);
  // add the file
  formData.append("file", file);

  request.open("POST", url, true);
  request.send(formData);
}

Once the client uploads the file, it will be visible on the S3 bucket at the indicated location. The metadata will also be attached to the object.

⬇️ Generate a pre-signed download url

Now that they can upload files, your users might also want to download them. Just like with uploads, you can generate a pre-signed download URL.

To do this, we need a new Mutation that takes the file name (object key) as input and returns the URL to download the file.

type Mutation {
  generateDownloadUrl(fileName: String!): DownloadUrlResponse!
}

type DownloadUrlResponse {
  url: AWSURL!
}

Here too, a Lambda resolver generates the signed url. Optionally, this is also where you can perform authorization checks.

import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3';
import { AppSyncResolverEvent } from 'aws-lambda';
import { get } from 'env-var';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

type GenerateDownloadUrlInput = {
  fileName: string;
};

const s3Client = new S3Client({
  region: process.env.AWS_REGION,
});

const BUCKET_NAME = get('BUCKET_NAME').required().asString();

export const handler = async (
  event: AppSyncResolverEvent<GenerateDownloadUrlInput>,
) => {
  const { fileName } = event.arguments;

  const command = new GetObjectCommand({
    Bucket: BUCKET_NAME,
    Key: fileName,
  });

  const url = await getSignedUrl(s3Client, command, {
    expiresIn: 60 * 5,
  });

  return {
    data: {
      url: url,
    },
  };
};

Response example

{
  "data": {
    "generateDownloadUrl": {
      "url": "https://appsyncs3presignedurlstac-attachmentsbucket8e14a36-hckzhkpulj39.s3.us-east-1.amazonaws.com/uploads/user123/picture.jpg?X-Amz-Algorithm=..."
    }
  }
}

Conclusion

By using pre-signed S3 URLs, you keep file transfers out of your GraphQL API while still maintaining full control over who can upload or download what, and under which conditions. AppSync becomes the gatekeeper — validating intent, enforcing security rules, and issuing short-lived permissions — while Amazon S3 handles the heavy lifting with speed and scale.

This approach gives you:

• Secure, direct uploads and downloads

• Clear separation between control plane and data plane

• Fine-grained control over authorization, file size, MIME types, and metadata

You can find the code of this project on Github.

Thanks for reading, and happy building! 🚀

Like any other service, there are some best practices that you should follow to make sure that your APIs remain secure, maintainable, and scaleable. In this blog post, I'd like to share the best practices I learned after several years of practice.

Avoid Lambda Resolvers

AWS AppSync integrates directly with 6 data sources outside Lambda: DynamoDB, OpenSearch, EventBridge, HTTP, Amazon RDS, and None (a special data source that does not connect to any store). If you only need to interact with any of those sources, use them. The benefits are the following:

• Lower latency: By interacting directly with the data source, you remove one "hop" in the data flow, and you also avoid Lambda cold starts. Your API will feel faster and more responsive to the end user.

• Lower cost: Even if your Lambda functions are warm, you are still billed for invocation and runtime. By removing Lambda, you will only be billed for the invoked underlying data store (e.g. DynamoDB).

• Maintenance & Monitoring: Removing a Lambda function will also lower the burden and cost of maintenance and observability. It's one thing less to monitor in your system.

In the early days, the dreaded VTL language has often been a reason - ahem, an excuse - to use Lambda functions. Today, with support for JS resolvers and its utility package, it has become easier than ever to write blazing-fast resolvers. And if you prefer, it is even possible to write AppSync resolvers in TypeScript.

Don't use Direct Lambda Resolver

I just explained why you should avoid Lambda resolvers, but sometimes you have no other choice. You might need to run complex business logic, invoke unsupported data sources (are you sure about that?), you are limited by the APPSYNC_JS runtime, or just prefer writing resolvers in your favorite language. If you do so, using the direct Lambda integration might be tempting. Don't.

The biggest problem with the direct Lambda integration is that it moves the handling of the GraphQL response back to the Lambda function. This is especially a problem for error handling because it means that if you want to return an error to the user, you must throw an error from the lambda function code. This raises several issues:

• Less control over the error returned in the request

AWS AppSync allows you to return detailed errors using the util.error helper function. This gives you more flexibility on what you want to return to the user, and how. A thrown error offers less flexibility. Worst, it might leak information about your implementation to the outside world. You also can't control which error to return. For example, AppSync has a special util.unauthorized() error util that you can use to notify the user about unauthorized access.

• It messes with Observability

Do you want to be woken up in the middle of the night because a user sent an invalid request? Throwing errors from the Lambda function will count as a failed execution. It will show up as such in your metrics and might trigger alarms unnecessarily. Controlled errors (i.e. Unauthorized, NotFound, ValidationError, etc) should not cause the Lambda function to fail. Only code errors (a.k.a. bugs) should.

What to do instead?

Instead of throwing errors from the Lambda function, return them as part of the response payload.

export const handler = (event) => {
  //...

  if (!isAuthorized(user)) {
    // ❌ DON'T
    // throw new Error('Unauthorized');

    // ✅ DO
    return {
      error: {
        name: 'Unauthorized',
        message: 'Not allowed',
      }
    }
  }

  if (!data) {
    // ❌ DON'T
    // throw new Error('NotFound');

    // ✅ DO
    return {
      error: {
        name: 'NotFound',
        message: 'Resource Not Found',
      }
    }
  }

  return { data };
}

You can then do simple checks in your response handler to see if any error was returned and handle it there.

export const response (ctx) => {
    const { data, error } = ctx.result;
    if (error) {
        const { name, message } = error;
        if (name === 'Unauthorized') {
            util.unauthorized();
        } else {
            util.error(name, message);
        }
    } else {
        return data;
    }
}

Always Use Pipepeline Resolvers

Pipeline resolvers allow you to execute several operations and/or connect to several data sources to resolve a single GraphQL field. For example, you might need to retrieve two DyamoDB items and merge them, or add an authorization layer in a multi-tenant application.

Even if a field only requires one operation, I tend to always default pipeline resolvers. I just create a one-function pipeline. Why? Because if I ever need to add an operation, I can just add a function in the pipeline without refactoring everything. Easy.

Another reason is that pipeline functions are re-useable (They can be in more than one resolver). Which brings me to my next point.

Create Re-useable Resolver Handlers

In a GraphQL API, some resolvers will often look similar. For example, resolvers for Query.getUser(id: ID!) and Order.user both resolve to a user. The only difference might just be where the id argument comes from (e.g. ctx.arguments.id and ctx.source.userId respectively). The rest is identical: both get a user from the data source with an id. This means that you can use the same resolver code, or pipeline function for both use cases. You don't have to duplicate them.

One neat trick is to use the context object to pass the data that you need to the handler (e.g. using ctx.prev or ctx.stash).

Example:

// get user request handler
export function request(ctx) {
  return {
    operation: 'GetItem',
    key: util.dynamodb.toMapValues({ id: ctx.prev.userId }),
  };
}

Using pipeline resolvers makes it even easier. You can use the before handler to pass the value.

// Query.user "before" handler
export function request(ctx) {
  return {
    userId: ctx.arguments.id,
  };
}

// Order.user "before" handler
export function request(ctx) {
  return {
    userId: ctx.source.userId,
  };
}

This will reduce the amount of code and maintenance. Be careful though, if the requirements of one resolver change, but not the other, you might need to separate them again.

Use Resolver Batching

OK, if there is one good reason to use a Lambda resolver after all, it would be batching. By default, AppSync will run all the resolvers of the same level of nesting in parallel (up to a certain limit), and as far as I could observe, it also tries to duplicate identical queries (e.g. DynamoDB GetItem with the same key). But for even better efficiency and control over how nested resolvers are executed, you can use AppSync's batching feature on Lambda resolvers.

It allows you to receive all the nested resolver events in a single Lambda function invocation (this is configurable). My fellow Community Builder Rich Buggy wrote a great article about this. I recommend you give it a read.

Don't Keep Secrets in Resolvers Code

If you are using a resolver to access external resources (e.g. a third-party API) that require an API key, or secret, you should not hardcode them in the resolver's code. You should also not keep them in environment variables, as explained in the documentation. Instead, use Secret Manager, or the Systems Manager Parameter Store to retrieve them. You can achieve that with a Pipeline resolver that fetches the secret first, followed by the external request.

One small drawback is that the value will need to be fetched for every single request. Unlike in a Lambda function, you cannot store the secret outside the handler and reuse it for further invocations. This can increase both latency and cost.

Use Caching

AppSync allows you to add a caching layer in front of your API. You can either choose to cache full requests or granularly decide which resolvers use cache or not. You can even control what the caching key is for each one of them.

Although caching adds an extra fixed cost (based on the instance type), it can actually save you other costs and even result in being cheaper compared to hitting the same data source over and over again.

Of course, needless to say, enabling cache will also boost the performance of your API for requests that have been previously saved, reducing latency.

Choose the Right Authorization Method

AWS AppSync supports 5 different authorizers. To ensure the security of your API, it's important to use the correct one for every use case.

• Cognito User Pools

Cognito User Pool is a fully managed AWS service that serves as a user directory. Use it for user-facing APIs which require identifying which user is executing the request. With Cognito User Pools, you can even have advanced control over who can access some mutations, queries, or subscriptions through the @aws_cognito_user_pools directive. This authorization filter will happen before the resolver is even called.

• API Key

This authorizer will provide you with one or more API keys that you use to invoke the API. This authorizer is a simple way to get started with AWS AppSync, and create quick proof of concepts or demos.

Another common use case for API keys is for "public" APIs. By default, AppSync does not allow any request to be unauthenticated. To go around that, you can create an API key and use it in your front end, but keep in mind that the API key will be publicly visible. For more secure guest/anonymous requests, you can use Cognito Identity Pools in conjunction with an IAM authorizer.

• IAM

The IAM authorizer requires every request to be signed with IAM SigV4. Use it to invoke queries or mutations from known and trusted actors with an IAM role, such as your back end. This is also the authorizer required for the EventBridge-AppSync integration.

• OpenId Connect (OIDC)

OIDC can be used to integrate with third-party authorizers. e.g. Okta, or Auth0. It requires a valid JWT created by a vendor. AppSync will validate the token after it receives it before allowing, or denying the request.

Use it for existing user bases outside AWS, or if you need/want to use an external provider.

• Lambda Authorizer

This method gives you the most flexibility, but it also puts all the responsibility of security on you. Only use it if you know what you are doing, or have a specific advanced use case.

Protect your API

After you deploy an API, you want to protect it against bad actors.

I just talked about it: one of the most important aspects of security is to only allow access to users/services that you trust by using the right authorizers. But you don't have to limit it to that. You also need to protect it against possible attacks.

There are several things you can do to prevent that.

• Set query depth limits

The nature of GraphQL allows you to nest types and fields together. This allows the client to retrieve all the data it needs in a single request. By default, there is no limit to how much you can nest queries, and this could be used as an exploit by attackers to overload your system.

Imagine the following query:

query user(id: "123") {
    name
    friends {
        name
        friends {
            name
            friends {
                name
                friends { ... }
            }
        }
    }
}

I could keep going with it as much as I'd like. Although AppSync has some limits in place (e.g. 30 seconds execution time), such queries can increase the complexity of requests, which increases the consumption of tokens. It will also likely incur additional costs (e.g. more DynamoDB requests).

To protect you against that, you can configure how deep queries can go. Any request that goes over that limit will be blocked.

• Add a Web Application Firewall (WAF)

AWS WAF is a fully managed service designed to protect APIs against common exploits. It is integrated with AWS AppSync and can be used as an extra layer of security.

With AWS WAF, you can for example block or allow access to certain IP addresses, rate limit callers, block access from certain countries, etc. AWS WAF acts as an extra layer on top of AppSync, meaning that only allowed requests will hit your API.

• Use private APIs

AppSync has support for private APIs. If your API is used for internal purposes only, you can deploy it in a private network. Only systems/clients that are in the same VPC will be able to access it.

• Disable Introspection

One of the great benefits of GraphQL is that it's self-documented. There is a special request that you can make to any API, called an introspection query. This allows the caller to explore the schema of the API and all its types and fields. However, there are cases where you might want to disallow public users from introspecting the schema. For example, you might want them to avoid discovering hidden, or private queries and mutations.

If this is your case, AppSync allows you to disable introspection through the API settings.

Disable Verbose Logging on Production

AppSync CloudWatch logs can be very useful; especially to debug AWS AppSync requests. But it can also be very expensive when enabled on production.

To avoid excessive costs, you can keep the log level to ERROR to ensure that only errors are logged for further investigation.

Alternatively, you can use log sampling. This is not supported natively by AWS AppSync, but Yan Cui explains how you can achieve it in this blog post.

Use Infrastructure as Code

Last, but not least, use infrastructure as code (IaC). This one might seem obvious but many tutorials, demos, etc. that you can find online often rely on the AWS console. In a real-world application, you want to keep your API's code versioned and make it reproducible and re-deployable in various environments. The choice of IaC is up to you though. It could be the CDK, the Serverless Framework, or SAM, for example. Just use IaC!

Conclusion

AWS AppSync is an amazing service for writing efficient, scaleable GraphQL API with serverless technology. Following the best practices will only make it better.

To improve this experience even more, I created GraphBolt, a desktop application to help you test and debug AppSync APIs, and follow those best practices. Try it today for free!

If you are new to AppSync, I also created a free workshop where you can get started.

Thanks for reading!

If you like this kind of content, feel free to follow me on Hashnode, X, and LinkedIn. You can also subscribe to this blog's newsletter to receive notifications about new posts.

Last year, AWS announced support for Amazon EventBridge as a Data Source for AWS AppSync, and more recently, it was the turn of EventBridge to integrate with AppSync. The two-way integration between those two services opens many opportunities.

In this article, I will show you how you can create an event-driven system with real-time updates using AWS AppSync, EventBridge, and Step Functions.

What We Will Build

We will build a simple food-ordering service where users can place an order for their meal. The order will be received by the back-end through an AWS AppSync GraphQL API. The Orders service will process it and send real-time updates to the user about the different status updates (e.g. Paid, Preparing, Out for delivery, Delivered, etc.). The Orders service is a Step Functions workflow that orchestrates the different steps of the process, while Amazon EventBridge asynchronously coordinates all the services.

Here is a high-level diagram of what this looks like.

To build this, we will use the CDK as our Infrastructure as Code (IaC).

The AppSync API

First, we'll need an AppSync API. It will be the entry point for our users to place orders. Here is the simplified schema.

type Product {
  name: String!
  quantity: Int!
}

type Order {
  id: ID!
  status: OrderStatus!
  products: [Product]!
  createdAt: AWSDateTime!
  updatedAt: AWSDateTime!
}

input ProductInput {
  name: String!
  quantity: Int!
}

input CreateOrderInput {
  products: [ProductInput!]!
}

input NotifyOrderUpdatedInput {
  id: ID!
  status: OrderStatus!
  updatedAt: AWSDateTime!
}

type OrderUpdated @aws_api_key @aws_iam {
  id: ID!
  status: OrderStatus!
  updatedAt: AWSDateTime!
}

enum OrderStatus {
  PENDING
  PAID
  PREPARING
  OUT_FOR_DELIVERY
  DELIVERED
  CANCELLED
}

type Mutation {
  createOrder(input: CreateOrderInput!): Order
  notifyOrderUpdated(input: NotifyOrderUpdatedInput!): OrderUpdated! @aws_iam
}

type Subscription {
  onOrderUpdated(id: ID!): OrderUpdated
    @aws_subscribe(mutations: ["notifyOrderUpdated"])
}

The schema contains 2 mutations:

• createOrder: can be used by the front end to place an order

This mutation is attached to a pipeline resolver with two functions: createOrder and putEvent.

createOrder uses a DynamoDB data source to persist the order in a table, and prepares an order.created event containing its details before storing it in the stash.

// createOrder resolver
import { util } from '@aws-appsync/utils';
import { put } from '@aws-appsync/utils/dynamodb';

export const request = (ctx) => {
  const order = {
    id: util.autoId(),
    status: 'PENDING',
    products: ctx.arguments.input.products,
    createdAt: util.time.nowISO8601(),
    updatedAt: util.time.nowISO8601(),
  };

  return put({
    key: {
      id: order.id,
    },
    item: order,
  });
};

export const response = (ctx) => {
  // Here, we prepare the `order.created` event for the next step
  ctx.stash.event = { detailType: 'order.created', detail: ctx.result };

  return ctx.result;
};

putEvent is attached to an EventBridge data source. It receives an event (from the stash) and puts it into the event bus. We'll see later how it is used.

// putEvent resolver
import { util } from '@aws-appsync/utils';

export function request(ctx) {
  const { event } = ctx.stash;

  if (!event) {
    console.error('No event found in stash');
    util.error('InternalError');
  }

  return {
    operation: 'PutEvents',
    events: [
      {
        source: 'order.api',
        ...event,
      },
    ],
  };
}

export function response(ctx) {
  return ctx.prev.result;
}

• notifyOrderUpdate will be used by the back end to send real-time updates.

It is attached to a resolver with a None data source because this mutation does not persist anything in any store. Its only purpose is to trigger a notification to the onOrderUpdated subscription.

The API also uses two authorizers: API_KEY (for users to place orders) and IAM for the back end to call the notifyOrderUpdated mutation (this is a requirement from the EventBridge integration).

The Step Functions State Machine

The second component of our application is a Step Functions state machine. This state machine orchestrates the different steps of processing an order: payment, waiting for the restaurant to prepare the meal, waiting for delivery, etc.

For simplicity, I used wait tasks to simulate the workflow. In a real application, the state machine would wait for real processes (e.g. payment gateway), or humans (e.g. meal preparation) before transitioning between the different states (hint: probably using the callback pattern).

The most important part is the EventBridgePutEvents tasks. After every step in the order processing workflow, it receives the updated order and puts an event into the event bus with all its details. In the next section, we'll see how they are being handled.

// lib/state-machine-construct.ts

createNotifyUpdate(name: string) {
  return new EventBridgePutEvents(this, `Notify ${name}`, {
    entries: [
      // generates an event from the task input
      {
        source: 'order.processing',
        detailType: 'order.updated',
        detail: TaskInput.fromObject({
          'order.$': '$.order',
        }),
        eventBus: this.eventBus,
      },
    ],
    resultPath: '$.eventBridgeResult',
  });
}

Put It All Together

So far, we've learned about the AppSync API, and the Step Functions workflow. Did you notice that both those services place events into EventBridge? What I haven't shown you yet is how those events are used and how they play together.

In our application, the first thing that happens is a user placing an order. That's the starting point of the whole process, and it results in an order.created event. If you look at our state machine construct, you'll notice that it subscribes to those events to start a new execution.

new Rule(this, 'OrderHandlerRule', {
  eventBus: eventBus,
  eventPattern: {
    // subscribe to order.created events, coming from the api.
    source: ['order.api'],
    detailType: ['order.created'],
  },
  targets: [
    new SfnStateMachine(sm, {
      input: RuleTargetInput.fromObject({
        order: EventField.fromPath('$.detail'),
      }),
    }),
  ],
});

This means, that each time an order is placed, a Step Functions workflow will start with the order details as its input.

Then, as the state machine goes over all the different steps of the order processing, it emits order.updated events, which in turn are picked up by an EventBridge-AppSync rule.

new CfnRule(scope, 'UpdateOrder', {
  eventBusName: eventBus.eventBusName,
  eventPattern: {
    source: ['order.processing'],
    'detail-type': ['order.updated'],
  },
  targets: [
    {
      id: 'OrderUpdated',
      arn: (api.node.defaultChild as CfnGraphQLApi).attrGraphQlEndpointArn,
      roleArn: ebRuleRole.roleArn,
      appSyncParameters: {
        graphQlOperation: `mutation NotifyOrderUpdated($input: NotifyOrderUpdatedInput!) { notifyOrderUpdated(input: $input) { id status updatedAt } }`,
      },
      inputTransformer: {
        inputPathsMap: {
          id: '$.detail.order.id',
          status: '$.detail.order.status',
          updatedAt: '$.detail.order.updatedAt',
        },
        inputTemplate: JSON.stringify({
          input: {
            id: '<id>',
            status: '<status>',
            updatedAt: '<updatedAt>',
          },
        }),
      },
    },
  ],
});

This is a direct integration between EventBridge and AWS AppSync. EventBridge invokes the notifyOrderUpdated mutation and uses the event's details attributes to build the input variables of the request. In turn, the mutation emits a message to all subscribers of the onOrderUpdated subscription.

With all this in place, the front end can use the onOrderUpdate subscription to get real-time updates as soon as it places an order.

subscription OnOrderUpdated($id: ID!) {
  onOrderUpdated(id: $id) {
    id
    status
    updatedAt
  }
}

Here is a simulation using GraphBolt. Look at the subscription messages coming in. Pay attention to the status field.

Conclusion

Leveraging event-driven architecture with AWS AppSync, Step Functions, and EventBridge offers a seamless solution for building efficient, asynchronous, and real-time applications. In this article, I explained with a practical use case how you can achieve it, and the best part is that we almost did not write any code for it (outside the infrastructure).

If you liked this content, please share! You can also follow me on X, Hashnode, and LinkedIn for more, or subscribe to this blog's newsletter.

Thank you!

In this blog post, I will explain them and tell you how to make the best out of them.

Say "Hi" to Projects 👋

When you work on a project, you often need to work on the same service in its different stages (also known as Environments). This is especially true for serverless applications where you often have 3 or more of them. i.e. You probably have dev, staging, prod; plus, every dev might have their own account/environment, without mentioning a bunch of ephemeral (or ci/cd) stacks. That's a lot of environments all representing the same application or service(s)!

More concretely, if you're deploying the same AppSync API to those different environments, you probably want to be able to manage (and test) them in one convenient place.

In this new version of GraphBolt, you can now create Projects to logically group all your API instances, even if they live in different AWS accounts and regions.

After selecting AWS profiles and regions, GraphBolt will automatically explore your accounts and discover your AppSync APIs. You can then choose which ones to include in the project.

After creating a Project, you can access all your favorite GraphBolt tools, within a single and homogenized scope. This means that, for example, Operation and Resolver collections now belong to the Project, and not to individual APIs (as it was the case before). When selecting an operation (i.e. a Query, Mutation, or Subscription), you can execute it against any of the APIs included in the project.

Tabbing Navigation 🗂

Until now you were only able to work on one thing at a time, whether it was an operation, a resolver, etc. This was very limiting in some cases, like executing a Subscription followed by a Mutation to check if the message is delivered as expected. This has been a long-standing issue with GraphBolt.

Starting from this version, every "view" will open in a new tab, making it much easier to work with several things at the same time.

Request History 🕓

Did you know that GraphBolt has been silently keeping track of all your requests for a while now? (Don't worry everything is still stored on your machine, I don't see them). However, there was no way to access them from the UI. This is now a done thing! You'll find a new drop-down menu on top of the response panel that lets you see the execution history.

Per Request Authentication 🔒

So far, the authentication settings have been global per API. This was a problem when some operations use a different authentication method than others. For example, you could have a "public" query named getPost() which uses the API key authorizer, while the createPost() mutation uses Cognito User Pools.

You can now use a different authentication method for every individual operation.

What Else?

Those are by far my favorite features, but I did not stop there. This new version also includes some bug fixes and other smaller improvements, notably in the UI and the onboarding process.

What's Next?

There are plenty of other great features I'd like to add that I will work on in the following weeks/month. What features would YOU like to see? Let me know!

Conclusion

This new release is one of the biggest ever. Projects and tabs were much-needed features and I truly hope that they will boost your AppSync development workflow.

Start using it now, and don't hesitate to send me your questions and feedback.

Today, we are one step closer to that. We just released a new version of GraphBolt with three new amazing tools.

1. API details ℹ️

In the new Details tab, you will find basic information about your APIs such as the GraphQL endpoints (for queries and real-time), the ARN, tags, etc.

2. Metrics 📈

GraphBolt is committed to making developers' lives easier, but once your APIs hit production, you want to keep an eye on how they perform, right?

GraphBolt now features CloudWatch metrics. Number of Requests, Errors, Latency, Consumed tokens, Caching, and even the new Real-time metrics. It's all there!

3. Stats 📊

Have you ever wondered which one of your AppSync resolvers is the most invoked? Or which one is the slowest? Maybe there is something you can do about it and improve latency and UX.

Under Statistics, you will find charts showing the most invoked resolvers and their execution time (average, min, and max). You can even choose a time range from which you would like to get data.

All these features are available for download today!

GraphBolt is still in beta and as such, it remains free to use for everyone.

One of the great benefits of AppSync is the ability to connect GraphQL schemas to data sources like DynamoDB by writing simple functions called resolvers, without having to worry too much about data fetching logic. However, when things are not working as expected, it becomes necessary to understand how those resolvers interact with your data sources and the GraphQL schema in order to fix the problem.

Like many other services, AppSync gives you the option to send logs into AWS CloudWatch. When enabled, AppSync writes records describing how resolvers interact with the schema and the data sources.

In this article, I will teach you how to find your way in those logs and understand them so you can effectively diagnose issues.

Enabling the logs

Before you can dive deep into the logs, you must first enable verbose logging in your API settings. Settings under Logging should look like this:

• Enable Logging is on ✅

• "Include request headers, response headers, context, and evaluated mapping templates, regardless of field logging level": is checked ✅

• Field resolver log level: All

You can also use your favorite IaC to do so.

If everything is set up properly, you should start seeing logs in CloudWatch after you execute a query.

ℹ️ AWS AppSync Log Groups are usually named as follows: /aws/appsync/apis/{apiId}

The anatomy of the AppSync log groups

AppSync logs provide very useful information about each step of the execution of the request. Most of the logs are JSON formatted, which makes them easy to parse, read, and search. Let's get to know them:

Request logs

At the top and bottom of the log stack, you will find some general details about the executed request.

• GraphQL Query

This line gives you information about the query that was executed. It comprises the query itself, the operation name (if any), and the variables (if any). It is prefixed with the request ID.

Example

f89cbf60-2431-44ca-a6f5-44eec32e42e6 GraphQL Query: query GetPost($postId: ID!) {
  getPost(id: $postId) {
    id
    title
    content
    author {
      name
    }
  }
}, Operation: GetPost, Variables: {"postId":"22d38480-ca09-4d6d-9582-428c15af0bb2"}

• RequestSummary

It provides a summary of the execution among which the status (the HTTP response code) and latency expressed in nanoseconds.

{
  "logType": "RequestSummary",
  "requestId": "f89cbf60-2431-44ca-a6f5-44eec32e42e6",
  "graphQLAPIId": "seiizsmq2zfsfk5nkxfl4h46j4",
  "statusCode": 200,
  "latency": 101516439
}

• Request Headers

The request headers of the HTTP request. Prefixed with the request ID.

f89cbf60-2431-44ca-a6f5-44eec32e42e6 Request Headers: {content-length=[228], cloudfront-viewer-country=[FR], sec-fetch-site=[cross-site], x-amzn-requestid=[f89cbf60-2431-44ca-a6f5-44eec32e42e6], x-amz-user-agent=[aws-amplify/3.0.7], x-forwarded-port=[443], via=[2.0 2f66f74411c5a2447c09372eb79e674e.cloudfront.net (CloudFront)], authorization=[****m3s1Lw], sec-ch-ua-mobile=[?0], cloudfront-viewer-asn=[12322], cloudfront-is-desktop-viewer=[true], host=xxxxx.appsync-api.us-east-1.amazonaws.com], content-type=[application/json], sec-fetch-mode=[cors], x-forwarded-proto=[https], accept-language=[en-GB], x-forwarded-for=[88.162.xxx.xxx, 18.68.xxx.xxx], accept=[*/*], cloudfront-is-smarttv-viewer=[false], sec-ch-ua=[" Not A;Brand";v="99", "Chromium";v="104"], x-amzn-trace-id=[Root=1-65245176-4d851b414d05bec92fc373de], cloudfront-is-tablet-viewer=[false], sec-ch-ua-platform=["macOS"], x-amzn-remote-ip=[xx.xxx.xx.xxx], cloudfront-forwarded-proto=[https], accept-encoding=[gzip, deflate, br], x-amz-cf-id=[839VZnCL8GNXiA_RIql-x5zBvjIb2RWGgQW1H3hr4V8sYqQKnC2Vsg==], user-agent=[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) GraphBolt/0.4.0 Chrome/104.0.5112.102 Electron/20.1.0 Safari/537.36], cloudfront-is-mobile-viewer=[false], x-amzn-appsync-is-vpce-request=[false], sec-fetch-dest=[empty]}

Note: For security reasons, authorization and other sensitive values are obfuscated in CloudWatch. In this example, I have also hidden other private information myself.

• Response Headers

The response headers of the HTTP request. Prefixed with the request ID.

f89cbf60-2431-44ca-a6f5-44eec32e42e6 Response Headers: {X-Amzn-Trace-Id=Root=1-65245176-4d851b414d05bec92fc373de, Content-Type=application/json; charset=UTF-8}

💡 Tip: If X-Ray is enabled in the AppSync settings, X-Amzn-Trace-Id is the trace id. You can easily copy it to find the x-ray traces.

• Tokens Consumed

The total number of tokens consumed by the request. For more info about tokens, see the documentation.

Also see: The AWS AppSync Limits You Need To Know

f89cbf60-2431-44ca-a6f5-44eec32e42e6 Tokens Consumed: 1

Resolver logs

Aside from the logs referring to the global execution of the request, you will also find a series of lines that relate to every resolver execution in the request. Those logs are in JSON format and provide information about the execution of the resolver's handler functions.

As a general rule, you will usually find a pair of records for each resolver execution (one resolver might be executed more than once): one for the request handler, and one for the response handler (a.k.a. request and response mapping templates). You can recognize them by the logType property: respectively RequestFunctionEvaluation and ResponseFunctionEvaluation, or RequestMapping and ResponseMapping, depending on the runtime (APPSYNC_JS or VTL).

For pipeline resolvers, the same applies to every function in the pipeline (i.e. you will find a pair of logs for every function execution). In addition to that, you'll see an additional pair corresponding to the before and after handlers, wrapping all the functions logs: BeforeRequestFunctionEvaluation and AfterResponseFunctionEvaluation(APPSYNC_JS), or BeforeMapping and AfterMapping (VTL).

Here are the relevant fields that you will find in the records and their meaning:

Here is an example of a full log record.

{
  "logType": "RequestFunctionEvaluation",
  "fieldName": "getPost",
  "resolverArn": "arn:aws:appsync:us-east-1:379730309663:apis/seiizsmq2zfsfk5nkxfl4h46j4/types/Query/resolvers/getPost",
  "functionName": "getPost",
  "fieldInError": false,
  "evaluationResult": {
    "operation": "GetItem",
    "key": {
      "id": {
        "S": "22d38480-ca09-4d6d-9582-428c15af0bb2"
      }
    }
  },
  "parentType": "Query",
  "path": [
    "getPost"
  ],
  "requestId": "fea8bf05-19a9-4b82-b8fa-a854d8f29737",
  "context": {
    "arguments": {
      "id": "22d38480-ca09-4d6d-9582-428c15af0bb2"
    },
    "prev": {
      "result": {}
    },
    "stash": {},
    "outErrors": []
  },
  "errors": [],
  "graphQLAPIId": "seiizsmq2zfsfk5nkxfl4h46j4",
  "functionArn": "arn:aws:appsync:us-east-1:379730309663:apis/seiizsmq2zfsfk5nkxfl4h46j4/functions/hdmi2bvayzh6zhoudhm6uc5voe"
}

In the above example, we can see the record for the request handler of the Query.getPost resolver. Since it's in a pipeline resolver, we also see the functionName (getPost). We also find the query that was sent to the data source (DynamoDB) under evaluationResult.

Tracing logs

Tracing logs provide extra information about the resolution of a field, including those that are not attached to a resolver.

Example:

{
  "duration": 41818411,
  "logType": "Tracing",
  "path": [
    "getPost"
  ],
  "fieldName": "getPost",
  "startOffset": 135988,
  "resolverArn": "arn:aws:appsync:us-east-1:379730309663:apis/seiizsmq2zfsfk5nkxfl4h46j4/types/Query/resolvers/getPost",
  "requestId": "81f08e54-f589-4775-b96f-7bb8ed218539",
  "parentType": "Query",
  "returnType": "Post!",
  "graphQLAPIId": "seiizsmq2zfsfk5nkxfl4h46j4"
}

Most fields are identical to resolver logs; and here are the other interesting ones:

Custom logs

Did you know that you can write custom logs from your resolvers?

In a JavaScript resolver, you can call console.log() or console.error(). If you're still using VTL, use $util.log.info() or $util.log.error(). You will see them appear in CloudWatch along with the file name (if you included a source map), the line number, and the log level (INFO or ERROR). Use them to log custom data or debugging information.

console.log('Hello from a JS resolver request handler!!');

fea8bf05-19a9-4b82-b8fa-a854d8f29737 INFO - resolvers/getPost.ts:11:2: "Hello from a JS resolver request handler!!"

You can even write plain objects, and they will appear as JSONs.

console.log({ hey: 'there' });

2ab4b798-8941-47d1-9f66-453389e5d67c INFO - resolvers/getPost.ts:12:2: {"hey":"there"}

Finding the right logs

AppSync logs in CloudWatch can be very (very) verbose, and it's not always easy to find what you're looking for among hundreds of lines of logs.

Luckily, AWS AppSync returns the request ID in the x-amzn-requestid HTTP response header.

If you're using an app like Postman to execute queries, you can easily find it and copy it (You can also find it in the Network tab of your browser's developer console). You can then paste it into CloudWatch to filter the logs of that particular request.

Note: You'll need to wrap the ID in quotes.

Alternatively, if you know exactly what you are looking for, you can also use a JSON filter.

For example, if you are interested in the request handler of the getPost query, for a request with the ID 278d0f86-83e8-417f-8985-6fa57ab7e5bd, you can apply the following filter:

{ $.requestId = "278d0f86-83e8-417f-8985-6fa57ab7e5bd"
  && $.logType = "RequestFunctionEvaluation"
  && $.fieldName = "getPost" }

ℹ️ Note: Logs can take a few seconds to show up in CloudWatch. If you just executed the request, you might need to refresh a few times before you see them.

Debugging requests

Once you understand how logs are structured, how they appear in CloudWatch, and how to find them, it becomes much easier to debug requests. Let's take a common mistake as an example.

Imagine you wrote the following JS resolver code:

export function request(ctx) {
  return {
    operation: 'GetItem',
    key: util.dynamodb.toMapValues({
      id: ctx.arguments.postId,
    }),
  };
}

export function response(ctx) {
  return ctx.result;
}

You're sending this query.

query {
  getPost(id: "f9f621c6-d9da-4880-a148-69535a118494") {
    id
    title
    content
  }
}

And this is the GraphQL schema corresponding to that request.

type Query {
  getPost(id: ID!): Post!
}

After executing the request, you receive this response:

{
  "data": null,
  "errors": [
    {
      "path": [
        "getPost"
      ],
      "locations": null,
      "message": "Cannot return null for non-nullable type: 'Post' within parent 'Query' (/getPost)"
    }
  ]
}

You are certain that the post exists, you checked in your DynamoDB table. So what is going on? Let's have a look at the logs. First, let's identify the request handler logs to see if we see anything. We're looking for a RequestFunctionEvaluation log for the getPost resolver.

After looking in CloudWatch, I found it:

{
  "logType": "RequestFunctionEvaluation",
  "path": [
    "getPost"
  ],
  "fieldName": "getPost",
  "resolverArn": "arn:aws:appsync:us-east-1:111111111111:apis/seiizsmq2zfsfk5nkxfl4h46j4/types/Query/resolvers/getPost",
  "requestId": "f9f621c6-d9da-4880-a148-69535a118494",
  "context": {
    "arguments": {
      "id": "22d38480-ca09-4d6d-9582-428c15af0bb2"
    },
    "stash": {},
    "outErrors": []
  },
  "fieldInError": false,
  "evaluationResult": {
    "operation": "GetItem",
    "key": {
      "id": {}
    }
  },
  "errors": [],
  "parentType": "Query",
  "graphQLAPIId": "seiizsmq2zfsfk5nkxfl4h46j4"
}

The first thing you would usually look at would be the evaluationResult which is the request that was sent to the data source (in this case DynamoDB). Here, something does not look right. You can see that the key value of the GetItem operation is empty. The ID is not sent correctly to DynamoDB.

We can confirm that by looking at the corresponding ResponseFunctionEvaluation record. DynamoDB complains about an invalid key (See context.error).

{
  "logType": "ResponseFunctionEvaluation",
  "path": [
    "getPost"
  ],
  "fieldName": "getPost",
  "resolverArn": "arn:aws:appsync:us-east-1:379730309663:apis/seiizsmq2zfsfk5nkxfl4h46j4/types/Query/resolvers/getPost",
  "functionName": "getPost",
  "requestId": "81f08e54-f589-4775-b96f-7bb8ed218539",
  "context": {
    "arguments": {
      "id": "22d38480-ca09-4d6d-9582-428c15af0bb2"
    },
    "prev": {
      "result": {}
    },
    "stash": {},
    "error": {
      "message": "The provided key element does not match the schema (Service: DynamoDb, Status Code: 400, Request ID: HUDP47TVTALEGM8NB90JVLK5RRVV4KQNSO5AEMVJF66Q9ASUAAJG)",
      "type": "DynamoDB:DynamoDbException"
    },
    "outErrors": []
  },
  "fieldInError": false,
  "errors": [],
  "parentType": "Query",
  "graphQLAPIId": "seiizsmq2zfsfk5nkxfl4h46j4",
  "functionArn": "arn:aws:appsync:us-east-1:379730309663:apis/seiizsmq2zfsfk5nkxfl4h46j4/functions/hdmi2bvayzh6zhoudhm6uc5voe"
}

This is a sign that something is wrong with the request handler. Let's double-check it, and pay attention to the key part.

    key: util.dynamodb.toMapValues({
      id: ctx.arguments.postId,
    })

We're sending ctx.arguments.postId. Let's check the arguments that were received by the handler to confirm the value (They are in the context property).

{
    "arguments": {
      "id": "22d38480-ca09-4d6d-9582-428c15af0bb2"
    },
    "stash": {},
    "outErrors": []
  }

If you look closely, there is a typo in the request handler. The GraphQL query (schema) takes id as an argument, but the request handler sends postId 🤦‍♂️.

The request handler should look like this

export function request(ctx) {
  return {
    operation: 'GetItem',
    key: util.dynamodb.toMapValues({
      id: ctx.arguments.id,  // <-- fixed argument
    }),
  };
}

After re-deploying, everything works as expected. 🎉

As you can see, the logs can help you easily identify issues due to invalid request/response handlers code. context and evaluationResult are very useful to see what happened inside the resolver and how it interacted with the data source.

💡 Tip: With the use of TypeScript to write AppSync resolvers, it also becomes easier to detect those kinds of mistakes in advance.

GraphBolt

GraphBolt is a tool that makes it much easier for developers to test their AppSync APIs. It packs everything you need in one desktop application, including a powerful GraphQL client. It also makes it super easy to debug requests.

Let's take our previous use case, and see how GraphBolt would have helped us find the issue faster.

After executing the request directly from the app, I see an error in the response panel (1). At this point, I have no clue where the problem is.

I click on the debug button (2). After a few seconds, it shows information about the last executed request, including resolver evaluations and their data sources, x-ray traces, request and response headers, and CloudWatch logs.

I'm interested in the getPost resolver which did not return the expected result, so I click on it in the list to get more details. A modal opens. It contains information about the execution of the resolver.

By looking at the getPost Request tab, I can see the context object received by the request handler of my pipeline function on the left side; and its evaluation result on the right side. I can immediately see that the operation sent to DynamoDB is wrong.

The getPost Response tab also shows me the error returned by DynamoDB.

All that, in just a few seconds, without leaving the app and going through CloudWatch logs 🙌

GraphBolt has plenty of other features to help you test, and debug AppSync APIs. Try it for free today!

Conclusion

Debugging AppSync requests is not a free lunch, but with the right knowledge and tools, it becomes easier. In the article, I showed you how to decode AppSync CloudWatch logs and use them to debug GraphQL requests. I also showed you how GraphBolt can help you make this process even quicker and easier.

Happy GraphQL coding!

Image credits: Image generated with krea.ia, and edited with Adobe Firefly.

The AppSync team recently added support for EventBridge as a data source. In this article, we will see how it works and how you can use it with a practical example.

How it works

The EventBridge data source is a new addition that streamlines integration between AppSync and EventBridge. Previously, events could be sent to EventBridge via an HTTP data source, but that required more complex resolvers, better knowledge of the events API, and more IaC to sign HTTP requests with sigv4. Now, with the new native EventBridge data source, AWS AppSync automatically handles all of the complex tasks, leaving you to simply write a simple resolver request operation.

Here is what a request looks like:

{
  operation: 'PutEvents',
  events: [
    {
      source: 'AppSync', // it can be anything
      detailType: 'myEvent',
      detail: {
        foo: 'bar',
      },
    },
  ],
}

You can put up to 10 events in the same request. Just pass them as an array in the events property.

Note that the EventBridge Data "Source" is a bit special in that it's write-only. The only operation you can do is PutEvents.

The response to the request returns information relative to the events that were put on the bus.

{
  "Entries": [
    {
      "EventId": "3298e5f9-66a6-ece5-f0c7-9ff791bcfd1e"
    }
  ],
  "FailedEntryCount": 0
}

📚 For more information, see the documentation

A Practical Example

Great, but how can it be used? Let's take an example.

Imagine that you are building a blog application. Each time that someone publishes a new blog post, you want to notify the followers of the author via email to let them know about it (Similar to Hashnode's Newsletter feature).

One way to do this is by using an Event-Driven approach. EventBridge is often associated with Event-Driven applications. This involves publishing an event, such as postCreated to an event bus after the post is created. Next, you can set up an EventBridge rule that triggers a Lambda function or a Step Function workflow. This workflow can then retrieve all subscribers for the author, construct an email, and send it out.

Let's have a look at how we can implement this using AppSync pipeline resolvers and the new EventBridge data source integration.

To achieve this, we are going to use two functions in our pipeline. Let's call them createPost and putEvent.

The first step is createPost, which utilizes a DynamoDB data source. With the use of a PutItem operation, it inserts the post into a table. Following it, putEvent takes advantage of the new EventBridge integration to place an event on the event bus.

The handlers are as follows:

// createPost.js
import { util } from '@aws-appsync/utils';

export function request(ctx) {
  const item = {
    ...ctx.args.post,
    createdAt: util.time.nowISO8601(),
    updatedAt: util.time.nowISO8601(),
  };
  return {
    operation: 'PutItem',
    key: {
      id: util.dynamodb.toDynamoDB(util.autoId()),
    },
    attributeValues: util.dynamodb.toMapValues(item),
    condition: {
      expression: 'attribute_not_exists(#id)',
      expressionNames: {
        '#id': 'id',
      },
    },
  };
}

export function response(ctx) {
  ctx.stash.event = {
    detailType: 'postCreated',
    detail: ctx.result,
  };

  return ctx.result;
}

In the response handler, we first build the event that is going to be put into EventBridge. It contains the detailType (postCreated) and the detail information which is the post item that was just inserted into DynamoDB. Before we return, we stash it. The returned value of the resolver is the post itself.

Pre-constructing the event in createPost allows us to keep the putEvent function more generic and reusable. i.e. It only has to call PutEvents with the data coming from the previous step (stashed) and does not have to deal with specificities.

// putEvent.js
import { util } from '@aws-appsync/utils';

export function request(ctx) {
  if (!ctx.stash.event) {
    util.error('InternalError', 'Event missing in stash');
  }

  return {
    operation: 'PutEvents',
    events: [
      {
        source: 'blog-appsync-api',
        ...ctx.stash.event,
      },
    ],
  };
}

export function response(ctx) {
  if (ctx.error) {
    util.error('Failed putting event in EventBride', 'Error');
  }

  return ctx.prev.result;
}

The response handler returns the result from the previous function, that is the post item that was inserted. This is also the value that is returned by the pipeline resolver to the client. The EventBridge step is completely transparent.

Let's test it.

GraphBolt is a tool that helps developers build and test AppSync APIs. Thanks to the embedded GraphQL client, you can easily execute queries and mutations. I'm going to use it to call the createPost Mutation.

The Mutation returned the inserted post, meaning that the resolver works as expected. But how about the event?

GraphBolt also comes with a debugger that lets you see what happened under the hood of a resolver. Let's check it out.

As you can see, the putEvent function was invoked, and it put an event into EventBridge with the content of the post, as expected (right side of the image).

To double-check it, I also created an EventBridge rule that triggers a Lambda function and prints the event into CloudWatch.

💡 You can find a fully functional example of this on GitHub, using the Serverless Framework.

Conclusion

In this article, we learned about the new EventBridge data source integration in AWS AppSync. We saw how this integration streamlines the process of building event-driven applications by allowing developers to easily publish events from AppSync resolvers to EventBridge. We also saw a practical example of how to implement this. Overall, this new feature enhances the capabilities of AWS AppSync and provides developers with a powerful tool for building modern serverless APIs.

In a previous article, I showed how you can bundle AppSync JavaScript resolvers and improve your Developer Experience by writing reusable code. In this issue, we'll see how we can take it one step further: write AppSync JavaScript resolvers in TypeScript.

TypeScript is a superset of JavaScript that adds static typing and other features to the language. Together with GraphQL, which is also typed, they are the perfect duo.

Here is what we'd like to achieve:

• Auto-generation of TypeScript types from a GraphQL SDL (Schema Definition Language)

• Use the generated types along with generic AppSync types in resolver handlers

• Transpile and bundle TypeScript into valid AppSync JavaScript code (APPSYNC_JS runtime)

• Deploy an AppSync API with the TypeScript CDK

💡 You can find the complete code used in this article on GitHub

TypeScript Codegen

With the help of the GraphQL codegen tool, you can easily generate TypeScript types from GraphQL schemas. I previously wrote about how you can generate TypeScript types for AppSync Lambda resolvers. The good news is that the process is exactly the same; so I won't go into details here. If you want to know more, please refer to that article.

TL;DR;

Given this GraphQL schema and this codegen file, we obtain the generated types after running the following command:

graphql-codegen

Side note: An alternative option is to use the Amplify CLI.

💡 Tip: I recommend that you commit the generated type files to the repository. Just don't forget to re-generate them when your schema changes!

Writing Resolvers In TypeScript

Out of the box, the AppSync utils package (@aws-appsync/utils) comes with a very convenient set of types that we can use as a base for our resolvers.

One of them, and probably the most relevant one, is Context, which provides the definition for the first argument of the request and response handler functions (usually named context or ctx).

This type takes arguments to specify the shape of the different use case-specific properties, namely: args (or arguments), stash, prev, source and result. We can use it combined with the types we generated earlier from the schema to unlock all the benefits of type-safety and IntelliSense.

Let's take an example. This is the createPost resolver from my demo project.

import { Context, DynamoDBPutItemRequest, util } from '@aws-appsync/utils';
import { createItem } from '../lib/helpers';
import { Post, MutationCreatePostArgs } from '../types/appsync';

export function request(
  ctx: Context<MutationCreatePostArgs>,
): DynamoDBPutItemRequest {
  // add timestamps
  const item = createItem(ctx.args.post);

  return {
    operation: 'PutItem',
    key: {
      id: util.dynamodb.toDynamoDB(util.autoId()),
    },
    attributeValues: util.dynamodb.toMapValues({
      publishDate: util.time.nowISO8601(),
      ...item,
    }),
  };
}

export function response(ctx: Context<MutationCreatePostArgs, object, object, object, Post>) {
  return ctx.result;
}

The usage of Context<MutationCreatePostArgs> in the request handler's argument provides us with a fully typed context object, where args corresponds to the mutation's input type generated from the SDL (in this case it's MutationCreatePostArgs).

Additionally, note that the request handler returns the DynamoDBPutItemRequest type, which is also provided by the AppSync utils package and defines the schema for a DynamoDB resolver's PutItem request.

Finally, in the response handler, I used Context<MutationCreatePostArgs, object, object, object, Post> to specify the type of the result property.

You'll find type declarations for all the available Data Sources, built-in utilities, and more!

Transpiling and Bundling

Writing AppSync resolvers in TypeScript greatly improves Developer Experience, but you can't just send TypeScript code to AppSync. Just like when you write Lambda functions in TypeScript, you need to transpile them to JavaScript first. For AppSync resolvers, the code also needs to be bundled.

Unfortunately, as of writing these lines, the CDK is still incapable of transpiling and bundling code automatically. Hopefully, this will improve over time, but for now, we'll have to do it manually.

For that, we're going to use esbuild. If you read the previous article about bundling AppSync JavaScript resolvers, the command is almost the same, except that in this case, esbuild will also take care of transpiling TypeScript to JavaScript.

Let's review the command:

esbuild src/resolvers/*.ts \
  --bundle \
  --outdir=build \
  --external:"@aws-appsync/utils" \
  --format=esm \
  --platform=node \
  --target=esnext \
  --sourcemap=inline \
  --sources-content=false

• src/resolvers/*.ts takes every file ending in *.ts in the src/resolver folder as entry points.

• --bundle bundles every entry point into one single file.

• --outdir=build is where the output is written. Each entry file will generate one output.

• --external:"@aws-appsync/utils" ignores the AppSync utils package from the bundle (It is provided by AppSync at runtime).

• --format=esm generates the output as ES modules (this is an AppSync requirement).

• --platform=node AppSync runs in an environment similar to NodeJS (i.e. not a browser)

• --target=esnext AppSync uses the latest ES module version.

• --sourcemap=inline AppSync supports source maps, but they must be inline in the bundled file.

• --sources-content=false Do not include content in the source map.

ℹ️ Source maps are optional, but they are useful if you want to see references to your source files in logs and runtime error messages.

Deploying

Using the esbuild command works and is fine for testing and debugging purposes, but ideally, we'd like the code to be bundled automatically at deployment time. In this demo, I'm going to use the CDK (the TypeScript version of course).

Luckily, esbuild comes with a JavaScript API. We're going to use it in order to pre-build the resolvers directly from the CDK stack. Let's first write a helper function:

export const bundleAppSyncResolver = (entryPoint: string): Code => {
  const result = esbuild.buildSync({
    entryPoints: [entryPoint],
    external: ['@aws-appsync/utils'],
    bundle: true,
    write: false,
    platform: 'node',
    target: 'esnext',
    format: 'esm',
    sourcemap: 'inline',
    sourcesContent: false,
  });

  return Code.fromInline(result.outputFiles[0].text);
};

The function takes a file path and calls esbuild's buildSync method with the same parameters as the command we executed earlier. We also add write: false, which just asks esbuild to return the code as a value, instead of writing it to disk. Finally, we return the transpiled and bundled code inline.

We can now use it in our stack definition:

// createPost resolver function
const createPost = new appsync.AppsyncFunction(this, 'CreatePost', {
  name: 'createPost',
  api,
  dataSource: postsDS,
  // transpile, bundle and pass the code inline
  code: bundleAppSyncResolver('src/resolvers/createPost.ts'),
  runtime: appsync.FunctionRuntime.JS_1_0_0,
});

You can find the complete code of the stack on GitHub.

Deploying the stack works as usual:

cdk deploy

ℹ️ Inline code is limited to 4KiB. If you hit that limit, you can pre-budle your code using the esbuild CLI, and point to the generated js file: Code.fromAsset('build/resolvers/createPost.js').

Conclusion

By using TypeScript to write AppSync resolvers, developers can benefit from static typing and enjoy IntelliSense capabilities without the need of relying on Lambda functions. The use of codegen effortlessly provides all the types that are specific to the AP's schema, which can later be used in combination with built-in AppSync types. And with the help of esbuild, the code can easily be transpiled and bundled into valid APPSYNC_JS runtime.

API Limits

You can create up to 25 AppSync APIs per region. However, if you need more, this limit can be increased. Furthermore, each API can have a maximum of 50 API keys, and up to 50 authentication providers. Those can't be extended, but they should be more than enough to cover most use cases.

Execution Time

AWS AppSync limits the execution time for mutations, queries, and subscriptions to 30 seconds. This limit ensures that the API responds to requests in a timely manner.

Queries that last too long provide a very bad user experience for your customers. There are several things you can do to reduce the execution time:

• When possible, prefer VTL or JavaScript resolvers over Lambda functions, it can save you some cold starts.

• Avoid resolvers that are too deeply nested. Nested resolvers are executed in series because child resolvers expect the parent's data (ctx.source).

• If you're using pipeline resolvers, keep the number of functions to a minimum.

• Optimize your requests. For example, make sure you use proper indexes when you query DynamoDB or RDS data sources.

💡 Tools such as X-Ray or GraphBolt can help you understand how queries are executed and detect slow resolvers and bottlenecks.

Resolvers

The maximum number of resolvers that can be executed in a single request is 10,000. Remember that one resolver might be executed more than once in the same request. This often happens when resolving nested fields from a list, for example (n+1 problem). Be careful, the number of executions can compound pretty quickly, especially if you deep-nest resolvers (my friends, the friends of my friends, and their friends).

Still using VTL? The maximum size of VTL mapping templates is 64 kilobytes, while their maximum size after they are evaluated is 5 megabytes, and the number of iterations in a foreach loop is limited to 1,000.

If you're using JavaScript resolvers, you can't use more than 32,000 characters.

For pipeline resolvers, the maximum number of functions that can be used is 10 per pipeline. If you need more complex logic, it's probably time to switch to a Lambda resolver (or an Express Step Function).

Request Tokens

AWS AppSync allocates request tokens to mutation and query requests based on the amount of resources they consume, with a maximum rate of 2,000 up to 10,000 per second per region (depending on the region), shared by all the APIs!

Requests consuming less than or equal to 1,500 KB-seconds of memory and vCPU time are allocated one token. After that, one more token is added per additional chunk of 1,500 KB-seconds consumed.

Although you can request a quota increase, a high token consumption could indicate the need to optimize requests and improve API performance. It also affects the number of requests that your APIs can receive per second. The more tokens they consume, the fewer concurrent requests they will accept.

Subscriptions

The maximum size of a message that can be received through subscriptions is 240 kilobytes.

There is also a default limit of 100 subscriptions per WebSocket connection (that is, per client), but it can be adjusted.

Finally, a hard limit of 100 subscription invalidation requests per second applies.

Update: April 2, 2024

AWS added new adjustable quotas for subscriptions:

• maximum number of inbound subscription invocations (caused by a mutation invocation): 10,000 per second.

• maximum number of outbound subscription messages delivered to clients (per 5kb payload): 1,000,000 per second.

• maximum number of WebSocket connection requests: 2,000 per second

Other Limits

Maximum size of the schema document: 1 megabyte.

The maximum number of AppSync custom domains that you can create in a region is 25, but you can request more if you need them.

If you're using caching, you won't be able to use more than 25 caching keys per resolver, and each item has a maximum Time To Live (TTL) of 1 hour.

Conclusion

AWS AppSync offers a range of features and functionalities for building GraphQL APIs, but it's important to be aware of the service's limits. Remember that they are there for a reason; they are guard rails to prevent you from adopting bad practices that could affect the performance of your APIs and make sure they perform and scale.

In Part 1, we gave a brief overview of the concept of event-driven architectures, coupling, and defined all AWS services needed to build the API.

In Part 2, we created a new CDK project, and added CDK resources for our GraphQL API, alongside a GraphQL Schema, an SQS queue, a DynamoDB table, and an SNS Topic.

In Part 3, we created resources and lambda functions for the step functions workflow.

In the fourth Part, we elaborated more on the step functions workflow, added SNS and deployed the code to the cloud.

API TESTING WITH GRAPHBOLT

When it comes to testing GraphQL APIs, there are a couple of options you can use. The very first one is the AWS AppSync Console. It comes loaded with stuff like

• Authentication

• All Queries, Mutations, and Subscriptions

• API Testing Interface

• View and edit VTL templates, Functions, and Pipelines

• And a lot more.

Irrespective of all its pros, it has a couple of cons that slow down developer productivity and have you thinking about alternatives. For example

• Console Time out. I don't like signing into the console every 20 minutes of inactivity. This timeout can be increased to 60 minutes in the AWS settings. But security-wise, this isn't ideal. I don't need to expose my entire AWS Console because I'm testing an AppSync API.

• Debugging your endpoint from AppSync is no fun. You need to open up the endpoint in Cloudwatch and search for the error through a hundred logs. This is one of the main reasons why observability platforms are on the rise every single day. Searching through Cloudwatch logs is stressful.

There are API clients out there that make managing, testing, and debugging GraphQL APIs Fun.

Say Hi to Graphbolt.

GRAPHBOLT

GraphBolt is a one-stop shop for everything related to AppSync to help you manage, build, test, and debug AppSync APIs.

The query client allows you to execute queries and mutations, just like with Postman or Insomnia. But it is Tailored for AppSync. Meaning, things like authentication (Cognito, API key, etc) are built-in and auto-detected. You only have to choose the right one. API keys are auto-detected and you only need to choose one vs to go copy and paste it in postman.

Let's take a quick look at its interface

At the top of the image, GraphBolt displays the auto-detected AWS Profile you've configured on your computer. I've highlighted it with a yellow rectangle. You can click the dropdown to see all detected AWS profiles.

Top right-hand side, there's a lock 🔒 sign highlighted in pink. Used in configuring authentication.

On the left-hand side of the image, I've highlighted 3 icons with a blue rectangle. The first icon is the Query Client. It provides an interface for building and running Graphql endpoints.

The second icon is the Query Inspector. It provides an interface for debugging. The third icon is Mapping Template Designer. It provides an interface to build and evaluate VTL templates and javascript resolvers.

On the right-hand side of the image, I've highlighted the bug icon with a red rectangle. That icon also takes you to the query inspector screen, to debug the API endpoint you just ran.

Then we have the green, purple, and black rectangles.

The green rectangle is for building the query.

Purple is for providing variables to a query.

Black is for displaying the response to the request when ran. It contains the body and header of the response.

Clicking on the bug icon takes you to this screen. (Query Inspector)

It has the Resolvers, Query, Request Headers, Response Headers, and Raw logs for the GraphQl Query you just ran.

I've highlighted an eye icon on the right, we'll see its use later.

With GraphBolt, you don't need to open up CloudWatch in order to debug your API. Just execute a request. Seeing something wrong? Click the 🪲 button (top right-hand side) and you’re taken to the query inspector screen.

There, you can visualize all the resolvers that were executed. All errors will be highlighted in red.

By clicking on the 👁 of any resolver, you can dig in and instantly see the $context object, the compiled mapping template, and the response from the data source.

Finally, you will also see the result of the resolver (returned value).

This allows you to understand what is going on.

We'll definitely be using GraphBolt to test this and subsequent APIs.

So the first endpoint I want to test is the postOrder endpoint. I want to place an order and see if it gets executed successfully or not. I expect a message to be sent to SQS, the postOrder function should poll and start a step functions workflow, and the email should be sent to confirm if payment was successful or not.

Let's check to see if we have a valid API key first. Click on the lock icon

The key is valid. If you don't have a valid key or no key at all, go to the settings menu in the AppSync console and create one.

GraphBolt would automatically detect the key once it has been created.

I'll make a request with this input. I want 6 cartons of pizza from Dominos Pizza.

    name:"Pizza",
    quantity:6,
    restaurantId:"Dominos Pizza"

Then hit send.

Log into AWS Console and navigate to the postOrder lambda functions CloudWatch logs. You should see a log message from SQS. The postOrder function polled the message from SQS.

If you look in detail at the message, you would see the input we sent to AppSync in the body of the message.

{
   "Records":[
      {
         "messageId":"80ddd9e2-740c-4c20-b6ce-c141382a21c8",
         "receiptHandle":"AQEBzS2lYHfSFDM79iWKIYQ3tydLNRuZWs7BqQlRTXwXMf6lBjigjjOIpXSBds6f9qhgTvOIwWc7kA4OmauYPBWkgxNFdVmx6ktmO6ph5MrJTGmWHM2cAQ45TQeKQn1tBjtI/ilOFeKghLasFnsqNVTN+8phsdbUI1ZOFfMqeALk9B2gvYIroQDLpjZpUgZsCSwA0xIoF+9fyWPO24Yax3XkRT0AneYiy08Ckwy6/RR2M+o6uceC+4XjxlzMuV16yhuuxtQdIiE6gBh5v9wYJYp5haZHUZFd0jwJLgkjOMInytIO249X4/eLlHTUWL/iVUJ3OQt9J7EObHBCYrfH5ARXsyc/oPW7B4A/RNWgO2AAvKYlLVw0sDWpRIynbJml46B+nY59ho+wc8vn5jcAmOMFZA==",
         "body":"{\"input\": {\"name\": \"Pizza\", \"quantity\": 6, \"restaurantId\": \"Dominos Pizza\"}}",
         "attributes":{
            "ApproximateReceiveCount":"1",
            "SentTimestamp":"1676395715258",
            "SenderId":"AROAR5S2TJZSTGYER2WGB:send-sqs-function",
            "ApproximateFirstReceiveTimestamp":"1676395715261"
         },
         "messageAttributes":{

         },
         "md5OfBody":"fe25d90fe8123ea670065bc94209c114",
         "eventSource":"aws:sqs",
         "eventSourceARN":"arn:aws:sqs:us-east-2:132260253285:sqs-queue",
         "awsRegion":"us-east-2"
      }
   ]
}

Next, open up the step functions workflow and see how the workflow played out.

From the workflow, we see that the payment was successful.

Now, we expect to receive a SUCCESS email

If you keep hitting the send button in GraphBolt, you'll receive a mix of SUCCESS and FAILED emails.

Be sure to also open up DynamoDB and check on the saved data.

Before destroying or clearing up all created resources for your application. Yes, you have to destroy the application once you are done, in order to not rack up unnecessary AWS Bills.

Test other queries like getAllOrders or getSingleOrder.

get all orders

getSingleOrder

What Next

We've barely scratched the surface when it comes to building Event-Driven Applications. I urge you to take this app as a starting point and build out something close to production-grade. I left a lot of room for improvement in the app.

For example, in the step functions workflow, you can easily take out a couple of the lambda functions and create a direct service integration between step functions and DynamoDB.

Step functions support >200 AWS Services.

Also, in the complete_order and cancel_failed_order lambda functions, we perform a DynamoDB update and then send an email through SNS.

Writing to the database and sending the email aren’t in one transactional scope.

What if the email fails to send when you've already done the DynamoDB update?

These are sweet edge cases you can resolve.

So instead of having lambda update DynamoDB as well as send an email, you can update DynamoDB directly in step functions, then use DynamoDB streams to connect to an SNS topic, through an event bridge pipe.

If you try this out, please do let me know how it goes.

Conclusion

In this tutorial series, we looked at how to create an Event Driven Application using AWS Services. We built the api, using AppSync, Graphql and python, then tested with GraphBolt. We assumed you have a basic understanding of step functions and suggested a couple of articles where you could step up in case you've never heard of step functions before or you've heard, but never used it.

I really do hope you enjoyed this piece. If you did, do leave a comment or a like.

Happy Coding ✌🏾

References

• https://aws.amazon.com/blogs/compute/introducing-maximum-concurrency-of-aws-lambda-functions-when-using-amazon-sqs-as-an-event-source/

• https://aws.amazon.com/blogs/compute/understanding-how-aws-lambda-scales-when-subscribed-to-amazon-sqs-queues/

In Part 1, we gave a brief overview of the concept of event-driven architectures, coupling, and defined all AWS services needed to build the API.

In Part 2, we created a new CDK project, and added CDK resources for our GraphQL API, alongside a GraphQL Schema, an SQS queue, a DynamoDB table, and an SNS Topic.

In Part 3, we created resources and lambda functions for the step functions workflow.

Let's continue from where we left off in part 3.

From the GraphQL schema, the endpoint is under Query

orders: [ Order ]!

We'll be using a Lambda function resolver to resolve this endpoint. Other alternatives are VTL resolvers or the newly added javascript resolvers. For now, Javascript resolvers are only compatible with applications built with Javascript (nodeJs or Typescript).

From my experience building GraphQL APIs, it's always better to use VTL resolvers for querying purposes. They are quicker, have zero cold starts, and are ideal in situations where there is less data manipulation.

But for this tutorial, we'll be querying the order data using a Lambda resolver. Sorry 😅

The first step towards creating our resolver is to first create a handler with the code to query DynamoDB.

Create a python package called get_orders within the lambda-fns folder. Create a python file called get_order.py within the get_orders folder.

Type in the following code to query all orders from the table.

def process_response(data):
    print(data)
    data["id"] = data["id"]["S"]
    data["quantity"] = data["quantity"]["N"]
    data["name"] = data["name"]["S"]
    data["restaurantId"] = data["restaurantId"]["S"]
    data["orderStatus"] = data["orderStatus"]["S"]

    return data


def fetch_all_orders(dynamo_client, table_name):
    results = []
    last_evaluated_key = None
    while True:
        if last_evaluated_key:
            response = ddb_client.scan(
                TableName=table_name, ExclusiveStartKey=last_evaluated_key
            )
        else:
            response = dynamo_client.scan(TableName=table_name)

        last_evaluated_key = response.get("LastEvaluatedKey")
        response = map(process_response, response["Items"])
        response = list(response)
        results.extend(response)

        if not last_evaluated_key:
            break
    print(f"fetch_all_orders returned {results}")
    return results


def handler(event, context):
    items = fetch_all_orders(ddb_client, TABLE_NAME)
    return items

We are using a scan and LastEvaluatedKey to paginate through the data in the database. This is not efficient and performant. As a matter of fact, don't do this. This is for tutorial purposes only.

In a real application, you'll want to make use of the query function and Global Secondary indexes (GSI) for better performance and scalability.

The next step is to define the lambda resources, the data source, and the lambda resolver.

I created a separate folder for files like this. So within the root folder, create a python package called lambda_resources. Within that folder, create a python file called get_all_orders_lambda_resource.py.

Type in the following code.

def get_all_orders_lambda_resource(stack, api, schema, db_role, lambda_execution_role):
    with open("lambda_fns/get_orders/get_orders.py", "r") as file:
        get_all_order_lambda_function = file.read()

    get_all_order_function = lambda_.CfnFunction(
        stack,
        "gets",
        code=lambda_.CfnFunction.CodeProperty(zip_file=get_all_order_lambda_function),
        role=db_role.role_arn,
        # the properties below are optional
        architectures=["x86_64"],
        description="lambda-ds",
        environment=lambda_.CfnFunction.EnvironmentProperty(
            variables={"ORDER_TABLE": "ORDER"}
        ),
        function_name="get-orders-function",
        handler="index.handler",
        package_type="Zip",
        runtime="python3.9",
        timeout=123,
        tracing_config=lambda_.CfnFunction.TracingConfigProperty(mode="Active"),
    )

We've seen similar code like this above. So we'll go ahead and create a new AppSync data source and then attach our lambda to it.

Within the same file, type in this code

 lambda_get_all_order_config_property = appsync.CfnDataSource.LambdaConfigProperty(
        lambda_function_arn=get_all_order_function.attr_arn
    )

    lambda_get_all_order_data_source = appsync.CfnDataSource(
        scope=stack,
        id="lambda-getAll-order-ds",
        api_id=api.attr_api_id,
        name="lambda_getAll_order_ds",
        type="AWS_LAMBDA",
        lambda_config=lambda_get_all_order_config_property,
        service_role_arn=lambda_execution_role.role_arn,
    )

Using the LambdaConfigProperty we specify the lambda functions ARN for AppSync's Datasource. Then in the CfnDataSource method, we specify the AppSync's API id, the type of Datasource (AWS_LAMBDA) the name of the Datasource, the lambda configuration, and the AWS IAM role ARN for the Datasource.

The next step is defining the resolver.

    # Resolvers

    ## list orders resolver
    get_all_orders_resolver = appsync.CfnResolver(
        stack,
        "list-orders",
        api_id=api.attr_api_id,
        field_name="orders",
        type_name="Query",
        data_source_name=lambda_get_all_order_data_source.name,
    )

In the above code, using the CfnResolver method from AppSync, we add in the AppSync API id, the field_name and type_name as specified in the GraphQL schema, and also the name of the Datasource we created above.

Lastly, since the resolver depends on the GraphQL schema and Datasource, we need to specify that also, using the add_dependency method.

    get_all_orders_resolver.add_dependency(schema)
    get_all_orders_resolver.add_dependency(lambda_get_all_order_data_source)

We are just adding a CloudFormation dependency between both resources.

The final step is to call this function get_all_orders_lambda_resource(stack, api, schema, db_role, lambda_execution_role) from the stack file and pass in all required parameters.

Deploy

I'll assume you've already added your account and region as environment variables to your app file in app.py.

Run

cdk synth in order to synthesize your app.

Then run

cdk bootstrap to provision CloudFormation resources to the environment (region and account) we added to the app above.

Finally, we have to deploy the app. Remember that we need to pass in an email address as a parameter during deployment so that it can be used to subscribe to SNS for emails.

Run this command to deploy your app

cdk deploy --parameter subscriptionEmail=YOUR_EMAIL_ADDRESS

If the app is successfully deployed, you should receive a subscription email from amazon. Check your spam folder.

Clicking on the link in the email subscribes that email to the SNS topic we created above sns topic.

If you log into the AWS console and navigate to SNS, under your topic, you should see all subscribed emails.

Subscribers

Navigate to AppSync from the AWS console. Select and open your project from the list of APIs.

Conclusion

In this article, we added an endpoint to get all orders in the database. We saw how to build the lambda function, create a Datasource and a resolver, then crowned it all up by deploying the app to the cloud, while passing in an email address as a parameter.

So what's left now is testing, and we'll be seeing that in the final episode of this series. Stay tuned. If you loved this piece, please like or comment. Happy Coding ✌🏾

In Part 1, we gave a brief overview of the concept of event-driven architectures, coupling, and defined all AWS services needed to build the API.

In Part 2, we created a new CDK project, and added CDK resources for our GraphQL API, alongside a GraphQL Schema, an SQS queue, a DynamoDB table, and an SNS Topic.

Let's proceed to build from where we left off in part 2.

AWS Step Functions Resources

We won't be looking at how to create a step functions workflow from scratch. I'll assume you at least have a basic understanding of the service.

If you don't, don't worry. I've written a couple of articles to get you up and running in no time.

• Building Apps with Step Functions

• Building a Step Functions Workflow With CDK, AppSync, and Python

The Step Functions workflow has 4 Lambda functions. Here's what it looks like visually. I exported this image from the Step Functions Visual Workflow.

I also exported the workflow as a JSON file and saved it inside a python package called step_function_workflow in the project directory.

You can create a python package with the same name and get the file here. workflow.json

For the 4 Lambda functions, navigate to the lambda_fns folder, create 4 python packages, and within each package, create a python file. These python files would serve as lambda functions.

1. Package name: initialize_order, File name: initialize_order.py

2. Package name: process_payment, File name: process_payment.py

3. Package name: complete_order, File name: complete_order.py

4. Package name: cancel_failed_order, File name: cancel_failed_order.py

Create a file called step_function.py within the step_function_workflow python package. We'll define all the lambda functions resources needed by the workflow within this file.

We have to create a Lambda resource for each of the above 4 python files, add them to the step functions workflow and then return a step functions instance.

In order to create a Lambda function resource, we need to first import the aws_lambda class from CDK.

from aws_cdk import aws_lambda as lambda_

Create a function with these parameters. We'll be passing in the stack, lambda permissions, and an SNS topic we created in the stack file.

def create_step_function(stack, lambda_step_function_role, cfn_topic):

Next, using the function CfnFunction found within the aws_lambda class, let's create the lambda function resource for initialize_order.py.

Firstly, we need to return a stream of the Lambda function, using the open method. We'll repeat these same steps for the other lambda functions.

with open("lambda_fns/initialize_order/initialize_order.py", 'r') as file:
    initialize_order = file.read()

Then, we define the resource the Lambda function needs

 initialize_order_function = \
        lambda_.CfnFunction(stack, "initialize-order-function",
        code=lambda_.CfnFunction.CodeProperty(
            zip_file=initialize_order
        ),
        role=lambda_step_function_role.role_arn,

        # the properties below are optional
        architectures=["x86_64"],
        description="lambda-ds",
        environment=lambda_.CfnFunction.EnvironmentProperty(
            variables={
                "ORDER_TABLE": "ORDER",
                "TOPIC_ARN": cfn_topic.attr_topic_arn
            }
        ),
        function_name="initialize-order-function",
        handler="index.handler",
        package_type="Zip",
        runtime="python3.9",
        timeout=123,
        tracing_config=lambda_.CfnFunction.TracingConfigProperty(
            mode="Active"
        )
        )

Here's what's happening in the above code.

To create a Lambda function, we need a deployment package and an execution role. Our deployment package in this case is a zip file, containing the initialize_order Lambda code. You can also use a container image as your deployment package.

code=lambda_.CfnFunction.CodeProperty(zip_file=initialize_order),

We also need an execution role that grants the function permission to access other AWS services such as Step Functions, CloudWatch, etc.

role=lambda_step_function_role.role_arn,

Here are the policies we attached to the lambda function role in the stack file.

• Full DynamoDB access (Because we'll be saving the order to the database)

• Full SNS access for sending emails

• Full CloudWatch access for logging

lambda_step_function_role = \
            iam.Role(self, "LambdaStepFunctionRole",
             assumed_by=iam.ServicePrincipal("lambda.amazonaws.com"),
             managed_policies=[db_full_access_role,
                               cloud_watch_role_full_access,
                               sns_policy])

Then we set the environment variables that we'll be needing in our Lambda function. In this case, it's the DynamoDB table name and SNS topic Amazon resource name (ARN).

Next, we state the runtime (python 3.9), handler name (which we'll define later within the initialize_order.py file, and the Lambda function timeout in seconds. Note that a lambda function can't run above 15 minutes.

Repeat these same steps for the other 3 lambda functions.

Use the file on Github as a reference: step_function.py

At the bottom of the step_function.py file, we have to define the Step Functions workflow template and also add the created Lambda functions ARNs.

    sf_workflow = \
        Template(workflow).substitute(InitializeOrderArn=initialize_order_function.attr_arn,
        ProcessPaymentArn=process_payment_function.attr_arn,
        CompleteOrderArn=complete_order_function.attr_arn,
        CancelFailedOrderArn=cancel_failed_order_function.attr_arn, dollar="$")

    return sf_workflow

Inside the workflow.json file, there's a line like this

"FunctionName": "$InitializeOrderArn"

We use the substitute method above to replace that line with the ARN to where the Lambda function resides InitializeOrderArn=initialize_order_function.attr_arn.

Now, go to the stack file of your application, and type in the following code to create a Step Functions state machine.

Firstly, we need to call the create_step_function(self, lambda_step_function_role, cfn_topic) function we defined above and pass its return type to our state machine's definition.

Also, the state machine needs permission to interact with other AWS services. So we grant Step Functions permissions to execute lambda functions.

 workflow = create_step_function(self, lambda_step_function_role, cfn_topic)

        simple_state_machine = \
            stepfunctions.CfnStateMachine(self, "SimpleStateMachine",
             definition=json.loads(workflow),
             role_arn=lambda_execution_role.role_arn
             )

At this point, we've created resources for 75% of the application. Let's go ahead and start creating the endpoints to consume these resources.

Endpoints

Create Order

When a user places an order, a series of events happen

• An AppSync Mutation is invoked. The input to the Mutation is bundled and sent as a message to an SQS Queue.

• A Lambda function (post order Lambda) polls the messages from the SQS Queue, extracts the order information, and starts up a Step Functions workflow with order information as input.

• The Step Functions workflow contains 4 different Lambda functions (Initialize Order, Process Payment, Complete Order, Cancel Order).

• The Initialize Order function creates a new order in the database.

• The Process Payment function randomly assigns a success or failed payment status to the order.

• If payment was successful, the Complete Order function updates the order in the database and sends an email to the client.

• If payment failed, the order status is updated in the database and an email is sent with failed status to the client through the failed email Lambda function.

Create a python package inside the lambda-fns folder called post_order. Then inside that folder, create a python file called post_order.py.

This Lambda function would poll messages from the SQS queue and start a Step Functions workflow. Therefore, it'll need permission to receive SQS queue messages and to start a Step Functions workflow.

Let's define the code within the post_order.py file.

def handler(event, context):
    new_order_list = []
    print("event ", event)
    for record in event["Records"]:
        message_id = record["messageId"]
        request_body = json.loads(record["body"])
        order_data = request_body["input"]
        print(f'post_orders reqeust_body {order_data} type: {type(order_data)}')
        sfn_input = assemble_order(message_id, order_data)
        response = start_sfn_exec(sfn_input, message_id)
        print(f'start sfn execution: {response}')
        new_order_list.append(response["executionArn"])
    return new_order_list

Messages from a queue are polled as Records. We have to iterate over each record in order to extract the order information, assemble the order, and use it to start a step functions workflow.

sfn_input = assemble_order(message_id, order_data)

This method simply adds more random data to the created order and returns the order.

def assemble_order(message_id, order_data):
    now = datetime.now()
    order_data["user_id"] = "demo_user"
    order_data["id"] = message_id
    order_data["orderStatus"] = DEFAULT_ORDER_STATUS
    order_data["createdAt"] = now.isoformat()
    return json.dumps(order_data, cls=DecimalEncoder)

response = start_sfn_exec(sfn_input, message_id)

Here's where we start the step functions workflow, using the Step Function ARN and the order data as input.

def start_sfn_exec(sfn_input, sfn_exec_id):
    response = sfn.start_execution(
        stateMachineArn=STATE_MACHINE_ARN,
        name=sfn_exec_id,
        input=json.dumps(sfn_input, cls=DecimalEncoder)
    )
    print(f'post_orders start sfn_exec_id {sfn_exec_id} and input {sfn_input}')
    return response

Now, define the resources and permissions this Lambda function needs. It's pretty similar to the Step Functions lambda resources we defined above.

Within the stack folder, mine is event_driven_cdk_app create a file called post_order_resource.py and type in the following code.

def create_post_order_lambda_resource(stack, simple_state_machine, sqs_receive_message_role, queue):
    with open("lambda_fns/post_order/post_order.py", 'r') as file:
        post_function_file = file.read()

    post_function = lambda_.CfnFunction(stack, "post",
                    code=lambda_.CfnFunction.CodeProperty(
                        zip_file=post_function_file
                    ),
                    role=sqs_receive_message_role.role_arn,

                    # the properties below are optional
                    architectures=["x86_64"],
                    description="lambda-ds",
                    environment=lambda_.CfnFunction.EnvironmentProperty(
                        variables={
                            "ORDER_TABLE": "ORDER",
                            "STATE_MACHINE_ARN": simple_state_machine.attr_arn
                        }
                    ),
                    function_name="post-order-function",
                    handler="index.handler",
                    package_type="Zip",
                    runtime="python3.9",
                    timeout=123,
                    tracing_config=lambda_.CfnFunction.TracingConfigProperty(
                        mode="Active"
                    )
                    )

    lambda_.EventSourceMapping(scope=stack, id="MyEventSourceMapping",
           target=post_function,
           batch_size=5,
           enabled=True,
           event_source_arn=queue.attr_arn)

It's very identical to the one we defined above, except for the last part

    lambda_.EventSourceMapping(scope=stack, id="MyEventSourceMapping",
                               target=post_function,
                               batch_size=5,
                               enabled=True,
                               event_source_arn=queue.attr_arn)

Remember, this function is responsible for polling messages from an SQS queue. We subscribe to the SQS queue using the EventSourceMapping function of the lambda API by passing in the SQS queue's ARN as the event source event_source_arn=queue.attr_arn.

The batch_size indicates the number of messages to be polled from SQS at a time.

When this function invokes a Step Function workflow, the first Lambda function to run in the workflow is initialize_order.py.

We created the file in the initialize_order python package, but we didn't add any code to it. Open up initialize_order.py and type in the following code.

All we do is save the order to the database and forward the order details to the next function, which is process_payment.py

def persist_order(order_item):
    print(f'persist_order item {order_item} to table {TABLE_NAME}')
    response = table.put_item(Item=order_item)
    message = {"order_status": order_item["orderStatus"], "order_id": order_item["id"]}
    print(f'new order pending payment {message}')
    return {

        "message": json.dumps(order_item, indent=4, cls=DecimalEncoder)
    }

You can grab the complete code here

The process_payment.py function randomly assigns a payment state (ok or error) and an error message if the randomly assigned state was error, and then moves to the next function.

The next function is determined by a choice state based on the state of the payment. The next function would be complete_order.py if the payment state was ok or cancel_failed_order.py if the payment state was error.

In either of these functions, the order is updated in the database, and an email is sent using SNS.

For the complete order function, the order is updated like so

    response = table.update_item(
        Key={
            "user_id": event["saveResults"]["user_id"],
            "id": event["saveResults"]["id"]
        },
        UpdateExpression="set orderStatus = :s",
        ExpressionAttributeValues={
            ":s": order_status
        },
        ReturnValues="UPDATED_NEW"
    )

And an email is sent as such

  sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps(message),
        Subject=f'Orders-App: Update for order {message["order_id"]}'

    )

See the complete code here complete_order

Same thing with cancel_failed_order.py function Cancel Failed Order

At this point, we could deploy and test the API. But I think it'll be better to add at least one more endpoint before deploying.

I would love to see a list of all the orders made.

But that'll be for the next article.

Conclusion

In this post, we added a Step functions workflow containing 4 Lambda functions to our API. We saw how to initialize an order and also send SNS email notifications of the status of an order. We'll continue building the API in the next post.

I'll love to know your thoughts about the series so far. Please leave a like or comment.

And until next time folks ✌🏾

AWS AppSync is a fully managed service that allows developers to build scalable and performant GraphQL APIs. It is also serverless, meaning that you will only pay for what you use.

AWS recently introduced support for JavaScript resolvers. This was a long-awaited feature. Before that, developers had to learn and write resolvers in a language called VTL, which has a steep learning curve and is hard to debug.

Shortly after the announcement, I wrote an article explaining what JS resolvers are, and why they are beneficial, but also explaining the limitations they have. For example, you likely cannot use your favorite external libraries. For two main reasons:

1. import (or require) is not allowed. All your code must be in the same file.

2. The code must comply with all the strict rules that AppSync has, which is likely not the case for most packages on npm today.

But that does not mean you cannot write your own reusable code that can be shared between resolvers. As developers, we often try to be DRY. If some code has to be written more than once, then it should be encapsulated into a function.

Right after JavaScript resolvers were announced, I did a quick proof of concept that shows that it is possible; as long as you follow the rules.

In this article, I will show you how I achieved it with a simple practical example.

Pre-requisites and Assumptions

In order to keep this article focused on the subject and take it to the point, I will take a few shortcuts and assumptions.

1. You are already familiar with JavaScript resolvers.

   I will not go into details about how JS resolvers work. If you are new to JS resolvers, please refer to this article and the documentation before you keep reading.

2. Pure JavaScript.

   Even though we will use esbuild (more on that later), for the purpose of this article, I will not try to use TypeScript and stick to pure JavaScript. This will remove the extra complexity that TypeScript adds because of code transpiling. In a follow-up article, we will see how we can add TypeScript into the mix.

3. Take IaC out of the equation.

   I will not show you how to couple this with any Infrastructure as Code (IaC), nor how to deploy the resulting resolvers either. At the time of writing this article, and as far as I know, no IaC solution out there today supports this out of the box. You will need to figure out that part yourself. However, I will show you how to test the result, so you can see if the code is valid and working as expected 🙂

Let's get started

Enough talk, let's write some code!

Imagine we want to add timestamp values (createdAt and updatedAt) for every item inserted into DynamoDB by all of our Mutations (e.g. createPost, createAuthor, createComment, etc.). Because we don't want to repeat that code in every single resolver, we might want to do something like this:

// createPost.js
import { util } from '@aws-appsync/utils';
import { createItem } from './helpers';

export function request(ctx) {

  // add timestamps
  const item = createItem(ctx.args.post);

  return {
    operation: 'PutItem',
    key: {
      id: util.dynamodb.toDynamoDB(util.autoId()),
    },
    attributeValues: util.dynamodb.toMapValues(item),
  };
}

export function response(ctx) {
  return ctx.result;
}

// helpers.js
export function createItem(item) {
  return {
    ...item,
    createdAt: util.time.nowISO8601(),
    updatedAt: util.time.nowISO8601(),
  };
}

As you can see, we created a helper function named createItem() in a new file (helpers.js) that adds the creation and update timestamps to the object that is passed as an argument and then returns it. We import that function into our resolver and use it. We can do so in every place where we need it (e.g. createAuthor.js, createComment.js, etc).

This is great, but it also breaks one of the AppSync JS rules: import is not allowed. (Note: with the exception of @aws-appsync/utils, which is a special package provided by AppSync).

However, what we can do is bundle our code into one single file before sending it to AppSync. To do so, we are going to leverage a widely used code bundler: esbuild.

There are a few things to keep in mind though:

1. By default, esbuild converts ECMAScript module syntax to CommonJS.

   AppSync only supports ES modules, so we must ensure that esbuild outputs those instead. To solve that issue, we can use the format option with a value of esm.

2. We don't want to bundle the AppSync utils library.

   AppSync provides the @aws-appsync/utils package by default. So, we don't want to have it bundled into our final code. The external option allows us to exclude it from the bundle.

Let's execute the following command:

esbuild createPost.js \
  --bundle \
  --outdir=build \
  --external:"@aws-appsync/utils" \
  --format=esm

If you look in the build directory, you should find a createPost.js file which contains the bundled code as follows:

// createPost.js
import { util as util2 } from "@aws-appsync/utils";

// helpers.js
import { util } from "@aws-appsync/utils";
function createItem(item) {
  return {
    ...item,
    createdAt: util.time.nowISO8601(),
    updatedAt: util.time.nowISO8601()
  };
}

// createPost.js
function request(ctx) {
  const item = createItem(ctx.args.post);
  return {
    operation: "PutItem",
    key: {
      id: util2.dynamodb.toDynamoDB(util2.autoId())
    },
    attributeValues: util2.dynamodb.toMapValues(item)
  };
}
function response(ctx) {
  return ctx.result;
}
export {
  request,
  response
};

We now have all the necessary code for our resolver in one single file that is compliant with AppSync 🎉

Testing

Great, now there is one more thing that we need to make sure of. Does our resolver work as expected? Does it contain any invalid code?

AWS provides a CLI that allows us to test JavaScript resolvers. It serves two purposes:

• If the code contains any error or is invalid, the command will fail and let us know where the problem is. This is especially useful in this case because esbuild might easily break compatibility with AppSync when bundling the code.

• We can inject mock data in order to check the result that would be sent to the data source. This allows us to check that our code logic works as expected.

Let's try it

aws appsync evaluate-code \
  --code file://build/createPost.js \
  --function request \
  --context file://context.json \
  --runtime name=APPSYNC_JS,runtimeVersion=1.0.0

with the following context.json file.

{
  "arguments": {
    "post": {
      "title": "Hello, world",
      "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit."
    }
  }
}

result:

{
  "evaluationResult": "{\"operation\":\"PutItem\",\"key\":{\"id\":{\"S\":\"927934a0-4fb4-4ee2-977a-4f4d458062fb\"}},\"attributeValues\":{\"createdAt\":{\"S\":\"2023-03-02T12:28:15.087Z\"},\"title\":{\"S\":\"Hello, world\"},\"content\":{\"S\":\"Lorem ipsum dolor sit amet, consectetur adipiscing elit.\"},\"updatedAt\":{\"S\":\"2023-03-02T12:28:15.087Z\"}}}",
  "logs": []
}

This looks a bit messy, but if you look closely, you will see that a DynamoDB PutItem request was generated. It contains our mocked data, which was enriched with the createdAt and udpatedAt fields from our helper function, as expected 🎉.

Before we call it a day, let me show you another way to test this.

GraphBolt is a tool for developers to build, test and debug AppSync APIs. You can also use it to easily test JS resolvers in a user-friendly way thanks to the Mapping Template Designer tool it provides.

Copy the same bundled resolver code and paste it into the tool. Select JavaScript as the runtime. Finally, enter the mock data in the top section (under args).

Hit the Eval button, and when prompted, chose request as the function to be evaluated.

And voilà!

On the right hand, you should see the result of the code evaluation, with the expected PutItem request.

Now that we confirmed that the code is valid, you can use it in your AppSync API using your favorite IaC.

💡 You can find the code for the example used in this tutorial, and more, on GitHub.

Conclusion

The introduction of AppSync JavaScript resolvers was a game-changer. They let developers write code quicker in a language they are more familiar with. They also open the possibility to write reusable code more easily. In this article, I showed you how to write simple helper functions that you can use in more than one resolver while still following AppSync's rules. Finally, I showed you two ways you can both validate the bundled code and test its logic.

Let's proceed.

Prerequisite

Before proceeding, please confirm you have all these dependencies installed on your computer

• AWS CLI

• AWS CDK

Creating a new CDK python project

From the command line interface (Terminal), create and change the directory into the newly created folder using the command

mkdir eventDrivenCdk && cd $_

I named my project eventDrivenCdk, feel free to give yours a different name.

Within the newly created project, initialize a python CDK project using the command

cdk init --language=python

This project is set up like a standard Python project. The initialization process also creates a virtualenv within this project, stored under the .venv directory.

To create the virtualenv it assumes that there is a python3 (or python for Windows) executable in your path with access to the venvpackage.

If for any reason the automatic creation of the virtualenv fails, you can create the virtualenv manually.

To manually create a virtualenv on MacOS and Linux:

$ python3 -m venv .venv

After the init process completes and the virtualenv is created, you can use the following step to activate your virtualenv.

$ source .venv/bin/activate

If you are a Windows platform, you would activate the virtualenv like this:

% .venv\Scripts\activate.bat

Once the virtualenv is activated, you can install the required dependencies.

Add boto3 to the requirements.txt before running the command.

$ pip install -r requirements.txt

Boto3 is the aws sdk for python.

Graphql Schema

In the root directory, create a file called schema.graphql and type in the following code. This file is a description of our Graphql API. It contains all the types, queries, mutations, and subscriptions for our Graphql API.

type Schema {
    query: Query
    mutation: Mutation
}

type Order {
    name: String!
    quantity: Int!
    restaurantId: String!
}

input OrderInput {
    name: String!
    quantity: Int!
    restaurantId: String!
}

input UpdateOrderInput {
    id: String!
    name: String!
    quantity: Int!
    restaurantId: String!
}

type Query {
    orders: [Order ]!
    order(id: String!): Order!
}

type Mutation {
    postOrder(input: OrderInput!): Order!
    updateOrder(input: UpdateOrderInput!): Order!
    deleteOrder(id: String!): String
}

From the schema, our API has 3 mutations and 2 queries. Before delving into the implementation details of these endpoints, we need to first define all the resources the app needs in order to run effectively.

Defining the GraphQL API in Stack

The first step is to import the appsync class from the aws-cdk-lib.

import aws_cdk.aws_appsync as appsync

Then, use the CfnGraphQLApi method within the appsync class to create the API. This method takes a myriad of parameters, but for our use case, all we need is an API name, the authentication type, x-ray, and cloudwatch for tracing and logging.

        api = appsync.CfnGraphQLApi(self, "Api",
                                    name="event_driven_cdk",
                                    authentication_type="API_KEY",
                                    xray_enabled=True,
                                    log_config=log_config
                                    )
   log_config = appsync.CfnGraphQLApi.LogConfigProperty(
            cloud_watch_logs_role_arn=appsync_cloud_watch_role.role_arn,
            exclude_verbose_content=False,
            field_log_level="ALL")

This line cloud_watch_logs_role_arn=appsync_cloud_watch_role.role_arn, gives AppSync permissions to push logs to CloudWatch. Here's how we define the role and attach its policies.

  cloud_watch_role_full_access = iam.ManagedPolicy.from_managed_policy_arn(self, "cloudWatchLogRole",
                                                                                 'arn:aws:iam::aws:policy/CloudWatchLogsFullAccess')

 appsync_cloud_watch_role = iam.Role(self, "AppSyncCloudWatchRole",
                                            assumed_by=iam.ServicePrincipal("appsync.amazonaws.com"),
                                            managed_policies=[
                                                cloud_watch_role_full_access
                                            ])

After creating the API, the next logical step is to attach the schema. We use the CfnGraphQLSchema method from the AppSync class to achieve this. This method takes in a scope, an id, an api_id which should be a unique AWS Appsync GraphQL API identifier, and a definition (the schema file itself).

dirname = path.dirname(__file__)

with open(os.path.join(dirname, "../schema.graphql"), 'r') as file:
    data_schema = file.read().replace('\n', '')
schema = appsync.CfnGraphQLSchema(scope=self, id="schema", api_id=api.attr_api_id, definition=data_schema)

Defining the Queue

Let's define and attach the SQS queue to AppSync. Firstly, we import the SQS class from cdk.

import aws_cdk.aws_sqs as sqs

Then, we'll create 2 queues and use one as the Dead letter queue (DLQ). A Dead letter Queue is a message queue that'll store all the messages that couldn't be processed successfully. The developer can always go back to the DLQ and redrive the unsuccessful messages.

        # SQS
        queue = sqs.CfnQueue(
            self, "CdkAccelerateQueue",
            visibility_timeout=300,
            queue_name="sqs-queue"
        )

        deadLetterQueue = sqs.Queue(
            self, "CdkAccelerateDLQueue",
            visibility_timeout=Duration.minutes(10),
            queue_name="dead-letter-queue"
        )

        sqs.DeadLetterQueue(max_receive_count=4, queue=deadLetterQueue)

The visibility_timeout is the time taken for a consumer to process and delete a message once dequeued. While this timeout is valid, the message is made unavailable to other consumers. If the timeout expires when the message hasn't been successfully processed and delivered, the message is sent back into the queue and made available for other consumers to pick up.

If you don't specify a value for the visibility_timeout, AWS CloudFormation uses the default value of 30 seconds. Default: Duration.seconds(30)

The max_receive_count is the number of times a message can be unsuccessfully dequeued before being moved to the dead-letter queue. We set the value to 4, meaning after 4 unsuccessful dequeue attempts, that message would be sent to the DLQ.

Now, let's attach the SQS queue to Appsync.

api.add_dependency(queue)

That's all. Remember the name of the GraphQL API we created above was api.

Defining DynamoDB Resources

Import the dynamodb class for cdk.

import aws_cdk.aws_dynamodb as dynamodb

Create a table called ORDER with a composite key.

user_id for the primary key and id as the sort key.

        # DynamoDB
        dynamodb.CfnTable(self, "Table",
                          key_schema=[dynamodb.CfnTable.KeySchemaProperty(
                              attribute_name="user_id",
                              key_type="HASH"
                          ),
                              dynamodb.CfnTable.KeySchemaProperty(
                                  attribute_name="id",
                                  key_type="RANGE"
                              )],
                          billing_mode="PAY_PER_REQUEST",
                          table_name="ORDER",
                          attribute_definitions=[dynamodb.CfnTable.AttributeDefinitionProperty(
                              attribute_name="user_id",
                              attribute_type="S"
                          ),
                              dynamodb.CfnTable.AttributeDefinitionProperty(
                                  attribute_name="id",
                                  attribute_type="S"
                              )]
                          )

Defining SNS Resources

Create an SNS topic with the topic name sns-topic. As usual, we'll import the SNS class from AWS CDK

from aws_cdk import aws_sns as sns

Then use CfnTopic and CfnTopicPolicy methods from the SNS class to create and grant policies to the SNS topic.

        cfn_topic = sns.CfnTopic(self, "MyCfnTopic",
                                 display_name="sns-topic",
                                 fifo_topic=False,
                                 topic_name="sns-topic"
                                 )

        sns_publish_policy = sns.CfnTopicPolicy(self, "MyCfnTopicPolicy",
                                                policy_document=iam.PolicyDocument(
                                                    statements=[iam.PolicyStatement(
                                                        actions=["sns:Publish", "sns:Subscribe"
                                                                 ],
                                                        principals=[iam.AnyPrincipal()],
                                                        resources=["*"]
                                                    )]
                                                ),
                                                topics=[cfn_topic.attr_topic_arn]
                                                )

Bear in mind that, we've assumed the client is already signed in to our application at this point. So we have their email address. We'll use that email address to subscribe to the SNS topic to receive email updates on success or failed order payments.

For the purpose of this tutorial, we'll pass in the email address as a parameter from the CLI, when deploying the application later.

For now, we need to add email as a subscriber using the CfnSubscription method from the SNS class.

        email_address = CfnParameter(self, "subscriptionEmail")
        sns.CfnSubscription(self, "EmailSubscription",
                            topic_arn=cfn_topic.attr_topic_arn,
                            protocol="email",
                            endpoint=email_address.value_as_string

                            )

Conclusion

In this episode,

• We Created A GraphQL API and attached a schema to it.

• Created and attached a queue to the AppSync API.

• Created a Dead Letter Queue to catch unprocessed messages.

• Created an SNS topic, with an email as a subscriber.

• Gave IAM roles and policies to all created resources.

In the next episode, we'll continue creating more resources for the API. Please stay tuned. Thanks for reading. If you enjoyed it, please leave a like or a comment. I'll love to know what you think. Take Care.

The World is Asynchronous

-- Dr Werner Vogels ReInvent 2022

The concept of Event Driven Architecture (EDA) is not new. As a matter of fact, it's been around for ages.

But lately, it's been the talk of every tech clique. There's no way you can scroll through tech Twitter without coming across a tweet concerning EDA.

But what are EDA's and why is every organization adopting this architectural pattern?

Event Driven Architectures?

Minimizing the degree of interdependence between different components of a system is the main consideration when building event-driven architectures. The term loosely coupled or Decoupling is generally used.

Lots of integration patterns have been created throughout the years, to tackle coupling within an application.

So how do you determine the level of coupling within your application's components and what strategies do you use in order to guarantee a loosely coupled system?

IT strategist and cloud/enterprise Architect, Gregor Hohpe Says

The appropriate level of coupling depends on the level of control you have over the endpoints

Generally, if your application depends on third-party APIs which you have less control over, you don't want to build a strong attachment with those APIs. Because, if they go down, your application goes down.

You want a scenario where you'll be able to easily pull back your resources in case of a system hiccup. This goes same for all the other components of your system, whether internal or external.

As we progress through the application, we'll see how to use AWS Services to ensure your system is loosely coupled

EDA's use events to coordinate communication between loosely coupled services.

But what are events?

Events

An event is a change in state, or an update, like an order added to the cart in a restaurant application. Events can either carry the state (the item purchased, its price, and quantity) or events can be identifiers (a notification that a payment was successful or not).

EDAs are made up of 3 parts.

Event Producers

This can be a mobile app or an e-commerce site

Event Routers

This can be an event store

Event Consumers

This can be a database, saas application, microservice, etc

So an event producer publishes an event to the event router, the event router filters and pushes the event to event consumers. The event router abstracts the producer from the consumer by allowing them to communicate asynchronously.

Why you should adopt EDA

• With services that scale on demand and fail independently, your application is bound to be reliable and resilient.

• Event routers coordinate events automatically between producers and consumers, so you no longer need to write code to handle that.

• Event-driven architectures are push-based, so everything happens on-demand as the event presents itself in the router. This way, you’re not paying for continuous polling to check for an event.

• EDAs are responsive to the needs of customers as they expect systems to respond to events as they happen.

• EDAs are cost effective, allowing customers to pay only for the services they need when they need them.

Now that we have a brief understanding of what EDAs are, let's go ahead and dive into the main topic for the blog post.

Article Scope

We'll use the concept of EDA to design and build a modern serverless Graphql API on AWS. We will be doing so, by mixing and matching different AWS services in a technical manner.

Out Of Scope

We'll not be looking at introductions to any of the AWS Services used in this tutorial such as

• AWS Step functions

• AWS SNS

• AWS SQS

But I'll provide supporting documents to upskill if need be.

This tutorial is aimed at the following audiences:

• Software engineers looking for a quick hands-on intro to Event-Driven Architectures

Use Case

Let's say you want to order pizza from a restaurant, through the restaurant's mobile or web application.

In an ideal scenario, the application would let you make (add to cart) your choice of pizza, and the quantity you want, with all necessary add-ons like extra cheese, and chili, and only let you sign in to or create an account when you wish to place the order.

In this tutorial, we'll assume you've already created an account, and you are about to place an order. Once your order has been placed, we'll run a fake payment check to determine if you've made payment or not, and would email you based on the status of the order, be it success or failure.

We'll be using a couple of AWS Services to illustrate how all these can be accomplished, in a scalable, decoupled, event-driven manner.

Here are the AWS services we'll be using in this application.

AWS Services

AWS AppSync

AWS AppSync is a fully managed service allowing developers to deploy scalable and engaging real-time GraphQL backends on AWS.

It leverages WebSockets connections under the hood to provide real-time capabilities, by publishing data updates to connected subscribers.

Amazon SQS

A fully managed message queueing service to decouple producers and consumers. SQS is a fundamental building block for building decoupled architectures

AWS Lambda

AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software-as-a-service (SaaS) applications and only pay for what you use.

AWS StepFunctions

AWS Step Functions is a visual workflow service that helps developers use AWS services to build distributed applications, automate processes, orchestrate microservices, and create data and machine learning (ML) pipelines.

AWS SNS

Amazon Simple Notification Service (SNS) sends notifications in two ways, Application-to-Application (A2A) and Application-to-Person (A2P).

A2A provides high-throughput, push-based, many-to-many messaging between distributed systems, microservices, and event-driven serverless applications. These applications include Amazon Simple Queue Service (SQS), Amazon Kinesis Data Firehose, AWS Lambda, and other HTTPS endpoints.

A2P functionality lets you send messages to your customers with SMS texts, push notifications, and email.

For this application, we'll be using A2P.

Amazon DynamoDB

Amazon DynamoDB is a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale. DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools.

Solutions Architecture

From the architecture above, the Amplify icon signifies a front-end application. It can be a mobile or web app. So a client places an order for 5 cartons of pizza from Domino's pizza.

Here's the request payload

"name": "Pizza",
"quantity": 6,
"restaurantId": "Dominos

AppSync receives the request and sends a message, containing the payload to an attached SQS queue. We use SQS to decouple the event producers (AppSync) from the consumers (in this case a Lambda), so that they can communicate asynchronously.

So as order requests keep flooding in, let's say on a world pizza day when demand is really high, all requests are being sent to SQS. Lambda is a common choice as a consumer for SQS as it supports native integration. So you get to write and maintain less code between both of them.

When a Lambda function subscribes to an SQS queue, Lambda polls the queue as it waits for messages to arrive. Lambda consumes messages in batches, starting at five concurrent batches with five functions at a time. If there are more messages in the queue, Lambda adds up to 60 functions per minute, and up to 1,000 functions, to consume those messages.

This means that Lambda can scale up to 1,000 concurrent Lambda functions processing messages from the SQS queue.

Failed messages are sent back into the queue, to be retried by Lambda. A DLQ (Dead letter queue) is put in place to prevent failed messages from getting added to the queue multiple times.

Once in DLQ, these messages can be reassessed and resent to the Lambda for processing by humans, through a redrive policy.

Once Lambda successfully processes a message, it extracts the order payload and invokes a step functions workflow with the payload as the input request.

We use step functions to orchestrate the payment process. No real APIs are being called. All we do is mimic a real-life scenario.

Inside the Step Functions, we randomly determine if an order was paid or not, save the order into DynamoDB, and then send success or failure emails using SNS to the client who made the order.

We also create endpoints to get_order, update_order, and delete_order.

Enough talk. Let's see some code.

🚨Note: We would be looking at code snippets and not the complete source code for the application.

For the complete code, please visit the GitHub page at Event Driven CDK

Let's end here and continue in part 2.

Hope you enjoyed reading this piece as I enjoyed writing it.

Do you see any errors?

Have any suggestions?

Loved the article?

Please like or leave a comment.

See you in part 2✌🏾

In this article, I will try to shed some light on this new feature, what JS resolvers are, how they can improve developer experience, what possibilities they unlock; but also their limitations.

What JS Resolvers Are Not

Before we start, it is important to make some clarification about what JS resolvers are not. There might be some misconceptions that will arise, mostly because they allow writing resolvers in a way that many developers already write their Lambda functions with (i.e. JavaScript). We should clear that out of the way from the beginning: JS resolvers are not a replacement for Lambda function resolvers, they will never be. You won't be able to do whatever you want; they come with a set of limitations that we will explore later. They are, however, a good alternative to Lambda in some cases and should simplify the choice between both options.

What Are JS Resolvers, Then?

They are a new way for AppSync to communicate with your data sources. This is what we have always known as “mapping templates”; except that instead of writing them in the VTL language, you can now use (a subset of) JavaScript, which brings tons of benefits (see below). You still need to attach your JS resolvers to a data source, and the only thing they should do is generate a request for that data source and format the response it returns.

From the doc:

The request handler takes a context object as an argument and returns a request payload in the form of a JSON object used to call your data source. The response handler receives a payload back from the data source with the result of the executed request

What benefits do they bring?

Now that we made clear what JS resolvers are, let's talk about their benefits; because there are! (other than "I don't like VTL 🤓")

Better Developer Exprience

Being able to write AppSync handlers in JavaScript obviously makes it easier for everyone. JS is a language that most developers are familiar with, and it has so many benefits over VTL such as better IDE support or code linting. If you want type support, you can also write them in TypeScript as long as you transpile them to JavaScript later.

Room for extension

JavaScript makes it possible to write reusable code by creating custom functions, utilities, or libraries. This was not (easily) possible with VTL. This should make it easier to write better code faster.

No cold start and no extra cost

JS resolvers, unlike Lambda functions, will not suffer cold start, and there is no penalty for choosing JS over VTL in terms of performance. They are also available at no extra cost. Being able to write resolver handlers as JS will potentially allow many to get rid of unnecessary Lamba functions; reducing latency and cost at the same time.

Limitations

If JS resolvers have some benefits, you should be aware of their limitations. These limitations might evolve over time, but some of them will likely never go away. Keep in mind that if JS resolvers don't allow you to do some things, it might as well be to protect you from using bad practices.

Only Pipeline resolvers are supported

For the time being, only Pipeline resolvers are supported. Unit resolvers still require the usage of VTL mapping templates.

Hint: 💡 If you still want to use JS with a unit resolver, you can always write a pipeline resolver that has only one function.

Unsupported JS features

It is primordial to remember that JS resolvers use a subset of ECMAScript 6. It means that all features are not available. For example, you can't use: for loops (for-in and for-of are supported), try, catch, finally, continue, do-while or while loops.

throw is also not supported. If you want to "throw" an error, you should use the util.error() util, which is basically the equivalent to the $ctx.util.error() that you already know.

Arrow functions are not supported either.

No Async/Await

async/await are not supported, for obvious reasons. JS resolvers should be completely synchronous and return as fast as possible.

No network access

You won't be able to do any network request in a JS resolver (e.g. you can't use fetch/axios).

💡 Hint: If you need to do that you probably want to use an HTTP resolver instead.

Size limitation and import

If you follow the above rules, you are free to do everything you want. While you can't use the import statement (except with @aws-appsync/utils), you should be able to include custom libraries and code if you bundle them in a single file before you upload them to AppSync. (e.g. using esbuild).

There is a big catch though: all the code in the bundle MUST be a valid ES6 module, follow all the above AppSync JS rules, and the final package must not exceed 32KB.

Unfortunately, this means that you will likely never be able to include almost any npm package out there at the moment. But, it opens the door to creating dedicated AppSync-compatible libraries that strictly follow the requirements. A quick POC shows that this is possible.

Please refer to the documentation for a full list of supported and non-supported features.

What Else Has Changed?

The AppSync utils

In VTL, we were used to invoking functions like $util.dynamodb.toDynamoDBJson(). Those utilities still make sense and are useful in JS. They now live in a new npm package: @aws-appsync/utils.

This new package also contains an API for interacting with AppSync: extensions.

CloudWatch logs

Those who, like me, were used to diving into the CloudWatch AppSync logs will notice new log types: BeforeRequestFunctionEvaluation, RequestFunctionEvaluation, ResponseFunctionEvaluation, AfterResponseFunctionEvaluation.

These log records are equivalent to the ones that you used to see when debugging VTL mapping templates (namely BeforeMapping, AfterMapping, RequestMapping, and ResponseMapping), but they are specific to JS resolvers. They look roughly the same as their VTL counterpart.

{
  "logType": "RequestFunctionEvaluation",
  "fieldName": "getPost",
  "resolverArn": "arn:aws:appsync:us-east-1:12345678912:apis/abcdefghijklmnop/types/Query/resolvers/getPost",
  "functionName": "getPost",
  "fieldInError": false,
  "evaluationResult": {
    "operation": "GetItem",
    "key": {
      "id": {
        "S": "b03a2a76-1f2c-4150-9b3c-aab88a04f49e"
      }
    }
  },
  "parentType": "Query",
  "path": [
    "getPost"
  ],
  "requestId": "83e25c50-8d97-4939-8475-440c0ed8b36d",
  "context": {
    "arguments": {
      "id": "b03a2a76-1f2c-4150-9b3c-aab88a04f49e"
    },
    "prev": {
      "result": {}
    },
    "stash": {},
    "outErrors": []
  },
  "errors": [],
  "graphQLAPIId": "ykwu6mw2mrc3ho4jczeg6p42pa",
  "functionArn": "arn:aws:appsync:us-east-1:12345678912:apis/abcdefghijklmnop/functions/uvwyyz"
}

You can also write custom logs by using the console.log() function. You will see them appear in CloudWatch.

83e25c50-8d97-4939-8475-440c0ed8b36d INFO - code.js:5:3: "Hello from JS Resolver!"

Conclusion

JS resolvers will greatly improve the developer experience. They should help developers adopt direct integration with data sources (e.g. DynamoDB) more often, while Lambda functions are still useful for more advanced and complex use cases.

There is still a lot to explore, though. This is only the beginning. I am sure the community will come up with great ways to use them and extend them 🚀

Credits: Cover image generated by dall·e