Search for a command to run...
How to Generate Files on Demand and Deliver Download URLs Using GraphQL Subscriptions
👨💻 Sofware Engineer · 💬 Consultant · 🚀 Indie Maker · ⚡️ Serverless
🔧 serverless-appsync-plugin · serverless-appsync-simulator · middy-appsync
🐈 and 🍵
No comments yet. Be the first to comment.
Using Pre-Signed URLs for Scalable, Secure File Handling in GraphQL APIs
Find the path to achieving AppSync excellence, by following those simple best practices
Over the past years, event-driven architecture has become more and more popular. With the arrival of services such as EventBridge and its integration with many other AWS services, building such applications has become easier than ever. Last year, AWS...
What's new in V1.0.0?
In a previous article, I explained how to generate Amazon S3 pre-signed download URLs using AWS AppSync. But what happens when the file doesn’t exist yet?
Sometimes the file must be generated on demand. Maybe it’s a report for a user-selected date range, a data export based on filters, or something that only makes sense at that specific moment.
You might be tempted to generate that file directly within the AppSync resolver (e.g. a Lambda function), store it on S3 and then return the pre-signed URL; but this would be an anti-pattern.
AppSync resolvers should stay thin
Resolvers are not meant to perform heavy processing. Long-running or compute-heavy tasks don’t belong there.
Hard time limits
AppSync operations have a 30-second timeout. File generation, especially for large datasets, can easily exceed that.
Reliability & retries
File generation can fail. You may want retries, dead-letter queues, monitoring, and proper error handling — all of which are far easier to implement asynchronously.
In this article, I’ll show you how to properly decouple file generation from the AppSync request using Asynchronous Lambda invocation, Amazon EventBridge, and AppSync Subscriptions.
AWS AppSync supports long-running operations by allowing asynchronous Lambda function invocations as a data source (resolver). When the Lambda function is invoked asynchronously, the AppSync resolver returns immediately, while the Lambda function is allowed to run for up to 15 minutes.
We can leverage this capability to trigger our file-generation process without blocking the user's request.
Once the Lambda function finishes generating the file, it can store it in an Amazon S3 bucket, generate a pre-signed download URL, and send it back to the user through a Subscription using the AppSync–EventBridge integration.
We use three GraphQL operations:
generateReport
This is the Mutation the user calls to express the intent to generate a file. It receives, for example, a start date and an end date and returns a requestId that uniquely identifies the process.
onReportGenerated
This is a Subscription that the client uses to subscribe to updates. It takes a requestId as input — the same requestId returned by the generateReport Mutation.
It is essentially a way of saying:
“Notify me once this specific report has been generated.”
reportGenerated
This is another Mutation that the backend invokes to trigger the above Subscription.
The client invokes the generateReport Mutation.
The AppSync JS resolver generates a random requestId, invokes the GenerateReport Lambda function asynchronously, and passes it the requestId along with the user inputs.
AppSync immediately returns the requestId to the client.
The client receives the requestId and uses it to subscribe via onReportGenerated.
Once the Lambda function finishes generating the file, it stores it in an Amazon S3 bucket.
The Lambda function publishes a reportGenerated event to the event bus, including the requestId and the pre-signed download URL.
The AppSync reportGenerated Mutation target is triggered through the EventBridge integration.
AppSync pushes the event back to the client via the active Subscription.
The client receives the download URL and initiates the file download in the browser.
When using asynchronous Lambda invocations, error handling behaves differently from synchronous calls.
By default, if an asynchronous Lambda execution fails, AWS automatically retries it twice after the initial failure (three attempts in total). If all retry attempts fail and no additional configuration is in place, the event is discarded.
If you need more control over failure handling, retries, and observability, there are several options to consider:
Lambda Destinations
You can configure Lambda function destinations for asynchronous invocations. This allows you to route failed events to another target — such as:
An SQS queue (acting as a DLQ)
An SNS topic
EventBridge
Another Lambda function
This gives you visibility into failures and the ability to reprocess or inspect failed events.
Introduce an SQS Queue
Instead of invoking the Lambda function asynchronously and directly from AppSync, you can place the event onto an Amazon SQS queue.
The Lambda function then consumes messages from the queue.
This approach provides:
A configurable retry policy
Built-in dead-letter queue support with redrive capability
Better control over failure handling
Buffering and backpressure management
In this article, we explored how to handle on-demand file generation in a clean, event-driven way using asynchronous Lambda invocations, EventBridge, and AppSync Subscriptions. Instead of blocking a GraphQL request, we decouple processing from the client interaction, allowing long-running tasks to execute reliably while still delivering real-time feedback when the file is ready.
