The World is Asynchronous
-- Dr Werner Vogels ReInvent 2022
The concept of Event Driven Architecture (EDA) is not new. As a matter of fact, it's been around for ages.
But lately, it's been the talk of every tech clique. There's no way you can scroll through tech Twitter without coming across a tweet concerning EDA.
But what are EDA's and why is every organization adopting this architectural pattern?
Event Driven Architectures?
Minimizing the degree of interdependence between different components of a system is the main consideration when building event-driven architectures. The term loosely coupled or Decoupling is generally used.
Lots of integration patterns have been created throughout the years, to tackle coupling within an application.
So how do you determine the level of coupling within your application's components and what strategies do you use in order to guarantee a loosely coupled system?
IT strategist and cloud/enterprise Architect, Gregor Hohpe Says
The appropriate level of coupling depends on the level of control you have over the endpoints
Generally, if your application depends on third-party APIs which you have less control over, you don't want to build a strong attachment with those APIs. Because, if they go down, your application goes down.
You want a scenario where you'll be able to easily pull back your resources in case of a system hiccup. This goes same for all the other components of your system, whether internal or external.
As we progress through the application, we'll see how to use AWS Services to ensure your system is loosely coupled
EDA's use events to coordinate communication between loosely coupled services.
But what are events?
Events
An event is a change in state, or an update, like an order added to the cart in a restaurant application. Events can either carry the state (the item purchased, its price, and quantity) or events can be identifiers (a notification that a payment was successful or not).
EDAs are made up of 3 parts.
Event Producers
This can be a mobile app or an e-commerce site
Event Routers
This can be an event store
Event Consumers
This can be a database, saas application, microservice, etc
So an event producer publishes an event to the event router, the event router filters and pushes the event to event consumers. The event router abstracts the producer from the consumer by allowing them to communicate asynchronously.
Why you should adopt EDA
With services that scale on demand and fail independently, your application is bound to be reliable and resilient.
Event routers coordinate events automatically between producers and consumers, so you no longer need to write code to handle that.
Event-driven architectures are push-based, so everything happens on-demand as the event presents itself in the router. This way, you’re not paying for continuous polling to check for an event.
EDAs are responsive to the needs of customers as they expect systems to respond to events as they happen.
EDAs are cost effective, allowing customers to pay only for the services they need when they need them.
Now that we have a brief understanding of what EDAs are, let's go ahead and dive into the main topic for the blog post.
Article Scope
We'll use the concept of EDA to design and build a modern serverless Graphql API on AWS. We will be doing so, by mixing and matching different AWS services in a technical manner.
Out Of Scope
We'll not be looking at introductions to any of the AWS Services used in this tutorial such as
AWS Step functions
AWS SNS
AWS SQS
But I'll provide supporting documents to upskill if need be.
This tutorial is aimed at the following audiences:
- Software engineers looking for a quick hands-on intro to Event-Driven Architectures
Use Case
Let's say you want to order pizza from a restaurant, through the restaurant's mobile or web application.
In an ideal scenario, the application would let you make (add to cart) your choice of pizza, and the quantity you want, with all necessary add-ons like extra cheese, and chili, and only let you sign in to or create an account when you wish to place the order.
In this tutorial, we'll assume you've already created an account, and you are about to place an order. Once your order has been placed, we'll run a fake payment check to determine if you've made payment or not, and would email you based on the status of the order, be it success or failure.
We'll be using a couple of AWS Services to illustrate how all these can be accomplished, in a scalable, decoupled, event-driven manner.
Here are the AWS services we'll be using in this application.
AWS Services
AWS AppSync is a fully managed service allowing developers to deploy scalable and engaging real-time GraphQL backends on AWS.
It leverages WebSockets connections under the hood to provide real-time capabilities, by publishing data updates to connected subscribers.
A fully managed message queueing service to decouple producers and consumers. SQS is a fundamental building block for building decoupled architectures
AWS Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software-as-a-service (SaaS) applications and only pay for what you use.
AWS Step Functions is a visual workflow service that helps developers use AWS services to build distributed applications, automate processes, orchestrate microservices, and create data and machine learning (ML) pipelines.
Amazon Simple Notification Service (SNS) sends notifications in two ways, Application-to-Application (A2A) and Application-to-Person (A2P).
A2A provides high-throughput, push-based, many-to-many messaging between distributed systems, microservices, and event-driven serverless applications. These applications include Amazon Simple Queue Service (SQS), Amazon Kinesis Data Firehose, AWS Lambda, and other HTTPS endpoints.
A2P functionality lets you send messages to your customers with SMS texts, push notifications, and email.
For this application, we'll be using A2P.
Amazon DynamoDB is a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale. DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools.
Solutions Architecture
From the architecture above, the Amplify icon signifies a front-end application. It can be a mobile or web app. So a client places an order for 5 cartons of pizza from Domino's pizza.
Here's the request payload
"name": "Pizza",
"quantity": 6,
"restaurantId": "Dominos
AppSync receives the request and sends a message, containing the payload to an attached SQS queue. We use SQS to decouple the event producers (AppSync) from the consumers (in this case a Lambda), so that they can communicate asynchronously.
So as order requests keep flooding in, let's say on a world pizza day when demand is really high, all requests are being sent to SQS. Lambda is a common choice as a consumer for SQS as it supports native integration. So you get to write and maintain less code between both of them.
When a Lambda function subscribes to an SQS queue, Lambda polls the queue as it waits for messages to arrive. Lambda consumes messages in batches, starting at five concurrent batches with five functions at a time. If there are more messages in the queue, Lambda adds up to 60 functions per minute, and up to 1,000 functions, to consume those messages.
This means that Lambda can scale up to 1,000 concurrent Lambda functions processing messages from the SQS queue.
Failed messages are sent back into the queue, to be retried by Lambda. A DLQ (Dead letter queue) is put in place to prevent failed messages from getting added to the queue multiple times.
Once in DLQ, these messages can be reassessed and resent to the Lambda for processing by humans, through a redrive policy.
Once Lambda successfully processes a message, it extracts the order payload and invokes a step functions workflow with the payload as the input request.
We use step functions to orchestrate the payment process. No real APIs are being called. All we do is mimic a real-life scenario.
Inside the Step Functions, we randomly determine if an order was paid or not, save the order into DynamoDB, and then send success or failure emails using SNS to the client who made the order.
We also create endpoints to get_order, update_order, and delete_order.
Enough talk. Let's see some code.
🚨Note: We would be looking at code snippets and not the complete source code for the application.
For the complete code, please visit the GitHub page at Event Driven CDK
Let's end here and continue in part 2.
Hope you enjoyed reading this piece as I enjoyed writing it.
Do you see any errors?
Have any suggestions?
Loved the article?
Please like or leave a comment.
See you in part 2✌🏾
