Last month during Re:Invent preview of EventBridge Schema Registry had been announced. One of unique feature that the service brings is the automatic discovery of any custom event publish to EventBridge. As soon as the discovery will be enabled on Event Bus with the service will aggregate the event over period of time and register them as OpenAPI document into the registry. You might ask yourself what is exactly the use case in which you will need a feature like that? Typically the discovery of event will be particularly useful whenever you don’t own the source that publishes the events typically when it’s owned by other team in you organization or even a third party. Even for consuming the AWS EventBridge events prior to the release of the service typical onboarding process required fallowing the contract of the events defined through the AWS documentation and mapping that into language of choice. That could be really tedious.
As for today the service allows integration with EventBridge, though it is definitely not limited to it. In matter of fact it will be able to easy integrate with any JSON base event source. To demonstrate that capabilities with little amount of code we can demonstrate how to automate the discovery of schema from any SNS topic.
To achieve that a little bit of wiring needs to be done to pipe the events from SNS to EventBridge. Unfortunately EventBridge is not a natively supported target for SNS at the moment. Instead we can use AWS Lambda to consume the event from SNS and forward it to EventBridge.
This leads us to very straight forward integration.
To wire everything together we can create a SAM template.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
aws-schema-discovery-sns
Sample SAM Template for aws-schema-discovery-sns
Globals:
Function:
Timeout: 60
Resources:
SnsDiscoveryTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: 'sns-discovery'
SnsSchemaDiscoveryFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: app/
Handler: app.lambdaHandler
Runtime: nodejs12.x
Environment:
Variables:
event_source: sns.topic
event_detail_type: !GetAtt SnsDiscoveryTopic.TopicName
Events:
SnsEvents:
Type: SNS
Properties:
Topic: !Ref SnsDiscoveryTopic
Policies:
- Statement:
- Sid: EventBridgePutEventsPolicy
Effect: Allow
Action:
- events:PutEvents
Resource: '*'
Outputs:
SnsSchemaDiscoveryFunction:
Description: 'SNS Schema Discovery function Lambda Function ARN'
Value: !GetAtt SnsSchemaDiscoveryFunction.Arn
SnsSchemaDiscoveryFunctionIamRole:
Description: 'Implicit IAM Role created for SNS Schema Discovery function'
Value: !GetAtt SnsSchemaDiscoveryFunctionRole.Arn
This will deploy through CloudFormation a Lambda function that will be subscribed to dedicated SNS topic. Publishing any message to the SNS will forward it to EventBridge and providing that the schema discovery had been enabled on the default Event Bus auto discovered schema will be registered.
Next thing is to develop the Lambda code. To keep things since let’s use JavaScript for that.
const AWS = require('aws-sdk');
exports.lambdaHandler = async (event, context) => {
try {
var eventbridge = new AWS.EventBridge();
var entries = [];
event.Records.forEach(record => entries.push({
Source: process.env.event_source,
DetailType: process.env.event_detail_type,
Detail: record.Sns.Message,
Time: new Date()
}));
return eventbridge.putEvents({
Entries: entries
}).promise();
} catch (err) {
console.log(err, err.stack);
throw err;
}
};
The code is very simple. It will deliver any event published to SNS back to EventBridge. It uses the configured source and detail-type attributes to identify this particular event.
The last two things is to enable the discovery on the EventBus. This can be done from Console, CloudFormation or using CLI command.
aws schemas create-discoverer --source-arn arn:aws:events:us-east-1:${ACCOUNT_ID}:event-bus/default
Now by publishing an event to SNS topic we will get automatic registered schema.
To demonstrate that everything is working let’s publish a sample JSON document into the SNS topic.
{
"Records": [{
"AwsRegion": "us-east-1",
"AwsAccountId": "12345678910",
"MessageId": "4e4fac8e-cf3a-4de3-b33e-e614fd25c66f",
"Message": {
"instance-id":"i-abcd1111",
"state":"pending"
}
}]
}
The above event results in fallowing OpenAPI document created in the registry:
{
"openapi": "3.0.0",
"info": {
"version": "1.0.0",
"title": "SnsDiscovery"
},
"paths": {},
"components": {
"schemas": {
"AWSEvent": {
"type": "object",
"required": ["detail-type", "resources", "detail", "id", "source", "time", "region", "version", "account"],
"x-amazon-events-detail-type": "sns-discovery",
"x-amazon-events-source": "sns.topic",
"properties": {
"detail": {
"$ref": "#/components/schemas/SnsDiscovery"
},
"account": {
"type": "string"
},
"detail-type": {
"type": "string"
},
"id": {
"type": "string"
},
"region": {
"type": "string"
},
"resources": {
"type": "array",
"items": {
"type": "object"
}
},
"source": {
"type": "string"
},
"time": {
"type": "string",
"format": "date-time"
},
"version": {
"type": "string"
}
}
},
"SnsDiscovery": {
"type": "object",
"required": ["Records"],
"properties": {
"Records": {
"type": "array",
"items": {
"$ref": "#/components/schemas/SnsDiscoveryItem"
}
}
}
},
"SnsDiscoveryItem": {
"type": "object",
"required": ["AwsRegion", "Message", "AwsAccountId", "MessageId"],
"properties": {
"Message": {
"$ref": "#/components/schemas/Message"
},
"AwsAccountId": {
"type": "string"
},
"AwsRegion": {
"type": "string"
},
"MessageId": {
"type": "string"
}
}
},
"Message": {
"type": "object",
"required": ["instance-id", "state"],
"properties": {
"instance-id": {
"type": "string"
},
"state": {
"type": "string"
}
}
}
}
}
}
The source code of the SAM application is available at Github repo. Feel free to try it out.
I just do things 