How to better handle updateSequenceNumber with AWS EventBridge?

Matthew Hardesty_vzukrh

Goal

I want to publish custom and deployment events to Compass when certain events occur in AWS QuickSight (like dashboard versions being updated, data sets being updated, data sources being deleted, etc...). These events don't require any separate build tool, but they're still important to surface. 

This is — almost — trivially possible to do with AWS EventBridge.

1. Set up event pattern
2. Set target to Compass API
3. Configure input transformer to map source event data to Compass API schema

Problem

The problem I've encountered is trying to populate updateSequenceNumber — it only accepts an integer. However, AWS event data doesn't have any integer available to represent this — at all.

It would make a lot more sense to me to allow this to be any sortable string instead of requiring an integer. For example:

• ISO 8601 timestamp ("2024-11-27T01:44:15Z")
• Sortable UUID (that includes hyphens)

Workaround

If Compass can't support sortable values for updateSequenceNumber, it makes accomplishing this task obnoxious since the workaround requires adding an intermediate step to simply convert a timestamp into epoch, and this would incur additional cost that, in my view, should be totally unnecessary. 

The questions are ultimately:

1. Have I overlooked something obvious (either with the Compass API or AWS EventBridge)?
2. Is there some other simpler workaround?
3. What is the likelihood that Compass could
   1. Not require sequence number (i.e., if not provided, just auto increment some sequence number you track yourself)?
   2. Allow sequence number to accept any sortable string?

Steffen Opel [Utoolity]

Hi @Matthew Hardesty, and welcome to the Atlassian Community!

I hope the Compass team will chime in still, just wanted to add our resp. experience from the perspective of building AWS integrations on the Compass platform from day one (also atop EventBridge):

1. You have not overlooked anything obvious, we also stumbled over the very problem you encountered here.

2. I'm not aware of any simpler workaround and we had to tackle it in the same fashion for Develop with AWS (Compass), i.e. by converting the ISO timestamps to numbers.

3. 1) That's a smart proposal that might fit with the ongoing work on custom webhooks, so I highly recommend to engage with Josh Campbell there (see below).

3. 2) While the Compass team has been a joy to work with and very receptive to change requests during beta, now that Compass is in GA for over a year they probably consider the API stable and need to treat careful with changes. Given updateSequenceNumber is core to how the event ingestion works, they are most likely unable to change it to a string after the fact, not the least due to also being used by various in-house solutions and public apps meanwhile, including ours.

Cheers,
Steffen

PS: Your use case is pretty interesting, insofar I've not expected QuickSight to be a candidate service for event ingestion into the Compass activity feed - if you'd rather not maintain this yourself, feel free to reach out, maybe we can integrate it into our app (it's currently free, though we'll transition to a commercial offering once that's supported by the Atlassian Marketplace).,