Now Available: Import Metrics from GitHub!

[#1]

Hey everyone, Henry here from the Compass Ecosystem Engineering team. We’re happy to announce that V3 of the GitHub app for Compass is now available! This version of the app will allow your linked Compass components to import key metrics from GitHub, including deployment frequency, # of open pull requests, and pull request cycle time.

To get started, make sure you have the GitHub app for Compass configured. If you already have the app installed, you may need to upgrade by clicking the “upgrade” button on the Github app card. 

Once your app is up-to-date and connected to a GitHub organization, try adding a link to one of the organization’s GitHub repositories to a Compass Component. Refresh the page and you should see the relevant metrics populate automatically. Here's a video of the upgrade process and the metrics feature in action.


Let us know what else you’d like to see from the GitHub Compass app, or what metrics matter most to your team!

Cheers,
Henry

[Henry Olson]

Just a follow-up note: We're updating our docs to include more details, but if you want to set up a GitHub Action that creates a deployment, you can checkout an example here:

https://github.com/compass-t2/compassy/blob/main/.github/workflows/blank.yml

The `deployment` key is specifically what creates a Deployment Event in Compass and contributes to the Deployment Frequency metric.

[Richard Simpson]

@Henry Olson this looks fantastic and just digging into this! Is it possible that what defines a deployment will be configurable in the future? Or that we could directly signal to Compass? (For example, today we use something like https://www.elastic.co/guide/en/kibana/current/apm-annotation-api.html)

Also, will Compass include build metrics in the future? We're currently looking for a way to help teams understand the health of various repositories, and metrics + scorecards seems perfect for that but we're currently missing the ability to have build health included.

[Henry Olson]

Hi @Richard Simpson , thanks for the feedback, glad you're enjoying the new feature! It's definitely possible to create custom Deployment events in Compass using the GraphQL API (https://developer.atlassian.com/cloud/compass/graphql/). Specifically, you would need to call `createEventSource`, `attachEventSource`, and `createCompassEvent`. I can help provide further guidance if needed! 

Right now, the GitHub app has only one way to define deployments, but we can definitely investigate other ways of defining events in your GitHub actions. We also don't currently have an integration with Kibana, but I'll make sure @Dave Parrish _Atlassian_ and the rest of our product team are aware of your use case :) 

Lastly, Builds are also available in the Compass API, they're just not surfaced in the UI or in any of our integrations currently. We'll soon be releasing a CircleCI integration that will ingest build events and populate metrics based on them, but we'll also need to backport this functionality to some of our other apps. We do have some built-in metrics like Build Success Rate that can be populated manually or through a Forge app in the meantime. You'll need to use the API to `createMetricSource` on a component and `insertMetricValue` to populate it. I hear you though, we're working to add more ways of ingesting these metrics since they're vital to determining component health!

[Richard Simpson]

@Henry Olson following up on this now. Specifically looking to populate the Build Success and Build Time pre-defined metrics from GitHub-linked repos (and really hope this comes to the official integration soon). I _believe_ I see how this should be done and your comments were very helpful, but I'd also love if there's any sort of example on guidance for building that out as it's a daunting-ish amount of work.

[Henry Olson]

Hey @Richard Simpson , apologies for the late reply, I've been OOO. We're working on scoping and planning Derived Metrics, which would allow you just submit Build events to Compass and have Compass calculated metrics like Build Success and Build Time automatically. 

In the meantime, I recommend taking a read through https://developer.atlassian.com/cloud/compass/integrations/create-a-data-provider-app/ . We've exposed a new compass:dataProvider forge module which allows you to easily submit events and metrics when specific link types are added to a component. In your case, you could check for Github-related links, and create the appropriate metrics. However, you'd need a Github developer token to get this to work in a new app.

I've noted your request for our official Github App and will work with @Dave Parrish _Atlassian_ to prioritize it :) 

[Richard Simpson]

@Henry Olson Derived Metrics sounds excellent! Can't wait to see how it unfolds.

I actually ended up building a whole two apps via the dataProvider module! Was a massive help and let me PoC getting a lot of data we wanted it the system.

[Henry Olson]

That's great to hear, I'm glad it was useful! If you have any feedback on the module itself, don't hesitate to share, we want this stuff to be as developer-friendly as possible.

[Chase Pierce]

I was able to get a workflow that functions to finally show deployments to production correctly, but is there any plan to integrate build rate success into this as well, or is there some hidden functionality that I'm not seeing?

Something that might be problematic for this working on my end is, I have the following "deploy" workflow (because I have multiple deployment types, this one represents the production one, which basically looks like:

jobs:

  build:
    uses: ./.github/workflows/build.yml
    secrets: inherit

  deployment:
    needs: [ build ]
    runs-on: ubuntu-latest
    steps:

    - run: echo "deployment"

Where the shared workflow contains a single job called `docker`. Do I need to keep the job names as `build` across the board, and does this functionality work for workflow's being shared, given that the full paths come out to something more like:

Docker/build/docker

Rather than

deploy/build

If it wasn't shared.

[Henry Olson]

Hi @Chase Pierce , we currently don't ingest builds or buildSuccessRate in the GitHub Compass app, but this is definitely something we're planning on adding in the future! I've tagged @Dave Parrish _Atlassian_ here to make sure he's aware of the ask as well.

I believe as long as you have the `deployment` key under `jobs` in your workflow, that should trigger the GitHub deployment event which hits our app's webhook. The `build` job shouldn't be necessary for getting deployment events into Compass. I'm not 100% certain on your use case though - are you receiving deployment events when your workflow runs as expected, or are they not coming through to Compass? I'm happy to help debug if you're not receiving events when you think you should be.