An update on Compass and our journey so far

[#1]

We announced Compass back in April at Team 2021 to enormous interest - clearly the idea of a tool to help transform the way your software teams work with distributed systems resonates with many of you!

As part of Atlassian's Point A program we've been working closely with many of you to help co-create Compass, understanding and learning more about how we can help harness the power of distributed systems and the culture of DevOps, and iterating our way forward. So thanks to everyone who has tried the product and given us feedback during the alpha so far!

Moving to an 'open' alpha

Based on your feedback so far we're confident in the problems we're trying to solve and the direction we're headed. We strongly believe that developer experience platforms like Compass can transform the way your teams work and allow your engineers to be more creative, happy, and productive.

This is why we're excited to announce the next step in our journey by removing the waitlist and allowing for self-serve signups.

It's a culmination of all the hard work and effort the team has put in over the last 6 months (more about that below), and we feel the alpha is ready for wider use and feedback. Now all you have to do is go to https://www.atlassian.com/software/compass and you can install Compass on any existing or new site.

We look forward to hearing your feedback so we can continue crafting the product with your help into something your teams will love to use!

A look back at what's shipped

We have made some incredible progress with the alpha over the last few months, with a focus on the following areas:

• Get started with your catalog.

• Add information about your components.

• Build a model of team ownership and component health.

• Extend the product UI and craft your own dev experience.

Let’s take a look at these in more detail.

Get started with your catalog

Creating and importing components

The starting point for creating a great developer experience is tracking the catalog of software components that your teams build, assemble, and operate. We made it easier to import thousands of components through various methods, including SCM tools like Bitbucket and GitHub.

Search and labels

We made it easy to search and find components, helping you get the information you need to understand your architecture fast. And with labels you can arbitrarily group components and use those labels to search - with more ways to use labels coming in the future.

Atlassian editor smart links

The smart links functionality in the Atlassian editor allows you to reveal additional context about links you drop into a confluence page, Jira issue, or comment. By pasting a Compass component URL, it will unfurl to reveal more information, making it easy to reference different parts of your architecture as you discuss and collaborate!

Config-as-code

Compass provides a great way for GitOps practitioners to scale and manage thousands of component definitions through the use of ‘config-as-code’. This means you can use source code, via a compass.yml file, to act as the source of truth for any particular component, tying directly into your existing development workflows.

Add information about your components

Jira integration

Connecting the work moving through Jira to the individual components in your distributed architecture is important for teams that build and operate software, allowing them to find the balance between innovation and stability, providing a true total cost of ownership that cuts across both dev and ops.

Activity feed with deployment

This is a powerful new capability that allows your teams to understand what is happening to each component across various tools & events - starting with deployment events for your services.

Dependency management and announcements

As your architecture and DevOps practices scale, the ability to track the relationships of components and co-ordinate activities across them and related teams grows in importance. Compass helps keep track of these dependencies and allows teams to broadcast messages as an initial starting point.

Component-level API specs

Software isn’t written anymore, it’s assembled from various components owned by autonomous teams, and successfully assembling software across a wide set of components requires an understanding of how to use them. This includes understanding the APIs and contracts made available to interact with each individual component. Here, we are using the OpenAPI specification to render API information simply by adding a swagger.json file to that component.

Build a model of team ownership and component health

Team ownership and dashboards

Based on Atlassian’s own experience supporting and building products for over 200,000 customers, Compass provides an opinionated stance that all components should be owned by a single team. Atlassian Teams are a relatively new concept that will span across all of our products allowing you to tie work, documents, and components together. This allows you to aggregate component health information about each team and across your entire organisation.

Scorecards

Scorecards are a powerful way to tie disparate information across many different tools and then codify best practices for your teams to follow. You can see which components or teams are scoring low on development practices or operational maturity, giving you a birds eye view you need to take action and improve where it is needed.

Extend the product UI and craft your own dev experience

There is no one-size-fits-all when it comes to how you build and operate software, and that's why we've spent time making Compass a developer experience platform that's as extensible as possible. Not only can you interact with all the information you see in the product through our APIs, but you can also extend the UI itself to tailor the developer experience to suit your own needs.

We have extension points on the component detail view and global view at the moment, with many more extension points to come! Using Atlassian Forge, you can let Atlassian handle running the apps so you can focus on the application logic.

You can get started by checking out our developer docs.

See how you can create a static ‘hello world’ app in under 60 seconds!

What's coming next

Being an Alpha product and part of Point A means that we're rapidly iterating and our roadmap is quite fluid. This means it's a great time to let us know what you would like to see and we can respond quickly to your feedback.

Note: Please keep in mind these are design mockups to convey the concept, they will change as we test & learn while building each new capability.

Below are just a few highlights of things we are spending time exploring:

Component metrics and scorecards

Customers love the idea of scorecards, but need a flexible way to add more information about components so they can construct more complex sets of criteria across dev and ops. Using the new concept of metrics, you can add all kinds of information about those components - some of which we will provide by default as best practices (when you install Bitbucket for example) and others you can define yourself. Read more here.

Activity feed - new event types and timeline view

We will continue to invest in the activity feed by allowing new event types and then track those events across your dependencies as well. We will move this into a new timeline view to key on time to make it easier to triage incidents, looking across everything that has happened related to that particular component at a given point in time. This pulls in course-grain information across tools for you to then dive into specific areas in those tools for the details.

Improve dependencies by tracking incidents

We will continue to invest in dependencies by overlaying incidents that occur to a particular component, as well as those you depend on. In a world of many components and assembling software, understanding the health and quality of your dependencies is just as important as keeping your own components in a healthy state.

Improved search and filtering

As more information gets added to the catalog, we want to provide you with an easy way to search and find information through a powerful search mechanism. This would allow you to search on different criteria, save filters, and lots more!

Team dashboard - CheckOps and UI extension points

In DevOps, teams are responsible for both building and operating their software. On the operations side, this can be a shared responsibility through the use of scorecards - owned by a platform team, and then the teams that need to keep those scorecards in a passing state. One way we can save your team a ton of time is by aggregating all this information into a weekly check-in for your development team to assess whether they are doing enough to keep their components healthy. There is a balance between building new features and paying down technical debt - this gives you a great way to see that often hidden cost of engineering health.

Look out for more information about the above features here in the Community (be sure to join the group to be notified of new posts!). We’re excited about this next phase of the alpha and look forward to collaborating with you in 2022 and beyond

[Ammar Ahmed Butt]

Awesome !!!!!!! 

[Dave Liao]

@Kelvin Yap - glad to see you on Community, Kelvin!

• First time I've seen "CheckOps" on Community, and Google-ing that term gets me neat results. 🙂
• Had no idea we could extend Compass programmatically, but since Jira Cloud supports this (via Forge) why not Compass?

Looking forward to what's next for Compass! 🧭

[Nikki Zavadska [Appfire]]

An amazing introduction to Compas Alpha, thank you @Kelvin Yap !

I have a question related to components and connecting Compas components to Jira projects. Are the Compas components created automatically in the Jira project when I create a connection and are the changes for example to the names and so on synchronized?

[Taylor]

Hi @Nikki Zavadska [Appfire],

There will be a longer post on this soon, but today there is no automatic creation/syncing of the components field across Jira and Compass today.

The current version is a first iteration, we have plans to improve from the current state so we would love to hear about your use case. What are you trying to achieve and what is the ideal solution to solve that problem?

Currently, the integration provides a simple mechanism to tie work to components being tracked in Compass, so that your work (maybe eng health type of work) can appear alongside that individual component, like a microservice or shared library, to see if there is a big backlog of technical debt that needs to be addressed. Jira has a lot of use cases, so it's been interesting to hear how teams have been wanting to use this.

How it works today

1. You go to an individual component in Compass. You select the Jira project to connect that Jira project to the Compass component. You can select 1 or more projects for each component.
2. This will then sync any Jira issues that have the component field with the same name as the Compass component. 

It's on you to update the Jira component names in that project to match. 

Improvements coming soon may include something around:

• Automatic syncing -> specify you want Compass to 'take over' the component field to power the values there perhaps? But you would want to mix and add components that are not from Compass? 
• Better UI in Compass ->  allow for some simple filters/search. We are starting to see some use cases where teams are using it to show 'eng health' items specifically as a way to assess technical debt for each microservice. Other developers can 'raise' issues they have on them if they are relying on that service as an another example. So being able to 'create issue' from within Compass and it automatically adds it to the right project, with the component name, and maybe an issue type specified + some other details?
• Team managed vs. Company managed projects -> Currently, the component field is only on Company-managed projects. We need to think through how this will work for team-managed projects (where the component field does not exist yet) using either a custom field called components we can help setup for customers or maybe keying on something more universal like label within Jira to match the same name. We would have less control over the 'set of values' for that field though as label has a lot more use cases than just using it for components. But could be a simpler workaround while we build a longer-term solution.

We would love to hear more about your use case! Maybe we could talk about it via video call? I'm sure @Katie Silver on the team would be interested to hear more too. 

Cheers!

[Harald Sømnes Hanssen]

Will this product be available for on-premise?

[Taylor]

@Harald Sømnes Hanssen Hey there! 

We have no plans for offering a self-hosted version of Compass at this time. Cloud only. 

Would you mind sharing why you are looking for something on-premise/self-hosted? Do you have concerns about data residency, security, or something else? 

Would you need the product to work well with other self-hosted tools you might be using?

Cheers!

[Harald Sømnes Hanssen]

@Taylor Hey :)

My company is in the healthcare sector and therefore exposing internal services such as the development tools on the www, is problematic. 

Yeah, we're currently using JIRA, BitBucket, Confluence as well as TeamCity, Octopus Deploy and ProGet. All of these tools are self hosted and would require Compass to be able to utilize the APIs of the self-hosted tools.

[Dave Liao]

@Harald Sømnes Hanssen - any plans to integrate Compass with any Data Center products?

[Harald Sømnes Hanssen]

@Dave Liao I've asked around in the different teams we have, about what they think about Compass. And the immediate response is "this is something we need". So yeah, Compass would be for sure integrated with any Data Center products if applicable.

[Taylor]

@Harald Sømnes Hanssen Thank you for the extra context there and sharing the toolchain you are using - all makes sense on your need for self-hosting. It's also really nice to hear that your teams think Compass is something they would find useful.

If you have the time, I'd love to hop on a call to better understand the problems your team faces and why Compass would be a good fit - what aspects of it are a more immediate pain point vs. something you want to build towards to improve how your teams build & operate the software? I can also capture a bit more details to feedback into our internal discussion around supporting self-hosted/data centre products with Compass in a simple, intuitive, secure way.

If so, just contact me directly at tpechacek@atlassian.com and we can set something up early next week or in early January after the holidays.

In the meantime, we have invested a lot in the extensibility of the product. This means that everything you see in the product has a public API behind it you could use to wire up to your own tooling. See https://developer.atlassian.com/cloud/compass/ and https://developer.atlassian.com/cloud/compass/integrations/graphql-api-toolkit/ specifically around the API. 

I understand things like Git being self-hosted, but what data would you store in Compass that is sensitive enough to have the same standards. A lot of the information can be pushed into Compass (self-hosted -> cloud) which is a secure one-way method of communication. Then, that provides the reference points to build a developer experience and component catalog around with links out to the respective internal services, where a user would have to auth to see anything of substance.

I would love to hear a few use cases of how you want to use Compass more specifically, and which tools would need to be involved. We could take that and brainstorm to see if/how we might support that today or in the near future.

Cheers!

[Adam]

@Kelvin Yap  - Thanks for the updates.

a few questions:

1. Activities: You mentioned new sources, such as GitHub, are coming. Is there a thought to support the same deployments and environment detection used in Jira Softwares Deployments screen. Specifically, where we can send build and deployment info within jenkins files.
2. Services: While you may not being willing to say so, is the idea that at some point Compass defined services replace or add to those listed in JSM (currently from OpsGenie)?
   • The additions and future features around deployments, incidents, etc. seem to say yes. My team is nearing a point to start creating services in JSM/OpsGenie but we'd rather do everything in one place.
3. Perhaps assuming the answer to #2 is yes, might there be plans to source incidents, service availability, SLAs, etc. and then generate reports (PDFs, spreadsheets) to provide our clients with information to document achieved SLAs?

[andrew_pass]

Looking awesome, love the timeline stuff and can't wait for the ability to link in deployment information from 3rd tools such as Jenkins and display that here similar to JIRA and hopefully feed that easily into the score cards.

Any plans to ingest markdown files from a repository and render these within Compass for pure technical documentation that might be stored within a repo rather than Confluence?

Cheers :)