Simple Yes/No Field in Scorecarsd

[Mario Cappellano]

There are many attributes I'd like to track using Compass Health Scorecards, but which are not currently possible.

For example, we're in the process of importing all of our services into Compass from gitlab. We've never graded the criticality of our services on such a granular level before, so while ideally we'd like to grade each one immediately, in practice we can't do that, it will take time.

However when we bring them in, the default Tier given each component is Tier 4.

I needed a way of tracking whether a component had been graded as Tier 4 legitimately, or if it had not actually been graded at all. The only solution I can find within Compass today, is a pretty ugly regex attribute. Basically when my team grade the criticality of a component, they set the Tier level and then add a Documentation link to a specific URL. If the regex in the scorecard sees that specific link on the component, it passes the healthcheck. This is a horrible, hacky, method.

A better solution, (which would be useful in multiple ways), would be for Compass to allow a new type of boolean Criterion, available for all Scorecards, which is called something like "Yes/No". You should be allowed to name that criterion whatever you like, and to decide on the default value, either Yes, or No.

That way I could have an attribute like "Criticality Graded?", each component would default to 'No', and my team could easily work their way through the list, changing them to Yes's as they go. Or, I could have an attribute named "DR Playbook Tested", with a Yes/No answer as to whether a playbook is documented and a dummy run of it performed, or not.

(Additionally, just because 'Tier' is the accepted standard naming convention within Atlassian, it would be nicer if companies could change things to suit their own practices. For my current workplace, Minor, Major, Significant, and Critical make more sense, but I can't use that within Compass.)

[Alastair Wilkes · Accepted solution]

Hey Mario! PM for Scorecards here. Thanks for your thoughtful feedback.

Good news - you can use the Checkbox-type custom field for this, which is equivalent to True/False. You can evaluate the field in Scorecards and check for a true value.

However, there is a minor quirk I want you to be aware of: if the custom checkbox field has not been set yet for a component (i.e., it hasn't been checked), the UI will display "criteria status: error" for that component. To resolve the error state, you need to check and then uncheck the field on the component to properly set the value as 'false'. However, you don't have to resolve the error state; the scorecard will still properly be marked as "Failed," so the end result is correct. (We need to improve this!) Edit June 10 2024 - this is fixed.

Thanks for the feedback about Tiers, too - being able to 1) automatically set a null value for Tier when you bring in a component and 2) rename Tier labels/add more tiers would be useful. I've logged that feedback.

Alastair

[Mario Cappellano]

Hi Alastair,

Ahh, excellent, got it working, great to see you guys have added all these new custom options as well now, good to see new features rolling out. 👍

[Shane Chapman]

Hey @Alastair Wilkes ,

I've followed the doc Add a custom field as a scorecard criterion but my custom field is not showing up.  I added a "Blue / Green Enabled" checkbox field type.  It shows up in the components but not the scorecard.  However, I can see the custom metrics I created in the scorecard.

Here is how my custom field is setup:

• Custom field - Blue / Green Enabled
• Customer field type - Checkbox
• Component types - All component Types

Thanks,
Shane

[Alastair Wilkes]

Hi @Shane Chapman - I think I just reproduced what you're seeing. I think there might be a bug here; try changing the "Application model" to Automatic and then selecting some component types. The custom field should appear in the dropdown then. I'll work with the team to fix that.

[Shane Chapman]

That's it!  I'm excited to implement the full capabilities of Compass.

Thanks for the quick help, @Alastair Wilkes!

[Mario Cappellano]

Hi @Alastair Wilkes , is there a date for a fix to this one? I did have my custom components available in scorecards, but now I can't manage to get them to appear. I've tried the above workaround to no avail.

[Alastair Wilkes]

Hi @Mario Cappellano! We deployed a fix for this today. Let me know if they are still not showing up.

[Mario Cappellano]

No change here - appears on the component but not available in the scorecards - but I'm checking only a couple hours post your comment, maybe it will take a while for my tenancy to get the update? 🤷‍♂️

[Alastair Wilkes]

Oh, which kind of custom field is it? I ask because scorecards currently don't support dropdown custom fields and user custom fields yet, so that might be the reason. (We're working on this.)

Another reason might be that the component types on the scorecard application filters don't match the component types for the custom field. But you said you switched to manual and that didn't fix it, so it's probably not that.

If it's not either of these, feel free to open a support ticket so we can look more closely. Thanks!