There are many attributes I'd like to track using Compass Health Scorecards, but which are not currently possible.
For example, we're in the process of importing all of our services into Compass from gitlab. We've never graded the criticality of our services on such a granular level before, so while ideally we'd like to grade each one immediately, in practice we can't do that, it will take time.
However when we bring them in, the default Tier given each component is Tier 4.
I needed a way of tracking whether a component had been graded as Tier 4 legitimately, or if it had not actually been graded at all. The only solution I can find within Compass today, is a pretty ugly regex attribute. Basically when my team grade the criticality of a component, they set the Tier level and then add a Documentation link to a specific URL. If the regex in the scorecard sees that specific link on the component, it passes the healthcheck. This is a horrible, hacky, method.
A better solution, (which would be useful in multiple ways), would be for Compass to allow a new type of boolean Criterion, available for all Scorecards, which is called something like "Yes/No". You should be allowed to name that criterion whatever you like, and to decide on the default value, either Yes, or No.
That way I could have an attribute like "Criticality Graded?", each component would default to 'No', and my team could easily work their way through the list, changing them to Yes's as they go. Or, I could have an attribute named "DR Playbook Tested", with a Yes/No answer as to whether a playbook is documented and a dummy run of it performed, or not.
(Additionally, just because 'Tier' is the accepted standard naming convention within Atlassian, it would be nicer if companies could change things to suit their own practices. For my current workplace, Minor, Major, Significant, and Critical make more sense, but I can't use that within Compass.)
