Jira (Spring?) cleaning

As @Bridget mentioned - it is spring in Australia, time for house cleaning. Well, we, Europeans are half a year behind or forward (depends on perspective), but it does not matter. Rules of the game are the same.

So, let's talk on Jira cleaning

First things first - why would you do the cleaning? There might be several factors for, but two are the most important:

  • governance - within it I would include security, unification, change control, program/project control, users and admins review etc.
  • performance - within it I would include availability and capacity, utility of tools and so on

Your focus on what is more important might fluctuate, depending on actual, current needs and goals. Still, both need to be taken into consideration. Why? Proper governance cleaning will allow You to keep/improve performance measures, while proper performance based cleaning might affect governance by changing known, approved ways of working.

I did my time for one of the project running huge performance analysis for Jira DC instance. Instance performance was terrible, governance even worse. And first advise given was to clean it. Get rid of 75-80% of 1200+ workflows, 3000+ custom Fields, and a lot of other stuff. But to clean it wisely - was another challenge. We have spent 9 weeks on analysing thoroughly all aspects - from underlying infrastructure through architecture, technical and business configuration, ending up at usage patterns. For the purpose of this article I will leave behind Architecture and technical part, and focus on business config cleaning and usage patterns.

Business config cleaning

As You might know from Atlassian documentation there are multiple factors You need to maintain clean:

  • Workflows & Workflow schemes
  • Custom Fields, contexts, configuration schemes
  • Screens and screen schemes (including Issue Types screen schemes)
  • Permission schemes
  • Notification Schemes
  • Security Schemes
  • Priorities, Resolutions, Statuses etc.
  • Shared objects - JQL filters, filter subscriptions, dashboards, gadgets

What does it mean "keep it clean"? By my means - keep it as functional as You need, but structured, unified as possible, re-usable. Jira gives You great capability, as the open platform, to make every project uniquely configured. But gives you also even more important option - to reutilise and unify ways of working.

If I recall example above - instance with 1200+ workflows - 800+ of them were THE SAME workflow. used in different projects. Exactly the same, out of the box workflow. By using shared configuration You would downgrade volumetric of active workflows by 2/3s. Governance wise, and performance wise. 

Keeping the same example - avoid and correct need from admins, project admins, change requestors to leave their unique "legacy". In 3000+ custom fields in mentioned instance around 120 were different fields for "results description": analysis results, test results, QA results, UAT results, etc. And I don't mention split between expected and actual results. I mean lack of unified naming related to whole business configuration (there were different issue types for each of these activities, different story - also extensive too much). 

I will get back here to my idea that with systems like Jira - application administrator role is not enough. Either You, as admin, are capable of solution architecture design, or You need solution architect to discuss with key users and make decisions on proper use of business features. Admin will simply follow the request to add workflow, project, Custom Field, architect will question, analyse, justify everything accordingly to governance and to achieve performance.

There is a lot more of course. Assuming that documentation for mentioned project took almost 300 pages, this article is to short to go through every case. But to give You some more insight about important, tricky, or unexpected findings:

  • 1800+ projects and 1800+ workflow/Field/permission schemes (little less - 70% of that - notifications schemes and screen schemes, and around 20% of that volume - security schemes) - almost every project had unique set - although after analysing - again - 70%+ had exactly the same configuration, but multiplied. 
  • Longest workflow - 51 statuses/steps, with 47 allowing transition from all others, in total 10 WF with 40 + (!!!) statuses
  • In total - around 300 text multiline fields - about 50% could be cut off based on unified naming conventions, another 30% could be switched to any select/checkbox/radio (and here is the tip - text fields have biggest impact on performance, as they are not indexed, but use full text search, with single ones - does not matter, but at large scale...)
  • Natural, organic growth of Custom Fields volume per month at level of 20-30 new fields monthly due to misuse - set of fields was created for each biweekly sprint in one project (field names were referring to Sprint name, so every sprint had the same set of fields/data, but differently named) - that gives new 240-360 CF per year.
  • 100+ Jira administrators with full rights to create new objects.

Bottomline - It was a mess to maintain, pain in the back for performance, and definitely not the instance You can use any governance on. By the end of the project - around 70% of findings were transferred into cleaning actions, limiting business config to around:

  • 100 Issue types
  • 150 active workflows
  • 30 Workflow schemes
  • 500 Custom Fields
  • 20 unified permission schemes based on roles
  • 10 notification schemes
  • limitation of rights to shared objects creation and management
  • and so on

Let's face it - I was not totally happy - I would cut it off more (especially WFs and CFs). But then it is still balancing between governance and performance on the one side, and utility on the other. 

Usage Patterns - important but often forgotten

What is often forgotten - is usage patterns. Partly it relates to previous paragraph - way of working within a teams and departments, and use of Jira features (especially shared objects).

There will be a number of things to consider:

  • natural and occasional growth of projects/issues
  • natural and occasional growth of users volumes

In all cases natural - related to extension of Jira into new business areas, occasional e.g. due to merges, acquisitions etc.

  • Business and technical roles mappings
  • Related to above user profiles (what is used, how, when, for what reason)

And here again - You need to avoid situations where You loose control over it (100+ admins - remember?). Growths can be anticipated, analysed, and adopted, if only there is a good communication and understanding of Jira's role in business processes (critical or non critical system, and core, enabling, enhancing role - I would say in most of cases it is not critical, but enabling - means - secondary just to critical systems). That means that governance over Jira health and performance is not one-off, neither occasional task, but ongoing process, requiring owner, rules, and enough effort.

Then - usage pattern themselves - they are transferable into roles, schemes, specific rights and so on. It has to follow identity management policies - and even in agile, distributed organisations - need to follow some ground rules, gathered in governance practice. You should keep standards, and base on standards for project roles and technical roles. Exceptions - as per name - are one-off setups for very specific, business wise needs. Let's make not mistake here - none of business mapping is so unique that couldn't be just variable from standardised way of working. If someone states otherwise - usually it is due to some tendency to run micromanagement, or shadowing the responsibilities. 

Micromanagement trends usually will result in will to keep projects totally under control of one-two persons, giving them all, including Jira admin rights, to do anything they want, regardless of sense. 

"Collective responsibility" trend on the other hand - drives to giving exactly the same rights (often, including Jira admin).

Both results in lack control over changes to specific projects. And then Your Jira holds 10 of them - it is not the issue. However, when it holds 1000... It is.

3 top tips for successful cleaning of Jira

Ending this story I would like to share with you 3 top tips I assume are the most important. And even if You consider them truisms, memorise them, and recall with every Jira cleaning operation:

  1. Know Your instance - on daily basis, understand patterns, changes, know the reasons and habits of your users and admins
  2. Re-utilise as much as possible inline with business needs
  3. Keep ownership of configuration in strong, knowledgeable hands - with tech, and business architecture acumen.

Then You should be rather safe than worry.

 

 

12 comments

Bridget
Community Manager
Community Managers are Atlassian Team members who specifically run and moderate Atlassian communities. Feel free to say hello!
October 22, 2019

@PJ Wysota this Article is AMAZING! 

PJ Wysota
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
October 22, 2019

Oh, thanks!

Kat Warner
Marketplace Partner
Marketplace Partners provide apps and integrations available on the Atlassian Marketplace that extend the power of Atlassian products.
October 22, 2019

Great article @PJ Wysota . I feel much better about the metaphoric dust bunnies in the Jira system I use now.

Image result for dust bunny gif

Shashi October 24, 2019

@PJ Wysota ,

Did you try to automate to clean up any stuff or was it manually done?

Any links to the article which you visited for clean up would be much appreciated

Shashi October 24, 2019

How did you measure the performance? How did you collect those metrics?

PJ Wysota
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
October 28, 2019

Hi @Shashi

That is in fact a lot of questions in one-two questions.

1. Performance references from Atlassian - a number of them - main I used in my work:

https://developer.atlassian.com/platform/marketplace/dc-apps-performance-and-scale-testing/

https://confluence.atlassian.com/adminjiraserver/performance-and-scale-testing-965568707.html and similar for previous versions

https://confluence.atlassian.com/enterprise/jira-data-center-sample-deployment-and-monitoring-strategy-953148860.html

https://confluence.atlassian.com/adminjiraserver/performance-and-scaling-965568705.html

https://confluence.atlassian.com/enterprise/managing-custom-fields-in-jira-effectively-945523781.html

https://community.atlassian.com/t5/Jira-questions/Custom-fields-and-Jira-Performance/qaq-p/81234

https://confluence.atlassian.com/jirakb/troubleshoot-performance-issues-in-jira-server-336169888.html

https://confluence.atlassian.com/jirakb/common-causes-for-jira-server-crashes-and-performance-issues-203394749.html

https://atlasauthority.com/blog/how-indexing-postgres-and-flushing-caches-impacts-jira-performance/

 

2. As for performance tests:

- start with JMelody or alternative

- use tools like: https://community.developer.atlassian.com/t/jira-performance-tests-jpt-beta-is-now-available/23708

- use references like: 

https://confluence.atlassian.com/enterprise/atlassian-performance-testing-examples-946041214.html

https://www2.valiantys.com/en/jira-performance-at-scale

https://confluence.spartez.com/display/SPARTEZ/Testing+a+Million+Issue+JIRA+-+The+Verdict

3. Last but not least is maintaining performance/availability/capacity policies. And this governance practice is the toughest part, as You need to apply it from committed application management level. In simplest words it would cover:

A. Setting up plan/baseline (how, by whom Jira should be used as per designed implementation) + capacity/performance measures for this scenario 

B. If differs, actual/current use scenarios (how, by whom) + capacity/performance measures + gap analysis against planned/baseline

C. Setting up monitoring on tech resources to measure fluctuations and changes on server/application/database levels

D. Maintaining any changes to Jira use patterns (organic growth of projects/issues/shared components, additional projects/business units, volumetric additions of served users/customers, resource consuming add-ons or volumetric config changes) under restrictive change management process including performance testing for potential scenarios 

E. Maintaining system/app components - as usually newer versions provide performance improvements

F. Upgrading system, parametrisation of JVM, scaling up Jira or migrating to DC

NotJoel
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
January 8, 2020

Awesome writeup, keep em coming!

Jess Seitz
Community Manager
Community Managers are Atlassian Team members who specifically run and moderate Atlassian communities. Feel free to say hello!
January 21, 2020

Fantastic article @PJ WysotaWow 51 steps in one workflow made my jaw drop, really interesting to hear how you weigh balancing governance with utility. What performance improvements were you able to see from this deep cleaning? I'm also curious if you were able to utilize project and issue archiving to clean up outdated issues? 

civa7649 September 11, 2020

I realize I am commenting nearly a year after this article was written but its content is still true and helpful.  I am in the midst of my own spring (fall?) cleaning and am finding similar trends.  From what I can see, we have many many workflows especially, that appear to be the SAME default workflow but with the new project's key in front.    Is this a default of JIRA?  Why does it make a copy of the default workflow with a name change?  

 

So I am tempted to go repoint the workflows to the original default and then delete the 'copy.'  But how do I know for SURE that it hasn't been changed.  Is it really a manual eyeballing of the workflow to confirm ?   Thanks in advance to anyone who can advise.  

PJ Wysota
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
September 18, 2020

Well, eyeballing would be the only sure thing. Or Insight for Jira with Jira environment add-on. 

But then, IMHO - I would repoint to original, make standard and announce, that it is a way untill someone has good justification to make amendments.

Jousef October 6, 2020

Hey PJ, awesome article, we are using it frequently!

Do you have any tips for cleaning up Confluence as well?
giphy

We have almost 200 spaces, probably a quarter of them aren't being used anymore or redundant in some way. What would be a good approach to find out which spaces should be archived?

Would love to hear your or anyone's input!

Thanks and all the best from cloudy Berlin,
Jousef

Gonchik Tsymzhitov
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 20, 2021

Hi! 

Thanks for use cases I am going to enroll it here

https://github.com/gonchik/cleanup-scripts

Comment

Log in or Sign up to comment
TAGS
AUG Leaders

Atlassian Community Events