Zephyr for JIRA Cloud - Differentiate Test and Production defects

Himanshu Kansara August 11, 2015

Hi Guys,

 I am trying to establish metrics to differentiate defects logged during testing phase and those which are found in production. What should be the best way to accomplish this? Is adding a field to differentiate the bug in test and production is only way to achieve this? I would appreciate if someone respond with industry standard pointers. How do people achieve this?

Thanks,
Himanshu

3 answers

1 vote
Brian Ross August 12, 2015

Hello Himanshu,

I have been on both sides of development and production support and have seen this handled many different ways.

The solution is going to depend on your organization structure.  Is there a dedicated team of people fixing production issues as they occur or are the "build new stuff" developers having to take velocity hits on their sprints to resolve production issues?  How are the Production issues being reported?  What are the odds of the notes that were created to help fix the issue could get dragged into a legal review?

 

If you are a small shop where five people "do everything", you don't want to make their lives more complex by logging into separate systems.  I would just create a JIRA user called "Production", have the "Production User" be the creator of all production issues and then you can easily filter and generate information related to that person.  When you do this, you need to have a pretty good template that needs to be filled out to document what is going on.

 

If you have resources at your disposal, I would have a Production Engineering Support Team where all they do is fix production issues, back port production fixes into the development code base and lend assistance to the sprint velocity if they really need something to do.  This team would have a completely different defect tracking system than the developers working on sprints.  This tracking system would have an awesome (and easy) way to convert emails, comments, notes, issues, etc... into knowledge documents to help build a searchable knowledge exchange.  This will help people trouble shoot issues as they come up.  This way you are not fully dependent on the one person that has been with the company 12 years to fix all issues.  Anyone can search the KX to do tier one and two trouble shooting.  This will keep your Tier three people fresh to fix the super hard issues that no one thought of.

 

This will help keep the distractions down to a minimum to the devs (since they are not seeing all the crazy going on in Prod) and give your company a little legal room to work with if notes related to production issues get subpoena and you don't want to give up active work products or show protected work in your ticketing system the devs use.

 

The bigger issue you will have to work out is, how and who is going to determine the Priority and Severity of these production issues?  What is the Escalation path when stuff is not getting fixed?  What is your company standard for something to be a major issue?  What do you call major issue?  What is your expected turn around time for each of the Priority / Severity levels?  This sounds easy to define but everyone struggles with it and will abuse them (All "my" stuff is important and needs to be worked first so it is a Priority One issue!).

 

Another item to work out in any system, who is the first point of contact in reporting an issue?

 

No real answers here, just stuff you need to know the answers to before getting to far down this road.  My opinion is; don't track dev and prod issues in the same system.

 

 

 

0 votes
Brian Ross August 13, 2015

Base lining the information given so far:

 

There are 20 developers that can end up working on any issue found in the development environment or the production environment.

 

There are 5 QA analysts that test any of the work the 20 developers complete.

 

There is a “QA group” (that is separate from the 5 QA analysts) that start to review the production issues as they come in.

 

Customers are in the active dev environment reporting issues seen in dev and reporting enhancement requests seen in production.  I hope they are not side stepping the “QA group” to put critical, "system down" issues in the ticket que.

 

From a people stand point, you have the basics, but it will cause tension in the organization.

 

I am willing to speculate parts of the following are going to occur:

 

  • Sprint / iteration velocity is going to slow down.  It won’t matter if you earmark a certain percent of time to dev and prod support tasks.  A lot of dev tasks can’t stop their work on a dime and it takes a while to get going back up to speed.  When team members are switching back and forth between dev and prod environments, they become inefficient. 

 

  • Customers are going to get cranky that they have to wait to complete your current sprint to plan the next sprint to get their fixes coded and then deployed.  Even if you are “fixing it now”, my first sentence in my first point becomes more valid.

 

With the amount of people outlined, that is just going to be the way it is.  The only way to improve it with the people you have is to determine how much work is going into resolving production issues and come up with a way to determine how many FTE’s it will take to cover that.  Then a decision can be made if you want to move that many FTE’s into a role to only work on those Production items (it could be full time or rotational; each have pros and cons).

 

I would start my focus on the “QA group”, working on revising a process and procedure for them to work from.  The first thing I would outline to all parties involved is “All production issues go through the QA group”.  If the QA group can’t fix the issue with their triage, then they can start negotiating with the customer on how important the issue is.  Based on that discussion, the JIRA issue is created and a Priority and Severity is set.  A discussion can start on who will work it and when it will be done. 

 

This goes back to your original question on what to do in JIRA.  That is a company decision based on the knowledge and capabilities of the people that are currently in place. 

 

It sounds like JIRA is going to be the one-size-fits-all vehicle for the team, so how to configure it.  You could add additional fields to the issues template, you could add additional issues to select from, you can modify the items in the priorities field to indicate DEV vs PROD priority rankings.  Based on the decisions you come up with here, will help drive the rest of your procedures on who does what and what time frames everyone works in.

 

That last sentence can get side tracked into a whole other list of what to consider and do; but that is going out of the scope of the original question.

 

This is just peeling back the first layer of items to consider when using one ticketing system to cover two completely different jobs.

 

Hope this helps your discussion for your team on how to proceed.

0 votes
Himanshu Kansara August 13, 2015

Thanks Brian for detailed response.  This helped us to get better insight. Yes, I agree with you that there is no real answer.

We have a same team of developer around 20 working on new as well as production issues. 5 testers working to test the development items. Customer is using the same platform to log issues found in development as well as production.

There is a dedicated JIRA service desk, here existing QA Team work as Level 1 support for all issues reported by customer. The service desk here serves as a platform to address questions about understanding, if any change is required in data or if there is an issue due to down time of the issue.  If level 1 confirms that this is an issue that require action from development team, then depending upon the nature of the issue it creates Task, Bug or Improvement.

Since, the existing environment is accessible to everyone, this makes the situation tricky.

I hope this will give you better visibility of the problem I have.

 

Thanks again for your reply.

 

 

Suggest an answer

Log in or Sign up to answer