Zephyr reporting by sprints

Thomas Toth

Hi,

We'd like to use the Zephyr add-on together with Jira-Agile. While Zephyr enables us to create the test cases, I'm having troubles with the Test Summary and Reporting side in our Agile environment. While I can link Test Cases to Stories, the summary/reporting by sprint seems quite difficult.

How can I create a report which shows how many Test Cases are covering each user story and what their current/latest execution status is? This will help me determine which story is not covered by tests yet or needs more testing effort. Can I filter this by sprint?

How can I creat a report/summary which shows which/how many Test Cases are in the current sprint, based on the stories in the current sprint? This will help me oversee the work by sprint.

Given that we have >100s Test Cases, how can I easly create a Test Cycle based on the stories of the current iteration? Can I filter the Test Cases by sprint?

Can I create Test Cycles by Sprint and another parameter (e.g. component, team)? Having only one level of grouping makes it very difficult to create meaningful cycles as we have multiple teams working in every sprint. Ideally I would like to have a Test Cycle by Team for each Sprint, whereby the Sprint is the parameter from Jira-Agile. I find Version in an Agile environment not very useful.

How can I create a report to show which of the Test Cases are currently not linked to a story or requirement? This willl help me catch any tests which are not currently linked.

I know these are a lot of questions, however I was hoping that Zephyr will help me manage these aspects and at the moment I don't seem to be able to do any of these.

Thanks for any help.

Cheers,

Tom

G3

Hello Thomas...

Your questions are many and I have lengthy process configuration answers/discussion for you (and others). My response will generally be in the direction of addressing these points in your post...

• 'the summary/reporting by sprint seems quite difficult'
• 'How can I creat a report/summary which shows which/how many Test Cases are in the current sprint, based on the stories in the current sprint?'
• '...help me oversee the work by sprint'

Please bear with me if some of my response comes across a bit unclear. My coworkers have collaboratively fought the 'process and configuration legion' repeatedly, learning together and honing how we conduct work. It's an ever-growing experience.

The information this post contains would really be more suited for a live demo or process series (which I'm open to write). I'll can explain more if you (and others) have questions.

We run tight sprints in my workplace. Development team progress is tracked through JIRA Agile. 'Zephyr for JIRA' (Z4J) is used for manual test cases and cycles. Confluence, and other various Atlassian tools/plug-ins/add-ons are used for reporting, documentation, and customer interfacing.

As a QA/testing manager I need the ability to track all development sprint activity, monitor all testing being executed on the development items in each sprint (and other testing coverage), observe overall QA/testing team activity, provide reporting details for a number of metrics... and more.

I'll try to describe how I've got things set-up to provide visibility across both development and testing realms using JIRA Agile and Z4J. I'll focus this post on how JIRA Agile relates to development and testing groups where I work, and break it down in sections...

JIRA AGILE - Development VS. QA/Testing Board Configurations:

First, as mentioned, JIRA Agile is used by everyone in my shop... from development groups to testers. We all (devs/testers) have worked together to create numerous filters. Some of those filters help developers only see development issues in their Agile projects. This helps each project filter out Z4J test cases from their backlog and JIRA Agile views... then use one or more Agile board/s to capture their development items during sprints.

So, as a QA/test manager I ALSO use a custom JIRA Agile board I've configured for my testing team (ACTUALLY one board for Scrum, and one for Kanban, but that's a whole other discussion). My team's Agile board still ONLY contains development issues, BUT is configured in a way that makes viewing the issues relevant for QA/testers...

JIRA AGILE - QA/Testing Board SWIMLANES:

I'll start with 'Swimlanes'. I use multiple swimlanes in the QA Agile board. Each custom swimlane is for each project (or a group of collective sub-projects). All of my swimlanes use concise, custom filters I created to capture those development items for each project (or, again, a group of collective sub-projects). I DO NOT TRACK Z4J TEST CASES using JIRA Agile (there are numerous reasons for this, bigger discussion, maybe another time). I track Z4J test cases separately, by different means.

Creating multiple swimlanes enables my one-stop QA/Testing JIRA Agile board to have multiple collapsible/expandable sections. Those readily identified sections/swimlanes contain NOT ONLY those development tickets/issues respective to each unique project (or development group), BUT ALSO those development issues being worked in the active sprints (next section). This 'swimlane' setup makes EVERYTHING centrally located and rapidly viewable.

JIRA AGILE - QA/Testing Board ACTIVE SPRINTS:

When each project starts a sprint that active sprint name appears at the top of my QA Agile board. When I select that sprint my board efficiently filters down to JUST those development tickets related to that project (or that group's active sprint). This also leads to the same reason why the swimlanes and filters become important... for rapid, one-board, all project, quickly identifiable viewing.

JIRA AGILE - QA/Testing Board COLUMNS:

Next up are the 'Columns'. I have five colums set-up that are all based on the status of a development ticket with respect to testing responsibility. The columns are named (left to right) "Pre QA", "Almost QA", "In QA", "Blocked", "Done". The development tickets move across my team's agile board columns depending on the status (workflow) of that issue. The testers know what QA/testing actions they should be taking on a development issue, based on the column where that development issue is sitting.

Here's a process example... when a development issue (ex. 'Story') hits the "Almost QA" column... my team knows the issue has been mostly completed/developed, is being code reviewed, and the very next stage/column will be "In QA". So, to quickly back up, the team begins writing and linking Z4J test cases AS SOON AS the sprint becomes active on day one (when the development issues are visibly in the "Pre QA" column). By the time a development ticket hits the "Almost QA" column, test cases should already be prepared, final test case reviews conducted, steps should be finalized, etc... because that development ticket is coming to QA ownership for testing REALLY soon.

So what happens with the Z4J test issues (cases/executions) that QA/testing has prepared and linked to the development issues?

I can continue covering that in a future post if there's interest.

Cheers.