You're on your way to the next level! Join the Kudos program to earn points and save your progress.
Level 1: Seed
25 / 150 points
1 badge earned
Challenges come and go, but your rewards stay with you. Do more to earn more!
What goes around comes around! Share the love by gifting kudos to your peers.
Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!
Join now to unlock these features and more
Are there plans to include the option to look at the Zephyr for JIRA gadgets based on test steps, as opposed to just tests?
We'd very much like to see a burndown chart that included all test steps in a cycle, for example.
Is there any update on this request. Like all the other users below , we work on a complex environment with each scenario having more then 400 steps spread across multiple teams. If we can track the execution status at step level we know which team currently owns the execution and report in terms in all over step progress in a day. As a 400 steps scenario may take several days to complete , not having the capability to step level reporting , will show the same status ( In progress) for all these days.
Ok, I understand your reasoning. But I am curious yet on what you do with testcase results.
For example, if you have 15 testcases in a cycle and 200 steps, what does your chart show? Are there only 200 step values in the chart and no testcase results in the chart? Or do you also want the testcase results and someone have each step result included in a family of a single testcase's results?
Note, I think no matter which way we are going here, this is a feature request. But understanding your end goal can allow us to accuractly consider the feature request.
Thanks, Chris- I really appreciate the attention to detail. I suppose the absolute best case scenario would be to be view both - a chart that shows the 200 steps or a chart that shows the 15 tests, or possibly even a chart that shows a combination view (tests and test steps completed vs tests and test steps remaining).
The main goal, using your example, is to be able to take into account situations when you have 15 test issues, 10 of which have 5 test steps each (50 of 200) and 5 of which have 30 steps each (150 of 200). So when you look at test burndown with the current gadget, if the 10 tests with 5 test steps each are complete, you'd think you were 66.66% done with testing. But that is misleading. If you base it on test steps, then you are 25% done with testing - a very significant difference.
The reality so far in our usage of Zephyr for JIRA is that the number of test steps per test case varies quite a bit, so only looking at stats based on testcase completion doesn't tell the whole story.
Hope this makes sense, and again thanks for being so responsive and eager to improve the product for the users!
We hear you here at Deviniti. We've had numerous requests to add this to our tool too.
In case you are open to considering alternatives, we have just released TestFLO 8.6.0 - Test Management for Jira. We have a Test Plan Execution report where you can now choose if it's precision should be based on test case statuses, or step statuses:
Here's the documentation regarding this report:
Just be aware TestFLO is only available for Jira Server and Data Center.
Product Manager @ Deviniti
Just two words of introduction. In TestFLO we have two concepts:
You can create TCs from TCTs, so TCs are executable copies of TCTs. TCs are groupped in Test Plans (technically TCs are sub-tasks of Test Plans).
At this moment, you can import Zephyr's test cases as our test case templates using our CSV importer. You would need to export your test cases from Zephyr, and trtansform the resulting file into the format acceptable by our CSV TCT importer.
If importing just the test specifications is enough for you, most likely you will be able to do it on your own (or we can help).
Regarding Zephyr's test executions, you will need our help if you need to migrate these too. For some customers, migrating just the test specifications, without all the execution history is sufficient.
We are considering creating a dedicated importer from Zephyr to automate the whole process, including test executions.
I would like to encourage you to:
If you'd like to consider an alternative, then TestFLO - Test Management for Jira, has a progress bar that shows the test case progress based on the status of individual steps.
You can see it on the Test Plan for all Test Cases within it:
You can also see it on the test case itself.
Disclaimer: I am the product manager of TestFLO.
I am also looking to report based on test steps, I think it is a necessary and common sensical requirement , reporting at a scenario level alone would not capture the effort the testers have to put in for testing. It will be hard to substantiate that not a lot of testing was done, as the scenario movement would be minimal, but a lot of testing would have been done considering each scenario has many test steps. Is there an update to this ticket?
Hi @sebastien legere ,
I'm TestFLO's product manager, and thank you for bringing this up.
Yes, we do have steps-based information on test cases, you can read more about it here (link).
Any progress on this request please?
I have similar needs. For example, we are doing end to end testing involving three check points to verify data flow in different systems. So, if my test case contain three steps for these three check points, it is important for us know any issues at a particular step level.
This report will help us see overall picture at different check points.
Thanks for all the input – is there any update or possible workaround for this…”Reporting by Test Steps”?
My request/requirement is similar if not the exactly same, we need the overall flexibility to report by Test Cycle the execution status by either “Test” (which we have currently) or “Test Steps” which doesn’t appear to be possible.
Any thoughts/suggestions would be appreciated.
I would also like to see burndown/report information using test steps (as well as test cases) for the same reason as above - a test case with 300 steps is counted in exactly the same way as a test case with 2 steps in a burndown chart. Using steps would reflect velocity more accuratly.
Thanks for the quick response! Essentially we just want a look at progress in terms of total test steps in a cycle instead of tests in a cycle. It would be great to be able to view the burndown chart based on test steps or an execution chart based on test steps. Basically just the option to be able to look at the entire testing effort in terms of all test steps in all tests in a cycle, instead of just all tests in a cycle.
The reason we want this is because looking at only tests can be a bit misleading as tests can vary widely in the number of test steps each one contains. So the ability to look at things at a test step level would give a better idea of how things are progressing.