Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,463,085
Community Members
 
Community Events
176
Community Groups

Reports based on test steps

Are there plans to include the option to look at the Zephyr for JIRA gadgets based on test steps, as opposed to just tests?

We'd very much like to see a burndown chart that included all test steps in a cycle, for example.

16 answers

1 accepted

1 vote
Answer accepted
Daniel Rising Star Feb 28, 2013

Hi Erik,

I have added this and your description to our feature request list. Thank you for your detail.

Regards,

Daniel

Thank you, Daniel.

Hi Eric,

Is there any update on this request. Like all the other users below , we work on a complex environment with each scenario having more then 400 steps spread across multiple teams. If we can track the execution status at step level we know which team currently owns the execution and report in terms in all over step progress in a day. As a 400 steps scenario may take several days to complete , not having the capability to step level reporting , will show the same status ( In progress) for all these days. 

Like # people like this

Hi Daniel

 

Has this feature been created? We would like to report on our test steps as well? 

Thanks

Same situation here.  We want to report metrics at the test step level.  Will this be new feature or is there a workaround using ad-ons?

Hi,

Are there any updates on this ticket?

1 vote

Hi Erik,

Ok, I understand your reasoning. But I am curious yet on what you do with testcase results.

For example, if you have 15 testcases in a cycle and 200 steps, what does your chart show? Are there only 200 step values in the chart and no testcase results in the chart? Or do you also want the testcase results and someone have each step result included in a family of a single testcase's results?

Note, I think no matter which way we are going here, this is a feature request. But understanding your end goal can allow us to accuractly consider the feature request.

Kind Regards,

Chris

Thanks, Chris- I really appreciate the attention to detail. I suppose the absolute best case scenario would be to be view both - a chart that shows the 200 steps or a chart that shows the 15 tests, or possibly even a chart that shows a combination view (tests and test steps completed vs tests and test steps remaining).

The main goal, using your example, is to be able to take into account situations when you have 15 test issues, 10 of which have 5 test steps each (50 of 200) and 5 of which have 30 steps each (150 of 200). So when you look at test burndown with the current gadget, if the 10 tests with 5 test steps each are complete, you'd think you were 66.66% done with testing. But that is misleading. If you base it on test steps, then you are 25% done with testing - a very significant difference.

The reality so far in our usage of Zephyr for JIRA is that the number of test steps per test case varies quite a bit, so only looking at stats based on testcase completion doesn't tell the whole story.

Hope this makes sense, and again thanks for being so responsive and eager to improve the product for the users!

Erik

What do you think, Chris?

It's been several years. Has this problem not been solved yet?

Zephyr can't calculate and report conveniently at the test steps level ?? So disappointing! I want to give Zephyr a negative score. 

Hi Troy,

We hear you here at Deviniti. We've had numerous requests to add this to our tool too.

In case you are open to considering alternatives, we have just released TestFLO 8.6.0 - Test Management for Jira. We have a Test Plan Execution report where you can now choose if it's precision should be based on test case statuses, or step statuses:

image_2021-02-25_130921.png

Here's the documentation regarding this report:
https://deviniti.com/support/addon/server/testflo-86/latest/test-plan-execution-report/

Just be aware TestFLO is only available for Jira Server and Data Center.

Regards,
Radek Cichocki
Product Manager @ Deviniti

Like Nina Zaradniak likes this

@Radoslaw Cichocki _Deviniti_ 

it is great that testFlo handles step report. is there a way to migrate existing zephyr test cases, cycles to testFlo?

Like Troy Wang likes this

Hi @jun

Just two words of introduction. In TestFLO we have two concepts:

You can create TCs from TCTs, so TCs are executable copies of TCTs. TCs are groupped in Test Plans (technically TCs are sub-tasks of Test Plans).

At this moment, you can import Zephyr's test cases as our test case templates using our CSV importer. You would need to export your test cases from Zephyr, and trtansform the resulting file into the format acceptable by our CSV TCT importer.

If importing just the test specifications is enough for you, most likely you will be able to do it on your own (or we can help).

Regarding Zephyr's test executions, you will need our help if you need to migrate these too. For some customers, migrating just the test specifications, without all the execution history is sufficient.

We are considering creating a dedicated importer from Zephyr to automate the whole process, including test executions.

I would like to encourage you to:

  1. Consider if importing just the test specifications is enough for you - you can have both tools for a moment, so you would start all your new test efforts in TestFLO while finishing up the old efforts in Zephyr. If so, you may be able to comlpete the process on your own (CSV TCT importer).
  2. Try out TestFLO as a trial version and see how it compares
  3. Contact our support team with any requests for help or with any questions you may have, they're a bunch of energetic people who love helping people like you. You can also ask them for a product demo, so they'll walk you through.

Regards,
Radek Cichocki

0 votes

Hello everybody,

If you'd like to consider an alternative, then TestFLO - Test Management for Jira, has a progress bar that shows the test case progress based on the status of individual steps.

You can see it on the Test Plan for all Test Cases within it:TP TC steps progress.png

You can also see it on the test case itself.

Regards,
Radek Cichocki

Disclaimer: I am the product manager of TestFLO.

Hi all,

 

I am also looking to report based on test steps, I think it is a necessary and common sensical requirement , reporting at a scenario level alone would not capture the effort the testers have to put in for testing. It will be hard to substantiate that not a lot of testing was done, as the scenario movement would be minimal, but a lot of testing would have been done considering each scenario has many test steps. Is there an update to this ticket?

We are having the same issue.  Would be nice if there was a solution other than "here's a totally different product we'd like to sell you on"

Any updates on this ticket please?

For those of you interested, TestFlo do provide some sort of information about test steps (number of steps, progression based on test steps in Test Plan). That's something we will investigate in our team.

Hi @sebastien legere ,

I'm TestFLO's product manager, and thank you for bringing this up.
Yes, we do have steps-based information on test cases, you can read more about it here (link).

If you'd like to play around with this, here's our live demo environment.
Once you click that link and go there, you can open any test case, e.g. OSA-1465, and find out how it behaves.

Regards,
Radek Cichocki

Like sebastien legere likes this

Hi @sebastien legere ,

Just so you know, yesterday we have released TestFLO 8.6.0 allowing our Test Plan Execution report to based on test case steps. The reporting precision is now as accurate as it can get.

(Documentation link)

Regards,
Radek Cichocki

Any update on this item? I'm looking for a way to get a report at a test step level in Zephyr.

Any update on this item. This is very important to integrate Zephyr with CI/CD and to create a test execution burn down chart basis on test step execution. Please share the update on this as soon as possible. 

 

Or Can anyone suggest any alternatives to this?

Same situation here... I'm looking for a way to get a report at a test step level. 

Any idea how to get this info?

any updates from Jira on this?

Hello,

Any progress on this request please?

I have similar needs. For example, we are doing end to end testing involving three check points to verify data flow in different systems. So, if my test case contain three steps for these three check points, it is important for us know any issues at a particular step level.

This report will help us see overall picture at different check points.

 

Regards,

Priyanka.

Hey Folks,

Thanks for all the input – is there any update or possible workaround for this…”Reporting by Test Steps”?

My request/requirement is similar if not the exactly same, we need the overall flexibility to report by Test Cycle the execution status by either “Test” (which we have currently) or “Test Steps” which doesn’t appear to be possible.

Any thoughts/suggestions would be appreciated.

Thanks,

Jay

Hi,

Do you have any updates regarding this case? It would be great to have burndown/report with test steps information.

Br,

Paulina

I would also like to see burndown/report information using test steps (as well as test cases) for the same reason as above - a test case with 300 steps is counted in exactly the same way as a test case with 2 steps in a burndown chart.  Using steps would reflect velocity more accuratly.

0 votes

Hi Eric,

How would that chart look? Can you explain it so we could understand your use case and desired result?

Kind Regards,
Chris

Hi Chris-

Thanks for the quick response! Essentially we just want a look at progress in terms of total test steps in a cycle instead of tests in a cycle. It would be great to be able to view the burndown chart based on test steps or an execution chart based on test steps. Basically just the option to be able to look at the entire testing effort in terms of all test steps in all tests in a cycle, instead of just all tests in a cycle.

The reason we want this is because looking at only tests can be a bit misleading as tests can vary widely in the number of test steps each one contains. So the ability to look at things at a test step level would give a better idea of how things are progressing.

Thanks,

Erik

Suggest an answer

Log in or Sign up to answer
TAGS

Atlassian Community Events