Multiple execution of the same test with different random seeds

In the SystemVerilog programming language, the same tests are run multiple times but with different seeds in the same build task. The same test could be run hundreds of times with a different seed each time in the same task. What report format makes the most sense? I have been using JUnit XML, but I have to trick the XML to make each test name different to get one report per seed. Is there a better test report format to use in this case?

2 answers

1 accepted

Looks like https://jira.atlassian.com/browse/BAM-2627answers the question:

"Bamboo 4.4 shipped with a specific TestNG result parsing task. This task is capable of parsing many tests with the same name. If you switch your plans to use this task, all your tests should be reported correctly."

Edit: I will accept this answer for now, but I have not tried it yet.

I'd aggregate passing testing and list failed tests like you are, unless the test failed 100% of the time then you could aggregate the failure as well.

Suggest an answer

Log in or Sign up to answer
Atlassian Community Anniversary

Happy Anniversary, Atlassian Community!

This community is celebrating its one-year anniversary and Atlassian co-founder Mike Cannon-Brookes has all the feels.

Read more
Community showcase
Renan Battaglin
Published May 18, 2017 in Bamboo

FAQ: How to Upgrade Bamboo Server

Bamboo 5.9 will no longer be supported after June 12, 2017. What does this mean? As part of our End of Life policy, Atlassian supports major versions for two years after the first major iteratio...

1,398 views 0 5
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you