it seems the testNG Results parser does not properly recognize tests that are executed through a data provider (or at least not in the way i'd expect).
In example, we have a test that is executed 100 times with different data. For 20 of those cases, the test fails. Bamboo will then show this as 1 failed test containing a single failed instance of this test method and one successful test. This omits both the 19 other failed tests and the 79 other successful tests, so i need to rely on the html report completely. I suspect this has to do with each of the 100 tests named exactly the same (as usual with data provider).
Is anyone familiar with this? Is this the intended state? Does anyone know how to display (in this case) all 100 test results in the frontend, instead of only one or two?
Yes, this relates to BAM-17738. We parse the xml file and record success or failure for a specific test using the "name" field as the identifier. I believe in your case the "name" for all your test cases are identical and hence Bamboo only displays the one entry.
I've added this community entry to the bug to help raise its visibility with the developers.
Atlassian support - Bamboo
G’day Community! As we gear up to introduce Bamboo Data Center to the world, we wanted to make sure that we shared a bit more about Bamboo, the product. Our team has put together an overview video ...
Connect with like-minded Atlassian users at free events near you!Find an event
Connect with like-minded Atlassian users at free events near you!
Unfortunately there are no Community Events near you at the moment.Host an event
You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events