TestNG Results with dataProvider

[René Habermann]

Hi,

it seems the testNG Results parser does not properly recognize tests that are executed through a data provider (or at least not in the way i'd expect).

In example, we have a test that is executed 100 times with different data. For 20 of those cases, the test fails. Bamboo will then show this as 1 failed test containing a single failed instance of this test method and one successful test. This omits both the 19 other failed tests and the 79 other successful tests, so i need to rely on the html report completely. I suspect this has to do with each of the 100 tests named exactly the same (as usual with data provider).

Is anyone familiar with this? Is this the intended state? Does anyone know how to display (in this case) all 100 test results in the frontend, instead of only one or two?

[Chris Berry]

Hi René.

Yes, this relates to BAM-17738. We parse the xml file and record success or failure for a specific test using the "name" field as the identifier. I believe in your case the "name" for all your test cases are identical and hence Bamboo only displays the one entry.

I've added this community entry to the bug to help raise its visibility with the developers.

thanks

Chris

Atlassian support - Bamboo