Not sure if we're missing something or we've gone down a path that is difficult to get to work in Xray.
We're essentially looking to get weekly. aggregate test run status reports for specific tests. Actually, the Xray Test Executions Report is essentially what we're looking for, but with some way to filter down to some specific tests (include some tests from a specific execution and exclude others). It would be nice if it had week over week changes, but that's a different problem.
We can get the specific data that we're looking for through the Xray Test Runs Report, but it doesn't provide aggregate information and there's some challenges with how we identify test runs that creates issues. We could work on resolving these if we thought it was a good path, but it didn't seem worth the effort since we can't get aggregate information.
We've tried just reporting on the TestRunStatus field, but due to how it's setup and how we execute tests, the TestRunStatus field isn't 100% accurate for our reporting needs.
So far, the only way we've been able to find a way to make this work is by making separate, specific test executions for these tests that we're looking to report on. However, that forces us to create so many test executions (potentially 16+ every week) that it isn't sustainable.
The tests we're interesting in reporting on are identified through components and/or labels.
This doesn't seem like an outlandish scenario. I feel like there must be some way to do this that we're missing.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.