Xray Data Center – "Not Applicable" step status causes Test Run to be PASS when all steps are NA

[Ido Cohen]

Hi,

I’m working with Jira Data Center and Xray.

I created a custom Test Step Status called "Not Applicable" to represent steps that are not relevant for a specific execution.

Requirement:
- "Not Applicable" should only apply at the Test Step level.
- It should behave as a neutral status (not affecting the Test Run result).
- If at least one step is PASS and others are Not Applicable → Test Run should be PASS.
- If any step FAILS → Test Run should be FAIL (default behavior).
- If ALL steps are "Not Applicable" → Test Run should NOT be PASS (since nothing was executed).

Current behavior:
- I mapped "Not Applicable" to PASS.
- When all steps are NA → Test Run becomes PASS.

What I tried:
- ScriptRunner Listener
- Xray REST API:
/rest/raven/1.0/api/testrun/{id}
- I can read and update Test Runs (GET/PUT works)
- But I cannot reliably detect the original "Not Applicable" step status from the API response (it seems mapped to PASS)

Question:
Is there a native way in Xray to define a neutral Step Status?

If not:
How can I correctly identify the original Step Status ("Not Applicable") via the API before mapping?

Thanks.

[Martin Runge]

Hi @Ido Cohen

Welcome to the community.

From my perspective, Xray does not natively support a neutral status that excludes steps from the overall Test Run calculation, so you must map "Not Applicable" to a custom status and use a ScriptRunner listener or the Xray REST API to programmatically evaluate the execution result based on your specific logic.