JSON Parse Error - Malformed Description Field with Unescaped JIRA Markup

[Sourabh Gandhi]

Category: JIRA REST API

Description:

I'm experiencing a JSON parsing error when consuming the JIRA REST API (`/rest/api/2/search`) due to malformed JSON in the response. The issue appears to be related to JIRA wiki markup (`{code:java}`) containing JSON-like content in the `description` field that is not properly escaped.

Error Details

JSONDecodeError: Expecting ',' delimiter or '}': line 1 column 2244235 (char 2244234)

API Request

GET /rest/api/2/search/jql?fields=*all&expand=changelog,transitions&validateQuery=strict&jql=updated >= '2019-01-01 00:00' order by updated asc

Problematic Response Structure

The issue occurs in the changelog history where the `description` field contains:

"description": "{code:java}{\"errorMessages\":[\"Invalid request payload. Refer to the REST API documentation and try again.\"]}"

This appears to be JIRA wiki markup (`{code:java}`) wrapping a JSON error message, but the quotes within are not escaped, breaking the JSON parser at the outer level.

Environment

- JIRA Type: Cloud (Atlassian)
- API Version: v2

Questions

1. Is this a known issue with how JIRA escapes content in description/changelog fields when they contain wiki markup with JSON-like content?

2. Is there a workaround to request these fields in a format that's guaranteed to be valid JSON (e.g., using `renderedFields` instead of raw fields)?

3. Should the API be escaping these special characters differently, or is there a content policy that prevents this type of data?

Any guidance on how to reliably retrieve all issues including those with complex description content would be greatly appreciated!

[Germán Morales _ Hiera]

Hi @Sourabh Gandhi , your investigation is solid. Pinpointing the exact byte offset where the parser breaks tells me you're on the right track.

Two things to add to what @Arkadiusz Wroblewski already said.

The change that buys you the most is moving from /rest/api/2/search/jql to /rest/api/3/search/jql. In v2, rich text fields like description, comment.body and environment come back as wiki markup: a single long string with embedded blocks like {code:java}...{code}. Any quoting issue inside that string can break the outer JSON parser. In v3 the same fields come back in ADF (Atlassian Document Format), which is structured JSON. The content lives in nested node objects instead of a flat markup string, so the whole "unescaped quotes inside wiki markup" category disappears by construction.

Reference: developer.atlassian.com/cloud/jira/platform/rest/v3/intro

If for some reason you must stay on v2, expand=renderedFields returns the rich text fields as HTML, which is properly escaped and easier to consume than wiki markup.

About the bug question: Atlassian's serializer should always escape " inside a JSON string value, so seeing it unescaped at the byte level is unusual. If after switching to v3 you still see the same pattern in any field, that is the moment to open a Support ticket. Include the request ID from a failing response plus a sample issue key that reproduces. The edge logs aren't visible to customers, only Atlassian Support can chase that down.

[Arkadiusz Wroblewski]

Hello @Sourabh Gandhi 

the wiki markup example you shared should be valid JSON as long as the internal quotes are properly escaped with backslashes. Because this error is triggering deep inside a massive payload, around character 2.2 million, the best first step is to inspect a raw, unparsed snippet of the response at that exact offset to see what is actually breaking your parser.

To simplify troubleshooting, I highly recommend narrowing down your request scope. Instead of pulling all fields alongside heavy expansions like transitions or changelogs, limit your query to only the essential fields you need. If you primarily need historical data, Jira's dedicated bulk changelog fetch endpoint is a much cleaner alternative to bundling everything into a single search query. Start from that and let us know.

[Sourabh Gandhi]

Hi Arkadiusz — thank you, I followed your advice exactly: I narrowed the query to a single issue and inspected the raw unparsed response bytes at the failure offset.

Here's what I found.

On your point: "the example should be valid JSON as long as internal quotes are properly escaped"

That is correct in principle — but our investigation confirms the internal quotes are not being escaped by the Jira API. Fetching just that one issue and reading the raw bytes at the error position reveals:

```
"description": "{code:java}{"errorMessages":["Invalid request payload. Refer to the REST API documentation and try again."]}
```

The `"` characters inside the `{code:java}` block are returned **literally unescaped** in the JSON envelope. There are no backslashes. The JSON parser treats the `"` after `{code:java}{` as closing the string, which is why parsing fails at that exact character.

---

Exact Error (narrowed to single issue)

json.JSONDecodeError: Expecting ',' delimiter: line 1 column 34728 (char 34727)

- Endpoint: `GET /rest/api/2/search/jql`
- Params: `fields=*all&expand=changelog,transitions&validateQuery=strict`
- Response size: 34,826 bytes (single issue)
- Failing field: `issues[0].fields.description`

Raw bytes at offset 34727:

"description":"{code:java}{" ← parser stops here
errorMessages":["Invalid request payload. Refer to the REST API documentation and try again."]}

On your point: "use the dedicated bulk changelog endpoint"

Understood — we will look into separating the changelog fetch. However, even with `expand=changelog` removed, this same issue would still break parsing when the `description` field is included in `fields=*all`. The root problem is the unescaped content in the field value itself.

Questions for Atlassian:
1. Is this a known server-side escaping bug when `{code:java}` blocks contain JSON with embedded double quotes?
2. Does this affect only the `description` field, or can any free-text field trigger this?
3. Is there an API parameter (e.g., `renderedFields`) that guarantees properly escaped output?