API call to update expand section stopped working on 4/23/24

[Duane Kuroda]

I have been using the API to update a confluence page with 3 expand sections. It was working perfectly for weeks, then on 4/23/24 it stopped working, throwing the error:

{'statusCode': 400, 'data': {'authorized': True, 'valid': True, 'errors': [], 'successful': True}, 'message': 'com.atlassian.confluence.api.service.exceptions.BadRequestException: Content body cannot be converted to new editor format'}

Through various testing, the error has been localized to using a table in my expand section. This was working until 4/23. The table is a dataframe converted to html. I've tried to convert the dataframe to html in two ways. Example code below:

df3 = pd.read_csv('<filename and path>')

table3 = df3.to_html(index=False, escape=False)

also did the following:

def dataframe_to_confluence_table(df):
     html = ['<table>']
     for i, row in df.iterrows():
          html.append('<tr>')
          for val in row:
                html.append(f'<td>{val}</td>')
          html.append('</tr>')
     html.append('</table>')
     return ''.join(html)
# Convert DataFrame to Confluence-compatible HTML table
table3 = dataframe_to_confluence_table(df3)

And the expand section definition (that used to work, but now fails):

expand_section3 = create_expand_section('Expand to view Data in the last 3d', table3)

I'm frustrated since my code was working fine for weeks and now just stopped working. All the other code I have works, including my expand section with non-table text. The table is now not working.

Did something change or anyone have ideas on how to get this working like it was 2 days ago?

[Kyle Long · Accepted solution]

Hi,

It looks like your code is attempting to transform a pandas data frame into an html format and you are using this to make an api requests that are responding with "BadRequest". There could be a multitude of causes here, to narrow down the issue can you share an example of the api request with a sample of the contents(with dummy data)? (I would like to reproduce and find precisely what syntax is failing and check for api changes)

- Kyle

[Duane Kuroda]

My problem was apparently the same as almost described here -- https://community.atlassian.com/t5/Confluence-questions/Content-body-cannot-be-converted-to-new-editor-format-if-my-html/qaq-p/1790375#M216349, with a solution that worked for a month suggested by Ayush here -- https://community.atlassian.com/t5/Confluence-questions/quot-Content-body-cannot-be-converted-to-new-editor-format-quot/qaq-p/1332071

I went through some troubleshooting with Atlassian support, and could not not find a root cause that they could fix in the product. They have been responsive, but for the first two weeks we could not nail down specific causes. The error has popped up again, after working for a month, so I may need to re-engage support to figure out what is going on. As for now, support nor any of the kb or community articles have a concrete solution, but I may also have to check if recent macro problems may be at the source of the current issue.

[Kyle Long]

Hi Duane,

For context, atlassian uses an XML like storage format. It can include some html elements but not all. This is important to note because html/xml have reserved characters like "<", "'", "&" etc because the language uses these characters. So you need to replace any reserved character with the equivalent Character entity NAME or NUMBER:

Symbol	Name	Number
<	&lt;	&#62;
>	&gt;	&#62;
&	&amp;	&#38;
"	&quot;	&#34;
'	&apos;	&#39;

I am testing both (& --> &amp; &#38;) and it is working. I have attached an example curl command and a screenshot of the result:

curl --request PUT \
 --url "https://$domain/wiki/api/v2/pages/<your_pageid>" \
 --user "$myemail:$token" \
 --header 'Accept: application/json' \
 --header 'Content-Type: application/json' \
 --data '{
     "id": "<your_pageid>",
     "status": "current",
     "title": "confluence api test & vs &amp;",
     "body": {
         "representation": "storage",
         "value": "<ac:task-list><ac:task><ac:task-status>incomplete</ac:task-status><ac:task-body>checkmark box 1 &amp;&amp;&amp;</ac:task-body></ac:task><ac:task><ac:task-status>incomplete</ac:task-status><ac:task-body>checkmark &#38; &#38; &#38;box 2</ac:task-body></ac:task></ac:task-list>"
     },
     "version": {
         "number": 1,
         "message": "v4.0"
     }
}'

 
I suspect that you are getting the same error but might be using different character input e.g (' " < symbol etc). Can you verify the above example is working for you?

Kind regards,

Kyle

[Duane Kuroda]

Hi Kyle, 

The reserved characters explanation clarified what caused the breakage in my working code that had the &amp; replacement but did not have the other reserved characters accounted for. BTW, my code had no problems with the HTML table I created, where I was not modifying the special reserved characters. I was checking the HTML and rendering it as valid before I was attempting to stuff it into the expand macro. 

After reading your reply, I noticed the "XML like storage format" you mentioned, which meant that while it may render in HTML fine, it may not render in the confluence expand macro. That was, in fact what was breaking. modified my code to instead step through the dataframe and modify the special reserved characters in the dataframe cells. Converting the dataframe with the modified cell contents to HTML then loading it into the expand macro worked.

Your note should be starred and flagged as a key troubleshooting tip for both support and community when using macros and the API for programmatically generating confluence pages.

I will also mark your latest response as the accepted answer.

Thx!

[Kyle Long]

I'm glad this assisted you in solving your issue. I will make a note of this internally. Thank you :)