nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Is there a Python library that can parse Confluence page content?

Woodson Miles

My goal is to use the Confluence API to get the content of a page, parse it, edit it, and update that same page with the edited content.

At first, I assumed Confluence's storage format was HTML. Based on that, my original plan was to use Python's BeautifulSoup module to parse and edit the content once I retrieved it from the Confluence API. I now know that the Confluence page storage format is "XHTML-based". I've tried to parse it with various BeautifulSoup parsers (lxml, xml, html.parser) but they all get caught on the standard-breaking macro elements like this:

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="283daa7d-46af-4d6d-a177-a00b4a2bc342"><ac:plain-text-body><![CDATA[*\[CDRL:\]*]]></ac:plain-text-body></ac:structured-macro>

Is there a preferred method for parsing this?

Accepted answers

All comments

Bill Bailey

Have you seen these:

Woodson Miles

Hey Bill, thanks for your response.

Alas, while these 2 libraries are helpful wrappers for the Confluence API, as far as I can tell, neither have the ability to parse the XHTML that is pulled from the API representing the content of a page.

Stephen M_08b9y1

I am having the same issue! If you found a solution could you share? Thanks!

Woodson Miles

I never found a parser that worked perfectly for Confluence. Ultimately, I was forced to edit Confluence pages using regular expressions.

Steve Sadler_n9749v

I have had the same issue. Ultimately, I have resorted to using an html parser (where that works), an xml parser (where that works), and regex for everything else.

Uzair Ansari

@Woodson Miles I am using ConfluencePS PowerShell module to achieve this. It's quite simple to fetch and upload page contents using Get-ConfluencePage and Set-ConfluencePage commands.

You can get this module from PowerShell Gallery.

John Larring

Same issue here. Need to find a way to parse Confluence XHTML incl. macro notation.

Ifeoluwa Akande

This might not be the solution you desire; however, If you are interested in writing Confluence Wiki text to docx format (while maintaining the wiki formats), you can try jirawiki2docx python library.

https://pypi.org/project/jirawiki2docx/

smingolelli

co-pilot suggested the following which I can confirm worked:

import requests
from requests.auth import HTTPBasicAuth
from bs4 import BeautifulSoup

# Replace these variables with your Confluence details
confluence_url = 'https://<my site>.atlassian.net/wiki/rest/api/content/'
page_id = '<my page id>' # Replace with your page ID
username = '<my username>'
api_token = '<my token>'

def get_confluence_page_text(page_id):
url = f"{confluence_url}{page_id}?expand=body.view"
auth = HTTPBasicAuth(username, api_token)

response = requests.get(url, auth=auth)

if response.status_code == 200:
page_data = response.json()
page_html = page_data['body']['view']['value']

# Use BeautifulSoup to extract text from HTML
soup = BeautifulSoup(page_html, 'html.parser')
page_text = soup.get_text()

return page_text
else:
raise Exception(f"Failed to fetch page: {response.status_code} - {response.text}")

if __name__ == "__main__":
try:
page_text = get_confluence_page_text(page_id)
print(page_text)
except Exception as e:
print(e)

Uladzislau Hubar

In case anyone else is still looking for this: https://github.com/Unificon/confluence-content-parser

Welcome!

It looks like you're new here. Sign in or register to get started.

Quick Links

Find more posts tagged with

confluence

server

confluence-server

python

parse