reading large confluence page

[hemaswapnika_e]

confluence.get_page_by_id(page_id, expand='body.storage') is not reading all the data if the confluence page is very large. (1500 KB)

How to handle reading of such large confluence pages?

[hemaswapnika_e]

.

[Kristian Klima]

Hi @hemaswapnika_e and welcome.

Here's a little trick that worked for me on multiple occasions.

1. Divide the content of the supersized page into 2 or more parts.
2. Create a single page for each of the said split parts. (Optionally, you can hide them outside of the page tree above the root page).
3. On the page that you want to read, use the Page Include macros - one for each page with split content.
4. Save the page. The content should display properly.

[hemaswapnika_e]

Hi Kristian ,

1. On what basis we can split the page data. Do we have any limit parameter in confluence.get_page_by_id() method?

2. Once the page is splitted to multiple pages, where are they stored and how to read them?

3. Once the page is splitted to multiple pages, how to use the Page Include macros on these pages?

[Kristian Klima]

Don't think as a developer, think as a reader/user :)

You simply split the content manually.

So let's imagine that, on my supersized page A, I have three videos - V1, V2, V3.

Here's what I do:

1. I create three pages - B, C, D.
2. I put one video on each page - B=V1, C=V2, D=V3. Then I save the pages.
3. On page A, with all videos deleted, I place three Page include macros - one for each page (B, C, D). I save the page A.

(you can move pages B, C, and D above the space's root page to ensure they're not visible in the tree)

I used this trick when the content was too big for Confluence to save or display. And our tools are able to 'read' included content. Your mileage may vary but it's worth trying because it should work.

[hemaswapnika_e]

Could you pls share the code if possible

[Kristian Klima]

You don't need the code, you do it manually by creating pages in your space.

The whole preparation (splitting the content, creating new pages, inserting macros) is a GUI thing.

Once you have it all ready, you can see if your original method still runs into the page-too-large issue.

[hemaswapnika_e]

I am currently working on a POC where i need to read all the confluence pages based on the confluence space key. I cant manually created sub pages during this process.

Is there any way we can split the supersized page into sub pages using atlassian-python-api library or any other approach?

[Kristian Klima]

"I am currently working on a POC where i need to read all the confluence pages based on the confluence space key."

Nothing should change, the page A will still be where it's now, it's just it would display content from different pages (from the programmatical point of view, it would link to the content from different pages).

Unless you're working with a specific set of pages.

We run a solution that works with pages from a specific space and we add new pages, we just 'refresh' the tool and it processes all the pages based on the space key.

If you don't want to create NEW pages, you can experiment with the existing pages - simply move the content (manually) from page X to page Y, save Y, put the include macro in page X. If you can verify, that your POC processes page X as if it actually featured the content, you'll have your proof.