I want to scrape an entire Confluence 7.18.19 instance to analyze every bit of text of every space and it's child content with my NLP application.Is there a way to do this with the API? I am limited to the API unfortunately.Thanks in advance!
For future reference, refer to this post --> https://community.atlassian.com/t5/Confluence-questions/Re-How-Should-I-get-all-text-from-all-spaces-and-their-c/qaq-p/2653690/comment-id/303748#M303748
Hi - I hope that you are Doing good !!
Yes this use case is achievable with API'Get all spaces' endpoint of the Confluence REST API to obtain a list of all spaces in the instance
the following REST API (https://developer.atlassian.com/cloud/confluence/rest/api-group-space/#api-wiki-rest-api-space-get).
Happy to help any further !!
Thank you very much and have a fantastic day!Warm regards
Good morning,Thanks for your reply!If I execute a GET request to this endpoint "https://wiki-acc.mycompany.com/rest/api/space", I get the following response:(note that I replaced certain values with mock values)
{"results": [{"id": 8290324,"key": "spaceKey","name": "spaceName","type": "global","_links": {"webui": "/display/mySpace","self": "https://wiki-acc.mycompany.com/rest/api/space/mySpace"},"_expandable": {"metadata": "","icon": "","description": "","retentionPolicy": "","homepage": "/rest/api/content/7899603"}},.... more spaces ...],"start": 0,"limit": 25,"size": 25,"_links": {"self": "https://wiki-acc.mycompany.com/rest/api/space","next": "/rest/api/space?limit=25&start=25","base": "https://wiki-acc.mycompany.com","context": ""}}
This does not resemble the example response from https://developer.atlassian.com/cloud/confluence/rest/v1/api-group-space/#api-wiki-rest-api-space-getThe Confluence instance was updated to 7.19.19 today, maybe that has something to do with it?
Secondly, how am I supposed to get every bit of text from all child content of a space? It seems inefficient to me to query every single content endpoint, is that the only way to achieve that?
Good morning @Humashankar VJ ,I would really appreciate an answer to my question!To clarify; I want to get all the content from all spaces with type "global".This now costs more than 70 API calls. This is because there is a limit of 25 objects per call and I need to get the "space key" from every space manually to get their content via a for loop with calls to "https://wiki-acc.[mycompany].com/rest/api/space/[space_key]/content?expand=body.storage".If there is a way to limit the amount of API calls to achieve my goal, I would very much like to know. It now takes about 37 seconds to execute, which is too slow for my use case.Thanks in advance!
@Humashankar VJ It has been a week... would you please answer my question?
It looks like you're new here. Sign in or register to get started.
