How to use Confluence API to get actual data / values from Page Properties

Jessica Malenfant August 1, 2017

I'd like to use the Confluence API to get the actual data entered for a confluence page tracking statements of work / contracts. I have 1 page per contracted project that uses a global template for users to enter the SOW contract details. Ineed to get the data from each page in this space that I can then pull into a BI analytics tool like Qlik or Tableau to look at projects over time, budgets, deliverables, timelines, etc. and perform calculations, etc. The Page Properties Report macro helps to see a summary of these but I need to also do analysis / report on the data.

I can't figure out what url to use to get at any of this. Here's an example of what we have now

 

And I can connect to the API for the space, I just can't figure out how to get the data now.

https://ourinstancename.atlassian.net/confluence/rest/api/space?spaceKey=OURKEY

 

confluence page properties example.JPG 

 confluence space page tree.jpg

 

4 answers

0 votes
ekarlssonjohan April 12, 2019

Since I spent about 8 hours to figure this out I thought I might post an example here just in case someone else stumbles on the same issue.

This REST call will retrieve all page properties for each page where pages have the label: label=mylabel.

http://localhost:8090/rest/masterdetail/1.0/detailssummary/lines?cql=label%3Dmylabel&spaceKey=MYOWNSPACE

Ajay Verma January 23, 2020
Hi 

am using your url and getting the below error

{"errorCode":500,"errorType":"RENDERING:MACRO","errorMessage":"Unable to locate pages for Page Properties Report. Details: CQL string is \"(label=projectinfo) and macro = details order by lastModified desc\", error message is: \"Not permitted to use confluence : null\""}
Like Mousuma Roy likes this
jmakarlsson January 23, 2020

Hi,

actually I think I only manged to get it to work in a browser if I remember correctly. Also you need to be logged in to confluence. 

Ajay Verma January 23, 2020

Now I can read the data in jason thanks to you . but can you please confirm I have 10 columns in my macro page but I am getting only Tital from there . how can I get rest of the columns ?

jmakarlsson January 23, 2020

Not sure I understand, what does your url look like? 

However, maybe it works if you add page size and index? Like: 

https:// localhost:8090/rest/masterdetail/1.0/detailssummary/lines?cql=label%3Dmylabel&spaceKey=myspace&pageSize=30&pageIndex=0&headings=myheading

Ajay Verma January 23, 2020

HI

could you please help me to correct the below url is it correct or i missed something.

https://mysite/rest/masterdetail/1.0/detailssummary/lines?cql=label=projectinfo&spaceKey=project&pageSize=30&pageIndex=0&headings=Projektledare, Projekttyp, Fas, Overall, Budget, Tidplan, Kundnöjdhet, Lösning, Uppdaterad

 

also am not getting all records some records i am missing what I can do to get all records

jmakarlsson January 23, 2020

I tried your url with multiple headings and it works. Any difference if you remove the spaces between the "headings"? Could the swedish characters casue any problem? 

Note: that you might get HTML as part of the key (heading) value.

Regarding number of records, not sure what you mean. But what happens if you increase page size to 100? 

Ajay Verma January 23, 2020

Hi

Thanks for reverting . am not getting all records some are missing . like RI Förvaltning 2020 . already increase the page size to 100 below error I have recieved

{"currentPage":0,"totalPages":1,"renderedHeadings":["Acceptansdatum"," Ansvarig säljare","Budget","Fas","Hemlig handling","Hemlig uppgift","Individuell sekretess","Kund","Kundens projektledare","Kundens projektsponsor","Kundnöjdhet","Kundnummer","Lösning","Overall","PUB(GDPR) avtal","PUL","Projektledare","Projektstart","Projekttyp","Resurser","SUA-klass","System","Teknikinformationskontakt","Tidplan","Timpris","Uppdaterad","Ursprungligt acceptansdatum","Verksamhetskonsult"],"detailLines":[],"asyncRenderSafe":false}

What I can do to get all records
Ajay Verma January 23, 2020

I have checked and found one issue when I changed page index to 1 then I received the missing records but other one disappear . 

could you please confirm whats the mean of page index here  

jmakarlsson January 23, 2020

I guess you need to include "RI Förvaltning 2020" in the url. Or could there be a problem having spaces in the key/heading?

 

Regarding page index. If you have 1000 pages and page size is 200 then page index=0 will show records 0-200 and page index=1 will show 201-400 and so on. 

Ajay Verma January 23, 2020

I think index making a problem here . I have already tried to remove pag size and page index but still the same issue . 

any idea what i can do now to getting all records

jmakarlsson January 23, 2020

No idea actually. I don't have that problem. I set &pageSize=500000000&pageIndex=0 and I am able to get all pages. 

David Memmer January 23, 2020

even though you change the pagesize you cant make it as big as you want. There will always be a limit (at 500 i think) and you need to start paging

Ajay Verma January 24, 2020

When I change the page size like 70 or 100 its give me below error

 

{"currentPage":0,"totalPages":1,"renderedHeadings":["Acceptansdatum"," Ansvarig säljare","Budget","Fas","Hemlig handling","Hemlig uppgift","Individuell sekretess","Kund","Kundens projektledare","Kundens projektsponsor","Kundnöjdhet","Kundnummer","Lösning","Overall","PUB(GDPR) avtal","PUL","Projektledare","Projektstart","Projekttyp","Resurser","SUA-klass","System","Teknikinformationskontakt","Tidplan","Timpris","Uppdaterad","Ursprungligt acceptansdatum","Verksamhetskonsult"],"detailLines":[],"asyncRenderSafe":false}
WW
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
January 24, 2020

What you're getting as a result is not an error (I can't read the human language it's in, though) but it looks like your result is just empty.  That's this part:

"detailLines":[]

If you read my last post, sometimes I have to figure out what the total number of results is, then enter that as the page size.  No idea why, and it only seems to be for certain fields.  This may or may not be your problem, though.

You should also make sure your cql is good.  I think you solved the error issue with the spaces, but what you're actually searching for may not have any results.  Like, you might not have any pages with the projectinfo label.

Try simplifying the cql to something that has no special characters just to see if you get any results such as:

https://localhost:8090/rest/masterdetail/1.0/detailssummary/lines?cql=label%3Dprojectinfo&spaceKey=project&pageSize=30&pageIndex=0&headings=Budget

 

You can also try leaving off the pageSize and using the default to see if you get any results such as:

https://localhost:8090/rest/masterdetail/1.0/detailssummary/lines?cql=label%3Dprojectinfo&spaceKey=project&headings=Budget

(I think you may have done this already, but without a heading)

If you don't get any results from the above or get empty results again, then make sure:

  • the spelling of your label is correct and that you actually have pages with that label
  • that you have a table with 2 columns (for vertical tables) or 2 rows (for horizontal tables) actually inside of a Page Properties macro on the page.  You'd be surprised at how often they're setup wrong
  • there are values in the correct column/row for the pages
  • the column name is spelled correctly (Budget should be simple enough, though)

If running it with just Budget works, then your problem is most likely the special characters.  You should encode your URL before sending it.

In any case, EVERYONE ON THIS THREAD, if you haven't already, please go and VOTE for this issue CONFSERVER-37890.  It basically asks to increase or eliminate the current limit of 500 results.

Mousuma Roy May 15, 2022

I am using  this https://ourinstance.com/rest/masterdetail/1.0/detailssummary/lines?spaceKey=myspacekey&cql=label%3Dmylabel

I am getting this. 

{
"currentPage": 0,
"totalPages": 0,
"renderedHeadings": [],
"detailLines": [],
"asyncRenderSafe": true
}

my page with this label has 10 macros.
any idea why I am getting this
WW
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
May 16, 2022

You might not be authenticating properly or the label might be misspelled. It's hard to tell from the given results.

I'd need more details like what code you're using to make the call, if you're on Cloud or Server/Data Center, etc.

0 votes
Luis Gonzalez March 6, 2018

Hi!

The API answer seems to paginate, but there is no link to the next page... nor parameter to access a page in particular.

Trying to find a workaround, if &pageSize=99 added to the url, detailLines returns empty instead of the 30 detailLines that returns without that parameter.

Any suggestion? 

Thanks in advance,

Luis

WW
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
May 22, 2018

I was having this same issue, but I figured it out.  You should see the totalPages in the first part of the results.

{"currentPage":0,"totalPages":17, etc.

To see more pages, add &pageIndex=1 to the request to get the next page. 

If there are 17 pages, &pageIndex=16 would be the last page (0 would be what's included in the first page).

Here's what it would look like to see the second page, using the example given by Robert (above), with the additional pageIndex part:

/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type=page&spaceKey=MYSPACEKEY&headings=key1,key2,key3&pageIndex=1
David Memmer June 14, 2018

Adding &pageSize to the query works for me. Has there been an update ?

WW
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 14, 2018

It's weird.  Sometimes setting the pageSize to 1000 works, and sometimes, it will only work if I set it up to 62, which is the exact number of records that exist for the cql I put. 

I think it depends on what my headings are.  In particular, it doesn't like the heading "Compliance Plan."  If I change the spelling of it, it gets the data and leaves that field blank.  If I leave that heading in there, it returns no records.  I've tried headings that are longer and shorter, and they work.  I have no idea why it doesn't like that field.

Like I said, weird. 

WW
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
June 14, 2018

And, FYI, this API must tie in to the same code the page properties report macro does because there is the same issue of a limit of 500 results regardless of how many pages they display on.  To get more than 500 results, you'd have to do multiple calls/searches.

Like Rodriguez Jose Alberto likes this
0 votes
Robert Reiner _smartics_
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 1, 2017

We do not use this API since we use our document properties. :-)

But this REST call should work

https://example.com/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type%3Dpage&spaceKey=MYSPACEKEY&headings=key1%2Ckey2%2Ckey3

 Or with cURL

curl 'https://example.com/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type%3Dpage&spaceKey=MYSPACEKEY&headings=key1%2Ckey2%2Ckey3' | jq .

 

Jessica Malenfant August 1, 2017

Thanks so much. I actually looked at your products but it looks like your tools are only for server and not cloud - is that true?

As far as your example REST call:

I swapped out the example.com for our site and spaceKey for ours, but can’t figure out what the headings are equivalent to?  How do we drill down to the pages and its child pages? I tried w/ our page name like this:

 

 

https://example.com/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type%3Dpage&spaceKey=PDP&headings=Project+ABC+Contract%1  

error:

Unable to locate pages for Page Properties Report. Details: CQL string is \"(type%3Dpage) and macro = details order by lastModified desc\", error message is: \"Could not parse cql : (type%3Dpage) and macro = details order by lastModified desc\"  

 

Then tried a few variations that also failed:

  https://example.com/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type%3Dpage& spaceKey=DP&headings=keyContract+Summary%1  

 

https://example.com/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type%3Dpage& spaceKey=DP.Contract+Summary+-+Open+Contracts&headings= Project+ABC+Contract%1

 

Any insight you have (to this very novice coder!) is greatly appreciated!!!

Thanks!

Robert Reiner _smartics_
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 1, 2017

Thank you for checking out our Web API! Indeed it is only available on Confluence server.

Sorry, it seems that I messed up coping the URL.

The headings are the keys to the page properties.

page-properties.png

This works for me (I snipped off the protocol/server part) using a web browser:

/confluence/rest/masterdetail/1.0/detailssummary/lines?cql=type=page&spaceKey=MYSPACEKEY&headings=key1,key2,key3

Here the first characters of the result I get:

{"currentPage":0,"totalPages":17,"renderedHeadings":["key1","key2","key3"],"detailLines":[{"id":70189634,"title":"Example Page with Page Properties","relativeLink":"/display/MYSPACEKEY/Example+Page+with+Page+Properties","details":["value1","value2","value3"],"likesCount":-1,"commentsCount":-1}

I assume that you need to check out the CQL documentation to learn how to navigate through the pages. I would expect that selecting on pages (with something like key1=value1) should be possible, but unfortunately I do not know the syntax (due to the lack of documentation). :(

It seems that the REST API Browser is not available for Cloud. This would be the UI (as an overview over the supported parameters):

 

UI.png

Like Sebastian Esch likes this
Jessica Malenfant August 2, 2017

Thanks so much - I'll try this out today. Any thoughts about providing a cloud solution in addition to server? :)

Robert Reiner _smartics_
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 3, 2017

Hi Jessica,

I'm happy if I could give you some links to start with. Hope you figure out how to use the REST API despite the missing docs quickly ... ;)

Thinking about Cloud is what we do, but unfortunately this is still a very, very long way to go. Thank you for your feedback!!

Cheers,

Robert

0 votes
Robert Reiner _smartics_
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 1, 2017

Is this what you are looking for?

How to retrieve metadata from Page Properties using REST API

Unfortunately I have not found any useful documentation on how to use this API.

EDIT: Space Key is required and headings allows to specify the properties you are looking for.

This seems to be the source code on Bitbucket.

Jessica Malenfant August 1, 2017

Thanks! I actually just found that one! Bummer about no documentation! 

Did this end up working for you? I keep geting  a 404 error when i try:

https://OURINSTANCE.atlassian.net/confluence/rest/masterdetail/1.0/detailssummary/lines/api/space?spaceKey=OURKEY

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events