Pagination issues for boards and sprints?

[Romo Fernandez, Matias]

So I'm trying to use the software cloud rest API to implement a microservice that updates data from Jira/tempo upon request. But I'm struggling with the pagination because even if I use all the options I've scoured through the documentation, and even went through some of the issue reports, I have not been able to get past the pagination limit, at all, and I'm starting to get desperate. 

Things I've tried: 

1. Editing the startAt, maxResults, total fields in the request - Did not work.
2. Adding the XExperimental-Api header -Did not work.
3. Using different API versions (1/2/3) - Did not work.
4. Using a different API (Cloud platform, but I still can't get the pagination to work)

I've tried both using curl and the requests library in python 3.7 and neither is working.

If someone has solved this, please help me out. 

Thanks in advance. 

[Andy Heinzer · Accepted solution]

Hi @Romo Fernandez, Matias 

I understand that you are trying to get back some data from Jira Cloud REST API in regards to issues that appear on a board or sprint, but are running into problems with pagination in the response.  Sorry to hear that, I can see this has been frustrating so far.  The experimental header is sometimes required in the call, but it depends on which endpoint you are calling here.

Which specific REST endpoint are you using here?

I tried to walk through this myself using curl, and I choose the endpoint of Get issues for board GET /rest/agile/1.0/board/{boardId}/issue.  In my case, I found that the board ID number was just 1 (you can also find this as the rapidview id number in the URL of any classic board.  So if the URL was https://[yourURL].atlassian.net/secure/RapidBoard.jspa?rapidView=1&projectKey=SCRUM then your value is 1 here).

In my curl call I am using basic authorization where I created a token and used it to make a string of emailaddress:token and then base64 encode that string as explained in Basic auth for REST APIs.

From there I could just make a curl call of

curl -v -D- -X GET /
-H "Authorization: Basic [redacted]" /
-H "Content-Type: application/json" /
"https://[mycloudsite].atlassian.net/rest/agile/1.0/board/1/issue?startAt=0&maxResults=10"

This returned to me the first 10 issues on that board.  You can then just increment the startAt and make the next call such as

curl -v -D- -X GET /
-H "Authorization: Basic [redacted]" /
-H "Content-Type: application/json" /
"https://[mycloudsite].atlassian.net/rest/agile/1.0/board/1/issue?startAt=10&maxResults=10"

to return the next 10 issues on that board and so on.

If this is not working for you, then I would be interested to see if we can learn more about the syntax you are using, the endpoint you are calling, and then the response that is generated.

I hope this helps, please let me know.

Andy

[Romo Fernandez, Matias]

Hello Andy! 

This is quite similar to what I'm trying to implement, but I'm doing it both for boards and issues. As you mentioned, I run the same call, with maxResults set to 10. (Originally I had 50, which is the local maximum I believe). The only difference seems to be the URI where you're directly passing the params for startAt and maxResults (ignore the export, for now, it's just for tests):

curl --location --request GET 'https://[your-domain].atlassian.net/rest/agile/1.0/board' \
 --header 'Accept: application/json'  --header 'Authorization: Basic $API_KEY' \ 
--header 'startAt: 10' --header 'maxResults: 10' > exported_file.json

Is it possible that this occurs because of how you and I parse the URI differently? 

In python, I'm doing the same thing (same URL and headers), and if the URI is the origin of the mistake it certainly is what's happening and it should fix my issue. 

 def request(url, filename, headers):
    request = requests.request("GET", url, headers = headers).text
    output = json.loads(request)    
    with open(filename, 'w+') as file:        
        json.dump(output, file, sort_keys = True)    
        print('Data exported to {}, from {}'.format(filename,url))

 Specifically for boards, the numbers aren't sequential, so the IDs are all over the place, as long as I know the range that shouldn't be an issue I guess? Or maybe I can just guide myself off of the isLast parameter? To be more explanatory, I have 68 boards but the first ID is 6 and the last one is 98 so a lot of sparsity there. 

I'm going to try parsing the URI and see if I get better results. Will get back to you in a bit! 

[Andy Heinzer]

Interesting, I had not though of trying to use the path parameters passed as headers.  Does that actually work? 

In all the examples I have used with REST calls to Jira, the path parameters are appended to the end of the URI itself, with the first parameter prefixed by the '?' character, and each subsequent parameter separated with a '&' character.  This is the way that I would suggest trying to format the call, as I know this way works.  I believe that you would have to adjust your URL/URI to append the path parameters to it rather than try to pass them in as a header. 

You still do need the authorization and accept headers, but I think those should be the only headers needed here.

As for the ID of the boards, if memory serves these actually are sequentially numbered from the time they are created on that site.  You might not find a board with an ID number if that board has been deleted, or if your account does not have permissions to see it.  But historically speaking, I believe that the board numbers are usually sequential to the time they are created.

[Romo Fernandez, Matias]

No, it doesn't, hahaha. I hadn't curled APIs in a while so I'd forgotten. 

Using the URI format worked like a charm for me, I just had to fiddle around a bit to find the right extractions paths and cutoff points. 

Thank you so much for your time and help! Cheers!