Retrieve paginated results, startAt and maxResults

[michael.oshea.mo2]

Hi, 

• URL is clouddevelopment.atlassian.net/rest/api/3/search?jql=encodedBlahBlah
• Results are being paginated with a current maximum of 50 results (out of several thousand) as described here API v3 pagination (JSON snippet copy/pasted from this webpage)

{ "startAt" : 0, "maxResults" : 10, "total": 200, "isLast": false, "values": [ { /* result 0 */ }, { /* result 1 */ }, { /* result 2 */ } ] }

The v2 and v3 API endpoints appear to support pagination differently, as does Jira local hosting and cloud saas.

My question: After receiving the first 50 results (URL above), I need to make subsequent API invocations to retrieve results 51-100, 101-150, 151-200, 201-300 and so on; how? Rather than a specific answer, ideally I am looking for a definitive Atlassian Jira API documentation URL link detailing the parameters for the request payload and/or URL string. The community forums seem littered with dross from prior Jira API versions and/or locally hosted and not cloud, that behave differently.

Thanks

Mike

[Pramodh M · Accepted solution]

Hi @michael.oshea.mo2 

Here it is https://developer.atlassian.com/cloud/jira/platform/rest/v3/intro/#expansion

And if you navigate to each of the API's there are parameter present with an example for reference.

Let me know which API you are exactly looking at?

Thanks,
Pramodh

[Pramodh M]

@michael.oshea.mo2 

It's just adding

?startAt=0&maxResults=1000

at the end of the API URL

There's no rocket science here

And for easiness, once you make the paginated call, you will get the next call in response to the JSON body itself.

[michael.oshea.mo2]

And the answer is that a corporate outbound proxy is rewriting the URL ... dear dear.

Thanks Pramodh

[Heajin Kim]

I did use "?startAt=0&maxResults=1000" the url but this is what I got. I am not the Jira Admin so what else should I do ?

url : project in (xxx, yyy, zzz ) AND fixVersion = v1.11.0 AND status != Done AND "Start date[Date]"=0 AND maxResult=150

https://xxx.atlassian.net/issues/?jql=project%20in%20(%2C%20%2C%20%2C%202C%20%2C%20%2C%20%2C%20%2C%20)%20AND%20fixVersion%20%3D%20v1.11.0%20AND%20status%20!%3D%20Done%20AND%20%22Start%20date%5BDate%5D%22%3D0%20AND%20maxResult%3D150

I got this error
"Field 'maxResult' does not exist or you do not have permission to view it."

Any suggestion?

[michael.oshea.mo2]

Again thanks Pramodh, but after trying this and other combinations over the past few hours, all I can report is that this approach does not work. It was the reason I was asking, in bold above, for a definitive piece of documentation on the Atlassian website.

The URL I have encoded is

clouddevelopment.atlassian.net%2Frest%2Fapi%2F3%2Fsearch%3FstartAt%3D5%26project%3D%27testproject%27%20order%20by%20createdDate%20asc

Decoded this is

clouddevelopment.atlassian.net/rest/api/3/search?startAt=5&project='testproject' order by createdDate asc

The response retrieved is for the correct project as specified in the JQL, but the offset of 5 (or any integer) is ignored and the JSON response returned reports, as included in the original post, 

"startAt" : 0

It should report "startAt" : 5 and the results reflecting the window offet.

So, there might not be "rocket science" here, to use your words, but there is clearly some "voodoo", that could be cleared up if the API documentation detailed clearly, on the same documentation page, how the next window of paginated results is retrieved.

Mike

[Emmanuel Tavolaro]

Buenas tardes @michael.oshea.mo2

Creo poder ayudarte a resolver tu problema con extraer los registros por pagina, después de investigar y realizar pruebas, la única solución es generar un bucle que modifique el valor de comienzo hasta llegar al total de registros.

Te comparto un fragmento que utilizo para paginar:

start = 0

while True:

        url = "https://<your url>.atlassian.net/rest/api/3/search?startAt="+str(start)+"&maxResults=100"

        resp = requests.request("GET", url, headers=headers, auth=auth).text

        data = json.loads(resp)

        series = json_normalize(data,)

        df = pd.DataFrame(series)

        df.to_csv(path_or_buf=csvfile, sep=';', mode='a', header=None, index=False)

        #print(len(df), start) ----> len(df) : devuelve la cantidad de issues / start : contador de registros por pagina 

        if len(df) > 0:

            start += len(df)    ---- > se le asigna al contador la cantidad de issues que devuelve por pagina

            print('Numero parcial de archivos:',start)

        else:

            print('Archivos completados!')

            break

Espero que te sirva me resolvió algo que no pude encontrar en la documentación.

Saludos, Emmanuel Tavolaro

[michael.oshea.mo2]

Thanks for the reply Pramodh. The documentation link you provided is the same one I included in my original question, API v3 and cloud hosted.

This documentation link does not document how to paginate ..... there is no example in this API  documentation and merely just the JSON response detailing startAt, maxResults, .... that I also copy/pasted verbatim in my question.

Mike

[Chouhei Ngamba Ngangom]

Add  ?pagelen=100 at end of the url.

It works for me and the suggested max pagelen is 100

Details -> pagelen ref 

Hope its helpful!