How can I get page 2 with a pagelen=100 when querying a list of repos with bitbucket API 2.0?

[Luis Matos]

I have run into a cumbersome limitation of the bitbucket API 2.0 - I am hoping there is a way to make it more usable.

When one wants to retrieve a list of repositories from the bitbucket API 2.0, this url can be used:

https://api.bitbucket.org/2.0/repositories/{teamname}

This returns the first 10 repos in the list. To access the next 10, one simply needs to add a page parameter:

https://api.bitbucket.org/2.0/repositories/{teamname}?page=2

This returns the next 10. One can also adjust the number of results returned using the pagelen parameter, like so:

https://api.bitbucket.org/2.0/repositories/{teamname}?pagelen=100

The maximum number can vary per account, but 100 is the maximum any team is able to request with each API call. The cumbersome part is that I cannot find a way to get page 2 with a pagelength of 100. I have tried variations on the following:

https://api.bitbucket.org/2.0/repositories/{teamname}?pagelen=100&page=2
https://api.bitbucket.org/2.0/repositories/{teamname}?page=2&pagelen=100

I've also tried using parameters such as limit or size to no avail. Is the behavior I seek even possible? (Apparently, it is - see my replies to Noah's answer below)

[Noah Sutherland · Accepted solution]

It appears this is fixed. I was able to use pagelen=100&page=2 in my query and received the extra 15 repositories that were missing from page 1.

[Luis Matos]

That's good news, Noah - however, I am still not able to see a returned list of repos for that combination of parameters (`pagelen=100&page=2`).  (The ones that I listed above as working, still work for me.) Is there any other part of your command that could be helping to make the query work for you?

[Noah Sutherland]

Here is my whole bash script:

LOGIN=xxxxxxxxx:xxxxxxxxxxx

SERVER=api.bitbucket.org

curl -k -s "https://$LOGIN$SERVER/2.0/repositories/xxxxxxxx?role=member&pagelen=100" | jq -r ".values[].slug"

curl -k -s "https://$LOGIN$SERVER/2.0/repositories/xxxxxxxx?role=member&pagelen=100&page=2" | jq -r ".values[].slug"

[Luis Matos]

Wouldn't you know it - the only thing I was missing was to put the url in quotes.  All of the other queries work without quotes around the url.

Moral of the story: If you want to use multiple parameters with the 2.0 api, the entire url needs to be in quotes.

Thanks very much for your help.

[Deleted user]

pagelen > 10 does not work. It still only returns the first 10

[Kevin Gallagher]

Guys, you need to fix this. I can't list more than 25 branches in a repository with the v2 API, even on Bitbucket Server? C'mon now. This is a duplicate of: https://community.atlassian.com/t5/Answers-Developer-Questions/How-can-I-set-the-page-size-of-branches-on-V2-0/qaq-p/535608

[Luis Matos]

Hey, Kevin - I don't know if it's too late for you, but I posted my workaround below. And Noah helped me isolate the actual fix above...

[Slav Aleqsanyan]

100

[Serge Nazin]

see https://gist.github.com/soberich/0b3fb069a950f3c2b408a588db93a6e5

e.g.

#!/usr/bin/env sh

# prerequisites: `httpie`, `jq`, GNU's `parallel`. e.g. brew install <package> # there is max 100 page length n pages where n is 100 length page. Execute one one by one (there is no way you'll get more than 100 times parallelization)

# in `.values[].links.clone[1].href` `1` is for SSH, where `0` would be for HTTPS.

http https://<user>:<pass>@api.bitbucket.org/2.0/repositories/<account_name> pagelen==100 page==<page_num> | jq -r '.values[].links.clone[1].href' | parallel git clone

# Voila it takes approx 1-2 minutes to clone a 100 repos.

[Luis Matos]

I figured out a workaround:

It would seem this behavior is not possible at this time. I was able to get around this by creating a bash script that looped through each page of 10 results, adding each new 10 repos to a temporary file and then cloning into those 10 repos. The only manual thing that needs to be done is to update the upper limit in the for loop to be the last page expected.

Here is an example script:

for thisPage in {1..[Last Page Expected]}
  do
    curl https://api.bitbucket.org/2.0/repositories/[organization]?page=$thisPage -u [username]:[password] > repoinfo

  for repo_name in `cat repoinfo | sed -r 's/("slug": )/\n\1/g' | sed -r 's/"slug": "(.*)"/\1/' | sed -e 's/{//' | cut -f1 -d\" | tr '\n' ' '`
    do
      echo "Cloning " $repo_name
      git clone https://[username]@bitbucket.org/[organization]/$repo_name
      echo "---"
    done
  done

Much help was gleaned from:
https://haroldsoh.com/2011/10/07/clone-all-repos-from-a-bitbucket-source/
and http://adomingues.github.io/2015/01/10/clone-all-repositories-from-a-user-bitbucket/ Thanks!

[Matthew Feinberg]

You can limit your request to just a single project this way. Max page length is 100.

curl -s https://api.bitbucket.org/2.0/repositories/[organization]\?pagelen=100\&q="project.key=\"[PROJECTKEY]\""\&page=$thisPage -u [username]:[password]