Pagination not working when using search API with text searches

[webmethods-integration]

We are using search API of confluence cloud(https://developer.atlassian.com/cloud/confluence/advanced-searching-using-cql/).

We are performing text searches with the above API.

Example of our query is below:

https://mydomain/wiki/rest/api/search?limit=2&start=1&cql=type=page and space='TS' and title='test'

We have already have 7 records matching when we use only limit=10, one first call limit=2&start=0, I am getting 2 records but when I use limit=2&start=1 or limit=2&start=2 or 'start' more than 0, I am getting empty array in results of the API. Ideally we should get the paginated records in the results of the API, but from API we are not getting proper response.

Does Anybody faced the similar issue or had a solution for the above problem?

Thanks.

WMIO

[Michele Liberman]

Atlassian have removed the start option in server and cloud now and now you need to loop on the next link in the response. I've created a python function based on atlassian-python-api's to replace get_all_pages_by_label():

    def get_all_pages_by_label(self, label):

        """

        Get all page by label

        :param label:

        :return:

        """

        url = "rest/api/content/search"

        limit = 200

        params = {}

        results = []

        if label:

            params["cql"] = 'type={type} AND label="{label}"'.format(type="page", label=label)

        if limit:

            params["limit"] = limit # product max

        try:

            response = self.get(url, params=params)

            results.extend(response.get("results"))

            # loop on the results until there's nothing more to get

            while "next" in response.get("_links"):

                response = self.get(response.get("_links")["next"], params=params)

                results.extend(response.get("results"))

        except HTTPError as e:

            if e.response.status_code == 400:

                raise ApiValueError("The CQL is invalid or missing", reason=e)

            raise

        return results

[Andrés Guerrero]

Same case here ...   is there a bug open for this? 

[Ture Hoefner]

EDIT: sorry, I'm using Confluence SERVER search API.  I'll leave this post here in case other server devs end up here.  Pagination of the search API is hard to use and broken in some cases:

I'm using the search REST API and the "next" link in the results from my first call doesn't work.  It returns HTTP 400 Bad Request, saying that the "CQL query parameter is required but was empty".

I think it is odd that the search API (first request, no pagination params specified) works if an "=" that is contained in the value of the cql param is not URL encoded (like /rest/api/content/search?cql=macro=multiexcerpt). 

It also works if the "=" IS URL encoded (like /rest/api/content/search?cql=macro%3Dmultiexcerpt) which I would expect.  Why would the API allow unencoded cql parameter values?  That is so odd and feels like it is never going to work.

UPDATE:

This works for me (SERVER API).  I worked around any oddness in the API for parsing the cql parameter value by always encoding it and specifying it separate from the parameters for pagination:

1st request:

curl --user username:password -G "https://my.domain.net/rest/api/content/search" --data-urlencode "cql=macro=multiexcerpt"

2nd request:

curl --user username:password -G "https://my.domain.net/rest/api/content/search?start=25&limit=25" --data-urlencode "cql=macro=multiexcerpt"

[Zach W]

We're running into a similar issue where https://our-subdomain.atlassian.net/wiki/rest/api/search?cql=label=ourlabel&limit=10&start=10&expand=content.metadata.labels,content.history.lastUpdated

returns the same value regardless of what start is set to. We have 72 results listed, but we can't access them for some reason by changing start.

[Cengiz Beytas]

I'm seeing this same issue with Confluence Cloud version 1000.0.0-f582da98b37a.

I increase the start value and the CQL JSON reply shows the correct start value but the content is always the same. I can only access the first 200 results no matter what start is set to.

Is there a solution or workaround for this?

[Maksim]

For anybody else stumbling upon this issue, it looks like Atlassian finally disabled the "start" parameter in the cloud too. This is the issue that helped me find the deprecation notice. I could not find the deprecation notice in the Cloud Changelog.

The gist of it is that one must now use cursor/"next"-link instead of "start" parameter.