Duplicate query results by using Organization groovy API

Question

Hi,

I'm using Scriptrunner in our Jira Service Management Datacenter setup to manage Organizations.

When I run the script (see below) the Organization query returns duplicates i.e. the same Organization is returned on different pages in response' pagedresult. When I run the script multiple times consequently, then usually first couple of times I do get duplicates, but then Jira returns duplicate-less response.

What could be the root cause here?

Kind Regards,

Andrey

import javax.ws.rs.core.MultivaluedMap 
import static groovyx.net.http.ContentType.*

import org.apache.log4j.Level
import org.apache.log4j.Logger

import com.atlassian.jira.component.ComponentAccessor
import com.atlassian.servicedesk.api.organization.CustomerOrganization
import com.atlassian.servicedesk.api.organization.OrganizationService
import com.onresolve.scriptrunner.runner.customisers.WithPlugin
import com.atlassian.servicedesk.api.util.paging.SimplePagedRequest
import com.atlassian.servicedesk.api.util.paging.LimitedPagedRequestImpl

@WithPlugin("com.atlassian.servicedesk")

// set up a logger
def lgr = Logger.getLogger("GettingOrgs")
lgr.setLevel(Level.DEBUG)

def isLastPageReached = false
def startIndex = 0
ArrayList jsdOrgsList = []
def infiniteLoopPrevention = 10000
def duplicatesCounter = 0

def adminUser = ComponentAccessor.userManager.getUserByKey('JIRAUSER10300')
def organisationService = ComponentAccessor.getOSGiComponentInstanceOfType(OrganizationService)

while (!isLastPageReached && (infiniteLoopPrevention-- >0)) {

def organizationQuery = organisationService
.newOrganizationsQueryBuilder()
.serviceDeskId(1)
.pagedRequest(SimplePagedRequest.paged(startIndex, 6000))
.build()

def pagedRes = organisationService.getOrganizations(adminUser, organizationQuery)

pagedRes.getResults().each { pagedResElement->

if (jsdOrgsList.find{it.jsdId == pagedResElement.id} == null) {
jsdOrgsList.add([
jsdId: pagedResElement.id
])

} else {
duplicatesCounter++
}
}

startIndex +=50
isLastPageReached = !pagedRes.hasNextPage()
}

lgr.debug("duplicatesCounter = ${duplicatesCounter}")
lgr.debug("Unique orgs count on all pages = ${jsdOrgsList.size()}")
def sumtotal = duplicatesCounter + jsdOrgsList.size()
lgr.debug("Sum of unique count and duplicates = ${sumtotal}")

Andrey Kravtsov · Answer

Hello Jeroen,

thank you for your response. I have tested the query with different values like 50, 6000, 30... The result is always the same: duplicates sometimes occur.

So far, I've found an workaround: I've put the query into a loop with a threshold on iterations count. After a couple of iterations, jira returns the Orgs without duplicates. It means, the same code returns for the same data different results. This lets us assume, that we have some caching / nodes synchronization issue.

Regards,

Andrey

Jeroen Poismans · Answer

Hell Andrey,

I think I see the error:

.pagedRequest(SimplePagedRequest.paged(startIndex, 6000))

You ask the api to retrieve 6000 rows, starting from startIndex, but the limit for each paged request is 50, so the api only retrieves 50.

Try changing it it to:

.pagedRequest(SimplePagedRequest.paged(startIndex, 50))

Let me know if this helped!

Regards,

Jeroen