Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in
Celebration

Earn badges and make progress

You're on your way to the next level! Join the Kudos program to earn points and save your progress.

Deleted user Avatar
Deleted user

Level 1: Seed

25 / 150 points

Next: Root

Avatar

1 badge earned

Collect

Participate in fun challenges

Challenges come and go, but your rewards stay with you. Do more to earn more!

Challenges
Coins

Gift kudos to your peers

What goes around comes around! Share the love by gifting kudos to your peers.

Recognition
Ribbon

Rise up in the ranks

Keep earning points to reach the top of the leaderboard. It resets every quarter so you always have a chance!

Leaderboard

Come for the products,
stay for the community

The Atlassian Community can help you and your team get more value out of Atlassian products and practices.

Atlassian Community about banner
4,465,926
Community Members
 
Community Events
176
Community Groups

Safe way to query JQL in groovy script

I've been challenged recently with some performance issues on my environment.

While doing research, I came across this Kbase article and as I was reading the "Cause" section mention the use of PagerFilter.getUnlimitedFilter() I went: "Hey this looks familiar, I do this all the time!"

Sure enough, I probably got that example from a place like this Adaptavist Library Script.

So what should one do to use JQL in a script while not risk blowing up their heap space with an uncontrolled query?

Here is my solution...

I created a short IssueSearchHelper groovy class that I can leverage in my other scripts:

package path.to

import com.atlassian.jira.bc.issue.search.SearchService
import com.atlassian.jira.component.ComponentAccessor
import com.atlassian.jira.user.ApplicationUser
import com.atlassian.jira.web.bean.PagerFilter
import groovy.util.logging.Log4j
import org.apache.log4j.Level

@Log4j
class IssueSearchHelper {
final Integer MAX_PAGER_SIZE = 200
final Integer DEFAULT_PAGER_SIZE = 50

ApplicationUser user = ComponentAccessor.jiraAuthenticationContext.loggedInUser
SearchService searchService = ComponentAccessor.getComponent(SearchService)
Integer pageSize = DEFAULT_PAGER_SIZE
Integer maxIterations

static void setLogLevel(def level) {
//quick way to turn on logging
if (level instanceof String) log.setLevel(Level.toLevel(level))
if (level instanceof Level) log.setLevel(level)
}

void setPageSize(Integer size) {
if (!size) size = DEFAULT_PAGER_SIZE
//don't let anyone attempt to get more than the MAX_PAGER_SIZE per page
this.pageSize = (size > MAX_PAGER_SIZE) ? MAX_PAGER_SIZE : size
}

void setUser(ApplicationUser user) {
//default the user when user:null is specified
this.user = user ?: ComponentAccessor.jiraAuthenticationContext.loggedInUser
}

Long getSearchCount(String jql) {
def jqlParseResult = searchService.parseQuery(user, jql)
assert jqlParseResult.isValid(), jqlParseResult.errors

searchService.searchCount(user, jqlParseResult.query)
}

def withJql(String jql, Closure closure) {
def jqlParseResult = searchService.parseQuery(user, jql)
assert jqlParseResult.isValid(), jqlParseResult.errors
def issueCount = searchService.searchCount(user, jqlParseResult.query) as Integer
if(!issueCount) return null
Integer localMax = maxIterations ?: issueCount
Integer localPageSize = pageSize
if (localMax < pageSize) {
localPageSize = localMax
}
//we'll chunk the results from the end in case the closure causes issue to drop from the search
log.info "Creating an initial pager with for the last $localPageSize issues out of $issueCount"
def start = (issueCount as Integer) - localPageSize
def pager = new PagerFilter(start, localPageSize)
Integer iterationCount = 0
while (true) {
log.debug "Retrieving 1 page of results from $pager.start to $pager.end"
def results = searchService.search(user, jqlParseResult.query, pager).results
results.each {
def issue = ComponentAccessor.issueManager.getIssueObject(it.id)
log.trace "Obtained a Mutable issue from search result: $issue.key"
log.trace "Calling the supplied closure"
closure.call(issue)
}
iterationCount += localPageSize
if (pager.start == 0 || localMax && iterationCount >= localMax) {
break
}
log.trace "Creating another pager starting at $pager.previousStart with a size of $pager.max"
def remainingIterations = localMax - iterationCount
if (remainingIterations < localPageSize) {
pager = new PagerFilter(pager.start - remainingIterations, remainingIterations)
} else {
pager = new PagerFilter(pager.previousStart, localPageSize)
}
}
}
}

This encapsulates anything you need to do to perform repeatable actions on a number of issues.

It takes care of grabbing the necessary components and validating your JQL.

Here are a few examples of how you can instantiate the helper class:

import path.to.IssueSearchHelper

//get a default search helper with currentUser, default page size, unlimited results
def defaultSearchHelper = new IssueSearchHelper()

//get a SearchHelper where the issues are retreived based on a specific user permission
def someUser = ComponentAccessor.userManager.getUserByName('sample_username')
def userSearchHelper = new IssueSearchHelper(user:someUser)

//specify the maximum number of issue to process regardless of how many are present
def limitedSearchHelper = new IssueSearchHelper(maxIterations:10)

//specify the size of each page to process
def pageSizeSearchHelper = new IssueSearchHelper(pageSize:100)

//all paramters combined
def allArgsSearchHelper = new IssueSearchHelper(pageSize:100, maxIterations:250, user:somUser)

Here is how you would actually use it once instantiated:

import com.atlassian.jira.component.ComponentAccessor
import com.atlassian.jira.issue.Issue
import path.to.IssueSearchHelper

def is = ComponentAccessor.issueService

//get a default search helper with currentUser, default page size, unlimited results
def issueSearchHelper = new IssueSearchHelper()

//you can re-use the user variable defaulted by the helper, I'll use that below in the issueService call
def user = issueSearchHelper.user

def jql = 'Assignee = currentUser() and Updated > -1d'

//just return the total number of issue that match the JQL. maxIterations and pageSize are ignored
def count = issueSearchHelper.getSearchCount(jql)

//perform the same operation on each issues in the search
issueSearchHelper.withJql(jql){Issue issue->
log.info "Found issue $issue.key"

//make a random update
def iip = is.newIssueInputParameters()
iip.summary = issue.summary + " - updated"
def validationResult = is.validateUpdate(user, issue.id, iip)
assert validationResult.valid : validationResult.errorCollection.errors
is.update(user, validationResult)
}

Here is another example where you can use the helper in the middle of other code when you need to iterate over your issues to build something else, like a table:

import com.atlassian.jira.issue.Issue
import path.to..IssueSearchHelper
import groovy.xml.MarkupBuilder

def issueSearchHelper = new IssueSearchHelper()

def writer = new StringWriter()
def builder = new MarkupBuilder(writer)
builder.table(class:'aui'){
def jql = 'updated > -1d'
issueSearchHelper.withJql(jql){Issue issue->
tr{
td issue.key
td issue.summary
}
}
}
writer.toString()

This last example may still give you memory problems because the writer variable will still grow based on the number of issues your JQL returns, but at least you won't be ALSO filling a SearchResults object with all of the issues.

It's still possible to get yourself in trouble if you try hard enough:

def jql = 'created > 1970-01-01'

def allIssues = []
issueSearchHelper.withJql(jql){Issue issue->
allIssues << issue
}
//allIssues will be just as big as if you had used PagerFilter.unlimitedFilter

Now, it turns out, my performance problems probably had nothing to do with my scripted jql queries ... but now I know better than to risk it by using getUnlimitedFilter()

Have fun scripting :)

1 comment

For anyone watching this... I encountered an infinite loop situation because my supplied closure was updating issues such that they disappeared from the JQL. 

I edited the code in the original post with a fix. The new version will process the issues in the search in reverse order. Also, there is an exit condition that will be encountered no matter what.

Comment

Log in or Sign up to comment
TAGS

Atlassian Community Events