Observation: IssueImpl (MutableIssue) consumes 6 times less memory than DocumentIssueImpl

[Daniel Garcia]

Just an interesting (unexpected) observation

Script run in scriptrunner console. It executes pretty slowly because of the System.gc() calls

/**
* Best to run in test environment
*/

import com.atlassian.jira.bc.issue.search.SearchService
import com.atlassian.jira.component.ComponentAccessor
import com.atlassian.jira.issue.issuetype.IssueType
import com.atlassian.jira.issue.search.SearchResults
import com.atlassian.jira.user.ApplicationUser
import com.atlassian.jira.web.bean.PagerFilter
import com.atlassian.query.Query
import com.atlassian.jira.issue.Issue
import com.atlassian.jira.issue.MutableIssue

import java.lang.management.ManagementFactory
import java.lang.management.MemoryMXBean
import java.lang.management.MemoryUsage
import groovy.transform.Field

import static com.atlassian.jira.component.ComponentAccessor.getConstantsManager
import static com.atlassian.jira.component.ComponentAccessor.getIssueManager

class IssueSearch {
static List<Issue> findAll(String jql, ApplicationUser user=null, boolean ignorePermissions=false, Integer maxResults=null) {
SearchService searchService = ComponentAccessor.getComponent(SearchService)
if (user==null) {
user = ComponentAccessor.jiraAuthenticationContext?.getLoggedInUser()
}

SearchService.ParseResult parsedResult = searchService.parseQuery(user, jql)
if (!parsedResult.isValid()) {
throw new Exception("###### INVALID JQL QUERY: ${jql} ######")
}

findAll(parsedResult.query, user, ignorePermissions, maxResults)
}

static List<Issue> findAll(Query query, ApplicationUser user=null, boolean ignorePermissions=false, Integer maxResults=null) {
SearchService searchService = ComponentAccessor.getComponent(SearchService)
if (user==null) {
user = ComponentAccessor.jiraAuthenticationContext?.getLoggedInUser()
}

PagerFilter filter = maxResults==null ? PagerFilter.getUnlimitedFilter() : new PagerFilter(maxResults)
SearchResults results = ignorePermissions ? searchService.searchOverrideSecurity(user, query, filter) : searchService.search(user, query, filter)
// call collect to convert from UnmodifiableRandomAccessList to Arraylist
results.getResults().collect()
}
}

@Field long start = System.currentTimeMillis()
@Field StringBuilder out = new StringBuilder()
@Field MemoryMXBean mbean = ManagementFactory.getMemoryMXBean()

void output(String text) {
long elapsed = System.currentTimeMillis() - start
out << "${elapsed.toString().padLeft(8)} ${text}\n"
}

long measureConsumed(Closure closure) {
System.gc()
MemoryUsage before = mbean.getHeapMemoryUsage();

Object obj = closure()

System.gc()
MemoryUsage after = mbean.getHeapMemoryUsage()
long consumed = after.getUsed() - before.getUsed()

consumed
}

out << "<pre class=\"dark\">\n"

List<Long> consumed = []
10.times{
consumed.add(measureConsumed{
//List<Issue> issues = IssueSearch.findAll("issuetype = \"Task\"")
List<MutableIssue> issues = IssueSearch.findAll("issuetype = \"Task\"").collect{issueManager.getIssueObject(it.key)}
output("${issues.size()} issues (${issues[0].class})")
issues
})
}

out << "\n"
output("Results:")
consumed.each{output(String.format("%,d", it))}


out << "</pre>"
out.toString()

first execution retrieved 8469 issues as DocumentIssueImpl (line 78 uncommented)

    2929   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
    8808   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   14516   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   20244   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   25987   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   31850   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   37586   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   43301   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   49042   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)
   54773   8469 issues (class com.atlassian.jira.issue.DocumentIssueImpl)

   57646   Results:
   57647   144,305,192
   57647   142,606,336
   57647   144,703,488
   57647   144,179,200
   57647   143,654,912
   57648   123,731,968
   57648   144,213,720
   57648   143,654,912
   57648   143,654,912
   57648   142,082,048

second execution retrieved 8469 issues as MutableIssue (line 79 uncommented)

    9889   8469 issues (class com.atlassian.jira.issue.IssueImpl)
   24379   8469 issues (class com.atlassian.jira.issue.IssueImpl)
   36867   8469 issues (class com.atlassian.jira.issue.IssueImpl)
   50675   8469 issues (class com.atlassian.jira.issue.IssueImpl)
   63914   8469 issues (class com.atlassian.jira.issue.IssueImpl)
   78034   8469 issues (class com.atlassian.jira.issue.IssueImpl)
   90863   8469 issues (class com.atlassian.jira.issue.IssueImpl)
  104211   8469 issues (class com.atlassian.jira.issue.IssueImpl)
  117822   8469 issues (class com.atlassian.jira.issue.IssueImpl)
  131244   8469 issues (class com.atlassian.jira.issue.IssueImpl)

  134410   Results:
  134410   24,117,248
  134410   19,398,656
  134410   21,495,808
  134410   20,971,520
  134410   25,165,824
  134411   -524,288
  134411   28,963,328
  134411   21,495,808
  134411   22,020,096
  134411   21,031,936

second execution used much less memory but was twice as slow

Of course, to create a list of MutableIssues you need to have the list of DocumentIssueImpl first, so the peak memory usage would be similar. With that many issues best to chunk it up by pages. Maybe I can create a wrapper that returns a stream and deals with the paging internally