CSV Issue imports encounter "Fatal error during import" w/ validator

We need to import ~500-2000 issues daily from a legacy ticketing system. Many will be duplicates. We created a validator to locate the duplicates (based off their unique summary) and prevent their creation. The import is performed by end-users on our team, using the issue -> 'import from csv' link

We see a "Fatal error during import" everytime we use the validator

2016-05-05 07:51:29,439 INFO - Importer started!
2016-05-05 07:51:29,440 INFO - Engine is running in Import mode
2016-05-05 07:51:29,450 INFO - All issues will be imported to project: TestProject (TEST)
2016-05-05 07:51:29,454 INFO - Creating issue: ExternalIssue
{externalId=autoid-4493924815619197614, summary=CM1, issueType=CM Unknown Subtask}
2016-05-05 07:51:29,693 ERROR - Fatal error during import
java.lang.NullPointerException
at com.atlassian.jira.config.DefaultSubTaskManager.createSubTaskIssueLink(DefaultSubTaskManager.java:582)
at sun.reflect.GeneratedMethodAccessor2522.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:134)
at com.sun.proxy.$Proxy484.createSubTaskIssueLink(Unknown Source)
at sun.reflect.GeneratedMethodAccessor2522.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:58)
at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:62)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:56)
at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:39)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy4201.createSubTaskIssueLink(Unknown Source)
at com.atlassian.jira.plugins.importer.imports.importer.impl.CompatibilityUtil.createSubTaskIssueLink(CompatibilityUtil.java:137)
at com.atlassian.jira.plugins.importer.compatibility.CompatibilityBridgeUtils.createSubTaskIssueLink(CompatibilityBridgeUtils.java:172)
at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.createSubTaskLink(BulkCreateDataImporter.java:326)
at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.doImportSubtasks(BulkCreateDataImporter.java:298)
at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.runImporterEngine(BulkCreateDataImporter.java:175)
at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.doImport(BulkCreateDataImporter.java:148)
at com.atlassian.jira.plugins.importer.imports.importer.impl.ImporterCallable.call(ImporterCallable.java:26)
at com.atlassian.jira.plugins.importer.imports.importer.impl.ImporterCallable.call(ImporterCallable.java:15)
at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:528)
at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:491)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.atlassian.jira.task.ForkedThreadExecutor$ForkedRunnableDecorator.run(ForkedThreadExecutor.java:254)
at java.lang.Thread.run(Thread.java:745)

It doesn't matter if its a subtask, or a new issue its always resulted in the issue.

The validator looks like this,

the validator is a "Validation based on JQL query"
JQL Query:

project = "Project Name" AND summary ~ "%{00000}"


Condition: number of issues returned by the JQL query satisfies a boolean expression, with "Ephemeral number 1" (code

{00058}) storing the number of returned issues
Boolean expression:

{00058} = 0

 

Anybody have any ideas on how we can use this to prevent duplicates without it error out? Ive tried this as an administrator and it throws the same errors as well. We have attempted to import with a clean CSV file thats two lines long to ensure its not related to bad CSV data - and we always face the same issue. 

Thanks! 

5 answers

Hi,

Since JIRA 6.1 If I am correct you have build in validator.

Try to firstly import only issues without substasks and tell us about the result.

 

Chris Fuller Atlassian Team May 08, 2016

 

It doesn't matter if its a subtask, or a new issue its always resulted in the issue.

OP claims to have already done this.

yes, it's been attempted. We have the issue regardless. 100% of the imports are fine without the validator, and 100% fail with the validator. We have tried the admin and user level imports and encounter the same issue. I've also tried a couple different pieces of logic in the Boolean comparison, and queries locating different mapped fields and continue to encounter the fatal error

 

were on Jira 6.4+ right now - do you know of any easy alternatives? We can't map to the issue key for various reasons, or that would have been our go-to

1 vote
Chris Fuller Atlassian Team May 08, 2016

I think that you should report this as a bug.  For whatever reason, it is attempting to create this issue as a subtask (its issue type suggests that this is the correct thing for it to do), and validation succeeds but the actual creation attempt fails.  It then tries to link the issue as a subtask of the parent issue, but since it wasn't actually created, there is nothing to link and it gets an NPE.

Chris Fuller Atlassian Team May 08, 2016
private MutableIssue createIssue(final ExternalIssue ei, final IssueService.CreateValidationResult validationResult) {
   final IssueService.IssueResult issueResult = bridgeUtils.create(issueService, authenticationContext.getUser(), validationResult);
   final MutableIssue issue = issueResult.getIssue();
   if (!issueResult.isValid()) {
      printErrors(validationResult);
   } else {
      log.log("Issue created successfully with Key [%s], %s", issue.getKey(), ei);
      stats.incrementCreated();
      importInfo.setLastCreatedIssue(issue);
   }
   return issue;
}

 

If createIssue fails, it logs the errors (the call to printErrors does that), but in the end returns issue anyway, and that would be null.

 

Chris Fuller Atlassian Team May 08, 2016
if (validationResult.isValid()) {
   String issueKey = null;
   if (isValidationMode) {
      log.log("No problems found with issue '%s'", ei.getSummary());
      stats.incrementCreated();
   } else {
      final MutableIssue subTaskIssue = createIssue(ei, validationResult);
      createSubTaskLink(subTaskIssue, subtaskParentId);
      issueKey = subTaskIssue.getKey();
   }
   saveNecessaryInfo(issueInputParameters, issueKey, ei);
} else {
   printErrors(validationResult);
}

 

So subTaskIssue can be null here, and createSubTaskLink doesn't tolerate a null argument (as well it shouldn't).

 

Chris Fuller Atlassian Team May 08, 2016

Mysteries I didn't solve:

  • I don't see any way to reach this particular stack trace except when subtasks are involved.  If you are also seeing NPEs when creating ordinary issues, then the stack trace is probably different.
  • I can't think of a good reason for the validator to succeed but the actual issue creation to fail.  That may be a bug in JIRA itself, but the best way to investigate this is probably for you to report the problem with example import data for triggering the issue.

Sorry for the late response, I'm too new to post more than once or twice in X hours for this site
I reported this as a bug as you have suggested, https://ecosystem.atlassian.net/browse/JIM-1735 hopefully thats correct
 

 

A standard issue being imported would present the following

2016-05-09 06:07:52,534 INFO - Importer started!
2016-05-09 06:07:52,534 INFO - Engine is running in Import mode
2016-05-09 06:07:52,536 INFO - All issues will be imported to project: TestProject (TEST)
2016-05-09 06:07:52,536 INFO - Creating issue: ExternalIssue{externalId=autoid-5517151439465571060, summary=CM1, issueType=11704}
2016-05-09 06:07:52,624 ERROR - Fatal error during import
java.lang.NullPointerException
	at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.doImportIssues(BulkCreateDataImporter.java:236)
	at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.runImporterEngine(BulkCreateDataImporter.java:173)
	at com.atlassian.jira.plugins.importer.imports.bulkcreate.engine.BulkCreateDataImporter.doImport(BulkCreateDataImporter.java:148)
	at com.atlassian.jira.plugins.importer.imports.importer.impl.ImporterCallable.call(ImporterCallable.java:26)
	at com.atlassian.jira.plugins.importer.imports.importer.impl.ImporterCallable.call(ImporterCallable.java:15)
	at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:528)
	at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:491)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at com.atlassian.jira.task.ForkedThreadExecutor$ForkedRunnableDecorator.run(ForkedThreadExecutor.java:254)
	at java.lang.Thread.run(Thread.java:745)

Thanks for your responses Chris

Chris Fuller Atlassian Team May 09, 2016

Ok, I commented on the issue that you opened.  As I say there, although it is a different code path, it has essentially the same behaviour.  In both cases, validation is passing but the issue does not get created successfully anyway, and this results in createIssue returning a null that doesn't get handled gracefully.  It isn't clear to me what the root cause of all of this is, but there is definitely a bug here.  Even if it turns out that something is wrong with your input data, an NPE is obviously not the right way for it to tell you that. wink

I had this happen once and it was because I confused Epic link with Epic name.  I was importing stories that had an epic link, but had mapped to Epic name.   Validated fine, but upon import JIRA gave me the above error.

0 vote
Chris Fuller Atlassian Team May 08, 2016

What JIRA Version?

6.4x+ Mr. Fuller

Hi,

I was also facing the same issue while importing issues because I was not mapping a mandatory field, once I mapped all mandatory field, issue was resolved for me & I was able to import the issues successfully.

Please check if you are providing all the mandatory fields before importing. Not sure but may be this can help. :)

Suggest an answer

Log in or Join to answer
Community showcase
Emilee Spencer
Published Friday in Marketplace Apps

Marketplace Spotlight: DeepAffects

Hello Atlassian Community! My name is Emilee, and I’m a Product Marketing Manager for the Marketplace team. Starting with this post, I'm kicking off a monthly series of Spotlights to highlight Ma...

67 views 0 3
Read article

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot