How to import large JSON file into JIRA ?

Question

Hello,

I'm migrating from a JIRA SERVER 6.3 to JIRA SERVER 7.3 using Jira Importer Plugin (JIM).
I've created a python script generating a JSON file using REST API v2 calls. The file contains more than 280 projects, 56K issues and its size is about 850Mo.

Once the file is loaded on the server, when clicking on "Begin import", the request is sent and nothing happens. Hours later, the page is still waiting for a server response.

The JIRA SERVER 7.3 JVM memory had been raised to 4096Mo and Attachment file size has also been raised to 2048Mo in order to allow the file import.

I've tried with a 300Mo file, it works fine.
I've tried to monitor the memory during the import process, but it never goes beyond 70% usage when importing the 850Mo file.

Do you have any idea of the reason why the import fails ?

I would like to be able to do a 1 shot-import rather than split the file into smallers files.
Thanks in advance for your attention.

Best regards,
Alex

Claus Schneider (eficode) · Answer

I just ran into this again.. I had a server config of :

WIndows Server 64bit with:

set JVM_MINIMUM_MEMORY=384m
set JVM_MAXIMUM_MEMORY=2048m

I looked at the logs - no signs.. I opened a ticket with Atlassian and they gave me a hint that it might be memory, even though no log signs

I monitored the memory https://localhost:8080/secure/admin/ViewSystemInfo.jspa and I could see it bounced up to 90% and dropped again. It continue for a while and it seems it finally "gave up" ..

I changed the server config to:

set JVM_MINIMUM_MEMORY=2048m
set JVM_MAXIMUM_MEMORY=4096m

and tried again ..

I monitored the memory https://localhost:8080/secure/admin/ViewSystemInfo.jspa and I could see it bounced up to 90% and dropped again. It continue a few time, but then it finally passed to the next stage at it is supposed to do .. I then had JSON errors, but that is an other story and problem

I hope the hint also fixes it for you guys if you run into the problem

Claus Schneider ( Praqma ) · Answer

I have a similar problem, but with far smaller files. Under 10mb seems to work fine..

I increased the file attachment to 100mb/1000mb and then tried an import of ~55mb, but now it just hangs and there is no indication of any activity on the server itself...

Alexandre HUYARD · Answer

Hi Branden,

I thank you very much for your answer.
I'm not concerned by both subjects you mentionned as my file is 850Mo large (less than 2Go then).
The attachment size limit doesn't concern my case and has already been raised to the max.
The problem may be somewhere else (maybe file in-memory loading ?).
I'll try to raise the JVM memory again to see if it changes anything and keep you in touch.

Cheers,

Alex