Missing avatars and attachment issues when migrating from Cloud to Server

Hello,

 

I've just migrated from Jira Software Cloud to Server on a self-hosted instance.  Installation went fine, and the migration (following the Atlassian directions) went okay - issues are there, projects are fine, users migrated, etc.  However, two persistent issues are bothering me, one minor and one less minor:

 

Minor:  Attachment healthcheck seems to be throwing a spurious error.

In the Support tools (JIRA Administration -> System -> Support Tools), the Attachment Healthcheck reports an error ("The attachment (XXXX.jpg) for issue (XXXXX) could not be read.").  In the server logs, the healthcheck for attachments is reporting that the file it's looking for is not there:

 

2017-11-11 13:11:38,521 HealthCheck:thread-5 WARN anonymous 785x35x1 218r8q 88.98.211.66,127.0.0.1 /secure/SetupImport.jspa [c.a.j.p.healthcheck.support.AttachmentHealthCheck] There was a RuntimeException when testing the attachment {0}

com.atlassian.jira.issue.attachment.NoAttachmentDataException: Attachment does not exist in filesystem /var/atlassian/application-data/jira/data/attachments/XXXXX/10000/XXXX-17/27560

at com.atlassian.jira.issue.attachment.store.DefaultFileSystemAttachmentStore.getAttachmentData(DefaultFileSystemAttachmentStore.java:143)

at com.atlassian.jira.issue.attachment.store.DefaultFileSystemAttachmentStore.getAttachment(DefaultFileSystemAttachmentStore.java:125)

at com.atlassian.jira.issue.attachment.BackwardCompatibleAttachmentStore.getAttachment(BackwardCompatibleAttachmentStore.java:160)

at com.atlassian.jira.issue.attachment.BackwardCompatibleAttachmentStore.getAttachment(BackwardCompatibleAttachmentStore.java:206)

at com.atlassian.jira.issue.managers.DefaultAttachmentManager.streamAttachmentContent(DefaultAttachmentManager.java:727)

[...]

However, the files are definitely there on the filesystem, they have correct permissions (jira:jira), they're readable - and they actually show up on the issue in JIRA itself!  (I can pull up the issue in the JIRA GUI and the attachment referenced is there and I can download / open it).  The database seems to have the correct information as well.

Question:  Why is the healthcheck failing when the issue has the attachment correctly linked?

 

Less minor: All of the avatars are gone.

I'm reporting this together as I think it might be related, so:  all of the avatars are missing.  In the server logs, I get similar errors as above:  

2017-11-11 14:40:42,911 http-nio-8090-exec-16 ERROR      [o.a.c.c.C.[.[.0.0.1].[/].[viewUniversalAvatarServlet]] Servlet.service() for servlet [viewUniversalAvatarServlet] in context with path [] threw exception

java.io.FileNotFoundException: /var/atlassian/application-data/jira/data/avatars/filenamehere.png (No such file or directory)

at java.io.FileInputStream.open0(Native Method)

at java.io.FileInputStream.open(FileInputStream.java:195)

at java.io.FileInputStream.<init>(FileInputStream.java:138)

--

    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:8.5.6]

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_60]

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_60]

    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.5.6]

    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]

    , referer=http://35.189.112.121/projects/PROJID/issues/ISSUEID?filter=allopenissues, servletErrorMessage=}

Again, the files are correctly named and exist on the filesystem (under /var/atlassian/application-data/jira/data/avatars).  They are readable and owned by the jira user.

The import did not copy over either the avatar or attachment directories, so I had to do a manual move on the server from the import directory, but I don't see why JIRA refuses to see the files on the filesystem and use them.

Question: Why are the avatars not being read from disk?

Can anyone give me a steer on these issues?

 

1 answer

This widget could not be displayed.

Hi Steven,

The Cloud backups of Jira contain the XML backup zip file and in addition to those the contents of both the /data/attachments/ and /data/avatars/ folders.  However Jira Server when it creates backups, does not bundle these two subdirectories in.   You have to migrate them separately.

As a result, it might be necessary in some cases to restore the /data/ directories flat files BEFORE you restore the XML backup.   I say might because this kind of a migration, from Cloud to Server, can cause Jira Server to have to execute different upgrade/downgrade tasks.   It's possible that the filenaming convention and/or file subdirectory name convention can change between versions of Jira.   We tend to see this with Jira Server upgrades that jump a large number of revisions at once.   But I have not yet encountered this same kind of problem with a Cloud migration.  

What can happen is that Jira has references in the database for each attachment, when an upgrade/downgrade occurs, these filename references in the database get renamed, and if the files are on the disk in the correct path at that time, they too get renamed.  But if those files do not exist in the correct path at the time of the upgrade, the references get broken.   Adding the files in after the fact does not help because we don't have a means to re-trigger that upgrade task to run again to rename the files on disk correctly.  But if this is happening, then the filename in the error is what Jira is expecting, but the filenames on the disk might actually be different.

I'd like to better understand your specific situation here.  The healthcheck warnings are there to help you know if something is wrong, but they are not 100% infallible.  If you're attachments are working correctly, but the logs still generate this error, it might be safe to ignore, but I'd want to check on the specific files in order to make sure they are working and located in the specific directory.   A restart of Jira might help reset this healthcheck warning too.

As for the avatars, I would want to scrutinize the specific filepath Jira is looking for these, and compare it against the filesystem to make sure that file exists there as expected.

Adding these in after the fact, while Jira is running might not be enough to stop this warning.  It's possible you might have to restart Jira in order to reset this healthcheck warning as well.

I'd also want to know exactly which document you followed for this migration process.   We have a couple of different guides out there and I'm concerned if we don't have the exact process to follow documented correctly we might be causing other users to have this same problem.  But also migrations between server and cloud can change due to recent changes in the Cloud platform.  So if we have incorrect docs out there, I want to know so we can take the steps to correct them as soon as possible.

It might be worth spinning up a brand new Jira instance, restoring those files in place before you import the XML backup to see if this is one way to avoid this problem.

Regards,
Andy

Suggest an answer

Log in or Sign up to answer
Atlassian Summit 2018

Meet the community IRL

Atlassian Summit is an excellent opportunity for in-person support, training, and networking.

Learn more
Community showcase
Posted Wednesday in New to Jira

Are you planning to trial, or are currently trialling Jira Software? - We want to talk to you!

Hello! I'm Rayen, a product manager at Atlassian. My team and I are working hard to improve the trial experience for Jira Software Cloud. We are interested in   talking to 20 people planning t...

292 views 5 0
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you