Sporadically unable to attach relatively large files

[Jason Adam]

Hello!

We're running JIRA Core 7.6.0. Attachment size is the default 10MB.

When trying to attach files larger than 100KB, we are sporadically getting either "Internal Error" or "Could not save data from stream" errors. Doesn't matter the file type, and the same file can sometimes be attached fine but then a few minutes later throw one of these errors.

Servers were pointed to Java 8r162 and Java 8r172, no change.

This happens either in Firefox 52 or 60, or IE 11.

This is on Window 7 and Windows 2012 R2 Server.

In two of the environments we tested on, firewalls and reverse proxies were not used, the machines accessing JIRA were all in the same enclave. I also don't think this would be an issue anyway since sometimes large files (about 9MB) can be attached if we're lucky.

What I've tried so far based on other posts with similar issues, none of which fixed the issue:

Deleted the mimeTypes.rdf

Reinstalled both Firefox and IE

Turned on logging for the attachment package and all the HTTP loggers but all I see are HTTP 500 errors when this happens, no other log entries.

Checked permissions and everything is good, even tried opening it to Everyone.

I've also tried having JIRA point to a local folder since normally we use a symbolic link to a file share, but it doesn't matter.

The only thing I can possibly find in the HTTP dump log is that each time I try to attach a file, whether it works or not, interspersed in the HTTP messages is a JSON object and the final entry has an error on field __I18N__ saying "You don't have permissions to edit this field". Again, this happens whether the attachment works or not but maybe it's a threading issue sometimes causing an upload error? Probably just a red herring but that's the only thing other the 500 errors I can as an issue.

Would garbage collection or any other process be happening to interfere with this?  I would think given the regular nature of our failure to attach, it wouldn't be GC unless it's running quite frequently. 

Any help would be greatly appreciated. I have an HTTP dump log if needed.

[Jason Adam · Accepted solution]

Well, I discovered that when using IE's debug console, each time an attachment fails I get a XMLHttpRequest: Network Error 0x2eff (or 0x2efe) error.  Firefox isn't showing this, but guessing it's the same issue behind the scenes.

In looking for this error related to JIRA, I found this link: https://community.atlassian.com/t5/Jira-Core-questions/Communications-breakdown-error-on-workflow-transition/qaq-p/96449

Different issue, but possibly same root cause, which is the pipe character in the URL string when attaching files is for some reason causing an issue with our network and security topology. 

My question is, how can I go about using URL encoding for the pipe to see if that fixes the problem?  Is there a js file that I can change that builds the URL string where I can switch or replace the pipe on? 

[Jason Adam]

So I found the InlineAttach.js and did some replaces on the URL going to AJAX.  No go.  Looking in IE's network monitor I can see the POST no long has pipes but I'm still getting the same behavior.

@Earl McCutcheonAny help here would be greatly appreciated.  While most of the time our users will be attaching smallish files, this periodic inability to attach even a moderately sized file is exceptionally annoying.

[Earl McCutcheon]

Hi Jason,

Good catch, and that error looks like the following BUG, relating to the pipe character and Network configurations primarily any type of load balancer or Proxy that does not accept the character in the URL being RFC-2396 aware:

• https://jira.atlassian.com/browse/JRASERVER-33520

This should be fixed in Jira Versions 7.11.0, 7.10.1, & 7.6.7 if this is the case,  But I would also recommend trying to Bypass your Proxy or load balancer to confirm if this is indeed the hangup.

Regards,
Earl

[Jason Adam]

Awesome thanks @Earl McCutcheon!  I upgraded to 7.11.0 and have seen no errors with attachments.  I did see a couple of times that the larger attachments would hang for a couple of seconds waiting to show progress, and usually that would indicate it was about to fail, but the upload succeeded.  So woot! Think we're in the clear.

Thanks again for the responses.

[Earl McCutcheon]

Hi Jason,

Thats Great to hear and thanks for confirming that was the root cause (HighFive)

Regards,
Earl

[Earl McCutcheon]

Hello Jason,

On the Surface it sounds like a file system permission conflict as covered in this KB article:

• JIRA throws "Could not save attachment data from stream" when attempting to add attachments to issue

But you did note:

"Checked permissions and everything is good, even tried opening it to Everyone."

and I just wanted to confirm if you set the permissions on the filesystem level or the Jira Project permissions?

Regards,
Earl

[Jason Adam]

I believe it is set on both the project  directory and the directory the attachments are saved to, but I will check again (currently unable to access any of the environments)

if we’re able to save attachments sometimes but not others, would it really be permissions?  I would think all attachments would fail if permissions weren’t correct, versus what we’re seeing 

[Jason Adam]

Checked the permissions and the service account running the Atlassian JIRA service has full control both to the share that we write attachments to and to the entire folder structure that JIRA is installed to (both the Application Data and the JIRA folders). 

Any other thoughts on what this can be?  Again, this is periodic, so I don't think this is going to be a permissions issue or a setting on a proxy.  Those would either be an all or none issue.  I'm thinking there is some processing issue going on since we seem to be seeing the likelihood of an attachment failure being directly proportional to the size of the file.