gadget.common.error.500

[Deleted user]

Hi, 

I updated Jira to lastest version 8.7, but got this message: gadget.common.error.500

Health check message: 

Warning

Gadget feed URL

What does this check do?

Checks if Jira is able to access itself through the gadgets feed URL to ensure that dashboard gadgets will work.

Result

Jira is not able to access itself through the Gadget feed URL. This is necessary so that dashboard gadgets can be generated successfully. Please verify the current Base URL and if necessary, review your network configurations to resolve the problem.

I have NGINX pointed to Jira server, settings as bellow: 

server.xml:

<Connector port="8080" maxThreads="150" minSpareThreads="25" connectionTimeout="20000" enableLookups="false"
maxHttpHeaderSize="8192" protocol="HTTP/1.1" useBodyEncodingForURI="true" redirectPort="8443"
acceptCount="100" disableUploadTimeout="true" bindOnInit="false" secure="true" scheme="https"
proxyName="jira.domain.xyz" proxyPort="443"/>

in NGINX:

server {
server_name jira.domain.xyz;
listen 443 ssl;

location / {
add_header Front-End-Https on;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

proxy_pass http://jira.server.ip:8080;
proxy_read_timeout 90;

proxy_redirect http://jira.server.ip:8080 https://jira.domain.xyz;
}

ssl_certificate /etc/nginx/ssl/comodo/live/domain.xyz/domain.xyz.fullchain.crt;
ssl_certificate_key /etc/nginx/ssl/comodo/live/domain.xyz/domain.xyz.key;

I tried to check atlassian-jira.log but couldnt find anything special about gadget.

As I understand: Jira is able to access itself  is the cause. Trying curl -v jira.domain.xyz within jira server but not success. (other server work fine)

* Rebuilt URL to: jira.domain.xyz/
* Trying jira.server.ip..
* connect to jira.server.ip port 80 failed: Connection refused
* Failed to connect to jira.server.ip port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to jira.server.ip port 80: Connection refused

Since it worked before updating, I tried copy everything from old server.xml but not success. 

Appreciate any help,

Peter

[MAJ · Accepted solution]

Hi 

try changing the "localhost" to "127.0.0.1" then restart start

[Deleted user]

Hi Max, 

I changed bellow:

JVM_EXTRA_ARGS="-XX:-OmitStackTraceInFastThrow -Djava.locale.providers=COMPAT -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8080 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=8080"

Same error.

Best Regards.

[MAJ]

remove those changes i recommended and post your connection infor from tomcat where you connect via port 8080 also does your firewall permit that port access 

one way to fine out is # firewall-cmd --list-all  port 8080 should be in the ports list.

finally what are the details from executing : # env

thanks

[Deleted user]

Hi Max, 

I used ufw for firewall but I disabled it in case it affect jira: 
#ufw status
Status: inactive

Can see port 8080 is LISTEN by:
# netstat -tulpn | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 32430/java

This is my #env:

# env
SHELL=/bin/bash
TERM=xterm
OLDPWD=/etc/jira/bin
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SUDO_USER=ics-pettra
SUDO_UID=1007
USERNAME=root
MAIL=/var/mail/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
PWD=/etc/jira/logs
LANG=en_NZ.UTF-8
SHLVL=1
SUDO_COMMAND=/bin/bash
HOME=/home/ics-pettra
LANGUAGE=en_NZ:en
LOGNAME=root
LESSOPEN=| /usr/bin/lesspipe %s
SUDO_GID=1007
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/env

I dont know how to get connection info for tomcat, can you help?

Cheers.

[MAJ]

Nothing really stands out to me. Are your running jira as root ?

if so can make sure Jira owns the files in the home and install directories and the

your problem isn't the application you need to connect to outside from the server did you ever ping google from the cmd line? 

if you can't you need to resolve the network issue. 

[Deleted user]

Hi Max, 

I have jira installation directory under: /etc/jira and jira home under: /var/jira

Also already run chown jira:jira -R /etc/jira and chown jira:jira -R /var/jira

I actually run jira under root. But I can see jira is running under jira user: 

# ps -elf | grep jira
0 S jira 2470 1 32 80 0 - 980782 futex_ 01:17 ? 00:04:19 /etc/jira/jre//bin/java -Djava.util.logging.config.file=/etc/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m -Xmx768m -XX:InitialCodeCacheSize=32m -XX:ReservedCodeCacheSize=512m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory -XX:-OmitStackTraceInFastThrow -Djava.locale.providers=COMPAT -Datlassian.plugins.startup.options= -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xloggc:/etc/jira/logs/atlassian-jira-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCCause -XX:+ExplicitGCInvokesConcurrent -Dignore.endorsed.dirs= -classpath /etc/jira/bin/bootstrap.jar:/etc/jira/bin/tomcat-juli.jar -Dcatalina.base=/etc/jira -Dcatalina.home=/etc/jira -Djava.io.tmpdir=/etc/jira/temp org.apache.catalina.startup.Bootstrap start

This is status jira now: 

root@hostname:/etc/jira/bin# systemctl status jira

● jira.service
Loaded: loaded (/etc/init.d/jira; bad; vendor preset: enabled)
Active: active (running) since Tue 2020-02-25 01:17:30 UTC; 12min ago
Docs: man:systemd-sysv-generator(8)
Process: 2312 ExecStop=/etc/init.d/jira stop (code=exited, status=0/SUCCESS)
Process: 2428 ExecStart=/etc/init.d/jira start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/jira.service
└─2470 /etc/jira/jre//bin/java -Djava.util.logging.config.file=/etc/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m -Xmx768m -XX:I

Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: `UOJ
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: [1B blob data]
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: Atlassian Jira
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: Version : 8.7.1
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: [19B blob data]
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: If you encounter issues starting or stopping Jira, please see the Troubleshooting guide at https://docs.atlassian.com/jira/jadm-docs-087/Troubleshooting+
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: Server startup logs are located in /etc/jira/logs/catalina.out
Feb 25 01:17:30 ausy-fe-jira-m1 jira[2428]: Tomcat started.
Feb 25 01:17:30 ausy-fe-jira-m1 runuser[2435]: pam_unix(runuser:session): session closed for user jira
Feb 25 01:17:30 ausy-fe-jira-m1 systemd[1]: Started jira.service.

Ping google before, as well as Atlassian Marketplace

root@hostname:/etc/jira/bin# ping google.com
PING google.com (216.58.200.110) 56(84) bytes of data.
64 bytes from syd09s14-in-f14.1e100.net (216.58.200.110): icmp_seq=1 ttl=56 time=1.29 ms
64 bytes from syd09s14-in-f14.1e100.net (216.58.200.110): icmp_seq=2 ttl=56 time=1.20 ms
64 bytes from syd09s14-in-f14.1e100.net (216.58.200.110): icmp_seq=3 ttl=56 time=1.16 ms

Best Regards.

[MAJ]

try the curl command to the market place

[Deleted user]

curl -v "atlassian.marketplace.com"
* Rebuilt URL to: atlassian.marketplace.com/
* Trying 69.172.201.153...
* Connected to atlassian.marketplace.com (69.172.201.153) port 80 (#0)
> GET / HTTP/1.1
> Host: atlassian.marketplace.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx

...

</script>
</head>
<body>
<noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript>
</body>
</html>
* Connection #0 to host atlassian.marketplace.com left intact

[MAJ]

Look at that! :) 

looks like it's working bring up the system and check the connection within the application. 

also please like my response if all goes well 

thanks 

Happy Tracking! :) 

[Deleted user]

Hi Max, 

I actually could curl marketplace before and the gadget still get error. 

Anyway, I found an solution, thanks for your response about HTTP proxy. I add my proxy in setenv.sh and fixed it. Don't know why it need proxy for it, since that server has internet connection already.

Problem solved.

Thanks Max

[MAJ]

Dear User 

Are you using a proxy to access the internet within your organization? 

If so you will need to make sure the proxy is configure to allow the application to access marketplace.  Compare the setting on the server experiencing this problem with the others that appear to be working fine. 

Good Luck 

[Deleted user]

Hi Max, 

Thanks for your quick response. 

I didn't put any proxy on Server, and it also has internet. How can I test access marketplace exactly? and which file should I compare? 

I tried to transfer everything in server.xml, issue still there. 

Cheers.

[MAJ]

Max,

From the command line in linux execute the following command:

# curl -v "atlassian.marketplace.com"

If your response is an error your connection/application is not getting to the internet. This is a network problem you will need to resolve.

off the top of my head I would recommend the following checks:

• Check your network connection and or restart or reload it.
• /etc/init.d/network restart(Redhat)
• sudo systemctl restart network (CentOS)
• Try pinging something like www.google.com if you get a response then your getting out so you may need to debug using traceroute.

I don’t want to recommend much more beyond this because I’m not a network engineer but start there and let us know.

Kind Regards

[MAJ]

After doing a little more digging, try this: 

Edit the setenv.sh file located in your install directory under /jira/bin/
add the following to the JVM_EXTRA_ARGS
-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -Dhttps.proxyHost=localhost -Dhttps.proxyPort=8080

[Deleted user]

Thanks Max, 

1. curl -v "atlassian.marketplace.com" reply with the web content, seem like internet ok
* Rebuilt URL to: atlassian.marketplace.com/
* Trying 69.172.201.153...
* Connected to atlassian.marketplace.com (69.172.201.153) port 80 (#0)

2. I restart network

3. edit setenv.sh as bellow:

JVM_EXTRA_ARGS="-XX:-OmitStackTraceInFastThrow -Djava.locale.providers=COMPAT -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -Dhttps.proxyHost=localhost -Dhttps.proxyPort=8080"

4. Restart jira

Now they showing different error: gadget.common.config.unavailable

This is the log error I get:

2020-02-24 20:27:02,792+0000 analyticsEventProcessor:thread-1 ERROR admin [c.a.a.client.listener.ProductEventListener] Failed to send analytics event com.atlassian.analytics.api.events.MauEvent
@f46aa3b7
com.atlassian.cache.CacheException: com.atlassian.util.concurrent.LazyReference$InitializationException: javax.xml.stream.FactoryConfigurationError: Provider for class javax.xml.stream.XMLInputFactory cannot be created

...

Caused by: java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Provider com.ctc.wstx.stax.WstxInputFactory not found

I can't understand this log, so if you need anything else, please let me know.

Regards.

[Deleted user]

Also after edit setenv.sh, I have error: 

The Atlassian Marketplace server is not reachable. Available application updates will not appear on this page.