Connection problem with MySQL 5.7: (Got an error reading communication packets)

Armen_Mkrtchyan June 7, 2020

Hello,

I have set up of Atlassian Jira, Confluence and Bitbucket using docker containers. The initial configuration was on CentOS 7. After 6 month of working without any problem recently i have restarted server and now i am getting:

[Note] Aborted connection 2 to db: 'jiradb' user: 'jira' host: '172.18.0.3' (Got an error reading communication packets)

Same true for confluence instance. I have tried different recommendations regarding above mentioned error, but no one helps.
I have tried also following which did not help:
1. Make fresh install using CentOS 8 and using latest stable release of Docker
2. Export jira database using mysqldump and load it to the new installed mysql container

Below is my configuration details:


# cat /etc/redhat-release
CentOS Linux release 8.1.1911 (Core)

# docker version
Client: Docker Engine - Community
Version: 19.03.11
API version: 1.39
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:13:48 2020
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 18.09.1
API version: 1.39 (minimum version 1.12)
Go version: go1.10.6
Git commit: 4c52b90
Built: Wed Jan 9 19:06:30 2019
OS/Arch: linux/amd64
Experimental: false

my.cnf file:

[mysqld]
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
character-set-server=utf8
collation-server=utf8_bin

max_allowed_packet=2048M
innodb_log_buffer_size=32M
innodb_log_file_size=2GB

query-cache-size = 8M
query-cache-type = 1
query_cache_limit = 8M

join-buffer-size = 8M
#table-cache = 1024
table-definition-cache = 1024

open-files-limit = 4096

bind-address = 0.0.0.0
transaction-isolation=READ-COMMITTED

connect_timeout = 6000
net_read_timeout = 6000
net_write_timeout = 6000

 docker-compose.yml file:

version: '3'

services:

database:
image: mysql:5.7
restart: always
container_name: mysql
volumes:
- database-data:/var/lib/mysql
- /data/mysql/my.cnf:/etc/my.cnf
networks:
- atl-net
environment:
TZ: "America/Toronto"

jira:
depends_on:
- database
image: atlassian/jira-software
restart: always
container_name: jira
volumes:
- jira-data:/var/atlassian/application-data/jira
- ./mysql-connector-java-8.0.20.jar:/opt/atlassian/jira/lib/mysql-connector-java-8.0.20.jar
networks:
- atl-net
ports:
- "8080:8080"
environment:
- TZ="America/Toronto"
- JVM_MINIMUM_MEMORY=2048m
- JVM_MAXIMUM_MEMORY=2048m
- ATL_JDBC_URL=jdbc:mysql://database:3306/jiradb?useUnicode=true&characterEncoding=UTF8&sessionVariables=default_storage_engine=InnoDB
- ATL_JDBC_USER=jira
- ATL_JDBC_PASSWORD=XXXXXXX
- ATL_DB_DRIVER=com.mysql.jdbc.Driver
- ATL_DB_TYPE=mysql
- ATL_PROXY_NAME=jira.xxxx.com
- ATL_PROXY_PORT=443
- ATL_TOMCAT_PORT=8080
- ATL_TOMCAT_SCHEME=https
- ATL_TOMCAT_SECURE=true

confluence:
depends_on:
- database
- jira
image: atlassian/confluence-server
restart: always
container_name: confluence
volumes:
- confluence-data:/var/atlassian/application-data/confluence
- ./mysql-connector-java-8.0.20.jar:/opt/atlassian/confluence/confluence/WEB-INF/lib/mysql-connector-java-8.0.20.jar
networks:
- atl-net
ports:
- "8090:8090"
environment:
- TZ="America/Toronto"
- JVM_MINIMUM_MEMORY=2048m
- JVM_MAXIMUM_MEMORY=2048m
- ATL_JDBC_URL=jdbc:mysql://database/confluencedb?sessionVariables=default_storage_engine%3DInnoDB&useUnicode=true&characterEncoding=utf8
- ATL_JDBC_USER=confluence
- ATL_JDBC_PASSWORD=XXXXX
- ATL_DB_DRIVER=com.mysql.jdbc.Driver
- ATL_DB_TYPE=mysql
- ATL_PROXY_NAME=confluence.xxxx.com
- ATL_PROXY_PORT=443
- ATL_TOMCAT_PORT=8090
- ATL_TOMCAT_SCHEME=https
- ATL_TOMCAT_SECURE=true
- ATL_TOMCAT_MAXTHREADS=50

bitbucket:
depends_on:
- database
- jira
image: atlassian/bitbucket-server
restart: always
container_name: bitbucket
volumes:
- bitbucket-data:/var/atlassian/application-data/bitbucket
networks:
- atl-net
ports:
- "7990:7990"
- "7999:7999"
environment:
- TZ="America/Toronto"
- JVM_MINIMUM_MEMORY=1024m
- JVM_MAXIMUM_MEMORY=1024m
- JDBC_URL=jdbc:mysql://database:3306/stashdb?characterEncoding=utf8&useUnicode=true
- JDBC_USER=stash
- JDBC_PASSWORD=XXXX
- JDBC_DRIVER=com.mysql.jdbc.Driver
- SERVER_PROXY_NAME=bitbucket.xxxx.com
- SERVER_PROXY_PORT=443
- SERVER_SCHEME=https
- SERVER_SECURE=true


networks:
atl-net:

volumes:
jira-data:
driver: local
driver_opts:
type: none
device: /data/atlassian/application-data/jira
o: bind
confluence-data:
driver: local
driver_opts:
type: none
device: /data/atlassian/application-data/confluence
o: bind
bitbucket-data:
driver: local
driver_opts:
type: none
device: /data/atlassian/application-data/bitbucket
o: bind
database-data:
driver: local
driver_opts:
type: none
device: /data/mysql/mysql
o: bind

Bigger part of log file. Issued command is:
docker-compose up database jira

.....
.....
jira | 07-Jun-2020 07:38:07.368 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
jira | 07-Jun-2020 07:38:07.380 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 436 ms
jira | 07-Jun-2020 07:38:07.387 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
jira | 07-Jun-2020 07:38:07.387 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.50
jira | 07-Jun-2020 07:38:07.396 SEVERE [Catalina-startStop-1] org.apache.catalina.startup.HostConfig.beforeStart Unable to create directory for deployment: [/opt/atlassian/jira/conf/Catalina/localhost]
mysql | 2020-06-07T07:38:07.917100Z 0 [Note] InnoDB: Buffer pool(s) load completed at 200607 3:38:07
jira | 2020-06-07 07:38:10,243+0000 localhost-startStop-1 INFO [c.a.jira.startup.JiraHomeStartupCheck] The jira.home directory '/var/atlassian/application-data/jira' is validated and locked for exclusive use by this instance.
jira | 2020-06-07 07:38:10,293+0000 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
jira |
jira | ****************
jira | Jira starting...
jira | ****************
jira |
jira | 2020-06-07 07:38:10,426+0000 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]
jira |
jira | ___ Environment _____________________________
jira |
jira | JIRA Build : 8.9.0#809000-sha1:4ceb90abd8e813f4565a1705e597aeab0a82fc50
jira | Build Date : Mon May 18 00:00:00 GMT 2020
jira | JIRA Installation Type : Standalone
jira | Application Server : Apache Tomcat/8.5.50 - Servlet API 3.1
jira | Java Version : 1.8.0_252 - AdoptOpenJDK
jira | Current Working Directory : /var/atlassian/application-data/jira
jira | Maximum Allowable Memory : 1963MB
jira | Total Memory : 1963MB
jira | Free Memory : 1457MB
jira | Used Memory : 506MB
jira | Memory Pool: Code Cache : Code Cache: init = 33554432(32768K) used = 10244864(10004K) committed = 33554432(32768K) max = 536870912(524288K)
jira | Memory Pool: Metaspace : Metaspace: init = 0(0K) used = 21014056(20521K) committed = 21626880(21120K) max = -1(-1K)
jira | Memory Pool: Compressed Class Space : Compressed Class Space: init = 0(0K) used = 2348624(2293K) committed = 2490368(2432K) max = 1073741824(1048576K)
jira | Memory Pool: PS Eden Space : PS Eden Space: init = 537395200(524800K) used = 507838896(495936K) committed = 537395200(524800K) max = 537395200(524800K)
jira | Memory Pool: PS Survivor Space : PS Survivor Space: init = 89128960(87040K) used = 24311360(23741K) committed = 89128960(87040K) max = 89128960(87040K)
jira | Memory Pool: PS Old Gen : PS Old Gen: init = 1431830528(1398272K) used = 163872(160K) committed = 1431830528(1398272K) max = 1431830528(1398272K)
jira | JVM Input Arguments : -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms2048m -Xmx2048m -XX:InitialCodeCacheSize=32m -XX:ReservedCodeCacheSize=512m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory -XX:-OmitStackTraceInFastThrow -Djava.locale.providers=COMPAT -Djira.home=/var/atlassian/application-data/jira -Datlassian.plugins.startup.options=-fg -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xloggc:/opt/atlassian/jira/logs/atlassian-jira-gc-%t.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCCause -XX:+ExplicitGCInvokesConcurrent -Dignore.endorsed.dirs= -Dcatalina.base=/opt/atlassian/jira -Dcatalina.home=/opt/atlassian/jira -Djava.io.tmpdir=/opt/atlassian/jira/temp
jira | Java Compatibility Information : JIRA version = 8.9.0, Java Version = 1.8.0_252
jira |
jira | ___ Java System Properties _________________
jira |
jira | atlassian.plugins.startup.options : -fg
jira | atlassian.standalone : JIRA
jira | awt.toolkit : sun.awt.X11.XToolkit
jira | catalina.base : /opt/atlassian/jira
jira | catalina.home : /opt/atlassian/jira
jira | catalina.useNaming : true
jira | common.loader : "${catalina.base}/lib",
jira | "${catalina.base}/lib/*.jar",
jira | "${catalina.home}/lib",
jira | "${catalina.home}/lib/*.jar"
jira | file.encoding : UTF-8
jira | file.encoding.pkg : sun.io
jira | ignore.endorsed.dirs :
jira | java.awt.graphicsenv : sun.awt.X11GraphicsEnvironment
jira | java.awt.headless : true
jira | java.awt.printerjob : sun.print.PSPrinterJob
jira | java.class.version : 52.0
jira | java.home : /opt/java/openjdk/jre
jira | java.io.tmpdir : /opt/atlassian/jira/temp
jira | java.locale.providers : COMPAT
jira | java.naming.factory.initial : org.apache.naming.java.javaURLContextFactory
jira | java.naming.factory.url.pkgs : org.apache.naming
jira | java.protocol.handler.pkgs : org.apache.catalina.webresources
jira | java.runtime.name : OpenJDK Runtime Environment
jira | java.runtime.version : 1.8.0_252-b09
jira | java.specification.name : Java Platform API Specification
jira | java.specification.vendor : Oracle Corporation
jira | java.specification.version : 1.8
jira | java.util.concurrent.ForkJoinPool.common.threadFactory : org.apache.catalina.startup.SafeForkJoinWorkerThreadFactory
jira | java.util.logging.config.file : /opt/atlassian/jira/conf/logging.properties
jira | java.util.logging.manager : org.apache.juli.ClassLoaderLogManager
jira | java.vendor : AdoptOpenJDK
jira | java.vendor.url : http://java.oracle.com/
jira | java.vendor.url.bug : http://bugreport.sun.com/bugreport/
jira | java.version : 1.8.0_252
jira | java.vm.info : mixed mode
jira | java.vm.name : OpenJDK 64-Bit Server VM
jira | java.vm.specification.name : Java Virtual Machine Specification
jira | java.vm.specification.vendor : Oracle Corporation
jira | java.vm.specification.version : 1.8
jira | java.vm.vendor : AdoptOpenJDK
jira | java.vm.version : 25.252-b09
jira | jdk.tls.ephemeralDHKeySize : 2048
jira | jira.home : /var/atlassian/application-data/jira
jira | mail.mime.decodeparameters : true
jira | org.apache.catalina.security.SecurityListener.UMASK : 0027
jira | org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER : true
jira | org.dom4j.factory : com.atlassian.core.xml.InterningDocumentFactory
jira | os.arch : amd64
jira | os.name : Linux
jira | os.version : 4.18.0-147.8.1.el8_1.x86_64
jira | package.access : sun.,
jira | org.apache.catalina.,
jira | org.apache.coyote.,
jira | org.apache.jasper.,
jira | org.apache.tomcat.
jira | package.definition : sun.,
jira | java.,
jira | org.apache.catalina.,
jira | org.apache.coyote.,
jira | org.apache.jasper.,
jira | org.apache.naming.,
jira | org.apache.tomcat.
jira | server.loader :
jira | shared.loader :
jira | sun.arch.data.model : 64
jira | sun.boot.library.path : /opt/java/openjdk/jre/lib/amd64
jira | sun.cpu.endian : little
jira | sun.cpu.isalist :
jira | sun.io.unicode.encoding : UnicodeLittle
jira | sun.java.command : org.apache.catalina.startup.Bootstrap start
jira | sun.java.launcher : SUN_STANDARD
jira | sun.jnu.encoding : UTF-8
jira | sun.management.compiler : HotSpot 64-Bit Tiered Compilers
jira | sun.os.patch.level : unknown
jira | tomcat.util.buf.StringCache.byte.enabled : true
jira | tomcat.util.scan.StandardJarScanFilter.jarsToScan : log4j-taglib*.jar,
jira | log4j-web*.jar,
jira | log4javascript*.jar,
jira | slf4j-taglib*.jar
jira | tomcat.util.scan.StandardJarScanFilter.jarsToSkip : annotations-api.jar,
jira | ant-junit*.jar,
jira | ant-launcher.jar,
jira | ant.jar,
jira | asm-*.jar,
jira | aspectj*.jar,
jira | bootstrap.jar,
jira | catalina-ant.jar,
jira | catalina-ha.jar,
jira | catalina-jmx-remote.jar,
jira | catalina-storeconfig.jar,
jira | catalina-tribes.jar,
jira | catalina-ws.jar,
jira | catalina.jar,
jira | cglib-*.jar,
jira | cobertura-*.jar,
jira | commons-beanutils*.jar,
jira | commons-codec*.jar,
jira | commons-collections*.jar,
jira | commons-daemon.jar,
jira | commons-dbcp*.jar,
jira | commons-digester*.jar,
jira | commons-fileupload*.jar,
jira | commons-httpclient*.jar,
jira | commons-io*.jar,
jira | commons-lang*.jar,
jira | commons-logging*.jar,
jira | commons-math*.jar,
jira | commons-pool*.jar,
jira | dom4j-*.jar,
jira | easymock-*.jar,
jira | ecj-*.jar,
jira | el-api.jar,
jira | geronimo-spec-jaxrpc*.jar,
jira | h2*.jar,
jira | hamcrest-*.jar,
jira | hibernate*.jar,
jira | httpclient*.jar,
jira | icu4j-*.jar,
jira | jasper-el.jar,
jira | jasper.jar,
jira | jaspic-api.jar,
jira | jaxb-*.jar,
jira | jaxen-*.jar,
jira | jdom-*.jar,
jira | jetty-*.jar,
jira | jmx-tools.jar,
jira | jmx.jar,
jira | jsp-api.jar,
jira | jstl.jar,
jira | jta*.jar,
jira | junit-*.jar,
jira | junit.jar,
jira | log4j*.jar,
jira | mail*.jar,
jira | objenesis-*.jar,
jira | oraclepki.jar,
jira | oro-*.jar,
jira | servlet-api-*.jar,
jira | servlet-api.jar,
jira | slf4j*.jar,
jira | taglibs-standard-spec-*.jar,
jira | tagsoup-*.jar,
jira | tomcat-api.jar,
jira | tomcat-coyote.jar,
jira | tomcat-dbcp.jar,
jira | tomcat-i18n-*.jar,
jira | tomcat-jdbc.jar,
jira | tomcat-jni.jar,
jira | tomcat-juli-adapters.jar,
jira | tomcat-juli.jar,
jira | tomcat-util-scan.jar,
jira | tomcat-util.jar,
jira | tomcat-websocket.jar,
jira | tools.jar,
jira | websocket-api.jar,
jira | wsdl4j*.jar,
jira | xercesImpl.jar,
jira | xml-apis.jar,
jira | xmlParserAPIs-*.jar,
jira | xmlParserAPIs.jar,
jira | xom-*.jar
jira | user.country : US
jira | user.dir : /var/atlassian/application-data/jira
jira | user.home : /var/atlassian/application-data/jira
jira | user.language : en
jira | user.name : jira
jira | user.timezone : GMT
jira |
jira | 07-Jun-2020 07:38:10.577 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
jira | 2020-06-07 07:38:10,580+0000 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/atlassian/application-data/jira/dbconfig.xml
jira | 07-Jun-2020 07:38:10.581 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
jira | 07-Jun-2020 07:38:10.598 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3217 ms
jira | 2020-06-07 07:38:10,612+0000 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigHandler] Trying to get encrypted password from xml and decrypt it
jira | 2020-06-07 07:38:10,614+0000 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigHandler] Database password decryption not performed.
mysql | 2020-06-07T07:38:11.051764Z 2 [Note] Aborted connection 2 to db: 'jiradb' user: 'jira' host: '172.18.0.3' (Got an error reading communication packets)
jira | 2020-06-07 07:38:11,061+0000 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger] Running Jira startup checks.
jira | 2020-06-07 07:38:11,061+0000 JIRA-Bootstrap FATAL [c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked.
jira | 2020-06-07 07:38:11,100+0000 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Memory Usage:
jira | ---------------------------------------------------------------------------------
jira | Heap memory : Used: 273 MiB. Committed: 1963 MiB. Max: 1963 MiB
jira | Non-heap memory : Used: 45 MiB. Committed: 67 MiB. Max: 1536 MiB
jira | ---------------------------------------------------------------------------------
jira | TOTAL : Used: 318 MiB. Committed: 2030 MiB. Max: 3499 MiB
jira | ---------------------------------------------------------------------------------

 

1 answer

1 accepted

1 vote
Answer accepted
Armen_Mkrtchyan June 8, 2020

Finally problem is solved! I have added following line in my.cnf file:

[mysqld]
...
default-time-zone
=+00:00
...

You can change time-zone value to the required one.

MySQL database server is configured to a time zone like, EDT, PDT, CEST, etc. These time zone values are not recognizable since JAVA 8. Most probably after some time it was updated and after restart i start getting "(Got an error reading communication packets)" .

I found solution here: https://stackoverflow.com/questions/26515700/mysql-jdbc-driver-5-1-33-time-zone-issue

Deleted user September 10, 2021

Thanks a ton, just had the same problem after upgrading Confluence 7.9.1 to 7.13.

Addind the timezone setting in the my.cnf file solved the issue for me as well.

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
SERVER
VERSION
8.9.0
TAGS
AUG Leaders

Atlassian Community Events