Create
cancel
Showing results for 
Search instead for 
Did you mean: 
Sign up Log in

Next challenges

Recent achievements

  • Global
  • Personal

Recognition

  • Give kudos
  • Received
  • Given

Leaderboard

  • Global

Trophy case

Kudos (beta program)

Kudos logo

You've been invited into the Kudos (beta program) private group. Chat with others in the program, or give feedback to Atlassian.

View group

It's not the same without you

Join the community to find out what other Atlassian users are discussing, debating and creating.

Atlassian Community Hero Image Collage

Confluence server will not start after I increased partition

I have confluence and jira installed on the same server.  I ran low on disk spaced, and increased the partition.

After rebooting, jira worked without an issue, but confluence will not load, and gives me an error:

"Your database Storage Engine could not be determined"

The atlassian-confluence log says:

2020-12-03 12:21:45,608 ERROR [http-nio-8090-exec-7] [[Standalone].[localhost].[/].[action]] log Servlet.service() for servlet [action] threw exception
java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)

How can I check if confluence and jira are using the same DB?

What could be the issue?

1 answer

0 votes

That's odd, I wouldn't expect that from a "run out of space" case unless you were installing, not just running.

Start with a look at <confluence home>/confluence.cfg.xml - that has the database connection in it. 

Then check the logs in <confluence home>/logs - the error message with "storage engine" should be repeated in there, but with a lot more information

Thanks,

I looked at the syncrhony, confluence and diagnostics log file, and could not find any errors relating to storage engine.

I do see that confluence is using 

jdbc:mysql://localhost/confluence

and jira is using

jdbc:mysql://localhost:3306/jiradb?useUnicode=true&amp;characterEncoding=UTF8&amp;sessionVariables=default_storage_engine=InnoDB

Should they be different?  As I mentioned, jira is working fine.

I suspect the driver assumes the default port of 3306 for mysql which would mean they are both connected to the same database server.

I think we need to see more of the log as the connection fails.

There are three logs, which one would have that information?

This is the first error in the atlassian-confluence.log file:

2020-12-03 17:21:45,420 WARN [http-nio-8090-exec-6] [confluence.impl.vcache.SynchronousExternalCache] lambda$get$11 Failed to read entry from cache 'com.atlassian.bandana.BandanaPersister': Failed due to UNCLASSIFIED_FAILURE
-- traceId: 0062c323e0740be4
2020-12-03 17:21:45,421 ERROR [http-nio-8090-exec-6] [ContainerBase.[Standalone].[localhost].[/]] log Unhandled exception occurred whilst decorating page
-- traceId: 0062c323e0740be4
java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.

That's probably a symptom of an earlier failure, you'll want to roll further back up the log

Like Ken Ducey likes this

I rebooted the server. Please let me know if this is enough information, or if there is a way for me to send you the entire file.

Really appreciate your help!

2020-12-03 17:43:36,669 INFO [localhost-startStop-3] [atlassian.plugin.manager.DefaultPluginManager] shutdown Preparing to shut down the plugin system
2020-12-03 17:43:36,721 INFO [localhost-startStop-3] [atlassian.plugin.manager.DefaultPluginManager] shutdown Shutting down the plugin system
2020-12-03 17:43:36,721 WARN [localhost-startStop-3] [atlassian.plugin.servlet.DefaultServletModuleManager] onPluginFrameworkShutdownEvent PluginController passed via the PluginFrameworkShutdownEvent did not match that passed via PluginFrameworkStartedEvent
2020-12-03 17:43:36,737 WARN [localhost-startStop-3] [atlassian.plugin.jmx.JmxUtil] unregister Failed to unregister, instance not found:
2020-12-03 17:43:37,094 INFO [localhost-startStop-3] [com.atlassian.confluence.lifecycle] destroy Shutting down long running task service
2020-12-03 17:43:37,201 INFO [localhost-startStop-3] [com.atlassian.confluence.lifecycle] shutdownCacheManager Shutting down EhCache cache manager
2020-12-03 17:43:37,208 INFO [localhost-startStop-3] [com.atlassian.confluence.lifecycle] contextDestroyed Stopping Confluence
2020-12-03 17:44:27,323 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 6.15.2 [build 8100 based on commit hash 4410012ac87e845516b70bc69b6f7a893eabaa5a] - synchrony version 2.1.0-master-9d112c9d
2020-12-03 17:44:35,434 INFO [localhost-startStop-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization started
2020-12-03 17:44:43,465 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] <init> Loading EhCache cache manager
2020-12-03 17:45:28,559 INFO [localhost-startStop-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization completed in 53125 ms
2020-12-03 17:45:33,498 ERROR [localhost-startStop-1] [confluence.setup.dbcheck.MySQLChecker] checkIsolationLevel MySQL isolation level could not be read. Isolation level must be 'READ-COMMITTED'. See http://confluence.atlassian.com/x/GAtmDg
2020-12-03 17:45:34,635 ERROR [localhost-startStop-1] [atlassian.confluence.setup.BootstrapApplicationStartupListener] checkConfigurationOnStartup Your database Storage Engine could not be determined
2020-12-03 17:45:34,784 WARN [localhost-startStop-1] [atlassian.confluence.plugin.PluginFrameworkContextListener] contextInitialized Not starting full plugin system due to upgrade
2020-12-03 17:45:36,606 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] init Confluence is ready to serve
2020-12-03 17:46:32,834 ERROR [http-nio-8090-exec-3] [atlassian.confluence.util.AbstractBootstrapHotSwappingFilter] doFilter Could not get swap target filter
-- url: /rest/capabilities | traceId: 00c92ebbeef0a650
2020-12-03 17:46:33,660 WARN [http-nio-8090-exec-3] [confluence.impl.vcache.SynchronousExternalCache] lambda$get$11 Failed to read entry from cache 'com.atlassian.bandana.BandanaPersister': Failed due to UNCLASSIFIED_FAILURE
-- traceId: 00878a406bd410c0
2020-12-03 17:46:33,669 ERROR [http-nio-8090-exec-3] [ContainerBase.[Standalone].[localhost].[/]] log Unhandled exception occurred whilst decorating page
-- traceId: 00878a406bd410c0
java.util.concurrent.CompletionException: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateException] is not allowed.
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)

Could it be as simple as parameters missing from the URL in confluence.cfg.xml?

<property name="hibernate.connection.url">jdbc:mysql://localhost/confluence</property>

if this was wrong, would I get that error?

You might, for some duff values.

However, the log has a smoking gun in it - look for the error about "READ COMMITTED"

I changed th connection url to  <property name="hibernate.connection.url">jdbc:mysql://localhost:3306/confluence?sessionVariables=tx_isolation='READ-COMMITTED'</property>,

but still has the same error

What versions of Confluence and the mysql driver are in use?

I've seen combinations of them that ignore the properties and hence require that you do this setting on the mysql server.

how can I find that?

I believe it is Confluence 6.15.2 based on the readme file

the MySQL driver is 

mysql-connector-java-5.1.39-bin.jar

Ok, Atlassian recommend a slightly later driver for Confluence 6.15 - see https://confluence.atlassian.com/conf615/database-jdbc-drivers-967338417.html - but I don't think that's the problem.

I'd like to try the fix they've got in the document the error points to - not adding to the connection string, but changing the properties. 

If those two still don't fix it, then you'll need to make the changes on the mysql server side.

Thank you. I will try upgrading the driver.  Will Confluence know to use the new driver?


"I'd like to try the fix they've got in the document the error points to - not adding to the connection string, but changing the properties."

what document? is it this one? https://confluence.atlassian.com/conf615/database-setup-for-mysql-967338433.html That is changing the MySQL side, correct?

Thanks again for your help.

I went through that doc, downloaded the driver, changed the web.xml and confluence.cfg.xml files.  I did not change anything on the Tomcat side, because I don't believe I need to. Rebooted the server, and still have the exact same error.

I am able to access MySQL from the CLI using mysql -h localhost --port=3306 -u confluence -pPASSWORD 'confluence' so I am using the correct port, etc.


Anything else that could be causing this?

Sorry, I meant the document pointed to in the error message.  

Yes, you've found the right one there - I have to admit I've never had any luck changing the connection string parameters, I've always ended up putting the setting on the mysql server and restarting it.

Do you have any advice on what to do from here? I am not sure where to go.

I don't have a lot of expertise with MySQL, the server settings I checked appear to match.

Looking at the error again, "Your database Storage Engine could not be determined" - do you know anything more about that?  For example, could it be it is not finding the driver or something?

Thanks,

Ken

Have you tried making the changes in the my.cnf file that are suggested in the document?

Yes, below is a copy of the file

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# This is for Atlassian confluence server modified by Ken
character-set-server=utf8
max_allowed_packet=256M
collation-server=utf8_bin
default-storage-engine=INNODB
max_allowed_packet=256M
innodb_log_file_size=2GB
transaction-isolation=READ-COMMITTED
binlog_format=row

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mysqld according to the
# instructions in http://fedoraproject.org/wiki/Systemd

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

Ok, that looks good to me (almost identical to the one I've got on a client server at the moment).

If you've restarted MySQL to pick up the changes, then I'm stumped.  Your database is telling Confluence, via the driver, that it is set up in a way Confluence can't use.  I don't know where to look next, other than raising it with Atlassian over at https://support.atlassian.com/contact 

Like Ken Ducey likes this

I have a license, but it is a low level and I can't contact them directly.  I posted it in Stackoverflow.

Thanks again for your help, I do appreciate it!

Ken

If I deleted the confluence.cfg.xml, and rebooted the server, my understanding is it will run the installation procedure.  Could I do that and tell it to connect to the existing DB?

Also, after I rewrote the XML, file, it appears as though I am receiving a new error - in case this helps.

kbUrl : https://confluence.atlassian.com/confkb/startup-check-providing-logs-for-support-936509900.html?utm_source=Install&amp;utm_medium=in-product&amp;utm_campaign=csseng_fy18_q3_server_confluence_errorstate
caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'healthCheckAnalyticsSender' available
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:687)

If it re-runs setup, you'll find it fails when it spots the existing database is not empty.

You could connect it to a new empty database, and then edit dbconfig.xml to swap back to the old one, but as the connection settings are being reported as wrong, I'm not sure that would help.

Is there any way to have confluence support take a look at this?  I looked at upgrading the license but it is a substantial increase.

Solved it! it was likely an issue with the /var/lib/mysql/performance_schema/ not being owned by mysql user, or that the mysql db needed to be upgraded.

1) renamed the confluence.cfg.xml file

2) when starting up, ran setup automatically, put in settings for the existing DB

3) tried to "test connection" received an error about writing to './performance_schema'  - realized MySQL was not the owner of the folder /var/lib/mysql/performance_schema/ - so I changed that and also changed the permissions

4) then received an error about 'performance_schema.session_variables' doesn't exist' an upgraded MySQL using mysql_upgrade

5) test connection was successful! hit next, and then it warned me there was an existing DB it was about to overwrite. Stopped, and went back and then just rebooted server

6) I copied the old confluence.cfg.xml file back

7) rebooted server, and it worked!

Like Nic Brough _Adaptavist_ likes this

Oh, that's fantastic!  I'm really glad you carried on digging and got to a solution, that was not an easy or obvious one to find.

I hope it's ok if I add this to my debugging notes?  I'd never thought to check permissions on the database service (I think because I almost always work with databases that are remote from the Atlassian system server and a little bit "black box" - they're either running fine or they are unreachable remotely)

Absolutely, that is why I posted it - the key to debugging was the "test connection" button on the setup - it gave me more specific errors that led me to the problem.

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
SERVER
TAGS
Community showcase
Published in Confluence

Announcing Team Calendars in Confluence Data Center

Hi Community! We're thrilled to share that Team Calendars for Confluence is now a built-in feature for Confluence Data Center releases 7.11 and beyond.  A long time favorite,  Team Cale...

26 views 0 2
Read article

Community Events

Connect with like-minded Atlassian users at free events near you!

Find an event

Connect with like-minded Atlassian users at free events near you!

Unfortunately there are no Community Events near you at the moment.

Host an event

You're one step closer to meeting fellow Atlassian users at your local event. Learn more about Community Events

Events near you