Seeking Feedback on Datasource Connector Usage in Confluence

[James Ponting]

Hi All,

My name is James and I’m an Engineering Manager on the Confluence Data Center team. I’m here to gather some feedback from users who Administer Confluence instances about how they connect Confluence to a database.

As part of our goal to deliver the most value for the most users, we occasionally remove features or functions from the product that are rarely used or no longer fit our teams vision for the product in order to increase focus on other areas. To this end, we are currently investigating the removal of support for Tomcat (JNDI) Datasources as a method for connecting Confluence to the database.

If you're not using a Datasource connecter (you can check by reading this documentation), then the changes under consideration won't impact you :)

For those using Datasources, we would like to gather your feedback on the following points
1. Are you running Confluence Server or Data Center?
2. Are you using a Tomcat Datasource to work around a problem you encountered in your environment? If so, what was the problem?
3. Why have you and your team chosen to use Datasources in place of the default JDBC URL method?

We'll be accepting feedback for two weeks, and will include this feedback as one of many factors in our deliberations on next steps. In turn, I'll update this post with the decision we arrive at.

EDIT - I'm going to refrain from commenting on your feedback until the end of this process so as not to influence the feedback. I will reply to some of the comments once the feedback window has closed as there is some relevant information to share. Thanks for the feedback so far :)

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

[GBE]

Hi @James Ponting ,

I saw this post from your comment on https://jira.atlassian.com/browse/CONFSERVER-59524

We are using Confluence Server.

We are using those thirdparty plugins such as 'SQL for Confluence' from AppFire.

And followed their documentation to rely on the server.conf datasources.

https://bobswift.atlassian.net/wiki/spaces/SQL/pages/389316780/Data+source+configuration+-+application+server+-+8.x  

We used it extensively to make dynamic pages that gather data from thirdparty databases.

This plugin makes Confluence a unique entry point for both WIKI and thirdparty DB data.

[James Ponting]

Hey @GBE

Thanks for providing your thoughts.

The Appfire team have commented below, and we've provided our thoughts in resposne.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center 

[Leszek Czaplis]

Hi James,

On our side we were facing serious performance degradation/instability when C3P0 connector was used.

1. Confluence Server

2. Yes, please review details of PS-71193. Problem (in short) - frequent spikes of Active Threads reaching limit defined in server.xml. When limit is reached Confluence application is not reachable/not accessible

3. Premier Support advised us to switch to JNDI Datasource and this method made our instances stable.

Best regards,

Leszek Czaplis

[James Ponting]

Hey @Leszek Czaplis

Thanks for providing your feedback.

I'm familiar with the issue you're talking about. The bug itself is tracked at CONFSERVER-60152 - Running Confluence with Oracle Database Native Network Encryption Degrades Performance or Causes Outages and has been resolved as of Confluence 7.14.0.

When you're on a version of Confluence that includes this fix, I'd recommend reverting to the default configuration, as we've replaced c3p0 with HikariCP, which is more performant. It also has the benefit of resolving the issue.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center 

[IT OPUSG]

Hi, we've been using a datasource connector with our Confluence Server in the past, but only to temporarily troubleshoot performance issues through JavaMelody. Afterwards we reverted back to the default connection method.

Using a datasource connector allows JavaMelody to monitor the jdbc connections and the SQL requests. See: https://github.com/javamelody/javamelody/wiki/AtlassianPlugin

[Marek Cwynar]

Hi!

We use datasources for Jira and Confluence in order to use 
jira-confluence-javamelody pliugin - link

https://github.com/javamelody/jira-confluence-javamelody to monitor Jira and Confluence performance.

It provides not only performance analysis but also lists of current errors.

We have found that plugin very useful for troubleshoting.

[Ravi Varma]

Hi @James Ponting ,

Thanks @GBE  for your inputs.

As mentioned in Configure+Data+Sources+-+10.x , SQL Pro for Confluence can be configured to leverage existing  Tomcat (JNDI) Datasources. This is one of the preferred ways that customers find convenient while working with various SQL macros offered by our app (https://bobswift.atlassian.net/wiki/spaces/SQL/pages/943915165/Macro+Reference+-+10.x)

Few concerns/queries from our side would be

• The proposed changes discussed in this thread, would result in major re-engineering for the app. Dropping this feature, would have its own set of challenges but can be done

• We are more concerned with regard to the Confluence pages that are currently configured using  Tomcat (JNDI) Datasources based connections/profiles.

• What is the timeframe you are looking at, for bringing this change?

• To make it seamless for existing pages, Appfire needs to brainstorm on alternate workarounds, which would have its own limitations and assumptions.

• As this is not a replacement API but rather removal of one of the basic features of Tomcat, we have to explore unchartered territory

Thanks,

Ravi Varma

[Michael Andreacchio]

Thanks for the details @Ravi Varma

To be clear, we wish to end support for data source connections being established for when the Confluence Application makes a database connection. I don't have a complete understanding of how the SQL for Confluence app works but would it be possible for there to be a JDBC connection set up for Confluence, and then JNDI connections set up for the SQL for Confluence query/macro connections?

Our plans are not to entirely remove this feature, and particularly not at the Tomcat level. If a user wishes to still run Confluence using a datasource connection they will receive a warning during startup but be able to continue running the app regardless.

We are aiming to deliver this in the final quarter of this calendar year. There will be an early access program release made available for Appfire to test the functionality out on once it's implemented.

I hope the above helps alleviate your concerns. Please let me know if you have any further questions.

Michael Andreacchio

Confluence DC Product Management

[James Ponting]

I see @Michael Andreacchio beat me to the punch. What he's said is my understanding also.

If we do make any changes, we'll communicate them so that you can assess the impact on your product. We'd already identified yours and a handful of others, so it's on our radar.

Thanks for letting us know your perspective. We'll let you know where we land :)

Thanks,
James Ponting
Engineering Manager - Confluence Data Center 

[Ravi Varma]

@James Ponting    /   @Michael Andreacchio ,

Following are the DB sources for our app

1. SQL app administrator can configure databases on the fly by providing appropriate details and credentials. I see no impact here
2. Confluence administrator can configure datasources in server.xml using <Resource name="jdbc/salesDatabase" auth="Container" type="javax.sql.DataSource" etc SQL app administrator can configure app level database connection using this datasource. SQL app does not distinguish whether the underlying datasource is the primary source for the Confluence instance or some other external database. 

@Michael Andreacchio , as per your statement

Our plans are not to entirely remove this feature, and particularly not at the Tomcat level. If a user wishes to still run Confluence using a datasource connection they will receive a warning during startup but be able to continue running the app regardless.

After the proposed/upcoming changes, if datasources can be configured in server.xml using <Resource > then I hope it should not have major impact on our app. 

We will test our app, once the EAP release is available. 

Thanks,

Ravi Varma

[James Ponting]

Hey @Ravi Varma

What you've covered in in points 1 and 2 is my understanding also.

We'll let you know through the EAP process if anything changes, but we haven't made any changes yet (again, discovery phase). The EAP due out soon shouldn't cause any issues on the connector front for this reason.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center