Why is confluence.context.path not correctly resolved in server.xml when upgrading?

Question

Hello, i recently updated our test confluence instance from 6.4 to 6.13.1 and ran into a problem which in our case was not that big deal regarding the outcome but could result in not beeing able to use collaborative editing feature on hosted confluence servers in some situations. After upgrading i noticed a warning regarding ${confluence.context.path}#synchrony-proxy.xml beeing ignored which made me worry a bit. Further analysis made me forget about it somewhat. It seems that this variable is not correctly templated when installing or upgrading server.xml file. If I understand the article https://confluence.atlassian.com/doc/possible-confluence-and-synchrony-configurations-958779064.html correclty collaborative editing would only be an issue when using fallback and not "Direct to synchrony with reverse proxy" configuration i.e. not having /synchrony proxied for direct connect. Having that variable in server.xml file present would result in creating a file ${confluence.context.path}#synchrony-proxy.xml which cant be resolved internally then since the fallback would be /confluence/synchrony-proxy. Workaround My workaround was to delete the deployment descriptors present in /conf/Standalone/localhost/. I had multiple in them. * ${confluence.context.path}#synchrony-proxy.xml * synchrony-proxy.xml * confluence#synchrony-proxy.xml I believe the second one was copyied over from the old installation and the first one created by the upgrade. Last file was created because i manually replaced the variable with the context path of the confluence instance. Furthermore i noticed that resolving context-path in setenv.sh using confluence-context-path-extractor.jar sometimes just outputs ${confluence.context.path}. To be on the save side I modified the setenv.sh script to not use the jar and use some awk magic instead extracting the value from confluence.cfg.xml reading out the path to this file from confluence-init.properties _CONFLUENCE_HOME=$(grep -e ^confluence.home confluence/WEB-INF/classes/confluence-init.properties | cut -d "=" -f2 | awk '{$1=$1;print}') _CONFLUENCE_CONTEXT_PATH=$(sed -n 's:$.*$:\1:p' ${_CONFLUENCE_HOME}/confluence.cfg.xml | awk '{$1=$1;print}') #CONFLUENCE_CONTEXT_PATH=`$JRE_HOME/bin/java -jar $CATALINA_HOME/bin/confluence-context-path-extractor.jar $CATALINA_HOME` CONFLUENCE_CONTEXT_PATH=${_CONFLUENCE_CONTEXT_PATH} I did not investigate this any further and i sadly do not know under which circumstances this happens so I can only guess. I called the jar manually multiple times with confluence user via su and at one point only the variable printed out but not the value. Maybe it was some file permission issue. For now i will leave it that way since using bash gives back the result a bit faster Im looking forward to get a reply maybe the workaround can help others who had the same problem or want to get rid of the warning in the log file. Best wishes, Rüdiger Huppertz

unknown · Answer

Hello @Stephen Sifers , recently i found out that atlassian is providing official docker images and i was eager to try them out on our testinstance to evaulate if upgrading instances would be less of pain when it comes to customizing the instance. I was very happy to see that everything we need is covered. I have set up a docker compose file and even found a way to utilize systemd to start dependend services making it easy to upgrade since i would only need to change the version of CONFLUENCE_VERSION in dot env file and restart the service. The database is also a docker image now since i dont want to use the database on the OS since the system does unnatended-upgrades. Why am I coming back to this topic? Well it turned out that i cannot use the workaround by modifying the synchrony tomcat descriptor which defaults to /synchrony without having to use a custom Dockerfile so i tried to find a way to work with what i have and after some time i did. Before i changed the internal /synchrony context - which is provided by confluence tomcat instance - to /wiki/synchrony, proxy_pass via nginx to /wiki/synchrony and finally setting the synchrony context path via jvm property to that same path modifing CATALINA_OPTS. This is _not_ needed. Maybe it never was. The only thing that has to be set is still synchrony.context.path and that is the path someone would need to point a websocket capable proxy_pass directive to via nginx. Btw the synchrony context descriptor in Standalone/localhost/synchrony.xml now as of 7.4.0 docker image always points to /synchrony and never changes regardless of confluence.context.path. When using standalone this variable was included in the path. Seems to me this changed now? Maybe you can elaborate on this. Anyway my working docker env loocks like this: -- docker-compose confluence-vars.env JVM_MINIMUM_MEMORY=6g JVM_MAXIMUM_MEMORY=12g ATL_PROXY_NAME= ATL_PROXY_PORT=443 ATL_TOMCAT_PORT=8090 ATL_TOMCAT_MAXTHREADS=100 ATL_TOMCAT_SCHEME=https ATL_TOMCAT_CONTEXTPATH=/wiki ATL_JDBC_URL=jdbc:postgresql://postgres:5432/ ATL_JDBC_USER= ATL_JDBC_PASSWORD= ATL_DB_DRIVER=org.postgresql.Driver ATL_DB_TYPE=postgresql JVM_SUPPORT_RECOMMENDED_ARGS=-Dsynchrony.proxy.enabled=false -Dsynchrony.enable.xhr.fallback=false -Dsynchrony.port=8091 -Dsynchrony.context.path=/wiki/synchrony The default proxy.enable and port is probably not required since its the default. Nginx required parameters for location /wiki/synchrony proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_cache_bypass $http_upgrade; And Colab Editing works perfectly. Before the page i wanted to edit didnt even go into edit mode and showed an endless loading spinner in firefox and running into timeouts. Just wanted to share this since it seems that i missed checking if the Testinstance had the correct nginx configuration. This put me on the wrong track assuming i would need to modify the context descriptor of synchrony again. Best wishes, Rüdiger Huppertz

unknown · Answer

Hello Stephen, when I created this Post i did not write down the exact steps to reproduce. As i recently updated to confluence 7.0.1 i now had the same issue but was now able to reproduce the "error" or special case where synchrony would not work thus breaking collaborative editing feature. I have Installed / Upgraded to Confluence 7.0.1 via installer and use a reverse proxy to serve both jira and confluence under the same domain but different context path. ROOT (/) is Jira and /wiki is Confluence. This worked before 7.x with minor adjustments but with 7.x its breaking. Steps to reproduce * Upgrade Confluence to latest. * Change context path in server.xml and set property "confluence.webapp.context.path" in confluence.cfg.xml to /wiki * Point /wiki in reverse proxy to your confluence instance i.e. :8090/wiki * Point /wiki/synchrony in reverse proxy to your synchrony instance served by default by confluence instance (server.xml context path ${confluence.context.path}/synchrony) to :8091/synchrony * Visit confluence via url https://sub.domain.tld/wiki Things to note: Installer is used via -q -varfile option and will NOT start the instance automatically after installation since i need to apply custom changes first and then start with bin/start-confluence -fg to test. Edit a Page and notice the error dialog saying "Something is wrong" .. page does take forever to load never displaying content. In Network log (Firefox Browser) you can see a 404 or 502 status code when trying to resolve https://sub.domain.tld/synchrony not checking /wiki/synchrony afterwards. Monitored Behavior: Checks first https://sub.domain.tld/synchrony and gets a HTTP 404 or 502 depending if jira is running on domain or not. Note: sub.domain.tld is configured via reverse proxy for jira. When the instance is down it will return a 502. According to docs auto discovery should try and check /synchrony ${confluence.context.path}/synchrony and redirect to the internal endpoint. In this special case i believe this will not work. Applied fix (works for me) * Run Step 1 to (including) Step 4 as mentioned. * Comment or remove in server.xml section where synchrony-proxy is defined. * Change context path for instance in server.xml to /wiki * Remove all files in conf/Standalone/localhost ( may be present from previous upgrades if ${confluence.context.path} was manually replaced with /wiki ) * Modify setenv.sh and add before export CATALINA_OPTS "-Dsynchrony.proxy.enabled=false ${CATALINA_OPTS}"
CATALINA_OPTS="-Dsynchrony.enable.xhr.fallback=false ${CATALINA_OPTS}"
CATALINA_OPTS="-Dsynchrony.context.path=/wiki/synchrony ${CATALINA_OPTS}"# synchrony configuration
CATALINA_OPTS= 6. Point Reverse Proxy to http://:8091/wiki/synchrony 7. Start Service confluence. 8. Notice Colab editing works now. As of my understanding doing this will define the context.path for the synchrony instance to use. It also is telling confluence that it should not try to resolve synchrony via proxy (synchrony-proxy in server.xml). The synchrony instance is still started by confluence if collab editing is enabled and listens on port 8091 by default. I do believe confluence is not ment to be run "on top of" an existing jira instance thus does not know if synchrony is just not available or "busy". In any case i hope this helps to understand the issue i had with configuring colab editing correctly. Additional Notes Reverse Proxy (nginx) is configured to * listen for /wiki and ProxyPass to http://confluence:8091/wiki * listen for /wiki/synchrony and ProxyPass to http://confluence:8091/wiki/synchrony * Upgrade to WebSockets is allowed for /wiki/synchrony but not /wiki Best wishes, Rüdiger Huppertz

Stephen Sifers · Answer

Hello Rüdiger,

From reading through your post, this behavior does sound strange as to why this is happening. To understand a little bit more as to how this started happening, can you please let us know the following:

* Did you test the upgrade on a staging instance?* If so, was this same issue present within staging?

* When you upgraded, did you edit the server.xml files or replace it with the old files?
* Did you replace any files after upgrading with older versions?
* If so, which files were replaced with older versions?

* Which version of Java are you currently using?

We look forward to your response so we may see what is causing this type of issue.

Regards,
Stephen Sifers