Invalid character found in the request target on Confluence behind nginx

Tran Tien dung November 11, 2018

Hi everyone,

Currently I'm using Confluene 6.10.2 behind nginx. I have some pages with the page name including character '>' could not accessible, the error is:

HTTP Status 400 – Bad Request


Type Exception Report

Message Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986

Description The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).

Exception

java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986
	org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:474)
	org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:294)
	org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:764)
	org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1388)
	org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	java.lang.Thread.run(Thread.java:748)

Note The full stack trace of the root cause is available in the server logs.


Apache Tomcat/9.0.10

 

But when I access the page bypassing the reverse proxy, it's ok, so it could be a problem in nginx.

I read logs in Confluence and nginx, but did not find any stranges, please advise me how to fix the issue.

 

 

 

 

 

7 answers

2 votes
Tran Tien dung December 11, 2018

 Removing "/confluence" in proxy_pass line solved my issue. Even my confluence instance already configured context path /confluence. I don't know why I have to change it, these links with invalid character seen worked in the past.

    location /confluence {
        ...
        proxy_pass http://localhost:8090;
    }
2 votes
Henrique Bittencourt
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 13, 2018

Hi Tran!

I'm not a Nginx pro, but, doing a little bit of digging into some internet threads I just found Nginx ignores some characters when those are passed on the headers:

The ignore_invalid_headers directive allows to control whether only headers matching the rules documented will be handled and passed to upstream servers by nginx, or any header will do. 

A possible workaround is to set the ignore_invalid_headers to off in the Nginx conf file:

It is worth testing!

Best Regards.

Tran Tien dung November 13, 2018

Thanks Henrique, I tried as the link suggest, but no luck, the issue still persist. :(

1 vote
Markus Gut May 3, 2019

Since all of the workarounds described above, I added

relaxedPathChars="[]|" relaxedQueryChars="[]|{}^\`"<>"

to the server.xml, as proposed in JiraKB (https://confluence.atlassian.com/jirakb/changing-server-xml-to-handle-requests-with-special-characters-958453799.html). Now there is no more Tomcat error, instead Confluence throws a similar error:

Cause

java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in path at index 25: /display/HT/Ultra+Series+|+DHI-XVR8816S
    at com.github.kristofa.brave.servlet.ServletHttpServerRequest.getUri(ServletHttpServerRequest.java:27)

caused by: java.net.URISyntaxException: Illegal character in path at index 25: /display/HT/Ultra+Series+|+DHI-XVR8816S
    at java.net.URI$Parser.fail(URI.java:2848)

Stack Trace:[hide]

java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in path at index 25: /display/HT/Ultra+Series+|+DHI-XVR8816S
 at com.github.kristofa.brave.servlet.ServletHttpServerRequest.getUri(ServletHttpServerRequest.java:27)
 at com.atlassian.confluence.web.filter.ZipkinTracingFilter$1.lambda$create$0(ZipkinTracingFilter.java:38)
 at com.github.kristofa.brave.http.HttpServerRequestAdapter.getSpanName(HttpServerRequestAdapter.java:41)
 at com.github.kristofa.brave.ServerRequestInterceptor.handle(ServerRequestInterceptor.java:47)
 at com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:56)
 at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:52)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
 at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.johnson.filters.AbstractJohnsonFilter.doFilter(AbstractJohnsonFilter.java:59)
 at com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:32)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:39)
 at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.confluence.internal.diagnostics.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:42)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
 at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
 at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
 at org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:206)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
 at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
 at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
 at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770)
 at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
 at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.URISyntaxException: Illegal character in path at index 25: /display/HT/Ultra+Series+|+DHI-XVR8816S
 at java.net.URI$Parser.fail(URI.java:2848)
 at java.net.URI$Parser.checkChars(URI.java:3021)
 at java.net.URI$Parser.parseHierarchical(URI.java:3105)
 at java.net.URI$Parser.parse(URI.java:3063)
 at java.net.URI.<init>(URI.java:588)
 at com.github.kristofa.brave.servlet.ServletHttpServerRequest.getUri(ServletHttpServerRequest.java:25)
 ... 49 more

Any other ideas?

1 vote
Rilwan Ahmed
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
February 15, 2019

This is due to Tomcat increased their security and no longer allows raw square brackets in the query string in newer versions

1 vote
Mathy Vanvoorden January 21, 2019

This was also reported as a bug here:

https://jira.atlassian.com/browse/CONFSERVER-57582

The suggested workaround works for me, just add

relaxedQueryChars="[,]"

to your connector in server.xml

Tran Tien dung January 22, 2019

Unfortunately, this workaround not work in my issue. :(

Thanks you for let me know the bug related RSS feed.

1 vote
Shannon S
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 13, 2018

Tran Tien dung,

Can you confirm that you do NOT receive the 400 error when you're not using the NGINX setup? 

If you do not, I would recommend reviewing this article to ensure your setup was completed as Confluence expects it to be:

Regards,

Shannon

Tran Tien dung November 13, 2018

Hi Shannon,

Yes, bypassing nginx is ok and I configured nginx definiely the same this article.

Thanks.

Shannon S
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 14, 2018

Hi Tran,

Alright, thank you for confirming that. One more thing you can try is to enable access logs to all URLs.

Uncomment this line in confluence/WEB-INF/classes/log4j.properties:

log4j.category.com.atlassian.confluence.util.AccessLogFilter=INFO

And then add the alternative filter:

An alternative filter: For troubleshooting purposes, often it is useful to capture all accesses to Confluence. To do this use the following filter mapping in confluence/WEB-INF/web.xmlinstead of the above:

<filter-mapping>
   <filter-name>AccessLogFilter</filter-name>
   <url-pattern>/*</url-pattern>
</filter-mapping>

You'll want to first see what errors you see while behind NGINX, and then while bypassing it, and compare the two. This will help you to understand what the issue is. Be sure to use the same Confluence page URL when testing.

 At this point we can confirm that it's not an issue with Confluence, but your NGINX setup. It may be that NGINX doesn't support the language or characters you're using. In general, we do not provide support for NGINX, so you may find it useful to contact NGINX directly.

Regards,

Shannon

Tran Tien dung November 14, 2018

Hi Shannon,

Yes, definely from Nginx, I enable a access log as your advise, I can see any requests from clients to Confluence except a page name with character ">", seen like nginx  that request, it not even go to confluence upstream.

I try to contact nginx directly, thanks for your link.

Shannon S
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 15, 2018

You're welcome, Tran. 

 

What about the solution that was proposed to your question on StackOverflow?

 

Probably you'll need to add a rewrite rule.

 

if ($request_uri ~ ^(/.*)[>](.*)$) { 
    return 301 $1%3E$2;
}
if ($request_uri ~ ^(/.*)[<](.*)$) { 
    return 301 $1%3C$2;
}
You may want to place this inside a location block to limit the scope. See this caution on the use of if.

Did that help?

Shannon

Tran Tien dung November 15, 2018

Hi Shannon,

I tried that but no luck. In fact, nginx return a correct uri, so no need this rewrite rule, nginx just somehow reject the request.

Thanks.

Shannon S
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 16, 2018

Alright, I thought I'd just check that.

If you follow-up with that response let them know it didn't work, and perhaps they can help to find another solution, otherwise NGINX does have their own support and community.

Once you're able to resolve the issue you can come back here and let us know how it was solved. Even though we don't support NGINX issues here I'm sure we still have users coming here for similar issues and they would find your answer useful!

Regards,

Shannon

Like Jimmy Van (GLiNTECH) likes this
0 votes
Claus Westerkamp December 11, 2018

I confirm 400 is thrown regardless accessing the application port or webserver port

imho it is no nginx issue

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events