access_log format help

I've been tasked with adding JIRA logs to our log monitoring tool (logstash). Problem being no one at our company can give me a clear description of what the various fields in the access_log file are. 

192.3.2.1 687x406810x28 userX [17/Apr/2015:11:27:43 -0700] "GET /rest/zephyr/latest/audit?entityType=TESTSTEP&maxRecords=20&offset=0&issueId=950030&_=1429295263223 HTTP/1.1" 200 47 91 "https://jira.company.com/issues/?filter=39318" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.82 Safari/537.36" "2c0qst"

Especially what the 200 47 91  represents in the this example. The first one obviously looks like an HTTP status code, the last two I don't know.

Could someone provide a description or if there are any logstash users out there your grok filter?

 

Thanks,

 

-Chris

 

3 answers

2 votes
Chris Fuller Atlassian Team Apr 17, 2015

The log format is specified by the AccessLogValve in server.xml:

<!--
                ====================================================================================
                 Access Logging.
                 This should produce access_log.<date> files in the 'logs' directory.
                 The output access log lies has the following fields :
                 IP Request_Id User Timestamp  "HTTP_Method URL Protocol_Version" HTTP_Status_Code ResponseSize_in_Bytes RequestTime_In_Millis Referer User_Agent ASESSIONID
                 eg :
                 192.168.3.238 1243466536012x12x1 admin [28/May/2009:09:22:17 +1000] "GET /jira/secure/admin/jira/IndexProgress.jspa?taskId=1 HTTP/1.1" 200 24267 1070 "http://carltondraught.sydney.atlassian.com:8090/jira/secure/admin/jira/IndexAdmin.jspa" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10" "C2C99B632EE0F41E90F8EF7A201F6A78"
                 NOTES:
                 The RequestId is a millis_since_epoch plus request number plus number of concurrent users
                 The Request time is in milliseconds
                 The ASESSIONID is an hash of the JSESSIONID and hence is safe to publish within logs.  A session cannot be reconstructed from it.
                 See http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html for more information on Tomcat Access Log Valves
                ====================================================================================
            -->
            <Valve className="org.apache.catalina.valves.AccessLogValve" resolveHosts="false"
                   pattern="%a %{jira.request.id}r %{jira.request.username}r %t "%m %U%q %H" %s %b %D "%{Referer}i" "%{User-Agent}i" "%{jira.request.assession.id}r""/>

 

The three numbers you were asking about are %s %b %D there, which are respectively the HTTP status code, the number of bytes sent excluding HTTP headers (or - instead of 0), and the duration of the request in milliseconds.  So the request you asked about responded with 200 OK, delivered a payload of 47 bytes, and took a whopping 91ms to run.

 

0 votes

For the JIRA Tomcat access log format, which is different from the JIRA access log, in install.dir/logs/access_log* see https://jira.atlassian.com/browse/JRA-42894

I'm not sure I got that right!

 

This should match all JIRA access log entries as per default settings:

^%{IPORHOST:clientip} %{HTTPDUSER:requestid} %{HTTPDUSER:user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:HTTP_Method}) %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})\" %{NUMBER:response} %{NOTSPACE:bytes} %{NOTSPACE:request_time} %{QS:referrer} %{QS:user_agent} %{QS:sessionid}$

I can recommend these sites for testing & infos: 

http://grokconstructor.appspot.com/

http://grokdebug.herokuapp.com/

https://www.elastic.co/blog/do-you-grok-grok

Note:

You don't have to escape " in the online constructors but you do when using the filter in Logstash.

Suggest an answer

Log in or Join to answer
Community showcase
Sarah Schuster
Posted Jan 29, 2018 in Jira

What are common themes you've seen across successful & failed Jira Software implementations?

Hey everyone! My name is Sarah Schuster, and I'm a Customer Success Manager in Atlassian specializing in Jira Software Cloud. Over the next few weeks I will be posting discussion topics (8 total) to ...

2,777 views 11 18
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you
Atlassian Team Tour

Join us on the Team Tour

We're bringing product updates and pro tips on teamwork to ten cities around the world.

Save your spot