access_log format help

I've been tasked with adding JIRA logs to our log monitoring tool (logstash). Problem being no one at our company can give me a clear description of what the various fields in the access_log file are. 

192.3.2.1 687x406810x28 userX [17/Apr/2015:11:27:43 -0700] "GET /rest/zephyr/latest/audit?entityType=TESTSTEP&maxRecords=20&offset=0&issueId=950030&_=1429295263223 HTTP/1.1" 200 47 91 "https://jira.company.com/issues/?filter=39318" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.82 Safari/537.36" "2c0qst"

Especially what the 200 47 91  represents in the this example. The first one obviously looks like an HTTP status code, the last two I don't know.

Could someone provide a description or if there are any logstash users out there your grok filter?

 

Thanks,

 

-Chris

 

3 answers

This widget could not be displayed.
Chris Fuller Atlassian Team Apr 17, 2015

The log format is specified by the AccessLogValve in server.xml:

<!--
                ====================================================================================
                 Access Logging.
                 This should produce access_log.<date> files in the 'logs' directory.
                 The output access log lies has the following fields :
                 IP Request_Id User Timestamp  "HTTP_Method URL Protocol_Version" HTTP_Status_Code ResponseSize_in_Bytes RequestTime_In_Millis Referer User_Agent ASESSIONID
                 eg :
                 192.168.3.238 1243466536012x12x1 admin [28/May/2009:09:22:17 +1000] "GET /jira/secure/admin/jira/IndexProgress.jspa?taskId=1 HTTP/1.1" 200 24267 1070 "http://carltondraught.sydney.atlassian.com:8090/jira/secure/admin/jira/IndexAdmin.jspa" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10" "C2C99B632EE0F41E90F8EF7A201F6A78"
                 NOTES:
                 The RequestId is a millis_since_epoch plus request number plus number of concurrent users
                 The Request time is in milliseconds
                 The ASESSIONID is an hash of the JSESSIONID and hence is safe to publish within logs.  A session cannot be reconstructed from it.
                 See http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html for more information on Tomcat Access Log Valves
                ====================================================================================
            -->
            <Valve className="org.apache.catalina.valves.AccessLogValve" resolveHosts="false"
                   pattern="%a %{jira.request.id}r %{jira.request.username}r %t "%m %U%q %H" %s %b %D "%{Referer}i" "%{User-Agent}i" "%{jira.request.assession.id}r""/>

 

The three numbers you were asking about are %s %b %D there, which are respectively the HTTP status code, the number of bytes sent excluding HTTP headers (or - instead of 0), and the duration of the request in milliseconds.  So the request you asked about responded with 200 OK, delivered a payload of 47 bytes, and took a whopping 91ms to run.

 

This widget could not be displayed.

For the JIRA Tomcat access log format, which is different from the JIRA access log, in install.dir/logs/access_log* see https://jira.atlassian.com/browse/JRA-42894

I'm not sure I got that right!

 

This widget could not be displayed.

This should match all JIRA access log entries as per default settings:

^%{IPORHOST:clientip} %{HTTPDUSER:requestid} %{HTTPDUSER:user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:HTTP_Method}) %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})\" %{NUMBER:response} %{NOTSPACE:bytes} %{NOTSPACE:request_time} %{QS:referrer} %{QS:user_agent} %{QS:sessionid}$

I can recommend these sites for testing & infos: 

http://grokconstructor.appspot.com/

http://grokdebug.herokuapp.com/

https://www.elastic.co/blog/do-you-grok-grok

Note:

You don't have to escape " in the online constructors but you do when using the filter in Logstash.

Suggest an answer

Log in or Sign up to answer
Atlassian Summit 2018

Meet the community IRL

Atlassian Summit is an excellent opportunity for in-person support, training, and networking.

Learn more
Community showcase
Posted Wednesday in New to Jira

Are you planning to trial, or are currently trialling Jira Software? - We want to talk to you!

Hello! I'm Rayen, a product manager at Atlassian. My team and I are working hard to improve the trial experience for Jira Software Cloud. We are interested in   talking to 20 people planning t...

139 views 2 0
Join discussion

Atlassian User Groups

Connect with like-minded Atlassian users at free events near you!

Find a group

Connect with like-minded Atlassian users at free events near you!

Find my local user group

Unfortunately there are no AUG chapters near you at the moment.

Start an AUG

You're one step closer to meeting fellow Atlassian users at your local meet up. Learn more about AUGs

Groups near you