access_log format help

Chris Laplante April 17, 2015

I've been tasked with adding JIRA logs to our log monitoring tool (logstash). Problem being no one at our company can give me a clear description of what the various fields in the access_log file are. 

192.3.2.1 687x406810x28 userX [17/Apr/2015:11:27:43 -0700] "GET /rest/zephyr/latest/audit?entityType=TESTSTEP&maxRecords=20&offset=0&issueId=950030&_=1429295263223 HTTP/1.1" 200 47 91 "https://jira.company.com/issues/?filter=39318" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.82 Safari/537.36" "2c0qst"

Especially what the 200 47 91  represents in the this example. The first one obviously looks like an HTTP status code, the last two I don't know.

Could someone provide a description or if there are any logstash users out there your grok filter?

 

Thanks,

 

-Chris

 

3 answers

3 votes
crf
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 17, 2015

The log format is specified by the AccessLogValve in server.xml:

<!--
                ====================================================================================
                 Access Logging.
                 This should produce access_log.<date> files in the 'logs' directory.
                 The output access log lies has the following fields :
                 IP Request_Id User Timestamp  "HTTP_Method URL Protocol_Version" HTTP_Status_Code ResponseSize_in_Bytes RequestTime_In_Millis Referer User_Agent ASESSIONID
                 eg :
                 192.168.3.238 1243466536012x12x1 admin [28/May/2009:09:22:17 +1000] "GET /jira/secure/admin/jira/IndexProgress.jspa?taskId=1 HTTP/1.1" 200 24267 1070 "http://carltondraught.sydney.atlassian.com:8090/jira/secure/admin/jira/IndexAdmin.jspa" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10" "C2C99B632EE0F41E90F8EF7A201F6A78"
                 NOTES:
                 The RequestId is a millis_since_epoch plus request number plus number of concurrent users
                 The Request time is in milliseconds
                 The ASESSIONID is an hash of the JSESSIONID and hence is safe to publish within logs.  A session cannot be reconstructed from it.
                 See http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html for more information on Tomcat Access Log Valves
                ====================================================================================
            -->
            <Valve className="org.apache.catalina.valves.AccessLogValve" resolveHosts="false"
                   pattern="%a %{jira.request.id}r %{jira.request.username}r %t "%m %U%q %H" %s %b %D "%{Referer}i" "%{User-Agent}i" "%{jira.request.assession.id}r""/>

 

The three numbers you were asking about are %s %b %D there, which are respectively the HTTP status code, the number of bytes sent excluding HTTP headers (or - instead of 0), and the duration of the request in milliseconds.  So the request you asked about responded with 200 OK, delivered a payload of 47 bytes, and took a whopping 91ms to run.

 

0 votes
Seb Kouba March 17, 2017

This should match all JIRA access log entries as per default settings:

^%{IPORHOST:clientip} %{HTTPDUSER:requestid} %{HTTPDUSER:user} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:HTTP_Method}) %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})\" %{NUMBER:response} %{NOTSPACE:bytes} %{NOTSPACE:request_time} %{QS:referrer} %{QS:user_agent} %{QS:sessionid}$

I can recommend these sites for testing & infos: 

http://grokconstructor.appspot.com/

http://grokdebug.herokuapp.com/

https://www.elastic.co/blog/do-you-grok-grok

Note:

You don't have to escape " in the online constructors but you do when using the filter in Logstash.

0 votes
MattS
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
October 28, 2015

For the JIRA Tomcat access log format, which is different from the JIRA access log, in install.dir/logs/access_log* see https://jira.atlassian.com/browse/JRA-42894

MattS
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
April 11, 2017

I'm not sure I got that right!

 

Suggest an answer

Log in or Sign up to answer