We are trying to ship jira application logs via Fluent d. Need a multiline regex to parse

Naresh Kumar Kethavath April 26, 2021

Below is the sample log we using :

2021-04-16 05:26:12,470-0400 localq-reader-2 ERROR [c.a.j.c.distribution.localq.LocalQCacheOpReader] Abandoning sending: LocalQCacheOp{cacheName='com.atlassian.jira.crowd.embedded.ofbiz.EagerOfBizGroupCache.groupCache', action=PUT, key={10000,ccs-cmt engineering}, value=Group[CCS-CMT Engineering:10000], creationTimeInMillis=1618503048658} from cache replication queue: [queueId=queue_ip10128170230factsetcom_1_c4cc94603bb736075782a80b0b7d53d4, queuePath=/home/svc-jira/jira/home/localq/queue_ip10128170230factsetcom_1_c4cc94603bb736075782a80b0b7d53d4], failuresCount: 10/10. Removing from queue. Error: java.rmi.UnknownHostException: Unknown host: ip-10-136-98-135.factset.com; nested exception is:
java.net.UnknownHostException: ip-10-136-98-135.factset.com
com.atlassian.jira.cluster.distribution.localq.LocalQCacheOpSender$UnrecoverableFailure: java.rmi.UnknownHostException: Unknown host: ip-10-136-98-135.factset.com; nested exception is:
java.net.UnknownHostException: ip-10-136-98-135.factset.com
at com.atlassian.jira.cluster.distribution.localq.rmi.LocalQCacheOpRMISender.send(LocalQCacheOpRMISender.java:88)
at com.atlassian.jira.cluster.distribution.localq.LocalQCacheOpReader.run(LocalQCacheOpReader.java:84)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.rmi.UnknownHostException: Unknown host: ip-10-136-98-135.factset.com; nested exception is:
java.net.UnknownHostException: ip-10-136-98-135.factset.com
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:620)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342)
at com.atlassian.jira.cluster.distribution.localq.rmi.BasicRMICachePeerProvider.lookupRemoteCachePeer(BasicRMICachePeerProvider.java:64)
at com.atlassian.jira.cluster.distribution.localq.rmi.BasicRMICachePeerProvider.create(BasicRMICachePeerProvider.java:39)
at com.atlassian.jira.cluster.distribution.localq.rmi.CachingRMICachePeerManager.getCachePeerFor(CachingRMICachePeerManager.java:58)
at com.atlassian.jira.cluster.distribution.localq.rmi.CachingRMICachePeerManager.withCachePeer(CachingRMICachePeerManager.java:91)
at com.atlassian.jira.cluster.distribution.localq.rmi.LocalQCacheOpRMISender.send(LocalQCacheOpRMISender.java:63)
... 6 more

 

Please find the below configuration using to parse

<source>
@type tail
@[deleted]_level debug
path /etc/td-agent/config.d/sample_log.log
pos_file /etc/td-agent/config.d/test.pos
format multiline
format_firstline /\d{4}-\d{1,2}-\d{1,2}/
format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2},\d{3})\s+(?<thread>.*)\s(?<level>(WARN|DEBUG|INFO|ERROR|FATAL))\s+(.*)?\s*\[(?<class>[^\s]*)\]\s+(?<message>.*)/
tag file.debug
</source>

<filter file.**>
@type record_transformer
<record>
__clp_index test
__clp_type aws_uat_application_jira
</record>
</filter>


## match tag=debug.** and dump to console
<match file.debug>
@type stdout
@[deleted]output_stdout
</match>

<match file.**>
@type forward
expire_dns_cache 300
dns_round_robin true
@[deleted]_level info
transport tls
keepalive true
keepalive_timeout 300s
<server>
host fluentd.log-uat.factset.com
port 24224
</server>
<buffer>
flush_interval 5s
</buffer>
</match>

Facing errors in console any better multiline regex?

1 answer

Suggest an answer

Log in or Sign up to answer
0 votes
Charlie Misonne
Community Leader
Community Leader
Community Leaders are connectors, ambassadors, and mentors. On the online community, they serve as thought leaders, product experts, and moderators.
September 7, 2023

Hi @Naresh Kumar Kethavath

Did you manage to construct the regexes properly? We're trying to do the same.

TAGS
AUG Leaders

Atlassian Community Events