Logstash thread watchdog timeout crash
Description
Attachments
2
- 05 Apr 2013, 07:51 AM
- 05 Apr 2013, 07:51 AM
Gliffy Diagrams
Activity
Show:
Aaron Sterr July 4, 2013 at 11:43 PMEdited
This happened to me as well when I added a fourth pattern to one of my grok filters.
Here is the grok filter:
grok {
type => "lhdr"
tags => [ 'lhdr', 'lhdrlog' ]
pattern => '%{NUMBER} \[%{DATA:thread}\] %{WORD:priority} %{DATA:class} %{GREEDYDATA:message}'
pattern => '\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{WORD:priority}\]\[%{DATA:thread}\]\[%{DATA:class}\] %{GREEDYDATA:message}'
pattern => '\[(?<timestamp>%{SYSLOGTIMESTAMP} %{DATA})\] \[%{DATA:thread}\] %{GREEDYDATA:message}'
pattern => '%{DATA:priority} (?<timestamp>%{DATA}+) \[%{DATA:thread}\] %{DATA:class} %{GREEDYDATA:message}'
}
Removing the fourth pattern fixes the problem.
For reference, I am currently indexing ~20 million messages per day. It has been holding up so far.
Actually – I found another work around. I changed the pattern to this and now I am working:
%{DATA:priority} (?<timestamp>.+) \[%{DATA:thread}\] %{DATA:class} %{GREEDYDATA:message}
Maybe using the DATA pattern repeatedly was too expensive?
Details
Details
Assignee
Logstash Developers
Logstash DevelopersReporter
Philipp H
Philipp HFix versions
Affects versions
Created April 5, 2013 at 7:51 AM
Updated September 21, 2013 at 3:15 PM
Logstash crashes with my nginx grok filter every few minutes or hours. Before my grok filter was simpler and did not contain more than two patterns...
I have attached the crash log and my nginx pattern.
Logstash receives a few million messages per day in our setup.