Gelfd TooManyChunksError - gelf listener died
Description
Gliffy Diagrams
Activity
Dan Ballinger August 27, 2013 at 10:49 PMEdited
In Summary:
1) Too many chunks error causes the gelfd listener to die.
2) The listener tries to restart using the udp_listener.
3) There appears to be an issue with the udp_listener restart in inputs/gelf.rb
Dan Ballinger August 27, 2013 at 10:41 PMEdited
This message is followed by the following in 5 second intervals:
{:timestamp=>"2013-08-06T11:42:46.666000-0700", :message=>"gelf listener died", :exception=>#<IOError: not opened for reading>, :backtrace=>["org/jruby/ext/socket/RubyBasicSocket.java:410:in `close_read'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/inputs/gelf.rb:72:in `udp_listener'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/inputs/gelf.rb:59:in `run'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/agent.rb:761:in `run_input'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/agent.rb:407:in `start_input'"], :level=>:warn}
Environment:
logstash-1.1.13-flatjar.jar
elasticsearch 0.20.6
redis-2.6.11
FreeBSD 9.1-RELEASE-p3
Windows 2k8 server with .net gelf
GELF 1.0.2.12
input {
gelf {
debug => true
host => "0.0.0.0"
format => plain
port => 5515
type => gelf
}
}
output {
redis {
host => "127.0.0.1"
type => "redis-input"
these settings should match the output of the agent
data_type => "list"
key => "logstash"
We use json_event here since the sender is a logstash agent
message_format => "json_event"
}
}
logstash log:
{:timestamp=>"2013-08-06T11:42:41.650000-0700", :message=>"gelf listener died", :exception=>#<Gelfd::TooManyChunksError: 166 greater than 128>, :backtrace=>["file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/gelfd/chunked_parser.rb:40:in `parse_chunk'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/gelfd/chunked_parser.rb:8:in `parse'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/gelfd/parser.rb:10:in `parse'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/inputs/gelf.rb:83:in `udp_listener'", "org/jruby/RubyKernel.java:1409:in `loop'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/inputs/gelf.rb:79:in `udp_listener'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/inputs/gelf.rb:59:in `run'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/agent.rb:761:in `run_input'", "file:/usr/local/logstash/logstash-1.1.13-flatjar.jar!/logstash/agent.rb:407:in `start_input'"], :level=>:warn}