[ LOGSTASH 1.3.3 / “metrics” filter ] : Crashing with “add a new key into hash during iteration” exception.

Description

Guys, Hi. Thanks for all your work. It is outstanding! I have an issue with Logstash 1.3.3. I am noticing the following exception while using the "metrics" filter. The crash basically blocks my ingestion pipe-line and I have to re-start the service as a consequence. I am in turn using only one instance of the filter. I noticed a few threads talking about the same symptoms. They are however mentions of some multi-threaded configuration for that filter, which is not the case in my set-up. I am running only one instance. Nevertheless:

[A.] Here is the exception:
...
{:timestamp=>"2014-02-03T17:06:06.216000+0000", :message=>"Exception in filterworker", "exception"=>#<RuntimeError: can't add a new key into hash during iteration>, "backtrace"=>["org/jruby/RubyHash.java:985:in `[]='", "file:/test/monitoring/app/logstash/logstash-1.3.3-flatjar.jar!/logstash/filters/metrics.rb:222:in `initialize_metrics'", "org/jruby/RubyProc.java:271:in `call'", "org/jruby/RubyHash.java:702:in `default'", "org/jruby/RubyHash.java:1086:in `[]'", "file:/test/monitoring/app/logstash/logstash-1.3.3-flatjar.jar!/logstash/filters/metrics.rb:161:in `filter'", "org/jruby/RubyArray.java:1613:in `each'", "file:/test/monitoring/app/logstash/logstash-1.3.3-flatjar.jar!/logstash/filters/metrics.rb:160:in `filter'", "(eval):917:in `initialize'", "org/jruby/RubyProc.java:271:in `call'", "file:/test/monitoring/app/logstash/logstash-1.3.3-flatjar.jar!/logstash/pipeline.rb:255:in `filter'", "file:/test/monitoring/app/logstash/logstash-1.3.3-flatjar.jar!/logstash/pipeline.rb:196:in `filterworker'", "file:/test/monitoring/app/logstash/logstash-1.3.3-flatjar.jar!/logstash/pipeline.rb:136:in `start_filters'"], :level=>:error}
...

[B.] A snippet from my "logstash.conf":

metrics {
meter => [
"stats.logstash.ingestion-host.c788ach",
"stats.logstash.ingestion-host.c788ach.%{host}"
]
rates => [ 1, 5 ]
flush_interval => 12
add_tag => [ "metric_message" ]
}
...
output {
if ("metric_message" in [tags]) {
graphite {
host => "metric-ingest-line.testserver.com"
port => 2003
include_metrics => [ "stats.logstash.ingestion-host.*" ]
fields_are_metrics => true
}
} else {
rabbitmq {
...

Thanks in advance,

Regards, Andrei

Assignee

Logstash Developers

Reporter

Andrei Chiriaev

Labels

None

Affects versions

Configure