If filter replaces @timestamp with a string flushing of event fails

Description

For background I have apache logging in json_event format and I am using rsyslog to ship logs. I am using the syslog input and then filtering the resultant message through the json plugin to replace the current body with the parsed output. This replaced the @timestamp field with a string which results in

{:timestamp=>"2013-09-12T10:43:38.904000-0500", :message=>"Failed to flush outgoing items", :outgoing_count=>100, :exception=>#<NoMethodError: undefined method `tv_sec' for "2013-09-12T08:50:59-0500":String>, :backtrace=>["file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:239:in `sprintf'", "org/jruby/RubyString.java:3062:in `gsub'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:225:in `sprintf'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/elasticsearch_http.rb:64:in `flush'", "org/jruby/RubyArray.java:2412:in `collect'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/elasticsearch_http.rb:63:in `flush'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1332:in `each'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:216:in `buffer_flush'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:193:in `buffer_flush'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:159:in `buffer_receive'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/outputs/elasticsearch_http.rb:59:in `receive'", "(eval):161:in `initialize'", "org/jruby/RubyProc.java:255:in `call'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:247:in `output'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:212:in `outputworker'", "file:/usr/share/logstash/logstash-1.2.1-flatjar.jar!/logstash/pipeline.rb:140:in `start_outputs'"], :level=>:warn}

The solution might be to just not do that. That is my current work around. Otherwise it would be possible to test the field and convert it on demand if it is a string.

It would also be nice if a failing event wouldn't stall the entire pipeline. Maybe just log it and move on.

Activity

Show:
Chris Denneen
October 17, 2013, 3:48 PM

Noticed similar tv_sec behavior reading off redis queue:

Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `tv_sec' for "2013-10-16T18:04:52.000Z":String
at RUBY.sprintf(file:/opt/logstash/logstash.jar!/logstash/event.rb:226)
at org.jruby.RubyString.gsub(org/jruby/RubyString.java:3062)
at RUBY.sprintf(file:/opt/logstash/logstash.jar!/logstash/event.rb:212)
at RUBY.receive(file:/opt/logstash/logstash.jar!/logstash/outputs/elasticsearch.rb:153)
at RUBY.worker_setup(file:/opt/logstash/logstash.jar!/logstash/outputs/base.rb:65)

Andi B.
October 18, 2013, 12:11 PM

Same issue here.

my configuration:

the log message (logged with logstash/log4j-jsonevent-layout)

and the error message:

is there a known work around?

Chris Denneen
October 18, 2013, 5:07 PM

I was able to work around this with date filter like above tried:

input {
redis{
host => 'redis'
data_type => 'list'
key => 'logstash'
type => 'redis-input'
codec => 'plain' # used to work around json codec
}
}

filter {
json { source => message }
date { match => ["@timestamp", "ISO8601"] }
}

Klavs Klavsen
January 10, 2014, 1:28 PM

isn't this a duplicate of https://logstash.jira.com/browse/LOGSTASH-1623 ?
( just hit this same issue.. )

Philippe Weber
August 4, 2014, 10:32 AM

Closing as duplicate of
Follow-up occuring in https://github.com/elasticsearch/logstash/issues/1250

Duplicate

Assignee

Logstash Developers

Reporter

Nathan Huff

Labels

None

Affects versions

Configure