Encoding::ConverterNotFoundError: code converter not found (ASCII-8BIT to UTF-8)>

Description

Hi,
I am getting following error when I used geoip filter.

{:timestamp=>"2013-09-16T11:29:24.407000+0100", :message=>"Failed to flush outgoing items", :outgoing_count=>2, :exception=>#<Encoding::ConverterNotFoundError: code converter not found (ASCII-8BIT to UTF-8)>, :backtrace=>["org/jruby/RubyString.java:7590:in `encode'", "json/ext/GeneratorMethods.java:71:in `to_json'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/logstash/event.rb:169:in `to_json'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/logstash/outputs/elasticsearch.rb:163:in `flush'", "org/jruby/RubyArray.java:1617:in `each'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/logstash/outputs/elasticsearch.rb:158:in `flush'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1332:in `each'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:216:in `buffer_flush'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:112:in `buffer_initialize'", "org/jruby/RubyKernel.java:1489:in `loop'", "file:/C:/Projects/Search/log-stash/logstash-1.2.1-flatjar.jar!/stud/buffer.rb:110:in `buffer_initialize'"], :level=>:warn}

Config file and input file is also attached.

Activity

Show:
Aamir Khan
December 19, 2013, 10:57 AM

I just tried with the newer version failing this time as well

←[33mFailed to flush outgoing items {:outgoing_count=>52, :exception=>#<Encoding
::InvalidByteSequenceError: ""\xDB"" on ASCII-8BIT>, :backtrace=>["org/jruby/Rub
yString.java:7599:in `encode'", "json/ext/GeneratorMethods.java:71:in `to_json'"
, "file:/C:/Logging/logstash-1.3.1/logstash-1.3.1-flatjar.jar!/logstash/event.rb
:157:in `to_json'", "file:/C:/Logging/logstash-1.3.1/logstash-1.3.1-flatjar.jar!
/logstash/outputs/elasticsearch.rb:303:in `flush'", "org/jruby/RubyArray.java:16
13:in `each'", "file:/C:/Logging/logstash-1.3.1/logstash-1.3.1-flatjar.jar!/logs
tash/outputs/elasticsearch.rb:298:in `flush'", "file:/C:/Logging/logstash-1.3.1/
logstash-1.3.1-flatjar.jar!/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/Ru
byHash.java:1338:in `each'", "file:/C:/Logging/logstash-1.3.1/logstash-1.3.1-fla
tjar.jar!/stud/buffer.rb:216:in `buffer_flush'", "file:/C:/Logging/logstash-1.3.
1/logstash-1.3.1-flatjar.jar!/stud/buffer.rb:193:in `buffer_flush'", "file:/C:/L
ogging/logstash-1.3.1/logstash-1.3.1-flatjar.jar!/stud/buffer.rb:112:in `buffer_
initialize'", "org/jruby/RubyKernel.java:1519:in `loop'", "file:/C:/Logging/logs
tash-1.3.1/logstash-1.3.1-flatjar.jar!/stud/buffer.rb:110:in `buffer_initialize'
"], :level=>:warn}←[0m

Aamir Khan
December 24, 2013, 8:27 AM

Hi
I tried this morning with the newest release i.e. 1.3.2 and again the same error
What I noticed this time that postalcode pull some character which have invalid character sequence

←[32moutput received {:event=>#<LogStash::Event:0x6753044a @cancelled=false, @da
ta={"message"=>"98.138.253.109\r", "@version"=>"1", "@timestamp"=>"2013-12-24T08
:22:22.495Z", "host"=>"mipLoggingCenter", "path"=>"C:/ip.log", "customerip"=>"98
.138.253.109", "customerlocation"=>{"ip"=>"98.138.253.109", "country_code2"=>"HR
", "country_code3"=>"HRV", "country_name"=>"Croatia", "continent_code"=>"EU", "r
egion_name"=>"deros", "postal_code"=>"m\x9C\x16\xED\xA7\x11\r21", "latitude"=>57
6.1555, "longitude"=>484.84360000000004, "timezone"=>"Europe/Zagreb", "location"
=>[484.84360000000004, 576.1555]}}>, :level=>:info, :file=>"(eval)", :line=>"54"
}←[0m
{
"message" => "98.138.253.109\r",
"@version" => "1",
"@timestamp" => "2013-12-24T08:22:22.495Z",
"host" => "mipLoggingCenter",
"path" => "C:/ip.log",
"customerip" => "98.138.253.109",
"customerlocation" => {
"ip" => "98.138.253.109",
"country_code2" => "HR",
"country_code3" => "HRV",
"country_name" => "Croatia",
"continent_code" => "EU",
"region_name" => "deros",
"postal_code" => "m\x9C\x16\xED\xA7\x11\r21",
"latitude" => 576.1555,
"longitude" => 484.84360000000004,
"timezone" => "Europe/Zagreb",
"location" => [
[0] 484.84360000000004,
[1] 576.1555
]
}
}
Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (Inv
alidByteSequenceError) ""\x9C"" on ASCII-8BIT
at org.jruby.RubyString.encode(org/jruby/RubyString.java:7599)
at json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.ja
va:71)
at RUBY.to_json(file:/C:/Logging/logstash-1.3.2/logstash-1.3.2-flatjar.j
ar!/logstash/event.rb:168)
at RUBY.receive(file:/C:/Logging/logstash-1.3.2/logstash-1.3.2-flatjar.j
ar!/logstash/outputs/file.rb:72)
at RUBY.handle(file:/C:/Logging/logstash-1.3.2/logstash-1.3.2-flatjar.ja
r!/logstash/outputs/base.rb:86)
at RUBY.initialize((eval):56)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:271)
at RUBY.output(file:/C:/Logging/logstash-1.3.2/logstash-1.3.2-flatjar.ja
r!/logstash/pipeline.rb:259)
at RUBY.outputworker(file:/C:/Logging/logstash-1.3.2/logstash-1.3.2-flat
jar.jar!/logstash/pipeline.rb:218)
at RUBY.start_outputs(file:/C:/Logging/logstash-1.3.2/logstash-1.3.2-fla
tjar.jar!/logstash/pipeline.rb:145)

Romain Neutron
January 17, 2014, 4:16 PM

I got the same error with valid json on Logstash 1.3.2 :

{"@timestamp":"2014-01-17T17:10:05.583+01:00","@version":1,"message":"> GET /index_dev.php/_profiler/wdt/50161c","host":"10.143.115.20","type":"test39.preprod","channel":"default-logger","level":"INFO"}

Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (InvalidByteSequenceError) ""\x9C"" on ASCII-8BIT
at org.jruby.RubyString.encode(org/jruby/RubyString.java:7599)
at json.ext.GeneratorMethods$RbHash.to_json(json/ext/GeneratorMethods.java:71)
at RUBY.to_json(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/event.rb:168)
at RUBY.encode(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/codecs/json.rb:38)
at RUBY.receive(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/outputs/stdout.rb:50)
at RUBY.handle(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/outputs/base.rb:86)
at RUBY.initialize((eval):20)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:271)
at RUBY.output(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/pipeline.rb:259)
at RUBY.outputworker(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/pipeline.rb:218)
at RUBY.start_outputs(file:/home/utilities/logstash/logstash-1.3.2-flatjar.jar!/logstash/pipeline.rb:145)

Here's my conf :

input {
udp {
host => '10.143.115.20'
port => 12202
codec => json { charset => "UTF-8" }
}
}
output {
stdout { codec => json { charset => "UTF-8" } }
elasticsearch_http { host => '10.143.115.20' }
}

Jordan Sissel
February 13, 2014, 6:25 AM

This is fixed by https://github.com/logstash/logstash/pull/1058 with tests coverage in https://github.com/logstash/logstash/pull/1054

This will be fixed in logstash 1.4.0 when released.

Assignee

Jordan Sissel

Reporter

Aamir Khan

Labels

Fix versions

Affects versions

Configure