Yeah, fine to close down by me. Thanks a lot, man.
the flatjar/monolithic thing might be logstash specific, but it's not related to the original issue here.
Michael Klishin July 10, 2013 at 6:28 PM
Sure. Can this issue be closed then?
I would be interesting in bringing this up to the JRuby team's attention. Sounds like a pretty inconvenient thing for Logstash users that may want the convenience of deploying a single JAR.
DickD July 10, 2013 at 6:27 PM
Yup, that's the fella. Starts in about 30 secs with the monolithic jar, seems to use less heap too (NB: I am not a real doctor).
Sorry for the false alarm I'll do some throughput testing next (might be tomorrow now).
Michael Klishin July 10, 2013 at 6:26 PM
Edited
For the record: Bunny indeed uses Ruby's OpenSSL extension but HotBunnies does not (all the networking parts are implemented in Java, with zero dependencies).
java -server -jar logstash/build/logstash-1.1.10.dev-monolithic.jar agent -v -f bug.logstash.conf # HEAD from github, java 6 on mac or linux
logstash.conf - tcp input is never used, rabbitmq shows amqp connects fine.
logstash-bug:$ cat bug.logstash.conf
input {
tcp {
port => 1337
type => 'dummy'
}
}
output {
same behaviour if you replace 'rabbitmq' with 'amqp'
rabbitmq {
exchange => "logstash"
exchange_type => "topic"
host => "amqpserver"
vhost => "/logstash"
user => "loguser"
password => "logpass"
}
}
----------------------
After around 5 minutes, jvisualvm sampling shows 95% of cpu being used by:
RubyThread-12: jar:file:/Users/sisred/logstash-gc-hell/logstash/build/logstash-1.1.10.dev-monolithic.jar!/gems/bunny-0.9.0.pre7/lib/bunny/main_loop.rb:18 428513905000 428,513.905 (0.0%) 949.062
permgen and old generations are stable, but GCs of young gen. happen multiple times a second.
jvisualvm per-thread allocations show the same thread allocating 100Mb/sec:
RubyThread-12: jar:file:/Users/sisred/logstash-snmp-gc-hell/logstash/build/logstash-1.1.10.dev-monolithic.jar!/gems/bunny-0.9.0.pre7/lib/bunny/main_loop.rb:18 57059376016 57,059,376,016 (78.6%) 100,741,365