1.1.10-dev amqp/rabbitmq output causes high CPU usage

Description

java -server -jar logstash/build/logstash-1.1.10.dev-monolithic.jar agent -v -f bug.logstash.conf # HEAD from github, java 6 on mac or linux

  1. logstash.conf - tcp input is never used, rabbitmq shows amqp connects fine.

logstash-bug:$ cat bug.logstash.conf

input {
tcp {
port => 1337
type => 'dummy'
}
}

output {

  1. same behaviour if you replace 'rabbitmq' with 'amqp'
    rabbitmq {
    exchange => "logstash"
    exchange_type => "topic"
    host => "amqpserver"
    vhost => "/logstash"
    user => "loguser"
    password => "logpass"
    }
    }

----------------------

After around 5 minutes, jvisualvm sampling shows 95% of cpu being used by:

RubyThread-12: jar:file:/Users/sisred/logstash-gc-hell/logstash/build/logstash-1.1.10.dev-monolithic.jar!/gems/bunny-0.9.0.pre7/lib/bunny/main_loop.rb:18 428513905000 428,513.905 (0.0%) 949.062

permgen and old generations are stable, but GCs of young gen. happen multiple times a second.

jvisualvm per-thread allocations show the same thread allocating 100Mb/sec:

RubyThread-12: jar:file:/Users/sisred/logstash-snmp-gc-hell/logstash/build/logstash-1.1.10.dev-monolithic.jar!/gems/bunny-0.9.0.pre7/lib/bunny/main_loop.rb:18 57059376016 57,059,376,016 (78.6%) 100,741,365

Attachments

2

Gliffy Diagrams

Activity

Show:

DickD July 10, 2013 at 6:54 PM

Resolves my original issue. nice work.

DickD July 10, 2013 at 6:53 PM

Yeah, fine to close down by me. Thanks a lot, man.

the flatjar/monolithic thing might be logstash specific, but it's not related to the original issue here.

Michael Klishin July 10, 2013 at 6:28 PM

Sure. Can this issue be closed then?

I would be interesting in bringing this up to the JRuby team's attention. Sounds like a pretty inconvenient thing for Logstash users that may want the convenience of deploying a single JAR.

DickD July 10, 2013 at 6:27 PM

Yup, that's the fella. Starts in about 30 secs with the monolithic jar, seems
to use less heap too (NB: I am not a real doctor).

Sorry for the false alarm I'll do some throughput testing next (might
be tomorrow now).

Michael Klishin July 10, 2013 at 6:26 PM
Edited

For the record: Bunny indeed uses Ruby's OpenSSL extension but HotBunnies does not (all the networking parts are implemented in Java, with zero dependencies).

Fixed

Details

Assignee

Reporter

Fix versions

Affects versions

Created March 5, 2013 at 4:41 PM
Updated July 10, 2013 at 6:54 PM
Resolved July 10, 2013 at 6:54 PM