Inputs should be duplicatable to run more than one of the same plugin


Many many many many..

many people report performance problems with the AMQP input plugin. I think it's Bunny, but frankly I haven't had the energy to stomach more AMQP debugging to figure out why or where the slowdown is.

However, folks are also reporting that you can increase throughput if you start with this:

And simply duplicate exactly the same amqp config, you get more performance:

While I don't think this should be necessary (the input should be fast enough on its own, what is really CPU-bound here?), in many cases it is a nice workaround.

Since I think we'll need lmany of these work arounds, we might as well make it a feature, and I think we'll find it a good feature.

Proposed example:

For plugins to support this, we should make a mixin 'LogStash:lugin::Threadable' that adds 'config :threads, ... whatever' and otherwise set it such that the Agent (or something external to the plugin) can see how many threads to create.

The effect, for now, will be to exactly replicate the plugin N times where 'threads => N'


Nick Ethier
June 8, 2012, 6:16 PM
June 10, 2012, 3:46 PM

pull merged by jordan


Logstash Developers


Jordan Sissel



Fix versions

Affects versions