Many many many many..
many people report performance problems with the AMQP input plugin. I think it's Bunny, but frankly I haven't had the energy to stomach more AMQP debugging to figure out why or where the slowdown is.
However, folks are also reporting that you can increase throughput if you start with this:
And simply duplicate exactly the same amqp config, you get more performance:
While I don't think this should be necessary (the input should be fast enough on its own, what is really CPU-bound here?), in many cases it is a nice workaround.
Since I think we'll need lmany of these work arounds, we might as well make it a feature, and I think we'll find it a good feature.
For plugins to support this, we should make a mixin 'LogStash:lugin::Threadable' that adds 'config :threads, ... whatever' and otherwise set it such that the Agent (or something external to the plugin) can see how many threads to create.
The effect, for now, will be to exactly replicate the plugin N times where 'threads => N'
pull merged by jordan