Elasticsearch: introduce Routing parameter to save space


Elastic search allows to set a routing parameter when creating documents to have related documents in the same shard:
This makes sense for log messages as well: e.g. if you have extracted the session id from the message, it would be sensible to store all entries to that session in the same shard.

So I propose to introduce an additional configuration parameter "routing" similar to the existing parameters "index" and "index_type" that sets this parameter. If you have, for instance, stored the session id in the field sid, you might use

elasticsearch { ... routing => "%{sid}" ...}

I expect this to save space since session specific terms like user IDs and so forth are stored only in one shard.


Logstash Developers


Hans-Peter Störr

Fix versions

Affects versions