use local timezone instead of utc when creating elasticsearch indices
I noticed logstash creates new index in elasticsearch at utc 00:00:00 while using output elasticsearch_http. It would be nice if it was based on local timezone, thanks.
This feature is useful when reparse an parsed log file, either.
For now, I must find out where to begin parse in log file (usualy very big) to avoid duplicate docs in index for last day in Elasticsearch.
For infosec (at least) everything is done in UTC - purely because it keeps everything even when storing the data. The output should change that into local timezone.
yes, the biggest benefit is that, it will be clear and accurate to write scripts to delete old indexes by date.
Yes. For example, most of our event activity happens during US business hours. Similarly, most of our searches are interested in that time span as well. Finally, we keep the last X days of events by deleting older indexes from ElasticSearch where day is generally around some US timezone instead of UTC. I guess more than anything, just simplifies things greatly from an operational and query perspective.
This would probably affect Kibana quite a bit, however. We found Kibana 2 (during a prototype w/o logstash) assumes UTC and will therefore chop off a lot of data because it doesn't think a time span should exist in other indexes.
Is there an expected benefit to having logstash create indexes based on midnight in your local time?