use local timezone instead of utc when creating elasticsearch indices

Description

I noticed logstash creates new index in elasticsearch at utc 00:00:00 while using output elasticsearch_http. It would be nice if it was based on local timezone, thanks.

Activity

Show:
尹川
April 30, 2014, 6:36 AM

This feature is useful when reparse an parsed log file, either.
For now, I must find out where to begin parse in log file (usualy very big) to avoid duplicate docs in index for last day in Elasticsearch.

Jason Kendall
March 6, 2014, 9:03 PM

For infosec (at least) everything is done in UTC - purely because it keeps everything even when storing the data. The output should change that into local timezone.

Jie Pan
July 8, 2013, 5:32 AM

yes, the biggest benefit is that, it will be clear and accurate to write scripts to delete old indexes by date.

Andrew Ochsner
June 25, 2013, 6:09 PM

Yes. For example, most of our event activity happens during US business hours. Similarly, most of our searches are interested in that time span as well. Finally, we keep the last X days of events by deleting older indexes from ElasticSearch where day is generally around some US timezone instead of UTC. I guess more than anything, just simplifies things greatly from an operational and query perspective.

This would probably affect Kibana quite a bit, however. We found Kibana 2 (during a prototype w/o logstash) assumes UTC and will therefore chop off a lot of data because it doesn't think a time span should exist in other indexes.

Jordan Sissel
April 1, 2013, 7:30 PM

Is there an expected benefit to having logstash create indexes based on midnight in your local time?

Won't Fix

Assignee

Logstash Developers

Reporter

Jie Pan