Make default .sincedb location configurable
Description
Gliffy Diagrams
Activity
Show:
Former user January 2, 2012 at 5:13 PM
released ruby-filewatch 0.3.2, integrated into logstash's gemfile, and fixed some default sincedb_path handling in the file input. now working:
0 carrera(~/l/logstash) % ls -l /tmp/.sincedb.petef
ls: cannot access /tmp/.sincedb.petef: No such file or directory
2 carrera(~/l/logstash) % env SINCEDB_PATH=/tmp/.sincedb.petef ruby --1.9 bin/logstash agent -f etc/agent.conf &
[1] 23195
2 carrera(~/l/logstash) % logger -p mail.info logtest1
0 carrera(~/l/logstash) % 2012-01-02T17:12:12.578000Z file://carrera.databits.net//var/log/maillog: Jan 2 11:12:11 carrera root: logtest1
0 carrera(~/l/logstash) % ls -l /tmp/.sincedb.petef
-rw-r--r--. 1 petef petef 21 Jan 2 12:12 /tmp/.sincedb.petef
0 carrera(~/l/logstash) % fg
[1] + running env SINCEDB_PATH=/tmp/.sincedb.petef ruby --1.9 bin/logstash agent -f
^CW, [2012-01-02T12:12:23.506000 #23195] WARN -- : SIGINT received, shutting down. {"timestamp":"2012-01-02T12:12:23.499000 -0500","message":"SIGINT received, shutting down.","level":"warn"}
0 carrera(~/l/logstash) % logger -p mail.info logtest2
0 carrera(~/l/logstash) % logger -p mail.info logtest3
0 carrera(~/l/logstash) % env SINCEDB_PATH=/tmp/.sincedb.petef ruby --1.9 bin/logstash agent -f etc/agent.conf
2012-01-02T17:12:37.808000Z file://carrera.databits.net//var/log/maillog: Jan 2 11:12:27 carrera root: logtest2
2012-01-02T17:12:37.812000Z file://carrera.databits.net//var/log/maillog: Jan 2 11:12:29 carrera root: logtest3Former user January 1, 2012 at 4:13 PM
I'll check a SINCEDB_PATH env variable in filewatch.
Fixed
Details
Details
Assignee
Former user
Former user(Deactivated)Reporter
James Carpenter
James Carpenter(Deactivated)Fix versions
Affects versions
Created December 2, 2011 at 9:30 PM
Updated April 19, 2013 at 8:20 PM
Resolved January 2, 2012 at 5:13 PM
The logstash input file plugin currently defaults sincedb_path to #{ENV['HOME']}/.sincedb
It would be very useful to be able to set an environment and/or system variable such as DEFAULT_SINCE_DB_HOME (or DEFAULT_SINCE_DB_PATH) and/or provide a command line argument to specify this.
A cascade of environment variables would work as well. (i.e.: If $DEFAULT_SINCE_DB_HOME is unavailable, then use $HOME, etc.)
Whether this is done via a command line argument, environment variable, java system variable or a single global default section in the configuration file isn't all that important to me. Any of these solutions would solve my problem of providing a simply DRY way of specifying the default .sincedb location.
---------------
MOTIVATION:
When deploying the shipper to a large number of nodes with configuration files monitoring a large number of files, it would be awkward (NOT DRY) to specify the sincedb_path in every single file plugin specification in the config file on every node. At the same time, setting $HOME to the deployment directory (i.e. /opt/logstash-agent) doesn't make a lot of sense either.