I’m using the S3 input plugin for logstash - it won’t parse S3 logs when I -f a folder with the s3 config, but it will parse them when I pass -f the config file. My apache configuration works fine both ways. I delete the sincedb file each time I test, and am not too sure what to try next.
I have the following structure:
sudo /usr/bin/logstash/bin/logstash agent --verbose -f /etc/logstash/conf/s3.conf
sudo /usr/bin/logstash/bin/logstash agent --verbose -f /etc/logstash/conf/
If I swap apache.conf (my apache configuration file, two file inputs) with s3.conf, parsing apache logs works either way.
An ELF file is created in /tmp/ (jffi1616993874545511618.tmp), instead of the s3.sincedb file I'm expecting when it isn't working.
I'm using logstash 1.4.2 on Amazon Linux, with Java 1.6.0.