Please will you provide examples of use of the varnishlog and exec inputs, for parsing varnish?
I am trying to take varnish input, parse the field-names, and output to ElasticSearch as a single line/event per hit (ie a single line/event per URL hit, with many/all of the entries from varnish), like the "varnishncsa" script does. I also need to customise the field names (but I should be able to deal with that).
The varnishlog input creates lines containing the "varnish_fd" entry (which I assume is the common link for lines), but I don't know how I would group the lines together using logstash (as the lines with the same varnish_fd are not necessarily successive).
If I use the exec input to run "varnishncsa -F '-a -b -c -d' ", what do I put as the interval (as varnishncsa is a long-running process, so you only run it once)?
Thanks - Ivan
I've just started looking into the varnish input module and also noticed it would be difficult to aggregate the varnish_fd entries as you noted. So I too am at a loss as to how to properly get varnish info into logstash in a usable manner.