Am having problems with agent that reads from log files and outputs to ZeroMQ. Ran the logstash process with command" java -jar logstash-1.1.0-monolithic.jar agent -vvv -f /etc/logstash/agent.conf" and in the output I get two ClassNotFoundException for filewatch.TailService & Ffi-rzmqService.
I have uploaded the log output in the attached file as well as the logstash config file.
Am running on Ubuntu 11.10 32-bit with OpenJDK version 6 JRE.
I'm going to assume you have the zeromq libs installed on the system where you're running logstash and in a common place (i.e. /usr/local/lib/ or /usr/lib).
If not, you'll need to do that. Ruby uses ffi to talk to the locally installed zmq libs. It's not like, say, pyzmq where it actually links against it.
I've not tested against openjdk, only sun.
As for the file input, I've seen this before when you specify multiple inputs or outputs of the same type in the past. That's totally a valid usecase, mind you but I have seen it. I think it is a small race condition in registration of plugins.
Note that the zeromq support that came out with 1.1 is VERY limited and experimental. In fact I've rewritten all the zeromq stuff. It currently only exists in master right now but you can build a jar easy enough - just checkout the source and type make. Alternately I can upload a build somewhere for you to use. We're currently rolling out the master version at my company because we want the zeromq support.
If you want to read up on the changes, see here:
Exact same problem with :
a fresh master build
a 2.2 zeromq standard installation
a Sun JVM
After investigation, it looks like the stacktrace is "informative" since everything seems to work.
However I had to listen on *:2120 and not on <real-hostname>:2120.
Logstash won't start with the stacktrace attached in zmq-logstash.txt
Regarding the stacktrace indicating "ZeroMQ Error while binding", it is a ZeroMQ issue.
ZeroMQ cannot bind on DNS with tcp protocol.
For information, see http://stackoverflow.com/questions/6024003/why-doesnt-zeromq-work-on-localhost
All the ClassNotFoundException stacktraces on verbose mode are still disturbing.
Indeed. Here's what happens -
When you run logstash agent with '-vvv' (3 -v flags)
logstash sets $DEBUG = true in ruby
In ruby, when $DEBUG is true, you get all exceptions of all kinds, caught or otherwise, emitted to stderr.
This confuses things
To resolve this confusion, I have disabled '-vvv' (3 -v flags) in the next release of logstash. Setting $DEBUG was only useful in insane edge cases and only caused confusion to logtash users
Fixed by making '-vvv' no longer set $DEBUG.