ElasticSearch and Logstash Integration Error

Description

According to the installation guide, I have installed the version 0.17.6 of ElasticSearch and after trying to run Logstash, the following error occurred:

~/Applications/logstash_conf$ logstash -f agent-server.conf
NameError: cannot load Java class org.elasticsearch.action.ActionListener
get_proxy_or_package_under_package at org/jruby/javasupport/JavaUtilities.java:54
method_missing at /home/jon1984/.rvm/rubies/jruby-1.6.5/lib/ruby/site_ruby/shared/builtin/javasupport/java.rb:51
ActionListener at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch/actionlistener.rb:5
(root) at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch/actionlistener.rb:4
require at org/jruby/RubyKernel.java:1038
require at /home/jon1984/.rvm/rubies/jruby-1.6.5/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36
(root) at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch/actionlistener.rb:2
require at org/jruby/RubyKernel.java:1038
require at /home/jon1984/.rvm/rubies/jruby-1.6.5/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36
(root) at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch/request.rb:2
require at org/jruby/RubyKernel.java:1038
require at /home/jon1984/.rvm/rubies/jruby-1.6.5/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36
(root) at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch/indexrequest.rb:3
require at org/jruby/RubyKernel.java:1038
require at /home/jon1984/.rvm/rubies/jruby-1.6.5/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36
(root) at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch/client.rb:1
require at org/jruby/RubyKernel.java:1038
require at /home/jon1984/.rvm/rubies/jruby-1.6.5/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36
register at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/jruby-elasticsearch-0.0.12/lib/jruby-elasticsearch.rb:81
run_output at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/logstash-1.0.17/lib/logstash/agent.rb:522
run_with_config at /home/jon1984/.rvm/gems/jruby-1.6.5/gems/logstash-1.0.17/lib/logstash/agent.rb:370
call at org/jruby/RubyProc.java:270
call at org/jruby/RubyProc.java:224

This is the current agent-server.conf file:

input {
amqp {

  1. ship logs to the 'rawlogs' fanout queue.
    type => "all"
    host => "172.17.112.98"
    exchange_type => "fanout"
    name => "rawlogs"
    }
    }

filter {
grok {
type => "syslog" # for logs of type "syslog"
pattern => "%{SYSLOGLINE}"

  1. You can specify multiple 'pattern' lines
    }

grok {
type => "apache-access" # for logs of type 'apache-access'
pattern => "%{COMBINEDAPACHELOG}"
}

date {
type => "syslog"

  1. The 'timestamp' and 'timestamp8601' names are for fields in the

  2. logstash event. The 'SYSLOGLINE' grok pattern above includes a field

  3. named 'timestamp' that is set to the normal syslog timestamp if it

  4. exists in the event.
    timestamp => "MMM d HH:mm:ss" # syslog 'day' value can be space-leading
    timestamp => "MMM dd HH:mm:ss"
    timestamp8601 => ISO8601 # Some syslogs use ISO8601 time format
    }

date {
type => "apache-access"
timestamp => "dd/MMM/yyyy:HH:mm:ss Z"
}
}

output {
stdout { }

  1. If your elasticsearch server is discoverable with multicast, use this:

  2. elasticsearch { }

  1. If you can't discover using multicast, set the address explicitly
    elasticsearch {
    host => "172.17.112.98"
    port => "9301"
    }
    }

I have also installed the river rabbitmq plugin for Elastic Search.

Logstash centralised setting works well without Elastic Search, as I can see the log output from the other servers. However I want to integrate Elastic Search with it, so any idea what has gone wrong?

Many thanks!

Regards,
Jonathan

Gliffy Diagrams

Activity

Show:

Jonathan Moo December 7, 2011 at 3:28 AM

Yes the elasticsearch.jar finally compiled. Thanks!

Jordan Sissel December 6, 2011 at 10:25 AM

Sounds like logstash can't find your elasticsearch jar.

Best recommendation is to use the logstash release jar files - they contain pretty much all the dependencies and are easy to run with simply 'java -jar logstash.jar ...'

Is this OK?

Cannot Reproduce

Details

Assignee

Reporter

Affects versions

Created December 6, 2011 at 10:21 AM
Updated April 19, 2013 at 8:19 PM
Resolved January 5, 2012 at 6:17 AM