Details

    • Type: Bug/Feature
    • Status: Resolved (View workflow)
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Hello,

      My configuration is as follows:
      1. logstash is gathering logs from RabbitMQ server;
      2. after that logstash puts logs in ElasticSearch directly

      Test case:
      1. when ElasticSearch dies and new logs appears in RabbitMQ, logstash takes logs from RabbitMQ and purges the queue
      2. then logstash tries to put logs in ElasticSearch (which is stopped) and tries to retry this operation every 1 minute
      3. if ElasticSearch server is started again everyting is ok, logs are sent to it from logstash
      4. but if I stop logstash service when ElasticSearch is stopped - I lose logs.

      I didn't find anything related to this issue with logstash-ElasticSearch direct configuration, so, maybe it is possible to implement such feature to not purge RabbitMQ queue until logstash pushed logs in ElasticSearch?

      Thanks in advance.

      P.S. I have the most recent versions of all software:
      RabbitMQ - 3.1.1
      logstash - logstash-1.1.13-flatjar.jar
      ElasticSearch - 0.90.1

      logstash configuration:

      input {
      rabbitmq

      { host => "localhost" queue => "indexer-queue" exchange => "amq.direct" key => "logstash-routing-key" exclusive => false durable => true auto_delete => false type => "logstash-indexer-input" }

      rabbitmq

      { host => "stg-app-2" queue => "indexer-queue" exchange => "amq.direct" key => "logstash-routing-key" exclusive => false durable => true auto_delete => false type => "logstash-indexer-input" }

      }

      output {
      stdout

      { debug => true debug_format => "json"}

      elasticsearch

      { host => "localhost" cluster => "logstash" }

      }

        Gliffy Diagrams

          Attachments

            Issue links

              Activity

                People

                • Assignee:
                  coolacid Jason Kendall
                  Reporter:
                  romanmelko Roman Melko
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: