Uploaded image for project: 'logstash'
  1. LOGSTASH-2100

date filter adding tag even if match fails and @timestamp not updated

    Details

      Description

      Logstash is adding the dated tag so, according to the docs, the date filter succeeded. Yet, @timestamp is still ms-resolution and clearly has not been updated from timestamp when we see logstash catch up on old log items.

      Config in client:

           grok {
             match => [ 'message', '%{SYSLOGBASE}\s*(?<syslogmsg>.*)' ]
             add_tag => [ 'syslog', 'grokked' ]
           }
           date {
             match => [ 'timestamp', 'MMM dd HH:mm:ss' ]
             add_tag => [ 'dated' ]
           }
      

      Example record sent to output:

      {
        "_index": "logstash-2014.04.08",
        "_type": "syslog",
        "_score": null,
        "_source": {
          "message": "Apr  7 21:32:23 HOSTNAME sshd[29113]: Connection closed by 1.1.1.1 [preauth]",
          "@version": "1",
          "@timestamp": "2014-04-08T01:32:24.385Z",
          "type": "syslog",
          "host": "HOSTNAME",
          "path": "/var/log/auth.log",
          "timestamp": "Apr  7 21:32:23",
          "logsource": "HOSTNAME",
          "program": "sshd",
          "pid": "29113",
          "syslogmsg": "Connection closed by 1.1.1.1 [preauth]",
          "tags": [
            "syslog",
            "grokked",
            "dated",
            "ingest"
          ]
        },
        "sort": [
          "sshd",
          1396920744385
        ]
      }
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                wiibaa Philippe Weber
                Reporter:
                garthk Garth Kidd
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: