Logstash is adding the dated tag so, according to the docs, the date filter succeeded. Yet, @timestamp is still ms-resolution and clearly has not been updated from timestamp when we see logstash catch up on old log items.
Config in client:
1
2
3
4
5
6
7
8
grok {
match => [ 'message', '%{SYSLOGBASE}\s*(?<syslogmsg>.*)' ]
add_tag => [ 'syslog', 'grokked' ]
}
date {
match => [ 'timestamp', 'MMM dd HH:mm:ss' ]
add_tag => [ 'dated' ]
}Example record sent to output:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
{
"_index": "logstash-2014.04.08",
"_type": "syslog",
"_score": null,
"_source": {
"message": "Apr 7 21:32:23 HOSTNAME sshd[29113]: Connection closed by 1.1.1.1 [preauth]",
"@version": "1",
"@timestamp": "2014-04-08T01:32:24.385Z",
"type": "syslog",
"host": "HOSTNAME",
"path": "/var/log/auth.log",
"timestamp": "Apr 7 21:32:23",
"logsource": "HOSTNAME",
"program": "sshd",
"pid": "29113",
"syslogmsg": "Connection closed by 1.1.1.1 [preauth]",
"tags": [
"syslog",
"grokked",
"dated",
"ingest"
]
},
"sort": [
"sshd",
1396920744385
]
}