Events between logstash instances using lumberjack lose grok'd fields
Description
Gliffy Diagrams
Activity
Philippe Weber August 1, 2014 at 7:46 AM
A little more aggressive garbage-collect on jira.com, expecting follow-up activities to take place on github issues
Jason Kendall March 5, 2014 at 3:45 PM
Is this still an issue with 1.3.3?
Boyd Meier July 11, 2013 at 6:27 PM
I've been experiencing the same issue. I've tracked it down to an issue with the field encoding in the protocol. The encoding handles hashes and strings, but not arrays. Added fields are added as arrays, and so they disappear from the encoded event. As a temporary fix, I added logic which handles arrays as key value instances with repeated keys and the value treated as a string. This works for my specific use case, but is not necessarily a generic solution.
The commit is at https://github.com/bwmeier/logstash/commit/d4840900d33b72c3a1baa47d74d020d798c5b14d
Paul Tinsley March 10, 2013 at 1:05 PM
That was my perspective as well reading the code...
Just to make sure it was not something else causing the issue I just replaced lumberjack on both sides with:
All the fields come across just fine.
Jordan Sissel March 10, 2013 at 4:46 AM
Strangely, the lumberjack output should be sending things properly.
THe above is the code from the lumberjack output in logstash.
Logstash config on remote host:
Sample grok debug from remote host:
logstash config on central host:
Indexed event from central box: