I've checked the new version, today. Many thanks for that excellent work. It fixes a lot.
By checking the new mappings for GELF data as an input, I found an issue with the field names. GELF differentiate between standard (manadatory) fileds an so called user fields. Die difference is quite simple to see in my example below:
"short_message": "Tue Jul 23 21:16:05.112 [initandlisten] ...",
"full_message": "Tue Jul 23 21:16:05.112 [initandlisten] ...",
"_user_1": "my value 1",
"_user_2": "my value 2",
All user fields start with an underscore.
The new mapping introduced with version 1.2.0 remaps all GELF field >>without<< an underscore for user fields. It would be very helpful if the naming convention for GELF is used for logstash as well.
Many thanks for help,
found the discussion adressing the new logstash event schema with https://logstash.jira.com/browse/LOGSTASH-675. Unsure, if it is possible the change the schema again.
The issue I've with the new schema is that i've to rewite all my programs for that mapping used for Graylog2 in the past. Anyway, I'll implement an abstraction layer for my programs to remap the schema Graylog2/Logstash in ES and vice versa. It should not that big deal to do this.
From my side, you can close that ticket or leave it open for further discussion.
Im afraid the schema won't be changed.
Is there anything we can do to make it easier?
sorry for delayed answer. I was on a longer business trip.
Thanks for your answer. I found a way to indicate what mappings are related to the GELF protocols resp. GELF as a source. Logstash offers with the gelf input to add additional mappings. This way it is easy to "mark" the documents/messages with that additional mapping as send via GELF:
Optional/hash. Add any number of key/value pairs that will result in
additional mappings with the ES database for all messages. This may
help with processing/filtering events later eg. by the web interface
or using Query DSL request with ES
add_field => [ "input", "gelf" ]
That means independently from the resulting mappings in ES by Logstash, adding an additional field clearly identifies the source.
Thanks a lot,
Reporter solved is issues with event schema transition