mutate filter should keep fields the same type

Description

Right now, you can mutate @tags and replace it with a string. That's not awesome.

I was going to say "Always keep @tags an array" but really it's better to say "Always keep arrays as arrays" - so a replace on an array should make it a single-element array... etc

Activity

Show:
Jordi Casas
February 8, 2013, 7:43 AM

Hi,

Have you any prediction to work about this issue?

I tested that in "alter" plugin that works fine. An array field after alter it's still an array.

Thanks,

Jordi.

Philippe Weber
October 11, 2013, 12:43 PM
Edited

As mentioned in add_field should default to single and promote if there is an array

Also gsub in the mutate filter should to the same, current returns an single element array.

Jason Kendall
February 21, 2014, 3:24 PM
Edited

With working with ernie1 on IRC it appears that tags do not convert to array if they are provided as a string.

Example: nxlog was used to add a tag to the event, but provided it as a string. When using add_tag it would concat the old tag and the new tag since it was not already an array.

Problem exists in the base filter - https://github.com/elasticsearch/logstash/blob/master/lib/logstash/filters/base.rb#L180

At no point does it try and convert the nxlog string to an array to add new array elements. Since this is in the base, it'll effect all filters.

Björn Ingeson
February 26, 2014, 7:25 AM
Edited

Example of Jasons comment.

Nxlog send a json via tcp to logstash.

The tags field will be treated as a string in logstash and add_tag will concatenate additional tags, example:
"testtaggeoed1geoed2"

Manually changing the json to:

Will make the add_tag add elements to the array correctly, example:
["testtag","geoed1","geoed2"]

Assignee

Logstash Developers

Reporter

Jordan Sissel

Labels

None

Affects versions

Configure