filter chaining question

Thank you very much Philippe, your input has been very valuable.

I will document my experience and hopefully others will be able to learn from my confusion.

All is working now, I implemented your recommended changes.

It clicked in my head this morning that event["something"] is completely different than event[something] <--- without the surrounding double quotes.

I also got the params working for rest-client. I think my first failed attempt as shown at the beginning of this ticket was due to my lack of understanding how event[something] is different than event["something"].

Thanks again!

Yes you're right for the resource creation, but you should be able to put it as a field variable and declare it once in the register method
Have a (maybe second?) look to rest_client readme, https://github.com/rest-client/rest-client
you should be able to pass your param only when calling the get method

Should look more like this:

I got it working through a lot of trial and error. But it's still very confusing on how logstash handles an event when passed through the filter chain.

The documentation is not very clear; it's very high-level. Maybe I was looking for more details than needed.

Here is what I ended up doing: I don't like the solution, and will have to refactor.

logstash.conf

crowd.rb

I am not happy with this because I am under the impression I am supposed to create the rest-client object in the def register than update the params in the def filter as it's needed.

I wasn't able to get the rest-client params to work with GET, the Atlassian REST service uses ? parameters instead of /user/username/value type.

So when logstash is first started it loads the crowd filter into memory, but creates a new rest-client object every time it's called. This is my impression.... Is this what is happening?

Thanks for your help Philippe!

As told, just think of the event as an associative array.
Let's take 2 case

1. You decide that your filter will only lookup the field username, so you "hardcode"

2. You need the field name to be a parameter of the filter, so you do

and you invoke two instance of your filter

3 Or you change the filter to accept an array of value to convert now that you understand the flow

Thanks Philippe, I spent an hour Friday after your comment, and again this morning.

I still seem to be misunderstanding the flow of data from logstash input -> filter (grok -> custom filter)

event before grok filter

event after grok filter

So at this point grok has successfully parsed the event

logstash conf

In my custom filter I have event.inspect and can see the data I want is in the event.

event.inspect output

Above the field user1 or user2 would provide the info.

In my custom filter how do I reference these fields?

crowd.rb filter