New filter: MySQL query

Description

i think it would be awesome, if i could query a MySQL database (or any) against the content of a field from my Input to add the result of the query to the output.

In my case that would be helpful to match cookie IDs to User IDs from a webserverlog for better usertracking.

Activity

Show:
Philippe Weber
April 24, 2014, 9:21 AM

hello Daniel,
I will try to make a first version in the coming weeks, as I wanted to have a look on how to load jdbc driver into logstash in the context of LOGSTASH-1764.

To be sure to go in the right direction
can you please describe your use case a little more

Do you expect to replace a single value each time or a list of value?
Which kind of sql statement do you expect to use ? number of parameters?...

I was thinking of something like

What do you think?

Daniel Klockenkämper
April 24, 2014, 10:32 AM
Edited

Hey Philippe,

first of all its very nice to get feedback in such a short span of time. Thanks!

I will get a Webserverlog - every message will contain a request with one cookie-id. The cookie-id will be also saved by our application in the mysql database and will have a relation to the customer-id. Every time the customer logs in, he will get a new random cookie-id - to track the user easily in a frontend on elasticsearch, i'd like to map all cookie-ids to customer-ids when parsing data in logstash.

So what we get is one cookie-id per request that will only match one customer id (the relation in the database is n to 1, filter needs only match 1 to 1)

I think your Model of the filter looks good as far as that matches the usecase i described - in my case it is a simple select that will replace a single value in every message. The "connection_string" you mention above will be the value for "DriverManager.getConnection("jdbc:mysql://"connection_string)"?

Maybe other persons will have usecases where the relation is 1 to n and the output has to be an array .

João Duarte
February 6, 2015, 3:42 PM

Assignee

Philippe Weber

Reporter

Daniel Klockenkämper

Labels

Fix versions

Configure