Logstash filtering problem
Description
Gliffy Diagrams
Activity
Philippe Weber September 26, 2013 at 12:21 PM
Closing old incomplete discussion
Philippe Weber May 23, 2013 at 4:26 AM
What I'm asking for is a real sample data not another test case.
Do you intend to use others logstash filters or only use it for pushing whole file content to elasticsearch?
Plus the multiline filter works by identifying either the first line or the last line of a multiline events, a common example is java logging that always start with a timestamp (2013-05-23 12:00:00) and may be followed by a stacktrace, then one possibility is to use this config. that means that any line that is not starting with "20" will be part of the previous event
rohit May 22, 2013 at 8:05 PM
okayy!!!!this is my simple input file....
"line1
line2
line3
line4
line5
line6" (quotes are for clarification)
now when i netcat (nc localhost 3333 < file_name.txt ) by using above logstash config it is taking as 6 different events (means 6 different files) instead of single event(single event)..but i want it as a single event(single file) to be indexed in eleastic search.so help me with that. i tried multiline adn it dint worked well.can u help me with that config??
Thanks...
Philippe Weber May 22, 2013 at 7:20 PM
With your example case I don't see clearly the added value of logstash for you against direct call to elasticsearch with your file
Either using the multiline filter, or simply using the exec input with the more yourfile bash command seems valid solutions.
Could you explain some more
Details
Details
Assignee
Philippe WeberReporter
rohit
Hi,
my config file looks like this
input{
tcp{
input => "demo"
port => 3333
}
output{
elasticsearch_http{
host => "localhost"
port => 9200
flush_size => 1 //using elastic search index api
}
}
so my input file looks like this
"demo1
demo2
demo3
demo4
"
when iam netcatting file through tcp file splits into 4 events(due to \n at the end of each line) and elastic search is indexing them as 4 separate events..
but i want whole file as only one event.can anybody help me in this issue??