TCP input events are expensive

Description

We are writing own small and customized log shippers in various languages.
The problem is that we have to open a new TCP socket, write some log data and close it afterwards in order to ship a new event to Logstash every time!

This is a highly expensive task, both on client and server side.
It would be cool if there is a configurable option for specifying a special character for separating log events and keep the socket open during the logging session.

//edit
I was not able to find any documentation, specifying the necessary requirements for sending an event via TCP which led to annoying try and error...

Gliffy Diagrams

Activity

Show:

Aaron Mildenstein February 6, 2015 at 9:39 PM

Since it's been 11 months with no response I am closing this with a status of incomplete.

With the new plugin nature of Logstash 1.5, if you have a custom implementation you need, you can write it yourself and keep it internal, or publish and share—your choice completely. Feel free to open a new issue at https://github.com/elasticsearch/logstash if you have any other issues.

Former user March 5, 2014 at 4:01 PM

OK. I'll take it on. I'm slow to push things back, as I wanted to stress test it on our 80 node hadoop cluster first.
But I was preparing to get the pull request out this week to clear my org ticket for this.