Many accumulating open connections


My logstash process has crashed with the error 'too many open files'. Once restarted, it has slowly been accumulating more open files, which are actually tcp sockets that seem to not be closing. The client sending messages to logstash is rsyslog, on port 514 (normal syslog messages.) I have quite a few of these rsyslog clients, but only one of them seems to have these connections remaining open.

My logstash config file:

Stack trace from last crash:

lsof -p <logstash_pid> output:


Jordan Sissel
March 13, 2012, 12:49 AM

I'm pretty sure this is because closed client sockets are left dangling and un-garbagecollected so they are never closed.

Two things:

  • evaluate whether or not we care about tracking the list of all client sockets (@tcp_clients in inputs/syslog.rb)

  • close the socket on any read error.

Richard Pijnenburg
December 17, 2012, 5:06 AM

Syslog input plugin will be removed.

Andrei Chiriaev
October 30, 2014, 9:37 AM

Guys, Hi. Had the same issue with 1.4.2. Anyone else reporting similar issues? Regards, Andrei
{:timestamp=>"2014-10-29T21:58:06.532000+0000", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::Tcp type=>\"nxlog_json\", tags=>[\"input_nxlog_json\"], host=>\"\", mode=>\"server\">\n Error: Too many open files", :level=>:error}
{:timestamp=>"2014-10-29T21:58:10.704000+0000", :message=>"An error occurred. Closing connection", :client=>"", :exception=>#<IOError: Too many open files>, :backtrace=>["org/jruby/ `sysread'", "/opt/logstash/lib/logstash/inputs/tcp.rb:162:in `read'", "/opt/logstash/lib/logstash/inputs/tcp.rb:111:in `handle_socket'", "/opt/logstash/lib/logstash/inputs/tcp.rb:146:in `client_thread'", "/opt/logstash/lib/logstash/inputs/tcp.rb:144:in `client_thread'"], :level=>:error}

Aaron Mildenstein
February 6, 2015, 7:37 PM


Logstash Developers




Fix versions

Affects versions