Elasticsearch input, elasticsearch close connection due to scroll_id too long

Description

Hello,
I'm trying to output to S3 6 month of data coming from an elasticsearch.
I have a backlog of 1 week for an index named logstash-* and a backlog of 6 month for an other index.

If I configure my elasticearch input for logstash-* it works but for another-* it doesn't.

We can see errors on elasticsearch with this info :
[2013-10-09 17:43:12,075][WARN ][http.netty ] [i-338c3651] Caught exception while handling client http traffic, closing connection [id: 0x9c6588c8, /10.45.10.177:40916 => /10.45.10.92:9200]
org.elasticsearch.common.netty.handler.codec.frame.TooLongFrameException: An HTTP line is larger than 4096 bytes.

If I do a tcpdump I can see the scroll_id in the request is huge which could explain the 4096 bytes limit.

Is there anyway to post the scroll_id rather than to set it in the URL.
https://github.com/logstash/logstash/blob/master/lib/logstash/inputs/elasticsearch.rb#L122

My problem seems similar to that one :
https://groups.google.com/forum/#!msg/elasticsearch/jXKLdCrrIHM/gG_y--hOzlcJ

What do you think?

I guess it can be a big problem for people having a lot of data to migrate from Logstash 1.1 to 1.2 as the logic will be the same (use elasticsearch input)

Status

Assignee

Logstash Developers

Reporter

jujugrrr

Labels

Affects versions

Configure