Logstash crashs and Google BigQuery feature upgrade

Description

Hi
1. Running Logstash i get the following error over and over again:
IOError: closed stream
flush at org/jruby/RubyIO.java:2199
size at org/jruby/RubyFile.java:1108
method_missing at /opt/logstash/lib/logstash/outputs/google_bigquery.rb:565
receive at /opt/logstash/lib/logstash/outputs/google_bigquery.rb:194
handle at /opt/logstash/lib/logstash/outputs/base.rb:86
worker_setup at /opt/logstash/lib/logstash/outputs/base.rb:78

I'm trying to understand what am i doing wrong
i've tested my configuration and everything seems more than fine
as sometimes it works and sometimes it just fails with unknown reason

2. Google BigQuery output plugin
I think its a good idea to add the threshold attribute for allowed errors within an bq upload job, you can see example in the bigquery portal

3. Unknown bug of file control within Google BigQuery plugin
within the plugin you have two keys you can set:
deleter_interval_secs - set the interval in sec for deleting the temp file from disk
uploader_interval_secs - set the interval in sec from data collection to uploading the data to bigquery

I've noted that despite that the file was uploaded and was done been written with data, once it's deleted using the deleter_interval_secs key, Logstash craches because it can't find the file

Environment

None

Status

Assignee

Logstash Developers

Reporter

Dave Ezrakhovich

Labels

None

Affects versions

Priority

Configure