Logstash does not appear to cleanup after itself well (specifically S3)

Description

Eg: s3 creates a temp directory and copies the file from S3 into that directory. Once it's done reading the local copy it then deletes it.

The problem is that if logstash shuts down while it's still reading the directory, it leaves the file in there. (In my case, each file was 2.5GB in size so after a few logstash restarts the FS got full)

Presumably other plugins that use temp storage have similar problems? Seems like logstash should register a shutdown hook and then force plugin developers to implement an "onExit" callback?

Environment

None

Status

Assignee

Logstash Developers

Reporter

AlexI

Labels

Affects versions

Priority

Configure