File input prevents log4j file rolling on Windows

Description

I'm using the simple, single standalone server of logstash on Windows 7 x64. I have a basic Java program writing log files using the log4j rolling file appender and I am monitoring it with logstash's file input. Below is my logstash.conf and log4j.xml from my program

logstash.conf:
input {
file {
type => "my-log"
path => [ "C:/logs/mylog/*.log" ]
}
}

output {
stdout { }
elasticsearch { embedded => true }
}

log4j.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<log4j:configuration>
<appender name="FileAppender" class="org.apache.log4j.RollingFileAppender">
<param name="MaxBackupIndex" value="30"/>
<param name="MaxFileSize" value="1MB"/>
<param name="File" value="c:/logs/mylog/MyService.log"/>

<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%t] %c - %m%n"/>
</layout>
</appender>

<root>
<priority value="DEBUG" />
<appender-ref ref="FileAppender"/>
</root>
</log4j:configuration>

When logstash is monitoring the file it prevents the file from rolling. Once the file is full (1 MB in my example case for testing), the current .log file is truncated and it starts over. The other rolled files all roll as expected (the .1 file becomes .2, .2 becomes .3, etc.) leaving a hole where the .log file should have gone to .log.1. It is almost like logstash's open file handle is preventing the active .log file from being renamed.

If I stop logstash, the file will then roll just fine.

I have seen this before when another program other than the owning java process has an open file handle to the file. For example, if a developer is doing a tail on the .log file it will prevent it from rolling.

I did make a change to my logstash jar file per LOGSTASH-351. Since this modified the file handler reader, I'm wondering if that may have something to do with it. The problem is without making that change I get the error indicated in bug 351 and can't get logstash to pick up anything.

Assignee

Logstash Developers

Reporter

D

Labels

None

Fix versions

Affects versions

Configure