I’m assuming you are using the Flume Persistent Appender which uses Berkeley DB 
based on the logs below.  From the log it appears that Berkeley DB’s file 
handle is stale and meets to be closed and reopened.  I haven’t checked but 
Berkeley DB might have a setting for this, otherwise the Flume Persistent 
Manager would need to deal with this condition. That would be a big change as 
the Database object is currently immutable.

Ralph

On Apr 4, 2014, at 12:04 PM, Arkin Yetis <[email protected]> wrote:

> We use the Flume Appender. Our logging stopped after a certain point in
> time and we noticed the exception at the end of this message in our
> application logs. It looks like there was an issue with the filesystem. But
> although the filesystem has recovered, the appender (or probably the
> persistence mechanism it uses) was stuck in this state and it took an
> application restart for it to continue logging. It does not look like there
> is a recovery mechanism or if there is one it failed.
> Would you like me to open a log4j JIRA ticket for this? Or is this
> something that can be prevented by something simple you can share over
> e-mail such as a certain configuration setting?
> 
> Thanks,
> - Arkin
> 
> Exception stack is:
> 1. Stale NFS file handle (java.io.IOException)
>  java.io.RandomAccessFile:-2 (null)
> 2. Environment invalid because of previous exception: (JE 5.0.73)
> /app/logs/abs-workflow/flumeDir java.io.IOException: Stale NFS file handle
> LOG_READ: IOException on read, log is likely invalid. Environment is
> invalid and must be closed. fetchTarget of 0x542/0x4af13c parent IN=5 IN
> class=com.sleepycat.je.tree.BIN lastFullVersion=0x543/0x62d6c5
> lastLoggedVersion=0x543/0x62d6c5 parent.getDirty()=true state=0
> (com.sleepycat.je.EnvironmentFailureException)
>  com.sleepycat.je.log.FileManager:1883 (null)
> 
> ********************************************************************************
> Root Exception stack trace:java.io.IOException: Stale NFS file handle
>    at java.io.RandomAccessFile.readBytes(Native Method)
>    at java.io.RandomAccessFile.read(RandomAccessFile.java:338)
>    at
> com.sleepycat.je.log.FileManager.readFromFileInternal(FileManager.java:1918)
>    at com.sleepycat.je.log.FileManager.readFromFile(FileManager.java:1869)
>    at com.sleepycat.je.log.FileManager.readFromFile(FileManager.java:1807)
>    at com.sleepycat.je.log.FileSource.getBytes(FileSource.java:56)
>    at
> com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:919)
>    at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:848)
>    at
> com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:809)
>    at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1412)
>    at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1251)
>    at com.sleepycat.je.dbi.CursorImpl.fetchCurrent(CursorImpl.java:2261)
>    at
> com.sleepycat.je.dbi.CursorImpl.getCurrentAlreadyLatched(CursorImpl.java:1466)
>    at com.sleepycat.je.dbi.CursorImpl.getNext(CursorImpl.java:1593)
>    at
> com.sleepycat.je.cleaner.UtilizationProfile.getObsoleteDetail(UtilizationProfile.java:632)
>    at
> com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:439)
>    at
> com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:289)
>    at
> com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:148)
>    at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:163)
>    at java.lang.Thread.run(Thread.java:662)
> ********************************************************************************


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to