That is probably a good idea anyway. You will get much better performance from local disk. The only downside is if your local disk is small.
Ralph On Apr 5, 2014, at 5:52 PM, Arkin Yetis <[email protected]> wrote: > Yes, I was referring to the Flume Persistent Appender. I will open an > enhancement request in JIRA.In the meanwhile, we will consider using local > disk instead of a remote filesystem to decrease the likelihood of the issue > occurring. > > Thanks, > - Arkin > > > On Fri, Apr 4, 2014 at 2:46 PM, Ralph Goers <[email protected]>wrote: > >> I'm assuming you are using the Flume Persistent Appender which uses >> Berkeley DB based on the logs below. From the log it appears that Berkeley >> DB's file handle is stale and meets to be closed and reopened. I haven't >> checked but Berkeley DB might have a setting for this, otherwise the Flume >> Persistent Manager would need to deal with this condition. That would be a >> big change as the Database object is currently immutable. >> >> Ralph >> >> On Apr 4, 2014, at 12:04 PM, Arkin Yetis <[email protected]> wrote: >> >>> We use the Flume Appender. Our logging stopped after a certain point in >>> time and we noticed the exception at the end of this message in our >>> application logs. It looks like there was an issue with the filesystem. >> But >>> although the filesystem has recovered, the appender (or probably the >>> persistence mechanism it uses) was stuck in this state and it took an >>> application restart for it to continue logging. It does not look like >> there >>> is a recovery mechanism or if there is one it failed. >>> Would you like me to open a log4j JIRA ticket for this? Or is this >>> something that can be prevented by something simple you can share over >>> e-mail such as a certain configuration setting? >>> >>> Thanks, >>> - Arkin >>> >>> Exception stack is: >>> 1. Stale NFS file handle (java.io.IOException) >>> java.io.RandomAccessFile:-2 (null) >>> 2. Environment invalid because of previous exception: (JE 5.0.73) >>> /app/logs/abs-workflow/flumeDir java.io.IOException: Stale NFS file >> handle >>> LOG_READ: IOException on read, log is likely invalid. Environment is >>> invalid and must be closed. fetchTarget of 0x542/0x4af13c parent IN=5 IN >>> class=com.sleepycat.je.tree.BIN lastFullVersion=0x543/0x62d6c5 >>> lastLoggedVersion=0x543/0x62d6c5 parent.getDirty()=true state=0 >>> (com.sleepycat.je.EnvironmentFailureException) >>> com.sleepycat.je.log.FileManager:1883 (null) >>> >>> >> ******************************************************************************** >>> Root Exception stack trace:java.io.IOException: Stale NFS file handle >>> at java.io.RandomAccessFile.readBytes(Native Method) >>> at java.io.RandomAccessFile.read(RandomAccessFile.java:338) >>> at >>> >> com.sleepycat.je.log.FileManager.readFromFileInternal(FileManager.java:1918) >>> at >> com.sleepycat.je.log.FileManager.readFromFile(FileManager.java:1869) >>> at >> com.sleepycat.je.log.FileManager.readFromFile(FileManager.java:1807) >>> at com.sleepycat.je.log.FileSource.getBytes(FileSource.java:56) >>> at >>> >> com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:919) >>> at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:848) >>> at >>> >> com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:809) >>> at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1412) >>> at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1251) >>> at com.sleepycat.je.dbi.CursorImpl.fetchCurrent(CursorImpl.java:2261) >>> at >>> >> com.sleepycat.je.dbi.CursorImpl.getCurrentAlreadyLatched(CursorImpl.java:1466) >>> at com.sleepycat.je.dbi.CursorImpl.getNext(CursorImpl.java:1593) >>> at >>> >> com.sleepycat.je.cleaner.UtilizationProfile.getObsoleteDetail(UtilizationProfile.java:632) >>> at >>> >> com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:439) >>> at >>> com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:289) >>> at >>> com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:148) >>> at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:163) >>> at java.lang.Thread.run(Thread.java:662) >>> >> ******************************************************************************** >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
