Re: FileLockExclusiveReadLockStrategy doesn't hold lock and error on commit
Hi Yeah the FileLock actually becomes closed when you close its channel. So I have logged a ticket https://issues.apache.org/jira/browse/CAMEL-8353 On Thu, Feb 12, 2015 at 6:56 PM, wrusch wrote: > The built in FileLockExclusiveReadLockStrategy for camel errors on every > message commit: > > WARN - MarkerIgnoringBase.warn(136) | Error during commit. > Exchange[test2.txt]. Caused by: [java.nio.channels.ClosedChannelException - > null] > java.nio.channels.ClosedChannelException > at sun.nio.ch.FileLockImpl.release(FileLockImpl.java:58) > at > org.apache.camel.component.file.strategy.FileLockExclusiveReadLockStrategy.releaseExclusiveReadLock(FileLockStrategy.java:137) > at > org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.commit(GenericFileProcessStrategySupport.java:75) > at > org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:92) > at > org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:124) > at > org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:80) > at > org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:54) > at > org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:104) > at > org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:229) > at > org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:65) > at > org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:642) > at > org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:610) > at > org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:240) > at > org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:251) > at > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:447) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > at > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:433) > at > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) > at > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) > at > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187) > at > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > I spent a fair amount of time looking at this and basically the lock is > getting created and released immediately at lines 120,121 where the lock is > aquired. > > This means that the release is not only pointless it generates errors > because the channel is closed already and the lock has been released. > > I thought that an easy fix would be to simply not close the channel in the > finally clause of acquireExclusiveReadLock method, so that it could be > closed properly in the releaseExclusiveReadLock method. However when I try > and do this it breaks the rename strategy because for some reason the rename > strategy thinks it can rename a file that is locked and tries to release the > lock after trying to rename the file. > > Error: > WARN - MarkerIgnoringBase.warn(136) | Error during commit. > Exchange[test2.txt]. Caused by: > [org.apache.camel.component.file.GenericFileOperationFailedException - Error > renaming file from > C:\source\workspace\3.3-BoiseU3\integrations\camel\test-output\filedrop\test2.txt > to test-output\filedrop\.done\test2.txt] > org.apache.camel.component.file.GenericFileOperationFailedException: Error > renaming file from > C:\source\workspace\3.3-BoiseU3\integrations\camel\test-output\filedrop\test2.txt > to test-output\filedrop\.done\test2.txt > at > org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java
Re: FileLockExclusiveReadLockStrategy doesn't hold lock and error on commit
I don't think this is a platform issue. It could be that the filelock strategy as a whole is simply not compatible with the rename file strategy at all and what is currently there is just a hack pretending to do it correctly. The error is happening because the lock is being released immediately and not during commit. During the commit it tries to release it again so you get the error. Nothing OS specific about that unless you are saying that on linux they just swallow the errors or flat out don't honor the locks. The problem here is I can't fix it on my end because the component I need access to(the rename strategy) is not a pluggable component. The main point is if you need to release a lock before you can rename a file, the rename strategy needs to do that in the correct order. Instead of rename file -> then release lock release lock -> then rename file I cannot fix this. Which means I cannot fix the strategy as a whole. The only thing I can do is hide the original problem without fixing it. The original problem being that the file lock strategy doesn't hold the lock till commit, it releases it immediately. What OS specific issue do you see here? -- View this message in context: http://camel.465427.n5.nabble.com/FileLockExclusiveReadLockStrategy-doesn-t-hold-lock-and-error-on-commit-tp5762668p5762706.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: FileLockExclusiveReadLockStrategy doesn't hold lock and error on commit
Hi Windows file system works different than unix based and therefore you can hit issue like that, that otherwise works on unix systems. Its a bit wonky having to add logic to cater for this on windows. The other read locks works better on windows. And using readLock=fileLock is not guaranteed to work globally, meaning you can have another JVM that can also grab a fileLock on the same file. Using marker files / changed is better. On Thu, Feb 12, 2015 at 6:56 PM, wrusch wrote: > The built in FileLockExclusiveReadLockStrategy for camel errors on every > message commit: > > WARN - MarkerIgnoringBase.warn(136) | Error during commit. > Exchange[test2.txt]. Caused by: [java.nio.channels.ClosedChannelException - > null] > java.nio.channels.ClosedChannelException > at sun.nio.ch.FileLockImpl.release(FileLockImpl.java:58) > at > org.apache.camel.component.file.strategy.FileLockExclusiveReadLockStrategy.releaseExclusiveReadLock(FileLockStrategy.java:137) > at > org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.commit(GenericFileProcessStrategySupport.java:75) > at > org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:92) > at > org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:124) > at > org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:80) > at > org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:54) > at > org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:104) > at > org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:229) > at > org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:65) > at > org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:642) > at > org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:610) > at > org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:240) > at > org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:251) > at > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:447) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > at > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:433) > at > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) > at > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) > at > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187) > at > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > I spent a fair amount of time looking at this and basically the lock is > getting created and released immediately at lines 120,121 where the lock is > aquired. > > This means that the release is not only pointless it generates errors > because the channel is closed already and the lock has been released. > > I thought that an easy fix would be to simply not close the channel in the > finally clause of acquireExclusiveReadLock method, so that it could be > closed properly in the releaseExclusiveReadLock method. However when I try > and do this it breaks the rename strategy because for some reason the rename > strategy thinks it can rename a file that is locked and tries to release the > lock after trying to rename the file. > > Error: > WARN - MarkerIgnoringBase.warn(136) | Error during commit. > Exchange[test2.txt]. Caused by: > [org.apache.camel.component.file.GenericFileOperationFailedException - Error > renaming file from > C:\source\workspace\3.3-BoiseU3\integrations\camel\test-output\filedrop\test2.txt > to test-output\filedrop\.done\test2.txt] > org.apache.camel.component
FileLockExclusiveReadLockStrategy doesn't hold lock and error on commit
The built in FileLockExclusiveReadLockStrategy for camel errors on every message commit: WARN - MarkerIgnoringBase.warn(136) | Error during commit. Exchange[test2.txt]. Caused by: [java.nio.channels.ClosedChannelException - null] java.nio.channels.ClosedChannelException at sun.nio.ch.FileLockImpl.release(FileLockImpl.java:58) at org.apache.camel.component.file.strategy.FileLockExclusiveReadLockStrategy.releaseExclusiveReadLock(FileLockStrategy.java:137) at org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.commit(GenericFileProcessStrategySupport.java:75) at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:92) at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:124) at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:80) at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:54) at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:104) at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:229) at org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:65) at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:642) at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:610) at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:240) at org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:251) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:447) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:433) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187) at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) I spent a fair amount of time looking at this and basically the lock is getting created and released immediately at lines 120,121 where the lock is aquired. This means that the release is not only pointless it generates errors because the channel is closed already and the lock has been released. I thought that an easy fix would be to simply not close the channel in the finally clause of acquireExclusiveReadLock method, so that it could be closed properly in the releaseExclusiveReadLock method. However when I try and do this it breaks the rename strategy because for some reason the rename strategy thinks it can rename a file that is locked and tries to release the lock after trying to rename the file. Error: WARN - MarkerIgnoringBase.warn(136) | Error during commit. Exchange[test2.txt]. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Error renaming file from C:\source\workspace\3.3-BoiseU3\integrations\camel\test-output\filedrop\test2.txt to test-output\filedrop\.done\test2.txt] org.apache.camel.component.file.GenericFileOperationFailedException: Error renaming file from C:\source\workspace\3.3-BoiseU3\integrations\camel\test-output\filedrop\test2.txt to test-output\filedrop\.done\test2.txt at org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:77) at org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:113) at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:88) at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(Gener
