Hi, sorry didn't fix it, still the same error.
-----Original Message----- From: Claus Ibsen [mailto:[email protected]] Sent: Tuesday, September 02, 2014 4:07 PM To: [email protected] Subject: Re: camel 2.13.2 zipdataformat Hi Thanks. I suspect its because this code creates a new input stream in the message body https://github.com/apache/camel/blob/master/components/camel-zipfile/src/main/java/org/apache/camel/dataformat/zipfile/ZipIterator.java#L106 That is not explicit closed. Can you try changing answer.setBody(new ZipInputStreamWrapper(zipInputStream)); To answer.setBody(zipInputStream); And give it a test on your Windows system? On Tue, Sep 2, 2014 at 3:13 PM, Sven Nold <[email protected]> wrote: > Hi, > > same result, but workaround is to use zipSplitter like this: > > <bean id="zipSplitter" > class="org.apache.camel.dataformat.zipfile.ZipSplitter" /> > > <route id="readZip" streamCache="true"> > <from > uri="file://target?fileName=any.xxx&moveFailed=.error&move=.done&delay=5000"/> > <!-- <unmarshal ref="zipFileDataFormat"/> --> > <camel:split streaming="true"> > <!-- <camel:simple>${body}</camel:simple> --> > <camel:ref>zipSplitter</camel:ref> > <camel:to > uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/> > <camel:to uri="file://target/unzipped"/> > </camel:split> > <camel:to > uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/> > </route> > > ZipSplitter seems not to have this issue. Also works for 2.13.2. which could > be helpful for you @Thomas. > > So from my point of view CAMEL-7577 is not RESOLVED. > > Best regards, > > -----Ursprüngliche Nachricht----- > Von: Claus Ibsen [mailto:[email protected]] > Gesendet: Dienstag, 2. September 2014 14:29 > An: [email protected] > Betreff: Re: camel 2.13.2 zipdataformat > > You need to build the code to make sure the snapshot is using recent code. > > On Tue, Sep 2, 2014 at 2:22 PM, Sven Nold <[email protected]> wrote: >> Hi Claus, >> >> I changed to 2.13.3-SNAPSHOT but still the same issue. >> >> Best regards, >> >> Sven >> >> -----Ursprüngliche Nachricht----- >> Von: Claus Ibsen [mailto:[email protected]] >> Gesendet: Dienstag, 2. September 2014 13:52 >> An: [email protected] >> Betreff: Re: camel 2.13.2 zipdataformat >> >> Hi >> >> See CAMEL-7577 >> >> On Tue, Sep 2, 2014 at 1:49 PM, Sven Nold <[email protected]> wrote: >>> Hi, >>> >>> interesting is also that if I only have on zipEntry and define the >>> zipDataFormat as useIterator=false it works fine. So my guess is that >>> ZipIterator somehow holds a lock on the file: >>> >>> <bean id="zipFileDataFormat" >>> class="org.apache.camel.dataformat.zipfile.ZipFileDataFormat"> >>> <property name="usingIterator" value="false"/> >>> </bean> >>> >>> <bean id="zipAggregationStrategy" >>> class="org.apache.camel.processor.aggregate.zipfile.ZipAggregationSt >>> r >>> a >>> tegy"/> >>> >>> <camelContext xmlns="http://camel.apache.org/schema/spring";> >>> <!-- write the zip so we can process something repeatedly --> >>> <route id="writeZip"> >>> <from >>> uri="file://src/main/data?antInclude=message1.xml&noop=true"/> >>> <aggregate strategyRef="zipAggregationStrategy" >>> completionFromBatchConsumer="true" eagerCheckCompletion="true" >>> completionSize="2"> >>> <correlationExpression> >>> <constant>true</constant> >>> </correlationExpression> >>> <to uri="file://target?fileName=any.xxx"/> >>> </aggregate> >>> </route> >>> >>> <!-- extract the zip --> >>> <route id="readZip" streamCache="true"> >>> <from >>> uri="file://target?fileName=any.xxx&moveFailed=.error&move=.done&delay=5000"/> >>> <unmarshal ref="zipFileDataFormat"/> >>> <!--<camel:split streaming="true"> >>> <camel:simple>${body}</camel:simple> >>> <camel:to >>> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/> >>> <camel:to uri="file://target/unzipped"/> >>> </camel:split> --> >>> <camel:to >>> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/> >>> </route> >>> </camelContext> >>> >>> Works fine. >>> >>> Best regards, >>> >>> >>> >>> -----Ursprüngliche Nachricht----- >>> Von: Sven Nold [mailto:[email protected]] >>> Gesendet: Dienstag, 2. September 2014 13:38 >>> An: [email protected] >>> Betreff: AW: camel 2.13.2 zipdataformat >>> >>> Hi, >>> >>> thanks for this test now I think here is the problem: >>> >>> [l-1) thread #1 - file://target] GenericFileOnCompletion WARN Error >>> during commit. Exchange[any.zip]. Caused by: >>> [org.apache.camel.component.file.GenericFileOperationFailedException - >>> Error renaming file from >>> C:\temp\camel-examples-master\zip-file-processing\target\any.zip to >>> target\.done\any.zip] >>> org.apache.camel.component.file.GenericFileOperationFailedException: Error >>> renaming file from >>> C:\temp\camel-examples-master\zip-file-processing\target\any.zip to >>> target\.done\any.zip >>> at >>> org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:77) >>> at >>> org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:113) >>> at >>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:88) >>> at >>> org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:124) >>> at >>> org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:80) >>> at >>> org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:54) >>> at >>> org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:100) >>> at >>> org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:228) >>> at >>> org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:61) >>> at >>> org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:613) >>> at >>> org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:581) >>> at >>> org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:240) >>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:106) >>> at >>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:423) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211) >>> at >>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175) >>> at >>> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187) >>> at >>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: java.io.IOException: Renaming file from >>> 'C:\temp\camel-examples-master\zip-file-processing\target\any.zip' to >>> 'target\.done\any.zip' failed: Cannot delete file >>> 'C:\temp\camel-examples-master\zip-file-processing\target\any.zip' after >>> copy succeeded >>> at >>> org.apache.camel.util.FileUtil.renameFileUsingCopy(FileUtil.java:456) >>> at org.apache.camel.util.FileUtil.renameFile(FileUtil.java:428) >>> at >>> org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:74) >>> ... 25 more >>> >>> This is why it grabs the file all the time. >>> >>> Regarding your questions: >>> >>> 1) well thanks to your test you can see error above >>> 2) yes it is processing it over and over again >>> 3) no doesn't move to .error >>> 4) no file is not updated, in your test case I am not quite sure, will >>> increase delay or as you suggested preMove option. >>> >>> Due to known problems off windows and zip files I even renamed it to .xxx >>> to keep away explorer from analyzing, but still it seems there is a handle >>> on that file. >>> And again if I remove the unmarshal and split it works fine. >>> >>> Thanks for any help. >>> >>> Best regards, >>> >>> >>> -----Ursprüngliche Nachricht----- >>> Von: Jimmi Dyson [mailto:[email protected]] >>> Gesendet: Dienstag, 2. September 2014 10:37 >>> An: [email protected] >>> Betreff: Re: camel 2.13.2 zipdataformat >>> >>> Hi, >>> >>> I just added an example at >>> https://github.com/jimmidyson/camel-examples/tree/master/zip-file-processing. >>> This example creates the zip file first (so that it's repeatable) & then >>> does pretty much what you're doing: read from file uri, unmarshal, log each >>> file contents (careful with large files) & finally writes each file to a >>> file. Seems to be doing what it should... >>> >>> Few questions if the example doesn't help you: >>> >>> - Are there any hints in the logs (try increasing logging level if not)? >>> - If the file is back in the source dir does it try to process again? >>> - Is there anything in the .error (moveFailed dir)? >>> - Could the source file be being updated while you're processing it - if >>> so, can you try the preMove option? >>> >>> Thanks, >>> >>> >>> On 2 September 2014 08:46, Walzer, Thomas >>> <[email protected]> >>> wrote: >>> >>>> Hi Sven, >>>> >>>> I seem to have a similar problem. I also use 2.13.2. >>>> My code is almost identical. The only difference is my in-url, that >>>> looks >>>> like: >>>> >>>> file:////someserver/somedir/in?move=archive/${date:now:yyyyMMdd}/${ >>>> f i l e:name}.${date:now:HHmmss}&antInclude=*.ZIP >>>> >>>> I get a random number of retrievals (about 4-20) and therefore the >>>> same number of files in the „archive“ directory. The timestamps are >>>> 6 seconds apart. >>>> >>>> Cheers, Thomas. >>>> >>>> Am 02.09.2014 um 08:44 schrieb Sven Nold <[email protected]>: >>>> >>>> > Hi Guys, >>>> > >>>> > I am trying to read Zipentries and process them separately. But >>>> > using >>>> the file component it seems the file is not correctly closed. I can >>>> see the file for a few seconds in the .done and source directory. >>>> But then it disappears in the .done folder and is back in the source >>>> directory : >>>> > >>>> > >>>> > <bean id="zipFileDataFormat" >>>> class="org.apache.camel.dataformat.zipfile.ZipFileDataFormat"> >>>> > <property name="usingIterator" value="true" /> </bean> >>>> > >>>> > <route id="readZip"> >>>> > <from >>>> uri="file://C:/temp/?fileName=any.zip&moveFailed=.error&move=.done&delay=10000" >>>> /> >>>> > <unmarshal ref="zipFileDataFormat" /> >>>> > <!-- <split streaming="true"> --> >>>> > <!-- <simple>$simple{body}</simple> --> >>>> > <!-- <to >>>> uri="log://OUT?level=INFO&showAll=true&multiline=true" /> >>>> --> >>>> > <!-- </split> --> >>>> > <to uri="log://OUT?level=INFO&showAll=true&multiline=true" >>>> /> >>>> > </route> >>>> > >>>> > If I remove the unmarshal the file component works fine (it moves >>>> processed file to .done). >>>> > >>>> > Any Ideas? I guess I could do a workaround by using noop=true and >>>> > moving >>>> the file myself, but it would be nice if it works as expected. >>>> > >>>> > Best regards, >>>> > >>>> > Sven >>>> >>>> >> >> >> >> -- >> Claus Ibsen >> ----------------- >> Red Hat, Inc. >> Email: [email protected] >> Twitter: davsclaus >> Blog: http://davsclaus.com >> Author of Camel in Action: http://www.manning.com/ibsen >> hawtio: http://hawt.io/ >> fabric8: http://fabric8.io/ > > > > -- > Claus Ibsen > ----------------- > Red Hat, Inc. > Email: [email protected] > Twitter: davsclaus > Blog: http://davsclaus.com > Author of Camel in Action: http://www.manning.com/ibsen > hawtio: http://hawt.io/ > fabric8: http://fabric8.io/ -- Claus Ibsen ----------------- Red Hat, Inc. Email: [email protected] Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen hawtio: http://hawt.io/ fabric8: http://fabric8.io/
