Hi,
same result, but workaround is to use zipSplitter like this:
<bean id="zipSplitter" class="org.apache.camel.dataformat.zipfile.ZipSplitter"
/>
<route id="readZip" streamCache="true">
<from
uri="file://target?fileName=any.xxx&moveFailed=.error&move=.done&delay=5000"/>
<!-- <unmarshal ref="zipFileDataFormat"/> -->
<camel:split streaming="true">
<!-- <camel:simple>${body}</camel:simple> -->
<camel:ref>zipSplitter</camel:ref>
<camel:to
uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
<camel:to uri="file://target/unzipped"/>
</camel:split>
<camel:to
uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
</route>
ZipSplitter seems not to have this issue. Also works for 2.13.2. which could be
helpful for you @Thomas.
So from my point of view CAMEL-7577 is not RESOLVED.
Best regards,
-----Ursprüngliche Nachricht-----
Von: Claus Ibsen [mailto:[email protected]]
Gesendet: Dienstag, 2. September 2014 14:29
An: [email protected]
Betreff: Re: camel 2.13.2 zipdataformat
You need to build the code to make sure the snapshot is using recent code.
On Tue, Sep 2, 2014 at 2:22 PM, Sven Nold <[email protected]> wrote:
> Hi Claus,
>
> I changed to 2.13.3-SNAPSHOT but still the same issue.
>
> Best regards,
>
> Sven
>
> -----Ursprüngliche Nachricht-----
> Von: Claus Ibsen [mailto:[email protected]]
> Gesendet: Dienstag, 2. September 2014 13:52
> An: [email protected]
> Betreff: Re: camel 2.13.2 zipdataformat
>
> Hi
>
> See CAMEL-7577
>
> On Tue, Sep 2, 2014 at 1:49 PM, Sven Nold <[email protected]> wrote:
>> Hi,
>>
>> interesting is also that if I only have on zipEntry and define the
>> zipDataFormat as useIterator=false it works fine. So my guess is that
>> ZipIterator somehow holds a lock on the file:
>>
>> <bean id="zipFileDataFormat"
>> class="org.apache.camel.dataformat.zipfile.ZipFileDataFormat">
>> <property name="usingIterator" value="false"/>
>> </bean>
>>
>> <bean id="zipAggregationStrategy"
>> class="org.apache.camel.processor.aggregate.zipfile.ZipAggregationStr
>> a
>> tegy"/>
>>
>> <camelContext xmlns="http://camel.apache.org/schema/spring";>
>> <!-- write the zip so we can process something repeatedly -->
>> <route id="writeZip">
>> <from
>> uri="file://src/main/data?antInclude=message1.xml&noop=true"/>
>> <aggregate strategyRef="zipAggregationStrategy"
>> completionFromBatchConsumer="true" eagerCheckCompletion="true"
>> completionSize="2">
>> <correlationExpression>
>> <constant>true</constant>
>> </correlationExpression>
>> <to uri="file://target?fileName=any.xxx"/>
>> </aggregate>
>> </route>
>>
>> <!-- extract the zip -->
>> <route id="readZip" streamCache="true">
>> <from
>> uri="file://target?fileName=any.xxx&moveFailed=.error&move=.done&delay=5000"/>
>> <unmarshal ref="zipFileDataFormat"/>
>> <!--<camel:split streaming="true">
>> <camel:simple>${body}</camel:simple>
>> <camel:to
>> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
>> <camel:to uri="file://target/unzipped"/>
>> </camel:split> -->
>> <camel:to
>> uri="log:org.apache.camel?level=INFO&showAll=true&multiline=true&showStreams=true"/>
>> </route>
>> </camelContext>
>>
>> Works fine.
>>
>> Best regards,
>>
>>
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Sven Nold [mailto:[email protected]]
>> Gesendet: Dienstag, 2. September 2014 13:38
>> An: [email protected]
>> Betreff: AW: camel 2.13.2 zipdataformat
>>
>> Hi,
>>
>> thanks for this test now I think here is the problem:
>>
>> [l-1) thread #1 - file://target] GenericFileOnCompletion WARN Error
>> during commit. Exchange[any.zip]. Caused by:
>> [org.apache.camel.component.file.GenericFileOperationFailedException - Error
>> renaming file from
>> C:\temp\camel-examples-master\zip-file-processing\target\any.zip to
>> target\.done\any.zip]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Error
>> renaming file from
>> C:\temp\camel-examples-master\zip-file-processing\target\any.zip to
>> target\.done\any.zip
>> at
>> org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:77)
>> at
>> org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:113)
>> at
>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:88)
>> at
>> org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:124)
>> at
>> org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:80)
>> at
>> org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:54)
>> at
>> org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:100)
>> at
>> org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:228)
>> at
>> org.apache.camel.util.UnitOfWorkHelper.doneUow(UnitOfWorkHelper.java:61)
>> at
>> org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:613)
>> at
>> org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.after(CamelInternalProcessor.java:581)
>> at
>> org.apache.camel.processor.CamelInternalProcessor$InternalCallback.done(CamelInternalProcessor.java:240)
>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:106)
>> at
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>> at
>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:423)
>> at
>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211)
>> at
>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175)
>> at
>> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)
>> at
>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.io.IOException: Renaming file from
>> 'C:\temp\camel-examples-master\zip-file-processing\target\any.zip' to
>> 'target\.done\any.zip' failed: Cannot delete file
>> 'C:\temp\camel-examples-master\zip-file-processing\target\any.zip' after
>> copy succeeded
>> at
>> org.apache.camel.util.FileUtil.renameFileUsingCopy(FileUtil.java:456)
>> at org.apache.camel.util.FileUtil.renameFile(FileUtil.java:428)
>> at
>> org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:74)
>> ... 25 more
>>
>> This is why it grabs the file all the time.
>>
>> Regarding your questions:
>>
>> 1) well thanks to your test you can see error above
>> 2) yes it is processing it over and over again
>> 3) no doesn't move to .error
>> 4) no file is not updated, in your test case I am not quite sure, will
>> increase delay or as you suggested preMove option.
>>
>> Due to known problems off windows and zip files I even renamed it to .xxx to
>> keep away explorer from analyzing, but still it seems there is a handle on
>> that file.
>> And again if I remove the unmarshal and split it works fine.
>>
>> Thanks for any help.
>>
>> Best regards,
>>
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Jimmi Dyson [mailto:[email protected]]
>> Gesendet: Dienstag, 2. September 2014 10:37
>> An: [email protected]
>> Betreff: Re: camel 2.13.2 zipdataformat
>>
>> Hi,
>>
>> I just added an example at
>> https://github.com/jimmidyson/camel-examples/tree/master/zip-file-processing.
>> This example creates the zip file first (so that it's repeatable) & then
>> does pretty much what you're doing: read from file uri, unmarshal, log each
>> file contents (careful with large files) & finally writes each file to a
>> file. Seems to be doing what it should...
>>
>> Few questions if the example doesn't help you:
>>
>> - Are there any hints in the logs (try increasing logging level if not)?
>> - If the file is back in the source dir does it try to process again?
>> - Is there anything in the .error (moveFailed dir)?
>> - Could the source file be being updated while you're processing it - if
>> so, can you try the preMove option?
>>
>> Thanks,
>>
>>
>> On 2 September 2014 08:46, Walzer, Thomas
>> <[email protected]>
>> wrote:
>>
>>> Hi Sven,
>>>
>>> I seem to have a similar problem. I also use 2.13.2.
>>> My code is almost identical. The only difference is my in-url, that
>>> looks
>>> like:
>>>
>>> file:////someserver/somedir/in?move=archive/${date:now:yyyyMMdd}/${f
>>> i l e:name}.${date:now:HHmmss}&antInclude=*.ZIP
>>>
>>> I get a random number of retrievals (about 4-20) and therefore the
>>> same number of files in the „archive“ directory. The timestamps are
>>> 6 seconds apart.
>>>
>>> Cheers, Thomas.
>>>
>>> Am 02.09.2014 um 08:44 schrieb Sven Nold <[email protected]>:
>>>
>>> > Hi Guys,
>>> >
>>> > I am trying to read Zipentries and process them separately. But
>>> > using
>>> the file component it seems the file is not correctly closed. I can
>>> see the file for a few seconds in the .done and source directory.
>>> But then it disappears in the .done folder and is back in the source
>>> directory :
>>> >
>>> >
>>> > <bean id="zipFileDataFormat"
>>> class="org.apache.camel.dataformat.zipfile.ZipFileDataFormat">
>>> > <property name="usingIterator" value="true" /> </bean>
>>> >
>>> > <route id="readZip">
>>> > <from
>>> uri="file://C:/temp/?fileName=any.zip&moveFailed=.error&move=.done&delay=10000"
>>> />
>>> > <unmarshal ref="zipFileDataFormat" />
>>> > <!-- <split streaming="true"> -->
>>> > <!-- <simple>$simple{body}</simple> -->
>>> > <!-- <to
>>> uri="log://OUT?level=INFO&showAll=true&multiline=true" />
>>> -->
>>> > <!-- </split> -->
>>> > <to uri="log://OUT?level=INFO&showAll=true&multiline=true"
>>> />
>>> > </route>
>>> >
>>> > If I remove the unmarshal the file component works fine (it moves
>>> processed file to .done).
>>> >
>>> > Any Ideas? I guess I could do a workaround by using noop=true and
>>> > moving
>>> the file myself, but it would be nice if it works as expected.
>>> >
>>> > Best regards,
>>> >
>>> > Sven
>>>
>>>
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: [email protected]
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> hawtio: http://hawt.io/
> fabric8: http://fabric8.io/
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: [email protected]
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/
