Yep, didn't think about memory consumption.
It actually just occurred to me that with such requirements it would be much
more logical to use something like a File component (i.e. a file-based
component, like FtpComponent) - it already addresses sorting, split needs
etc. (Perhaps this is what Claus referred to as 'VFS' in his original
reply?)
So you could do something like:
from("ftp://...";).to("file://downloadedFiles").to("decompress://zip?sort=${file:name}").process(...
but I'm not sure such usage is possible at all.
Alternatively we could extract 2 separate operations on the composite
archives - list included file names and extract file with given name from
the archive. Then we could use existing patterns/custom processors to
transform the list of files according to requirements, and extract files by
one. Only this would perform bad for solid archives I suppose :-(
Christian Schneider wrote:
>
> Vladimir Okhotnikov schrieb:
>> The approach seems okay. The only thing that bothers me is that I'm still
>> not sure the split/aggregate behavior is specific to compression, and
>> will
>> not be useful for other data formats.
>>
> "split" and "aggregate" are already present in the DSL. That is why I
> first thought about using them. The question is if they can be used for
> this case. I guess
> the extraction could be done in an expression like in the example
> split(body(String.class).tokenize("\n"))
> split(body(String.class).decompress(Zip))
>
> The only thing is that I think decompress should work in a streaming way
> to handle large archives. I guess the above example would read the whole
> archive into a string.
>> Christian Schneider wrote:
>>
>>> Btw. In our scenario we had the requirement that the files from the zip
>>> had to be processed in a certain order. In our case the processing
>>> should be done in order of the filenames.
>>> Any idea how this could be expressed?
>>>
>>>
>>
>> decompress(Zip).resequencer()... ?
>>
> Logically this would work but I fear it could consume much memory.
> Resequencer can´t be made streaming but perhaps it can use a disk.
>
> Greetings
>
> Christian
>
>
>
>
>
> --
>
> Christian Schneider
> ---
> http://www.liquid-reality.de
>
>
>
--
View this message in context:
http://www.nabble.com/Zip-format-problem-tp25723682p25741749.html
Sent from the Camel - Users mailing list archive at Nabble.com.
