Re: zip file best practices

[Willem Jiang]

I think we should throw the exception out to let the user know there is 
something wrong with the Zip file. If the ZipIterator eat up the exception, it 
could be every hard for the user to find out why the camel route doesn’t work 
as he expects.
 
I just fill a JIRA[1] to throw the exception out from the ZipIterator. 

[1]https://issues.apache.org/jira/browse/CAMEL-7409 

--  
Willem Jiang

Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem



On May 4, 2014 at 5:37:18 PM, Claus Ibsen (claus.ib...@gmail.com) wrote:
> On Sat, May 3, 2014 at 10:08 AM, Serge Shikov wrote:
> > Here is part of ZipIterator:
> >
> >
> >
> > Note ignored IOException's. I think is should be logged and processed.
> > Probably messages should be generated even if we can't unzip some entries.
> >
> >
> >
> > --
> > View this message in context: 
> > http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5750812.html
> >   
> > Sent from the Camel - Users mailing list archive at Nabble.com.
>  
> Hi
>  
> Yeah sounds like we should detect those io exceptions and propagate
> them back so Camel can react. Maybe we can have an option on the zip
> iterator to ignore those exceptions so people can partially read
> corrupted jars.
>  
> Fell free to log a JIRA ticket
> http://camel.apache.org/support
>  
>  
> --
> Claus Ibsen
> -
> Red Hat, Inc.
> Email: cib...@redhat.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> hawtio: http://hawt.io/
> fabric8: http://fabric8.io/
>  

Re: zip file best practices

[Claus Ibsen]

On Sat, May 3, 2014 at 10:08 AM, Serge Shikov  wrote:
> Here is part of ZipIterator:
>
>
>
> Note ignored IOException's. I think is should be logged and processed.
> Probably messages should be generated even if we can't unzip some entries.
>
>
>
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5750812.html
> Sent from the Camel - Users mailing list archive at Nabble.com.

Hi

Yeah sounds like we should detect those io exceptions and propagate
them back so Camel can react. Maybe we can have an option on the zip
iterator to ignore those exceptions so people can partially read
corrupted jars.

Fell free to log a JIRA ticket
http://camel.apache.org/support


-- 
Claus Ibsen
-
Red Hat, Inc.
Email: cib...@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/

Re: zip file best practices

[Serge Shikov]

Here is part of ZipIterator:



Note ignored IOException's. I think is should be logged and processed.
Probably messages should be generated even if we can't unzip some entries.



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5750812.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Serge Shikov]

Hi, everybody

Could somebody explain, how are errors handled within ZipSplitter and
ZipFileDataFormat?

Just for example, I have a project where I should receive zip files from
external source via http. 

What if zip central directory was corrupted during transfer? Does this
component check CRC for zip entries?

Just for example - I have tested some use cases with my camel unzipping
route, and found that if I try to unzip something (not a zip archive at
all), I'll get just one message after splitter, and original file as its
body. 

And no exceptions, log messages etc. I don't think this is correct.

Regards

Serge



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5750811.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Henryk Konsek]

> Yes, that's good idea. I was thinking about writing another processor to
> do this stuff, but maybe better will be to just improve a bit Camel.

Fell free to create a pull request and ping me when it will be ready.
I'll review it and merge to the master on you behalf.

Cheers.

-- 
Henryk Konsek
http://henryk-konsek.blogspot.com

Re: zip file best practices

[Jakub Bubin]

W dniu 2014-03-03 15:47, hekonsek [via Camel] pisze:
> > I spent a day or two trying the zip component. Finally got it working,
>
> Yeah, Zip File component need some improvements. It lacks some
> functionality and is sometimes a bit unintuitive in a usage.

Good to hear that I was right with my guesses.

>
> > but I
> > think there is still one thing to cover in this component: recursive 
> reading
> > entries from zip package.
>
> We love [1] contributors . Our community would highly appreciate
> pull request with such enhancement. I personally think that we don't
> even need 'recursive' option - we can just automatically detected if
> given zip entry is a directory or file.

Yes, that's good idea. I was thinking about writing another processor to 
do this stuff, but maybe better will be to just improve a bit Camel.

Cheers

-- 
DATAX Sp. z o.o., ul. Fabryczna 10, 53-609 Wrocław, Poland, NIP PL 522-01-00-172
Sąd Rejestrowy dla Wrocławia - Fabrycznej, VI Wydz. Gosp. Krajowego Rejestru 
Sądowego
KRS 126740, Kapitał zakładowy: 131.200 PLN (opłacony w całości)





--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5748254.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Henryk Konsek]

> I spent a day or two trying the zip component. Finally got it working,

Yeah, Zip File component need some improvements. It lacks some
functionality and is sometimes a bit unintuitive in a usage.

> but I
> think there is still one thing to cover in this component: recursive reading
> entries from zip package.

We love [1] contributors :) . Our community would highly appreciate
pull request with such enhancement. I personally think that we don't
even need 'recursive' option - we can just automatically detected if
given zip entry is a directory or file.

Cheers.

[1] https://camel.apache.org/contributing.html

-- 
Henryk Konsek
http://henryk-konsek.blogspot.com

Re: zip file best practices

[Jakub Bubin]

Hi Guys,

I spent a day or two trying the zip component. Finally got it working, but I
think there is still one thing to cover in this component: recursive reading
entries from zip package.

Use case: I want to unzip .zip file which contains many files and process
them separately. But files in package are not packed directly to the zip,
but somewhere in the directory tree (i.e. files.zip -> directory1 ->
directory2 -> file1.txt, file1.txt and so on..).

I think it should be possible with parameter 'recursive' set to 'true'. At
this moment I think it doesn't work this way.



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5748252.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[JonasF]

Ok, thanks I got it working now! The issue was in my missunderstanding that I 
needed to unmarshall the zip file into a zipFile before calling the split.



On 24 Jan 2014, at 22:00, alexey-s [via Camel] 
 wrote:

> I never used to separate files in zip archive. 
> I use a component splitter to read nested XML files. 
> 
> Example use in spring DSL: 
> http://stackoverflow.com/questions/20294005/using-zipsplitter-in-apache-camel-spring-dsl
> 
> Copy/paste and add comments 
> 
>  class="org.apache.camel.dataformat.zipfile.ZipSplitter" />
> ... 
> 
> zipSplitter  
>  
>  
>  
> 
> 
> 
> 
> 
> 
> 
> Aleksey 
> 
> If you reply to this email, your message will be added to the discussion 
> below:
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5746463.html
> To unsubscribe from zip file best practices, click here.
> NAML



SampleZipSplitter.xml (1K) 





--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5746466.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[alexey-s]

I never used to separate files in zip archive.
I use a component splitter to read nested XML files. 

Example use in spring DSL:
http://stackoverflow.com/questions/20294005/using-zipsplitter-in-apache-camel-spring-dsl

Copy/paste and add comments


...

zipSplitter 










Aleksey



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5746463.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[JonasF]

Oh, good! Please provide one working spring XML example! 



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5746454.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[JonasF]

I spent a few hours trying to get this to work in Spring DSL as well without
any luck. I tried debugging it and things looked correct but then somehow I
lost the context and the wrong data was coming out (i.e. the ZipSplitter was
called and did its job but the answer got lost somehow). I ran out of
time/patience and implemented a processor which performed the decompression
to a temporary file and passed on each file as a new exchange.

Here is what my xml looked like when it almost worked...



...

 
 
 
 
  



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5746412.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[alexey-s]

ZipSplitter uses the input stream from the archive file (message body). Sends
output stream for each attached file. 
One should not expect from him GenericFile
Nothing complicated.



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5746434.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[alexey-s]

Can be improved.
I thought it would be interesting to the user to remember the name of the
zip file.
Now ZipItaror not give this information.

answer.getHeaders().putAll(inputMessage.getHeaders());
answer.setHeader("zipFileName", current.getName());
answer.setHeader(Exchange.FILE_NAME, current.getName());

It has been
${header.camelFileName} = archive_file.zip
${header.zipFileName} = attached_file_name

It has become
${header.camelFileName} = attached_file_name
${header.zipFileName} = attached_file_name

What for?


Alexey



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5741856.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[peter]

I believe there was an original request on this post for someone to please
show how to use the ZipSplitter with Spring DSL. But I have not seen this
anywhere and I have not been able to get it to work. Here is a snippet of my
attempt:



  http://camel.apache.org/schema/spring";>




${file:name.ext} == 'max'

myZipSplitter


${header.ZipFileName.ext} ==
'pdz'








.
.
.
Could someone please post a correct example of usage of the ZipFile
Dataformat (using the ZipSplitter expression) or a statement that it cannot
be used with Spring?



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5741806.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Willem jiang]

Hi Juan,

Thanks for reporting it. I will check out your patch when I get some time next 
week.  

--  
Willem Jiang

Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) 
(English)
  http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem





On Saturday, July 6, 2013 at 3:50 AM, Gardella juan wrote:

> Hello Willieem,
>  
> It seems there is a problem with the ZipSplitter that you did. I had added a
> test "ZipProcessRouteBuilderIT" to reproduce the error at
> https://issues.apache.org/jira/browse/CAMEL-6139. If you use my ZipSplitter
> the test works.  
>  
> Thanks,
> Juan
>  
>  
>  
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5735214.html
> Sent from the Camel - Users mailing list archive at Nabble.com 
> (http://Nabble.com).

Re: zip file best practices

[Gardella juan]

Hello Willieem,

It seems there is a problem with the ZipSplitter that you did. I had added a
test "ZipProcessRouteBuilderIT" to reproduce the error at
https://issues.apache.org/jira/browse/CAMEL-6139. If you use my ZipSplitter
the test works. 

Thanks,
Juan



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5735214.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Willem jiang]

Hi Henryk,

I'm sorry I just checked the code Zip data format before I applied patch.
I cannot agree with you more after checking the code of ZipFile DataFormat.
Current ZipFile DataFormat doesn't support to unmarshal the Zip file which has 
more than one entry.
So it make sense to create the Iterator of ZipEntry inside the ZipFile 
DataFormat.

I will revisit the patch again and move the code to camel-zipfile module.  

--  
Willem Jiang

Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) 
(English)
  http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem





On Saturday, March 16, 2013 at 4:27 AM, Henryk Konsek wrote:

> > Otherwise its hard for our end users
> > to know such functionality exists.
>  
>  
>  
> Is there any particular reason to provide separated splitter in
> camel-core instead of enhancing existing ZipFile Data Format? Am I
> missing something? :) The following snippet would look just perfect
> for me...
>  
> from(...).unmarshal().zipFile().split(body(Iterable.class)).to(...)
>  
> In my humble opinion ZipFile data format is responsible for handle
> ZipEntry streams, so we shouldn't create ZipSplitter in camel-core but
> rather improve existing ZipFile code.
>  
> Best regards.
>  
> --
> Henryk Konsek
> http://henryk-konsek.blogspot.com

Re: zip file best practices

[Henryk Konsek]

> Otherwise its hard for our end users
> to know such functionality exists.

Is there any particular reason to provide separated splitter in
camel-core instead of enhancing existing ZipFile Data Format? Am I
missing something? :) The following snippet would look just perfect
for me...

from(...).unmarshal().zipFile().split(body(Iterable.class)).to(...)

In my humble opinion ZipFile data format is responsible for handle
ZipEntry streams, so we shouldn't create ZipSplitter in camel-core but
rather improve existing ZipFile code.

Best regards.

--
Henryk Konsek
http://henryk-konsek.blogspot.com

Re: zip file best practices

[Claus Ibsen]

On Wed, Mar 13, 2013 at 3:32 AM, Willem.Jiang  wrote:
> I just have a chance to check the code, it is not a new component.
> I think we can treat it as an expression, which can help the splitter to
> build a Iterator from the uncompressed stream. And it is mainly used with
> split DSL.
>
> Willem
>

I suggest to add a nice example in the unit test and use SNIPPET tags
to have this being used as an example we can show in the Camel docs,
such as on the Splitter EIP page. Otherwise its hard for our end users
to know such functionality exists.


>
>
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5729057.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cib...@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

Re: zip file best practices

[Willem.Jiang]

I just have a chance to check the code, it is not a new component.
I think we can treat it as an expression, which can help the splitter to
build a Iterator from the uncompressed stream. And it is mainly used with
split DSL.

Willem



--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5729057.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Henryk Konsek]

Hi guys,

> Puff, sorry. The Jira is https://issues.apache.org/jira/browse/CAMEL-6139

Actually we already got ZipFile component [1], however it supports
only single-entry zip files at the moment.

Instead of creating separated module with zip file splitter I suggest
to add multi-entry zip files support to the
ZipFileDataFormat#unmarshal method, which could just return the
iterator. We would be able to split zip file as follows then...

from("direct:multi-entry-zip-file").unmarshal().zip().split().to("direct:single-zip-entry");

Are there any particular reasons against reusing and improving
existing ZipFile data format?

Best regards.

[1] http://camel.apache.org/zip-dataformat.html

--
Henryk Konsek
http://henryk-konsek.blogspot.com

Re: zip file best practices

[Gardella juan]

Hi,

It is a new feature so I don't touch any existent class of camel. So, in
what module I have to do a patch?

Juan


2013/3/4 Claus Ibsen-2 [via Camel] 

> Hi
>
> Thanks for your contribution.
>
> Though its easier if its a patch. So I suggest to read here about how
> to create and submit patches to Apache Camel
> http://camel.apache.org/contributing.html
>
>
> On Mon, Mar 4, 2013 at 3:05 PM, Gardella juan
> <[hidden email] >
> wrote:
>
> > The source code has a bug. I fixed it and I want to contribute because
> the
> > solution is based in alexey-s solution. I 've attached the source code.
> >
> > camel-zipsplitter.zip
> > 
> >
> > Thanks,
> > Juan
> >
> >
> >
> > --
> > View this message in context:
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5728454.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: [hidden email]
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
>
>
> --
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5728457.html
>  To unsubscribe from zip file best practices, click 
> here
> .
> NAML
>




--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5728459.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Claus Ibsen]

Hi

Thanks for your contribution.

Though its easier if its a patch. So I suggest to read here about how
to create and submit patches to Apache Camel
http://camel.apache.org/contributing.html


On Mon, Mar 4, 2013 at 3:05 PM, Gardella juan
 wrote:
> The source code has a bug. I fixed it and I want to contribute because the
> solution is based in alexey-s solution. I 've attached the source code.
>
> camel-zipsplitter.zip
> 
>
> Thanks,
> Juan
>
>
>
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5728454.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cib...@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

Re: zip file best practices

[alexey-s]

You can not get a ZipFile from InputStream. We can only ZipInputStream. When
processing files within the zip file, you can not close the stream reader.
This is an InputStream for the entire zip file. 

Enclose an example of processing of xml file inside the zip file. In the
example of a very large zip file.
http://camel.465427.n5.nabble.com/file/n5714149/camel-spliter-stax.tar.gz
camel-spliter-stax.tar.gz 


Aleksey

--
View this message in context: 
http://camel.465427.n5.nabble.com/zip-file-best-practices-tp5713437p5714149.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: zip file best practices

[Claus Ibsen]

On Mon, May 28, 2012 at 5:55 PM, Christian Müller
 wrote:
> What do you think about the following proposal:
>
> 1) Add a type converter which supports a conversion from InputStream ->
> ZipFile and from InputStream -> ZipInputStream
> 2) Implement a splitter which extract the (multiple) ZipEntry, add some
> header to each of them (name of the entry, number of entries, size, ...)
> and put them into its own exchange.
>

Yeah that sounds reasonable. A prototype is usually a good idea to try
it out and see what works.


> For the other way (create a zip archive which contain multiple entries),
> you could provide an Aggregator (AggregationStrategy) which work with
> ZipEntries...
>

I wonder if its not easier to allow to append to an existing file,
which is a .zip file as well?


> My 0.02$,
> Christian
>
> On Fri, May 25, 2012 at 1:29 PM, Tyler Durvik  wrote:
>
>> I am working on a solution for this that I can contribute to Camel.  I
>> have the files being unzipped in a processor.  The processor then sets
>> a header value which is a comma-separated string listing the files.
>> The problem I see is that I want the splitter to generate a new Camel
>> Message object for each file in zip file.  Then in each Message object
>> I set a filename header for each the file that message represents.
>> The problem is that the splitter generates 2 messages, each with the
>> same file header value.  So here is my flow:
>>
>> Zip File
>>
>> |
>>
>> Unzip Processor - uncompress zip which is contained in body, then set
>> header.  key = filenames, values=file1,file2
>>
>> |
>>
>> Splitter - based on "filenames" header,  set new key = filename
>>
>> |
>>
>> Message 1 - key = filename, value = file1
>> Message 2 - key = filename, value = file1
>>
>>
>> What I want is the header of Message 2 to contain file2.  I will write
>> unit test to verify this if anyone does not understand or know how to
>> help.
>>
>> Thank you for your help,
>> Tyler
>>
>>
>>
>> On Thu, May 24, 2012 at 1:18 AM, Claus Ibsen 
>> wrote:
>> > On Wed, May 23, 2012 at 10:43 PM, Tyler Durvik 
>> wrote:
>> >> I am receiving data message from client and the body of the message is
>> >> a zip file containing multiple files.  What is the best method to
>> >> process the zip data?  I assume that I should use a StreamMessage.  Is
>> >> there support for Camel to unzip the body as I receive it like
>> >> marshaling or do I need to write an unzip processor or splitter.
>> >>
>> >> Thank you for your time,
>> >> Tyler
>> >
>> > You would need to write code to unzip that yourself.
>> >
>> > The zip/gzip data formats is for compressing/decompression message
>> > bodies, not zip files.
>> >
>> > There is a JIRA ticket to see if we can add support for zip files as
>> well.
>> > Contributions is as always welcome.
>> >
>> > For example maybe a iterator that can walk the zip file entries, which
>> > you can then use with the Splitter EIP to process each file by file.
>> >
>> >
>> >
>> > --
>> > Claus Ibsen
>> > -
>> > CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
>> > FuseSource
>> > Email: cib...@fusesource.com
>> > Web: http://fusesource.com
>> > Twitter: davsclaus, fusenews
>> > Blog: http://davsclaus.blogspot.com/
>> > Author of Camel in Action: http://www.manning.com/ibsen/
>>



-- 
Claus Ibsen
-
FuseSource
Email: cib...@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

Re: zip file best practices

[Christian Müller]

What do you think about the following proposal:

1) Add a type converter which supports a conversion from InputStream ->
ZipFile and from InputStream -> ZipInputStream
2) Implement a splitter which extract the (multiple) ZipEntry, add some
header to each of them (name of the entry, number of entries, size, ...)
and put them into its own exchange.

For the other way (create a zip archive which contain multiple entries),
you could provide an Aggregator (AggregationStrategy) which work with
ZipEntries...

My 0.02$,
Christian

On Fri, May 25, 2012 at 1:29 PM, Tyler Durvik  wrote:

> I am working on a solution for this that I can contribute to Camel.  I
> have the files being unzipped in a processor.  The processor then sets
> a header value which is a comma-separated string listing the files.
> The problem I see is that I want the splitter to generate a new Camel
> Message object for each file in zip file.  Then in each Message object
> I set a filename header for each the file that message represents.
> The problem is that the splitter generates 2 messages, each with the
> same file header value.  So here is my flow:
>
> Zip File
>
> |
>
> Unzip Processor - uncompress zip which is contained in body, then set
> header.  key = filenames, values=file1,file2
>
> |
>
> Splitter - based on "filenames" header,  set new key = filename
>
> |
>
> Message 1 - key = filename, value = file1
> Message 2 - key = filename, value = file1
>
>
> What I want is the header of Message 2 to contain file2.  I will write
> unit test to verify this if anyone does not understand or know how to
> help.
>
> Thank you for your help,
> Tyler
>
>
>
> On Thu, May 24, 2012 at 1:18 AM, Claus Ibsen 
> wrote:
> > On Wed, May 23, 2012 at 10:43 PM, Tyler Durvik 
> wrote:
> >> I am receiving data message from client and the body of the message is
> >> a zip file containing multiple files.  What is the best method to
> >> process the zip data?  I assume that I should use a StreamMessage.  Is
> >> there support for Camel to unzip the body as I receive it like
> >> marshaling or do I need to write an unzip processor or splitter.
> >>
> >> Thank you for your time,
> >> Tyler
> >
> > You would need to write code to unzip that yourself.
> >
> > The zip/gzip data formats is for compressing/decompression message
> > bodies, not zip files.
> >
> > There is a JIRA ticket to see if we can add support for zip files as
> well.
> > Contributions is as always welcome.
> >
> > For example maybe a iterator that can walk the zip file entries, which
> > you can then use with the Splitter EIP to process each file by file.
> >
> >
> >
> > --
> > Claus Ibsen
> > -
> > CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
> > FuseSource
> > Email: cib...@fusesource.com
> > Web: http://fusesource.com
> > Twitter: davsclaus, fusenews
> > Blog: http://davsclaus.blogspot.com/
> > Author of Camel in Action: http://www.manning.com/ibsen/
>

Re: zip file best practices

[Tyler Durvik]

I am working on a solution for this that I can contribute to Camel.  I
have the files being unzipped in a processor.  The processor then sets
a header value which is a comma-separated string listing the files.
The problem I see is that I want the splitter to generate a new Camel
Message object for each file in zip file.  Then in each Message object
I set a filename header for each the file that message represents.
The problem is that the splitter generates 2 messages, each with the
same file header value.  So here is my flow:

Zip File

|

Unzip Processor - uncompress zip which is contained in body, then set
header.  key = filenames, values=file1,file2

|

Splitter - based on "filenames" header,  set new key = filename

|

Message 1 - key = filename, value = file1
Message 2 - key = filename, value = file1


What I want is the header of Message 2 to contain file2.  I will write
unit test to verify this if anyone does not understand or know how to
help.

Thank you for your help,
Tyler



On Thu, May 24, 2012 at 1:18 AM, Claus Ibsen  wrote:
> On Wed, May 23, 2012 at 10:43 PM, Tyler Durvik  wrote:
>> I am receiving data message from client and the body of the message is
>> a zip file containing multiple files.  What is the best method to
>> process the zip data?  I assume that I should use a StreamMessage.  Is
>> there support for Camel to unzip the body as I receive it like
>> marshaling or do I need to write an unzip processor or splitter.
>>
>> Thank you for your time,
>> Tyler
>
> You would need to write code to unzip that yourself.
>
> The zip/gzip data formats is for compressing/decompression message
> bodies, not zip files.
>
> There is a JIRA ticket to see if we can add support for zip files as well.
> Contributions is as always welcome.
>
> For example maybe a iterator that can walk the zip file entries, which
> you can then use with the Splitter EIP to process each file by file.
>
>
>
> --
> Claus Ibsen
> -
> CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
> FuseSource
> Email: cib...@fusesource.com
> Web: http://fusesource.com
> Twitter: davsclaus, fusenews
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/

Re: zip file best practices

[Christian Müller]

The ZipDataFormat expects an InputStream and will return a ByteArray by
unmarshalling the payload.
If you read a ZIP file, you can convert it to an InputStream. But than, you
have to create a java.util.zip.ZipFile or a java.util.zip.ZipInputStream to
extract all entries from the archive. Than you can iterate over the
entries, extract it and process all of them further...

Best,
Christian

On Thu, May 24, 2012 at 2:23 PM, Tyler Durvik  wrote:

> Thank you.  I have one more question.  What is the difference between
> zip the body and a zip file as the body of a message?
>
>
>
> On Thu, May 24, 2012 at 1:18 AM, Claus Ibsen 
> wrote:
> > On Wed, May 23, 2012 at 10:43 PM, Tyler Durvik 
> wrote:
> >> I am receiving data message from client and the body of the message is
> >> a zip file containing multiple files.  What is the best method to
> >> process the zip data?  I assume that I should use a StreamMessage.  Is
> >> there support for Camel to unzip the body as I receive it like
> >> marshaling or do I need to write an unzip processor or splitter.
> >>
> >> Thank you for your time,
> >> Tyler
> >
> > You would need to write code to unzip that yourself.
> >
> > The zip/gzip data formats is for compressing/decompression message
> > bodies, not zip files.
> >
> > There is a JIRA ticket to see if we can add support for zip files as
> well.
> > Contributions is as always welcome.
> >
> > For example maybe a iterator that can walk the zip file entries, which
> > you can then use with the Splitter EIP to process each file by file.
> >
> >
> >
> > --
> > Claus Ibsen
> > -
> > CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
> > FuseSource
> > Email: cib...@fusesource.com
> > Web: http://fusesource.com
> > Twitter: davsclaus, fusenews
> > Blog: http://davsclaus.blogspot.com/
> > Author of Camel in Action: http://www.manning.com/ibsen/
>

Re: zip file best practices

[Tyler Durvik]

Thank you.  I have one more question.  What is the difference between
zip the body and a zip file as the body of a message?



On Thu, May 24, 2012 at 1:18 AM, Claus Ibsen  wrote:
> On Wed, May 23, 2012 at 10:43 PM, Tyler Durvik  wrote:
>> I am receiving data message from client and the body of the message is
>> a zip file containing multiple files.  What is the best method to
>> process the zip data?  I assume that I should use a StreamMessage.  Is
>> there support for Camel to unzip the body as I receive it like
>> marshaling or do I need to write an unzip processor or splitter.
>>
>> Thank you for your time,
>> Tyler
>
> You would need to write code to unzip that yourself.
>
> The zip/gzip data formats is for compressing/decompression message
> bodies, not zip files.
>
> There is a JIRA ticket to see if we can add support for zip files as well.
> Contributions is as always welcome.
>
> For example maybe a iterator that can walk the zip file entries, which
> you can then use with the Splitter EIP to process each file by file.
>
>
>
> --
> Claus Ibsen
> -
> CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
> FuseSource
> Email: cib...@fusesource.com
> Web: http://fusesource.com
> Twitter: davsclaus, fusenews
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/

Re: zip file best practices

[Claus Ibsen]

On Wed, May 23, 2012 at 10:43 PM, Tyler Durvik  wrote:
> I am receiving data message from client and the body of the message is
> a zip file containing multiple files.  What is the best method to
> process the zip data?  I assume that I should use a StreamMessage.  Is
> there support for Camel to unzip the body as I receive it like
> marshaling or do I need to write an unzip processor or splitter.
>
> Thank you for your time,
> Tyler

You would need to write code to unzip that yourself.

The zip/gzip data formats is for compressing/decompression message
bodies, not zip files.

There is a JIRA ticket to see if we can add support for zip files as well.
Contributions is as always welcome.

For example maybe a iterator that can walk the zip file entries, which
you can then use with the Splitter EIP to process each file by file.



-- 
Claus Ibsen
-
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cib...@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/