Hi You dont have to use the aggregator in the splitter. You could just route each splitted message to a file endpoint and have it append the target file.
On Tue, Jul 6, 2010 at 11:31 AM, Christian Mueller <christian.muel...@gmail.com> wrote: > > Hello dev, > > at present, I use Camel 2.2-fuse-02-00 (inside FUSE ESB 4.2-fuse-02-00) and > plan to upgrade to 2.3-fuse-01-00 if it's available. > > I have a requirement to read big fixed length files (up to 500 MB), split > the content in individual messages, process/transform these messages into > xml, aggregate all transformed messages and than write it into a new file. > > My questions are about the aggregator: > - If I use the 'streaming()' option in the splitter, is the > 'CamelSplitComplete' exchange property the best strategy to detect in the > aggregator that all messages are processed? Requires this, that I can not > process/transform the individual messages in parallel? My understanding is > also, that I can not use the 'completionFromBatchConsumer' option in the > aggregator, because it use the 'CamelBatchSize' header which is not set in > streaming mode. Do you see other options? > - My understanding is, that the aggregation is done in memory and not made > for aggregating big messages (500 MB or so). Would it be useful/possible to > extend the aggregator to stream the content into a file (may be with a > header and a footer)? Or should this part of a custom aggregating strategy? > - Another solution could be to store all transformed messages into > individual files and after all messages are processed, the aggregation > strategy kicks in a custom processor which read all files in stream it into > a new (big) file. > - Do you see other solutions for this? > > May be a good topic for the wiki, what I could write... > > Regards, > Christian > -- > View this message in context: > http://camel.465427.n5.nabble.com/Splitting-big-files-and-aggregate-the-big-responses-tp858166p858166.html > Sent from the Camel Development mailing list archive at Nabble.com. > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
