Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-04 Thread Brad Johnson
When I run into this kind of edge case problem I usually start thinking about Java objects. If you have an object with an increment method you invoke when coming in and then have a List inside that you add your aggregated objects into, then when the list length and incremented size are the same,

Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-04 Thread Craig Washington
That’s correct On 11/4/16, 4:13 PM, "Brad Johnson" wrote: It appears that the splitter is exiting while your aggregator is still stuck waiting for the time out if I'm groking the code right. On Nov 4, 2016 3:49 PM, "Craig Washington" wrote: > Thanks Brad. > Jus

Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-04 Thread Brad Johnson
It appears that the splitter is exiting while your aggregator is still stuck waiting for the time out if I'm groking the code right. On Nov 4, 2016 3:49 PM, "Craig Washington" wrote: > Thanks Brad. > Just now taking a look at the resequencer. In my case the final message > and ordering > aren’t

Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-04 Thread Craig Washington
Thanks Brad. Just now taking a look at the resequencer. In my case the final message and ordering aren’t important, only the timing of the split/group/completion. The resequncer may work, likewise I suppose I could get away with using delay() and some sufficiently large value after the split,

Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-04 Thread Craig Washington
@Claus I could be wrong but the composed message processor (splitter-only) doesn’t appear to solve it. I’m not actually interested in the final composed response. I’m only interested that the message was split, processed in batches. My actual route will split a huge file (w/streaming, parallel

Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-04 Thread Brad Johnson
@Craig, This may or may not be appropriate to your situation depending on your situation, but I recently ran into a situation where I was parallel processing and making external REST calls to another company so one couldn't count on uniform transaction speed and an exception absolutely bogged a th

Re: Split, aggregate, then wait for completionTimeout before continuing

2016-11-03 Thread Claus Ibsen
See the composed message processor EIP with the splitter only http://camel.apache.org/composed-message-processor.html On Thu, Nov 3, 2016 at 11:32 PM, Craig Washington wrote: > Hello, > I have a simple use case where I'd like to do the following: > * split a message and process each part > * agg

Split, aggregate, then wait for completionTimeout before continuing

2016-11-03 Thread Craig Washington
Hello, I have a simple use case where I'd like to do the following: * split a message and process each part * aggregate parts into groups of size N for group-processing (w/timeout to ensure no parts are lost) * continue route ONLY after all aggregated parts have completed The simplified route and