I didn't think you were appending the data. But realize that when you call direct or direct-vm it is a request/response style of route. So you are calling route 1 which calls route 2 which calls route 1 and are not returning as far as I know. So you are essentially making the equivalent of a recursive method call. You should end up with a stack overflow that way eventually not because of the size of the individual messages but due to the number of calls from 1 to 2 and 2 to 1 and 1 to 2 and so on. Essentially you keep filling up the stack with each new call.
There is a chapter on the POJO approach in the Camel in Action book with the same sort of recommendation. All of chapter 4 is about it and one section, for example, is called "using beans the easy way" and another covers annotations. Hopefully in the new book (V2) they'll emphasize it a bit more. Most of the time you don't need to specify the annotations if you are passing a bean in the body that Camel can identify as the type and method to invoke. Otherwise you can annotate the method call and type with @Body and if you need to pass in a header you use @Header. If you want to specify the method to call I think it's @Handler (can't be sure as I don't use it most of the time). http://camel.apache.org/bean.html http://camel.apache.org/parameter-binding-annotations.html On Fri, Oct 14, 2016 at 4:07 PM, Goyal, Arpit <arpit.go...@sap.com> wrote: > Hi Brad, > > TOPIC 1: > ----------- > "StackOverflowError" - this is what happened with the current Route 2 > calling Route 1 for LARGE # OF PAGES. It was not about the data, because > the body is not getting appended and is always limited to one page (at > least in most cases). > > JVM/stack-size exploded around about 200 calls to the loop :) But yeah a > good learning of what not to do :) Moved the Loop outside the route > completely and based on Route 2 response, triggered the Route 1 again. > > Route 1 to Route 2 linkage was done only for modularization purpose. > > TOPIC 2: > ---------- > Very interesting point you have raised here about using POJO and not > Processors. I actually wanted to know is there some explanation of what you > said in CAMEL IN ACTION book. Just got hand of hard-copy and see if I can > understand that better. > > I did use split and bean binding, but not sure if that is the right POJO > approach? I guess I should use @ExchangeProperty > - public List<Message> splitMessage(@Body Object body, Exchange > exchange) > > Current use cases which processor is doing: > - set / get exchange headers (multiple) > - set / get camel exchange property (so far single) > - saw annotation ExchangeProperty, but it has String type return > value. Whereas on exchange we are setting Objects. > > Regards, > Arpit. > > > -----Original Message----- > From: Brad Johnson [mailto:brad.john...@mediadriver.com] > Sent: Friday, October 14, 2016 8:54 AM > To: users@camel.apache.org > Subject: Re: Looping in routes using Direct Component - Performance impact? > > Have you looked at splitter/tokenizer/streaming for this instead of having > route 1 call route 2 which calls route 1? Interesting route setup as I've > never used routes in a recursive fashion like that. When using direct the > return doesn't happen until the end of the last route in the chain. In > this case it appears that you are continually delaying the return from the > route via this iterative approach. I'd have to check that via testing to > see what's really happening in that case but I've never done it like that > and would assume it would eventually run out of memory for large amounts of > data. As far as I know you can't just erase the history though you could > set the body to null or empty string. But I wouldn't do that. > > Just unmarshal your data and run it through the splitter in a streaming > fashion. If you don't need to process these in serial order you could then > drop them on a SEDA queue with multiple consumer threads. You can then also > set the queue size to limit how many documents are coming through at a > time. > > Also, unless you have a compelling reason don't use Processors as they > couple you to the framework. Just use a POJO and let Camel use reflection > to select the correct method for processing or use annotations. As a > general rule of thumb I use, if you don't know exactly why you need to use > a Processor you don't need to use it. > > > On Thu, Oct 13, 2016 at 6:23 PM, Goyal, Arpit <arpit.go...@sap.com> wrote: > > > I can't compromise my business logic and there is hardly any data; but > > each time camel Route 2 starts Route 1, the previous route processing > goes > > onto stack. I only have limited stack after that any operation can fail. > > > > To me below approach seems risky...I need to go async or remove the > > looping out of the route overall. > > > > Regards, > > Arpit. > > > > -----Original Message----- > > From: souciance [mailto:souciance.eqdam.ras...@gmail.com] > > Sent: Thursday, October 13, 2016 3:45 PM > > To: users@camel.apache.org > > Subject: Re: Looping in routes using Direct Component - Performance > impact? > > > > I think you should first run your route setup with a few pages, 10s of > > pages, hundreds of pages and then thousands of pages and then do > > performance tests to see where what is eating memory and optimise from > > there. > > > > On Fri, Oct 14, 2016 at 12:31 AM, Goyal, Arpit [via Camel] < > > ml-node+s465427n5788744...@n5.nabble.com> wrote: > > > > > Hi, > > > > > > We currently are on 2.16.3 version and can't use the loop function > (part > > > of 2.17). So we have the following routes based on which we create sort > > of > > > loop with switch case. Now we are worried about the performance (when > we > > > see stacktrace of camel processing) > > > > > > Route 1: from("direct:s1).to("read one page of data").process("process > > > one page").to("direct:s2"); > > > > > > Route 2: from("direct:s2").toD("go to external source").when("are more > > > pages left").to("direct:s1").otherwise().process("end processing"); > > > > > > Now if there are thousands of pages, it is possible that this loop > > creates > > > lot of objects in memory & can create performance issue? Is there a way > > for > > > Camel to be told, before going to route 1 again, forget all the past? > Is > > > disabling HISTORY would solve that? > > > > > > Regards, > > > Arpit. > > > > > > > > > ------------------------------ > > > If you reply to this email, your message will be added to the > discussion > > > below: > > > http://camel.465427.n5.nabble.com/Looping-in-routes-using- > > > Direct-Component-Performance-impact-tp5788744.html > > > To start a new topic under Camel - Users, email > > > ml-node+s465427n465428...@n5.nabble.com > > > To unsubscribe from Camel - Users, click here > > > <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro= > > unsubscribe_by_code&node=465428&code=c291Y2lhbmNlLmVxZGFtLnJhc2h0aU > > BnbWFpbC5jb218NDY1NDI4fDE1MzI5MTE2NTY=> > > > . > > > NAML > > > <http://camel.465427.n5.nabble.com/template/ > NamlServlet.jtp?macro=macro_ > > viewer&id=instant_html%21nabble%3Aemail.naml&base= > nabble.naml.namespaces. > > BasicNamespace-nabble.view.web.template.NabbleNamespace- > > nabble.view.web.template.NodeNamespace&breadcrumbs= > > notify_subscribers%21nabble%3Aemail.naml-instant_emails% > > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > > > > > > > > > > -- > > View this message in context: http://camel.465427.n5.nabble. > > com/Looping-in-routes-using-Direct-Component-Performance- > > impact-tp5788744p5788745.html > > Sent from the Camel - Users mailing list archive at Nabble.com. > > >
