Got it running better by setting the split operation not parallel and by
threading on the split results.

Here is the output from VisualVM: http://i.imgur.com/m4jFQdK.png

Here is the output from Camel: http://i.imgur.com/1BZWl4b.png.
The test starts at 10:55. The previous one is from the other test, with no
.threads().

As you can see, it is much faster but it still drops. I don't know if a
larger result set won't stop it because that slope tends to zero...

I think the problem is that I'm creating a whole lot of objects just to
create the XML. Maybe I should create the XML directly by string
concatenation without creating the objects first.

Maven Ran camel:run with four threads.

BTW, I changed from SpringDSL to JavaDSL.


@Component
public class ReadDB extends SpringRouteBuilder{
        
    @Override
    public void configure() throws Exception {
        
       
from("quartz2://myGroup/myTimerName?trigger.repeatInterval=20000&trigger.repeatCount=0")
        .to("sql:select * from sales where integration_id = (select
max(integration_id) as max_int from sales)")
        .split(simple("${body}")).streaming()
                .to("myLogger")
                .to("direct:sqlqueue");
    }

}

public class WriteToQueue extends SpringRouteBuilder{
        
    @Override
    public void configure() throws Exception {          
        from("direct:sqlqueue")
        .threads(4)
        .to("lineProcessor")
        .marshal("xstream-utf8")      
       
.to("rabbitmq://xx.xx.xx.xx:5672/myExchange?username=xxxx&password=xxxx")
        ;
    }

}




--
View this message in context: 
http://camel.465427.n5.nabble.com/Camel-gradually-slowing-down-with-large-sql-results-tp5741560p5741594.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Reply via email to