Well apparently something is alleviated, but I'm trying to understand what... The 2 queues are sequential, and the 1st queue is almost always empty, so I cannot see why I have a bigger buffer.
On Sat, Aug 4, 2012 at 1:17 PM, Sam (Stephen Samuel) <samspad...@gmail.com>wrote: > With the intermediate queue you have a bigger buffer, a 2nd queue. > There might be a bottleneck on certain calls from your initial > endpoint which having the 2nd queue alleviates. > > On Sat, Aug 4, 2012 at 9:48 AM, Xenofon Papadopoulos > <xenofon.papadopou...@upstreamsystems.com> wrote: > > Thanks for the tips. I'm trying, however, to understand why there is > better > > behavior with the intermediate queue, given that all other parameters > > (threads etc) remain the same. > > > > Also this is a test; in production we normally use about 800 concurrent > > consumers, and the resource use in the servers is pretty low. > > > > > > On Sat, Aug 4, 2012 at 6:44 AM, Ben O'Day <ben.o...@initekconsulting.com > >wrote: > > > >> tough to say given the unknowns about your setup, but here are a couple > of > >> things to consider... > >> > >> -for kahadb, set enableJournalDiskSyncs to false to get much better > >> throughput > >> -try fewer consumer threads...100 is a lot and you generally have > >> diminished/negative results with larger thread counts > >> -tune your CachingConnectionFactory (notably the sessionCacheSize > should be > >> increased from the default)...that said, beware of using this with > polling > >> consumers connections (see > >> > >> > http://stackoverflow.com/questions/5916638/autoreconnect-problem-with-activemq-and-cachingconnectionfactory > >> ) > >> -alternatively, try using the AMQ PooledConnectionFactory and see if you > >> get better performance > >> -see this thread on AMQ performance: > >> > >> > http://stackoverflow.com/questions/7463415/activemq-performance-gotchas-and-precautions > >> > >> > >> -Ben > >> > >> > >> > >> On Fri, Aug 3, 2012 at 8:44 AM, Xenofon Papadopoulos < > >> xenofon.papadopou...@upstreamsystems.com> wrote: > >> > >> > Sorry in advance if I should have posted that in ActiveMQ list, but > here > >> is > >> > our case. > >> > We are running the same test with two different setups: > >> > > >> > Setup 1: > >> > ------------ > >> > We are using a single ActiveMQ broker and a single Camel with the > routes: > >> > > >> > from( "jetty://http://..."; ).inOnly( "activemq:queue:queue2" ) > >> > > >> > from("activemq:queue:queue2?maxConcurrentConsumers=100").bean( > >> myProcessor, > >> > "delay" ) > >> > > >> > The activeMQ component uses > >> > a org.springframework.jms.connection.CachingConnectionFactory > >> > > >> > myProcess.delay() delays for 200 ms > >> > > >> > > >> > Setup 2: > >> > ------------ > >> > We are using two ActiveMQ brokers and a single Camel with the routes: > >> > > >> > from( "jetty://http://..."; ).inOnly( "activemq1:queue:queue1" ) > >> > > >> > from( > >> > "activemq1:queue:queue1?maxConcurrentConsumers=100").inOnly( > >> > "activemq2:queue:queue2" > >> > ) > >> > > >> > from("activemq2:queue:queue2?maxConcurrentConsumers=100").bean( > >> > myProcessor, "delay" ) > >> > > >> > Both activeMQ components use their > >> > own org.springframework.jms.connection.CachingConnectionFactory > >> > ------------------------ > >> > > >> > All ActiveMQ brokers are identical, with: > >> > > >> > - kahadb > >> > - <policyEntry queue=">" queuePrefetch="20" > producerFlowControl="false" > >> > optimizedDispatch="true" useCache="true"> > >> > - <memoryUsage limit="4 gb"/> > >> > - <transportConnector name="openwire" uri="tcp://0.0.0.0:xxxx"/> > >> > > >> > We test the setup by sending http requests from jmeter with 40 threads > >> and > >> > 1200 tps filter. We monitor the enqueue and dequeue rates of all the > >> > queues, and notice that: > >> > > >> > Setup 1: > >> > ------------ > >> > queue2 enqueue: ~900 tps > >> > queue2 dequeue: ~50 tps > >> > > >> > > >> > Setup 2: > >> > ------------ > >> > queue1 enqueue: ~1200 tps > >> > queue1 dequeue: ~1200 tps > >> > queue2 enqueue: ~1200 tps > >> > queue2 dequeue: ~150 tps > >> > > >> > This means that the 2nd setup causes the brokers to behave much better > >> and > >> > gives us a much higher throughput for our processor. > >> > How can this be explained? Is there any configuration that will let us > >> > achieve the same performance with a single broker? > >> > > >> > Thank you > >> > Xenofon > >> > > >> > > > > > > > > -- > > *Xenofon Papadopoulos > > * > > *Upstream > > *4 Kastorias & Messinias Street > > 153 44 Gerakas Attikis > > Mob +30 694 027 5392 > > DL +30 210 661 8277 > > Fax +30 210 661 8550 > > www.upstreamsystems.com > > > > -- > -Sam > -- *Xenofon Papadopoulos * *Upstream *4 Kastorias & Messinias Street 153 44 Gerakas Attikis Mob +30 694 027 5392 DL +30 210 661 8277 Fax +30 210 661 8550 www.upstreamsystems.com
