Hi Frase, You’ve done a good job at guessing the limits - I’ll let Gordon educate us on the rest.
The threading model, though - definitely not per connection. There’s a pool of threads that can handle I/O - by default there are (number of cores) + 1, but you can change it by command line option. The I/O model is proactive (as opposed to reactive) and those threads handle what ever handles get I/O completions. Yes, at some point, it gets saturated but it’s driven by the amount of I/O going on concurrently, not by the sheer number of connections. So go for it - let us know how high you (well, the number of clients) can get :-) -Steve > On Jul 11, 2015, at 12:52 PM, Fraser Adams <fraser.ad...@blueyonder.co.uk> > wrote: > > Hey all, > Suppose I have a queue on qpidd, I know that I can have multiple consumer > clients subscribing to that queue node - I've used that several times to > provide a means of scaling out consumers, so if I have "n" consumers each > consumer receives roughly 1/n of the messages published onto the queue. > > I've never really pushed the limits of this and have tended to only have 10 - > 20 consumers, but it got me wondering: > a) What's the maximum theoretical limit for the number of consumers on such a > shared resource > b) what's the maximum practical limit - I'm not sure how the qpidd threading > model works (I *think* it's per connection), so I'm wondering at what point > we get into a position where the "stampeding herds" lock contention problem > kicks in. > > I know I should probably just stand up a test and give it a whirl, but > figured I'd ask first :-) > > I *think* that the answer to a) is that the theoretical limit is the maximum > number of link handles and is a 32 bit unsigned int so would be 4294967295, > though I suspect something else would get a bit sad before that limit is > reached. > > > On point b. has anyone (most likely Gordon) explored the scaling limits of > qpidd? Obviously when Qpid started out servers tended to have something > between one and four cores, but now of course Moore's law tends to be > followed by increasing the number of cores, so I'm curious as to how qpidd > scales. I'm thinking of things like ActiveMQ Apollo where as I understand it > it uses hawt-dispatch (which I think is the Java equivalent of Apple's Grand > Central Dispatch pattern) and I know with Proton there has been a lot of work > migrating to a more reactive pattern, which makes me wonder if that's an > acknowledgement of any potential scaling limits in qpidd at present? If there > are known limits is there a roadmap to change the threading model. I'm mostly > just curious at the moment, but I guess it's a question that's going to crop > up more and more. > > Cheers, > Frase > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org >
