-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> If the thinking is that: > >> asynchronous = pub/sub = efficiency > >> Then, an important consideration is: How does qpid implements *durable* >> subscription to topics (pub/sub)? The durable nature of the >> subscription usually requires that brokers implement using queues where >> messages are routed to subscriber queues based on topic/subject and >> selectors. Asynchronous request/response assumes guaranteed delivery. >> This means that the message must be queued so it can be delivered to >> consumers that are not connected when the message is published. This is >> the definition of durable subscription. So, publishing a message to 10k >> agents probably still requires 10k queues. > >> But, if we anticipate this kind of mass operation, then asynchronous >> will be much more efficient and worth the extra complexity because we >> can have all the agents performing the operation in parallel. If we did >> this synchronously, we'd be limited to the Task thread limit.
+1 to this whole train of thought. That brings me to the question of throttling (yes, I'm playing devil's advocate against myself). If we use sync RMI over the async task queue, then we can only send out X many package install invocations at a time, where X is the number of threads in the queue. If we have 10K consumers and 50 threads, it's gonna be rough. So if we use an async model to send the package install requests to the consumers, then we don't block on the size of the task queue. However, then we have a situation where 10K consumers suddenly smash our repos at the same time asking for package bits. I like how Jeff put it. The sync RMI approach gives us a form of throttling. It's just in the wrong place. What we'll need is something in place to throttle and/or load balance the requests on the repos themselves. I'm sure that's on a backlog somewhere, just throwing it out there now since it's relevant to the discussion. - -- Jason Dobies RHCE# 805008743336126 Freenode: jdob -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMNhufAAoJEOMmcTqOSQHCy4sIAIden/rf0Cgn1C5sYm0ydgh7 ZwXdqIdny+cxWKXmDVfN7KptLZ6udduyVdTNxxGOqrhUqZWpzYeTdAlpjTUafTdW /m6vPgdSzpa61mncfzEoqHkljnU4k1PMWQPGwT/4d+9KAR05o1eHO8UDUdJdNuoj eQqzeRB0pHxNJJ9ZPlxwILUf9vSKuegrP7QUHY+5mIcHZFY9pozp4IMgVe75ov0n Ky/kaSAg4jhyw5gtHTRYDYmOR7ROTW/GQ0xRMTBl5Fs0C++3TEsa9FerSRLDMVHK Dt9bXpLLtXLclHmvsry85ojVxA+rtpMdHhH8HJF3NlX0b45mH7jWBp73sZQ87O4= =+C+o -----END PGP SIGNATURE----- _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
