-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Robert,
On 11/03/2010 08:55 PM, Robert Fleischman wrote: > Question regarding parallelism in libunbound: > > Using "async" requests, multiple threads can request async resolutions > to libunbound. > It appears these threads do a "tube_write()" to a SINGLE background > thread insides libunbound.a > > This is concerning! What if 10 threads are all asking for something > that is in the cache? This thread is a full resolver and can handle a lot of queries in its requestlist (mostly it will be waiting for a network event, so they should not quickly interfere with each other). To enhance its capacity, you can read a config file (or use the set_option call) and basically do what the documentation howto optimise says. You can set it to handle, say 4000 queries, in its requestlist, and permit it to have lots of sockets (which your app must be able to have, eg. ulimit). By default libunbound uses pretty few sockets since we do not know what the app can sustain (and it probably just wants to validate a couple queries). For high traffic, you can increase it. > How can a user of libunbound get more parallelism during resolution? I think you can set num-threads and get multiple libunbound threads in the background, but the tube_write is always done to thread0 this does not help you as those other threads are not told to do work. If you perform a *blocking* resolve from several threads at a time, then, temporarily, that thread joins the unbound-server-work-thread-set, and works on its own query. At this point you (temporarily until the query resolves) are running a multithreaded resolver. The unbound-ctx contains the shared cache. > --- > > If the non-async methods are used, the requesting threads block, > reverting to thread-per-request parallelism! That can be even worse. > You'd have to have a thread-per-outstanding request. Even more > concerning! > > The "unbound" server doesn't seem to have this problem. (num-threads > parameter is ignored by libunbound.a users). You could have two or three unbound-contexts. And choose a random one to put an async query to. They do not share their cache, but then you easily have multiple resolution threads. (if you choose the ctx based on the hash of the query then you always ask the same context the same query and caching happens). Your callback thread can use the poll_fd that libunbound provides to see which context has answers that have become ready and execute those callbacks. Best regards, Wouter -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iEYEARECAAYFAkzhAnUACgkQkDLqNwOhpPi2hgCfckE8wPwWPjGsgjzARy/Kawiz cSUAoIt1NucXGmZtqyTheXEEXOF/gGs9 =Xoyd -----END PGP SIGNATURE----- _______________________________________________ Unbound-users mailing list [email protected] http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users
