On Tue, Nov 27, 2007 at 10:21:12AM +0100, Dmitry Adamushko wrote: > On 26/11/2007, Micah Dowty <[EMAIL PROTECTED]> wrote: > > > > The application doesn't really depend on the load-balancer's decisions > > per se, it just happens that this behaviour I'm seeing on NUMA systems > > is extremely bad for performance. > > > > In this context, the application is a virtual machine runtime which is > > executing either an SMP VM or it's executing a guest which has a > > virtual GPU. In either case, there are at least three threads: > > > > - Two virtual CPU/GPU threads, which are nice(0) and often CPU-bound > > - A low-latency event handling thread, at nice(-10) > > > > The event handling thread performs periodic tasks like delivering > > timer interrupts and completing I/O operations. > > Are I/O operations initiated by these "virtual CPU/GPU threads"? > > If so, would it make sense to have per-CPU event handling threads > (instead of one global)? They would handle I/O operations initiated > from their respective CPUs to (hopefully) achieve better data locality > (esp. if the most part of the involved data is per-CPU). > > Then let the load balancer to evenly distribute the "virtual CPU/GPU > threads" or even (at least, as an experiment) fix them to different > CPUs as well? > > sure, the scenario is highly dependent on a nature of those > 'events'... and I can just speculate here :-) (but I'd imagine > situations when such a scenario would scale better).
We already do this when it makes sense to. The high priority "event" thread is mostly used for asynchronous events- timers (driven by /dev/rtc), completion of disk or USB I/O, etc. These are things that happen asynchronously to the virtual CPU, and which may or may not need to interrupt the virtual CPU at some point. Another good example is audio. We have another thread (which ideally is high-priority) which processes audio events asynchronously from the virtual CPUs. The CPUs manipulate some shared memory to queue up audio data, but while the audio is playing there is no direct control path between the two threads. The virtual CPU reads/writes audio registers whenever it needs to emulate accesses to the audio device, and the audio thread wakes up any time the hardware needs more samples to play. The two threads are asynchronous with very little shared synchronization. Hopefully that clarifies our situation a bit. Thanks again, --Micah - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
