On 28 August 2013 23:35, Matthew Dillon <[email protected]> wrote:
> :Hi there, > : > :I've been studying dragonfly for a while now and this is something I > :could never figure out, I'm pretty sure I'm missing something and I'd > :appreciate if someone could give me some pointers. > : > :Suppose the following cenario. > : > :Cenario A [A] > :cpu0: on whichever context > :cpu0: spin_lock(&a) (now in a critical section) > :cpu0: wakeup(ident1) (ident1 has a sleeper on cpu1) > :cpu0: lwkt_send_ipiq3(cpu1, wakeup(ident1)) > > Illegal. You should not call wakeup() with a spin lock held. > You most definitely should not be calling any IPIQ functions with > a spin lock held. > > You have two solutions here. First, use a regular lock or a token > instead of a spin lock. Second, flag the condition and issue the > wakeup() or IPIQ function after releasing the spin lock. > > In DragonFly, spin locks are only intended to be used for very > short sequences of self-contained or mostly self-contained code. > > :But ipiq for cpu1 is full, so I enable interrupts, although I'm still on > :a critical section, process my own ipiq and spin until cpu1 queue is not > :full, but at this point I'm still holding spin_lock a. > > Critical sections do not disable interrupts, they simply prevent normal > preemptive processing (preemption by an interrupt thread or IPI) > from occuring. Interrupts will still occur, but they will simply flag > the pending condition in mycpu->gd_reqflags, EOI the interrupt, and > immediately return. The interrupt is then processed normally when the > critical section is exited. > > However, if you issue an IPIQ operation inside a critical section which > would otherwise be forced to spin-wait due to a full IPIQ queue, it > will synchronously execute any incoming IPIs while it is spin-waiting > to avoid deadlocking the system. > > :Both realize the other is full and process their own queues, but they > break the > :atomicity required by the two calls to crit_enter() earlier > : > :What am I missing ? (Sorry for the noise). > : > :Thanks > > Programs which enter critical sections can always assume that there > will > be no asynchronous atomicy breakage, but there are no guarantees > regarding > synchronous atomicy. For example, you can tsleep() just fine while > holding a critical section. Nominal IPIQ operations fall under the > same > rules. Incoming IPIs will not be executed asynchronously but they can > still be executed synchronously if your outgoing IPI operation would > otherwise be forced to spin-wait. > > Thanks for the detailed reply. I assumed synchronous atomicity had to be kept across a critical section, I had considered the tsleep case but thought it was special, since you're sleeping, it's already implied you accept losing atomicity.
