Re: mutexes, locks and so on...

2010-11-14 Thread David Holland
On Fri, Nov 12, 2010 at 02:21:34PM +0100, Johnny Billquist wrote:
 > >  >  then I realized that this solution would break if people actually
 > >  >  wrote code like
 > >  >  lock(a)
 > >  >  lock(b)
 > >  >  release(a)
 > >  >  release(b)
 > >
 > >...which is very common.
 > 
 > It is? I would have thought (and hoped) that people normally did:
 > lock(a)
 > lock(b)
 > unlock(b)
 > unlock(a)

Nope. You might get away with this if we always did strict two-phase
locking in the kernel, but we don't (no kernel does) to avoid
excessive contention on e.g. the vnode for / and other such locks.
Meanwhile, lock coupling tends to appear anytime one is transitioning
through a data structure and wants to maintain consistency.

Thus the typical usage is something like

   lock(a)
   b = a->b
   lock(b)
   unlock(a)
   c = b->c
   lock(c)
   unlock(b)
   do_work(c)
   unlock(c)

It can be shown that this preserves conflict serializability as long
as nothing ever follows the structure in the opposite order (c -> b ->
a).

The traditional place you find code like this is in pathname
translation, but in a MP kernel it pops up in lots of other places
too.

 > I agree that it's not wrong, but untidy. Keeping track of ipl
 > levels could have been kept within the mutex instead, thus
 > simplifying both the lock and unlock code, at the expense that
 > people actually had to unlock mutexes in the reverse order they
 > acquired them. Just as with the splraise/splx before.

That however isn't workable.

(And it still wouldn't be workable even in a kernel that had separate
spinlocks and sleep-locks.)

-- 
David A. Holland
dholl...@netbsd.org


Re: Please do not yell at people for trying to help you.

2010-11-14 Thread David Holland
On Fri, Nov 12, 2010 at 08:31:39PM +, Eduardo Horvath wrote:
 > No it doesen't because all those macros assume the value is being 
 > transferred from one register to another rather than regiser to memory.  
 > The assignment:
 > 
 > foo.size = htole64(size);
 > 
 > Cannot be replaced with:
 > 
 > __inline __asm("stxa %1, [%0] ASI_LITLE" : &foo.size : size);

The right way to fix this is in the compiler; teach the compiler about
opposite-endian variables and let it pick the right instructions for
accessing them, and the problem goes away.

-- 
David A. Holland
dholl...@netbsd.org


Re: mutexes, locks and so on...

2010-11-14 Thread der Mouse
Just a note to avoid having incorrect information in the archives
without a correction.  I wrote:

> [%] It occurs to me, the VAX's BBSSI and BBCCI _are_ CAS, just
> restricted to a one-bit-wide operand (and with the data-to-swap-in
> specified by choice of instruction rather than an operand).

This is false.  CAS, as the term is normally used, turns out to be not
the compare-and-swap the acronym expands to but
compare-and-conditionally-swap.  However, BBSSI and BBCCI always
perform the write.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B


Re: mutexes, locks and so on...

2010-11-14 Thread David Holland
On Sat, Nov 13, 2010 at 01:45:40AM +0900, Izumi Tsutsui wrote:
 > > Wow. I guess you can add me to the list of people leaving.
 > 
 > There is no perfect world and we don't have enough resources.
 > 
 > Any help to keep support for ancient machines are appreciate, but
 > complaints like "we should support it" which prevents improvements
 > of mainstream will just make NetBSD rotten.

What "prevents improvements of mainstream" are we talking about here?
We have someone who wants to provide tuned vax-specific locking
primitives. The absolute worst possible cost to the "mainstream" that
this incurs is a bit of extra cpp and config hackery.

(Can we all please get a grip?)

-- 
David A. Holland
dholl...@netbsd.org


Re: CVS commit: src/sys/arch/powerpc/oea

2010-11-14 Thread Bernd Ernesti
On Mon, Nov 15, 2010 at 11:24:21AM +0900, Masao Uebayashi wrote:
> On Sun, Nov 14, 2010 at 05:52:51AM +, David Holland wrote:
> > On Sun, Nov 14, 2010 at 03:32:44AM +, Masao Uebayashi wrote:
> >  > XXX What is the conclusion about direct vs. indirect #include from 
> > headers?
> > 
> > Every header file should include the things it requires to compile.
> > Therefore, there should in principle be no cases where a header file
> > (or source file) needs to #include something it doesn't itself use.
> 
> This clarifies my long-unanswered question, thanks!
> 
> I've (re)built about 300 kernels in the last days.  I've found:
> 
> - sys/sysctl.h's struct kinfo_proc should be moved into sys/proc.h
>   (I've done this locally).  Otherwise all sysctl node providers
>   includes sys/proc.h and uvm/uvm_extern.h.
>   (This is where I started...)
> 
> - sys/syscallargs.h should be split into pieces, otherwise all its
>   users have to know unrelated types (sys/mount.h, sys/cpu.h).
> 
> - sys/proc.h's tsleep(), wakeup(), and friends should be moved into
>   some common header, because it's widely used API.  sys/proc.h will
>   be used only for "struct proc" related things.

What are the issues here that we need to fix this right now?

Can you please post changes and/or start a thread about all this
before you do them on tech-kern?

Bernd



Re: CVS commit: src/sys/arch/powerpc/oea

2010-11-14 Thread Adam Hamsik

On Nov,Monday 15 2010, at 7:16 AM, Bernd Ernesti wrote:

> On Mon, Nov 15, 2010 at 11:24:21AM +0900, Masao Uebayashi wrote:
>> On Sun, Nov 14, 2010 at 05:52:51AM +, David Holland wrote:
>>> On Sun, Nov 14, 2010 at 03:32:44AM +, Masao Uebayashi wrote:
 XXX What is the conclusion about direct vs. indirect #include from headers?
>>> 
>>> Every header file should include the things it requires to compile.
>>> Therefore, there should in principle be no cases where a header file
>>> (or source file) needs to #include something it doesn't itself use.
>> 
>> This clarifies my long-unanswered question, thanks!
>> 
>> I've (re)built about 300 kernels in the last days.  I've found:
>> 
>> - sys/sysctl.h's struct kinfo_proc should be moved into sys/proc.h
>>  (I've done this locally).  Otherwise all sysctl node providers
>>  includes sys/proc.h and uvm/uvm_extern.h.
>>  (This is where I started...)
>> 
>> - sys/syscallargs.h should be split into pieces, otherwise all its
>>  users have to know unrelated types (sys/mount.h, sys/cpu.h).
>> 
>> - sys/proc.h's tsleep(), wakeup(), and friends should be moved into
>>  some common header, because it's widely used API.  sys/proc.h will
>>  be used only for "struct proc" related things.
> 
> What are the issues here that we need to fix this right now?

I think that it's quite good time to fix, it would be much harder to do this 
after 6.0 branch.

Regards

Adam.