Re: CVS commit: src/share/mk

2014-01-24 Thread Izumi Tsutsui
 Sorry that I was lost, but I still believe that
 creating-directories-as-make-target is to avoid.

Do you have any suggestion without boring work to rename all
#include machine/foo.h lines and expand the machine path
in MI files to ${MACHINE}/include?

Creating symlinks for stand programs has been annoying for me
for 15 years, but see no alternative.

---
Izumi Tsutsui



Re: CVS commit: src/distrib/sets

2014-01-24 Thread Alan Barrett

On Fri, 24 Jan 2014, Masao Uebayashi wrote:

I agree that in an ideal reproducible world timestamp (== physical
time and its order) has no value.  But it is useful to detect
unnecessary rebuild - reproducible but built repeatedly 
unnecessarily.  I see some value in it.


You could extract that from METALOG.  If something is built 
twice, it will have two entries in METALOG, but only one entry in 
METALOG.sanitised.


Depending on the host platform, there might not be any timestamps 
in METALOG.  On a NetBSD host, I see time= keywords in METALOG, 
but on a Linux host, I do not see them.


--apb (Alan Barrett)


Re: CVS commit: src/share/mk

2014-01-24 Thread Matt Thomas

On Jan 24, 2014, at 1:16 AM, Izumi Tsutsui tsut...@ceres.dti.ne.jp wrote:

 Do you have any suggestion without boring work to rename all
 #include machine/foo.h lines and expand the machine path
 in MI files to ${MACHINE}/include?

I wish that cc/clang supported -isystem:NAME PATH and iquote:NAME PATH
which would look starting at PATH for includes that start with {,}NAME

If we had that, we wouldn't need the damn links at all. :)

-isystem:arm ${NETBSDSRCTOP}/sys/arch/arm/include



Re: CVS commit: src/share/mk

2014-01-24 Thread Warner Losh

On Jan 24, 2014, at 2:16 AM, Izumi Tsutsui wrote:

 Sorry that I was lost, but I still believe that
 creating-directories-as-make-target is to avoid.
 
 Do you have any suggestion without boring work to rename all
 #include machine/foo.h lines and expand the machine path
 in MI files to ${MACHINE}/include?
 
 Creating symlinks for stand programs has been annoying for me
 for 15 years, but see no alternative.

You could install them all into /usr/include/arch and make all the machine/*.h 
files look like:

#include arch/ # MACHINE # /foo.h

which could be generated from a master list or globular expansion of 
arch/*/include (just to get the list, any individual arch/machine would be free 
to not worry about them). OF course, this solved the installed problem, and not 
the symlinks created by config(1), but I'm guessing that's not quite so 
irritating...

Warner



Re: CVS commit: src/share/mk

2014-01-24 Thread Izumi Tsutsui
 You could install them all into /usr/include/arch and make all the 
 machine/*.h files look like:
 
 #include arch/ # MACHINE # /foo.h

As mrg@ wrote, kernel (and all standalone) builds
should not require installed DESTDIR.

If we install them into OBJDIR dynamically,
I guess we will get another race.

---
Izumi Tsutsui


Re: CVS commit: src/distrib/vax/ramdisk

2014-01-24 Thread Matt Thomas

On Jan 24, 2014, at 4:19 AM, Martin Husemann mar...@netbsd.org wrote:

 Module Name:  src
 Committed By: martin
 Date: Fri Jan 24 12:19:10 UTC 2014
 
 Modified Files:
   src/distrib/vax/ramdisk: Makefile
 
 Log Message:
 Slightly bump ramdisk size, so the content fits even when compiled with
 gcc 4.8 (usually binaries are smaller or same size, this increase is
 strange)

Alas, as gcc improves support for riscish architectures, these enhancements
make CISCish code much worse.

I tried an experiment and limited the plus:QI and plus:HI patterns to
const_int and register and the code produced improved considerably.

I'm reworking vax.md to limit patterns will help.


Re: CVS commit: src/share/mk

2014-01-24 Thread Greg Troxel

chris...@zoulas.com (Christos Zoulas) writes:

 On Jan 22,  7:29am, m...@3am-software.com (Matt Thomas) wrote:
 -- Subject: Re: CVS commit: src/share/mk

 | I always wondered why we don't use ln -sf=20
 | and avoid the race.

 That does not work because if the destnation is a directory it will
 try to link in the destination directory... (I tried). This is why
 I suggested that it needs to be done differently.

I feel like I'm completely missing something.  A make rule is triggered,
and it does something.  There are not in general requirements for make
rules to be atomic, and it seems like the problem is parallelism in make
which is improper, not the rule.


pgpT7UAvMAynh.pgp
Description: PGP signature


Re: CVS commit: src/sys/opencrypto

2014-01-24 Thread Christos Zoulas
In article 7458.1390534...@splode.eterna.com.au,
matthew green  m...@eterna.com.au wrote:

 Log Message:
 Implement in-module ref-counting, and do not allow auto-unload if there
 are existing references.
 
 Note that manual unloading is not prevented.
 
 OK christos@
 
 XXX Also note that there is still a small window where the ref-count can
 XXX be decremented, and then the process/thread preempted.  If auto-unload
 XXX happens before that thread can return from the module's code, bad
 XXX things (tm) could happen.

in this case, please simply disallow unload for this module always.
if the race is fixed, it can be enabled again.

I think that most module unloads suffer from this race, any ideas how to
fix it?

christos



Re: CVS commit: src/sys/opencrypto

2014-01-24 Thread Paul Goyette
Some modules get auto-loaded by the syscall mechanism.  We already have 
a mechanism to determine if any lwp's are currently executing these 
syscalls, and if yes we prevent auto-unload.


For device-driver modules, there is currently no equivalent mechanism. 
They get auto-loaded from code in specfs/vfsops as a result of trying to 
access the block/char device.  We would need to specify exactly what 
constitutes a reference and then keep track.  Is it only open that 
creates a reference?  Someone mentioned that it might be possible to 
read/write without opening the device...  Also need to consider what 
happens when the device is cloned ...


There is a refcount in the struct module, and a generic module_hold() / 
module_rele() mechanism to manipulate.  But it is currently used only 
for tracking inter-module references (ie, dependencies) as far as I can 
see.  It could be used here, but as currently implemented (linear search 
for module name) it's probably too expensive.


I'm not sure how we would handle miscellaneous modules...  :)




On Fri, 24 Jan 2014, Christos Zoulas wrote:


In article 7458.1390534...@splode.eterna.com.au,
matthew green  m...@eterna.com.au wrote:



Log Message:
Implement in-module ref-counting, and do not allow auto-unload if there
are existing references.

Note that manual unloading is not prevented.

OK christos@

XXX Also note that there is still a small window where the ref-count can
XXX be decremented, and then the process/thread preempted.  If auto-unload
XXX happens before that thread can return from the module's code, bad
XXX things (tm) could happen.


in this case, please simply disallow unload for this module always.
if the race is fixed, it can be enabled again.


I think that most module unloads suffer from this race, any ideas how to
fix it?

christos


!DSPAM:52e2a490250671524318036!




-
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:   |
| Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com|
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
| Kernel Developer |  | pgoyette at netbsd.org  |
-


Re: CVS commit: src/sys/opencrypto

2014-01-24 Thread Taylor R Campbell
   Date: Fri, 24 Jan 2014 17:35:41 + (UTC)
   From: chris...@astron.com (Christos Zoulas)

   In article 7458.1390534...@splode.eterna.com.au,
   matthew green  m...@eterna.com.au wrote:
   
Log Message:
XXX Also note that there is still a small window where the ref-count can
XXX be decremented, and then the process/thread preempted.  If auto-unload
XXX happens before that thread can return from the module's code, bad
XXX things (tm) could happen.
   
   in this case, please simply disallow unload for this module always.
   if the race is fixed, it can be enabled again.

   I think that most module unloads suffer from this race, any ideas how to
   fix it?

Shouldn't devsw_detach or config_fini_component handle this?  Why does
the crypto device need special reference counting that other devices
don't?


Re: CVS commit: src/sys/opencrypto

2014-01-24 Thread Paul Goyette

On Fri, 24 Jan 2014, Taylor R Campbell wrote:


Shouldn't devsw_detach or config_fini_component handle this?  Why does
the crypto device need special reference counting that other devices
don't?


The crypto device isn't special in this regard.  Pretty much all device 
driver modules need this sort of ref-counting.  Without it, the race 
exists and at some future time something will have reused/overwritten 
the memory previously occupied by the module, with non-deterministic 
results.


Crypto is special only in that it tried to do some clean-up during the 
detach.  The pool_destroy() code correctly noticed that there were still 
some outstanding allocations that had not been returned.  So we sort of 
got lucky here and found out about the problem immediately, rather 
than waiting for some non-deterministic future.



-
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:   |
| Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com|
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
| Kernel Developer |  | pgoyette at netbsd.org  |
-