Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-07-03 Thread Manuel Bouyer
On Sun, Jul 03, 2011 at 08:31:00PM +0300, Jukka Ruohonen wrote:
> On Sun, Jul 03, 2011 at 07:27:00PM +0200, Manuel Bouyer wrote:
> > it's not only about Xen, it's about all kernels for any port which
> > already have DIAGNOSTIC and want to keep it even for release
> > (e.g. i386 ALL).
> 
> As far as I understand, i386/ALL is just for testing the compilation of
> various options and drivers. I doubt whether it even boots.

Exactly. My point is that adding DIAGNOSTIC in some central place will
cause it either to fail to build, or to not have DIAGNOSTIC in the
release branch when it's removed from the central place.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-07-03 Thread Jukka Ruohonen
On Sun, Jul 03, 2011 at 07:27:00PM +0200, Manuel Bouyer wrote:
> it's not only about Xen, it's about all kernels for any port which
> already have DIAGNOSTIC and want to keep it even for release
> (e.g. i386 ALL).

As far as I understand, i386/ALL is just for testing the compilation of
various options and drivers. I doubt whether it even boots.

- Jukka.


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-07-03 Thread Manuel Bouyer
On Sun, Jul 03, 2011 at 06:23:49PM +0100, Mindaugas Rasiukevicius wrote:
> Manuel Bouyer  wrote:
> > On Sun, Jul 03, 2011 at 04:17:02PM +0100, Mindaugas Rasiukevicius wrote:
> > > Manuel Bouyer  wrote:
> > > > > 
> > > > > - Obviously, defined policy/responsibility to disable these options
> > > > > for release kernels.  In fact, if we go this way - then options
> > > > > should be removed from all MD kernel configs and managed in MI
> > > > > src/sys/conf/std.
> > > > 
> > > > I'm not sure this is doable: some ports may want to keep DIAGNOSTIC in
> > > > release branches, while others may want to exclude DIAGNOSTIC from
> > > > some kernels on HEAD (for example because of space constraints).
> > > 
> > > Why not?  Such ports can still define the option in their configs.
> > > Also,
> > 
> > No because you'll then have "already have options `DIAGNOSTIC'" from
> > config.
> 
> Yes, that would be an extra headache for release branch (perhaps one
> could make it easier with some config magic).  However, expectation
> of release branch is a stable kernel.  Why do you want to distribute
> Xen kernel with inconsistent options (vs x86) and against policy?

it's not only about Xen, it's about all kernels for any port which
already have DIAGNOSTIC and want to keep it even for release
(e.g. i386 ALL).

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-07-03 Thread Mindaugas Rasiukevicius
Manuel Bouyer  wrote:
> On Sun, Jul 03, 2011 at 04:17:02PM +0100, Mindaugas Rasiukevicius wrote:
> > Manuel Bouyer  wrote:
> > > > 
> > > > - Obviously, defined policy/responsibility to disable these options
> > > > for release kernels.  In fact, if we go this way - then options
> > > > should be removed from all MD kernel configs and managed in MI
> > > > src/sys/conf/std.
> > > 
> > > I'm not sure this is doable: some ports may want to keep DIAGNOSTIC in
> > > release branches, while others may want to exclude DIAGNOSTIC from
> > > some kernels on HEAD (for example because of space constraints).
> > 
> > Why not?  Such ports can still define the option in their configs.
> > Also,
> 
> No because you'll then have "already have options `DIAGNOSTIC'" from
> config.

Yes, that would be an extra headache for release branch (perhaps one
could make it easier with some config magic).  However, expectation
of release branch is a stable kernel.  Why do you want to distribute
Xen kernel with inconsistent options (vs x86) and against policy?

-- 
Mindaugas


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-07-03 Thread Manuel Bouyer
On Sun, Jul 03, 2011 at 04:17:02PM +0100, Mindaugas Rasiukevicius wrote:
> Manuel Bouyer  wrote:
> > > 
> > > - Obviously, defined policy/responsibility to disable these options for
> > >   release kernels.  In fact, if we go this way - then options should be
> > >   removed from all MD kernel configs and managed in MI src/sys/conf/std.
> > 
> > I'm not sure this is doable: some ports may want to keep DIAGNOSTIC in
> > release branches, while others may want to exclude DIAGNOSTIC from some
> > kernels on HEAD (for example because of space constraints).
> 
> Why not?  Such ports can still define the option in their configs.  Also,

No because you'll then have "already have options `DIAGNOSTIC'" from config.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-07-03 Thread Mindaugas Rasiukevicius
Manuel Bouyer  wrote:
> > 
> > - Obviously, defined policy/responsibility to disable these options for
> >   release kernels.  In fact, if we go this way - then options should be
> >   removed from all MD kernel configs and managed in MI src/sys/conf/std.
> 
> I'm not sure this is doable: some ports may want to keep DIAGNOSTIC in
> release branches, while others may want to exclude DIAGNOSTIC from some
> kernels on HEAD (for example because of space constraints).

Why not?  Such ports can still define the option in their configs.  Also,
some slow ports like VAX might want to keep debugging options disabled even
in -current, e.g. 'no options DEBUG' would do.

However, I do not think that ports should arbitrary enable/disable these
options for release/current branches, unless there is a very good reason.
IMO, Xen kernel should not have DIAGNOSTIC for release branch (it is your
call, of course), especially when user is no warned about that.

Let's be consistent in what we distribute.

-- 
Mindaugas


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-22 Thread Manuel Bouyer
On Wed, Jun 22, 2011 at 02:06:34AM +, David Holland wrote:
> On Thu, Jun 16, 2011 at 07:49:43PM +0200, Manuel Bouyer wrote:
>  > > I'm not in favor of LOCKDEBUG by default, for reasons already stated 
> here.
>  > 
>  > Also, could LOCKDEBUG have ABI issues with modules ?
>  > I've only ever used LOCKDEBUG with non-MODULAR kernels ...
> 
> The whole reason LOCKDEBUG is abysmally slow is that it does crazy
> things to preserve the same ABI. Personally I think this is a poor
> tradeoff...

But if LOCKDEBUG changes the ABI, then you need a different set of modules.
not that I care much about modules (and especially having them independant
from the kernel) ...

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-21 Thread matthew green


FWIW, i much prefer using DDB_ONPANIC=2 than setting the enter command
to "bt".  unfortunately only options(4) meantions this, not sysctl(4)
or ddb(4).


.mrg.


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-21 Thread David Holland
On Thu, Jun 16, 2011 at 07:49:43PM +0200, Manuel Bouyer wrote:
 > > I'm not in favor of LOCKDEBUG by default, for reasons already stated here.
 > 
 > Also, could LOCKDEBUG have ABI issues with modules ?
 > I've only ever used LOCKDEBUG with non-MODULAR kernels ...

The whole reason LOCKDEBUG is abysmally slow is that it does crazy
things to preserve the same ABI. Personally I think this is a poor
tradeoff...

-- 
David A. Holland
dholl...@netbsd.org


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-21 Thread David Young
On Fri, Jun 17, 2011 at 12:34:53PM +0200, Manuel Bouyer wrote:
> On Fri, Jun 17, 2011 at 12:24:13PM +0200, Adam Hamsik wrote:
> > 
> > On Jun,Thursday 16 2011, at 5:30 PM, Mindaugas Rasiukevicius wrote:
> > > - DDB_ONPANIC=1 and DDB_COMMANDONENTER="bt;show regsisters" and perhaps
> > >  also "call ddb_vgapost" in the beginning (not sure if there are any
> > >  potential side effects?).  Otherwise, not getting information from DDB
> > >  is just counter-productive, plus we get not very useful reports, when
> > >  backtrace is missing.
> > 
> > I like this we should definitely set DDB_COMMANDONENTER to something like 
> > above.
> 
> be carefull though; this can easily output more than the 25 lines of a VGA
> display; and so usefull information (like the panic message) may have
> dissapeared. 

It can also lead to an infinite loop where a fault while ddb examines
the stack for 'bt' leads ddb to execute 'bt;show registers' again which
leads to another fault.  Just another reason to be careful.

Dave

-- 
David Young OJC Technologies
dyo...@ojctech.com  Urbana, IL * (217) 344-0444 x24


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-17 Thread Manuel Bouyer
On Fri, Jun 17, 2011 at 12:24:13PM +0200, Adam Hamsik wrote:
> 
> On Jun,Thursday 16 2011, at 5:30 PM, Mindaugas Rasiukevicius wrote:
> 
> > Mindaugas Rasiukevicius  wrote:
> >> I have few concerns:
> >> 
> >> - If we enable DIAGNOSTIC, then we should also enable DEBUG, as it also
> >>  covers many relevant diagnostic checks.
> >> 
> >> - Alternatively, it should be clearly defined what goes under DEBUG,
> >>  i.e. what is considered a "heavier check".  I think code diverged in
> >>  a way that the difference between DEBUG and DIAGNOSTIC is small.
> >> 
> >> - Since performance is degraded and -current users concerned about it
> >>  will need to compile their own kernels anyway - I believe LOCKDEBUG
> >>  should be enabled as well.  Perhaps LOCKDEBUG should become a part
> >>  of DEBUG - it is at least clearly a "heavier check". :)
> >> 
> >> - There MUST be a very clear indication to users - a warning in a visible
> >>  place that the kernel has diagnostic options enabled, and performance
> >>  is significantly degraded.
> >> 
> >> - Obviously, defined policy/responsibility to disable these options for
> >>  release kernels.  In fact, if we go this way - then options should be
> >>  removed from all MD kernel configs and managed in MI src/sys/conf/std.
> > 
> > - DDB_ONPANIC=1 and DDB_COMMANDONENTER="bt;show regsisters" and perhaps
> >  also "call ddb_vgapost" in the beginning (not sure if there are any
> >  potential side effects?).  Otherwise, not getting information from DDB
> >  is just counter-productive, plus we get not very useful reports, when
> >  backtrace is missing.
> 
> I like this we should definitely set DDB_COMMANDONENTER to something like 
> above.

be carefull though; this can easily output more than the 25 lines of a VGA
display; and so usefull information (like the panic message) may have
dissapeared. 

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-17 Thread Adam Hamsik

On Jun,Thursday 16 2011, at 5:30 PM, Mindaugas Rasiukevicius wrote:

> Mindaugas Rasiukevicius  wrote:
>> I have few concerns:
>> 
>> - If we enable DIAGNOSTIC, then we should also enable DEBUG, as it also
>>  covers many relevant diagnostic checks.
>> 
>> - Alternatively, it should be clearly defined what goes under DEBUG,
>>  i.e. what is considered a "heavier check".  I think code diverged in
>>  a way that the difference between DEBUG and DIAGNOSTIC is small.
>> 
>> - Since performance is degraded and -current users concerned about it
>>  will need to compile their own kernels anyway - I believe LOCKDEBUG
>>  should be enabled as well.  Perhaps LOCKDEBUG should become a part
>>  of DEBUG - it is at least clearly a "heavier check". :)
>> 
>> - There MUST be a very clear indication to users - a warning in a visible
>>  place that the kernel has diagnostic options enabled, and performance
>>  is significantly degraded.
>> 
>> - Obviously, defined policy/responsibility to disable these options for
>>  release kernels.  In fact, if we go this way - then options should be
>>  removed from all MD kernel configs and managed in MI src/sys/conf/std.
> 
> - DDB_ONPANIC=1 and DDB_COMMANDONENTER="bt;show regsisters" and perhaps
>  also "call ddb_vgapost" in the beginning (not sure if there are any
>  potential side effects?).  Otherwise, not getting information from DDB
>  is just counter-productive, plus we get not very useful reports, when
>  backtrace is missing.

I like this we should definitely set DDB_COMMANDONENTER to something like above.


Regards

Adam.



Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Steven Bellovin

On Jun 16, 2011, at 3:29 36AM, Manuel Bouyer wrote:

> Hello,
> for the second time (at last) a kernel issue raised the question of
> adding back 'options DIAGNOSTIC' to GENERIC/INSTALL kernels in HEAD.
> Several people agreed tha this would be a good thing.
> 

Good idea.  (I'm reminded of a comment, I think by Hoare,
that I saw many years ago; unfortunately, I was not able to
find it recently when I looked.  Briefly, he likened using
array bounds checking during debugging but turning it off
in production code to sailors practicing on shore wearing
life preservers but leaving them home when they went to sea.)


--Steve Bellovin, https://www.cs.columbia.edu/~smb







Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Martin Husemann
On Fri, Jun 17, 2011 at 12:42:34AM +0400, Aleksej Saushev wrote:
> I'd say that at least "call ddb_vgapost" should enter default sysctl.conf,
> at least commented out. It isn't trivial to learn that the functionality
> exists, and it is the only way to get to debugger on modern machines
> without serial interface.

What is the list of archs that have ddb_vgapost?
Should sysinst nm /netbsd and grep for it? Or just enter it and ignore the
ddb error when the symbol is not found?

Martin




Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Aleksej Saushev
Mindaugas Rasiukevicius  writes:

> - DDB_ONPANIC=1 and DDB_COMMANDONENTER="bt;show regsisters" and perhaps
>   also "call ddb_vgapost" in the beginning (not sure if there are any
>   potential side effects?).  Otherwise, not getting information from DDB
>   is just counter-productive, plus we get not very useful reports, when
>   backtrace is missing.

I'd say that at least "call ddb_vgapost" should enter default sysctl.conf,
at least commented out. It isn't trivial to learn that the functionality
exists, and it is the only way to get to debugger on modern machines
without serial interface.


-- 
HE CE3OH...



Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Manuel Bouyer
On Thu, Jun 16, 2011 at 05:50:56PM +0200, Manuel Bouyer wrote:
> > 
> > - Alternatively, it should be clearly defined what goes under DEBUG,
> >   i.e. what is considered a "heavier check".  I think code diverged in
> >   a way that the difference between DEBUG and DIAGNOSTIC is small.
> > 
> > - Since performance is degraded and -current users concerned about it
> >   will need to compile their own kernels anyway - I believe LOCKDEBUG
> >   should be enabled as well.  Perhaps LOCKDEBUG should become a part
> >   of DEBUG - it is at least clearly a "heavier check". :)
> 
> I'm not in favor of LOCKDEBUG by default, for reasons already stated here.

Also, could LOCKDEBUG have ABI issues with modules ?
I've only ever used LOCKDEBUG with non-MODULAR kernels ...

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Manuel Bouyer
On Thu, Jun 16, 2011 at 04:30:18PM +0100, Mindaugas Rasiukevicius wrote:
> Mindaugas Rasiukevicius  wrote:
> > I have few concerns:
> > 
> > - If we enable DIAGNOSTIC, then we should also enable DEBUG, as it also
> >   covers many relevant diagnostic checks.
> > 
> > - Alternatively, it should be clearly defined what goes under DEBUG,
> >   i.e. what is considered a "heavier check".  I think code diverged in
> >   a way that the difference between DEBUG and DIAGNOSTIC is small.
> > 
> > - Since performance is degraded and -current users concerned about it
> >   will need to compile their own kernels anyway - I believe LOCKDEBUG
> >   should be enabled as well.  Perhaps LOCKDEBUG should become a part
> >   of DEBUG - it is at least clearly a "heavier check". :)
> > 
> > - There MUST be a very clear indication to users - a warning in a visible
> >   place that the kernel has diagnostic options enabled, and performance
> >   is significantly degraded.
> > 
> > - Obviously, defined policy/responsibility to disable these options for
> >   release kernels.  In fact, if we go this way - then options should be
> >   removed from all MD kernel configs and managed in MI src/sys/conf/std.
> 
> - DDB_ONPANIC=1 and DDB_COMMANDONENTER="bt;show regsisters" and perhaps
>   also "call ddb_vgapost" in the beginning (not sure if there are any
>   potential side effects?).  Otherwise, not getting information from DDB
>   is just counter-productive, plus we get not very useful reports, when
>   backtrace is missing.

I also agree.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Manuel Bouyer
On Thu, Jun 16, 2011 at 04:20:06PM +0100, Mindaugas Rasiukevicius wrote:
> I have few concerns:
> 
> - If we enable DIAGNOSTIC, then we should also enable DEBUG, as it also
>   covers many relevant diagnostic checks.

Historically, DIAGNOSTIC was enabled and DEBUG was not. Usually (but maybe
my view is wrong) DIAGNOSTIC enables debug/consistency checks while
DEBUG enables debug printfs. DEBUG can make the kernel way more noisy,
and is not necesserely usefull to detect a problem (which is what
DIAGNOSTIC is about).

KASSERT() & friend is enabled by DIAGNOSTIC, this is mostly what I'm
concerned about.

> 
> - Alternatively, it should be clearly defined what goes under DEBUG,
>   i.e. what is considered a "heavier check".  I think code diverged in
>   a way that the difference between DEBUG and DIAGNOSTIC is small.
> 
> - Since performance is degraded and -current users concerned about it
>   will need to compile their own kernels anyway - I believe LOCKDEBUG
>   should be enabled as well.  Perhaps LOCKDEBUG should become a part
>   of DEBUG - it is at least clearly a "heavier check". :)

I'm not in favor of LOCKDEBUG by default, for reasons already stated here.

> 
> - There MUST be a very clear indication to users - a warning in a visible
>   place that the kernel has diagnostic options enabled, and performance
>   is significantly degraded.

we already have a /etc/motd which contains warnings on HEAD. this could be
added here.


> 
> - Obviously, defined policy/responsibility to disable these options for
>   release kernels.  In fact, if we go this way - then options should be
>   removed from all MD kernel configs and managed in MI src/sys/conf/std.

I'm not sure this is doable: some ports may want to keep DIAGNOSTIC in release
branches, while others may want to exclude DIAGNOSTIC from some kernels
on HEAD (for example because of space constraints).

> 
> > I know that DIAGNOSTIC was commented out so that someone could install
> > a HEAD snapshot and run benchmarks out of the box, but as a side effect
> > a lot of but are left hidden and only show up when someone tries
> > to run a Xen kernel (which still have DIAGNOSTIC). See kern/45051 for
> > another one.
> 
> Many developers do use these options (e.g. I always enable all options
> when developing something), but some bugs just occur rarely.  For example,
> at least few developers were running diagnostic kernels, but did not get
> the assert reported by drochner@ (also, many developers simply do not
> upgrade their kernels that often).  PR/45051 is also a rare case - I have
> added that assert in pool(9) subsystem a year ago, exactly for a reason to
> get these kind of reports.  Surely many had run diagnostic kernels in a
> year time, but it might need specific workload to trigger.

Actually lots of DIAGNOSTIC checks firing on Xen, also fire on
GENERIC when built with DIAGNOSTIC ...
So I guess not so many peoples runs DIAGNOSTIC kernels ...

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Izumi Tsutsui
rmind@ wrote:

>   Otherwise, not getting information from DDB
>   is just counter-productive, plus we get not very useful reports, when
>   backtrace is missing.

FYI, OpenBSD puts the following message on entering ddb:
---
db_printf("RUN AT LEAST 'trace' AND 'ps' AND INCLUDE "
"OUTPUT WHEN REPORTING THIS PANIC!\n");
#ifdef MULTIPROCESSOR
db_printf("IF RUNNING SMP, USE 'mach ddbcpu <#>' AND "
"'trace' ON OTHER PROCESSORS, TOO.\n");
#endif
db_printf("DO NOT EVEN BOTHER REPORTING THIS WITHOUT "
"INCLUDING THAT INFORMATION!\n");
---
Izumi Tsutsui


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Martin Husemann
On Thu, Jun 16, 2011 at 04:30:18PM +0100, Mindaugas Rasiukevicius wrote:
> > - Since performance is degraded and -current users concerned about it
> >   will need to compile their own kernels anyway - I believe LOCKDEBUG
> >   should be enabled as well.  Perhaps LOCKDEBUG should become a part
> >   of DEBUG - it is at least clearly a "heavier check". :)

Well, a DEBUG kernel is usable (and, IIRC a DIAGNOSTIC+LOCKDEBUG kernel
as well), but a DEBUG+LOCKDEBUG kernel is absolutely unusable for anything
besides debugging.

A simple solution (but with a gotcha): have releng builds both produce
GENERIC and GENERIC-DEBUG (or whatever you call it), then run automatic
test runs on both and compare.

Encourage users to run the -DEBUG version on test machines, but point to
GENERIC for performance critical things.


Martin


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Mindaugas Rasiukevicius
Mindaugas Rasiukevicius  wrote:
> I have few concerns:
> 
> - If we enable DIAGNOSTIC, then we should also enable DEBUG, as it also
>   covers many relevant diagnostic checks.
> 
> - Alternatively, it should be clearly defined what goes under DEBUG,
>   i.e. what is considered a "heavier check".  I think code diverged in
>   a way that the difference between DEBUG and DIAGNOSTIC is small.
> 
> - Since performance is degraded and -current users concerned about it
>   will need to compile their own kernels anyway - I believe LOCKDEBUG
>   should be enabled as well.  Perhaps LOCKDEBUG should become a part
>   of DEBUG - it is at least clearly a "heavier check". :)
> 
> - There MUST be a very clear indication to users - a warning in a visible
>   place that the kernel has diagnostic options enabled, and performance
>   is significantly degraded.
> 
> - Obviously, defined policy/responsibility to disable these options for
>   release kernels.  In fact, if we go this way - then options should be
>   removed from all MD kernel configs and managed in MI src/sys/conf/std.

- DDB_ONPANIC=1 and DDB_COMMANDONENTER="bt;show regsisters" and perhaps
  also "call ddb_vgapost" in the beginning (not sure if there are any
  potential side effects?).  Otherwise, not getting information from DDB
  is just counter-productive, plus we get not very useful reports, when
  backtrace is missing.

-- 
Mindaugas


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Mindaugas Rasiukevicius
Manuel Bouyer  wrote:
> for the second time (at last) a kernel issue raised the question of
> adding back 'options DIAGNOSTIC' to GENERIC/INSTALL kernels in HEAD.
> Several people agreed tha this would be a good thing.
> 
> So here's the formal question: would someone object if I add back
> 'options DIAGNOSTIC' to i386 and amd64 GENERIC and INSTALL kernels,
> with a comment saying this should be disabled on release branch
> (it would be up to releng to comment it out as part of the release
> process) ?

I have few concerns:

- If we enable DIAGNOSTIC, then we should also enable DEBUG, as it also
  covers many relevant diagnostic checks.

- Alternatively, it should be clearly defined what goes under DEBUG,
  i.e. what is considered a "heavier check".  I think code diverged in
  a way that the difference between DEBUG and DIAGNOSTIC is small.

- Since performance is degraded and -current users concerned about it
  will need to compile their own kernels anyway - I believe LOCKDEBUG
  should be enabled as well.  Perhaps LOCKDEBUG should become a part
  of DEBUG - it is at least clearly a "heavier check". :)

- There MUST be a very clear indication to users - a warning in a visible
  place that the kernel has diagnostic options enabled, and performance
  is significantly degraded.

- Obviously, defined policy/responsibility to disable these options for
  release kernels.  In fact, if we go this way - then options should be
  removed from all MD kernel configs and managed in MI src/sys/conf/std.

> I know that DIAGNOSTIC was commented out so that someone could install
> a HEAD snapshot and run benchmarks out of the box, but as a side effect
> a lot of but are left hidden and only show up when someone tries
> to run a Xen kernel (which still have DIAGNOSTIC). See kern/45051 for
> another one.

Many developers do use these options (e.g. I always enable all options
when developing something), but some bugs just occur rarely.  For example,
at least few developers were running diagnostic kernels, but did not get
the assert reported by drochner@ (also, many developers simply do not
upgrade their kernels that often).  PR/45051 is also a rare case - I have
added that assert in pool(9) subsystem a year ago, exactly for a reason to
get these kind of reports.  Surely many had run diagnostic kernels in a
year time, but it might need specific workload to trigger.


P.S. PR/45051 problem is that bus_dma(9) uses pmap_enter(9), and it can
occasionally happen from interrupt context.  It is kernel-only mapping
i.e. on pmap_kernel().  Is there any reason (apart from historical) why
pmap_kenter_pa(9) is not used?

-- 
Mindaugas


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Manuel Bouyer
On Thu, Jun 16, 2011 at 03:20:42PM +0100, Julio Merino wrote:
> On 6/16/11 9:54 AM, Martin Husemann wrote:
> > On Thu, Jun 16, 2011 at 09:29:36AM +0200, Manuel Bouyer wrote:
> >> So here's the formal question: would someone object if I add back
> >> 'options DIAGNOSTIC' to i386 and amd64 GENERIC and INSTALL kernels,
> >> with a comment saying this should be disabled on release branch
> >> (it would be up to releng to comment it out as part of the release 
> >> process) ?
> > 
> > I am in favour of this proposal (and would add sparc64)
> 
> Me too.
> 
> > - but in case we
> > do not agree, we should at least make sure that all regular automatic test
> > runs are using kernels with DIAGNOSTIC enabled.
> 
> There is still the possibility that DIAGNOSTIC might hide real bugs in
> the automated tests, although unlikely.

I think in HEAD we have more bugs exposed by DIAGNOSTIC than hidden by it.

> 
> We'd get around this by _also_ running continuous tests on release
> branches.  This way, we'd have coverage for both diagnostic and
> non-diagnostic kernels where the options make most sense.

Sure. I'm not sure about the state of atf in netbsd-5, but it's definitvely
something we want for netbsd-6.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Julio Merino
On 6/16/11 9:54 AM, Martin Husemann wrote:
> On Thu, Jun 16, 2011 at 09:29:36AM +0200, Manuel Bouyer wrote:
>> So here's the formal question: would someone object if I add back
>> 'options DIAGNOSTIC' to i386 and amd64 GENERIC and INSTALL kernels,
>> with a comment saying this should be disabled on release branch
>> (it would be up to releng to comment it out as part of the release process) ?
> 
> I am in favour of this proposal (and would add sparc64)

Me too.

> - but in case we
> do not agree, we should at least make sure that all regular automatic test
> runs are using kernels with DIAGNOSTIC enabled.

There is still the possibility that DIAGNOSTIC might hide real bugs in
the automated tests, although unlikely.

We'd get around this by _also_ running continuous tests on release
branches.  This way, we'd have coverage for both diagnostic and
non-diagnostic kernels where the options make most sense.

-- 
Julio Merino / @jmmv


Re: add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Martin Husemann
On Thu, Jun 16, 2011 at 09:29:36AM +0200, Manuel Bouyer wrote:
> So here's the formal question: would someone object if I add back
> 'options DIAGNOSTIC' to i386 and amd64 GENERIC and INSTALL kernels,
> with a comment saying this should be disabled on release branch
> (it would be up to releng to comment it out as part of the release process) ?

I am in favour of this proposal (and would add sparc64) - but in case we
do not agree, we should at least make sure that all regular automatic test
runs are using kernels with DIAGNOSTIC enabled.

Martin


add DIAGNOSTIC back to GENERIC/INSTALL

2011-06-16 Thread Manuel Bouyer
Hello,
for the second time (at last) a kernel issue raised the question of
adding back 'options DIAGNOSTIC' to GENERIC/INSTALL kernels in HEAD.
Several people agreed tha this would be a good thing.

So here's the formal question: would someone object if I add back
'options DIAGNOSTIC' to i386 and amd64 GENERIC and INSTALL kernels,
with a comment saying this should be disabled on release branch
(it would be up to releng to comment it out as part of the release process) ?

I know that DIAGNOSTIC was commented out so that someone could install
a HEAD snapshot and run benchmarks out of the box, but as a side effect
a lot of but are left hidden and only show up when someone tries
to run a Xen kernel (which still have DIAGNOSTIC). See kern/45051 for
another one.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--