re: 2.6.23.8: OOM killer kills wrong jobs

[Dan Kegel]

Marting Mokreja wrote:
> first of all, sorry for not being up to date with how the OOM killer
> works. I think there used to be a kernel config option to disable
> OOM killer and instead kill the process which actually asks for the
> memory and supposedly caused the memory lack. That is what I would
> like to have on my system. I a have a 1GB RAM laptop

You probably just need to add more swap space on your system,

Any time the OOM killer fires, something's wrong with the
system, and it's more productive to deal with that than to
wish for a more accurate OOM killer; see http://lwn.net/Articles/111408/

When I was working at a company that used embedded Linux,
I eventually figured this out, and patched the kernel to panic on OOM
conditions; that gave users the right incentive to avoid
configuring jobs that caused the system to run out of memory.
- Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: 2.6.23.8: OOM killer kills wrong jobs

[Dan Kegel]

Marting Mokreja wrote:
 first of all, sorry for not being up to date with how the OOM killer
 works. I think there used to be a kernel config option to disable
 OOM killer and instead kill the process which actually asks for the
 memory and supposedly caused the memory lack. That is what I would
 like to have on my system. I a have a 1GB RAM laptop

You probably just need to add more swap space on your system,

Any time the OOM killer fires, something's wrong with the
system, and it's more productive to deal with that than to
wish for a more accurate OOM killer; see http://lwn.net/Articles/111408/

When I was working at a company that used embedded Linux,
I eventually figured this out, and patched the kernel to panic on OOM
conditions; that gave users the right incentive to avoid
configuring jobs that caused the system to run out of memory.
- Dan
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Relax permissions for reading hard drive serial number?

[Dan Kegel]

On Nov 29, 2007 7:37 AM, Xavier Bestel <[EMAIL PROTECTED]> wrote:
> > One sticking point is that apps like Photoshop and probably
> > Punkbuster want to retrieve the hard drive's serial number
>
> So they can't be installed on a network drive ?

I think Adobe supports that, though perhaps not with the
retail version.  Big companies with network drives are
probably an important revenue source for them.

I haven't looked closely at what happens when you try installing
onto network drives.  If you are really interested, it's pretty easy to
try yourself; just run the app under wine with
WINEDEBUG=+cdrom,+disk and look in the log for calls like
CreateFile(".\\PhysicalDrive0", ...).
There's some chance the code always checks drive 0 instead
of the drive you're installing onto.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Relax permissions for reading hard drive serial number?

[Dan Kegel]

On 2000-09-07, in
http://marc.info/?l=linux-kernel=96836765403118=2, Linus wrote:
> Hmm.. I have this feeling that it would be much nicer to
> just implement the NT system calls directly.
> ...
> I wouldn't be adverse to supporting Wine better...

A few years on, Wine has matured to the point where it's
ready to run quite a few apps, even some protected by Safedisc.
One sticking point is that apps like Photoshop and probably
Punkbuster want to retrieve the hard drive's serial number,
but Linux restricts access to that info:
$ ls -l /proc/ide/hda/identify
-r 1 root root 0 2007-11-29 07:05 /proc/ide/hda/identify
(And even if /proc/ide goes away, the other ways of retrieving
this info are similarly restricted to root.)

Any reason the access control on this info shouldn't be eased a bit,
e.g. make /proc/ide/*/identify be mode 444?

An alternative might be for Wine to simply fake the serial number,
but I would like to avoid that if possible.

See also http://bugs.winehq.org/show_bug.cgi?id=10018
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Relax permissions for reading hard drive serial number?

[Dan Kegel]

On 2000-09-07, in
http://marc.info/?l=linux-kernelm=96836765403118w=2, Linus wrote:
 Hmm.. I have this feeling that it would be much nicer to
 just implement the NT system calls directly.
 ...
 I wouldn't be adverse to supporting Wine better...

A few years on, Wine has matured to the point where it's
ready to run quite a few apps, even some protected by Safedisc.
One sticking point is that apps like Photoshop and probably
Punkbuster want to retrieve the hard drive's serial number,
but Linux restricts access to that info:
$ ls -l /proc/ide/hda/identify
-r 1 root root 0 2007-11-29 07:05 /proc/ide/hda/identify
(And even if /proc/ide goes away, the other ways of retrieving
this info are similarly restricted to root.)

Any reason the access control on this info shouldn't be eased a bit,
e.g. make /proc/ide/*/identify be mode 444?

An alternative might be for Wine to simply fake the serial number,
but I would like to avoid that if possible.

See also http://bugs.winehq.org/show_bug.cgi?id=10018
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Relax permissions for reading hard drive serial number?

[Dan Kegel]

On Nov 29, 2007 7:37 AM, Xavier Bestel [EMAIL PROTECTED] wrote:
  One sticking point is that apps like Photoshop and probably
  Punkbuster want to retrieve the hard drive's serial number

 So they can't be installed on a network drive ?

I think Adobe supports that, though perhaps not with the
retail version.  Big companies with network drives are
probably an important revenue source for them.

I haven't looked closely at what happens when you try installing
onto network drives.  If you are really interested, it's pretty easy to
try yourself; just run the app under wine with
WINEDEBUG=+cdrom,+disk and look in the log for calls like
CreateFile(.\\PhysicalDrive0, ...).
There's some chance the code always checks drive 0 instead
of the drive you're installing onto.
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Valgrinding the kernel?

[Dan Kegel]

On 7/6/07, Jeff Dike <[EMAIL PROTECTED]> wrote:

UML is cloning a thread in order to test the host's ptrace.  However,
it looks like valgrind is branching to 0x9 for some reason.

This particular bit is going to be problematic for other reasons, but
if valgrind ever looks like it has a chance of working, I can work
around that in UML.


Could you give it a shot?   Maybe the problems after
that will be more pedestrian.  I'm willing to focus a little
effort on this.

Thanks,
Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Valgrinding the kernel?

[Dan Kegel]

On 7/6/07, Jeff Dike [EMAIL PROTECTED] wrote:

UML is cloning a thread in order to test the host's ptrace.  However,
it looks like valgrind is branching to 0x9 for some reason.

This particular bit is going to be problematic for other reasons, but
if valgrind ever looks like it has a chance of working, I can work
around that in UML.


Could you give it a shot?   Maybe the problems after
that will be more pedestrian.  I'm willing to focus a little
effort on this.

Thanks,
Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Valgrinding the kernel?

[Dan Kegel]

It'd be nice to see if Valgrind could catch uninitialized
references in the kernel, if only to see if Coverity is
missing anything that happens in practice.

Back in December 2002, Valgrind started to run UML:
http://user-mode-linux.sourceforge.net/diary.html
http://marc.info/?l=linux-kernel=104035199923121=2
but it wasn't quite usable, and it seems broken since then.
The last note I could find about this was from Jeff In July 2005:
http://marc.info/?l=linux-kernel=112273702329952=2

Has there been any motion since then?

Thanks,
Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Valgrinding the kernel?

[Dan Kegel]

It'd be nice to see if Valgrind could catch uninitialized
references in the kernel, if only to see if Coverity is
missing anything that happens in practice.

Back in December 2002, Valgrind started to run UML:
http://user-mode-linux.sourceforge.net/diary.html
http://marc.info/?l=linux-kernelm=104035199923121w=2
but it wasn't quite usable, and it seems broken since then.
The last note I could find about this was from Jeff In July 2005:
http://marc.info/?l=linux-kernelm=112273702329952w=2

Has there been any motion since then?

Thanks,
Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: make modules Segfault

[Dan Kegel]

George Van Tuyl <[EMAIL PROTECTED]> wrote:
> gcc: Internal error: Segmentation fault (program cpp0)
> ...
> make[3]: Leaving directory `/usr/src/linux-2.4.31/drivers/net/wan'
> ...
> Gnu C  2.96
> ...

cpu family: 6
model : 4
model name: AMD Athlon(tm) Processor
stepping  : 4
cpu MHz   : 1400.110
...

> I  expect you to tell me to upgrade everything.

Nah.  Stop overclocking, and *then* upgrade everything :-)

Seriously, it seems like your machine is flaky.
And even if it were a kernel source problem,
gcc should never have an internal error.
But gcc-2.96 is so old that it's not supported anymore.
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: make modules Segfault

[Dan Kegel]

George Van Tuyl [EMAIL PROTECTED] wrote:
 gcc: Internal error: Segmentation fault (program cpp0)
 ...
 make[3]: Leaving directory `/usr/src/linux-2.4.31/drivers/net/wan'
 ...
 Gnu C  2.96
 ...

cpu family: 6
model : 4
model name: AMD Athlon(tm) Processor
stepping  : 4
cpu MHz   : 1400.110
...

 I  expect you to tell me to upgrade everything.

Nah.  Stop overclocking, and *then* upgrade everything :-)

Seriously, it seems like your machine is flaky.
And even if it were a kernel source problem,
gcc should never have an internal error.
But gcc-2.96 is so old that it's not supported anymore.
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: CONFIG_ALPHA_GENERIC problem with gcc-4.1

[Dan Kegel]

Dan Kegel wrote:

I've been doing builds of linux-2.6.11 as a sanity check
for new versions of gcc, and a problem just popped up
in arch/alpha/Makefile (see 
http://gcc.gnu.org/ml/gcc/2005-07/msg00397.html)


Never mind.  rth kindly explained to me that
it's a gcc or binutils problem.
The alpha kernel compiles in instructions
for ev6 machines even when building for generic/ev5,
and then uses them at runtime only if it
detects that it's safe.
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: CONFIG_ALPHA_GENERIC problem with gcc-4.1

[Dan Kegel]

Dan Kegel wrote:

I've been doing builds of linux-2.6.11 as a sanity check
for new versions of gcc, and a problem just popped up
in arch/alpha/Makefile (see 
http://gcc.gnu.org/ml/gcc/2005-07/msg00397.html)


Never mind.  rth kindly explained to me that
it's a gcc or binutils problem.
The alpha kernel compiles in instructions
for ev6 machines even when building for generic/ev5,
and then uses them at runtime only if it
detects that it's safe.
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

CONFIG_ALPHA_GENERIC problem with gcc-4.1

[Dan Kegel]

I've been doing builds of linux-2.6.11 as a sanity check
for new versions of gcc, and a problem just popped up
in arch/alpha/Makefile (see http://gcc.gnu.org/ml/gcc/2005-07/msg00397.html)
I think I can work around this myself by using CONFIG_ALPHA_EV6
instead of CONFIG_ALPHA_GENERIC, but here's my analysis
of the problem; maybe the alpha kernel maintainer can take it from here?

Take the following with a grain of salt; I don't know much
about alpha or gcc, I'm just doing a little QA.

arch/alpha/Makefile says:
36   # If GENERIC, make sure to turn off any instruction set extensions that
37   # the host compiler might have on by default.  Given that EV4 and EV5
38   # have the same instruction set, prefer EV5 because an EV5 schedule is
39   # more likely to keep an EV4 processor busy than vice-versa.
40   ifeq ($(CONFIG_ALPHA_GENERIC),y)
41 mcpu := ev5
42 mcpu_done := y
43   endif
...
84 # For TSUNAMI, we must have the assembler not emulate our instructions.
85 # The same is true for IRONGATE, POLARIS, PYXIS.
86 # BWX is most important, but we don't really want any emulation ever.
87 CFLAGS += $(cflags-y) -Wa,-mev6

Thus when you pick CONFIG_ALPHA_GENERIC, gcc is invoked with
the contradictory options -mcpu=ev5 -Wa,-mev6

This probably means that even on ev5, some ev6 instructions are used.
In particular, see include/asm-alpha/compiler.h:

#if defined(__alpha_bwx__)
#define __kernel_ldbu(mem)  (mem)
#define __kernel_ldwu(mem)  (mem)
#define __kernel_stb(val,mem)   ((mem) = (val))
#define __kernel_stw(val,mem)   ((mem) = (val))
#else
#define __kernel_ldbu(mem)  \
  ({ unsigned char __kir;   \
 __asm__("ldbu %0,%1" : "=r"(__kir) : "m"(mem));\
 __kir; })

That inline assembly is fine on ev5, but only if the assembler
is emulating the ldbu instruction with a macro -- exactly
the kind of thing arch/alpha/Makefile is trying to
inhibit when it says -Wa,-mev6.

This is an issue now because building the kernel with CONFIG_ALPHA_GENERIC
fails on the current gcc-4.1 snapshot with

> {standard input}:496: Error: macro requires $at register while noat in effect
> make[1]: *** [arch/alpha/kernel/core_cia.o] Error 1

and it looks like a kernel problem, not a gcc problem:
don't try to use ev6 instructions on ev5 or earlier processors.

That probably means conditionalizing that -Wa,ev6 properly,
but if that's hard, maybe it means dropping support for ev4 and ev5 processors,
and mapping CONFIG_ALPHA_GENERIC to ev6.  I wouldn't know...
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

CONFIG_ALPHA_GENERIC problem with gcc-4.1

[Dan Kegel]

I've been doing builds of linux-2.6.11 as a sanity check
for new versions of gcc, and a problem just popped up
in arch/alpha/Makefile (see http://gcc.gnu.org/ml/gcc/2005-07/msg00397.html)
I think I can work around this myself by using CONFIG_ALPHA_EV6
instead of CONFIG_ALPHA_GENERIC, but here's my analysis
of the problem; maybe the alpha kernel maintainer can take it from here?

Take the following with a grain of salt; I don't know much
about alpha or gcc, I'm just doing a little QA.

arch/alpha/Makefile says:
36   # If GENERIC, make sure to turn off any instruction set extensions that
37   # the host compiler might have on by default.  Given that EV4 and EV5
38   # have the same instruction set, prefer EV5 because an EV5 schedule is
39   # more likely to keep an EV4 processor busy than vice-versa.
40   ifeq ($(CONFIG_ALPHA_GENERIC),y)
41 mcpu := ev5
42 mcpu_done := y
43   endif
...
84 # For TSUNAMI, we must have the assembler not emulate our instructions.
85 # The same is true for IRONGATE, POLARIS, PYXIS.
86 # BWX is most important, but we don't really want any emulation ever.
87 CFLAGS += $(cflags-y) -Wa,-mev6

Thus when you pick CONFIG_ALPHA_GENERIC, gcc is invoked with
the contradictory options -mcpu=ev5 -Wa,-mev6

This probably means that even on ev5, some ev6 instructions are used.
In particular, see include/asm-alpha/compiler.h:

#if defined(__alpha_bwx__)
#define __kernel_ldbu(mem)  (mem)
#define __kernel_ldwu(mem)  (mem)
#define __kernel_stb(val,mem)   ((mem) = (val))
#define __kernel_stw(val,mem)   ((mem) = (val))
#else
#define __kernel_ldbu(mem)  \
  ({ unsigned char __kir;   \
 __asm__(ldbu %0,%1 : =r(__kir) : m(mem));\
 __kir; })

That inline assembly is fine on ev5, but only if the assembler
is emulating the ldbu instruction with a macro -- exactly
the kind of thing arch/alpha/Makefile is trying to
inhibit when it says -Wa,-mev6.

This is an issue now because building the kernel with CONFIG_ALPHA_GENERIC
fails on the current gcc-4.1 snapshot with

 {standard input}:496: Error: macro requires $at register while noat in effect
 make[1]: *** [arch/alpha/kernel/core_cia.o] Error 1

and it looks like a kernel problem, not a gcc problem:
don't try to use ev6 instructions on ev5 or earlier processors.

That probably means conditionalizing that -Wa,ev6 properly,
but if that's hard, maybe it means dropping support for ev4 and ev5 processors,
and mapping CONFIG_ALPHA_GENERIC to ev6.  I wouldn't know...
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: function ordering (was: Re: [RFC] exit_thread() speedups in x86

[Dan Kegel]

Arjan van de Ven wrote:

hmm. I wonder if a slightly different approach (based on the __slow)
idea would make sense
1) Use -ffunction-sections option from gcc to put each function in it's
own section
2) Use readprofile/oprofile data to collect an (external to the code)
list of hot/cold functions (we can put a default list in the kernel
source somewhere and allow people to measure their own if they want)
3) Use this list to make a linker script to order the functions


Somebody recently implemented something similar; see
http://www.bclennox.com/ldreorder/
It used valgrind rather than oprofile.
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: function ordering (was: Re: [RFC] exit_thread() speedups in x86

[Dan Kegel]

Arjan van de Ven wrote:

hmm. I wonder if a slightly different approach (based on the __slow)
idea would make sense
1) Use -ffunction-sections option from gcc to put each function in it's
own section
2) Use readprofile/oprofile data to collect an (external to the code)
list of hot/cold functions (we can put a default list in the kernel
source somewhere and allow people to measure their own if they want)
3) Use this list to make a linker script to order the functions


Somebody recently implemented something similar; see
http://www.bclennox.com/ldreorder/
It used valgrind rather than oprofile.
- Dan

--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [LTP] Re: [ANNOUNCE] April Release of LTP now Available

[Dan Kegel]

Andrew Morton wrote:
LTP-20050405
It seems to have an x86ism in it which causes the compile to fail on ppc64:
socketcall01.c: In function `socketcall':
socketcall01.c:80: error: asm-specifier for variable `__sc_4' conflicts with 
asm clobber list
That might be a problem with your toolchain.
Other mentions of that error message on Google
suggest that it's due to a kernel header problem.
I bet your toolchain uses kernel headers from 2.4.21 or earlier...
check includes/asm-ppc64/unistd.h to see if it's got the
line
  /* On powerpc a system call basically clobbers the same registers like a
in it.  If not, it may be missing the patch mentioned below.
See
http://ozlabs.org/pipermail/linuxppc64-dev/2003-April/000211.html
http://ozlabs.org/pipermail/linuxppc-dev/2002-October/014492.html
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9379
http://www.hu.kernel.org/pub/linux/kernel/v2.4/snapshots/incr/patch-2.4.22-bk57-bk58
- Dan
--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [LTP] Re: [ANNOUNCE] April Release of LTP now Available

[Dan Kegel]

Andrew Morton wrote:
LTP-20050405
It seems to have an x86ism in it which causes the compile to fail on ppc64:
socketcall01.c: In function `socketcall':
socketcall01.c:80: error: asm-specifier for variable `__sc_4' conflicts with 
asm clobber list
That might be a problem with your toolchain.
Other mentions of that error message on Google
suggest that it's due to a kernel header problem.
I bet your toolchain uses kernel headers from 2.4.21 or earlier...
check includes/asm-ppc64/unistd.h to see if it's got the
line
  /* On powerpc a system call basically clobbers the same registers like a
in it.  If not, it may be missing the patch mentioned below.
See
http://ozlabs.org/pipermail/linuxppc64-dev/2003-April/000211.html
http://ozlabs.org/pipermail/linuxppc-dev/2002-October/014492.html
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9379
http://www.hu.kernel.org/pub/linux/kernel/v2.4/snapshots/incr/patch-2.4.22-bk57-bk58
- Dan
--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: 2.6.11.3 build problem in arch/alpha/kernel/srcons.c with gcc-4.0

[Dan Kegel]

Andrew Morton wrote:
Dan Kegel <[EMAIL PROTECTED]> wrote:
Anyone with an alpha care to suggest a fix for this?
arch/alpha/kernel/srmcons.c: In function 'srmcons_open':
arch/alpha/kernel/srmcons.c:196: warning: 'srmconsp' may be used uninitialized 
in this function
make[1]: *** [arch/alpha/kernel/srmcons.o] Error 1
make: *** [arch/alpha/kernel] Error 2
I get this when building the 2.6.11.3 kernel with a recent gcc-4.0 snapshot.

It's beyond gcc's ability to figure out that the code is OK.  Options would
be to disable -Werror, or to artificially initialise that variable.
I'll initialize it to something - guess it doesn't matter what.
Thanks,
Dan
--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2.6.11.3 build problem in arch/alpha/kernel/srcons.c with gcc-4.0

[Dan Kegel]

Anyone with an alpha care to suggest a fix for this?
arch/alpha/kernel/srmcons.c: In function 'srmcons_open':
arch/alpha/kernel/srmcons.c:196: warning: 'srmconsp' may be used uninitialized 
in this function
make[1]: *** [arch/alpha/kernel/srmcons.o] Error 1
make: *** [arch/alpha/kernel] Error 2
I get this when building the 2.6.11.3 kernel with a recent gcc-4.0 snapshot.
Thanks,
Dan
--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2.6.11.3 build problem in arch/alpha/kernel/srcons.c with gcc-4.0

[Dan Kegel]

Anyone with an alpha care to suggest a fix for this?
arch/alpha/kernel/srmcons.c: In function 'srmcons_open':
arch/alpha/kernel/srmcons.c:196: warning: 'srmconsp' may be used uninitialized 
in this function
make[1]: *** [arch/alpha/kernel/srmcons.o] Error 1
make: *** [arch/alpha/kernel] Error 2
I get this when building the 2.6.11.3 kernel with a recent gcc-4.0 snapshot.
Thanks,
Dan
--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: 2.6.11.3 build problem in arch/alpha/kernel/srcons.c with gcc-4.0

[Dan Kegel]

Andrew Morton wrote:
Dan Kegel [EMAIL PROTECTED] wrote:
Anyone with an alpha care to suggest a fix for this?
arch/alpha/kernel/srmcons.c: In function 'srmcons_open':
arch/alpha/kernel/srmcons.c:196: warning: 'srmconsp' may be used uninitialized 
in this function
make[1]: *** [arch/alpha/kernel/srmcons.o] Error 1
make: *** [arch/alpha/kernel] Error 2
I get this when building the 2.6.11.3 kernel with a recent gcc-4.0 snapshot.

It's beyond gcc's ability to figure out that the code is OK.  Options would
be to disable -Werror, or to artificially initialise that variable.
I'll initialize it to something - guess it doesn't matter what.
Thanks,
Dan
--
Trying to get a job as a c++ developer?  See 
http://kegel.com/academy/getting-hired.html
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH] gcc-4.0 compatibility fixes for i386, m68k, ppc64, x86_64

[Dan Kegel]

gcc-4.0 fails with
"error: array type has incomplete element type"
(see http://gcc.gnu.org/ml/gcc/2005-02/msg00053.html)
on several files in linux-2.6.11.3.
Who knows, maybe all this is fixed in the -mm tree already,
but what the heck, here are a few fixes I haven't seen anyone
else post yet.  These fix build problems for me, and
are all trivial.  I haven't tested the resulting binaries.
See also related sets of fixes at
http://user.it.uu.se/~mikpe/linux/patches/2.6/patch-gcc4-fixes-v2-2.6.11
and
http://archives.andrew.net.au/lm-sensors/msg29809.html
I don't duplicate any of those here, I think.
--- part 1 ---
Fixes
In file included from include/asm/thread_info.h:16,
 from include/linux/thread_info.h:21,
 from include/linux/spinlock.h:12,
 from include/linux/capability.h:45,
 from include/linux/sched.h:7,
 from arch/i386/kernel/asm-offsets.c:7:
include/asm/processor.h:87: error: array type has incomplete element type
make[1]: *** [arch/i386/kernel/asm-offsets.asm] Error 1
--- linux-2.6.11.3/include/asm-i386/processor.h.old Tue Mar 15 06:45:26 2005
+++ linux-2.6.11.3/include/asm-i386/processor.h Tue Mar 15 06:46:45 2005
@@ -81,6 +81,64 @@
 #define X86_VENDOR_UNKNOWN 0xff
 /*
+ * Size of io_bitmap.
+ */
+#define IO_BITMAP_BITS  65536
+#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)
+#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long))
+#define INVALID_IO_BITMAP_OFFSET 0x8000
+#define INVALID_IO_BITMAP_OFFSET_LAZY 0x9000
+
+struct tss_struct {
+   unsigned short  back_link,__blh;
+   unsigned long   esp0;
+   unsigned short  ss0,__ss0h;
+   unsigned long   esp1;
+   unsigned short  ss1,__ss1h; /* ss1 is used to cache 
MSR_IA32_SYSENTER_CS */
+   unsigned long   esp2;
+   unsigned short  ss2,__ss2h;
+   unsigned long   __cr3;
+   unsigned long   eip;
+   unsigned long   eflags;
+   unsigned long   eax,ecx,edx,ebx;
+   unsigned long   esp;
+   unsigned long   ebp;
+   unsigned long   esi;
+   unsigned long   edi;
+   unsigned short  es, __esh;
+   unsigned short  cs, __csh;
+   unsigned short  ss, __ssh;
+   unsigned short  ds, __dsh;
+   unsigned short  fs, __fsh;
+   unsigned short  gs, __gsh;
+   unsigned short  ldt, __ldth;
+   unsigned short  trace, io_bitmap_base;
+   /*
+* The extra 1 is there because the CPU will access an
+* additional byte beyond the end of the IO permission
+* bitmap. The extra byte must be all 1 bits, and must
+* be within the limit.
+*/
+   unsigned long   io_bitmap[IO_BITMAP_LONGS + 1];
+   /*
+* Cache the current maximum and the last task that used the bitmap:
+*/
+   unsigned long io_bitmap_max;
+   struct thread_struct *io_bitmap_owner;
+   /*
+* pads the TSS to be cacheline-aligned (size is 0x100)
+*/
+   unsigned long __cacheline_filler[35];
+   /*
+* .. and then another 0x100 bytes for emergency kernel stack
+*/
+   unsigned long stack[64];
+} __attribute__((packed));
+
+#define IO_BITMAP_OFFSET offsetof(struct tss_struct,io_bitmap)
+#define ARCH_MIN_TASKALIGN 16
+
+/*
  * capabilities of CPUs
  */
@@ -308,16 +366,6 @@
 #define HAVE_ARCH_PICK_MMAP_LAYOUT
-/*
- * Size of io_bitmap.
- */
-#define IO_BITMAP_BITS  65536
-#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)
-#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long))
-#define IO_BITMAP_OFFSET offsetof(struct tss_struct,io_bitmap)
-#define INVALID_IO_BITMAP_OFFSET 0x8000
-#define INVALID_IO_BITMAP_OFFSET_LAZY 0x9000
-
 struct i387_fsave_struct {
longcwd;
longswd;
@@ -371,54 +419,6 @@
 } mm_segment_t;
 struct thread_struct;
-
-struct tss_struct {
-   unsigned short  back_link,__blh;
-   unsigned long   esp0;
-   unsigned short  ss0,__ss0h;
-   unsigned long   esp1;
-   unsigned short  ss1,__ss1h; /* ss1 is used to cache 
MSR_IA32_SYSENTER_CS */
-   unsigned long   esp2;
-   unsigned short  ss2,__ss2h;
-   unsigned long   __cr3;
-   unsigned long   eip;
-   unsigned long   eflags;
-   unsigned long   eax,ecx,edx,ebx;
-   unsigned long   esp;
-   unsigned long   ebp;
-   unsigned long   esi;
-   unsigned long   edi;
-   unsigned short  es, __esh;
-   unsigned short  cs, __csh;
-   unsigned short  ss, __ssh;
-   unsigned short  ds, __dsh;
-   unsigned short  fs, __fsh;
-   unsigned short  gs, __gsh;
-   unsigned short  ldt, __ldth;
-   unsigned short  trace, io_bitmap_base;
-   /*
-* The extra 1 is there because the CPU will access an
-* additional byte beyond the end of the IO permission
-* bitmap. The extra byte must be all 1 bits, and must
-* be within the limit.
-*/
-   unsigned long   io_bitmap[IO_BITMAP_LONGS + 1];
-   /*
-* Cache the current maximum 

[PATCH] gcc-4.0 compatibility fixes for i386, m68k, ppc64, x86_64

[Dan Kegel]

gcc-4.0 fails with
error: array type has incomplete element type
(see http://gcc.gnu.org/ml/gcc/2005-02/msg00053.html)
on several files in linux-2.6.11.3.
Who knows, maybe all this is fixed in the -mm tree already,
but what the heck, here are a few fixes I haven't seen anyone
else post yet.  These fix build problems for me, and
are all trivial.  I haven't tested the resulting binaries.
See also related sets of fixes at
http://user.it.uu.se/~mikpe/linux/patches/2.6/patch-gcc4-fixes-v2-2.6.11
and
http://archives.andrew.net.au/lm-sensors/msg29809.html
I don't duplicate any of those here, I think.
--- part 1 ---
Fixes
In file included from include/asm/thread_info.h:16,
 from include/linux/thread_info.h:21,
 from include/linux/spinlock.h:12,
 from include/linux/capability.h:45,
 from include/linux/sched.h:7,
 from arch/i386/kernel/asm-offsets.c:7:
include/asm/processor.h:87: error: array type has incomplete element type
make[1]: *** [arch/i386/kernel/asm-offsets.asm] Error 1
--- linux-2.6.11.3/include/asm-i386/processor.h.old Tue Mar 15 06:45:26 2005
+++ linux-2.6.11.3/include/asm-i386/processor.h Tue Mar 15 06:46:45 2005
@@ -81,6 +81,64 @@
 #define X86_VENDOR_UNKNOWN 0xff
 /*
+ * Size of io_bitmap.
+ */
+#define IO_BITMAP_BITS  65536
+#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)
+#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long))
+#define INVALID_IO_BITMAP_OFFSET 0x8000
+#define INVALID_IO_BITMAP_OFFSET_LAZY 0x9000
+
+struct tss_struct {
+   unsigned short  back_link,__blh;
+   unsigned long   esp0;
+   unsigned short  ss0,__ss0h;
+   unsigned long   esp1;
+   unsigned short  ss1,__ss1h; /* ss1 is used to cache 
MSR_IA32_SYSENTER_CS */
+   unsigned long   esp2;
+   unsigned short  ss2,__ss2h;
+   unsigned long   __cr3;
+   unsigned long   eip;
+   unsigned long   eflags;
+   unsigned long   eax,ecx,edx,ebx;
+   unsigned long   esp;
+   unsigned long   ebp;
+   unsigned long   esi;
+   unsigned long   edi;
+   unsigned short  es, __esh;
+   unsigned short  cs, __csh;
+   unsigned short  ss, __ssh;
+   unsigned short  ds, __dsh;
+   unsigned short  fs, __fsh;
+   unsigned short  gs, __gsh;
+   unsigned short  ldt, __ldth;
+   unsigned short  trace, io_bitmap_base;
+   /*
+* The extra 1 is there because the CPU will access an
+* additional byte beyond the end of the IO permission
+* bitmap. The extra byte must be all 1 bits, and must
+* be within the limit.
+*/
+   unsigned long   io_bitmap[IO_BITMAP_LONGS + 1];
+   /*
+* Cache the current maximum and the last task that used the bitmap:
+*/
+   unsigned long io_bitmap_max;
+   struct thread_struct *io_bitmap_owner;
+   /*
+* pads the TSS to be cacheline-aligned (size is 0x100)
+*/
+   unsigned long __cacheline_filler[35];
+   /*
+* .. and then another 0x100 bytes for emergency kernel stack
+*/
+   unsigned long stack[64];
+} __attribute__((packed));
+
+#define IO_BITMAP_OFFSET offsetof(struct tss_struct,io_bitmap)
+#define ARCH_MIN_TASKALIGN 16
+
+/*
  * capabilities of CPUs
  */
@@ -308,16 +366,6 @@
 #define HAVE_ARCH_PICK_MMAP_LAYOUT
-/*
- * Size of io_bitmap.
- */
-#define IO_BITMAP_BITS  65536
-#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)
-#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long))
-#define IO_BITMAP_OFFSET offsetof(struct tss_struct,io_bitmap)
-#define INVALID_IO_BITMAP_OFFSET 0x8000
-#define INVALID_IO_BITMAP_OFFSET_LAZY 0x9000
-
 struct i387_fsave_struct {
longcwd;
longswd;
@@ -371,54 +419,6 @@
 } mm_segment_t;
 struct thread_struct;
-
-struct tss_struct {
-   unsigned short  back_link,__blh;
-   unsigned long   esp0;
-   unsigned short  ss0,__ss0h;
-   unsigned long   esp1;
-   unsigned short  ss1,__ss1h; /* ss1 is used to cache 
MSR_IA32_SYSENTER_CS */
-   unsigned long   esp2;
-   unsigned short  ss2,__ss2h;
-   unsigned long   __cr3;
-   unsigned long   eip;
-   unsigned long   eflags;
-   unsigned long   eax,ecx,edx,ebx;
-   unsigned long   esp;
-   unsigned long   ebp;
-   unsigned long   esi;
-   unsigned long   edi;
-   unsigned short  es, __esh;
-   unsigned short  cs, __csh;
-   unsigned short  ss, __ssh;
-   unsigned short  ds, __dsh;
-   unsigned short  fs, __fsh;
-   unsigned short  gs, __gsh;
-   unsigned short  ldt, __ldth;
-   unsigned short  trace, io_bitmap_base;
-   /*
-* The extra 1 is there because the CPU will access an
-* additional byte beyond the end of the IO permission
-* bitmap. The extra byte must be all 1 bits, and must
-* be within the limit.
-*/
-   unsigned long   io_bitmap[IO_BITMAP_LONGS + 1];
-   /*
-* Cache the current maximum 

Re: A signal fairy tale

[Dan Kegel]

Dan Kegel wrote:
> Pseudocode:
> 
>   sigemptyset();
>   sigaddset(SIGUSR1, );
>   fd=sigopen();
>   m=read(fd, buf, n*sizeof(siginfo_t))
>   close(fd);
> 
> should probably be equivalent to
> 
>   sigemptyset();
>   sigaddset(SIGUSR1, );
>   struct sigaction newaction, oldaction;
>   newaction.sa_handler = dummy_handler;
>   newaction.sa_flags = SA_SIGINFO;
>   newaction.sa_mask = 0;
>   sigaction(SIGUSR1, , );

I forgot to mask off the signal to avoid traditional delivery:

sigprocmask(SIG_BLOCK, , );

>   for (i=0; i  if (sigwaitinfo(, buf+i))
> break;
>   m = n * sizeof(siginfo_t);
>   sigaction(SIGUSR1, , 0);

sigprocmask(SIG_UNBLOCK, );
 
> (apologies if any of the above is wrong)

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: CLOSE_WAIT Problem

[Dan Kegel]

Chriss wrote:
> I wrote a simple server application and installed it on a linux machine
> in Slovakia, running Mandrake 7.2 (2.2.18).
> That machine loses tcp/ip packages, as it uses a Microwave connection.
> So my server works all the time, and the tcp/ip connections are set to
> TIME_WAIT, but after a couple of hours
> my server application won't get any connections anymore and the netstat
> shows a lot of CLOSE_WAITs that belong to the server.
> I've installed the same server on two SuSE 7.1 (2.2.18) machines in
> Austria, and the problem never occured.
> So does anyone know how to avoid that CLOSE_WAITs, or at least how to
> get rid of  them?

Dunno if this will help, but:

They're supposed to go away by themselves after 2MSL (about 120 seconds).
Other people (on many operating systems) have reported similar problems, btw:

http://uwsg.iu.edu/hypermail/linux/net/9611.2/0043.html
http://www.sunmanagers.org/pipermail/sunmanagers/2001-April/002894.html
http://www2.real-time.com/tclug-list/1999/Jun/msg00254.html
http://mail-index.netbsd.org/netbsd-bugs/1996/04/16/0004.html

The last one has a fix for an old bug in netbsd that could cause this.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: O_DIRECT please; Sybase 12.5

[Dan Kegel]

Alan Cox wrote:
> 
> > the boss say "If Linux makes Sybase go through the page cache on
> > reads, maybe we'll just have to switch to Solaris.  That's
> > a serious performance problem."
> 
> Thats something you'd have to benchmark. It depends on a very large number
> of factors including whether the database uses mmap, the average I/O size
> and the like

I'll probably benchmark raw vs. non-raw I/O with Sybase ASE 12.5
on our application once we've come up to speed on basic performance
issues (we're database newbies).
 
> > It supports raw partitions, which is good; that might satisfy my
> > boss (although the administration will be a pain, and I'm not
> > sure whether it's really supported by Dell RAID devices).
> > I'd prefer O_DIRECT :-(
> 
> We already support raw direct I/O to devices themselves so they should support
> that - if not then Oracle I believe already does.

Haven't seen Sybase talk about O_DIRECT.  Not sure we want to
pony up the Sybase license fees.  (I'm still in denial about
databases in general, and hope I can switch to PostgreSQL
at some point.)

BTW, 
http://eval.veritas.com/webfiles/whitepapers/sybaseedition/sybase14_performance_paper.pdf
seems to show that raw beats O_DIRECT hands down on Solaris.
Will that hold on Linux, or is your (forthcoming?) O_DIRECT
higher performance than the one on Solaris?

Thanks,
Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

O_DIRECT please; Sybase 12.5

[Dan Kegel]

At work I had to sit through a meeting where I heard
the boss say "If Linux makes Sybase go through the page cache on
reads, maybe we'll just have to switch to Solaris.  That's
a serious performance problem."
All I could say was "I expect Linux will support O_DIRECT
soon, and Sybase will support that within a year."  

Er, so did I promise too much?  Andrea mentioned O_DIRECT recently
( http://marc.theaimsgroup.com/?l=linux-kernel=99253913516599=2,
 http://lwn.net/2001/0510/bigpage.php3 )
Is it supported yet in 2.4, or is this a 2.5 thing?

And what are the chances Sybase will support that flag any time
soon?  I just read on news://forums.sybase.com/sybase.public.ase.linux
that Sybase ASE 12.5 was released today, and a 60 day eval is downloadable
for NT and Linux.  I'm downloading now; it's a biggie.

It supports raw partitions, which is good; that might satisfy my
boss (although the administration will be a pain, and I'm not
sure whether it's really supported by Dell RAID devices).
I'd prefer O_DIRECT :-(

Hope somebody can give me encouraging news.

Thanks,
Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: A signal fairy tale

[Dan Kegel]

Christopher Smith wrote:
> 
> At 07:57 PM 6/27/2001 -0700, Daniel R. Kegel wrote:
> >From: Christopher Smith <[EMAIL PROTECTED]>
> > >I guess the main thing I'm thinking is this could require some significant
> > >changes to the way the kernel behaves. Still, it's worth taking a "try it
> > >and see approach". If anyone else thinks this is a good idea I may hack
> > >together a sample patch and give it a whirl.
> >
> >What's the biggest change you see?  From my (two-martini-lunch-tainted)
> >viewpoint, it's just another kind of signal masking, sorta...
> 
> Yeah, the more I think about it, the more I think this is just another
> branch in the signal delivery code. Not necessarily too huge a change. I'll
> hack on this over the weekend I think.

Cool, have fun!

Feature checklist for future reference:

If sigopen() has been called, and the file descriptor is still open,
sigaction(x, 0, ) should show foo != SIG_DFL for compatibility
with the traditional signal allocation scheme.

If sigopen() has been called, and the file descriptor is still open,
sending a signal to the thread (or if posix, the process) that called
sigopen() should cause the signal to stick until picked up by
read(), or until close() is called on the fd(), in which case it will
be delivered or picked up as normal.

sigaction(x, , 0) should return EBUSY if fd=sigopen(x) has been
called but close(fd) has not yet been called.

Pseudocode:

  sigemptyset();
  sigaddset(SIGUSR1, );
  fd=sigopen();
  m=read(fd, buf, n*sizeof(siginfo_t)) 
  close(fd);

should probably be equivalent to

  sigemptyset();
  sigaddset(SIGUSR1, );
  struct sigaction newaction, oldaction;
  newaction.sa_handler = dummy_handler;
  newaction.sa_flags = SA_SIGINFO;
  newaction.sa_mask = 0;
  sigaction(SIGUSR1, , );
  for (i=0; ihttp://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: A signal fairy tale

[Dan Kegel]

Christopher Smith wrote:
> 
> At 07:49 PM 6/27/2001 -0700, Daniel R. Kegel wrote:
> >Balbir Singh <[EMAIL PROTECTED]> wrote:
> > >sigopen() should be selective about the signals it allows
> > >as argument. Try and make sigopen() thread specific, so that if one
> > >thread does a sigopen(), it does not imply it will do all the signal
> > >handling for all the threads.
> >
> >IMHO sigopen()/read() should behave just like sigwait() with respect
> >to threads.  That means that in Posix, it would not be thread specific,
> >but in Linux, it would be thread specific, because that's how signals
> >and threads work there at the moment.
> 
> Actually, I believe with IBM's new Posix threads implementation, Linux
> finally does signal delivery "the right way". In general, I think it'd be
> nice if this API *always* sucked up signals from all threads. This makes
> sense particularly since the FD is accessible by all threads.

Although I'm looking forward to the day when Linux threading
(perhaps thanks to IBM's enhancements to Gnu Pth) becomes Posix compliant,
for now we need to consider both Posix threads and LinuxThreads.  
I think the proper behavior for sigopen() under the two threading systems would be:

Posix threads: sigopen() would capture signals delivered to the process,
as well as signals delivered by pthread_kill() to the thread that called sigopen().

Current LinuxThreads: sigopen() would only capture signals delivered 
to the thread that called sigopen().

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: A signal fairy tale

[Dan Kegel]

Christopher Smith wrote:
 
 At 07:49 PM 6/27/2001 -0700, Daniel R. Kegel wrote:
 Balbir Singh [EMAIL PROTECTED] wrote:
  sigopen() should be selective about the signals it allows
  as argument. Try and make sigopen() thread specific, so that if one
  thread does a sigopen(), it does not imply it will do all the signal
  handling for all the threads.
 
 IMHO sigopen()/read() should behave just like sigwait() with respect
 to threads.  That means that in Posix, it would not be thread specific,
 but in Linux, it would be thread specific, because that's how signals
 and threads work there at the moment.
 
 Actually, I believe with IBM's new Posix threads implementation, Linux
 finally does signal delivery the right way. In general, I think it'd be
 nice if this API *always* sucked up signals from all threads. This makes
 sense particularly since the FD is accessible by all threads.

Although I'm looking forward to the day when Linux threading
(perhaps thanks to IBM's enhancements to Gnu Pth) becomes Posix compliant,
for now we need to consider both Posix threads and LinuxThreads.  
I think the proper behavior for sigopen() under the two threading systems would be:

Posix threads: sigopen() would capture signals delivered to the process,
as well as signals delivered by pthread_kill() to the thread that called sigopen().

Current LinuxThreads: sigopen() would only capture signals delivered 
to the thread that called sigopen().

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: A signal fairy tale

[Dan Kegel]

Christopher Smith wrote:
 
 At 07:57 PM 6/27/2001 -0700, Daniel R. Kegel wrote:
 From: Christopher Smith [EMAIL PROTECTED]
  I guess the main thing I'm thinking is this could require some significant
  changes to the way the kernel behaves. Still, it's worth taking a try it
  and see approach. If anyone else thinks this is a good idea I may hack
  together a sample patch and give it a whirl.
 
 What's the biggest change you see?  From my (two-martini-lunch-tainted)
 viewpoint, it's just another kind of signal masking, sorta...
 
 Yeah, the more I think about it, the more I think this is just another
 branch in the signal delivery code. Not necessarily too huge a change. I'll
 hack on this over the weekend I think.

Cool, have fun!

Feature checklist for future reference:

If sigopen() has been called, and the file descriptor is still open,
sigaction(x, 0, foo) should show foo != SIG_DFL for compatibility
with the traditional signal allocation scheme.

If sigopen() has been called, and the file descriptor is still open,
sending a signal to the thread (or if posix, the process) that called
sigopen() should cause the signal to stick until picked up by
read(), or until close() is called on the fd(), in which case it will
be delivered or picked up as normal.

sigaction(x, foo, 0) should return EBUSY if fd=sigopen(x) has been
called but close(fd) has not yet been called.

Pseudocode:

  sigemptyset(s);
  sigaddset(SIGUSR1, s);
  fd=sigopen(s);
  m=read(fd, buf, n*sizeof(siginfo_t)) 
  close(fd);

should probably be equivalent to

  sigemptyset(s);
  sigaddset(SIGUSR1, s);
  struct sigaction newaction, oldaction;
  newaction.sa_handler = dummy_handler;
  newaction.sa_flags = SA_SIGINFO;
  newaction.sa_mask = 0;
  sigaction(SIGUSR1, newaction, oldaction);
  for (i=0; in; i++)
 if (sigwaitinfo(s, buf+i))
break;
  m = n * sizeof(siginfo_t);
  sigaction(SIGUSR1, oldaction, 0);

(apologies if any of the above is wrong)

But, um, Chris, could you check your library code to make sure you did
the sigaction stuff?  Could it be that you forgot that, and if you did
that properly, the main application would notice that you'd allocated
a signal, and not suck up your signals?

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

O_DIRECT please; Sybase 12.5

[Dan Kegel]

At work I had to sit through a meeting where I heard
the boss say If Linux makes Sybase go through the page cache on
reads, maybe we'll just have to switch to Solaris.  That's
a serious performance problem.
All I could say was I expect Linux will support O_DIRECT
soon, and Sybase will support that within a year.  

Er, so did I promise too much?  Andrea mentioned O_DIRECT recently
( http://marc.theaimsgroup.com/?l=linux-kernelm=99253913516599w=2,
 http://lwn.net/2001/0510/bigpage.php3 )
Is it supported yet in 2.4, or is this a 2.5 thing?

And what are the chances Sybase will support that flag any time
soon?  I just read on news://forums.sybase.com/sybase.public.ase.linux
that Sybase ASE 12.5 was released today, and a 60 day eval is downloadable
for NT and Linux.  I'm downloading now; it's a biggie.

It supports raw partitions, which is good; that might satisfy my
boss (although the administration will be a pain, and I'm not
sure whether it's really supported by Dell RAID devices).
I'd prefer O_DIRECT :-(

Hope somebody can give me encouraging news.

Thanks,
Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: CLOSE_WAIT Problem

[Dan Kegel]

Chriss wrote:
 I wrote a simple server application and installed it on a linux machine
 in Slovakia, running Mandrake 7.2 (2.2.18).
 That machine loses tcp/ip packages, as it uses a Microwave connection.
 So my server works all the time, and the tcp/ip connections are set to
 TIME_WAIT, but after a couple of hours
 my server application won't get any connections anymore and the netstat
 shows a lot of CLOSE_WAITs that belong to the server.
 I've installed the same server on two SuSE 7.1 (2.2.18) machines in
 Austria, and the problem never occured.
 So does anyone know how to avoid that CLOSE_WAITs, or at least how to
 get rid of  them?

Dunno if this will help, but:

They're supposed to go away by themselves after 2MSL (about 120 seconds).
Other people (on many operating systems) have reported similar problems, btw:

http://uwsg.iu.edu/hypermail/linux/net/9611.2/0043.html
http://www.sunmanagers.org/pipermail/sunmanagers/2001-April/002894.html
http://www2.real-time.com/tclug-list/1999/Jun/msg00254.html
http://mail-index.netbsd.org/netbsd-bugs/1996/04/16/0004.html

The last one has a fix for an old bug in netbsd that could cause this.
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: A signal fairy tale

[Dan Kegel]

Dan Kegel wrote:
 Pseudocode:
 
   sigemptyset(s);
   sigaddset(SIGUSR1, s);
   fd=sigopen(s);
   m=read(fd, buf, n*sizeof(siginfo_t))
   close(fd);
 
 should probably be equivalent to
 
   sigemptyset(s);
   sigaddset(SIGUSR1, s);
   struct sigaction newaction, oldaction;
   newaction.sa_handler = dummy_handler;
   newaction.sa_flags = SA_SIGINFO;
   newaction.sa_mask = 0;
   sigaction(SIGUSR1, newaction, oldaction);

I forgot to mask off the signal to avoid traditional delivery:

sigprocmask(SIG_BLOCK, s, oldmask);

   for (i=0; in; i++)
  if (sigwaitinfo(s, buf+i))
 break;
   m = n * sizeof(siginfo_t);
   sigaction(SIGUSR1, oldaction, 0);

sigprocmask(SIG_UNBLOCK, s);
 
 (apologies if any of the above is wrong)

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: O_DIRECT please; Sybase 12.5

[Dan Kegel]

Alan Cox wrote:
 
  the boss say If Linux makes Sybase go through the page cache on
  reads, maybe we'll just have to switch to Solaris.  That's
  a serious performance problem.
 
 Thats something you'd have to benchmark. It depends on a very large number
 of factors including whether the database uses mmap, the average I/O size
 and the like

I'll probably benchmark raw vs. non-raw I/O with Sybase ASE 12.5
on our application once we've come up to speed on basic performance
issues (we're database newbies).
 
  It supports raw partitions, which is good; that might satisfy my
  boss (although the administration will be a pain, and I'm not
  sure whether it's really supported by Dell RAID devices).
  I'd prefer O_DIRECT :-(
 
 We already support raw direct I/O to devices themselves so they should support
 that - if not then Oracle I believe already does.

Haven't seen Sybase talk about O_DIRECT.  Not sure we want to
pony up the Sybase license fees.  (I'm still in denial about
databases in general, and hope I can switch to PostgreSQL
at some point.)

BTW, 
http://eval.veritas.com/webfiles/whitepapers/sybaseedition/sybase14_performance_paper.pdf
seems to show that raw beats O_DIRECT hands down on Solaris.
Will that hold on Linux, or is your (forthcoming?) O_DIRECT
higher performance than the one on Solaris?

Thanks,
Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

A signal fairy tale

[Dan Kegel]

Once upon a time a hacker named Xman
wrote a library that used aio, and decided
to use sigtimedwait() to pick up completion
notifications.  It worked well, and his I/O
was blazing fast (since was using a copy
of Linux that was patched to have good aio).
But when he tried to integrate his library
into a large application someone else had
written, woe! that application's use of signals
conflicted with his library.  "Fsck!" said Xman.
At that moment a fairy appeared, and said
"Young man, watch your language, or I'm going to
have to turn you into a goon!  I'm the good fairy Eunice.  
Can I help you?"  Xman explained his problem to Eunice,
who smiled and said "All you need is right here,
just type 'man 2 sigopen'".  Xman did, and saw:

 
SIGOPEN(2)Linux Programmer's Manual   SIGOPEN(2)
 
NAME
   sigopen - open a signal as a file descriptor
 
SYNOPSIS
   #include 
 
   int sigopen(int signum);
 
DESCRIPTION
   The sigopen system call opens signal number signum as a file descriptor.
   That signal is no longer delivered normally or available for pickup
   with sigwait() et al.  Instead, it must be picked up by calling
   read() on the file descriptor returned by sigwait(); the buffer passed to
   read() must have a size which is a multiple of sizeof(siginfo_t).
   Multiple signals may be picked up with a single call to read().
   When that file descriptor is closed, the signal is available once more 
   for traditional use.
   A signal number cannot be opened more than once concurrently; sigopen() 
   thus provides a way to avoid signal usage clashes in large programs.

RETURN VALUE
   signal returns the new file descriptor, or -1 on error (in which case, errno
   is set appropriately).

ERRORS
   EWOULDBLOCK signal is already open

NOTES
   read() will block when reading from a file descriptor opened by sigopen()
   until a signal is available unless fcntl(fd, F_SETFL, O_NONBLOCK) is called
   to set it into nonblocking mode.

HISTORY
   sigopen() first appeared in the 2.5.2 Linux kernel.

Linux  July, 2001 1   

When he finished reading, he knew just how to solve his
problem, and he lived happily ever after.  

The End.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

A signal fairy tale

[Dan Kegel]

Once upon a time a hacker named Xman
wrote a library that used aio, and decided
to use sigtimedwait() to pick up completion
notifications.  It worked well, and his I/O
was blazing fast (since was using a copy
of Linux that was patched to have good aio).
But when he tried to integrate his library
into a large application someone else had
written, woe! that application's use of signals
conflicted with his library.  Fsck! said Xman.
At that moment a fairy appeared, and said
Young man, watch your language, or I'm going to
have to turn you into a goon!  I'm the good fairy Eunice.  
Can I help you?  Xman explained his problem to Eunice,
who smiled and said All you need is right here,
just type 'man 2 sigopen'.  Xman did, and saw:

 
SIGOPEN(2)Linux Programmer's Manual   SIGOPEN(2)
 
NAME
   sigopen - open a signal as a file descriptor
 
SYNOPSIS
   #include signal.h
 
   int sigopen(int signum);
 
DESCRIPTION
   The sigopen system call opens signal number signum as a file descriptor.
   That signal is no longer delivered normally or available for pickup
   with sigwait() et al.  Instead, it must be picked up by calling
   read() on the file descriptor returned by sigwait(); the buffer passed to
   read() must have a size which is a multiple of sizeof(siginfo_t).
   Multiple signals may be picked up with a single call to read().
   When that file descriptor is closed, the signal is available once more 
   for traditional use.
   A signal number cannot be opened more than once concurrently; sigopen() 
   thus provides a way to avoid signal usage clashes in large programs.

RETURN VALUE
   signal returns the new file descriptor, or -1 on error (in which case, errno
   is set appropriately).

ERRORS
   EWOULDBLOCK signal is already open

NOTES
   read() will block when reading from a file descriptor opened by sigopen()
   until a signal is available unless fcntl(fd, F_SETFL, O_NONBLOCK) is called
   to set it into nonblocking mode.

HISTORY
   sigopen() first appeared in the 2.5.2 Linux kernel.

Linux  July, 2001 1   

When he finished reading, he knew just how to solve his
problem, and he lived happily ever after.  

The End.

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [OT] Threads, inelegance, and Java

[Dan Kegel]

Russell Leighton wrote:
> The lack of a good operating system _dependent_ interface
> makes running fast hard in Java when you need to do IO...
> yes, there is always JNI so you can add a little C to mmap a file or whatever,

JDK 1.4 beta comes with a way to memory-map files, and various
other I/O improvements (like poll(); see http://www.jcp.org/jsr/detail/51.jsp).
Chris Smith will have some nio benchmarks up on http://www.jlinux.org/
next week or so showing how well their nonblocking network I/O performs.

Sun is slowly getting their act together on the I/O front with java.
Still, the competition from C# and for that matter gcj will probably be 
a good thing, keep 'em on their toes.

(Disclaimer: I served on the JSR-51 expert group, representing the linux 
point of view, kinda.)
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [OT] Threads, inelegance, and Java

[Dan Kegel]

Russell Leighton wrote:
 The lack of a good operating system _dependent_ interface
 makes running fast hard in Java when you need to do IO...
 yes, there is always JNI so you can add a little C to mmap a file or whatever,

JDK 1.4 beta comes with a way to memory-map files, and various
other I/O improvements (like poll(); see http://www.jcp.org/jsr/detail/51.jsp).
Chris Smith will have some nio benchmarks up on http://www.jlinux.org/
next week or so showing how well their nonblocking network I/O performs.

Sun is slowly getting their act together on the I/O front with java.
Still, the competition from C# and for that matter gcj will probably be 
a good thing, keep 'em on their toes.

(Disclaimer: I served on the JSR-51 expert group, representing the linux 
point of view, kinda.)
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Alan Cox quote? (was: Re: accounting for threads)

[Dan Kegel]

[EMAIL PROTECTED] wrote:
> 
> On an unrelated note:
> 
> I noticed the quote below in your message. Is this a true quote or just a
> joke going around? I have tried believing it is just a joke but I am
> scared it is not.
> 
> >--
> > "A Computer is a state machine.
> >  Threads are for people who can't program state machines."
> >   - Alan Cox

Alan, did you really say that, or are people taking your name in vain?

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: getrusage vs /proc/pid/stat?

[Dan Kegel]

"J . A . Magallon" wrote:
> I have just the same problem. getrusage() did not catch the CPU time for
> children, even if the man page said that. Now I am using times(2), that
> seems to work in Solaris, but gives nothing in Linux.
> 
> I you look at time(1) manpage, it says time is implemented over the times(2)
> system call. But if I include that call, it gives me only zero.
>
> ? time gives good results for summed CPU time, but my own call
> to times(2) fails ???

It could be that you have to wait() for the child before times()
includes it in 'children time'.

By the way, the source for time is easy to find.  Here's debian's
(just search for time.c, then click on 'main'):
http://src.openresources.com/debian/src/utils/HTML/mains.html

If that doesn't help, maybe the code I sent you that reads /proc/pid/stat
for all threads of interest will.  Either way, let me know...
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: getrusage vs /proc/pid/stat?

[Dan Kegel]

J . A . Magallon wrote:
 I have just the same problem. getrusage() did not catch the CPU time for
 children, even if the man page said that. Now I am using times(2), that
 seems to work in Solaris, but gives nothing in Linux.
 
 I you look at time(1) manpage, it says time is implemented over the times(2)
 system call. But if I include that call, it gives me only zero.

 ? time gives good results for summed CPU time, but my own call
 to times(2) fails ???

It could be that you have to wait() for the child before times()
includes it in 'children time'.

By the way, the source for time is easy to find.  Here's debian's
(just search for time.c, then click on 'main'):
http://src.openresources.com/debian/src/utils/HTML/mains.html

If that doesn't help, maybe the code I sent you that reads /proc/pid/stat
for all threads of interest will.  Either way, let me know...
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Alan Cox quote? (was: Re: accounting for threads)

[Dan Kegel]

[EMAIL PROTECTED] wrote:
 
 On an unrelated note:
 
 I noticed the quote below in your message. Is this a true quote or just a
 joke going around? I have tried believing it is just a joke but I am
 scared it is not.
 
 --
  A Computer is a state machine.
   Threads are for people who can't program state machines.
- Alan Cox

Alan, did you really say that, or are people taking your name in vain?

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: getrusage vs /proc/pid/stat?

[Dan Kegel]

Pete Wyckoff wrote:
> 
> [EMAIL PROTECTED] said:
> > I'd like to monitor CPU, memory, and I/O utilization in a
> > long-running multithreaded daemon under kernels 2.2, 2.4,
> > and possibly also Solaris (#ifdefs are ok).
> 
> getrusage() isn't really the system call you want for this.

I'll buy that.  Looks like a lot of unices don't implement that
call fully, and Linux is one of them.

What is the proper way to measure total CPU time used by a multithreaded program?
The only way I can figure to do it is to sum /proc/pid/stat across
the threads of interest (see below).  Is this the easiest way, 
or am I missing something?  (Forgive the C++.  I'd recode this in C
if it were for general use.)

= LinuxTimes.h ==
#include 
#include 

/*--
 Source and test case at http://www.kegel.com/lt.tar.gz

 Monitor the CPU usage of a bunch of threads in the same process.
 This is a simulation of the system call times() 
 providing traditional semantics under LinuxThreads.
 On e.g. Solaris, you don't need this; you just call the standard times().
--*/
class LinuxTimes {
const static int MAXTHREADS = 100;

/// number of threads being monitored
int m_nthreads;

/// fd open to /proc/pid/stat for each thread
int m_proc_pid_stat_fd[MAXTHREADS];

/// make addSelf threadsafe
pthread_mutex_t m_lock;

public:

LinuxTimes() { m_nthreads = 0; pthread_mutex_init(_lock, NULL); }

/**
 New threads call this to add themselves to the group.
 Threadsafe.
 @return 0 on success, Unix error code on failure
 */
int addSelf();

/**
 Calculate user and system time accumulated by all threads in group.
 Return result in tms_utime and tms_stime fields of given struct tms.
 Similar to 'man 2 times' on Solaris (where all CPU time of all threads
 is counted as CPU time towards the process).
 @return 0 on success, Unix error code on failure
 */
int times(struct tms *buf);
};

= LinuxTimes.cc ==

#include "LinuxTimes.h"
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include   

/**
 New threads call this to add themselves to the group.
 Threadsafe.
 @return 0 on success, Unix error code on failure
 */
int LinuxTimes::addSelf()
{
int fd;
char fname[256];
int err = 0;

if (pthread_mutex_lock(_lock))
return EINVAL;

if (m_nthreads >= MAXTHREADS) {
err = E2BIG;
} else {
// Under LinuxThreads, each thread has its own pid
sprintf(fname, "/proc/%d/stat", getpid());
fd = open(fname, O_RDONLY);
if (fd == -1) 
err = errno;
else {
m_proc_pid_stat_fd[m_nthreads++] = fd;
}
}

if (pthread_mutex_unlock(_lock))
return EINVAL;

return err;
}

/* Skip to char after nth whitespace.  Returns NULL on failure. */
static const char *skipNspace(const char *p, int n)
{
while (*++p)
if (isspace(*p) && ! --n) 
return p+1;
return NULL;
}

/**
 Calculate user and system time accumulated by all threads in group.
 Return result in tms_utime and tms_stime fields of given struct tms.
 Similar to 'man 2 times' on Solaris (where all CPU time of all threads
 is counted as CPU time towards the process).
 @return 0 on success, Unix error code on failure
 */
int LinuxTimes::times(struct tms *buf)
{
int i;
int nread;

buf->tms_utime = 0;
buf->tms_stime = 0;
for (i=0; itms_utime += atoi(p);

// Skip to stime field
p = skipNspace(p, 1);
if (!p) return EINVAL;
buf->tms_stime += atoi(p);
}

return 0;
}

==

Thanks,
Dan

-- 
"A computer is a state machine.
 Threads are for people who can't program state machines."
 - Alan Cox
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: getrusage vs /proc/pid/stat?

[Dan Kegel]

Pete Wyckoff wrote:
 
 [EMAIL PROTECTED] said:
  I'd like to monitor CPU, memory, and I/O utilization in a
  long-running multithreaded daemon under kernels 2.2, 2.4,
  and possibly also Solaris (#ifdefs are ok).
 
 getrusage() isn't really the system call you want for this.

I'll buy that.  Looks like a lot of unices don't implement that
call fully, and Linux is one of them.

What is the proper way to measure total CPU time used by a multithreaded program?
The only way I can figure to do it is to sum /proc/pid/stat across
the threads of interest (see below).  Is this the easiest way, 
or am I missing something?  (Forgive the C++.  I'd recode this in C
if it were for general use.)

= LinuxTimes.h ==
#include sys/times.h
#include pthread.h

/*--
 Source and test case at http://www.kegel.com/lt.tar.gz

 Monitor the CPU usage of a bunch of threads in the same process.
 This is a simulation of the system call times() 
 providing traditional semantics under LinuxThreads.
 On e.g. Solaris, you don't need this; you just call the standard times().
--*/
class LinuxTimes {
const static int MAXTHREADS = 100;

/// number of threads being monitored
int m_nthreads;

/// fd open to /proc/pid/stat for each thread
int m_proc_pid_stat_fd[MAXTHREADS];

/// make addSelf threadsafe
pthread_mutex_t m_lock;

public:

LinuxTimes() { m_nthreads = 0; pthread_mutex_init(m_lock, NULL); }

/**
 New threads call this to add themselves to the group.
 Threadsafe.
 @return 0 on success, Unix error code on failure
 */
int addSelf();

/**
 Calculate user and system time accumulated by all threads in group.
 Return result in tms_utime and tms_stime fields of given struct tms.
 Similar to 'man 2 times' on Solaris (where all CPU time of all threads
 is counted as CPU time towards the process).
 @return 0 on success, Unix error code on failure
 */
int times(struct tms *buf);
};

= LinuxTimes.cc ==

#include LinuxTimes.h
#include ctype.h
#include errno.h
#include fcntl.h
#include stdio.h
#include stdlib.h
#include string.h
#include sys/types.h
#include unistd.h  

/**
 New threads call this to add themselves to the group.
 Threadsafe.
 @return 0 on success, Unix error code on failure
 */
int LinuxTimes::addSelf()
{
int fd;
char fname[256];
int err = 0;

if (pthread_mutex_lock(m_lock))
return EINVAL;

if (m_nthreads = MAXTHREADS) {
err = E2BIG;
} else {
// Under LinuxThreads, each thread has its own pid
sprintf(fname, /proc/%d/stat, getpid());
fd = open(fname, O_RDONLY);
if (fd == -1) 
err = errno;
else {
m_proc_pid_stat_fd[m_nthreads++] = fd;
}
}

if (pthread_mutex_unlock(m_lock))
return EINVAL;

return err;
}

/* Skip to char after nth whitespace.  Returns NULL on failure. */
static const char *skipNspace(const char *p, int n)
{
while (*++p)
if (isspace(*p)  ! --n) 
return p+1;
return NULL;
}

/**
 Calculate user and system time accumulated by all threads in group.
 Return result in tms_utime and tms_stime fields of given struct tms.
 Similar to 'man 2 times' on Solaris (where all CPU time of all threads
 is counted as CPU time towards the process).
 @return 0 on success, Unix error code on failure
 */
int LinuxTimes::times(struct tms *buf)
{
int i;
int nread;

buf-tms_utime = 0;
buf-tms_stime = 0;
for (i=0; im_nthreads; i++) {
char scratch[1024]; // FIXME: is that big enough?
int fd = m_proc_pid_stat_fd[i];

// rewind to start of stat file.  (Not all /proc entries support this.)
if (lseek(fd, 0, SEEK_SET))
return EBADF;

// Read in ASCII data and parse out utime and stime fields
// (see 'man proc')
nread = read(fd, scratch, sizeof(scratch)-1);
if (nread  50) // FIXME: cheesy
return EINVAL;
scratch[nread] = 0;

// Skip to end of command field
// FIXME: what if executable has ) in its filename?  Bleh.
const char *p = strchr(scratch, ')') + 2;

// Skip to utime field
p = skipNspace(p, 11);
if (!p) return EINVAL;
buf-tms_utime += atoi(p);

// Skip to stime field
p = skipNspace(p, 1);
if (!p) return EINVAL;
buf-tms_stime += atoi(p);
}

return 0;
}

==

Thanks,
Dan

-- 
A computer is a state machine.
 Threads are for people who can't program state machines.
 - Alan Cox
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  

getrusage vs /proc/pid/stat?

[Dan Kegel]

I'd like to monitor CPU, memory, and I/O utilization in a 
long-running multithreaded daemon under kernels 2.2, 2.4,
and possibly also Solaris (#ifdefs are ok).

getrusage() looked promising, and might even work for CPU utilization.
Dunno if it returns info for all child threads yet, haven't tried it.
In Linux, though, getrusage() doesn't return any info about RAM.

I know I can get the RSS and VSIZE under Linux by parsing /proc/pid/stat,
but was hoping for a faster interface (although I suppose a seek,
a read, and an ascii parse isn't *that* slow).  Is /proc/pid/stat
the only way to go under Linux to monitor RSS?
- Dan

-- 
"A computer is a state machine.
 Threads are for people who can't program state machines."
 - Alan Cox
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

getrusage vs /proc/pid/stat?

[Dan Kegel]

I'd like to monitor CPU, memory, and I/O utilization in a 
long-running multithreaded daemon under kernels 2.2, 2.4,
and possibly also Solaris (#ifdefs are ok).

getrusage() looked promising, and might even work for CPU utilization.
Dunno if it returns info for all child threads yet, haven't tried it.
In Linux, though, getrusage() doesn't return any info about RAM.

I know I can get the RSS and VSIZE under Linux by parsing /proc/pid/stat,
but was hoping for a faster interface (although I suppose a seek,
a read, and an ascii parse isn't *that* slow).  Is /proc/pid/stat
the only way to go under Linux to monitor RSS?
- Dan

-- 
A computer is a state machine.
 Threads are for people who can't program state machines.
 - Alan Cox
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: disk-based fds in select/poll

[Dan Kegel]

Pierre Phaneuf <[EMAIL PROTECTED]> wrote:
> It's fairly widely-known that select/poll returns immediately when
> testing a filesystem-based file descriptor for writability or
> readability.
> 
> On top of this, even when in non-blocking mode, read() could block if
> the pages needed aren't in core. sendfile() behaves in a similar way.
> 
> What would be needed to alleviate this?
> ...
> I remember seeing a suggestion by Linus for an event-based I/O
> interface, similar to kqueue on FreeBSD but much simpler. I'd just say
> "I want it too!", ok? :-)
>
> SGI's AIO might be a solution here, does it use threads? I'm trying to
> avoid context switching as much as possible, to keep the CPU cache as
> warm as possible.

IMHO, you want AIO.  SGI's is fine for now.  I hear rumors that there will be
something even better coming in 2.5, though I have no details.

Or you could use explicit userspace threads... say, divide up your
network connections among 8 or so threads.  Then if one thread blocks,
the others are there to usefully soak up the CPU time.

Readiness events for readahead completion on disk files used to 
seem like a neat idea to me, but now AIO seems more appealing
in the long run, since they handle random access properly.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

re: disk-based fds in select/poll

[Dan Kegel]

Pierre Phaneuf [EMAIL PROTECTED] wrote:
 It's fairly widely-known that select/poll returns immediately when
 testing a filesystem-based file descriptor for writability or
 readability.
 
 On top of this, even when in non-blocking mode, read() could block if
 the pages needed aren't in core. sendfile() behaves in a similar way.
 
 What would be needed to alleviate this?
 ...
 I remember seeing a suggestion by Linus for an event-based I/O
 interface, similar to kqueue on FreeBSD but much simpler. I'd just say
 I want it too!, ok? :-)

 SGI's AIO might be a solution here, does it use threads? I'm trying to
 avoid context switching as much as possible, to keep the CPU cache as
 warm as possible.

IMHO, you want AIO.  SGI's is fine for now.  I hear rumors that there will be
something even better coming in 2.5, though I have no details.

Or you could use explicit userspace threads... say, divide up your
network connections among 8 or so threads.  Then if one thread blocks,
the others are there to usefully soak up the CPU time.

Readiness events for readahead completion on disk files used to 
seem like a neat idea to me, but now AIO seems more appealing
in the long run, since they handle random access properly.

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux scalability?

[Dan Kegel]

Sean Hunter wrote:
> On Sat, May 19, 2001 at 10:31:01AM +0200, Sasi Peter wrote: 
> > On Fri, 18 May 2001, Sean Hunter wrote: 
> > 
> > > Why would you want to run a web server with 8 processors rather than four 
> > > webservers with 2 each? 
> > 
> > As you might already know, after the interviews to Mingo I assumed, that a 
> > major portion of the achievements was enabled by the 2.4 scalability 
> > enhacements. That is why I wrote to LKML, to ask about the 2.4 
> > scalability, if anybody out there could tell us about the linux kernel's 
> > scalability possibily compared to W2k scalability... 
>
> Yup. The problem is that you're trying to measure scalability in performance 
> of an i/o-bound task by comparing a machine with greater i/o resource but less 
> processing power with one with greater processing but poorer i/o. Surprisingly 
> enough, the one with the best i/o wins. This isn't really a fair comparison 
> between the two platforms. 

The document tree (21 - 26 GB) is small enough to fit in RAM (32 GB),
so the speed of the disk is not likely to have a noticable impact.
(See http://boudicca.tux.org/hypermail/linux-kernel/2001week20/1276.html )

A lot of people during the Mindcraft discussion made the mistake
of calling the test unfair.
Regardless of whether the initial test was fair, it actually showed 
interesting performance weaknesses in Linux, ones the kernel team
has successfully addressed.

> My point was that in the real world having this configuration for a webserver 
> is unlikely to be sensible at all. 

That's certainly true.  On the other hand, worrying about how many
nanoseconds a system call takes isn't really an issue in the
real world, but kernel hackers love to optimize system call overhead
anyway.  This is the same sort of intellectual challenge.  Plus,
it impresses the beancounters, and they're the ones who buy the
systems and keep us all employed.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux scalability?

[Dan Kegel]

Sean Hunter wrote:
 On Sat, May 19, 2001 at 10:31:01AM +0200, Sasi Peter wrote: 
  On Fri, 18 May 2001, Sean Hunter wrote: 
  
   Why would you want to run a web server with 8 processors rather than four 
   webservers with 2 each? 
  
  As you might already know, after the interviews to Mingo I assumed, that a 
  major portion of the achievements was enabled by the 2.4 scalability 
  enhacements. That is why I wrote to LKML, to ask about the 2.4 
  scalability, if anybody out there could tell us about the linux kernel's 
  scalability possibily compared to W2k scalability... 

 Yup. The problem is that you're trying to measure scalability in performance 
 of an i/o-bound task by comparing a machine with greater i/o resource but less 
 processing power with one with greater processing but poorer i/o. Surprisingly 
 enough, the one with the best i/o wins. This isn't really a fair comparison 
 between the two platforms. 

The document tree (21 - 26 GB) is small enough to fit in RAM (32 GB),
so the speed of the disk is not likely to have a noticable impact.
(See http://boudicca.tux.org/hypermail/linux-kernel/2001week20/1276.html )

A lot of people during the Mindcraft discussion made the mistake
of calling the test unfair.
Regardless of whether the initial test was fair, it actually showed 
interesting performance weaknesses in Linux, ones the kernel team
has successfully addressed.

 My point was that in the real world having this configuration for a webserver 
 is unlikely to be sensible at all. 

That's certainly true.  On the other hand, worrying about how many
nanoseconds a system call takes isn't really an issue in the
real world, but kernel hackers love to optimize system call overhead
anyway.  This is the same sort of intellectual challenge.  Plus,
it impresses the beancounters, and they're the ones who buy the
systems and keep us all employed.

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux scalability?

[Dan Kegel]

Sasi Peter <[EMAIL PROTECTED]> wrote:
> I am just writing an essay, an have mentioned TUX as a performance and 
> scalability linearity recort holder with TUX, referencing the specweb99 
> website summary page: 
> 
> http://www.spec.org/osg/web99/results/web99.html 
> 
> However, taking a closer look, it turns out, that the above statement 
> holds true only for 1 and 2 processor machines. Scalability already 
> suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten 
> by IIS 5.0 (8001), and these were measured on the same kind of box! 
>
> How come, TUX is s good at the lowend (1 and 2 CPUs), and scales this 
> bad? 

Let's look at the scores.  (BTW, SPECweb99 gets harder
as the scores get better; the document tree required to achieve a score of
3222 is twice as large as that required to achieve a score of 1438.)

  SPECweb99 result summary:
date#cpu  #nics L2 cache/cpu  RAM  tree score  sw   modelMHz
1/2001  1 1 256K  2G5G  1438   tux2 Compq Proliant DL320 800
6/2000  1 1 256K  2G4G  1270   tux1 Dell Poweredge 6400  667
6/2000  2 2 256K  4G7G  2200   tux1 Dell Poweredge 4400  800
3/2001  2 4 256K  4G10G 3222   tux2 Dell Poweredge 2500  1000

2/2001  1 3 2M8G9G  2700   tux2 IBM xSeries 370  900
2/2001  2 4 2M16G   13G 3999   tux2 IBM xSeries 370  900
6/2000  4 4 2M8G14G 4200   tux1 Dell Poweredge 6400  700
7/2000  8 8 2M32G   21G 6387   tux1 Dell Poweredge 8450  700
11/2000 8 8 2M32G   24G 7500   tux2 Dell Poweredge 8450  700
12/2000 8 8 2M32G   21G 6407   tux1 IBM Netfinity 8500R  700

3/2001  2 3 256K  4G8G  2499   IIS5/SWC HP NetserverLP2000r  1000
4/2001  8 8 2M32G   26G 8000   IIS5/SWC Dell Poweredge 8450  700

IIS5/SWC only has two results on record, at 2 and 8 CPUs.  They're hard
to compare, because they have different cache and RAM sizes and CPU speeds,
but it's safe to say that it performs poorly at 2 CPUs (compared to the 3/2001 
results from Dell) and scales nearly linearly to perform comparatively well at 8 CPUs.

Looking at the IBM 1 and 2 CPU results, twice the CPU only got 1.4 times
the performance.  Not sure TUX is scaling especially well even at 2 CPU's.
(And you can't blame this on disk drives, please don't try.)

So I agree, Tux doesn't seem to scale as well to multiple CPUs as does IIS5/SWC.

About comparing the Tux and IIS/SWC results on the Dell 8 CPU box:
the Tux measurement is 5 months older than the IIS/SWC measurement.
It's interesting to speculate how tux2 would do if tested today; 
It looks like tux2 is about 12% faster than tux1 on 8-CPU machines.
In other words, 5 months of further development on tux and the 2.4 kernel yielded 
a 12% speedup.  Since IIS was only 4% faster than TUX, If Tux were measured today, 
it might have improved enough to beat IIS/SWC, who knows.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux scalability?

[Dan Kegel]

Sasi Peter [EMAIL PROTECTED] wrote:
 I am just writing an essay, an have mentioned TUX as a performance and 
 scalability linearity recort holder with TUX, referencing the specweb99 
 website summary page: 
 
 http://www.spec.org/osg/web99/results/web99.html 
 
 However, taking a closer look, it turns out, that the above statement 
 holds true only for 1 and 2 processor machines. Scalability already 
 suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten 
 by IIS 5.0 (8001), and these were measured on the same kind of box! 

 How come, TUX is s good at the lowend (1 and 2 CPUs), and scales this 
 bad? 

Let's look at the scores.  (BTW, SPECweb99 gets harder
as the scores get better; the document tree required to achieve a score of
3222 is twice as large as that required to achieve a score of 1438.)

  SPECweb99 result summary:
date#cpu  #nics L2 cache/cpu  RAM  tree score  sw   modelMHz
1/2001  1 1 256K  2G5G  1438   tux2 Compq Proliant DL320 800
6/2000  1 1 256K  2G4G  1270   tux1 Dell Poweredge 6400  667
6/2000  2 2 256K  4G7G  2200   tux1 Dell Poweredge 4400  800
3/2001  2 4 256K  4G10G 3222   tux2 Dell Poweredge 2500  1000

2/2001  1 3 2M8G9G  2700   tux2 IBM xSeries 370  900
2/2001  2 4 2M16G   13G 3999   tux2 IBM xSeries 370  900
6/2000  4 4 2M8G14G 4200   tux1 Dell Poweredge 6400  700
7/2000  8 8 2M32G   21G 6387   tux1 Dell Poweredge 8450  700
11/2000 8 8 2M32G   24G 7500   tux2 Dell Poweredge 8450  700
12/2000 8 8 2M32G   21G 6407   tux1 IBM Netfinity 8500R  700

3/2001  2 3 256K  4G8G  2499   IIS5/SWC HP NetserverLP2000r  1000
4/2001  8 8 2M32G   26G 8000   IIS5/SWC Dell Poweredge 8450  700

IIS5/SWC only has two results on record, at 2 and 8 CPUs.  They're hard
to compare, because they have different cache and RAM sizes and CPU speeds,
but it's safe to say that it performs poorly at 2 CPUs (compared to the 3/2001 
results from Dell) and scales nearly linearly to perform comparatively well at 8 CPUs.

Looking at the IBM 1 and 2 CPU results, twice the CPU only got 1.4 times
the performance.  Not sure TUX is scaling especially well even at 2 CPU's.
(And you can't blame this on disk drives, please don't try.)

So I agree, Tux doesn't seem to scale as well to multiple CPUs as does IIS5/SWC.

About comparing the Tux and IIS/SWC results on the Dell 8 CPU box:
the Tux measurement is 5 months older than the IIS/SWC measurement.
It's interesting to speculate how tux2 would do if tested today; 
It looks like tux2 is about 12% faster than tux1 on 8-CPU machines.
In other words, 5 months of further development on tux and the 2.4 kernel yielded 
a 12% speedup.  Since IIS was only 4% faster than TUX, If Tux were measured today, 
it might have improved enough to beat IIS/SWC, who knows.

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [QUESTION] "No 100Hz timer" thread - any LKML archive?

[Dan Kegel]

Tomas Telensky ([EMAIL PROTECTED]) wrote:
> does anybody know about any archive/digest service for this mailing 
> list? Majordomo at vger doesn't support this. Or does anybody of you 
> archive all e-mails? 
> 
> I'm especially interested in the _WHOLE_ thread "No 100Hz timer !" now. 
> (but the question is asked in general). If you have the complete thread, 
> I would be very glad if you could expose it on WWW for me to download. 
> 
> I've met different better or worse http interfaces to archives 
> (included www.mail-archive.com). But I'm not satisfied with them - no 
> header's, I must wget... I want to have a folder of e-mails and sort 
> them by threads in mutt - this is only possible when headers are 
> preserved. 

See http://marc.theaimsgroup.com/?l=linux-kernel for a good search tool.
Dunno about raw archives...

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [QUESTION] No 100Hz timer thread - any LKML archive?

[Dan Kegel]

Tomas Telensky ([EMAIL PROTECTED]) wrote:
 does anybody know about any archive/digest service for this mailing 
 list? Majordomo at vger doesn't support this. Or does anybody of you 
 archive all e-mails? 
 
 I'm especially interested in the _WHOLE_ thread No 100Hz timer ! now. 
 (but the question is asked in general). If you have the complete thread, 
 I would be very glad if you could expose it on WWW for me to download. 
 
 I've met different better or worse http interfaces to archives 
 (included www.mail-archive.com). But I'm not satisfied with them - no 
 header's, I must wget... I want to have a folder of e-mails and sort 
 them by threads in mutt - this is only possible when headers are 
 preserved. 

See http://marc.theaimsgroup.com/?l=linux-kernel for a good search tool.
Dunno about raw archives...

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: bug in redhat gcc 2.96

[Dan Kegel]

Stefan Hoffmeister wrote:
> >Since there is no kgcc in RH71,
> 
> There is an compat-egcs RPM (on CD2?) that contains kgcc. Took me a while
> to find that.

OH.  I kept looking for a package called 'kgcc'.  Silly me.

Guess it's time for a "How to compile a kernel on Red Hat 7.1" FAQ.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: bug in redhat gcc 2.96

[Dan Kegel]

Stefan Hoffmeister wrote:
 Since there is no kgcc in RH71,
 
 There is an compat-egcs RPM (on CD2?) that contains kgcc. Took me a while
 to find that.

OH.  I kept looking for a package called 'kgcc'.  Silly me.

Guess it's time for a How to compile a kernel on Red Hat 7.1 FAQ.
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: fs.file-max

[Dan Kegel]

Federico Edelman Anaya ([EMAIL PROTECTED]) wrote:

> What can I do to test the FD limit? ... Because, the FD limit is set in 
> /proc/sys/fs/file-max, sample: 
> 
> echo "2048" > /proc/sys/fs/file-max 

That sets the systemwide limit to 2048.  

> ulimit -n 8192 

That sets the per-process limit (for this process
and its children) to 2048.  
 
> In this case ... the FD limit = 8192 :( ... when the limit should be 
> 2048? 

No, the two limits are independant (except, obviously, that
that process will reach the systemwide fd limit before it
exhausts its per-process fd limit).
 
> I wrote a perl script for the test ... anybody known a "C" program for 
> test the FD limit? 

http://www.kegel.com/dkftpbench/#tuning

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: fs.file-max

[Dan Kegel]

Federico Edelman Anaya ([EMAIL PROTECTED]) wrote:

 What can I do to test the FD limit? ... Because, the FD limit is set in 
 /proc/sys/fs/file-max, sample: 
 
 echo 2048  /proc/sys/fs/file-max 

That sets the systemwide limit to 2048.  

 ulimit -n 8192 

That sets the per-process limit (for this process
and its children) to 2048.  
 
 In this case ... the FD limit = 8192 :( ... when the limit should be 
 2048? 

No, the two limits are independant (except, obviously, that
that process will reach the systemwide fd limit before it
exhausts its per-process fd limit).
 
 I wrote a perl script for the test ... anybody known a C program for 
 test the FD limit? 

http://www.kegel.com/dkftpbench/#tuning

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Whether can we put our company's linux driver into linux kernel?

[Dan Kegel]

[EMAIL PROTECTED] wrote:
> I want to contact with the author of linux kernel. 
> Anybody knows how to contact with them? 
> Our leader hope put our own driver into linux kernel. I am not sure whether 
> it was permitted. 
> So, I need to contact with the authors of linux kernel. 
> If you know how to do it, please tell me. 

Hi Yiping,
the linux kernel community is happy to accept drivers from Via!

It's easiest if Via agrees to license the source code
for the driver under the GPL (see http://www.fsf.org/copyleft/gpl.html )
If that's ok, then just put the source code up as a .tar file on a
web or ftp server, and send an email to the linux-kernel mailing list
explaining where it is and how to use it.

See also http://www.linuxhq.com/kernel/v2.4/doc/SubmittingDrivers.html

Thanks,
Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Whether can we put our company's linux driver into linux kernel?

[Dan Kegel]

[EMAIL PROTECTED] wrote:
 I want to contact with the author of linux kernel. 
 Anybody knows how to contact with them? 
 Our leader hope put our own driver into linux kernel. I am not sure whether 
 it was permitted. 
 So, I need to contact with the authors of linux kernel. 
 If you know how to do it, please tell me. 

Hi Yiping,
the linux kernel community is happy to accept drivers from Via!

It's easiest if Via agrees to license the source code
for the driver under the GPL (see http://www.fsf.org/copyleft/gpl.html )
If that's ok, then just put the source code up as a .tar file on a
web or ftp server, and send an email to the linux-kernel mailing list
explaining where it is and how to use it.

See also http://www.linuxhq.com/kernel/v2.4/doc/SubmittingDrivers.html

Thanks,
Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: just-in-time debugging?

[Dan Kegel]

 Tony Hoyle ([EMAIL PROTECTED]) wrote:
> Is there a way that gdb/ddd could be invoked when a program is about 
> to dump core...?

Yes, I use that to get a symbolic stack dump after a crash,
although I find that the gdb so invoked doesn't accept interactive
commands, and I have to use 'kill -9' afterwards.
Here's the code I use:

void dump_stack(int signum)
{
(void) signum;
char s[160];
// The right command to execute depends on the program.  Adjust to taste.
system("echo 'info threads\nbt\nthread 3\nbt\nthread 4\nbt\nthread 5\nbt\n' > 
gdbcmd");
 
sprintf(s, "gdb -batch -x gdbcmd %s %d", argv0, (int) getpid());
printf("Crashed!  Starting debugger to get stack dump.  You may need to kill -9 
this process afterwards.\n");
system(s);
exit(1);
 
} 

main() {
   signal(SIGSEGV, dump_stack);
   signal(SIGBUS, dump_stack);   
...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: just-in-time debugging?

[Dan Kegel]

 Tony Hoyle ([EMAIL PROTECTED]) wrote:
 Is there a way that gdb/ddd could be invoked when a program is about 
 to dump core...?

Yes, I use that to get a symbolic stack dump after a crash,
although I find that the gdb so invoked doesn't accept interactive
commands, and I have to use 'kill -9' afterwards.
Here's the code I use:

void dump_stack(int signum)
{
(void) signum;
char s[160];
// The right command to execute depends on the program.  Adjust to taste.
system(echo 'info threads\nbt\nthread 3\nbt\nthread 4\nbt\nthread 5\nbt\n'  
gdbcmd);
 
sprintf(s, gdb -batch -x gdbcmd %s %d, argv0, (int) getpid());
printf(Crashed!  Starting debugger to get stack dump.  You may need to kill -9 
this process afterwards.\n);
system(s);
exit(1);
 
} 

main() {
   signal(SIGSEGV, dump_stack);
   signal(SIGBUS, dump_stack);   
...
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: FD in Kernel 2.4.x

[Dan Kegel]

> How can I increase the FD in the Kernel 2.4.3? 

echo 32768 > /proc/sys/fs/file-max

See also http://www.kegel.com/c10k.html#limits.filehandles

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: FD in Kernel 2.4.x

[Dan Kegel]

 How can I increase the FD in the Kernel 2.4.3? 

echo 32768  /proc/sys/fs/file-max

See also http://www.kegel.com/c10k.html#limits.filehandles

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Mark Hahn wrote:
> the main goal at this point is to make kernel proc-related 
> code more efficient, easy-to-use, etc. a purely secondary goal 
> is to make user-space tools more robust, efficient, and simpler. 
> 
> there are three things that need to be communicated through the proc 
> interface, for each chunk of data: its type, it's name and its value. 
> it's critical that data be tagged in some way, since that's the only 
> way to permit back-compatibility. that is, a tool looking for a particular 
> tag will naturally ignore new data with other tags. 

Agreed.

> [three example schemes in use in /proc today]
> I have a sense that all of these could be collapsed into a single 
> api where kernel systems would register hierarchies of tuples of 
> , where callback would be passed the tag, 
> and proc code would take care of "rendering" the data into 
> human readable text (default), binary, or even xml. 

Sounds reasonable to me.  Relieve the modules of having to
format their /proc entries by defining standard code that does
it.   And as an extra bonus, if tuples registration was table-driven,
the tables would define a grammar that could be fed to a parser
generator.

(It sounds a little bit like the snmpd code I'm working on,
actually.  How eerie.)

(It also sounds a little like (gasp) the windows registry,
but hey, that's ok.)

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Jesse Pollard wrote:
> > But one thing XML provides (potentially) is a DTD that defines meanings and 
>formats.
> > IMHO the kernel needs something like this for /proc (though not in DTD format!).
> >
> > Has anyone ever tried to write a formal syntax for all the entries
> > in /proc?   We have bits and pieces of /proc documentation in
> > /usr/src/linux/Documentation, but nothing you could feed directly
> > into a parser generator.  It'd be neat to have a good definition for /proc
> > in the LSB, and have an LSB conformance test that could look in
> > /proc and say "Yup, all the entries there conform to the spec and can
> > be parsed properly."...
> 
> From one point of view (that of the /proc entries...) each file
> is by definition in the proper format. That format is specified
> (in the /proc interface to the driver). Using "proc_printf" is a
> specification for the output.

When two different distributions ship different forks of
the kernel source, which differ in the arguments passed to proc_printf,
which one is right?  
There's no way to tell.  That's why saying "the source is the spec" doesn't cut it.

Also, the source is not a specification a parser generator can use.

A formal spec for /proc entries maintained by e.g. the LSB is needed;
it has to be separate from the source code (to avoid forking problems),
and it should be machine-readable (so we can build parsers from it).

> That DOES NOT mean that no improvements are possible. If the formats
> used by the various modules/drivers has some variation in format from
> access to acess, then the determination of that format must also be
> included. From what I've seen (via "cat /proc/") the files all
> have a fixed format. Sometimes the number of entries varies, but then
> the count should ALSO be included in the file (in a known place of
> course). The multi-entry files I've looked at (/proc/net) reach the
> EOF to end the list. This is not unreasonable.

Yeah, there's a general style that seems to work; it just needs to be
formalized.
 
> I'm not sure of the usefullness of the title lines that are printed. If
> looked at in raw form, yes the titles are nice. But the utilities
> that are aimed at examining the values should not have to discard them, nor
> should the drivers have to generate them.

I think they're good; they're a little bit like the XML tags you're proposing.
 
> I can live with them anyway, since they are already there.
> 
> The biggest problem I know of is being able to retrieve structure
> in an atomic manner. Not easy (in any system, not just Linux).

Something SNMP doesn't deal well with, either.  People seem to cope,
though.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Jesse Pollard wrote:
> Personally, I think
> proc_printf(fragment, "%d %d",get_portnum(usbdev), usbdev->maxchild);
> (or the string " ddd" with d representing a digit)
> 
> is shorter (and faster) to parse with
> fscanf(input,"%d %d",,);
> 
> Than it would be to try parsing
> 
> with an XML parser.
> 
> Sorry - XML is good for some things. It is not designed to be a
> interface language between a kernel and user space.
> 
> I am NOT in favor of "one file per value", but structured data needs
> to be written in a reasonable, concise manner. XML is intended for
> communication between disparate systems in an exreemly precise manner
> to allow some self documentation to be included when the communication
> fails.

Agreed.  

But one thing XML provides (potentially) is a DTD that defines meanings and formats.  
IMHO the kernel needs something like this for /proc (though not in DTD format!).

Has anyone ever tried to write a formal syntax for all the entries
in /proc?   We have bits and pieces of /proc documentation in 
/usr/src/linux/Documentation, but nothing you could feed directly 
into a parser generator.  It'd be neat to have a good definition for /proc
in the LSB, and have an LSB conformance test that could look in
/proc and say "Yup, all the entries there conform to the spec and can
be parsed properly."

(http://www.pathname.com/fhs/2.2-beta/fhs-2.2-beta.txt mentions /proc,
but doesn't standardize any of it, except to suggest that /etc/mtab
can be a symbolic link to /proc/mounts.)
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Tim Jansen wrote:
> 
> On Wednesday 25 April 2001 19:10, you wrote:
> > The command
> >   more foo/* foo/*/*
> > will display the values in the foo subtree nicely, I think.
> 
> Unfortunately it displays only the values. Dumping numbers and strings
> without knowing their meaning (and probably not even the order) is not very
> useful.

The meanings should be implied by the filenames, which are displayed (try it).
The order is alphabetical by filename.

> But the one-value per file approach is MORE work. It would be less work to
> create XML and factor out the directory structure in user-space :)
> Devreg collects its data from the drivers, each driver should contribute the
> information that it can provide about the device.
> Printing a few values in XML format using the functions from xmlprocfs is as
> easy as writing
> proc_printf(fragment, "\n",
> get_portnum(usbdev), usbdev->maxchild);

The corresponding one-value-per-file approach can probably be made to
be a single call per value.  IMHO that's more useful; it means that
(once we agree on definitions) programs don't need to parse XML to
access this data; they can go straight to the node in the document object
model tree ( = /proc ).  Think of /proc as a preparsed XML tree
that hasn't been standardized yet.
 
> The code is easy to read and not larger than a solution that creates static
> /proc entries, and holding the data completely static would take much more
> memory. And it takes less code than a solution that would create the values
> in /proc dynamically because this would mean one callback per file or a
> complicated way to specify several values with a single callback.

... but XML parsing is something we don't want to force on people
when we can provide the same data in a pre-parsed, much easier to access
form, IMHO.

Have you bothered to go back and read the old discussions on this topic?

> The driver should use its 
> own XML namespace, so whatever the driver adds will not break any 
> (well-written) user-space applications.

Are you trying to avoid writing a DTD?  IMHO it would be better to
have a single DTD for the entire tree, rather than a separate 
anything-goes namespace for each driver.  Yes, this is more work,
but all the Linux drivers are tightly integrated into the kernel
source tree, we may as well have a tightly-integrated DTD documenting
what each block, serial, synch, etc. driver must provide.

I think we both agree that there needs to be an easy, standardized way
to access this data.  IMHO there's a lot of standardizing that needs
to happen before you can start writing code -- otherwise your new code
won't help, and we'll be in the same mess we're in now.

The DTD can apply to both the existing /proc form and any proposed XML form
of config info exported by the kernel; there should be an easy transformation
between them.  And it has to come first!

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Device Registry (DevReg) Patch 0.2.0

[Dan Kegel]

Tim Jansen wrote:
> On Tuesday 24 April 2001 18:39, Martin Dalecki wrote: 
> >> Are there alternatives to get complex and extendable information out to 
> >> user space? 
> > Yes filesystem structures. 
> 
> How exactly can this work? A single value per file is not very helpful if you 
> have a thousand values. You could cluster them (for example one level in the 
> XML hierarchy == one file), but this will soon get very complicated. Its much 
> more work to implement in the kernel, its painful in user-space and you cant 
> just use a text editor to look at it (because you always have to look at 10 
> files per device). 

The command
  more foo/* foo/*/* 
will display the values in the foo subtree nicely, I think.

Think of the /proc tree as the XML parse tree already exploded for you.

The only problem with /proc as it stands is that there is no formal
syntax for its entries.  Some of them are hard to parse.

Before we add a new /proc entry that generates XML which summarizes
the rest of /proc, it might make sense to standardize /proc entries
and write a regression test to verify they are formatted correctly.
It would then be trivial to write a /proc to XML converter which
ran solely in userspace.

See 
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.0/0506.html
and
http://marc.theaimsgroup.com/?l=linux-kernel=%2Fproc+xml

for prior discussion on the matter.

I don't want to dismiss the reasons you want to use XML for this,
but tread carefully, lest you duplicate lots of code and introduce
cruft.  Better to factor the XML part out to a userspace library...

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Device Registry (DevReg) Patch 0.2.0

[Dan Kegel]

Tim Jansen wrote:
 On Tuesday 24 April 2001 18:39, Martin Dalecki wrote: 
  Are there alternatives to get complex and extendable information out to 
  user space? 
  Yes filesystem structures. 
 
 How exactly can this work? A single value per file is not very helpful if you 
 have a thousand values. You could cluster them (for example one level in the 
 XML hierarchy == one file), but this will soon get very complicated. Its much 
 more work to implement in the kernel, its painful in user-space and you cant 
 just use a text editor to look at it (because you always have to look at 10 
 files per device). 

The command
  more foo/* foo/*/* 
will display the values in the foo subtree nicely, I think.

Think of the /proc tree as the XML parse tree already exploded for you.

The only problem with /proc as it stands is that there is no formal
syntax for its entries.  Some of them are hard to parse.

Before we add a new /proc entry that generates XML which summarizes
the rest of /proc, it might make sense to standardize /proc entries
and write a regression test to verify they are formatted correctly.
It would then be trivial to write a /proc to XML converter which
ran solely in userspace.

See 
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0101.0/0506.html
and
http://marc.theaimsgroup.com/?l=linux-kernels=%2Fproc+xml

for prior discussion on the matter.

I don't want to dismiss the reasons you want to use XML for this,
but tread carefully, lest you duplicate lots of code and introduce
cruft.  Better to factor the XML part out to a userspace library...

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Tim Jansen wrote:
 
 On Wednesday 25 April 2001 19:10, you wrote:
  The command
more foo/* foo/*/*
  will display the values in the foo subtree nicely, I think.
 
 Unfortunately it displays only the values. Dumping numbers and strings
 without knowing their meaning (and probably not even the order) is not very
 useful.

The meanings should be implied by the filenames, which are displayed (try it).
The order is alphabetical by filename.

 But the one-value per file approach is MORE work. It would be less work to
 create XML and factor out the directory structure in user-space :)
 Devreg collects its data from the drivers, each driver should contribute the
 information that it can provide about the device.
 Printing a few values in XML format using the functions from xmlprocfs is as
 easy as writing
 proc_printf(fragment, usb:topology port=\%d\ portnum=\%d\/\n,
 get_portnum(usbdev), usbdev-maxchild);

The corresponding one-value-per-file approach can probably be made to
be a single call per value.  IMHO that's more useful; it means that
(once we agree on definitions) programs don't need to parse XML to
access this data; they can go straight to the node in the document object
model tree ( = /proc ).  Think of /proc as a preparsed XML tree
that hasn't been standardized yet.
 
 The code is easy to read and not larger than a solution that creates static
 /proc entries, and holding the data completely static would take much more
 memory. And it takes less code than a solution that would create the values
 in /proc dynamically because this would mean one callback per file or a
 complicated way to specify several values with a single callback.

... but XML parsing is something we don't want to force on people
when we can provide the same data in a pre-parsed, much easier to access
form, IMHO.

Have you bothered to go back and read the old discussions on this topic?

 The driver should use its 
 own XML namespace, so whatever the driver adds will not break any 
 (well-written) user-space applications.

Are you trying to avoid writing a DTD?  IMHO it would be better to
have a single DTD for the entire tree, rather than a separate 
anything-goes namespace for each driver.  Yes, this is more work,
but all the Linux drivers are tightly integrated into the kernel
source tree, we may as well have a tightly-integrated DTD documenting
what each block, serial, synch, etc. driver must provide.

I think we both agree that there needs to be an easy, standardized way
to access this data.  IMHO there's a lot of standardizing that needs
to happen before you can start writing code -- otherwise your new code
won't help, and we'll be in the same mess we're in now.

The DTD can apply to both the existing /proc form and any proposed XML form
of config info exported by the kernel; there should be an easy transformation
between them.  And it has to come first!

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Jesse Pollard wrote:
 Personally, I think
 proc_printf(fragment, %d %d,get_portnum(usbdev), usbdev-maxchild);
 (or the string  ddd with d representing a digit)
 
 is shorter (and faster) to parse with
 fscanf(input,%d %d,usbdev,maxchild);
 
 Than it would be to try parsing
 usb:topology port=d portnum=
 with an XML parser.
 
 Sorry - XML is good for some things. It is not designed to be a
 interface language between a kernel and user space.
 
 I am NOT in favor of one file per value, but structured data needs
 to be written in a reasonable, concise manner. XML is intended for
 communication between disparate systems in an exreemly precise manner
 to allow some self documentation to be included when the communication
 fails.

Agreed.  

But one thing XML provides (potentially) is a DTD that defines meanings and formats.  
IMHO the kernel needs something like this for /proc (though not in DTD format!).

Has anyone ever tried to write a formal syntax for all the entries
in /proc?   We have bits and pieces of /proc documentation in 
/usr/src/linux/Documentation, but nothing you could feed directly 
into a parser generator.  It'd be neat to have a good definition for /proc
in the LSB, and have an LSB conformance test that could look in
/proc and say Yup, all the entries there conform to the spec and can
be parsed properly.

(http://www.pathname.com/fhs/2.2-beta/fhs-2.2-beta.txt mentions /proc,
but doesn't standardize any of it, except to suggest that /etc/mtab
can be a symbolic link to /proc/mounts.)
- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: /proc format (was Device Registry (DevReg) Patch 0.2.0)

[Dan Kegel]

Mark Hahn wrote:
 the main goal at this point is to make kernel proc-related 
 code more efficient, easy-to-use, etc. a purely secondary goal 
 is to make user-space tools more robust, efficient, and simpler. 
 
 there are three things that need to be communicated through the proc 
 interface, for each chunk of data: its type, it's name and its value. 
 it's critical that data be tagged in some way, since that's the only 
 way to permit back-compatibility. that is, a tool looking for a particular 
 tag will naturally ignore new data with other tags. 

Agreed.

 [three example schemes in use in /proc today]
 I have a sense that all of these could be collapsed into a single 
 api where kernel systems would register hierarchies of tuples of 
 type,tag,callback, where callback would be passed the tag, 
 and proc code would take care of rendering the data into 
 human readable text (default), binary, or even xml. 

Sounds reasonable to me.  Relieve the modules of having to
format their /proc entries by defining standard code that does
it.   And as an extra bonus, if tuples registration was table-driven,
the tables would define a grammar that could be fed to a parser
generator.

(It sounds a little bit like the snmpd code I'm working on,
actually.  How eerie.)

(It also sounds a little like (gasp) the windows registry,
but hey, that's ok.)

- Dan
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [repost] Announce: Linux-OpenLVM mailing list

[Dan Kegel]

Dear Sistina:

I know very little about LVM, but from watching earlier projects
in the same situation you're in now, the path you need to follow
seems clear:
   Stop using CVS internally for development.
   It makes checking in changes without submitting them to 
   Linus too easy.

To get sync'd back up, *start with the standard kernel*,
and start generating clean, human-understandable patches one 
at a time that bring it up to where you want.

Once you've achieved that, have your programmers generate patches 
rather than checking in to CVS, and feed the patches to Linus 
at the same time you hand them out to your other programmers.
Individual programmers may need to do more testing this way, but
c'est la vie.

This is the only way to achieve union with the standard kernel.

So many projects have failed to learn this lesson...
ignore it at your peril.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [repost] Announce: Linux-OpenLVM mailing list

[Dan Kegel]

Dear Sistina:

I know very little about LVM, but from watching earlier projects
in the same situation you're in now, the path you need to follow
seems clear:
   Stop using CVS internally for development.
   It makes checking in changes without submitting them to 
   Linus too easy.

To get sync'd back up, *start with the standard kernel*,
and start generating clean, human-understandable patches one 
at a time that bring it up to where you want.

Once you've achieved that, have your programmers generate patches 
rather than checking in to CVS, and feed the patches to Linus 
at the same time you hand them out to your other programmers.
Individual programmers may need to do more testing this way, but
c'est la vie.

This is the only way to achieve union with the standard kernel.

So many projects have failed to learn this lesson...
ignore it at your peril.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux on the Unisys ES7000 and CMP2 machines?

[Dan Kegel]

jdow wrote:
> Miles, if these babies are the 32 processor monsters that UniSys 
> has been making recently there IS interest to get Linux on it. 
> But the people I know who have mentioned "interest", mostly from 
> a curiosity standpoint, have their hands neatly tied by Microsoft. 
> Ya see, the developers at UniSys have NT source licenses so they 
> can develop the HALs for the monsters. Microsoft insists that they 
> spend a considerable time away from OS development before working 
> on another OS. So, no Linux port is in the offing, I suspect. The 
> people who KNOW the machine are not allowed to do it.

I just saw one of these 32 processor machines at Internet World,
and the engineer said that AIX and the successor to SCO Unix would
also run on the machine.  Perhaps the engineers doing the AIX port
are under less restrictive terms.

(When the two people he was talking to asked about Linux on the machine, 
he said "We feel Linux can't do enterprise-level stuff like this."
He got a little defensive when we questioned his judgement.)

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux on the Unisys ES7000 and CMP2 machines?

[Dan Kegel]

jdow wrote:
 Miles, if these babies are the 32 processor monsters that UniSys 
 has been making recently there IS interest to get Linux on it. 
 But the people I know who have mentioned "interest", mostly from 
 a curiosity standpoint, have their hands neatly tied by Microsoft. 
 Ya see, the developers at UniSys have NT source licenses so they 
 can develop the HALs for the monsters. Microsoft insists that they 
 spend a considerable time away from OS development before working 
 on another OS. So, no Linux port is in the offing, I suspect. The 
 people who KNOW the machine are not allowed to do it.

I just saw one of these 32 processor machines at Internet World,
and the engineer said that AIX and the successor to SCO Unix would
also run on the machine.  Perhaps the engineers doing the AIX port
are under less restrictive terms.

(When the two people he was talking to asked about Linux on the machine, 
he said "We feel Linux can't do enterprise-level stuff like this."
He got a little defensive when we questioned his judgement.)

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Process vs. Threads

[Dan Kegel]

[EMAIL PROTECTED] wrote:
> I have been trying to differenciate threads and process in Linux. As 
> I am sure you already know, other OS, namely HPUX, implement threads 
> in a different way. ...
> Of course, I am talking about kernel 2.2.x, but AFAIK this has not 
> changed in the new kernels.

It's starting to change a bit; see the discussion following
  http://boudicca.tux.org/hypermail/linux-kernel/2000week36/0903.html
  ("thread group comments").

Can someone summarize the state of the thread changes in 2.4?  
A lot seemed to happen, but from what I gather, nothing user-visible yet.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Process vs. Threads

[Dan Kegel]

[EMAIL PROTECTED] wrote:
 I have been trying to differenciate threads and process in Linux. As 
 I am sure you already know, other OS, namely HPUX, implement threads 
 in a different way. ...
 Of course, I am talking about kernel 2.2.x, but AFAIK this has not 
 changed in the new kernels.

It's starting to change a bit; see the discussion following
  http://boudicca.tux.org/hypermail/linux-kernel/2000week36/0903.html
  ("thread group comments").

Can someone summarize the state of the thread changes in 2.4?  
A lot seemed to happen, but from what I gather, nothing user-visible yet.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Wine + kernel ?? How to do that?

[Dan Kegel]

[EMAIL PROTECTED] asked:
> hey, I hear that wine ( windows emulator ) can port into kernel and make 
> it running faster, How can I do it? 
> or anyone can make a patch to add wine code into kernel? 
> waiting for answer, Thanks 

It's not ready for prime time yet.

There is some discussion of the issues at 
  news://news.winehq.com/wine.devel 

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Wine + kernel ?? How to do that?

[Dan Kegel]

[EMAIL PROTECTED] asked:
 hey, I hear that wine ( windows emulator ) can port into kernel and make 
 it running faster, How can I do it? 
 or anyone can make a patch to add wine code into kernel? 
 waiting for answer, Thanks 

It's not ready for prime time yet.

There is some discussion of the issues at 
  news://news.winehq.com/wine.devel 

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: TCP_NOPUSH on FreeBSD, TCP_CORK on Linux (was: Is sendfile all that

[Dan Kegel]

Alexy wrote:

> > > How close is TCP_NOPUSH to behaving identically to TCP_CORK now? 
> 
> They have not so much of common. 
> 
> TCP_NOPUSH enables T/TCP and its presense used to mean that 
> T/TCP is possible on this system. Linux headers cannot 
> even contain TCP_NOPUSH.

But Tony Finch wrote:
> They are exactly the same. 

Alexy, Tony just checked in a change to FreeBSD to make TCP_NOPUSH behave the
same as TCP_CORK.

Tony, are people using the TCP_NOPUSH define as a way to detect
the presence of T/TCP support?  In that case, perhaps the right
thing to do to achieve source compatibility would be for FreeBSD
to also define TCP_CORK (and give it TCP_NOPUSH as a value, perhaps).

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: TCP_NOPUSH on FreeBSD, TCP_CORK on Linux (was: Is sendfile all that

[Dan Kegel]

Alan Cox wrote:
> 
> > How close is TCP_NOPUSH to behaving identically to TCP_CORK now?
> > If it does behave identically, it might be time to standardize
> > the symbolic name for this option, to make apps more portable
> > between the two OS's.  (It'd be nice to also standardize the
> > numeric value, in the interest of making the ABI's more compatible, too.)
> 
> That one isnt practical because of the way the implementations handle
> boolean options. BSD uses bitmask based option setting for the basic
> options and Linus uses switch statements

OK, well, at least a common symbolic name could be chosen.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

TCP_NOPUSH on FreeBSD, TCP_CORK on Linux (was: Is sendfile all that sexy?)

[Dan Kegel]

Tony Finch wrote:
> 
> Tony Finch <[EMAIL PROTECTED]> wrote:
> >Linus Torvalds <[EMAIL PROTECTED]> wrote:
> >>
> >>Without proper uncorking (and it really shouldn't be that hard to
> >>add), TCP_NOPUSH simply can't be used in the generic sense.
> >
> >It was easy :-) I've put up a patch for FreeBSD that adds proper
> >uncorking on my homepage  in the "Stuff"
> >section.
> 
> ... and it has been committed to -CURRENT, too.

How very cool.

How close is TCP_NOPUSH to behaving identically to TCP_CORK now?
If it does behave identically, it might be time to standardize
the symbolic name for this option, to make apps more portable
between the two OS's.  (It'd be nice to also standardize the
numeric value, in the interest of making the ABI's more compatible, too.)

Here are the definitions in the two OS's at the moment:

FreeBSD: netinet/tcp.h (from 
http://minnie.cs.adfa.oz.au/FreeBSD-srctree/newsrc/netinet/tcp.h.html#TCP_NOPUSH )

/*
 * User-settable options (used with setsockopt).
 */
#define TCP_NODELAY 0x01/* don't delay send to coalesce packets */
#define TCP_MAXSEG  0x02/* set maximum segment size */
#define TCP_NOPUSH  0x04/* don't push last block of write */

Linux: netinet/tcp.h:

/*
 * User-settable options (used with setsockopt).
 */
#define TCP_NODELAY 0x01/* don't delay send to coalesce packets */
#define TCP_MAXSEG  0x02/* set maximum segment size */
#define TCP_CORK0x03/* control sending of partial frames */  

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: No buffer space available??

[Dan Kegel]

Mathieu Dube wrote:
> when accept return -1 perror gives me "No buffer space available"
> What do you think that means??

Better ask a real net guru :-) 
Or look at the sources.

in /usr/src/linux/net/ipv4/af_inet.c

int inet_create() {
...
sk = sk_alloc(PF_INET, GFP_KERNEL, 1);   
if (sk == NULL)
goto do_oom;  
...
do_oom:
return -ENOBUFS;   

in /usr/src/linux/net/core/sock.c

/*
 *  All socket objects are allocated here. This is for future
 *  usage.
 */
 
struct sock *sk_alloc(int family, int priority, int zero_it)
{
struct sock *sk = kmem_cache_alloc(sk_cachep, priority);
 
if(sk) {
if (zero_it)
memset(sk, 0, sizeof(struct sock));
sk->family = family;
}
 
return sk;
}

void __init sk_init(void)
{
sk_cachep = kmem_cache_create("sock", sizeof(struct sock), 0,
  SLAB_HWCACHE_ALIGN, 0, 0);
 
} 

Poking around a little in mm/slab.c, I see that the name passed to
kmem_cache_create is used in generating the /proc/slabinfo report, so 
  cat /proc/slabinfo   | grep sock
shows you some info.  On my system, it prints two numbers, from
len += sprintf(buf+len, "%-17s %6lu %6lu\n", cachep->c_name, active_objs, 
num_objs);  
which tells you how many sockets are allocated.  Dunno how useful that is.

Like I said, you'll have to ask a real guru :-)

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: No buffer space available??

[Dan Kegel]

Mathieu Dube wrote:
 when accept return -1 perror gives me "No buffer space available"
 What do you think that means??

Better ask a real net guru :-) 
Or look at the sources.

in /usr/src/linux/net/ipv4/af_inet.c

int inet_create() {
...
sk = sk_alloc(PF_INET, GFP_KERNEL, 1);   
if (sk == NULL)
goto do_oom;  
...
do_oom:
return -ENOBUFS;   

in /usr/src/linux/net/core/sock.c

/*
 *  All socket objects are allocated here. This is for future
 *  usage.
 */
 
struct sock *sk_alloc(int family, int priority, int zero_it)
{
struct sock *sk = kmem_cache_alloc(sk_cachep, priority);
 
if(sk) {
if (zero_it)
memset(sk, 0, sizeof(struct sock));
sk-family = family;
}
 
return sk;
}

void __init sk_init(void)
{
sk_cachep = kmem_cache_create("sock", sizeof(struct sock), 0,
  SLAB_HWCACHE_ALIGN, 0, 0);
 
} 

Poking around a little in mm/slab.c, I see that the name passed to
kmem_cache_create is used in generating the /proc/slabinfo report, so 
  cat /proc/slabinfo   | grep sock
shows you some info.  On my system, it prints two numbers, from
len += sprintf(buf+len, "%-17s %6lu %6lu\n", cachep-c_name, active_objs, 
num_objs);  
which tells you how many sockets are allocated.  Dunno how useful that is.

Like I said, you'll have to ask a real guru :-)

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

TCP_NOPUSH on FreeBSD, TCP_CORK on Linux (was: Is sendfile all that sexy?)

[Dan Kegel]

Tony Finch wrote:
 
 Tony Finch [EMAIL PROTECTED] wrote:
 Linus Torvalds [EMAIL PROTECTED] wrote:
 
 Without proper uncorking (and it really shouldn't be that hard to
 add), TCP_NOPUSH simply can't be used in the generic sense.
 
 It was easy :-) I've put up a patch for FreeBSD that adds proper
 uncorking on my homepage http://apache.org/~fanf/ in the "Stuff"
 section.
 
 ... and it has been committed to -CURRENT, too.

How very cool.

How close is TCP_NOPUSH to behaving identically to TCP_CORK now?
If it does behave identically, it might be time to standardize
the symbolic name for this option, to make apps more portable
between the two OS's.  (It'd be nice to also standardize the
numeric value, in the interest of making the ABI's more compatible, too.)

Here are the definitions in the two OS's at the moment:

FreeBSD: netinet/tcp.h (from 
http://minnie.cs.adfa.oz.au/FreeBSD-srctree/newsrc/netinet/tcp.h.html#TCP_NOPUSH )

/*
 * User-settable options (used with setsockopt).
 */
#define TCP_NODELAY 0x01/* don't delay send to coalesce packets */
#define TCP_MAXSEG  0x02/* set maximum segment size */
#define TCP_NOPUSH  0x04/* don't push last block of write */

Linux: netinet/tcp.h:

/*
 * User-settable options (used with setsockopt).
 */
#define TCP_NODELAY 0x01/* don't delay send to coalesce packets */
#define TCP_MAXSEG  0x02/* set maximum segment size */
#define TCP_CORK0x03/* control sending of partial frames */  

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: TCP_NOPUSH on FreeBSD, TCP_CORK on Linux (was: Is sendfile all that

[Dan Kegel]

Alan Cox wrote:
 
  How close is TCP_NOPUSH to behaving identically to TCP_CORK now?
  If it does behave identically, it might be time to standardize
  the symbolic name for this option, to make apps more portable
  between the two OS's.  (It'd be nice to also standardize the
  numeric value, in the interest of making the ABI's more compatible, too.)
 
 That one isnt practical because of the way the implementations handle
 boolean options. BSD uses bitmask based option setting for the basic
 options and Linus uses switch statements

OK, well, at least a common symbolic name could be chosen.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: [Kiobuf-io-devel] Re: RFC: Kernel mechanism: Compound eventwait/notify + callback chains

[Dan Kegel]

sct wrote:

> On Wed, Jan 31, 2001 at 04:12:11PM +0530, [EMAIL PROTECTED] wrote: 
> > 
> > Thanks for mentioning this. I didn't know about it earlier. I've been 
> > going through the 4/00 kqueue patch on freebsd ... 
> 
> Linus has already denounced them as massively over-engineered... 

That shouldn't stop anyone from looking at them and learning, though.
There might be a good idea or two hiding in there somewhere.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: gprof cannot profile multi-threaded programs

[Dan Kegel]

Mohit Aron wrote:
> 
> > http://opensource.corel.com/cprof.html
> >
> > I haven't used it yet, myself.
> >
> 
> I have. cprof is no good - extremely slow and generates a 100MB trace
> even with a simple hello world program.

Oh.  Bleh.

http://wordindex.sourceforge.net/testdata/usenet.col-2817-1548/028-123.col.txt.txt
mentioned a workaround for gprof, I don't know if it's real:

> AFAIK gprof doesn't support multithreaded apps profiling, but you can
> workaround it if you call getitimer() in the main thread for ITIMER_PROF
> then using that value in a call to setitimer() in every thread you
> spawn. Other alternative is using the open source cprof by Corel [I
> never 

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: gprof cannot profile multi-threaded programs

[Dan Kegel]

Mohit Aron wrote:
 
  http://opensource.corel.com/cprof.html
 
  I haven't used it yet, myself.
 
 
 I have. cprof is no good - extremely slow and generates a 100MB trace
 even with a simple hello world program.

Oh.  Bleh.

http://wordindex.sourceforge.net/testdata/usenet.col-2817-1548/028-123.col.txt.txt
mentioned a workaround for gprof, I don't know if it's real:

 AFAIK gprof doesn't support multithreaded apps profiling, but you can
 workaround it if you call getitimer() in the main thread for ITIMER_PROF
 then using that value in a call to setitimer() in every thread you
 spawn. Other alternative is using the open source cprof by Corel [I
 never 

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: [Kiobuf-io-devel] Re: RFC: Kernel mechanism: Compound eventwait/notify + callback chains

[Dan Kegel]

sct wrote:

 On Wed, Jan 31, 2001 at 04:12:11PM +0530, [EMAIL PROTECTED] wrote: 
  
  Thanks for mentioning this. I didn't know about it earlier. I've been 
  going through the 4/00 kqueue patch on freebsd ... 
 
 Linus has already denounced them as massively over-engineered... 

That shouldn't stop anyone from looking at them and learning, though.
There might be a good idea or two hiding in there somewhere.
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: gprof cannot profile multi-threaded programs

[Dan Kegel]

See 
http://opensource.corel.com/cprof.html

I haven't used it yet, myself.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: gprof cannot profile multi-threaded programs

[Dan Kegel]

See 
http://opensource.corel.com/cprof.html

I haven't used it yet, myself.

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: [Fwd: Is sendfile all that sexy? (fwd)]]

[Dan Kegel]

Dean Gaudet wrote:
> consider the case where you're responding to a pair of pipelined HTTP/1.1 
> requests. with the HPUX and BSD sendfile() APIs you end up forcing a 
> packet boundary between the two responses. this is likely to result in 
> one small packet on the wire after each response. 
> 
> with the linux TCP_CORK API you only get one trailing small packet

Tony Finch tells me that BSD also supports TCP_CORK; in fact, it had it first.
He wrote:
> BSDs that include T/TCP (pretty much all of them since 1995) have an
> option called TCP_NOPUSH which is equivalent to Linux's TCP_CORK. A
> pity the Linux people didn't know about it when they implemented their
> version.
>  
> #if defined(TCP_CORK) && !defined(TCP_NOPUSH)
> #define TCP_NOPUSH TCP_CORK
> #endif

Can anyone verify it resolves the problem Dean pointed out?

Now, Linus, does that make you hate BSD less? :-)
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/

Re: [Fwd: [Fwd: Is sendfile all that sexy? (fwd)]]

[Dan Kegel]

Dean Gaudet wrote:
 consider the case where you're responding to a pair of pipelined HTTP/1.1 
 requests. with the HPUX and BSD sendfile() APIs you end up forcing a 
 packet boundary between the two responses. this is likely to result in 
 one small packet on the wire after each response. 
 
 with the linux TCP_CORK API you only get one trailing small packet

Tony Finch tells me that BSD also supports TCP_CORK; in fact, it had it first.
He wrote:
 BSDs that include T/TCP (pretty much all of them since 1995) have an
 option called TCP_NOPUSH which is equivalent to Linux's TCP_CORK. A
 pity the Linux people didn't know about it when they implemented their
 version.
  
 #if defined(TCP_CORK)  !defined(TCP_NOPUSH)
 #define TCP_NOPUSH TCP_CORK
 #endif

Can anyone verify it resolves the problem Dean pointed out?

Now, Linus, does that make you hate BSD less? :-)
- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/