Re: [rfc 08/45] cpu alloc: x86 support

2007-11-26 Thread John Richard Moser



Andi Kleen wrote:

On Tuesday 20 November 2007 04:50, Christoph Lameter wrote:

On Tue, 20 Nov 2007, Andi Kleen wrote:



You could in theory move the modules, but then you would need to implement
a full PIC dynamic linker for them  first and also increase runtime overhead
for them because they would need to use a GOT/PLT.


On x86-64?  The GOT/PLT should stay in cache due to temporal locality. 
The x86-64 instruction set itself handles GOT-relative addressing rather 
well; what's a 1% loss on x86 is like 0.01% on x86-64, so I'm thinking 
100 times better?


I think I got this by `-fpic -pie` compiling nbyte benchmark versus 
fixed position, each with and without on 32-bit (which made about a 1% 
difference) and on 64-bit (which made a 0.01% difference).  It was a 
long time ago.


Still, yeah I know.  Complexity.

(You have the ability to textrel these things too, and just rewrite 
non-PIC, depending on how you feel about that)

--
Bring back the Firefox plushy!
http://digg.com/linux_unix/Is_the_Firefox_plush_gone_for_good
https://bugzilla.mozilla.org/show_bug.cgi?id=322367
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [rfc 08/45] cpu alloc: x86 support

2007-11-26 Thread John Richard Moser



Andi Kleen wrote:

On Tuesday 20 November 2007 04:50, Christoph Lameter wrote:

On Tue, 20 Nov 2007, Andi Kleen wrote:



You could in theory move the modules, but then you would need to implement
a full PIC dynamic linker for them  first and also increase runtime overhead
for them because they would need to use a GOT/PLT.


On x86-64?  The GOT/PLT should stay in cache due to temporal locality. 
The x86-64 instruction set itself handles GOT-relative addressing rather 
well; what's a 1% loss on x86 is like 0.01% on x86-64, so I'm thinking 
100 times better?


I think I got this by `-fpic -pie` compiling nbyte benchmark versus 
fixed position, each with and without on 32-bit (which made about a 1% 
difference) and on 64-bit (which made a 0.01% difference).  It was a 
long time ago.


Still, yeah I know.  Complexity.

(You have the ability to textrel these things too, and just rewrite 
non-PIC, depending on how you feel about that)

--
Bring back the Firefox plushy!
http://digg.com/linux_unix/Is_the_Firefox_plush_gone_for_good
https://bugzilla.mozilla.org/show_bug.cgi?id=322367
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: via_drm bug

2007-06-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Dave Airlie wrote:
> On 6/10/07, John Richard Moser <[EMAIL PROTECTED]> wrote:
> This has been an on-going issue for I don't know how long.  I
> reported it a while ago but it's still in 2.6.22.
> 
> Here's another error log.  Loaded the Via driver in Xorg with kernel
> 2.6.22 on Ubuntu, got the following in dmesg.  The [drm:via_mem_alloc]
> error may be the problem.  This is on a 32-bit kernel.
> 
> 
>> I've cc'ed Thomas to see if he can help out, he knows the via drm ..
> 

I may not be able to contribute much testing soon, as this board is
moving out of this machine and I don't have another to put it into for
the moment.  After that it won't be my problem any more, but it'd still
be nice to have it fixed.

It's been bugging me for a while because it seems to be just a piece of
faulty memory management code (probably a bug in the DMA code), but it
hasn't been fixed in several revisions.  Oh well, I'm sure no one's
really looking, so it's good to bring it up and pull out the latest
debug data to say "Yes this is really still broken" once in a while :)

Interestingly enough, I haven't managed to get it to take down the
system since a while back (it used to hard-freeze the kernel).



>> Dave.
> 
> See also:
> 
> https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-via/+bug/72630
> 
> 
> 
> 
> [264565.164000] [drm] Initialized via 2.11.1 20070202 on minor 0
> [264565.168000] ACPI: PCI Interrupt :01:00.0[A] -> GSI 16 (level,
> low) -> IRQ 21
> [264565.168000] [drm] Initialized via 2.11.1 20070202 on minor 1
> [264565.184000] agpgart: Found an AGP 3.0 compliant device at
> :00:00.0.
> [264565.184000] agpgart: Xorg tried to set rate=x12. Setting to AGP3 x8
> mode.
> [264565.184000] agpgart: Putting AGP V3 device at :00:00.0 into 8x
> mode
> [264565.184000] agpgart: Putting AGP V3 device at :01:00.0 into 8x
> mode
> [264565.192000] [drm:via_mem_alloc] *ERROR* Attempt to allocate from
> uninitialized memory manager.
> [264565.296000] powernow-k8: error - out of sync, fix 0x2 0xa, vid 0x0
> 0x0
> [264565.296000] powernow-k8: ph2 null fid transition 0xa
> [264565.692000] irq 21: nobody cared (try booting with the "irqpoll"
> option)
> [264565.692000]  [] __report_bad_irq+0x24/0x80
> [264565.692000]  [] note_interrupt+0x262/0x2a0
> [264565.692000]  [] handle_IRQ_event+0x30/0x60
> [264565.692000]  [] handle_fasteoi_irq+0xbb/0xf0
> [264565.692000]  [] do_IRQ+0x3b/0x70
> [264565.692000]  [] smp_apic_timer_interrupt+0x55/0x80
> [264565.692000]  [] default_idle+0x0/0x60
> [264565.692000]  [] common_interrupt+0x23/0x30
> [264565.692000]  [] default_idle+0x0/0x60
> [264565.692000]  [] native_safe_halt+0x2/0x10
> [264565.692000]  [] default_idle+0x3c/0x60
> [264565.692000]  [] cpu_idle+0x53/0xe0
> [264565.692000]  [] start_kernel+0x31f/0x3b0
> [264565.692000]  [] unknown_bootoption+0x0/0x260
> [264565.692000]  ===
> [264565.692000] handlers:
> [264565.692000] [] (via_driver_irq_handler+0x0/0x1d0 [via])
> [264565.692000] Disabling IRQ #21
> 
> 
> 
> Device:
> 
> 01:00.0 VGA compatible controller: VIA Technologies, Inc. S3 Unichrome
> Pro VGA Adapter (rev 01)
> 
> 
> lshw segment:
> 
>*-display
> description: VGA compatible controller
> product: S3 Unichrome Pro VGA Adapter
> vendor: VIA Technologies, Inc.
> physical id: 0
> bus info: [EMAIL PROTECTED]:00.0
> version: 01
> size: 64MB
> width: 32 bits
> clock: 66MHz
> capabilities: vga bus_master cap_list
> configuration: latency=32 mingnt=2
> resources: iomemory:e400-e7ff
> iomemory:e800-e8ff irq:21
> 
> 
- -
To unsubscribe from this list: send the line "unsubscribe
linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
>>

- --
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
LiveCD Creator Poll:  http://ubuntuforums.org/showthread.php?t=435562
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRmuMhQs1xW0HCTEFAQJhQw/8DBaejrNgkrAa+8Rcm6Dh3vFfhImchI8D
LKGmz1SBM3suhlNPl5saEiSKAO+jIFa23dpFk7/ijTrVLNRNm8baLAYl8iLBuXiF
DpPZahWYRcbldIj/8osl3169blpiRY/wGaHb2CL50E6POIMo/OyW3nc1St/N7vst
gItuRKW4WtEZaxA5+tGHsc2T3A3JTB3IDS1fUe8uFjsJG9/ciqKQJc2p+xnqMiLM
M29cdwk7X8uJSidMcq

via_drm bug

2007-06-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

This has been an on-going issue for I don't know how long.  I
reported it a while ago but it's still in 2.6.22.

Here's another error log.  Loaded the Via driver in Xorg with kernel
2.6.22 on Ubuntu, got the following in dmesg.  The [drm:via_mem_alloc]
error may be the problem.  This is on a 32-bit kernel.

See also:

https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-via/+bug/72630



[264565.164000] [drm] Initialized via 2.11.1 20070202 on minor 0
[264565.168000] ACPI: PCI Interrupt :01:00.0[A] -> GSI 16 (level,
low) -> IRQ 21
[264565.168000] [drm] Initialized via 2.11.1 20070202 on minor 1
[264565.184000] agpgart: Found an AGP 3.0 compliant device at :00:00.0.
[264565.184000] agpgart: Xorg tried to set rate=x12. Setting to AGP3 x8
mode.
[264565.184000] agpgart: Putting AGP V3 device at :00:00.0 into 8x mode
[264565.184000] agpgart: Putting AGP V3 device at :01:00.0 into 8x mode
[264565.192000] [drm:via_mem_alloc] *ERROR* Attempt to allocate from
uninitialized memory manager.
[264565.296000] powernow-k8: error - out of sync, fix 0x2 0xa, vid 0x0 0x0
[264565.296000] powernow-k8: ph2 null fid transition 0xa
[264565.692000] irq 21: nobody cared (try booting with the "irqpoll" option)
[264565.692000]  [] __report_bad_irq+0x24/0x80
[264565.692000]  [] note_interrupt+0x262/0x2a0
[264565.692000]  [] handle_IRQ_event+0x30/0x60
[264565.692000]  [] handle_fasteoi_irq+0xbb/0xf0
[264565.692000]  [] do_IRQ+0x3b/0x70
[264565.692000]  [] smp_apic_timer_interrupt+0x55/0x80
[264565.692000]  [] default_idle+0x0/0x60
[264565.692000]  [] common_interrupt+0x23/0x30
[264565.692000]  [] default_idle+0x0/0x60
[264565.692000]  [] native_safe_halt+0x2/0x10
[264565.692000]  [] default_idle+0x3c/0x60
[264565.692000]  [] cpu_idle+0x53/0xe0
[264565.692000]  [] start_kernel+0x31f/0x3b0
[264565.692000]  [] unknown_bootoption+0x0/0x260
[264565.692000]  ===
[264565.692000] handlers:
[264565.692000] [] (via_driver_irq_handler+0x0/0x1d0 [via])
[264565.692000] Disabling IRQ #21



Device:

01:00.0 VGA compatible controller: VIA Technologies, Inc. S3 Unichrome
Pro VGA Adapter (rev 01)


lshw segment:

   *-display
description: VGA compatible controller
product: S3 Unichrome Pro VGA Adapter
vendor: VIA Technologies, Inc.
physical id: 0
bus info: [EMAIL PROTECTED]:00.0
version: 01
size: 64MB
width: 32 bits
clock: 66MHz
capabilities: vga bus_master cap_list
configuration: latency=32 mingnt=2
resources: iomemory:e400-e7ff
iomemory:e800-e8ff irq:21


- --
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
LiveCD Creator Poll:  http://ubuntuforums.org/showthread.php?t=435562
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRmtq2ws1xW0HCTEFAQKthhAAmEDCkQGl5tEfioxZE67/a1EkVKQisIwk
YWKKZ/+5uQo/wo1V/HWIekRICByYVjuK3vqnJmXo9VEk37L/ZRQgcDDiMyG5lRwR
UhtYFVEMkPyr923EKGXyQCjdNaYDHWHz4VWOtf1mpyPMP3WVByvOJcGWikv/wWcl
BLb37+tk9Wv7846zwoTwREStZHJbhoSac+VSidgV3koIjeeOCqFwicNP+6ycOSbE
PnBQ/sjsbagw7DdweDR6QvDzxtnS/KP4adGutXGJUQYN1s4R5YDIDF7Qk1M1hNN7
7IH7KNwNXwzKq3v6jCcYrprOthBrB7Xho2s8nM+fT3Kzwpj+UTy/wvesEQQNbDSD
jTVGFa4qjQIdgf4G/n88Wv4Pm2yHC1dRyPrrRz8TX2rv4A3cqlC+REYfVyHhS9Uh
2gNUOctawvAM1rL2peG9SDvsEJNYIzoV/jL61zit3Jpkevc34Aq1wrxj+cyxchJI
7zUtshxGCOOIiEwgJ9u/5E0T128hXGIX9Nm3Cb6bdj5aVRv+kso/bpMGGcn5+opa
JdUYg9avJMbJtlmfWtz7EEACLB8DeUTbCFKZ2ctoqCm4UKkyex2uX+sAAI1hbjpZ
oPYcXEs4bcb8BlZpntykybjlJUtGOeTlCyULeWATNHDF+txgTnpkp7U79Z73EXIL
vI9KkUJ2qho=
=hmgi
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


via_drm bug

2007-06-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

This has been an on-going issue for I don't know how long.  I
reported it a while ago but it's still in 2.6.22.

Here's another error log.  Loaded the Via driver in Xorg with kernel
2.6.22 on Ubuntu, got the following in dmesg.  The [drm:via_mem_alloc]
error may be the problem.  This is on a 32-bit kernel.

See also:

https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-via/+bug/72630



[264565.164000] [drm] Initialized via 2.11.1 20070202 on minor 0
[264565.168000] ACPI: PCI Interrupt :01:00.0[A] - GSI 16 (level,
low) - IRQ 21
[264565.168000] [drm] Initialized via 2.11.1 20070202 on minor 1
[264565.184000] agpgart: Found an AGP 3.0 compliant device at :00:00.0.
[264565.184000] agpgart: Xorg tried to set rate=x12. Setting to AGP3 x8
mode.
[264565.184000] agpgart: Putting AGP V3 device at :00:00.0 into 8x mode
[264565.184000] agpgart: Putting AGP V3 device at :01:00.0 into 8x mode
[264565.192000] [drm:via_mem_alloc] *ERROR* Attempt to allocate from
uninitialized memory manager.
[264565.296000] powernow-k8: error - out of sync, fix 0x2 0xa, vid 0x0 0x0
[264565.296000] powernow-k8: ph2 null fid transition 0xa
[264565.692000] irq 21: nobody cared (try booting with the irqpoll option)
[264565.692000]  [c015bcc4] __report_bad_irq+0x24/0x80
[264565.692000]  [c015bf82] note_interrupt+0x262/0x2a0
[264565.692000]  [c015b1e0] handle_IRQ_event+0x30/0x60
[264565.692000]  [c015c96b] handle_fasteoi_irq+0xbb/0xf0
[264565.692000]  [c0106b1b] do_IRQ+0x3b/0x70
[264565.692000]  [c0118735] smp_apic_timer_interrupt+0x55/0x80
[264565.692000]  [c0102e00] default_idle+0x0/0x60
[264565.692000]  [c0105223] common_interrupt+0x23/0x30
[264565.692000]  [c0102e00] default_idle+0x0/0x60
[264565.692000]  [c011db72] native_safe_halt+0x2/0x10
[264565.692000]  [c0102e3c] default_idle+0x3c/0x60
[264565.692000]  [c0102413] cpu_idle+0x53/0xe0
[264565.692000]  [c03dfa7f] start_kernel+0x31f/0x3b0
[264565.692000]  [c03df1f0] unknown_bootoption+0x0/0x260
[264565.692000]  ===
[264565.692000] handlers:
[264565.692000] [f8eb6480] (via_driver_irq_handler+0x0/0x1d0 [via])
[264565.692000] Disabling IRQ #21



Device:

01:00.0 VGA compatible controller: VIA Technologies, Inc. S3 Unichrome
Pro VGA Adapter (rev 01)


lshw segment:

   *-display
description: VGA compatible controller
product: S3 Unichrome Pro VGA Adapter
vendor: VIA Technologies, Inc.
physical id: 0
bus info: [EMAIL PROTECTED]:00.0
version: 01
size: 64MB
width: 32 bits
clock: 66MHz
capabilities: vga bus_master cap_list
configuration: latency=32 mingnt=2
resources: iomemory:e400-e7ff
iomemory:e800-e8ff irq:21


- --
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
LiveCD Creator Poll:  http://ubuntuforums.org/showthread.php?t=435562
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRmtq2ws1xW0HCTEFAQKthhAAmEDCkQGl5tEfioxZE67/a1EkVKQisIwk
YWKKZ/+5uQo/wo1V/HWIekRICByYVjuK3vqnJmXo9VEk37L/ZRQgcDDiMyG5lRwR
UhtYFVEMkPyr923EKGXyQCjdNaYDHWHz4VWOtf1mpyPMP3WVByvOJcGWikv/wWcl
BLb37+tk9Wv7846zwoTwREStZHJbhoSac+VSidgV3koIjeeOCqFwicNP+6ycOSbE
PnBQ/sjsbagw7DdweDR6QvDzxtnS/KP4adGutXGJUQYN1s4R5YDIDF7Qk1M1hNN7
7IH7KNwNXwzKq3v6jCcYrprOthBrB7Xho2s8nM+fT3Kzwpj+UTy/wvesEQQNbDSD
jTVGFa4qjQIdgf4G/n88Wv4Pm2yHC1dRyPrrRz8TX2rv4A3cqlC+REYfVyHhS9Uh
2gNUOctawvAM1rL2peG9SDvsEJNYIzoV/jL61zit3Jpkevc34Aq1wrxj+cyxchJI
7zUtshxGCOOIiEwgJ9u/5E0T128hXGIX9Nm3Cb6bdj5aVRv+kso/bpMGGcn5+opa
JdUYg9avJMbJtlmfWtz7EEACLB8DeUTbCFKZ2ctoqCm4UKkyex2uX+sAAI1hbjpZ
oPYcXEs4bcb8BlZpntykybjlJUtGOeTlCyULeWATNHDF+txgTnpkp7U79Z73EXIL
vI9KkUJ2qho=
=hmgi
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: via_drm bug

2007-06-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Dave Airlie wrote:
 On 6/10/07, John Richard Moser [EMAIL PROTECTED] wrote:
 This has been an on-going issue for I don't know how long.  I
 reported it a while ago but it's still in 2.6.22.
 
 Here's another error log.  Loaded the Via driver in Xorg with kernel
 2.6.22 on Ubuntu, got the following in dmesg.  The [drm:via_mem_alloc]
 error may be the problem.  This is on a 32-bit kernel.
 
 
 I've cc'ed Thomas to see if he can help out, he knows the via drm ..
 

I may not be able to contribute much testing soon, as this board is
moving out of this machine and I don't have another to put it into for
the moment.  After that it won't be my problem any more, but it'd still
be nice to have it fixed.

It's been bugging me for a while because it seems to be just a piece of
faulty memory management code (probably a bug in the DMA code), but it
hasn't been fixed in several revisions.  Oh well, I'm sure no one's
really looking, so it's good to bring it up and pull out the latest
debug data to say Yes this is really still broken once in a while :)

Interestingly enough, I haven't managed to get it to take down the
system since a while back (it used to hard-freeze the kernel).



 Dave.
 
 See also:
 
 https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-via/+bug/72630
 
 
 
 
 [264565.164000] [drm] Initialized via 2.11.1 20070202 on minor 0
 [264565.168000] ACPI: PCI Interrupt :01:00.0[A] - GSI 16 (level,
 low) - IRQ 21
 [264565.168000] [drm] Initialized via 2.11.1 20070202 on minor 1
 [264565.184000] agpgart: Found an AGP 3.0 compliant device at
 :00:00.0.
 [264565.184000] agpgart: Xorg tried to set rate=x12. Setting to AGP3 x8
 mode.
 [264565.184000] agpgart: Putting AGP V3 device at :00:00.0 into 8x
 mode
 [264565.184000] agpgart: Putting AGP V3 device at :01:00.0 into 8x
 mode
 [264565.192000] [drm:via_mem_alloc] *ERROR* Attempt to allocate from
 uninitialized memory manager.
 [264565.296000] powernow-k8: error - out of sync, fix 0x2 0xa, vid 0x0
 0x0
 [264565.296000] powernow-k8: ph2 null fid transition 0xa
 [264565.692000] irq 21: nobody cared (try booting with the irqpoll
 option)
 [264565.692000]  [c015bcc4] __report_bad_irq+0x24/0x80
 [264565.692000]  [c015bf82] note_interrupt+0x262/0x2a0
 [264565.692000]  [c015b1e0] handle_IRQ_event+0x30/0x60
 [264565.692000]  [c015c96b] handle_fasteoi_irq+0xbb/0xf0
 [264565.692000]  [c0106b1b] do_IRQ+0x3b/0x70
 [264565.692000]  [c0118735] smp_apic_timer_interrupt+0x55/0x80
 [264565.692000]  [c0102e00] default_idle+0x0/0x60
 [264565.692000]  [c0105223] common_interrupt+0x23/0x30
 [264565.692000]  [c0102e00] default_idle+0x0/0x60
 [264565.692000]  [c011db72] native_safe_halt+0x2/0x10
 [264565.692000]  [c0102e3c] default_idle+0x3c/0x60
 [264565.692000]  [c0102413] cpu_idle+0x53/0xe0
 [264565.692000]  [c03dfa7f] start_kernel+0x31f/0x3b0
 [264565.692000]  [c03df1f0] unknown_bootoption+0x0/0x260
 [264565.692000]  ===
 [264565.692000] handlers:
 [264565.692000] [f8eb6480] (via_driver_irq_handler+0x0/0x1d0 [via])
 [264565.692000] Disabling IRQ #21
 
 
 
 Device:
 
 01:00.0 VGA compatible controller: VIA Technologies, Inc. S3 Unichrome
 Pro VGA Adapter (rev 01)
 
 
 lshw segment:
 
*-display
 description: VGA compatible controller
 product: S3 Unichrome Pro VGA Adapter
 vendor: VIA Technologies, Inc.
 physical id: 0
 bus info: [EMAIL PROTECTED]:00.0
 version: 01
 size: 64MB
 width: 32 bits
 clock: 66MHz
 capabilities: vga bus_master cap_list
 configuration: latency=32 mingnt=2
 resources: iomemory:e400-e7ff
 iomemory:e800-e8ff irq:21
 
 
- -
To unsubscribe from this list: send the line unsubscribe
linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


- --
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
LiveCD Creator Poll:  http://ubuntuforums.org/showthread.php?t=435562
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRmuMhQs1xW0HCTEFAQJhQw/8DBaejrNgkrAa+8Rcm6Dh3vFfhImchI8D
LKGmz1SBM3suhlNPl5saEiSKAO+jIFa23dpFk7/ijTrVLNRNm8baLAYl8iLBuXiF
DpPZahWYRcbldIj/8osl3169blpiRY/wGaHb2CL50E6POIMo/OyW3nc1St/N7vst
gItuRKW4WtEZaxA5+tGHsc2T3A3JTB3IDS1fUe8uFjsJG9/ciqKQJc2p+xnqMiLM
M29cdwk7X8uJSidMcqXIUc+q+SfZpYOc2yqhx0wbDmtGwGfDjOwb3YX4a/7Tbxcx
8bloaZF+O6olLR2HGcWM4umWExbfXH3+jGj+GyNfCfV1Hn4/EWjdBWC1x6WAJIf/
lXE3kLIDBG3wd2lQq8Dt9oAntErOYBj208VvW3601b4KxthGJVMrfOQX2qvbFbLp
pQzceSruoMZlZ016+k+RVc2uNEFkFxIN8+7N7qDm4R/8qHaNj4YIieQgi0qOg+5c
B7vRq

Re: evading ulimits

2006-12-23 Thread John Richard Moser


[EMAIL PROTECTED] wrote:
> On Sat, 23 Dec 2006 19:42:10 EST, John Richard Moser said:
>>
>> Jan Engelhardt wrote:
>>>> I've set up some stuff on my box where /etc/security/limits.conf
>>>> contains the following:
>>>>
>>>> @users  softnproc   3072
>>>> @users  hardnproc   4096
>>>>
>>>> I'm in group users, and a simple fork bomb is easily quashed by this:
>>>>
>>>> [EMAIL PROTECTED]:~$ :(){ :|:; };:
>>>> bash: fork: Resource temporarily unavailable
>>>> Terminated
>>>>
>>>> Oddly enough, trying this again and again yields the same results; but,
>>>> I can kill the box (eventually; about 1 minute in I managed to `/exec
>>>> killall -9 bash` from x-chat, since I couldn't get a new shell open)
>>>> with the below:
>>> Note that trying to kill all shells is a race between killing them all firs
> t
>>> and them spawning new ones everytime. To stop fork bombs, use killall -STOP
>>> first, then kill them.
>>>
>> Yes I know; the point, though, is that they should die automatically
>> when the process count hits 4096.  They do with the first fork bomb;
>> they keep growing with the second, well past what they should.
> 
> This may be another timing issue - note that in the first case, you have :|:
> which forks off 2 processes with a pipe in between.  If the head process 
> fails,
> the second probably won't get started by the shell *either*.  In the second
> case, you launch *3*, and it's possible that the head process will start and
> live long enough to re-fork before a later one in the pipe gets killed.
> 
> Does the behavior change if you use 4095 or 4097 rather than 4096?  I'm
> willing to bet that your system exhibits semi-predictable behavior regarding
> the order the processes get created, and *which* process gets killed makes
> a difference regarding how fast the process tree gets pruned.
> 

I will test this later (sleeping now)

> Do you have any good evidence that the second version manages to create much
> more than 4096 processes, rather than just being more exuberant about doing 
> so?
> I'm guessing that in fact, in both cases the number of processes ends up
> pseudorandomly oscillating in the 4090-4906 range, but the exact order of
> operations makes the rate and impact different in the two cases.

I haven't gathered evidence; I rather look for evidence that fork() is
failing, like bash complaints.  I get those in abundance in the first
case; but see none in the second case.

Trying again, I can't reproduce the phenomena.  It crashes out in 3
seconds.  Either just weird that it ran for so long last time before I
had to manually kill it off; or a race.  I don't have sufficient data to
decide.


-- 
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
Anti-Spam:  https://bugzilla.mozilla.org/show_bug.cgi?id=229686
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: evading ulimits

2006-12-23 Thread John Richard Moser


Jan Engelhardt wrote:
>> I've set up some stuff on my box where /etc/security/limits.conf
>> contains the following:
>>
>> @users  softnproc   3072
>> @users  hardnproc   4096
>>
>> I'm in group users, and a simple fork bomb is easily quashed by this:
>>
>> [EMAIL PROTECTED]:~$ :(){ :|:; };:
>> bash: fork: Resource temporarily unavailable
>> Terminated
>>
>> Oddly enough, trying this again and again yields the same results; but,
>> I can kill the box (eventually; about 1 minute in I managed to `/exec
>> killall -9 bash` from x-chat, since I couldn't get a new shell open)
>> with the below:
> 
> Note that trying to kill all shells is a race between killing them all first
> and them spawning new ones everytime. To stop fork bombs, use killall -STOP
> first, then kill them.
> 

Yes I know; the point, though, is that they should die automatically
when the process count hits 4096.  They do with the first fork bomb;
they keep growing with the second, well past what they should.
> 
>   -`J'

-- 
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
Anti-Spam:  https://bugzilla.mozilla.org/show_bug.cgi?id=229686
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: evading ulimits

2006-12-23 Thread John Richard Moser


Jan Engelhardt wrote:
 I've set up some stuff on my box where /etc/security/limits.conf
 contains the following:

 @users  softnproc   3072
 @users  hardnproc   4096

 I'm in group users, and a simple fork bomb is easily quashed by this:

 [EMAIL PROTECTED]:~$ :(){ :|:; };:
 bash: fork: Resource temporarily unavailable
 Terminated

 Oddly enough, trying this again and again yields the same results; but,
 I can kill the box (eventually; about 1 minute in I managed to `/exec
 killall -9 bash` from x-chat, since I couldn't get a new shell open)
 with the below:
 
 Note that trying to kill all shells is a race between killing them all first
 and them spawning new ones everytime. To stop fork bombs, use killall -STOP
 first, then kill them.
 

Yes I know; the point, though, is that they should die automatically
when the process count hits 4096.  They do with the first fork bomb;
they keep growing with the second, well past what they should.
 
   -`J'

-- 
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
Anti-Spam:  https://bugzilla.mozilla.org/show_bug.cgi?id=229686
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: evading ulimits

2006-12-23 Thread John Richard Moser


[EMAIL PROTECTED] wrote:
 On Sat, 23 Dec 2006 19:42:10 EST, John Richard Moser said:

 Jan Engelhardt wrote:
 I've set up some stuff on my box where /etc/security/limits.conf
 contains the following:

 @users  softnproc   3072
 @users  hardnproc   4096

 I'm in group users, and a simple fork bomb is easily quashed by this:

 [EMAIL PROTECTED]:~$ :(){ :|:; };:
 bash: fork: Resource temporarily unavailable
 Terminated

 Oddly enough, trying this again and again yields the same results; but,
 I can kill the box (eventually; about 1 minute in I managed to `/exec
 killall -9 bash` from x-chat, since I couldn't get a new shell open)
 with the below:
 Note that trying to kill all shells is a race between killing them all firs
 t
 and them spawning new ones everytime. To stop fork bombs, use killall -STOP
 first, then kill them.

 Yes I know; the point, though, is that they should die automatically
 when the process count hits 4096.  They do with the first fork bomb;
 they keep growing with the second, well past what they should.
 
 This may be another timing issue - note that in the first case, you have :|:
 which forks off 2 processes with a pipe in between.  If the head process 
 fails,
 the second probably won't get started by the shell *either*.  In the second
 case, you launch *3*, and it's possible that the head process will start and
 live long enough to re-fork before a later one in the pipe gets killed.
 
 Does the behavior change if you use 4095 or 4097 rather than 4096?  I'm
 willing to bet that your system exhibits semi-predictable behavior regarding
 the order the processes get created, and *which* process gets killed makes
 a difference regarding how fast the process tree gets pruned.
 

I will test this later (sleeping now)

 Do you have any good evidence that the second version manages to create much
 more than 4096 processes, rather than just being more exuberant about doing 
 so?
 I'm guessing that in fact, in both cases the number of processes ends up
 pseudorandomly oscillating in the 4090-4906 range, but the exact order of
 operations makes the rate and impact different in the two cases.

I haven't gathered evidence; I rather look for evidence that fork() is
failing, like bash complaints.  I get those in abundance in the first
case; but see none in the second case.

Trying again, I can't reproduce the phenomena.  It crashes out in 3
seconds.  Either just weird that it ran for so long last time before I
had to manually kill it off; or a race.  I don't have sufficient data to
decide.


-- 
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
Anti-Spam:  https://bugzilla.mozilla.org/show_bug.cgi?id=229686
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


evading ulimits

2006-12-22 Thread John Richard Moser
I've set up some stuff on my box where /etc/security/limits.conf
contains the following:

@users  softnproc   3072
@users  hardnproc   4096

I'm in group users, and a simple fork bomb is easily quashed by this:

[EMAIL PROTECTED]:~$ :(){ :|:; };:
bash: fork: Resource temporarily unavailable
Terminated

Oddly enough, trying this again and again yields the same results; but,
I can kill the box (eventually; about 1 minute in I managed to `/exec
killall -9 bash` from x-chat, since I couldn't get a new shell open)
with the below:

[EMAIL PROTECTED]:~$ :(){ :|:|:; };:

How exactly does the ulimit work?  Why do I seem to be able to evade
limits on maximum number of processes by doing a bigger fork bomb?  I
would have thought that the above would have terminated much sooner,
since it was spawning x^3 processes instead of x^2 for iteration x and
should have hit 4096 a lot sooner.


-- 
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
Anti-Spam:  https://bugzilla.mozilla.org/show_bug.cgi?id=229686
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


evading ulimits

2006-12-22 Thread John Richard Moser
I've set up some stuff on my box where /etc/security/limits.conf
contains the following:

@users  softnproc   3072
@users  hardnproc   4096

I'm in group users, and a simple fork bomb is easily quashed by this:

[EMAIL PROTECTED]:~$ :(){ :|:; };:
bash: fork: Resource temporarily unavailable
Terminated

Oddly enough, trying this again and again yields the same results; but,
I can kill the box (eventually; about 1 minute in I managed to `/exec
killall -9 bash` from x-chat, since I couldn't get a new shell open)
with the below:

[EMAIL PROTECTED]:~$ :(){ :|:|:; };:

How exactly does the ulimit work?  Why do I seem to be able to evade
limits on maximum number of processes by doing a bigger fork bomb?  I
would have thought that the above would have terminated much sooner,
since it was spawning x^3 processes instead of x^2 for iteration x and
should have hit 4096 a lot sooner.


-- 
We will enslave their women, eat their children and rape their
cattle!
 -- Bosc, Evil alien overlord from the fifth dimension
Anti-Spam:  https://bugzilla.mozilla.org/show_bug.cgi?id=229686
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: libata and sata?

2006-12-18 Thread John Richard Moser


Alan wrote:
>> I no longer have two kernels to test through; I can't tell if the speed
>> is back or not.  Nothing in dmesg tells me if SATA is using DMA or
>> 32-bit IO support though, so I don't know... lack of knowledge over here
>> is killing me for troubleshooting this on my own.
> 
> The dmesg message shows the mode selected. It should be the highest speed
> but in one or two cases it selects UDMA33 only. I've fixed one of those
> caused by us relying on a bit not defined in older controllers. We've
> still got a case in the newer chips where BIOS setup doesn't set the
> flags properly. Old IDE has a hackish workaround for that and I'll
> probably end up porting it over.
> 
> 

It seems the highest speed here is UDMA/133.  That should be right...

I've let this go for now; except someone just brought up that copying
from one SATA drive to another slows Ubuntu to a crawl (which is what
I'm using, hence my dmesg should be relevant).  On my end I'm not
noticing; VLC used to hang the system horribly while trying to read like
20M videos (hard disk light on the whole time), now it behaves.


[   25.411977] sata_via :00:0f.0: version 2.0
[   25.411992] ACPI: PCI Interrupt :00:0f.0[B] -> Link [ALKA] -> GSI
20 (level, low) -> IRQ 18
[   25.412004] sata_via :00:0f.0: routed to hard irq line 11
[   25.412057] ata3: SATA max UDMA/133 cmd 0x9400 ctl 0x9802 bmdma
0xA400 irq 18
[   25.412363] ata4: SATA max UDMA/133 cmd 0x9C00 ctl 0xA002 bmdma
0xA408 irq 18
[   25.412380] scsi2 : sata_via
[   25.415286] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
[   25.598514] ieee1394: Host added: ID:BUS[0-00:1023]
GUID[]
[   25.613389] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   25.738290] usb 2-1: device not accepting address 2, error -71
[   25.764951] ata3.00: ATA-7, max UDMA/133, 240121728 sectors: LBA
[   25.764954] ata3.00: ata3: dev 0 multi count 16
[   25.765730] ata3.00: configured for UDMA/133
[   25.765741] scsi3 : sata_via
[   25.967113] ata4: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
[   25.977712] ATA: abnormal status 0x7F on port 0x9C07
[   25.977852] scsi 2:0:0:0: Direct-Access ATA  Maxtor 6Y120M0
 YAR5 PQ: 0 ANSI: 5

-- 
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: libata and sata?

2006-12-18 Thread John Richard Moser


Alan wrote:
 I no longer have two kernels to test through; I can't tell if the speed
 is back or not.  Nothing in dmesg tells me if SATA is using DMA or
 32-bit IO support though, so I don't know... lack of knowledge over here
 is killing me for troubleshooting this on my own.
 
 The dmesg message shows the mode selected. It should be the highest speed
 but in one or two cases it selects UDMA33 only. I've fixed one of those
 caused by us relying on a bit not defined in older controllers. We've
 still got a case in the newer chips where BIOS setup doesn't set the
 flags properly. Old IDE has a hackish workaround for that and I'll
 probably end up porting it over.
 
 

It seems the highest speed here is UDMA/133.  That should be right...

I've let this go for now; except someone just brought up that copying
from one SATA drive to another slows Ubuntu to a crawl (which is what
I'm using, hence my dmesg should be relevant).  On my end I'm not
noticing; VLC used to hang the system horribly while trying to read like
20M videos (hard disk light on the whole time), now it behaves.


[   25.411977] sata_via :00:0f.0: version 2.0
[   25.411992] ACPI: PCI Interrupt :00:0f.0[B] - Link [ALKA] - GSI
20 (level, low) - IRQ 18
[   25.412004] sata_via :00:0f.0: routed to hard irq line 11
[   25.412057] ata3: SATA max UDMA/133 cmd 0x9400 ctl 0x9802 bmdma
0xA400 irq 18
[   25.412363] ata4: SATA max UDMA/133 cmd 0x9C00 ctl 0xA002 bmdma
0xA408 irq 18
[   25.412380] scsi2 : sata_via
[   25.415286] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
[   25.598514] ieee1394: Host added: ID:BUS[0-00:1023]
GUID[]
[   25.613389] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   25.738290] usb 2-1: device not accepting address 2, error -71
[   25.764951] ata3.00: ATA-7, max UDMA/133, 240121728 sectors: LBA
[   25.764954] ata3.00: ata3: dev 0 multi count 16
[   25.765730] ata3.00: configured for UDMA/133
[   25.765741] scsi3 : sata_via
[   25.967113] ata4: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
[   25.977712] ATA: abnormal status 0x7F on port 0x9C07
[   25.977852] scsi 2:0:0:0: Direct-Access ATA  Maxtor 6Y120M0
 YAR5 PQ: 0 ANSI: 5

-- 
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


libata and sata?

2006-12-12 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

A while back my distro moved to libata for sata_via.  I was since
confused; my disk seemed a lot slower, and it looked like DMA was off.
I'm not sure how SATA works; is it even possible to enable/disable
32-bit IO and DMA?  Or are those just on?

sata_via   11524  4
libata112660  3 ata_generic,pata_via,sata_via

~$ sudo hdparm -d1 -c1 -u1 /dev/sda

/dev/sda:
 setting 32-bit IO_support flag to 1
 HDIO_SET_32BIT failed: Invalid argument
 setting unmaskirq to 1 (on)
 HDIO_SET_UNMASKINTR failed: Inappropriate ioctl for device
 setting using_dma to 1 (on)
 HDIO_SET_DMA failed: Inappropriate ioctl for device
 IO_support   =  0 (default 16-bit)

I no longer have two kernels to test through; I can't tell if the speed
is back or not.  Nothing in dmesg tells me if SATA is using DMA or
32-bit IO support though, so I don't know... lack of knowledge over here
is killing me for troubleshooting this on my own.

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRX7YeAs1xW0HCTEFAQIZ5w/9GCEP1LMCqTAa366wSg6WcwomBL1fsSik
xrZJ2ZGAzKyAkKA6DLDKjJ12fG1Pueri5qj7Cx8iRgPOFTnXg3+zNdC26uWJ2cSR
hlXI/GkjMAaB2yXidPHkfpsh1fROQoiWPy+wQ8ppXVfW/xwgFUPFst5OWoU/HSbn
8Y72tUDgsj5QjWtn6XendpND8kq59Aqw4A+WG8ThW5FyRNXa5zyd8IW6bSmLGKAa
+/PCp937aK6us8LAqlkjqHaqN+JavS+lTxFXb0ZhStKteoGo0z79IvuwnRbNtJR5
XwDBzWhic1IDTV5NOyxDQ0fBoVTllIKeY55v84lzbRrYJOIiKaze1dyDPPT/nAlo
2nS1rXP+X+3RxqUrD5kXorGSHdTqVQoBINfnntrKTkNqAKUNQ4YxlBmwzsoYsIHO
/h0hCEFoIp1wFokKEP6/Dz6TM63G3m+tadPLpSbQ2u4yEg8N8vSZeix61EieNoAU
nR8YZW52BK6rGc0/IrJRE2fVd90gxSjvN2T3o3Du+cjPHdYQhlvV3GFcIsFVIp3Y
GkXO9T29HwgZO/KuA02vhjqgP2WvC5qeV4ansbukGBBi8jaanm6qf7PcsN0/1bNr
bWqYCfzvIuIg/FPiK7XzkmVJ6cipBbCHEGqBQiQn6D0lpA715Me+T8uTH3HPUX4l
oclOnqemO+w=
=TU0m
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


libata and sata?

2006-12-12 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

A while back my distro moved to libata for sata_via.  I was since
confused; my disk seemed a lot slower, and it looked like DMA was off.
I'm not sure how SATA works; is it even possible to enable/disable
32-bit IO and DMA?  Or are those just on?

sata_via   11524  4
libata112660  3 ata_generic,pata_via,sata_via

~$ sudo hdparm -d1 -c1 -u1 /dev/sda

/dev/sda:
 setting 32-bit IO_support flag to 1
 HDIO_SET_32BIT failed: Invalid argument
 setting unmaskirq to 1 (on)
 HDIO_SET_UNMASKINTR failed: Inappropriate ioctl for device
 setting using_dma to 1 (on)
 HDIO_SET_DMA failed: Inappropriate ioctl for device
 IO_support   =  0 (default 16-bit)

I no longer have two kernels to test through; I can't tell if the speed
is back or not.  Nothing in dmesg tells me if SATA is using DMA or
32-bit IO support though, so I don't know... lack of knowledge over here
is killing me for troubleshooting this on my own.

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRX7YeAs1xW0HCTEFAQIZ5w/9GCEP1LMCqTAa366wSg6WcwomBL1fsSik
xrZJ2ZGAzKyAkKA6DLDKjJ12fG1Pueri5qj7Cx8iRgPOFTnXg3+zNdC26uWJ2cSR
hlXI/GkjMAaB2yXidPHkfpsh1fROQoiWPy+wQ8ppXVfW/xwgFUPFst5OWoU/HSbn
8Y72tUDgsj5QjWtn6XendpND8kq59Aqw4A+WG8ThW5FyRNXa5zyd8IW6bSmLGKAa
+/PCp937aK6us8LAqlkjqHaqN+JavS+lTxFXb0ZhStKteoGo0z79IvuwnRbNtJR5
XwDBzWhic1IDTV5NOyxDQ0fBoVTllIKeY55v84lzbRrYJOIiKaze1dyDPPT/nAlo
2nS1rXP+X+3RxqUrD5kXorGSHdTqVQoBINfnntrKTkNqAKUNQ4YxlBmwzsoYsIHO
/h0hCEFoIp1wFokKEP6/Dz6TM63G3m+tadPLpSbQ2u4yEg8N8vSZeix61EieNoAU
nR8YZW52BK6rGc0/IrJRE2fVd90gxSjvN2T3o3Du+cjPHdYQhlvV3GFcIsFVIp3Y
GkXO9T29HwgZO/KuA02vhjqgP2WvC5qeV4ansbukGBBi8jaanm6qf7PcsN0/1bNr
bWqYCfzvIuIg/FPiK7XzkmVJ6cipBbCHEGqBQiQn6D0lpA715Me+T8uTH3HPUX4l
oclOnqemO+w=
=TU0m
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: noexec=on doesn't work

2006-12-11 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Eric Piel wrote:
> 12/09/2006 09:03 PM, Kyle McMartin wrote/a écrit:
>> On Sat, Dec 09, 2006 at 02:34:47PM -0500, John Richard Moser wrote:
>>> I have filed this as a distro bug with Ubuntu; it may be their issue, I
>>> haven't dug deep enough to find out.  I am posting this here to disperse
>>> the information breadth-first instead of depth-first, which will shorten
>>> the bug's life cycle if it turns out to be an upstream bug.
>>>
>>
>> NX requires the 64-bit page table entries (ie, PAE) which requires
>> CONFIG_HIGHMEM64G.
> 
> Somehow there is a problem: a user can explicitly put "noexec=on" and it
> will be silently ignored if the kernel doesn't have PAE support. I guess
> that currently no message is written because "noexec=on" is the
> _default_. Still, it would be fair to the user who added "noexec=on" on
> its command line that if it is not respected, either because the
> hardware doesn't support it or because the kernel doesn't support it, we
> display a warning saying it's hopeless.
> 

Would have saved me and others a lot of trouble if this happened, yes; I
wouldn't have written a test case and wtf'd at it for 5 days.  :)

> I'll send a patch if it seems meaningful to you,

Telling may be better than letting the user think; then again any
knowledgeable user should know based on his config (yes I know, by this
logic I should have known about the HIGHMEM64G thing).

> c u
> Eric
> 
> 
> 
> 

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRX3VJgs1xW0HCTEFAQLmaRAAj1e55b6if2lLTEbFNtylIn2aikAuPC87
wCqzvmdp/NBxUcIgXESdQeCDPxPuNK6OUCT6dtPTNCMu15wn7bfq3QUsXCR6z4za
lI7nBzIhU1ZH6HaGMm2d2MAuXfOg1I+SFEokOzXwh8db6HXGvH8DjP0mDLtKVxKP
yYjUXd8ZK3RPwU7eHUPN/V9s1v0ekc/1uFIlBBQHmzA0la/D32NcwhuCVsTEA8Ne
iix3QqBTn3p3UnD7LhnqaIKfBQEDTKfRnuWeGsf6L764cbyMaoga/6E6S7E8P2Jw
X+D940tAylrG8uH0CnmCDVzEGEPmozvN8Kk+UmSSwzgiFMQ3RlJaBbYEX9VsvqBZ
uIC77KVRHsKc+/nRYfYnDWoXRapWJTqVJfC+Ouuj1pm3NNptaHjSgpsgtHde6MuJ
ZZvvFhjN1iedDSCzRRYP4OLKTvomdiIQ9XrKPdfkqUvSgJZS7/zvCn+q6mZZDlqc
SthGcf9wCTSplGNwXzeIwMA14DGN6zZabA4ZTHNeyrLMAjzCrzd4/T8DSNGTyT0d
EotN0paFP5p7rgY37o7D+smm7m2V+zfGMn8iQr64E/xUDlySbEJKuea7VANzTj/1
FtLSb4rQRgA0yNaeCFuNQkvaCtn0U0/Ot/E7GQM53Hjr43mq2Pienc6+U/1KHFje
cmZt2/ZbzeM=
=pgCd
-END PGP SIGNATURE-

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: noexec=on doesn't work

2006-12-11 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Eric Piel wrote:
 12/09/2006 09:03 PM, Kyle McMartin wrote/a écrit:
 On Sat, Dec 09, 2006 at 02:34:47PM -0500, John Richard Moser wrote:
 I have filed this as a distro bug with Ubuntu; it may be their issue, I
 haven't dug deep enough to find out.  I am posting this here to disperse
 the information breadth-first instead of depth-first, which will shorten
 the bug's life cycle if it turns out to be an upstream bug.


 NX requires the 64-bit page table entries (ie, PAE) which requires
 CONFIG_HIGHMEM64G.
 
 Somehow there is a problem: a user can explicitly put noexec=on and it
 will be silently ignored if the kernel doesn't have PAE support. I guess
 that currently no message is written because noexec=on is the
 _default_. Still, it would be fair to the user who added noexec=on on
 its command line that if it is not respected, either because the
 hardware doesn't support it or because the kernel doesn't support it, we
 display a warning saying it's hopeless.
 

Would have saved me and others a lot of trouble if this happened, yes; I
wouldn't have written a test case and wtf'd at it for 5 days.  :)

 I'll send a patch if it seems meaningful to you,

Telling may be better than letting the user think; then again any
knowledgeable user should know based on his config (yes I know, by this
logic I should have known about the HIGHMEM64G thing).

 c u
 Eric
 
 
 
 

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRX3VJgs1xW0HCTEFAQLmaRAAj1e55b6if2lLTEbFNtylIn2aikAuPC87
wCqzvmdp/NBxUcIgXESdQeCDPxPuNK6OUCT6dtPTNCMu15wn7bfq3QUsXCR6z4za
lI7nBzIhU1ZH6HaGMm2d2MAuXfOg1I+SFEokOzXwh8db6HXGvH8DjP0mDLtKVxKP
yYjUXd8ZK3RPwU7eHUPN/V9s1v0ekc/1uFIlBBQHmzA0la/D32NcwhuCVsTEA8Ne
iix3QqBTn3p3UnD7LhnqaIKfBQEDTKfRnuWeGsf6L764cbyMaoga/6E6S7E8P2Jw
X+D940tAylrG8uH0CnmCDVzEGEPmozvN8Kk+UmSSwzgiFMQ3RlJaBbYEX9VsvqBZ
uIC77KVRHsKc+/nRYfYnDWoXRapWJTqVJfC+Ouuj1pm3NNptaHjSgpsgtHde6MuJ
ZZvvFhjN1iedDSCzRRYP4OLKTvomdiIQ9XrKPdfkqUvSgJZS7/zvCn+q6mZZDlqc
SthGcf9wCTSplGNwXzeIwMA14DGN6zZabA4ZTHNeyrLMAjzCrzd4/T8DSNGTyT0d
EotN0paFP5p7rgY37o7D+smm7m2V+zfGMn8iQr64E/xUDlySbEJKuea7VANzTj/1
FtLSb4rQRgA0yNaeCFuNQkvaCtn0U0/Ot/E7GQM53Hjr43mq2Pienc6+U/1KHFje
cmZt2/ZbzeM=
=pgCd
-END PGP SIGNATURE-

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: PAE/NX without performance drain?

2006-12-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chuck Ebbert wrote:
> In-Reply-To: <[EMAIL PROTECTED]>
> 
> On Sat, 09 Dec 2006 15:39:30 -0500, John Richard Moser wrote:
> 
>> Is it possible to give some other way to get the hardware NX bit working
>> in 32-bit mode, without the apparently massive performance penalty of
>> HIGHMEM64?
> 
> If your hardware can run the x86_64 kernel, try using that with your
> i386 userspace.  It works here...
> 

I hear that breaks USB printing.  Also I'm interested in getting it
working for other people, i.e. shipping with working NX.

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXzlVgs1xW0HCTEFAQKGcQ//Q91/SNOSQQkRdOQLf6KAYfKbKSt5b9qm
EgpH7TdeJyFq+pZb90BKd0Sr7h245vr2q4+xufuZ51gxMIqc/+UZ6D0bttUbcE10
Pja/i0s3havWMccEs/60NqnM8xnV82IOUZORBPJKVoBo39pHBOGyRjkBJFjVOjQO
8baJN67fa1gTPxvnhS1Xb7LqxpJqGLygjCieofFxxLh7UJbvOVNoJLXyphMNA3AE
uwrut3HDLTPw7XW/klc1y8bFIOVf1RI9YLUiQZJPcyBTaEKntuFOzVbE1X1J5CDj
/96JE+oqqg+Y3ysyJY1kv7Rwo+zWAr/ARTcK+q9UlJXFabWDSb9WC6mS612YhISM
gus3P76oFZ27irVHHVlDKM6V9Uk7B2S5fZnjaiLV+yQ9IlxVInQohjn2aPpp+7Oj
zfWogpjjVMyWNnOdJ1PttvH2OCykNuMmE+YclsXt5GH8HobLlnOCBdfZYnH9hYmm
N//EoRC7HEX2mDcV0LwIhHiWOQxoPnhB3qDQnG1F/KNK7MYF9mpDtryoptDmu2y/
568V2sm/bx9JOKb7Dy5p2k8SAApUCnGZKVQ+JRJ7FxoIMpOZd5MVGy1cROroJW/x
LCuHNjm+tttMBuzn4R9LACp6QdNdW58ygbwzAL9HuHTeOtJjAVwdrvKOLjMCk23Q
qHv5gZB4NLs=
=ijVN
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: PAE/NX without performance drain?

2006-12-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chuck Ebbert wrote:
 In-Reply-To: [EMAIL PROTECTED]
 
 On Sat, 09 Dec 2006 15:39:30 -0500, John Richard Moser wrote:
 
 Is it possible to give some other way to get the hardware NX bit working
 in 32-bit mode, without the apparently massive performance penalty of
 HIGHMEM64?
 
 If your hardware can run the x86_64 kernel, try using that with your
 i386 userspace.  It works here...
 

I hear that breaks USB printing.  Also I'm interested in getting it
working for other people, i.e. shipping with working NX.

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXzlVgs1xW0HCTEFAQKGcQ//Q91/SNOSQQkRdOQLf6KAYfKbKSt5b9qm
EgpH7TdeJyFq+pZb90BKd0Sr7h245vr2q4+xufuZ51gxMIqc/+UZ6D0bttUbcE10
Pja/i0s3havWMccEs/60NqnM8xnV82IOUZORBPJKVoBo39pHBOGyRjkBJFjVOjQO
8baJN67fa1gTPxvnhS1Xb7LqxpJqGLygjCieofFxxLh7UJbvOVNoJLXyphMNA3AE
uwrut3HDLTPw7XW/klc1y8bFIOVf1RI9YLUiQZJPcyBTaEKntuFOzVbE1X1J5CDj
/96JE+oqqg+Y3ysyJY1kv7Rwo+zWAr/ARTcK+q9UlJXFabWDSb9WC6mS612YhISM
gus3P76oFZ27irVHHVlDKM6V9Uk7B2S5fZnjaiLV+yQ9IlxVInQohjn2aPpp+7Oj
zfWogpjjVMyWNnOdJ1PttvH2OCykNuMmE+YclsXt5GH8HobLlnOCBdfZYnH9hYmm
N//EoRC7HEX2mDcV0LwIhHiWOQxoPnhB3qDQnG1F/KNK7MYF9mpDtryoptDmu2y/
568V2sm/bx9JOKb7Dy5p2k8SAApUCnGZKVQ+JRJ7FxoIMpOZd5MVGy1cROroJW/x
LCuHNjm+tttMBuzn4R9LACp6QdNdW58ygbwzAL9HuHTeOtJjAVwdrvKOLjMCk23Q
qHv5gZB4NLs=
=ijVN
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


PAE/NX without performance drain?

2006-12-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Apparently (as I've been told today) using a hardware NX bit in a 32-bit
x86 kernel requires PAE mode.  PAE mode is enabled with HIGHMEM64, which
is (apparently) extremely slow.

Is it possible to give some other way to get the hardware NX bit working
in 32-bit mode, without the apparently massive performance penalty of
HIGHMEM64?


- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXsfAAs1xW0HCTEFAQIb3xAAhuVgOGGff6N3BQFUjej6PozDDcc56C2P
pS+6JOdUaFWNfBbBg9FWbeGkW8thwNOfRRaTgE3TXar44djwd8rmjfFx9siWenue
sYbdn61LYWsTRsRuS3noD49Dn3vj/sOv8pPEiz6ZPYd3kgkuipQHNVWUUjR7mne/
9o5P4ajae4gcml7z3CcQVO8CkCFpCqQUPwXz2yVBPGEi4DEJHrNIlr8mbP2uBPkD
nXcMY5KmHovDyueihoaVInzBdIhNGUSFEc6mfZS0bluCLaNUudWJCZDjEwunHS7M
ngySKIQC2U3I0tgdok00Szum2NRlwclDNpoQP4x9577v/rCTVKfOxv+CioK+DXG2
QnYBPuicI31f2++itubnidLgCiBtjbuwHaz0OMMg9Ix7xws4WX2BLykuenRAxN9f
F+TZuJJq8sD1CfTveomZq7UP8mpBECXB+HRTMNdpIy+QJ19Eg8p4N/l2pep1hvDv
UA6tafSopIXEzcQKQ6Yi1MI8Du79O1zpTWzS+Hwgl+t3XfkI2e04wq7D4aN2ZMlw
b3S3h6Lp4I9EgqPLBnu+s2/AkRa/AxZc3eGbgf8Fz75sbDYvRnuhXzSxAmmJru5D
d51uB0GuHbiser+axWIj886pAOLPa2KGYpAjm7gtmVWFBNhzx5gnQRJkrxIcg4ew
JOb4VR4yfB8=
=QCVY
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: noexec=on doesn't work

2006-12-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Kyle McMartin wrote:
> On Sat, Dec 09, 2006 at 02:34:47PM -0500, John Richard Moser wrote:
>> I have filed this as a distro bug with Ubuntu; it may be their issue, I
>> haven't dug deep enough to find out.  I am posting this here to disperse
>> the information breadth-first instead of depth-first, which will shorten
>> the bug's life cycle if it turns out to be an upstream bug.
>>
> 
> NX requires the 64-bit page table entries (ie, PAE) which requires
> CONFIG_HIGHMEM64G.
> 

Thanks.

> Why are you posting to linux-kernel@ without even checking the upstream
> kernel, anyway?
> 

Because I had no real clue how to do that and didn't have enough
information to fill in why it wouldn't work.  It also showed up on
Gentoo so I took that as close enough.  :)

Sorry.  :)

> plonk.
> 
>   Kyle M.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXsYGgs1xW0HCTEFAQKiqQ/+Ksb3v2gptqaYbCDIJgG723e2DT/Waz26
MtlvxJVPhkHB7JcFFsCojeLNj5rkvaZhuf1VOUfWjiRh9qi9PgrfGWcdBVUacdnK
gAmdoRTlxobID+HajrWT6kGx5P64XkccZJxMVD59yXbxLzAp8NAus21UbeYcdHsF
hCDCNjj6CcUF02hl/Nm1y1I3/rmh4T6hfR0AW/NpkR9Z01GgRkUDvGmsZESNRovA
kXmfjN7i7b/qH48619o+NivHXEUQK0orHAqV8EOoeF4D5TAewoJBNxUffGdWt20E
diOY21awg66gTxEqWA/Owd2DasrHmZ2WcI+73bade1wr2NR7thTtQNoCvqR/0AZK
l+2GE7E9Hreiay8aMLtZNW8DxIE0PpMA+PJPgwVZFbKLOb1x2VqXljSg1djUGjkJ
O2WtSdNNIL04LQkh8ffJTq7eozZfNjT8e/ctKYk/OvaBso8GQbpJVxr/GVYGmQzW
EOA9uAVaJeRBm7KHud/ExOI0PDxQbJfVF/zo5NdY/kV8mJPhSuR97s11+3XUOwrK
uqlRwA6k3cEoPP3tmi6nZ70oq9lTWMl+v05dESAu3r1J6FBSMMXfU6OUHjIKDwkY
MM5762xuNTsfuePtA9LrXzCMruU31rHIFxcWu1b8M/elJ/D4puBkp2SBxu7AIvdZ
ara7KkuWPJE=
=OplU
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


noexec=on doesn't work

2006-12-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'm running on an Athlon 64 in 32-bit mode, running 32-bit Ubuntu with
kernel 2.6.19 (Ubuntu version 2.6.19-7-generic for the curious;
compiled for 586).  Apparently, 'noexec=on' on the kernel command line
does nothing; the NX bit seems to not work.

Chunk of my /proc/cpuinfo:

flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext lm
3dnowext 3dnow up ts fid vid ttp

Attached to the relevant Ubuntu bug is a test program that attempts to
disable PROT_EXEC for a page of memory containing (I believe) the entry
point of a function.  It's compiled as such:


$ gcc -O2 -shared -fpic test_so.c -o test_so.so
$ gcc -O2 test.c -o test -ldl


Running it on AMD64-ubuntu gives the following output:

$ ./test
Test function run successfully!
Segmentation fault

This is good; I tried to execute non-executable memory, it segfaulted.
However, 32-bit Ubuntu on the Athlon64 gives the following:

$ ./test
Test function run successfully!
Test function run successfully!

Apparently noexec is not being honored.

I have filed this as a distro bug with Ubuntu; it may be their issue, I
haven't dug deep enough to find out.  I am posting this here to disperse
the information breadth-first instead of depth-first, which will shorten
the bug's life cycle if it turns out to be an upstream bug.

This also appears to happen on 2.6.15.

Ubuntu bug:
https://bugs.launchpad.net/distros/ubuntu/+source/linux-source-2.6.19/+bug/75157


- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXsP1gs1xW0HCTEFAQLAsQ//XUdfVAK6Fp225mUXv+ApLUqnZFyfca4z
n0e1WFVYGQcplKQpdn+MGxzoEXc4xo4GnC2n0qfbyp6l7GN9uLvZS3myLOB6nfMH
AFUeDmpc44Q40Nq94UKxrMmMvyfs0lEOBIRjQzvzWonYjNXFQuIAWS8xsFducjaF
IK7E79cvC9d/mlSOQMgcFSt8hW35MqAI0/Au3iViReE+Qm/qUw2PWw9lUpRTgVZf
SrKqQq8HNM5SCJDFyu/zVltEwAPqTvn6g0p9BlCMZhugiBIaoaeAvO8ZYhxr+0JG
DxIEoaA4Ij/AnL7u2LT1lz/oDKhH4RImko+z2NubmFNiBJiODmX6RLsciWy3mgtx
ulYuESmIRjb1hOwTlFqvpOMNncCyCGjZwCw8/O7SxosDFx4Gvd9IqPFJ6dKoj2fp
oXESE5uIWbJGtbPyB2jsaz5t3PZ9DGrzX/P9ykPLhGmwiM23Nrc36DW/BH9/+ESZ
Uv1SbIgIWV8tz2cO+YJgbrrSD+wuvizkzv4Va31tyPyPx3QfkFeLxOm7nxK1Yo6M
XmckCqXMNOUAMnENMf5N5tnYkIZSnEYrNTTWuIPRS8Xzo37HOfDJL1uyEJPPmxoR
K3/wH+LkecVIaWaP3+UH9VlieKnrC4+guFj4QdnXFzAqNQ55vEpDQs/sbFvjO0n9
IKTdKQKnsgs=
=0wST
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


noexec=on doesn't work

2006-12-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'm running on an Athlon 64 in 32-bit mode, running 32-bit Ubuntu with
kernel 2.6.19 (Ubuntu version 2.6.19-7-generic for the curious;
compiled for 586).  Apparently, 'noexec=on' on the kernel command line
does nothing; the NX bit seems to not work.

Chunk of my /proc/cpuinfo:

flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext lm
3dnowext 3dnow up ts fid vid ttp

Attached to the relevant Ubuntu bug is a test program that attempts to
disable PROT_EXEC for a page of memory containing (I believe) the entry
point of a function.  It's compiled as such:


$ gcc -O2 -shared -fpic test_so.c -o test_so.so
$ gcc -O2 test.c -o test -ldl


Running it on AMD64-ubuntu gives the following output:

$ ./test
Test function run successfully!
Segmentation fault

This is good; I tried to execute non-executable memory, it segfaulted.
However, 32-bit Ubuntu on the Athlon64 gives the following:

$ ./test
Test function run successfully!
Test function run successfully!

Apparently noexec is not being honored.

I have filed this as a distro bug with Ubuntu; it may be their issue, I
haven't dug deep enough to find out.  I am posting this here to disperse
the information breadth-first instead of depth-first, which will shorten
the bug's life cycle if it turns out to be an upstream bug.

This also appears to happen on 2.6.15.

Ubuntu bug:
https://bugs.launchpad.net/distros/ubuntu/+source/linux-source-2.6.19/+bug/75157


- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXsP1gs1xW0HCTEFAQLAsQ//XUdfVAK6Fp225mUXv+ApLUqnZFyfca4z
n0e1WFVYGQcplKQpdn+MGxzoEXc4xo4GnC2n0qfbyp6l7GN9uLvZS3myLOB6nfMH
AFUeDmpc44Q40Nq94UKxrMmMvyfs0lEOBIRjQzvzWonYjNXFQuIAWS8xsFducjaF
IK7E79cvC9d/mlSOQMgcFSt8hW35MqAI0/Au3iViReE+Qm/qUw2PWw9lUpRTgVZf
SrKqQq8HNM5SCJDFyu/zVltEwAPqTvn6g0p9BlCMZhugiBIaoaeAvO8ZYhxr+0JG
DxIEoaA4Ij/AnL7u2LT1lz/oDKhH4RImko+z2NubmFNiBJiODmX6RLsciWy3mgtx
ulYuESmIRjb1hOwTlFqvpOMNncCyCGjZwCw8/O7SxosDFx4Gvd9IqPFJ6dKoj2fp
oXESE5uIWbJGtbPyB2jsaz5t3PZ9DGrzX/P9ykPLhGmwiM23Nrc36DW/BH9/+ESZ
Uv1SbIgIWV8tz2cO+YJgbrrSD+wuvizkzv4Va31tyPyPx3QfkFeLxOm7nxK1Yo6M
XmckCqXMNOUAMnENMf5N5tnYkIZSnEYrNTTWuIPRS8Xzo37HOfDJL1uyEJPPmxoR
K3/wH+LkecVIaWaP3+UH9VlieKnrC4+guFj4QdnXFzAqNQ55vEpDQs/sbFvjO0n9
IKTdKQKnsgs=
=0wST
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: noexec=on doesn't work

2006-12-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Kyle McMartin wrote:
 On Sat, Dec 09, 2006 at 02:34:47PM -0500, John Richard Moser wrote:
 I have filed this as a distro bug with Ubuntu; it may be their issue, I
 haven't dug deep enough to find out.  I am posting this here to disperse
 the information breadth-first instead of depth-first, which will shorten
 the bug's life cycle if it turns out to be an upstream bug.

 
 NX requires the 64-bit page table entries (ie, PAE) which requires
 CONFIG_HIGHMEM64G.
 

Thanks.

 Why are you posting to linux-kernel@ without even checking the upstream
 kernel, anyway?
 

Because I had no real clue how to do that and didn't have enough
information to fill in why it wouldn't work.  It also showed up on
Gentoo so I took that as close enough.  :)

Sorry.  :)

 plonk.
 
   Kyle M.
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
 

- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXsYGgs1xW0HCTEFAQKiqQ/+Ksb3v2gptqaYbCDIJgG723e2DT/Waz26
MtlvxJVPhkHB7JcFFsCojeLNj5rkvaZhuf1VOUfWjiRh9qi9PgrfGWcdBVUacdnK
gAmdoRTlxobID+HajrWT6kGx5P64XkccZJxMVD59yXbxLzAp8NAus21UbeYcdHsF
hCDCNjj6CcUF02hl/Nm1y1I3/rmh4T6hfR0AW/NpkR9Z01GgRkUDvGmsZESNRovA
kXmfjN7i7b/qH48619o+NivHXEUQK0orHAqV8EOoeF4D5TAewoJBNxUffGdWt20E
diOY21awg66gTxEqWA/Owd2DasrHmZ2WcI+73bade1wr2NR7thTtQNoCvqR/0AZK
l+2GE7E9Hreiay8aMLtZNW8DxIE0PpMA+PJPgwVZFbKLOb1x2VqXljSg1djUGjkJ
O2WtSdNNIL04LQkh8ffJTq7eozZfNjT8e/ctKYk/OvaBso8GQbpJVxr/GVYGmQzW
EOA9uAVaJeRBm7KHud/ExOI0PDxQbJfVF/zo5NdY/kV8mJPhSuR97s11+3XUOwrK
uqlRwA6k3cEoPP3tmi6nZ70oq9lTWMl+v05dESAu3r1J6FBSMMXfU6OUHjIKDwkY
MM5762xuNTsfuePtA9LrXzCMruU31rHIFxcWu1b8M/elJ/D4puBkp2SBxu7AIvdZ
ara7KkuWPJE=
=OplU
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


PAE/NX without performance drain?

2006-12-09 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Apparently (as I've been told today) using a hardware NX bit in a 32-bit
x86 kernel requires PAE mode.  PAE mode is enabled with HIGHMEM64, which
is (apparently) extremely slow.

Is it possible to give some other way to get the hardware NX bit working
in 32-bit mode, without the apparently massive performance penalty of
HIGHMEM64?


- --
We will enslave their women, eat their children and rape their
cattle!
  -- Bosc, Evil alien overlord from the fifth dimension
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRXsfAAs1xW0HCTEFAQIb3xAAhuVgOGGff6N3BQFUjej6PozDDcc56C2P
pS+6JOdUaFWNfBbBg9FWbeGkW8thwNOfRRaTgE3TXar44djwd8rmjfFx9siWenue
sYbdn61LYWsTRsRuS3noD49Dn3vj/sOv8pPEiz6ZPYd3kgkuipQHNVWUUjR7mne/
9o5P4ajae4gcml7z3CcQVO8CkCFpCqQUPwXz2yVBPGEi4DEJHrNIlr8mbP2uBPkD
nXcMY5KmHovDyueihoaVInzBdIhNGUSFEc6mfZS0bluCLaNUudWJCZDjEwunHS7M
ngySKIQC2U3I0tgdok00Szum2NRlwclDNpoQP4x9577v/rCTVKfOxv+CioK+DXG2
QnYBPuicI31f2++itubnidLgCiBtjbuwHaz0OMMg9Ix7xws4WX2BLykuenRAxN9f
F+TZuJJq8sD1CfTveomZq7UP8mpBECXB+HRTMNdpIy+QJ19Eg8p4N/l2pep1hvDv
UA6tafSopIXEzcQKQ6Yi1MI8Du79O1zpTWzS+Hwgl+t3XfkI2e04wq7D4aN2ZMlw
b3S3h6Lp4I9EgqPLBnu+s2/AkRa/AxZc3eGbgf8Fz75sbDYvRnuhXzSxAmmJru5D
d51uB0GuHbiser+axWIj886pAOLPa2KGYpAjm7gtmVWFBNhzx5gnQRJkrxIcg4ew
JOb4VR4yfB8=
=QCVY
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Kernel profiles anyone?

2005-09-06 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Are there any recent kernel profiles?  I think from an acedemic
perspective it'd be nice to see some graphs and numbers nobody
understands showing where the longest running code paths in the kernel
occur.  It might also be nice for those latency whores (*runs to the
back and raises hand*) and those who want to increase overall
performance and efficiency; then there'd be a map showing . . .
something that only kernel hackers could possibly understand or care about.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDHkPIhDd4aOud5P8RAiQ4AJwPdaJAZchWNNtoO9zjz6AePpty/ACbBa/A
vX+9F3/Yuw68QtteomUXtqQ=
=faXl
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Kernel profiles anyone?

2005-09-06 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Are there any recent kernel profiles?  I think from an acedemic
perspective it'd be nice to see some graphs and numbers nobody
understands showing where the longest running code paths in the kernel
occur.  It might also be nice for those latency whores (*runs to the
back and raises hand*) and those who want to increase overall
performance and efficiency; then there'd be a map showing . . .
something that only kernel hackers could possibly understand or care about.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDHkPIhDd4aOud5P8RAiQ4AJwPdaJAZchWNNtoO9zjz6AePpty/ACbBa/A
vX+9F3/Yuw68QtteomUXtqQ=
=faXl
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


SELinux policies, memory protections

2005-08-13 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I was writing a section of my paper ("Designing a Secure and Friendly
Operating System") and basically describing and explaining why the
memory protection policy ("mprotect() restrictions") supplied by PaX is
a powerful security tool; and I had a thought.  PaX places the policy in
the kernel; but with LSM hooks in mprotect() and friends, it may be
possible to implement it entirely in policy.  This would be interesting,
as it would allow a system implementing the suggested enhancements to
avoid additional kernel code.

What I want to know is, what facilities does SELinux supply in the
current policy format to control the abuse of mprotect(), mmap(), and
friends; and where can I find an online reference on this?

For reference, PaX defines the following "good" mapping states where new
code can't be introduced (note that VM_READ and VM_MAYREAD are
completely ignored as they are of no consequence):

  VM_WRITE
  VM_MAYWRITE
  VM_WRITE | VM_MAYWRITE
  VM_EXEC
  VM_MAYEXEC
  VM_EXEC | VM_MAYEXEC

Of course the kernel won't allow VM_WRITE or VM_EXEC without VM_MAYWRITE
or VM_MAYEXEC, so this leaves us with

  VM_MAYWRITE
  VM_MAYEXEC
  VM_WRITE | VM_MAYWRITE
  VM_EXEC | VM_MAYEXEC


This gives us a few target guarantees:

1.  No mapping may be created in any state other than those listed above
(VM_READ and VM_READ|VM_MAYREAD are permissible in addition to any
allowed state)

2.  No mapping may transition to any state not described in (1)

3.  No existing mapping without VM_EXEC is to have VM_MAYEXEC or be
given VM_EXEC or VM_MAYEXEC at any time in the remainder of its life
cycle; this includes mappings that started with and later dropped VM_EXEC



And there are a few other things that I want guaranteed:

1.  Anonymous mappings are always created without VM_EXEC or VM_MAYEXEC

2.  Shared memory mappings are always created without VM_EXEC or
VM_MAYEXEC (this is the current case)

3.  File backed mmap() segments requesting PROT_WRITE get
VM_WRITE|VM_MAYWRITE (and VM_READ|VM_MAYREAD if PROT_READ was
requested), but never VM_EXEC or VM_MAYEXEC even if requested

4.  For certain applications, it may be necessary that we can
automatically grant VM_EXEC|VM_MAYEXEC on file-backed mappings not
requesting PROT_WRITE, if those applications assume that PROT_READ
implies PROT_EXEC (this is how PaX works)


Finally, the ability to detect if the affected area is part of a
relocation and account for that in policy would be important, because
PaX set to disable ELF text relocations breaks quite a number of things;
trying to reconstruct the memory policy from PaX with an SELinux policy
without being able to mimic the special case allowance of ELF text
relocations would be disasterous.



REFERENCES:

  http://pax.grsecurity.net/docs/mprotect.txt


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFC/r/+hDd4aOud5P8RAkCIAJ4utVh8rEZqLb3WDvM75gnkZ/+LOgCaAizc
zTSxkvYL0MFtC+QqEt//hbA=
=idfj
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


SELinux policies, memory protections

2005-08-13 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I was writing a section of my paper (Designing a Secure and Friendly
Operating System) and basically describing and explaining why the
memory protection policy (mprotect() restrictions) supplied by PaX is
a powerful security tool; and I had a thought.  PaX places the policy in
the kernel; but with LSM hooks in mprotect() and friends, it may be
possible to implement it entirely in policy.  This would be interesting,
as it would allow a system implementing the suggested enhancements to
avoid additional kernel code.

What I want to know is, what facilities does SELinux supply in the
current policy format to control the abuse of mprotect(), mmap(), and
friends; and where can I find an online reference on this?

For reference, PaX defines the following good mapping states where new
code can't be introduced (note that VM_READ and VM_MAYREAD are
completely ignored as they are of no consequence):

  VM_WRITE
  VM_MAYWRITE
  VM_WRITE | VM_MAYWRITE
  VM_EXEC
  VM_MAYEXEC
  VM_EXEC | VM_MAYEXEC

Of course the kernel won't allow VM_WRITE or VM_EXEC without VM_MAYWRITE
or VM_MAYEXEC, so this leaves us with

  VM_MAYWRITE
  VM_MAYEXEC
  VM_WRITE | VM_MAYWRITE
  VM_EXEC | VM_MAYEXEC


This gives us a few target guarantees:

1.  No mapping may be created in any state other than those listed above
(VM_READ and VM_READ|VM_MAYREAD are permissible in addition to any
allowed state)

2.  No mapping may transition to any state not described in (1)

3.  No existing mapping without VM_EXEC is to have VM_MAYEXEC or be
given VM_EXEC or VM_MAYEXEC at any time in the remainder of its life
cycle; this includes mappings that started with and later dropped VM_EXEC



And there are a few other things that I want guaranteed:

1.  Anonymous mappings are always created without VM_EXEC or VM_MAYEXEC

2.  Shared memory mappings are always created without VM_EXEC or
VM_MAYEXEC (this is the current case)

3.  File backed mmap() segments requesting PROT_WRITE get
VM_WRITE|VM_MAYWRITE (and VM_READ|VM_MAYREAD if PROT_READ was
requested), but never VM_EXEC or VM_MAYEXEC even if requested

4.  For certain applications, it may be necessary that we can
automatically grant VM_EXEC|VM_MAYEXEC on file-backed mappings not
requesting PROT_WRITE, if those applications assume that PROT_READ
implies PROT_EXEC (this is how PaX works)


Finally, the ability to detect if the affected area is part of a
relocation and account for that in policy would be important, because
PaX set to disable ELF text relocations breaks quite a number of things;
trying to reconstruct the memory policy from PaX with an SELinux policy
without being able to mimic the special case allowance of ELF text
relocations would be disasterous.



REFERENCES:

  http://pax.grsecurity.net/docs/mprotect.txt


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFC/r/+hDd4aOud5P8RAkCIAJ4utVh8rEZqLb3WDvM75gnkZ/+LOgCaAizc
zTSxkvYL0MFtC+QqEt//hbA=
=idfj
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Fault tolerance. . .

2005-07-24 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'm playing Skies of Arcadia Legends on my GameCube and noticing that
software bugs continuously produce errors (no scratch on the disk; I can
have an error, reset, play through it easy).  This leads me on and on,
but now it's lead me into thinking about fault tolerance.

Leaving the GameCube and moving to my computer, I'm wondering if
something could be done kernel-end to provide automatic application and
system fault recovery.  Thinking about it, I can come up with a few cute
ideas, though they don't really seem all that realistic.  Still, it
might be fun to share them and see what kinds of comments I get.

I'm thinking of application level fault tolerance using roll-back states
or something weird, to restore the system as affected by that
application to a point before the error.  The obvious visual effect
would be that if an application were to crash, it and potentially
interrelated applications would suddenly reset to a state a few seconds
to a few minutes earlier.

Like I said, this is a really dumb idea, but it's kind of cute to think
about.  In the 5 minutes I think about this, I'll type some crap here,
then go back to my game and leave this for you all so you can take a
break from real work and read something mildly ammusing.

Interrelating applications:

 - Tasks sharing memory are synchronized at state saves
 - Tasks having mappings to files where writing to the mapping writes to
the file have state saves synchronized starting where one could affect
another and ending where they no longer can affect eachother
  - When a process opens a file another process has open for writing,
the two both have a state save made
  - Following state saves are sychronized temporally between the tasks
  - When neither task can affect the other by writing to said file, the
state saves no longer are required to be synchronized
 - Tasks communicating over sockets are synchronized at save states
 - Tasks communicating via pipes are synchronized at save states
 - When a parent is rolled back to precede a fork(), the fork() child dies

"Synchronized" means that when the state is set to be saved for
roll-back in one task, interrelated tasks are also frozen and saved at
the same time.

Saving a state is easy:

 - Lock a spinlock
 - Add an entry to a linked list for a state save, with registers and such
 - Remove entries older than the limit for the oldest save
 - Open spinlock

Maintaining the state is also easy:

 - When a file is changed, track the changes and attach them to the last
state save
 - When memory pages are written to, cache the old copies first
(unfortunately each page has to be made CoW after every state save)

Restoring a state is also easy:

 - At untrapped sigsegv, sigabrt, roll back changes and related changes
 - Repeted incident means rolling back farther

This of course raises many questions and concerns that make this
rediculous and probably not entirely possible:

 - What about huge modifications to files in a short time?  Make a new
file, then write 10,000,000,000 bytes past the end and watch it crash.
 - What about lost work in interrelated applications?
 - Will the system state remain consistent?
 - Will it crash over and over and over?
 - Connecting to named pipes? (easily handled, not discussed here)
 - Crashes are usually trappable, and then programs exit cleanly.  They
won't care about this
 - How does a process know to change course if it gets restored?
 - We can probably use this to do some deep-and-dirty espionage, relying
on the fault tolerance when we crash something trying to leak information
 - Darth Vader will use this to find Luke Skywalker

Anyway, whatever.  Comment if you want, I just thought the idea was
cute.  Not practical, but cute.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFC5EeehDd4aOud5P8RAhK/AJ90BXofS/XPJcr5xsGFhqlf9jJiBQCfcbSG
v2Di7VqKv29jlRjoJiphy0c=
=5H6M
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Fault tolerance. . .

2005-07-24 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'm playing Skies of Arcadia Legends on my GameCube and noticing that
software bugs continuously produce errors (no scratch on the disk; I can
have an error, reset, play through it easy).  This leads me on and on,
but now it's lead me into thinking about fault tolerance.

Leaving the GameCube and moving to my computer, I'm wondering if
something could be done kernel-end to provide automatic application and
system fault recovery.  Thinking about it, I can come up with a few cute
ideas, though they don't really seem all that realistic.  Still, it
might be fun to share them and see what kinds of comments I get.

I'm thinking of application level fault tolerance using roll-back states
or something weird, to restore the system as affected by that
application to a point before the error.  The obvious visual effect
would be that if an application were to crash, it and potentially
interrelated applications would suddenly reset to a state a few seconds
to a few minutes earlier.

Like I said, this is a really dumb idea, but it's kind of cute to think
about.  In the 5 minutes I think about this, I'll type some crap here,
then go back to my game and leave this for you all so you can take a
break from real work and read something mildly ammusing.

Interrelating applications:

 - Tasks sharing memory are synchronized at state saves
 - Tasks having mappings to files where writing to the mapping writes to
the file have state saves synchronized starting where one could affect
another and ending where they no longer can affect eachother
  - When a process opens a file another process has open for writing,
the two both have a state save made
  - Following state saves are sychronized temporally between the tasks
  - When neither task can affect the other by writing to said file, the
state saves no longer are required to be synchronized
 - Tasks communicating over sockets are synchronized at save states
 - Tasks communicating via pipes are synchronized at save states
 - When a parent is rolled back to precede a fork(), the fork() child dies

Synchronized means that when the state is set to be saved for
roll-back in one task, interrelated tasks are also frozen and saved at
the same time.

Saving a state is easy:

 - Lock a spinlock
 - Add an entry to a linked list for a state save, with registers and such
 - Remove entries older than the limit for the oldest save
 - Open spinlock

Maintaining the state is also easy:

 - When a file is changed, track the changes and attach them to the last
state save
 - When memory pages are written to, cache the old copies first
(unfortunately each page has to be made CoW after every state save)

Restoring a state is also easy:

 - At untrapped sigsegv, sigabrt, roll back changes and related changes
 - Repeted incident means rolling back farther

This of course raises many questions and concerns that make this
rediculous and probably not entirely possible:

 - What about huge modifications to files in a short time?  Make a new
file, then write 10,000,000,000 bytes past the end and watch it crash.
 - What about lost work in interrelated applications?
 - Will the system state remain consistent?
 - Will it crash over and over and over?
 - Connecting to named pipes? (easily handled, not discussed here)
 - Crashes are usually trappable, and then programs exit cleanly.  They
won't care about this
 - How does a process know to change course if it gets restored?
 - We can probably use this to do some deep-and-dirty espionage, relying
on the fault tolerance when we crash something trying to leak information
 - Darth Vader will use this to find Luke Skywalker

Anyway, whatever.  Comment if you want, I just thought the idea was
cute.  Not practical, but cute.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFC5EeehDd4aOud5P8RAhK/AJ90BXofS/XPJcr5xsGFhqlf9jJiBQCfcbSG
v2Di7VqKv29jlRjoJiphy0c=
=5H6M
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


USB on zx5405us

2005-04-11 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

USB isn't working on my zv5405us on a 2.6.10 ubuntu kernel.  Or on
gentoo.  Or anything.  It works in WindowsXP though.

I can extract the error from dmesg.

Here's ACPI first (ACPI works btw)

Nvidia board detected. Ignoring ACPI timer override.
ACPI: RSDP (v000 PTLTD ) @
0x000f7260
ACPI: RSDT (v001 PTLTDRSDT   0x0604  LTP 0x) @
0x1ff7a87e
ACPI: FADT (v001 NVIDIA CK8  0x0604 PTL_ 0x000f4240) @
0x1ff7ee13
ACPI: MADT (v001 NVIDIA NV_APIC_ 0x0604  LTP 0x) @
0x1ff7ee87
ACPI: BOOT (v001 PTLTD  $SBFTBL$ 0x0604  LTP 0x0001) @
0x1ff7eee1
ACPI: SSDT (v001 PTLTD  POWERNOW 0x0604  LTP 0x0001) @
0x1ff7ef09
ACPI: DSDT (v001 NVIDIA  CK8 0x0604 MSFT 0x010e) @
0x
ACPI: Local APIC address 0xfee0
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:4 APIC version 16
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec0] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec0, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: BIOS IRQ0 pin2 override ignored.
ACPI: IRQ9 used by override.
Setting APIC routing to flat
Using ACPI (MADT) for SMP configuration information

ACPI wakeup devices:
USB0 USB1 USB2 PS2K PS2M MAC0
ACPI: (supports S0 S3 S4 S5)

NFORCE3-150: IDE controller at PCI slot :00:08.0
NFORCE3-150: chipset revision 165
NFORCE3-150: not 100% native mode: will probe irqs later
NFORCE3-150: BIOS didn't set cable bits correctly. Enabling workaround.
NFORCE3-150: :00:08.0 (rev a5) UDMA133 controller
ide0: BM-DMA at 0x2080-0x2087, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0x2088-0x208f, BIOS settings: hdc:DMA, hdd:pio



Finally, the USB messages themselves:

ohci_hcd :00:02.0: nVidia Corporation nForce3 USB 1.1
ohci_hcd :00:02.0: USB HC TakeOver failed!
ohci_hcd :00:02.0: can't reset
ohci_hcd :00:02.0: init :00:02.0 fail, -16
ohci_hcd: probe of :00:02.0 failed with error -16


lspci gives:

:00:02.0 USB Controller: nVidia Corporation nForce3 USB 1.1 (rev a5)
:00:02.1 USB Controller: nVidia Corporation nForce3 USB 1.1 (rev a5)
:00:02.2 USB Controller: nVidia Corporation nForce3 USB 2.0 (rev a2)


lsusb gives:

Bus 001 Device 001: ID :

Happens to be the USB2 stuff.


/proc/bus/usb/devices:

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480 MxCh= 6
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS= 8 #Cfgs=  1
P:  Vendor= ProdID= Rev= 2.06
S:  Manufacturer=Linux 2.6.10-5-amd64-generic ehci_hcd
S:  Product=nVidia Corporation nForce3 USB 2.0
S:  SerialNumber=:00:02.2
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   2 Ivl=256ms


A mass storage device that works on my desktop doesn't work here.  It
works in Windows though.


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCWz3chDd4aOud5P8RArB9AJ9AECG8+VrPUt8Zu7djzvl+Oi3lwgCfdD17
QbgPw+B1tbY66BjcFSpz9L4=
=0WK/
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


USB on zx5405us

2005-04-11 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

USB isn't working on my zv5405us on a 2.6.10 ubuntu kernel.  Or on
gentoo.  Or anything.  It works in WindowsXP though.

I can extract the error from dmesg.

Here's ACPI first (ACPI works btw)

Nvidia board detected. Ignoring ACPI timer override.
ACPI: RSDP (v000 PTLTD ) @
0x000f7260
ACPI: RSDT (v001 PTLTDRSDT   0x0604  LTP 0x) @
0x1ff7a87e
ACPI: FADT (v001 NVIDIA CK8  0x0604 PTL_ 0x000f4240) @
0x1ff7ee13
ACPI: MADT (v001 NVIDIA NV_APIC_ 0x0604  LTP 0x) @
0x1ff7ee87
ACPI: BOOT (v001 PTLTD  $SBFTBL$ 0x0604  LTP 0x0001) @
0x1ff7eee1
ACPI: SSDT (v001 PTLTD  POWERNOW 0x0604  LTP 0x0001) @
0x1ff7ef09
ACPI: DSDT (v001 NVIDIA  CK8 0x0604 MSFT 0x010e) @
0x
ACPI: Local APIC address 0xfee0
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:4 APIC version 16
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec0] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec0, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: BIOS IRQ0 pin2 override ignored.
ACPI: IRQ9 used by override.
Setting APIC routing to flat
Using ACPI (MADT) for SMP configuration information

ACPI wakeup devices:
USB0 USB1 USB2 PS2K PS2M MAC0
ACPI: (supports S0 S3 S4 S5)

NFORCE3-150: IDE controller at PCI slot :00:08.0
NFORCE3-150: chipset revision 165
NFORCE3-150: not 100% native mode: will probe irqs later
NFORCE3-150: BIOS didn't set cable bits correctly. Enabling workaround.
NFORCE3-150: :00:08.0 (rev a5) UDMA133 controller
ide0: BM-DMA at 0x2080-0x2087, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0x2088-0x208f, BIOS settings: hdc:DMA, hdd:pio



Finally, the USB messages themselves:

ohci_hcd :00:02.0: nVidia Corporation nForce3 USB 1.1
ohci_hcd :00:02.0: USB HC TakeOver failed!
ohci_hcd :00:02.0: can't reset
ohci_hcd :00:02.0: init :00:02.0 fail, -16
ohci_hcd: probe of :00:02.0 failed with error -16


lspci gives:

:00:02.0 USB Controller: nVidia Corporation nForce3 USB 1.1 (rev a5)
:00:02.1 USB Controller: nVidia Corporation nForce3 USB 1.1 (rev a5)
:00:02.2 USB Controller: nVidia Corporation nForce3 USB 2.0 (rev a2)


lsusb gives:

Bus 001 Device 001: ID :

Happens to be the USB2 stuff.


/proc/bus/usb/devices:

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480 MxCh= 6
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS= 8 #Cfgs=  1
P:  Vendor= ProdID= Rev= 2.06
S:  Manufacturer=Linux 2.6.10-5-amd64-generic ehci_hcd
S:  Product=nVidia Corporation nForce3 USB 2.0
S:  SerialNumber=:00:02.2
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:  If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   2 Ivl=256ms


A mass storage device that works on my desktop doesn't work here.  It
works in Windows though.


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCWz3chDd4aOud5P8RArB9AJ9AECG8+VrPUt8Zu7djzvl+Oi3lwgCfdD17
QbgPw+B1tbY66BjcFSpz9L4=
=0WK/
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: LSM hooks

2005-03-30 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
> * John Richard Moser ([EMAIL PROTECTED]) wrote:
> 
>>-BEGIN PGP SIGNED MESSAGE-
>>Hash: SHA1
>>
>>Well the LSM mailing list seems to be dead, even the archives stop at
>>Jan 15 2005.  My own mails don't come back to me (I'm subscribed).
> 
> 
> They're coming through just fine, not sure why the archives are stale
> though (I'll take a look).
> 
> 
>>So, Which version of Linux will first implement stacking in LSM as per
>>Serge Hallyn's patches?
> 
> 
> None are ready yet.  Serge is still wading through performance testing.
> There's no telling about merging without a magic eightball, a handle on
> the performance issues, and some bonafide users.
> 
> 
>>Where is the new documentation?
> 
> 
> It's in the archives ;-)  There's not much to document.  Each hook would
> potentially call a chain of modules rather than just one.
> 
> 
>>As for hooks, a few questions.
>>
>>1.  With shm_shmctl(), can I control permissions assigned to shared
>>memory using shmctl()?  I want to prevent memory protections on the shm
>>segment from being in certain combinations; however, if any changes
>>AFTER creation go through mprotect(), I can use file_mprotect() because
>>I will be preventing the same transitions everywhere.
>>
>>Shared memory mappings seem to always be in VM_WRITE | VM_MAYWRITE,
>>so they're sane by default at creation time.  Control over mprotect()
>>has this covered beyond that.
> 
> 
> And VM_MAYREAD and VM_MAYEXEC typically as well (mmap does that
> naturally), so mprotect is rather open for abuse here.
> 

Dang, I want read/write but never execute for shm.

> 
>>2.  Is shared memory always attached to without PROT_EXEC?  If not, how
>>would I control this?  What is the best hook?
> 
> 
> No, shamt(SHM_EXEC), for example, will attach to the segment with
> PROT_EXEC.
> 

Again, I need to control that. . . .

> 
>>3.  I want control over the memory protections on the stack and heap.
>>PT_GNU_STACK allows for an executable stack/heap.  Is there a way for me
>>to control this so that I can i.e. mandatorily make the stack/heap
>>PROT_READ|PROT_WRITE and never PROT_EXEC?  The only way I can see is to
>>add a hook in load_elf_binary(). . . .
>>
>>In other words, I want a module to be able to force the heap and stack
>>to be !EXEC.
> 
> 
> Thus far we intentionally left that type of work out, rather left it as
> something for access control. (you can easily stop from creating or
> changing such vmas).
> 
> 
>> do_brk() only has a check, but I wish to elegantly control what may
>>happen here.  I would need a hook that would allow me to supply
>>something to AND flags with after the following line:
>>
>>flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags;
>>
>>So for example:
>>
>>flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags;
>>  /* stacking should AND together each value and return the
>>   * result
>>   */
>>  flags &= security_vm_brk();
>>
>>I would of course have my module returning (~VM_MAYEXEC) as in PaX.
> 
> 
> I'm not convinced it makes sense as lsm work.
> 

Well, it's in PaX; I could just split it out from PaX directly, but I'm
feeling creative.  Besides, it seems dismally unlikely that porting in
chunks of PaX just "here set this in mainline and here's a config option
to disable it" is going to work; I'd suspect an LSM has a much higher
chance of getting in for some reason.

Pretty much, I'm fairly sure that with PT_PAX_FLAGS processing, parts of
PaX could survive just fine in pieces.  Particularly, mprotect()
restrictions and the high-entropy ASLR (the kernel will have ES'
low-entropy ASLR in 2.6.12, basically a performance boost due to some
issues with cache hits/misses that I don't care to pretend to fully
understand).  I'd also consider trampoline emulation to replace the
executable stack in some apps, on a per-binary basis of course.

In the case of ASLR, I'd also like to port over Brad Spengler's brute
force deterrance from GrSecurity.  The deterrance would extend the brute
force period of the 64K stack ASLR to 34.13 hours, ignoring the
potential for stack stuffing.  The 2M randomization would hold up for
1092 hours under a brute force.

With the 256M randomization from PaX, the period is 2,236,962 hours.
Obviously more is better, whether you want to argue "overkill" or not;
PT_PAX_FLAGS has some room, so perhaps a flag to lighten ASLR could
allow {Heavy,Light} along with the basic {On,Off}, and the higher level
of ASLR could be used in many ca

LSM hooks

2005-03-30 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Well the LSM mailing list seems to be dead, even the archives stop at
Jan 15 2005.  My own mails don't come back to me (I'm subscribed).


So, Which version of Linux will first implement stacking in LSM as per
Serge Hallyn's patches?

Where is the new documentation?


As for hooks, a few questions.

1.  With shm_shmctl(), can I control permissions assigned to shared
memory using shmctl()?  I want to prevent memory protections on the shm
segment from being in certain combinations; however, if any changes
AFTER creation go through mprotect(), I can use file_mprotect() because
I will be preventing the same transitions everywhere.

Shared memory mappings seem to always be in VM_WRITE | VM_MAYWRITE,
so they're sane by default at creation time.  Control over mprotect()
has this covered beyond that.


2.  Is shared memory always attached to without PROT_EXEC?  If not, how
would I control this?  What is the best hook?


3.  I want control over the memory protections on the stack and heap.
PT_GNU_STACK allows for an executable stack/heap.  Is there a way for me
to control this so that I can i.e. mandatorily make the stack/heap
PROT_READ|PROT_WRITE and never PROT_EXEC?  The only way I can see is to
add a hook in load_elf_binary(). . . .

In other words, I want a module to be able to force the heap and stack
to be !EXEC.

 do_brk() only has a check, but I wish to elegantly control what may
happen here.  I would need a hook that would allow me to supply
something to AND flags with after the following line:

flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags;

So for example:

flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags;
/* stacking should AND together each value and return the
 * result
 */
flags &= security_vm_brk();

I would of course have my module returning (~VM_MAYEXEC) as in PaX.

4.  file_mmap() is only a check; however, to be very unintrusive, I want
to be able to alter vm_flags in do_mmap_pgoff().  Particularly, I want
to be able to supply a mask to AND them with.  The current code looks like:

error = security_file_mmap(file, prot, flags);
if (error)
return error;

To accomplish this task, one of two venues would be taken.  The first,
shown below, adds a new hook in the same place:

error = security_file_mmap(file, prot, flags);
if (error)
return error;
/* Serge's stacking code should AND together each thing we get
 * back from each module to produce the most restrictive set
 */
vm_flags &= security_file_mmap_vm_flags(file, prot, flags);

The second alters the current hook, requiring all LSMs using the
file_mmap() hook to be rewritten:

{
int my_vm_flags = ~0;
error = security_file_mmap(file, prot, flags,
  _vm_flags);
if (error)
return error;
/* Serge's stacking code should AND together each
 *  my_vm_flags
 */
vm_flags &= my_vm_flags;
}

Having one hook is most elegant; breaking an API is least elegant.
Which would be more likely to be genuinely accepted as an LSM hook
modification?  I'm thinking adding a second hook, as it won't break
SeLinux and friends. . . .


5.  It looks as if I can modify the vma to handle relocations the same
way PaX does from security_file_mprotect().  That's fine.  Now all I
have to do is figure out htf pip's logic works, mainly where the data
that is set for certain checks comes from.  Relocations aren't magically
MAY_WRITE, even if they come like that by default.


It seems to me that in total, I must create at least two hooks:

1.  security_file_mmap_vm_flags(file, prot, flags)
mm/mmap.c:do_mmap_pgoff()
This will allow control over how mmap() segments are created,
with unintrusive modifications to their vma permissions.  The
modifications I intend to impliment are quite well tested.

This hook will return a bitmask to be ANDed with vm_flags.  The
stacking implementation can simply AND every result from every
module together to get the least restrictive set.

2.  security_vm_brk();
mm/mmap.c:do_brk()
This will allow the permissions on the heap to be unintrusively
controlled.

This hook will return a bitmask to be ANDed with flags.  The
stacking implimentation can AND every result from every module
together to get the least restrictive set.

There may be a third needed to control the stack's permissions; but I
don't know yet, as I can't find how that's even done.  For now, the
kernel is oopsing when i try to mount another xfs partition (from a 64
bit system), so I need to reboot.  :/


In case anyone is wondering, as an excercise (but potentially as
something I may aim at mainline), I'm trying to port 

LSM hooks

2005-03-30 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Well the LSM mailing list seems to be dead, even the archives stop at
Jan 15 2005.  My own mails don't come back to me (I'm subscribed).


So, Which version of Linux will first implement stacking in LSM as per
Serge Hallyn's patches?

Where is the new documentation?


As for hooks, a few questions.

1.  With shm_shmctl(), can I control permissions assigned to shared
memory using shmctl()?  I want to prevent memory protections on the shm
segment from being in certain combinations; however, if any changes
AFTER creation go through mprotect(), I can use file_mprotect() because
I will be preventing the same transitions everywhere.

Shared memory mappings seem to always be in VM_WRITE | VM_MAYWRITE,
so they're sane by default at creation time.  Control over mprotect()
has this covered beyond that.


2.  Is shared memory always attached to without PROT_EXEC?  If not, how
would I control this?  What is the best hook?


3.  I want control over the memory protections on the stack and heap.
PT_GNU_STACK allows for an executable stack/heap.  Is there a way for me
to control this so that I can i.e. mandatorily make the stack/heap
PROT_READ|PROT_WRITE and never PROT_EXEC?  The only way I can see is to
add a hook in load_elf_binary(). . . .

In other words, I want a module to be able to force the heap and stack
to be !EXEC.

 do_brk() only has a check, but I wish to elegantly control what may
happen here.  I would need a hook that would allow me to supply
something to AND flags with after the following line:

flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm-def_flags;

So for example:

flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm-def_flags;
/* stacking should AND together each value and return the
 * result
 */
flags = security_vm_brk();

I would of course have my module returning (~VM_MAYEXEC) as in PaX.

4.  file_mmap() is only a check; however, to be very unintrusive, I want
to be able to alter vm_flags in do_mmap_pgoff().  Particularly, I want
to be able to supply a mask to AND them with.  The current code looks like:

error = security_file_mmap(file, prot, flags);
if (error)
return error;

To accomplish this task, one of two venues would be taken.  The first,
shown below, adds a new hook in the same place:

error = security_file_mmap(file, prot, flags);
if (error)
return error;
/* Serge's stacking code should AND together each thing we get
 * back from each module to produce the most restrictive set
 */
vm_flags = security_file_mmap_vm_flags(file, prot, flags);

The second alters the current hook, requiring all LSMs using the
file_mmap() hook to be rewritten:

{
int my_vm_flags = ~0;
error = security_file_mmap(file, prot, flags,
  my_vm_flags);
if (error)
return error;
/* Serge's stacking code should AND together each
 *  my_vm_flags
 */
vm_flags = my_vm_flags;
}

Having one hook is most elegant; breaking an API is least elegant.
Which would be more likely to be genuinely accepted as an LSM hook
modification?  I'm thinking adding a second hook, as it won't break
SeLinux and friends. . . .


5.  It looks as if I can modify the vma to handle relocations the same
way PaX does from security_file_mprotect().  That's fine.  Now all I
have to do is figure out htf pip's logic works, mainly where the data
that is set for certain checks comes from.  Relocations aren't magically
MAY_WRITE, even if they come like that by default.


It seems to me that in total, I must create at least two hooks:

1.  security_file_mmap_vm_flags(file, prot, flags)
mm/mmap.c:do_mmap_pgoff()
This will allow control over how mmap() segments are created,
with unintrusive modifications to their vma permissions.  The
modifications I intend to impliment are quite well tested.

This hook will return a bitmask to be ANDed with vm_flags.  The
stacking implementation can simply AND every result from every
module together to get the least restrictive set.

2.  security_vm_brk();
mm/mmap.c:do_brk()
This will allow the permissions on the heap to be unintrusively
controlled.

This hook will return a bitmask to be ANDed with flags.  The
stacking implimentation can AND every result from every module
together to get the least restrictive set.

There may be a third needed to control the stack's permissions; but I
don't know yet, as I can't find how that's even done.  For now, the
kernel is oopsing when i try to mount another xfs partition (from a 64
bit system), so I need to reboot.  :/


In case anyone is wondering, as an excercise (but potentially as
something I may aim at mainline), I'm trying to port 

Re: LSM hooks

2005-03-30 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
 * John Richard Moser ([EMAIL PROTECTED]) wrote:
 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Well the LSM mailing list seems to be dead, even the archives stop at
Jan 15 2005.  My own mails don't come back to me (I'm subscribed).
 
 
 They're coming through just fine, not sure why the archives are stale
 though (I'll take a look).
 
 
So, Which version of Linux will first implement stacking in LSM as per
Serge Hallyn's patches?
 
 
 None are ready yet.  Serge is still wading through performance testing.
 There's no telling about merging without a magic eightball, a handle on
 the performance issues, and some bonafide users.
 
 
Where is the new documentation?
 
 
 It's in the archives ;-)  There's not much to document.  Each hook would
 potentially call a chain of modules rather than just one.
 
 
As for hooks, a few questions.

1.  With shm_shmctl(), can I control permissions assigned to shared
memory using shmctl()?  I want to prevent memory protections on the shm
segment from being in certain combinations; however, if any changes
AFTER creation go through mprotect(), I can use file_mprotect() because
I will be preventing the same transitions everywhere.

Shared memory mappings seem to always be in VM_WRITE | VM_MAYWRITE,
so they're sane by default at creation time.  Control over mprotect()
has this covered beyond that.
 
 
 And VM_MAYREAD and VM_MAYEXEC typically as well (mmap does that
 naturally), so mprotect is rather open for abuse here.
 

Dang, I want read/write but never execute for shm.

 
2.  Is shared memory always attached to without PROT_EXEC?  If not, how
would I control this?  What is the best hook?
 
 
 No, shamt(SHM_EXEC), for example, will attach to the segment with
 PROT_EXEC.
 

Again, I need to control that. . . .

 
3.  I want control over the memory protections on the stack and heap.
PT_GNU_STACK allows for an executable stack/heap.  Is there a way for me
to control this so that I can i.e. mandatorily make the stack/heap
PROT_READ|PROT_WRITE and never PROT_EXEC?  The only way I can see is to
add a hook in load_elf_binary(). . . .

In other words, I want a module to be able to force the heap and stack
to be !EXEC.
 
 
 Thus far we intentionally left that type of work out, rather left it as
 something for access control. (you can easily stop from creating or
 changing such vmas).
 
 
 do_brk() only has a check, but I wish to elegantly control what may
happen here.  I would need a hook that would allow me to supply
something to AND flags with after the following line:

flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm-def_flags;

So for example:

flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm-def_flags;
  /* stacking should AND together each value and return the
   * result
   */
  flags = security_vm_brk();

I would of course have my module returning (~VM_MAYEXEC) as in PaX.
 
 
 I'm not convinced it makes sense as lsm work.
 

Well, it's in PaX; I could just split it out from PaX directly, but I'm
feeling creative.  Besides, it seems dismally unlikely that porting in
chunks of PaX just here set this in mainline and here's a config option
to disable it is going to work; I'd suspect an LSM has a much higher
chance of getting in for some reason.

Pretty much, I'm fairly sure that with PT_PAX_FLAGS processing, parts of
PaX could survive just fine in pieces.  Particularly, mprotect()
restrictions and the high-entropy ASLR (the kernel will have ES'
low-entropy ASLR in 2.6.12, basically a performance boost due to some
issues with cache hits/misses that I don't care to pretend to fully
understand).  I'd also consider trampoline emulation to replace the
executable stack in some apps, on a per-binary basis of course.

In the case of ASLR, I'd also like to port over Brad Spengler's brute
force deterrance from GrSecurity.  The deterrance would extend the brute
force period of the 64K stack ASLR to 34.13 hours, ignoring the
potential for stack stuffing.  The 2M randomization would hold up for
1092 hours under a brute force.

With the 256M randomization from PaX, the period is 2,236,962 hours.
Obviously more is better, whether you want to argue overkill or not;
PT_PAX_FLAGS has some room, so perhaps a flag to lighten ASLR could
allow {Heavy,Light} along with the basic {On,Off}, and the higher level
of ASLR could be used in many cases (this is also well tested).

In either case, I need some way of controlling the stuff I want to split
out from PaX.  I have the code that reads PT_PAX_FLAGS; but I have
serious doubts that that control mechanism would actually get in, even
though in my experience it works rather well.

At the very least, Arjan doesn't seem to hot on the following logic:

executable_stack = EXSTACK_DISABLE_X;
if (binary_has(PT_PAX_FLAGS)) {
  if (!(PT_PAX_FLAGS  FL_PAX_PAGEEXEC))
executable_stack = EXSTACK_ENABLE_X;
}
else {
  if (PT_GNU_STACK)
executable_stack

Re: Aligning file system data

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Well then, the verdict is reached.

My original design is based around storing related data in the same
block so that the track cache allows me to evade doing reads while I
poke around.

The design will stay the same; but the dependency on the track cache
will dissappear.  I'll simply consider 32KiB or 64KiB to be a nice block
size, 64KiB being the biggest, and leverage the design on the kernel
reading whole blocks into main memory to play with at a time.

Back to designing my file system. . . .


The only lasting regrets I have is that I don't have a good, fast way to
do on-disk locking for a cluster file system.  This would make my FS a
complete solution. . . .

It doesn't matter, finishing the design is a while off anyway.  I still
have to define several extended journal transaction types to support
fault tolerant dynamic resizing (grow, shrink) while running.  I don't
see how to grow left; shrinking from the left is easy enough.  Wait,
suddenly I see how to grow left:  Superblock at the end, and a bit of
magic. . . .


Robert Hancock wrote:
> John Richard Moser wrote:
> 
>> How likely is it that I can actually align stuff to 31.5KiB on the
>> physical disk, i.e. have each block be a track?
> 
> 
> I don't think this is very likely. Even being able to find out what the
> physical disk arrangement is, or whether it is consistent in terms of
> track size, etc. seems unlikely.
> 
>>
>> Rather than leveraging the track cache, would it be less expensive for
>> me to simply read in blocks totaling about 16 or 32KiB all at once?
> 
> 
> For block sizes that small I think that the kernel should be smart
> enough to do this itself, there is no need to concern with such low
> level details in the application.
> 
>> How much more latency is involved in (B) than in (C)?  Does crossing a
>> track boundary incur anything expensive?
> 
> 
> Given that both the disk and the kernel will likely read far more than
> 32KB ahead I can't see much difference other than the overhead inside
> your application..
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD4DBQFCSjmPhDd4aOud5P8RAgB7AJiWq4Qiyfk1G0SJa+5ZCtJ//WH8AJ9ysogo
3z6+FLvkNgyU/k0o9HBf1w==
=OPXo
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Aligning file system data

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

How likely is it that I can actually align stuff to 31.5KiB on the
physical disk, i.e. have each block be a track?

Rather than leveraging the track cache, would it be less expensive for
me to simply read in blocks totaling about 16 or 32KiB all at once?


Let's say I have two situations...

A)
  My blocks are all 31.5KiB (512 bytes/sector * 63 sectors) and aligned
to tracks.  The track cache on the disk stores the entire block, so
repeted reads to the disk are 0mS seek.  I leverage this to read a
couple sectors at a time and seek as I care within the block while it's
cached, making several requests to the ATA device.

B)
  My blocks are all 32KiB and cross track boundaries.  All of them exist
in part in two separate tracks.  Upon reading a block, I request the
entire block and work with it in main memory.

Which situation has less overhead?

C)
  My blocks are all 31.5KiB and perfectly aligned within tracks.  I read
the entire block as in (B) and work with it in main memory.

How much more latency is involved in (B) than in (C)?  Does crossing a
track boundary incur anything expensive?


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSivPhDd4aOud5P8RAszeAJ4wPonhpXas8IprMBUq8/NdM57aegCdEBva
24LXB3O+7GEE0XKxPBFr1L0=
=iTEm
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
> On Tue, 2005-03-29 at 14:07 -0500, John Richard Moser wrote:
> 
>>-BEGIN PGP SIGNED MESSAGE-

[...]

>>/me shrugs.  It's a security blanket for him mostly; he fears automagic
>>security maintainence.
> 
> 
> who is "him" ?
> 

me in third person?  :)

> 
>>>>Remember also I'm very much against "let the compiler guess if you need
>>>>an executable stack"
>>>
>>>
>>>it's not guessing. the *compiler* emits the stack trampoline. So the
>>>*compiler* knows that it needs that stack.
>>>
>>
>>With a trampoline, things like Grub and a few libs need PT_GNU_STACK.
> 
> 
> sure they do. There's about a handful in an entire distro.

Right, so it's a toss-up:  Do you want to fix these few things, or do
you want to let trampolines exist?  Are trampolines that useful?

> 
> 
>>Of course you can't just suddenly say "OH!  Well PT_GNU_STACK should do
>>this instead!" because you'll break everything.
> 
> 
> I'm not a fan of any kind of emutrampoline. At all. But I am open to
> others making a different tradeoff; for me the option to have a
> trampoline at all is just a bypass of the non-exec stack... legit bypass
> one hopes but a bypass regardless. Some time ago we did an eval of how
> much stuff would need the emutramp (well or something equivalent) and
> the list was so small that I decided that the added risk and complexity
> were not worth it and that I rather had those 5 or so apps run with exec
> stack.
> 

Eh.

> 
>>>again what does tristate mean.. "I don't know" ? But gcc does know, with
>>>very very very few exceptions. Eg old mono is the exception because it
>>>didn't do a proper mprotect. Saying "I don't know" doesn't solve you
>>>anything, since in the end there needs to be a policy enforced anyway,
>>>it's just postponing the inevitable to a point with less knowledge.
>>>
>>
>>Remember I'm also thinking of restricted mprotect() situations as well,
>>because I'm trying to get it to the point where one set of markings has
>>a predictable effect on any kernel, be it vanilla, pax, or ES.
> 
> 
> well that is an entirely independent thing again. Really.
> I think mixing all these into one big flag is a mistake. 
> (And thats a lesson I learned the hard way, but Linus was right; don't
> mix independent things into one flag artificially. Extra flags are
> cheap. Don't complicate the world for a dozen bytes.)
> 
> 

two or four dozen bytes, eight dozen bytes in 10 years when 128 bit
systems come along, and 16i dozen planck qubytes when we get quantum
computers :)

Actually when I proposed adding PT_PAX_FLAGS to Ubuntu, the very first
argument I got was "Oh, yeah right, add just a few bytes here for this.
 Then later it'll be a few more bytes for something else.  Then a few
more for another thing.  It all adds up, especially when you have
thousands of binaries."

And if bitfield logic is "complicated," you should stop pretending to be
a programmer.


#define BIG   (1)
#define LONG  (1 << 1)
#define FAT   (1 << 2)
#define TALL  (1 << 3)
#define GREEN (1 << 4)

struct foo {
  char *myname;
  unsigned long flags;
};

struct foo *newfoo() {
  struct foo *out = malloc(sizeof(struct foo));
  out->myname = malloc(4);
  strcpy(out->myname, "bob");
  out->flags = BIG | TALL | GREEN;
  return out;
}


Easy enough?


struct foo {
  char myname[10];
  int big;
  int long;
  int fat;
  int tall;
  int green;
};

struct foo *newfoo() {
  struct foo *out = malloc(sizeof(struct foo));
  strcpy(out->myname, "bob");
  out->big = 1;
  out->tall = 1;
  out->green = 1;
  return out;
}


I don't find the above to be quite as elegant.  In fact, it looks ugly
and wasteful; even checking ...

if (myfoo->flags & BIG)

versus

if (myfoo->big)

> 
> 
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSb1rhDd4aOud5P8RAg2cAJ98SlxFCLcHvN+aWcVTM2VWxiRCEACfUPPl
24wpdtY/VyKHGs/YpPDo8Hk=
=mVc5
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



John Richard Moser wrote:
> 
> 
> Arjan van de Ven wrote:
> 
[...]

Three more notes, then I'll sleep.  These notes won't include the two
paragraph long explaination of falling back to PT_GNU_STACK if
PT_PAX_FLAGS isn't there; compatibility has been touched what, 5 times?

1.  I don't want to continue using PT_GNU_STACK for three reasons.  The
first being that PaX uses a tristate in PT_PAX_FLAGS; the second being
that PT_GNU_STACK is a whole ELF field and I'm inclined to take the more
space-efficient method; and the third being that PT_GNU_STACK is not a
tristate.

The last is particularly an important consideration to me:  a tristate
would allow for a compatibility/soft mode, but changing PT_GNU_STACK's
logic would change the current expected behavior and thus could be
unpredictable (break things).  I have no interest in breaking Fedora
horribly, nor wasting space with a full field where sharing with the
other parts of PT_PAX_FLAGS would do just fine.

2.  Although binutils can emit PT_GNU_STACK, the paxctl utility could
also be modified to detect PT_GNU_STACK in a binary without PT_PAX_FLAGS
and change it to PT_PAX_FLAGS, then nuke it.  This would allow the flags
to be changed without relinking (remember PT_GNU_STACK is to be ignored
if PT_PAX_FLAGS exists at all).  This is only of interest to
distributions which will use PT_PAX_FLAGS.

Note also that execstack would probably be wisely modified to set
PF_PAGEEXEC and PT_GNU_STACK both, just for future compatibility.  This
is of course a lot of work (I tried to make paxctl hack EI_PAX too, and
. . .well, it didn't work).

3.  PaX won't pay any attention to markings on libraries.  Exec Shield
and Mainline may, though I have no idea how.  If it can be done with
PT_GNU_STACK, it can be done with PT_PAX_FLAGS.  Such behavior is
acceptable, though libraries should be coded with the utmost care to
avoid this simply because the weakening of security around a library
weakens any and all programs using that library.


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSRWghDd4aOud5P8RAhRFAJ9Ezr6mMIEvk9R+4XpXq7+lZxgd0gCfYhBa
IuUU7Zeuk1J9kSJXCSqZlKU=
=m0YW
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
>>You need to consider that in the end I'd need PT_GNU_STACK to do
>>everything PaX wants
> 
> 
> why?
> Why not have independent flags for independent things?
> That way you have both cleanness of design and you don't break anything.
> 

Also, I should have pointed out that independent flags for each part of
this would require at the very least a 1 byte field per flag, totaling
6.  In practice, the fields are usually 1 processor word (4 or 8 bytes),
totalling 24 (or 48) bytes rather than 4 (or 8).

As these are all pretty much "control memory space related security
protections," convergence to a well-defined standard would be both clean
and non-stuff-breaking.  Now of course there's no standard, but several
things operating very similarly.  This gives us the opportunity to look
at how each reacts to each different proprietary marking, take the most
robust (which just happens to be PT_PAX_FLAGS, no flamewars please), and
define exactly what each setting controls.

I want something very well defined for PAGEEXEC (executable stack,
heap).  The MPROTECT setting should also be very well defined, which
will match with PaX because nobody else restricts mprotect().  EMUTRAMP
should be defined fairly loosely, but enough to say "stuff we can
predictably identify as exceptions to the rules are caught here."  All
of these alter the programming environment, so must be predictable to
the programmer; emutramp is a special case, which allows the programmer
to assume that he needs PAGEEXEC off while the administrator knows to
just EMUTRAMP in this case.

For the two randomizers, "sane for default" should define one and "not
sane for default" should define the other.  These have little to no
effect on most programs, VM space fragmentation aside.  We don't need to
define these too well, but enough to know what they're for.

SEGMEXEC is of course "nothing."  In truth, I want PaX to change to make
SEGMEXEC absolutely nothing unless PAGEEXEC is on, and only for x86.
This is because PAGEEXEC can be very clearly defined, and SEGMEXEC can
be defined as a specific modifier on PAGEEXEC.

Of course, the effect of any one of these being on is subject to
implementation; obviously if mprotect() restrictions aren't supported,
MPROTECT being on does nothing.  I doubt mainline and ES will take that
particular logic specifically, though either way I have no intention of
even trying to force them to.  EMUTRAMP, however, I hope would be able
to enhance mainline and ES both (that means Red Hat/Fedora) by allowing
some of the PT_GNU_STACK markings to come off.

In the future, an SeLinux hook should go into the pax_parse_elf_flags()
logic to allow SeLinux policy to control these settings for PaX,
mainline, and ES in one sweep:

+   /*Are these broke?  Then get out*/
+   if (0 > pax_check_flags(_flags))
+   return -EINVAL;
+
(insert hook here)
+   /*Add to the memory manager flags*/
+   current->mm->flags |= pax_flags;


[...]

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSQ6chDd4aOud5P8RAvr3AJ91i8c7W1CetmjWFGuItG6pCHEiigCbBfXb
H4RCVuOjFI71R45Q+TUw/AY=
=dLsN
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
>>You need to consider that in the end I'd need PT_GNU_STACK to do
>>everything PaX wants
> 
> 
> why?
> Why not have independent flags for independent things?
> That way you have both cleanness of design and you don't break anything.
> 

In the end I want to fine-tune for what the best behaviors are, and then
define exactly what the flags should do.

Right now, my rough sketch is:

MF_PAX_PAGEEXEC
  ON:   ET_EXEC enforced.  Stack NX.  Heap NX.  Code PROT_EXEC.
  OFF:  Stack and heap default to +X
  The PAGEEXEC flag will basically mandate the automated non-executable
  setting for the stack and heap.  When off, these areas are executable
  (like when PT_GNU_STACK is on)
MF_PAX_EMUTRAMP:
  ON:  Trampolines (in NX stack) and PLT will be detected and emulated
  OFF: Stack needs to be +X for trampolines to work
  The EMUTRAMP flag will basically allow certain known exceptional
  conditions to be trapped and allowed somehow automatically.  This
  includes mainly nested functions on a non-executable stack, and parisc
  procedural linkage tables (binutils patch can fix these apparently).
  This is off by default in hardmode, but on by default in soft or
  compatibility mode if PAGEEXEC is manually on.
MF_PAX_RANDMMAP:
  ON:  stack, heap, mmap() base randomized
  OFF: Nothing is randomized in memory
  RANDMMAP should probably be called "RANDADDR" instead.  When set, the
  kernel randomizes anything that can be randomized in the address
  space (support determining).
MF_PAX_RANDEXEC:
  ON:  Fixed-position things are also randomized
  OFF: Fixed-position things are at fixed positions
  RANDEXEC allows things that normally can't be placed randomly to be
  placed randomly if hacks exist to do it.  Any hacks 100% safe that
  don't cause excess overhead are for RANDMMAP; any that may cause
  instability or excessive overhead go under RANDEXEC.  OFF BY DEFAULT
  in any mode.
MF_PAX_MPROTECT:
  ON:  Enforce certain mprotect() restrictions
  OFF: Normal mprotect() restrictions
  A certain defined set of transitions and creation states are banned
  when this is on.  PaX has these now, nobody else has them and
  apparently nobody else wants them.
MF_PAX_SEGMEXEC:
  Absolutely no effect, reserved for PaX or anything else that
  implements PaX' specific SEGMEXEC emulation method.

Obviously mainline and ES won't do anything with RANDEXEC, MPROTECT, or
SEGMEXEC.

RANDMMAP will be useful to control ES and mainline ASLR (on-off switch).

PAGEEXEC being OFF will act exactly like PT_GNU_STACK being ON for
mainline and ES.

EMUTRAMP. . . I think I've got a patch for trampoline emulation, which
should let red hat use Exec Shield with fewer PT_GNU_STACK markings.
Mainline should benefit from this too.  It feels like porting this was
way too easy, so I'll ask pipacs to give it a quick look.

I also don't have a soft/hard mode for mainline.  I expect that mainline
will be more into making softmode (compatibility mode) into a
personality, which makes sense for softmode shells (which I've needed
before, i.e. to compile mono without softmoding my whole system).

Soft/Hard mode controls what flags are set by default.  Each flag has
ON/OFF/NEUTRAL settings (thus each is 2 bits).

HARDMODE:
 MF_PAX_PAGEEXEC, MF_PAX_RANDMMAP, MF_PAX_MPROTECT
SOFTMODE:
 (MF_PAX_EMUTRAMP if MF_PAX_PAGEEXEC is ON)
> 
>>The point is
>>to not break anything, yet to still make things easier for those
>>projects and distributions like Hardened Ubuntu.
> 
> 
> to achieve that you need to get the toolchain to omit this stuff
> automatically somehow. 
> 

Emit.

And there's a patch for binutils that Gentoo uses.  Ubuntu can use it too.

Remember that the way I'm doing it, PT_GNU_STACK is used if there is no
PT_PAX_FLAGS header.  Distributions not using PT_PAX_FLAGS will not care
about the kernel's ability to read PT_PAX_FLAGS, because it will still
behave the same with PT_GNU_STACK.  In other words, if Red Hat updated
the kernel with this stuff today (assuming it's non-buggy), it won't do
shit, and Fedora will still work exactly as expected.

> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSQlqhDd4aOud5P8RAgCIAJ4hGeiHD/wcB+B6u9up4v37CT0bAwCeKgcA
zX00s0dqVkkRhnxmmzQLZq0=
=4EMG
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
You need to consider that in the end I'd need PT_GNU_STACK to do
everything PaX wants
 
 
 why?
 Why not have independent flags for independent things?
 That way you have both cleanness of design and you don't break anything.
 

In the end I want to fine-tune for what the best behaviors are, and then
define exactly what the flags should do.

Right now, my rough sketch is:

MF_PAX_PAGEEXEC
  ON:   ET_EXEC enforced.  Stack NX.  Heap NX.  Code PROT_EXEC.
  OFF:  Stack and heap default to +X
  The PAGEEXEC flag will basically mandate the automated non-executable
  setting for the stack and heap.  When off, these areas are executable
  (like when PT_GNU_STACK is on)
MF_PAX_EMUTRAMP:
  ON:  Trampolines (in NX stack) and PLT will be detected and emulated
  OFF: Stack needs to be +X for trampolines to work
  The EMUTRAMP flag will basically allow certain known exceptional
  conditions to be trapped and allowed somehow automatically.  This
  includes mainly nested functions on a non-executable stack, and parisc
  procedural linkage tables (binutils patch can fix these apparently).
  This is off by default in hardmode, but on by default in soft or
  compatibility mode if PAGEEXEC is manually on.
MF_PAX_RANDMMAP:
  ON:  stack, heap, mmap() base randomized
  OFF: Nothing is randomized in memory
  RANDMMAP should probably be called RANDADDR instead.  When set, the
  kernel randomizes anything that can be randomized in the address
  space (support determining).
MF_PAX_RANDEXEC:
  ON:  Fixed-position things are also randomized
  OFF: Fixed-position things are at fixed positions
  RANDEXEC allows things that normally can't be placed randomly to be
  placed randomly if hacks exist to do it.  Any hacks 100% safe that
  don't cause excess overhead are for RANDMMAP; any that may cause
  instability or excessive overhead go under RANDEXEC.  OFF BY DEFAULT
  in any mode.
MF_PAX_MPROTECT:
  ON:  Enforce certain mprotect() restrictions
  OFF: Normal mprotect() restrictions
  A certain defined set of transitions and creation states are banned
  when this is on.  PaX has these now, nobody else has them and
  apparently nobody else wants them.
MF_PAX_SEGMEXEC:
  Absolutely no effect, reserved for PaX or anything else that
  implements PaX' specific SEGMEXEC emulation method.

Obviously mainline and ES won't do anything with RANDEXEC, MPROTECT, or
SEGMEXEC.

RANDMMAP will be useful to control ES and mainline ASLR (on-off switch).

PAGEEXEC being OFF will act exactly like PT_GNU_STACK being ON for
mainline and ES.

EMUTRAMP. . . I think I've got a patch for trampoline emulation, which
should let red hat use Exec Shield with fewer PT_GNU_STACK markings.
Mainline should benefit from this too.  It feels like porting this was
way too easy, so I'll ask pipacs to give it a quick look.

I also don't have a soft/hard mode for mainline.  I expect that mainline
will be more into making softmode (compatibility mode) into a
personality, which makes sense for softmode shells (which I've needed
before, i.e. to compile mono without softmoding my whole system).

Soft/Hard mode controls what flags are set by default.  Each flag has
ON/OFF/NEUTRAL settings (thus each is 2 bits).

HARDMODE:
 MF_PAX_PAGEEXEC, MF_PAX_RANDMMAP, MF_PAX_MPROTECT
SOFTMODE:
 (MF_PAX_EMUTRAMP if MF_PAX_PAGEEXEC is ON)
 
The point is
to not break anything, yet to still make things easier for those
projects and distributions like Hardened Ubuntu.
 
 
 to achieve that you need to get the toolchain to omit this stuff
 automatically somehow. 
 

Emit.

And there's a patch for binutils that Gentoo uses.  Ubuntu can use it too.

Remember that the way I'm doing it, PT_GNU_STACK is used if there is no
PT_PAX_FLAGS header.  Distributions not using PT_PAX_FLAGS will not care
about the kernel's ability to read PT_PAX_FLAGS, because it will still
behave the same with PT_GNU_STACK.  In other words, if Red Hat updated
the kernel with this stuff today (assuming it's non-buggy), it won't do
shit, and Fedora will still work exactly as expected.

 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSQlqhDd4aOud5P8RAgCIAJ4hGeiHD/wcB+B6u9up4v37CT0bAwCeKgcA
zX00s0dqVkkRhnxmmzQLZq0=
=4EMG
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
You need to consider that in the end I'd need PT_GNU_STACK to do
everything PaX wants
 
 
 why?
 Why not have independent flags for independent things?
 That way you have both cleanness of design and you don't break anything.
 

Also, I should have pointed out that independent flags for each part of
this would require at the very least a 1 byte field per flag, totaling
6.  In practice, the fields are usually 1 processor word (4 or 8 bytes),
totalling 24 (or 48) bytes rather than 4 (or 8).

As these are all pretty much control memory space related security
protections, convergence to a well-defined standard would be both clean
and non-stuff-breaking.  Now of course there's no standard, but several
things operating very similarly.  This gives us the opportunity to look
at how each reacts to each different proprietary marking, take the most
robust (which just happens to be PT_PAX_FLAGS, no flamewars please), and
define exactly what each setting controls.

I want something very well defined for PAGEEXEC (executable stack,
heap).  The MPROTECT setting should also be very well defined, which
will match with PaX because nobody else restricts mprotect().  EMUTRAMP
should be defined fairly loosely, but enough to say stuff we can
predictably identify as exceptions to the rules are caught here.  All
of these alter the programming environment, so must be predictable to
the programmer; emutramp is a special case, which allows the programmer
to assume that he needs PAGEEXEC off while the administrator knows to
just EMUTRAMP in this case.

For the two randomizers, sane for default should define one and not
sane for default should define the other.  These have little to no
effect on most programs, VM space fragmentation aside.  We don't need to
define these too well, but enough to know what they're for.

SEGMEXEC is of course nothing.  In truth, I want PaX to change to make
SEGMEXEC absolutely nothing unless PAGEEXEC is on, and only for x86.
This is because PAGEEXEC can be very clearly defined, and SEGMEXEC can
be defined as a specific modifier on PAGEEXEC.

Of course, the effect of any one of these being on is subject to
implementation; obviously if mprotect() restrictions aren't supported,
MPROTECT being on does nothing.  I doubt mainline and ES will take that
particular logic specifically, though either way I have no intention of
even trying to force them to.  EMUTRAMP, however, I hope would be able
to enhance mainline and ES both (that means Red Hat/Fedora) by allowing
some of the PT_GNU_STACK markings to come off.

In the future, an SeLinux hook should go into the pax_parse_elf_flags()
logic to allow SeLinux policy to control these settings for PaX,
mainline, and ES in one sweep:

+   /*Are these broke?  Then get out*/
+   if (0  pax_check_flags(pax_flags))
+   return -EINVAL;
+
(insert hook here)
+   /*Add to the memory manager flags*/
+   current-mm-flags |= pax_flags;


[...]

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSQ6chDd4aOud5P8RAvr3AJ91i8c7W1CetmjWFGuItG6pCHEiigCbBfXb
H4RCVuOjFI71R45Q+TUw/AY=
=dLsN
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



John Richard Moser wrote:
 
 
 Arjan van de Ven wrote:
 
[...]

Three more notes, then I'll sleep.  These notes won't include the two
paragraph long explaination of falling back to PT_GNU_STACK if
PT_PAX_FLAGS isn't there; compatibility has been touched what, 5 times?

1.  I don't want to continue using PT_GNU_STACK for three reasons.  The
first being that PaX uses a tristate in PT_PAX_FLAGS; the second being
that PT_GNU_STACK is a whole ELF field and I'm inclined to take the more
space-efficient method; and the third being that PT_GNU_STACK is not a
tristate.

The last is particularly an important consideration to me:  a tristate
would allow for a compatibility/soft mode, but changing PT_GNU_STACK's
logic would change the current expected behavior and thus could be
unpredictable (break things).  I have no interest in breaking Fedora
horribly, nor wasting space with a full field where sharing with the
other parts of PT_PAX_FLAGS would do just fine.

2.  Although binutils can emit PT_GNU_STACK, the paxctl utility could
also be modified to detect PT_GNU_STACK in a binary without PT_PAX_FLAGS
and change it to PT_PAX_FLAGS, then nuke it.  This would allow the flags
to be changed without relinking (remember PT_GNU_STACK is to be ignored
if PT_PAX_FLAGS exists at all).  This is only of interest to
distributions which will use PT_PAX_FLAGS.

Note also that execstack would probably be wisely modified to set
PF_PAGEEXEC and PT_GNU_STACK both, just for future compatibility.  This
is of course a lot of work (I tried to make paxctl hack EI_PAX too, and
. . .well, it didn't work).

3.  PaX won't pay any attention to markings on libraries.  Exec Shield
and Mainline may, though I have no idea how.  If it can be done with
PT_GNU_STACK, it can be done with PT_PAX_FLAGS.  Such behavior is
acceptable, though libraries should be coded with the utmost care to
avoid this simply because the weakening of security around a library
weakens any and all programs using that library.


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSRWghDd4aOud5P8RAhRFAJ9Ezr6mMIEvk9R+4XpXq7+lZxgd0gCfYhBa
IuUU7Zeuk1J9kSJXCSqZlKU=
=m0YW
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
 On Tue, 2005-03-29 at 14:07 -0500, John Richard Moser wrote:
 
-BEGIN PGP SIGNED MESSAGE-

[...]

/me shrugs.  It's a security blanket for him mostly; he fears automagic
security maintainence.
 
 
 who is him ?
 

me in third person?  :)

 
Remember also I'm very much against let the compiler guess if you need
an executable stack


it's not guessing. the *compiler* emits the stack trampoline. So the
*compiler* knows that it needs that stack.


With a trampoline, things like Grub and a few libs need PT_GNU_STACK.
 
 
 sure they do. There's about a handful in an entire distro.

Right, so it's a toss-up:  Do you want to fix these few things, or do
you want to let trampolines exist?  Are trampolines that useful?

 
 
Of course you can't just suddenly say OH!  Well PT_GNU_STACK should do
this instead! because you'll break everything.
 
 
 I'm not a fan of any kind of emutrampoline. At all. But I am open to
 others making a different tradeoff; for me the option to have a
 trampoline at all is just a bypass of the non-exec stack... legit bypass
 one hopes but a bypass regardless. Some time ago we did an eval of how
 much stuff would need the emutramp (well or something equivalent) and
 the list was so small that I decided that the added risk and complexity
 were not worth it and that I rather had those 5 or so apps run with exec
 stack.
 

Eh.

 
again what does tristate mean.. I don't know ? But gcc does know, with
very very very few exceptions. Eg old mono is the exception because it
didn't do a proper mprotect. Saying I don't know doesn't solve you
anything, since in the end there needs to be a policy enforced anyway,
it's just postponing the inevitable to a point with less knowledge.


Remember I'm also thinking of restricted mprotect() situations as well,
because I'm trying to get it to the point where one set of markings has
a predictable effect on any kernel, be it vanilla, pax, or ES.
 
 
 well that is an entirely independent thing again. Really.
 I think mixing all these into one big flag is a mistake. 
 (And thats a lesson I learned the hard way, but Linus was right; don't
 mix independent things into one flag artificially. Extra flags are
 cheap. Don't complicate the world for a dozen bytes.)
 
 

two or four dozen bytes, eight dozen bytes in 10 years when 128 bit
systems come along, and 16i dozen planck qubytes when we get quantum
computers :)

Actually when I proposed adding PT_PAX_FLAGS to Ubuntu, the very first
argument I got was Oh, yeah right, add just a few bytes here for this.
 Then later it'll be a few more bytes for something else.  Then a few
more for another thing.  It all adds up, especially when you have
thousands of binaries.

And if bitfield logic is complicated, you should stop pretending to be
a programmer.


#define BIG   (1)
#define LONG  (1  1)
#define FAT   (1  2)
#define TALL  (1  3)
#define GREEN (1  4)

struct foo {
  char *myname;
  unsigned long flags;
};

struct foo *newfoo() {
  struct foo *out = malloc(sizeof(struct foo));
  out-myname = malloc(4);
  strcpy(out-myname, bob);
  out-flags = BIG | TALL | GREEN;
  return out;
}


Easy enough?


struct foo {
  char myname[10];
  int big;
  int long;
  int fat;
  int tall;
  int green;
};

struct foo *newfoo() {
  struct foo *out = malloc(sizeof(struct foo));
  strcpy(out-myname, bob);
  out-big = 1;
  out-tall = 1;
  out-green = 1;
  return out;
}


I don't find the above to be quite as elegant.  In fact, it looks ugly
and wasteful; even checking ...

if (myfoo-flags  BIG)

versus

if (myfoo-big)

 
 
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSb1rhDd4aOud5P8RAg2cAJ98SlxFCLcHvN+aWcVTM2VWxiRCEACfUPPl
24wpdtY/VyKHGs/YpPDo8Hk=
=mVc5
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Aligning file system data

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

How likely is it that I can actually align stuff to 31.5KiB on the
physical disk, i.e. have each block be a track?

Rather than leveraging the track cache, would it be less expensive for
me to simply read in blocks totaling about 16 or 32KiB all at once?


Let's say I have two situations...

A)
  My blocks are all 31.5KiB (512 bytes/sector * 63 sectors) and aligned
to tracks.  The track cache on the disk stores the entire block, so
repeted reads to the disk are 0mS seek.  I leverage this to read a
couple sectors at a time and seek as I care within the block while it's
cached, making several requests to the ATA device.

B)
  My blocks are all 32KiB and cross track boundaries.  All of them exist
in part in two separate tracks.  Upon reading a block, I request the
entire block and work with it in main memory.

Which situation has less overhead?

C)
  My blocks are all 31.5KiB and perfectly aligned within tracks.  I read
the entire block as in (B) and work with it in main memory.

How much more latency is involved in (B) than in (C)?  Does crossing a
track boundary incur anything expensive?


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSivPhDd4aOud5P8RAszeAJ4wPonhpXas8IprMBUq8/NdM57aegCdEBva
24LXB3O+7GEE0XKxPBFr1L0=
=iTEm
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Aligning file system data

2005-03-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Well then, the verdict is reached.

My original design is based around storing related data in the same
block so that the track cache allows me to evade doing reads while I
poke around.

The design will stay the same; but the dependency on the track cache
will dissappear.  I'll simply consider 32KiB or 64KiB to be a nice block
size, 64KiB being the biggest, and leverage the design on the kernel
reading whole blocks into main memory to play with at a time.

Back to designing my file system. . . .


The only lasting regrets I have is that I don't have a good, fast way to
do on-disk locking for a cluster file system.  This would make my FS a
complete solution. . . .

It doesn't matter, finishing the design is a while off anyway.  I still
have to define several extended journal transaction types to support
fault tolerant dynamic resizing (grow, shrink) while running.  I don't
see how to grow left; shrinking from the left is easy enough.  Wait,
suddenly I see how to grow left:  Superblock at the end, and a bit of
magic. . . .


Robert Hancock wrote:
 John Richard Moser wrote:
 
 How likely is it that I can actually align stuff to 31.5KiB on the
 physical disk, i.e. have each block be a track?
 
 
 I don't think this is very likely. Even being able to find out what the
 physical disk arrangement is, or whether it is consistent in terms of
 track size, etc. seems unlikely.
 

 Rather than leveraging the track cache, would it be less expensive for
 me to simply read in blocks totaling about 16 or 32KiB all at once?
 
 
 For block sizes that small I think that the kernel should be smart
 enough to do this itself, there is no need to concern with such low
 level details in the application.
 
 How much more latency is involved in (B) than in (C)?  Does crossing a
 track boundary incur anything expensive?
 
 
 Given that both the disk and the kernel will likely read far more than
 32KB ahead I can't see much difference other than the overhead inside
 your application..
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD4DBQFCSjmPhDd4aOud5P8RAgB7AJiWq4Qiyfk1G0SJa+5ZCtJ//WH8AJ9ysogo
3z6+FLvkNgyU/k0o9HBf1w==
=OPXo
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Brandon Hale wrote:
>>>actually Linus was really against adding non-related things to this
>>>flag. And I think he is right...
>>>
> 
> 
> Makes sense to me.
> 
> 

[...]

> 
> IMO you have this backwards, John.  Rather than having the majority (ES,
> mainline NX stuff) respect your less popular branch, it would make sense
> to have PaX work as well as possible with PT_GNU_STACK, and politely
> request that any missing functionality be duplicated in ES. PT_GNU_STACK
> is the most widely available header, and we should leverage that
> ubiquity as much as possible.  Marking our binaries with PT_PAX_FLAGS
> and then begging everyone else to support our way of doing things will
> never work.
> 

You need to consider that in the end I'd need PT_GNU_STACK to do
everything PaX wants, and to make PF_X a tristate (On, Off, Neutral)
which mapped to PF_PAGEEXEC in PaX.  In other words, I'd have to
overhaul and most likely *break* everything else that relied on
PT_GNU_STACK, instead of passively adding logic for PT_PAX_FLAGS and
letting everyone else catch up at their leisure.

I'd rather not break anything and force everyone to upgrade *now*; but
instead add PT_PAX_FLAGS functionality for mainline/ES, let it lay there
for a year or so as people start moving over and accepting it, let the
kernel devs decide when it's time to depricate PT_GNU_STACK, and see it
naturally decay away once it's actually no longer needed.  The point is
to not break anything, yet to still make things easier for those
projects and distributions like Hardened Ubuntu.


> ---
> Brandon Hale

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSIKKhDd4aOud5P8RAkqEAJwNhFvfDc63qyPrBvMxs6naG1xuAQCfZKHn
upzqNI5/XzYVCmDKGM6q8ZY=
=YZkT
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
> On Mon, 2005-03-28 at 13:50 -0500, John Richard Moser wrote:
> 
>>-BEGIN PGP SIGNED MESSAGE-
>>Hash: SHA1
>>
>>
>>
>>Arjan van de Ven wrote:
>>
>>>>As I understand, PT_GNU_STACK uses a single marking to control whether a
>>>>task gets an executable stack and whether ASLR is applied to the
>>>>executable.
>>>
>>>
>>>you understand wrongly.
>>>
>>>PT_GNU_STACK just sets the exec permission for the stack (and the heap
>>>now mirrors the stack). Nothing more nothing less.
>>>
>>
>>So then this would be slightly more useful than I had previously
>>thought, bringing control over the randomization as well?
> 
> 
> actually Linus was really against adding non-related things to this
> flag. And I think he is right...
> 

I'm not interested in altering and hacking up PT_GNU_STACK; PT_PAX_FLAGS
already supplies enough to do what I want.  My goal is to have
PT_PAX_FLAGS code in mainline and Exec Shield, so that if it exists in
the binary it will be used; else PT_GNU_STACK will be fallen back to.

> Now.. do you have any examples of when you want a binary marked for no-
> randomisation ?? (eg something the setarch flag won't fix/won't be good
> enough for)

What's setarch do for one?  Anyway, ASLR has been known to break some
things.  Blackdown Java used to break IIRC; also there's the poorly
designed Oracle and the poorly designed solution of Oracle on a 32 bit
platform; and of course there's Emacs, which I heard was broken due to
Exec Shield's randomization.  Temporary work-arounds are sometimes needed.



Remember also that I'm not just trying to make a more robust setting for
ES and mainline; I'm trying to find a way to make it so that
distribution maintainers can set one set of flags and have it assure
that the program works in Mainline, Exec Shield, and PaX.  Just a little
less work for the distribution maintainers, which I think would be a
good thing considering that apparently Ubuntu Linux might support both
PaX and Exec Shield in the future, if I'm reading this[1] right.

[1] http://thread.gmane.org/gmane.linux.ubuntu.devel/6130


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSFeuhDd4aOud5P8RApQ+AKCPtp5b4/2rw+aRqEUg7r1FlphmQwCfX3Io
FUNq9xZlDsoo1poGBo5+zus=
=v0dv
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
>>As I understand, PT_GNU_STACK uses a single marking to control whether a
>>task gets an executable stack and whether ASLR is applied to the
>>executable.
> 
> 
> you understand wrongly.
> 
> PT_GNU_STACK just sets the exec permission for the stack (and the heap
> now mirrors the stack). Nothing more nothing less.
> 

So then this would be slightly more useful than I had previously
thought, bringing control over the randomization as well?

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSFINhDd4aOud5P8RAv36AJ9kFyE4u14CAVvWNu4bl11Gd125agCfVj3I
gNPQRd73mWJCLrPd5Ge/EnM=
=jqg0
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Greetings.

Currently I'm in need of some information about both vanilla and Exec
Shield kernels in regards to markings emitted by the toolchain,
specifically PT_GNU_STACK.  I'd like to check my assumptions, in
preparation for possibly making a non-intrusive change which would allow
the PT_PAX_FLAGS from PaX to be used in a more finegrained way than
PT_GNU_STACK is used now to control mainline and exec shield memory
protections.

As I understand, PT_GNU_STACK uses a single marking to control whether a
task gets an executable stack and whether ASLR is applied to the
executable.  I would like more information on this.  To start, I'll
supply some background on why I need this information.

PaX uses its own markings, PT_PAX_FLAGS, to control the way PaX applies
protections to processes.  In particular, PaX supplies the following
markings:

P  - PAGEEXEC (NX bit or emulated NX bit)
S  - SEGMEXEC (NX emulation on x86, ignored for our purposes)
E  - EMUTRAMP (Trampoline emulation, to avoid giving an executable stack
   in situations where nested functions are used)
M  - MPROTECT (mprotect() restrictions, to control kernel policy on how
   memory protections may be applied in userspace)
R  - RANDMMAP (mmap() base randomization, for libraries, PIEs, anonymous
   segments, etc; also controls all other randomization
   except RANDEXEC)
X  - RANDEXEC (random ET_EXEC base)

Each marking has three states:  On, Off, and Neutral.  The Neutral
marking takes the most restrictive setting by default, except in the
case of RANDEXEC, in which it's off because RANDEXEC can crash things
easily.  A sysctl in PaX can switch the Neutral logic, except for RANDEXEC.

For our purposes, SEGMEXEC is completely ignored; aside from x86 PaX,
SEGMEXEC has no impact on anything.  PAGEEXEC is equivalent to using the
processor supplied facilities to enforce PROT_EXEC (or emulation to do
such on x86).

I believe that these could map easily to PT_GNU_STACK's functionality:

P  - Page-based PROT_EXEC enforcement.  With this off, PROT_EXEC has no
 effect.  This is slightly different than normal PT_GNU_STACK
 (executable stack).
R  - Randomization.  With this off, the ASLR for stack, heap, mmap(),
 etc are disabled.
E  - Trampoline Emulation, port this from PaX in the future

If PT_PAX_FLAGS exists (the logic to check and read it can be derived
from PaX), these markings should be used; else, PT_GNU_STACK should be
fallen back to.  This would have no effect on any current distributions;
however, future development could mark for any situation and potentially
have one marking which simultaneously make the program work under PaX,
Exec Shield, and Vanilla.

The default state (when these are Neutral) would be the more
restrictive, of course; that is, PRe would be equal to ---.

Trampoline emulation would allow for a task to use nested functions with
a non-executable stack.  This would allow some PT_GNU_STACK markings to
come off for i.e. x86-64 and for Exec Shield.

Currently my understanding of how PT_GNU_STACK affects mainline and exec
shield kernels is very limited.  The above is what I came up under the
assumption that it simply allows the stack to be made executable and
allows ASLR to be turned off with one setting, rather than individual
fine-grained settings; I may need more information.  If anyone has any
suggestions, comments, etc, I'd like to hear them.

To do this, I'd pretty much simply have to port over the part of PaX
which checks for PT_PAX_FLAGS and if it can't find it falls back to
EI_PAX, changing that to falling back to PT_GNU_STACK and appropriate
code; and tweak the code to reflect mainline rather than a PaX kernel.
Not sure how I'd do that exactly, but I have a whole week off work and
class somehow so maybe I'll learn something.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSEsThDd4aOud5P8RAnh8AJ9dv11ozerlf5MKGzeFIyvLpucjtACfV+vF
Ll8bAJBbTCReRBwToCGVX/c=
=1qEV
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Greetings.

Currently I'm in need of some information about both vanilla and Exec
Shield kernels in regards to markings emitted by the toolchain,
specifically PT_GNU_STACK.  I'd like to check my assumptions, in
preparation for possibly making a non-intrusive change which would allow
the PT_PAX_FLAGS from PaX to be used in a more finegrained way than
PT_GNU_STACK is used now to control mainline and exec shield memory
protections.

As I understand, PT_GNU_STACK uses a single marking to control whether a
task gets an executable stack and whether ASLR is applied to the
executable.  I would like more information on this.  To start, I'll
supply some background on why I need this information.

PaX uses its own markings, PT_PAX_FLAGS, to control the way PaX applies
protections to processes.  In particular, PaX supplies the following
markings:

P  - PAGEEXEC (NX bit or emulated NX bit)
S  - SEGMEXEC (NX emulation on x86, ignored for our purposes)
E  - EMUTRAMP (Trampoline emulation, to avoid giving an executable stack
   in situations where nested functions are used)
M  - MPROTECT (mprotect() restrictions, to control kernel policy on how
   memory protections may be applied in userspace)
R  - RANDMMAP (mmap() base randomization, for libraries, PIEs, anonymous
   segments, etc; also controls all other randomization
   except RANDEXEC)
X  - RANDEXEC (random ET_EXEC base)

Each marking has three states:  On, Off, and Neutral.  The Neutral
marking takes the most restrictive setting by default, except in the
case of RANDEXEC, in which it's off because RANDEXEC can crash things
easily.  A sysctl in PaX can switch the Neutral logic, except for RANDEXEC.

For our purposes, SEGMEXEC is completely ignored; aside from x86 PaX,
SEGMEXEC has no impact on anything.  PAGEEXEC is equivalent to using the
processor supplied facilities to enforce PROT_EXEC (or emulation to do
such on x86).

I believe that these could map easily to PT_GNU_STACK's functionality:

P  - Page-based PROT_EXEC enforcement.  With this off, PROT_EXEC has no
 effect.  This is slightly different than normal PT_GNU_STACK
 (executable stack).
R  - Randomization.  With this off, the ASLR for stack, heap, mmap(),
 etc are disabled.
E  - Trampoline Emulation, port this from PaX in the future

If PT_PAX_FLAGS exists (the logic to check and read it can be derived
from PaX), these markings should be used; else, PT_GNU_STACK should be
fallen back to.  This would have no effect on any current distributions;
however, future development could mark for any situation and potentially
have one marking which simultaneously make the program work under PaX,
Exec Shield, and Vanilla.

The default state (when these are Neutral) would be the more
restrictive, of course; that is, PRe would be equal to ---.

Trampoline emulation would allow for a task to use nested functions with
a non-executable stack.  This would allow some PT_GNU_STACK markings to
come off for i.e. x86-64 and for Exec Shield.

Currently my understanding of how PT_GNU_STACK affects mainline and exec
shield kernels is very limited.  The above is what I came up under the
assumption that it simply allows the stack to be made executable and
allows ASLR to be turned off with one setting, rather than individual
fine-grained settings; I may need more information.  If anyone has any
suggestions, comments, etc, I'd like to hear them.

To do this, I'd pretty much simply have to port over the part of PaX
which checks for PT_PAX_FLAGS and if it can't find it falls back to
EI_PAX, changing that to falling back to PT_GNU_STACK and appropriate
code; and tweak the code to reflect mainline rather than a PaX kernel.
Not sure how I'd do that exactly, but I have a whole week off work and
class somehow so maybe I'll learn something.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSEsThDd4aOud5P8RAnh8AJ9dv11ozerlf5MKGzeFIyvLpucjtACfV+vF
Ll8bAJBbTCReRBwToCGVX/c=
=1qEV
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
As I understand, PT_GNU_STACK uses a single marking to control whether a
task gets an executable stack and whether ASLR is applied to the
executable.
 
 
 you understand wrongly.
 
 PT_GNU_STACK just sets the exec permission for the stack (and the heap
 now mirrors the stack). Nothing more nothing less.
 

So then this would be slightly more useful than I had previously
thought, bringing control over the randomization as well?

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSFINhDd4aOud5P8RAv36AJ9kFyE4u14CAVvWNu4bl11Gd125agCfVj3I
gNPQRd73mWJCLrPd5Ge/EnM=
=jqg0
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
 On Mon, 2005-03-28 at 13:50 -0500, John Richard Moser wrote:
 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:

As I understand, PT_GNU_STACK uses a single marking to control whether a
task gets an executable stack and whether ASLR is applied to the
executable.


you understand wrongly.

PT_GNU_STACK just sets the exec permission for the stack (and the heap
now mirrors the stack). Nothing more nothing less.


So then this would be slightly more useful than I had previously
thought, bringing control over the randomization as well?
 
 
 actually Linus was really against adding non-related things to this
 flag. And I think he is right...
 

I'm not interested in altering and hacking up PT_GNU_STACK; PT_PAX_FLAGS
already supplies enough to do what I want.  My goal is to have
PT_PAX_FLAGS code in mainline and Exec Shield, so that if it exists in
the binary it will be used; else PT_GNU_STACK will be fallen back to.

 Now.. do you have any examples of when you want a binary marked for no-
 randomisation ?? (eg something the setarch flag won't fix/won't be good
 enough for)

What's setarch do for one?  Anyway, ASLR has been known to break some
things.  Blackdown Java used to break IIRC; also there's the poorly
designed Oracle and the poorly designed solution of Oracle on a 32 bit
platform; and of course there's Emacs, which I heard was broken due to
Exec Shield's randomization.  Temporary work-arounds are sometimes needed.



Remember also that I'm not just trying to make a more robust setting for
ES and mainline; I'm trying to find a way to make it so that
distribution maintainers can set one set of flags and have it assure
that the program works in Mainline, Exec Shield, and PaX.  Just a little
less work for the distribution maintainers, which I think would be a
good thing considering that apparently Ubuntu Linux might support both
PaX and Exec Shield in the future, if I'm reading this[1] right.

[1] http://thread.gmane.org/gmane.linux.ubuntu.devel/6130


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSFeuhDd4aOud5P8RApQ+AKCPtp5b4/2rw+aRqEUg7r1FlphmQwCfX3Io
FUNq9xZlDsoo1poGBo5+zus=
=v0dv
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [ubuntu-hardened] Re: Collecting NX information

2005-03-28 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Brandon Hale wrote:
actually Linus was really against adding non-related things to this
flag. And I think he is right...

 
 
 Makes sense to me.
 
 

[...]

 
 IMO you have this backwards, John.  Rather than having the majority (ES,
 mainline NX stuff) respect your less popular branch, it would make sense
 to have PaX work as well as possible with PT_GNU_STACK, and politely
 request that any missing functionality be duplicated in ES. PT_GNU_STACK
 is the most widely available header, and we should leverage that
 ubiquity as much as possible.  Marking our binaries with PT_PAX_FLAGS
 and then begging everyone else to support our way of doing things will
 never work.
 

You need to consider that in the end I'd need PT_GNU_STACK to do
everything PaX wants, and to make PF_X a tristate (On, Off, Neutral)
which mapped to PF_PAGEEXEC in PaX.  In other words, I'd have to
overhaul and most likely *break* everything else that relied on
PT_GNU_STACK, instead of passively adding logic for PT_PAX_FLAGS and
letting everyone else catch up at their leisure.

I'd rather not break anything and force everyone to upgrade *now*; but
instead add PT_PAX_FLAGS functionality for mainline/ES, let it lay there
for a year or so as people start moving over and accepting it, let the
kernel devs decide when it's time to depricate PT_GNU_STACK, and see it
naturally decay away once it's actually no longer needed.  The point is
to not break anything, yet to still make things easier for those
projects and distributions like Hardened Ubuntu.


 ---
 Brandon Hale

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCSIKKhDd4aOud5P8RAkqEAJwNhFvfDc63qyPrBvMxs6naG1xuAQCfZKHn
upzqNI5/XzYVCmDKGM6q8ZY=
=YZkT
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: vfat broken in 2.6.10?

2005-03-24 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



OGAWA Hirofumi wrote:
> John Richard Moser <[EMAIL PROTECTED]> writes:
> 
> 
>>It appears dosfsck may not be working quite right.  I've taken this into
>>account, hence the second pass after each fsck.  This is either a
>>dosfsck issue, a usb-storage issue for the PNY compact flash drive, or
>>an issue with vfat itself.
>>
>>So either LKML needs to fix the drivers, or Ubuntu needs to upgrade/fix
>>dosfsck or some patch they've applied to the kernel.
> 
> 
> Can you try http://user.parknet.co.jp/hirofumi/tmp/fatfsprogs.tar.bz2,
> or most recently released dosfstools (2.11 or later)?

I would really, but I haven't mastered creating debian packages yet; on
Gentoo I just wrote ebuilds whenever I wanted to test something, then
uninstalled it if it broke.  Maybe someone else can do it. . . .

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCQwwlhDd4aOud5P8RAvDlAJ9EutR6A5BRF8Ej8JaXiDxNJKFbiwCfX56o
S4wcvdgMldvZjmhoRasFPLk=
=VubT
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: vfat broken in 2.6.10?

2005-03-24 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



OGAWA Hirofumi wrote:
 John Richard Moser [EMAIL PROTECTED] writes:
 
 
It appears dosfsck may not be working quite right.  I've taken this into
account, hence the second pass after each fsck.  This is either a
dosfsck issue, a usb-storage issue for the PNY compact flash drive, or
an issue with vfat itself.

So either LKML needs to fix the drivers, or Ubuntu needs to upgrade/fix
dosfsck or some patch they've applied to the kernel.
 
 
 Can you try http://user.parknet.co.jp/hirofumi/tmp/fatfsprogs.tar.bz2,
 or most recently released dosfstools (2.11 or later)?

I would really, but I haven't mastered creating debian packages yet; on
Gentoo I just wrote ebuilds whenever I wanted to test something, then
uninstalled it if it broke.  Maybe someone else can do it. . . .

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCQwwlhDd4aOud5P8RAvDlAJ9EutR6A5BRF8Ej8JaXiDxNJKFbiwCfX56o
S4wcvdgMldvZjmhoRasFPLk=
=VubT
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: vfat broken in 2.6.10?

2005-03-23 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Triffid Hunter wrote:
> i've seen the same problems with a fat32 partition image after an
> unclean shutdown. reading certain files would cause the filesystem to
> spontaneously become read-only with error messages similar to the ones
> you list below.

Clean umount, not unclean.  Not even removing the device, no shutdown.
I'm not damaging the filesystem except by actually using it.

It's remniscent of using NTFS with "full read-write" under Linux, where
you have lots of damage afterwards even though you didn't do anything
nasty to damage it.

> 
> my solution was to copy all the files off, rename the offending
> directory, and copy the relevant files back. unfortunately i can't
> remove the "dead" folder, as attempting to remove it sets the filesystem
> read-only :(
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCQi5AhDd4aOud5P8RAiedAJ40CyoAYYdu1jbSnl1j8J7/iXH2dQCbBuND
uZX0j3qIZe3TYQ2NMMelgks=
=XMEH
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


vfat broken in 2.6.10?

2005-03-23 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'm using Ubuntu Linux Hoary

[EMAIL PROTECTED]:~# uname -a
Linux icebox 2.6.10-5-686 #1 Tue Mar 15 15:16:01 UTC 2005 i686 GNU/Linux


[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/\uSCK.REN
  Duplicate directory entry.
  FirstSize 0 bytes, date 19:03:36 Mar 04 1980
  Second   Size 0 bytes, date 19:00:00 Dec 31 1979
1) Drop first
2) Drop second
3) Rename first
4) Rename second
5) Auto-rename first
6) Auto-rename second
? 5
  Renamed to FSCK.REN
Perform changes ? (y/n) y
/dev/sda1: 187 files, 167150/246288 clusters
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/dev/sda1: 187 files, 167150/246288 clusters

Now the FS is clean.

[EMAIL PROTECTED]:~# mount /dev/sda1  /media/usbdisk/
[EMAIL PROTECTED]:~# vim /media/usbdisk/a.txt
[EMAIL PROTECTED]:~# rm /media/usbdisk/a.txt

Created, deleted a.txt

[EMAIL PROTECTED]:~# umount /media/usbdisk/
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/\u.\000a\000.\000t.\000x\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it

Does that say a.txt?

? 1
/\ua\000.\000t\000x.\000t\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 1
/\u.\000a\000.\000t.\000x\000
  Start cluster beyond limit (7798784 > 246289). Truncating file.
/\u.\000a\000.\000t.\000x\000
  File size is 4294967295 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
/\ua\000.\000t\000x.\000t\000
  Start cluster beyond limit (4294901760 > 246289). Truncating file.
/\ua\000.\000t\000x.\000t\000
  File size is 4294967295 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
/\u.TXT
  Contains a free cluster (167160). Assuming EOF.
/\u.TXT
  File size is 5 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
Perform changes ? (y/n) y


I didn't pull the disk out at all, or anything.  Just umounted and
remounted and made/removed a file.  Also. . .


/dev/sda1: 189 files, 167150/246288 clusters
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/\u.\000a\000.\000t.\000x\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 3
  Renamed to FSCK0001.REN
/\ua\000.\000t\000x.\000t\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 3
  Renamed to FSCK0002.REN
Perform changes ? (y/n) y
/dev/sda1: 191 files, 167150/246288 clusters
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/dev/sda1: 191 files, 167150/246288 clusters

It appears dosfsck may not be working quite right.  I've taken this into
account, hence the second pass after each fsck.  This is either a
dosfsck issue, a usb-storage issue for the PNY compact flash drive, or
an issue with vfat itself.

So either LKML needs to fix the drivers, or Ubuntu needs to upgrade/fix
dosfsck or some patch they've applied to the kernel.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCQePohDd4aOud5P8RAhxRAJ9ydXWfLZbu0r/B4rY6zhaWScR3rQCfVNMY
lGsqNpIPDIBREqIWGVlQJFo=
=1Nlg
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


vfat broken in 2.6.10?

2005-03-23 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'm using Ubuntu Linux Hoary

[EMAIL PROTECTED]:~# uname -a
Linux icebox 2.6.10-5-686 #1 Tue Mar 15 15:16:01 UTC 2005 i686 GNU/Linux


[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/\uSCK.REN
  Duplicate directory entry.
  FirstSize 0 bytes, date 19:03:36 Mar 04 1980
  Second   Size 0 bytes, date 19:00:00 Dec 31 1979
1) Drop first
2) Drop second
3) Rename first
4) Rename second
5) Auto-rename first
6) Auto-rename second
? 5
  Renamed to FSCK.REN
Perform changes ? (y/n) y
/dev/sda1: 187 files, 167150/246288 clusters
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/dev/sda1: 187 files, 167150/246288 clusters

Now the FS is clean.

[EMAIL PROTECTED]:~# mount /dev/sda1  /media/usbdisk/
[EMAIL PROTECTED]:~# vim /media/usbdisk/a.txt
[EMAIL PROTECTED]:~# rm /media/usbdisk/a.txt

Created, deleted a.txt

[EMAIL PROTECTED]:~# umount /media/usbdisk/
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/\u.\000a\000.\000t.\000x\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it

Does that say a.txt?

? 1
/\ua\000.\000t\000x.\000t\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 1
/\u.\000a\000.\000t.\000x\000
  Start cluster beyond limit (7798784  246289). Truncating file.
/\u.\000a\000.\000t.\000x\000
  File size is 4294967295 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
/\ua\000.\000t\000x.\000t\000
  Start cluster beyond limit (4294901760  246289). Truncating file.
/\ua\000.\000t\000x.\000t\000
  File size is 4294967295 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
/\u.TXT
  Contains a free cluster (167160). Assuming EOF.
/\u.TXT
  File size is 5 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
Perform changes ? (y/n) y


I didn't pull the disk out at all, or anything.  Just umounted and
remounted and made/removed a file.  Also. . .


/dev/sda1: 189 files, 167150/246288 clusters
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/\u.\000a\000.\000t.\000x\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 3
  Renamed to FSCK0001.REN
/\ua\000.\000t\000x.\000t\000
  Bad file name.
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 3
  Renamed to FSCK0002.REN
Perform changes ? (y/n) y
/dev/sda1: 191 files, 167150/246288 clusters
[EMAIL PROTECTED]:~# fsck.vfat -r /dev/sda1
dosfsck 2.10, 22 Sep 2003, FAT32, LFN
/dev/sda1: 191 files, 167150/246288 clusters

It appears dosfsck may not be working quite right.  I've taken this into
account, hence the second pass after each fsck.  This is either a
dosfsck issue, a usb-storage issue for the PNY compact flash drive, or
an issue with vfat itself.

So either LKML needs to fix the drivers, or Ubuntu needs to upgrade/fix
dosfsck or some patch they've applied to the kernel.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCQePohDd4aOud5P8RAhxRAJ9ydXWfLZbu0r/B4rY6zhaWScR3rQCfVNMY
lGsqNpIPDIBREqIWGVlQJFo=
=1Nlg
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-12 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

You wanna give me a quick run-down on x86 of CPL and Ring levels?  It's
been bugging me.  I know they're there and have a basic idea that they
control what a context can do, don't know what CPL stands for, and
there's a visible gap in my knowledge.  I like to understand everything,
it makes things easier.

Felipe Alfaro Solana wrote:
> On Thu, 10 Mar 2005 17:32:39 -0500, John Richard Moser
> <[EMAIL PROTECTED]> wrote:
> 
>>CPL=3 scares me; context switches are expensive.  can they have direct
>>hardware access?  I'm sure a security model to isolate user mode drivers
>>could be in place. . .
>>
>>. . . huh.  Xen seems to run Linux at CPL=3 and give direct hardware
>>access, so I guess user mode drivers are possible *shrug*.  Linux isn't
>>a microkernel though.
> 
> 
> Xen hypervisor runs at Ring0, while the guest OSs it supports run at Ring1.
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCM8kShDd4aOud5P8RAon1AKCLNWEbY3Vq32k61m9jN2CbSoD98QCeJT8m
mhgyXtmGNFL+RPzJw8md9hE=
=B/i5
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-12 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

You wanna give me a quick run-down on x86 of CPL and Ring levels?  It's
been bugging me.  I know they're there and have a basic idea that they
control what a context can do, don't know what CPL stands for, and
there's a visible gap in my knowledge.  I like to understand everything,
it makes things easier.

Felipe Alfaro Solana wrote:
 On Thu, 10 Mar 2005 17:32:39 -0500, John Richard Moser
 [EMAIL PROTECTED] wrote:
 
CPL=3 scares me; context switches are expensive.  can they have direct
hardware access?  I'm sure a security model to isolate user mode drivers
could be in place. . .

. . . huh.  Xen seems to run Linux at CPL=3 and give direct hardware
access, so I guess user mode drivers are possible *shrug*.  Linux isn't
a microkernel though.
 
 
 Xen hypervisor runs at Ring0, while the guest OSs it supports run at Ring1.
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCM8kShDd4aOud5P8RAon1AKCLNWEbY3Vq32k61m9jN2CbSoD98QCeJT8m
mhgyXtmGNFL+RPzJw8md9hE=
=B/i5
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Peter Chubb wrote:
>>>>>>"John" == John Richard Moser <[EMAIL PROTECTED]> writes:
> 
> 
> 
> John> I've done more thought, here's a small list of advantages on
> John> using binary drivers, specifically considering UDI.  You can
> John> consider a different implementation for binary drivers as well,
> John> with most of the same advantages.
> 
> Almost all these advantages are also present for user-mode drivers...
> and getting drivers out of the kernel, where possible, is a much
> better approach IMHO than trying to maintain a leaky in-kernel
> interface.  The problem with in-kernel interfaces, even if set in
> concrete, is that any binary driver can go outside the interface ---
> there's no encapsulation --- and so break when the kernel changes.
> 

CPL=3 scares me; context switches are expensive.  can they have direct
hardware access?  I'm sure a security model to isolate user mode drivers
could be in place. . .

. . . huh.  Xen seems to run Linux at CPL=3 and give direct hardware
access, so I guess user mode drivers are possible *shrug*.  Linux isn't
a microkernel though.

> Peter C
> 
> 
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD4DBQFCMMsGhDd4aOud5P8RAjoOAJ9Owgnd5QOp9MfrQ8PzOLAt/A7k+wCYmxLc
wvXLkQX84Z2PF2J4oEIbVA==
=wi8f
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

People are still e-mailing me about this?

Lennart Sorensen wrote:
> On Thu, Mar 10, 2005 at 12:24:15PM -0500, John Richard Moser wrote:
> 
>>I've done more thought, here's a small list of advantages on using
>>binary drivers, specifically considering UDI.  You can consider a
>>different implementation for binary drivers as well, with most of the
>>same advantages.
>>
>> - Smaller kernel tree
>>   The kernel tree would no longer contain all of the drivers; they'd
>>   slowly have to bleed into UDI until most were there
> 
> 
> Users would have to go hunting for drivers to add to their kernel to get
> hardware supported.  Making a CD with a kernel and drivers for a wide
> variety of hardware would be a nightmare.
> 

/pub/kernel/2.6
/pub/drivers/

> 
>> - Better focused development
>>   The kernel's core would be the core.  Driver code would be isolated,
>>   so work on the kernel would affect the kernel only and not any
>>   drivers.  UDI is a standard interface that shouldn't be broken.  This
>>   means that work on the high-level drivers will not need to be sanity
>>   checked a thousand times against the PCI Bus interface or the USB
>>   host controler API or whatnot.
> 
> 
> But anything that runs in kernel memory space can still go trampling on
> memory in the kernel by accident and is very difficult to debug without
> the sources.
> 

True, but that only should happen if you code things to access exact
memory locations, rather than asking the kernel for memory or mappings.

> 
>> - Faster rebuilding for developers
>>   The isolation between drivers and core would make rebuilding involve
>>   the particular component (driver, core).  A "broken driver" would
>>   just require recoding and rebuilding the driver; a "broken kernel"
>>   would require building pretty much a skeletal core
> 
> 
> That can already be done basicly.  The makefiles work just fine for
> rebuilding only what has changed in general.
> 

I don't remember what I was thinking
> 
>> - UDI supplies SMP safety
>>   The UDI page brags[1]:
>>
>>   "An advanced scheduling model. Multiple driver instances can be run
>>in parallel on multiple processors with no lock management performed
>>by the driver. Free paralllism and scalability!"
>>
>>   Drivers can be considered SMP safe, apparently.  Inside the same
>>   driver, however, I have my doubts; I can see a driver maintaining a
>>   linked list that needs to be locked during insertions or deletions,
>>   which needs lock managment for the driver.  Still, no consideration
>>   for anything outside the driver need be made, apparently.
>> - Vendor drivers and religious issues
>>   Vendors can supply third party drivers until there are open source
>>   alternatives, since they have this religious thing where they don't
>>   want people to see their driver code, which is kind of annoying and
>>   impedes progress
> 
> 
> I imagine a driver writer could still easily do something not SMP safe,
> but I don't know that for sure.  It sounds like a very complex thing to
> promise a perfect solution for.
> 

internally drivers would need to be smp safe, eh.  Externally I guess
they're safe.

> 
>>Disadvantages:
>>
>> - Preemption
>>   Is it still possible to implement a soft realtime kernel that
>>   responds to interrupts quickly?
>> - Performance
>>   UDI's developers claim that the performance overhead is negligible.
>>   It's still added work, but it remains to be seen if it's significant
>>   enough to degrade performance.
>> - Religious battles
>>   People have this religious thing about binary drivers, which is kind
>>   of annoying and impedes progress
> 
> 
> Many of the disadvantages are a good reason why they have these opinions
> on binary drivers.  They do impede getting work done if you have to use
> them on your system and something isn't working right.
> 
> 
>> - Constriction
>>   This would of course create an abstraction layer that constricts the
>>   driver developer's ability to do low level complex operations for any
>>   portable binary driver
> 
> 
> You forgot the very important:
>- Only works on architecture it was compiled for.  So anyone not
>  using i386 (and maybe later x86-64) is simply out of luck.  What do
>  nvidia users that want accelerated nvidia drivers for X DRI do
>  right now if they have a powerpc or a sparc or an alpha?  How about
>  porting Linux to a new architecture.  With binary drivers you now
>

Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Stop mailing me, I lost interest when I figured out nobody else cared.

Diego Calleja wrote:
> El Thu, 10 Mar 2005 12:24:15 -0500,
> John Richard Moser <[EMAIL PROTECTED]> escribió:
> 
> [...]
> 
>> - Smaller kernel tree
> 
> [...]
> 
>> - Better focused development
> 
> [...]
> 
>> - Faster rebuilding for developers
> 
> 
> It can be done without UDI.
> 
> 

Then do so.

> 
>>- UDI supplies SMP safety
> 
> 
> Well designed drivers don't have SMP issues either...
> 

I figured as much but *shrug*

> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMKqihDd4aOud5P8RAqifAJ0aH26fq/x6AatH5S2ji2PRCsbOrQCgkGht
xKVJ2rbkE+WGxCkGqyqrckM=
=nQYU
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Ralf Baechle wrote:
> On Thu, Mar 10, 2005 at 11:28:39AM -0500, John Richard Moser wrote:
> 
> 
>>I've been looking at the UDI project[1] and thinking about binary
>>drivers and the like, and wondering what most peoples' take on these are
>>and what impact that UDI support would have on the kernel's development.
> 
> 
> UDI is already dead.  You just haven't noticed.  And just like with dead
> people it's death also had a cause :-)
> 

File last modified: 11:28 Tue December 14, 2004

3 months and it's dead.  Linux hasn't passed 2.6 in over 3 months, it
must be dead too :)
>   Ralf
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMIMEhDd4aOud5P8RAvHzAJ99xQyr4SF98zd7VbmcSDCsEUEpFgCePAP8
7WdLwkOCKXkaycLNdL8KBbI=
=o68B
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I've done more thought, here's a small list of advantages on using
binary drivers, specifically considering UDI.  You can consider a
different implementation for binary drivers as well, with most of the
same advantages.

 - Smaller kernel tree
   The kernel tree would no longer contain all of the drivers; they'd
   slowly have to bleed into UDI until most were there
 - Better focused development
   The kernel's core would be the core.  Driver code would be isolated,
   so work on the kernel would affect the kernel only and not any
   drivers.  UDI is a standard interface that shouldn't be broken.  This
   means that work on the high-level drivers will not need to be sanity
   checked a thousand times against the PCI Bus interface or the USB
   host controler API or whatnot.
 - Faster rebuilding for developers
   The isolation between drivers and core would make rebuilding involve
   the particular component (driver, core).  A "broken driver" would
   just require recoding and rebuilding the driver; a "broken kernel"
   would require building pretty much a skeletal core
 - UDI supplies SMP safety
   The UDI page brags[1]:

   "An advanced scheduling model. Multiple driver instances can be run
in parallel on multiple processors with no lock management performed
by the driver. Free paralllism and scalability!"

   Drivers can be considered SMP safe, apparently.  Inside the same
   driver, however, I have my doubts; I can see a driver maintaining a
   linked list that needs to be locked during insertions or deletions,
   which needs lock managment for the driver.  Still, no consideration
   for anything outside the driver need be made, apparently.
 - Vendor drivers and religious issues
   Vendors can supply third party drivers until there are open source
   alternatives, since they have this religious thing where they don't
   want people to see their driver code, which is kind of annoying and
   impedes progress

Disadvantages:

 - Preemption
   Is it still possible to implement a soft realtime kernel that
   responds to interrupts quickly?
 - Performance
   UDI's developers claim that the performance overhead is negligible.
   It's still added work, but it remains to be seen if it's significant
   enough to degrade performance.
 - Religious battles
   People have this religious thing about binary drivers, which is kind
   of annoying and impedes progress
 - Constriction
   This would of course create an abstraction layer that constricts the
   driver developer's ability to do low level complex operations for any
   portable binary driver

A Linux specific binary driver format might be more useful, but wouldn't
potentially allow for sharing the drivers across operating systems

[1] http://projectudi.sourceforge.net/about.php
- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMIK8hDd4aOud5P8RAo+EAJwIF7zEmiyxKzRJJ037TQ2FhYG5bACffTBX
JLhXPcidbQbf18LyTmjHgh0=
=gbyB
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Greg KH wrote:
> On Thu, Mar 10, 2005 at 11:28:39AM -0500, John Richard Moser wrote:
> 
>>I've been looking at the UDI project[1] and thinking about binary
>>drivers and the like, and wondering what most peoples' take on these are
>>and what impact that UDI support would have on the kernel's development.
> 
> 
> Please, the UDI stuff has been proven to be broken and wrong.  If you
> want to work on it, feel free to do so, just don't expect for anyone to
> accept the UDI layer into the kernel mainline.
> 

1.  What's broken about it
2.  Is it possible to fix it

I require information or else I can't assess things.

> What's that phrase about people forgetting history are doomed...
> 
> greg k-h
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMIGqhDd4aOud5P8RAl+PAJ4ndKTmqyRRc+qaJjPK62HBABb0rgCgjCTR
8kQ9MOOdZiH3FUsNG1Hoe3E=
=NIcs
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I've been looking at the UDI project[1] and thinking about binary
drivers and the like, and wondering what most peoples' take on these are
and what impact that UDI support would have on the kernel's development.

I know the immediate first reactions are probably "Portable drivers are
slow" and "We don't support closed source crap."  I think benchmarks are
needed, and closed drivers can always be replaced by rewritten open
ones.  Really critical drivers that need the extra few microseconds can
always be low-level instead of abstracted.

The major considerations I come across mainly involve development focus
and kernel tree size.  UDI drivers would be separated from the kernel,
so their development would be focused; a driver fix would not warrent a
2.6, 2.4, and 2.2 patch, plus backports in 100 distributions (which
don't concern the LKML).  They'd also be in their own tarball, so the
kernel tree size would be a lot smaller if most drivers were UDI, though
by how much I'm not sure.

I'm aware that drivers would have to be loaded to the kernel like this,
being separated.  Aside from having the kernel build eat drivers from
some /lib/modules/udi/ directory, grub has a "module" command that can
load multiple modules.  If the kernel used that to load drivers (does it
now?), an initrd wouldn't be needed; a very straightforward bootloader
configuration would come in its place.

There is a 2.4 UDI reference model out with SCSI and NIC driver support.
 Perhaps some experimentation with these would be interesting, since the
disk and the network are performance focuses.  I don't think the UDI
reference model is ready quite yet for mainline, but it's ready for some
cross-examination by unbiased kernel developers.

[1] http://projectudi.sourceforge.net/

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMHW2hDd4aOud5P8RAnhSAJ9+8VdgR6EX0JwjUpysXsTMRl1ewwCeOWJR
g6mNs6NuEltgNS6qtVat5Mo=
=DrWO
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I've been looking at the UDI project[1] and thinking about binary
drivers and the like, and wondering what most peoples' take on these are
and what impact that UDI support would have on the kernel's development.

I know the immediate first reactions are probably Portable drivers are
slow and We don't support closed source crap.  I think benchmarks are
needed, and closed drivers can always be replaced by rewritten open
ones.  Really critical drivers that need the extra few microseconds can
always be low-level instead of abstracted.

The major considerations I come across mainly involve development focus
and kernel tree size.  UDI drivers would be separated from the kernel,
so their development would be focused; a driver fix would not warrent a
2.6, 2.4, and 2.2 patch, plus backports in 100 distributions (which
don't concern the LKML).  They'd also be in their own tarball, so the
kernel tree size would be a lot smaller if most drivers were UDI, though
by how much I'm not sure.

I'm aware that drivers would have to be loaded to the kernel like this,
being separated.  Aside from having the kernel build eat drivers from
some /lib/modules/udi/ directory, grub has a module command that can
load multiple modules.  If the kernel used that to load drivers (does it
now?), an initrd wouldn't be needed; a very straightforward bootloader
configuration would come in its place.

There is a 2.4 UDI reference model out with SCSI and NIC driver support.
 Perhaps some experimentation with these would be interesting, since the
disk and the network are performance focuses.  I don't think the UDI
reference model is ready quite yet for mainline, but it's ready for some
cross-examination by unbiased kernel developers.

[1] http://projectudi.sourceforge.net/

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMHW2hDd4aOud5P8RAnhSAJ9+8VdgR6EX0JwjUpysXsTMRl1ewwCeOWJR
g6mNs6NuEltgNS6qtVat5Mo=
=DrWO
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Greg KH wrote:
 On Thu, Mar 10, 2005 at 11:28:39AM -0500, John Richard Moser wrote:
 
I've been looking at the UDI project[1] and thinking about binary
drivers and the like, and wondering what most peoples' take on these are
and what impact that UDI support would have on the kernel's development.
 
 
 Please, the UDI stuff has been proven to be broken and wrong.  If you
 want to work on it, feel free to do so, just don't expect for anyone to
 accept the UDI layer into the kernel mainline.
 

1.  What's broken about it
2.  Is it possible to fix it

I require information or else I can't assess things.

 What's that phrase about people forgetting history are doomed...
 
 greg k-h
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMIGqhDd4aOud5P8RAl+PAJ4ndKTmqyRRc+qaJjPK62HBABb0rgCgjCTR
8kQ9MOOdZiH3FUsNG1Hoe3E=
=NIcs
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I've done more thought, here's a small list of advantages on using
binary drivers, specifically considering UDI.  You can consider a
different implementation for binary drivers as well, with most of the
same advantages.

 - Smaller kernel tree
   The kernel tree would no longer contain all of the drivers; they'd
   slowly have to bleed into UDI until most were there
 - Better focused development
   The kernel's core would be the core.  Driver code would be isolated,
   so work on the kernel would affect the kernel only and not any
   drivers.  UDI is a standard interface that shouldn't be broken.  This
   means that work on the high-level drivers will not need to be sanity
   checked a thousand times against the PCI Bus interface or the USB
   host controler API or whatnot.
 - Faster rebuilding for developers
   The isolation between drivers and core would make rebuilding involve
   the particular component (driver, core).  A broken driver would
   just require recoding and rebuilding the driver; a broken kernel
   would require building pretty much a skeletal core
 - UDI supplies SMP safety
   The UDI page brags[1]:

   An advanced scheduling model. Multiple driver instances can be run
in parallel on multiple processors with no lock management performed
by the driver. Free paralllism and scalability!

   Drivers can be considered SMP safe, apparently.  Inside the same
   driver, however, I have my doubts; I can see a driver maintaining a
   linked list that needs to be locked during insertions or deletions,
   which needs lock managment for the driver.  Still, no consideration
   for anything outside the driver need be made, apparently.
 - Vendor drivers and religious issues
   Vendors can supply third party drivers until there are open source
   alternatives, since they have this religious thing where they don't
   want people to see their driver code, which is kind of annoying and
   impedes progress

Disadvantages:

 - Preemption
   Is it still possible to implement a soft realtime kernel that
   responds to interrupts quickly?
 - Performance
   UDI's developers claim that the performance overhead is negligible.
   It's still added work, but it remains to be seen if it's significant
   enough to degrade performance.
 - Religious battles
   People have this religious thing about binary drivers, which is kind
   of annoying and impedes progress
 - Constriction
   This would of course create an abstraction layer that constricts the
   driver developer's ability to do low level complex operations for any
   portable binary driver

A Linux specific binary driver format might be more useful, but wouldn't
potentially allow for sharing the drivers across operating systems

[1] http://projectudi.sourceforge.net/about.php
- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMIK8hDd4aOud5P8RAo+EAJwIF7zEmiyxKzRJJ037TQ2FhYG5bACffTBX
JLhXPcidbQbf18LyTmjHgh0=
=gbyB
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Ralf Baechle wrote:
 On Thu, Mar 10, 2005 at 11:28:39AM -0500, John Richard Moser wrote:
 
 
I've been looking at the UDI project[1] and thinking about binary
drivers and the like, and wondering what most peoples' take on these are
and what impact that UDI support would have on the kernel's development.
 
 
 UDI is already dead.  You just haven't noticed.  And just like with dead
 people it's death also had a cause :-)
 

File last modified: 11:28 Tue December 14, 2004

3 months and it's dead.  Linux hasn't passed 2.6 in over 3 months, it
must be dead too :)
   Ralf
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMIMEhDd4aOud5P8RAvHzAJ99xQyr4SF98zd7VbmcSDCsEUEpFgCePAP8
7WdLwkOCKXkaycLNdL8KBbI=
=o68B
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Stop mailing me, I lost interest when I figured out nobody else cared.

Diego Calleja wrote:
 El Thu, 10 Mar 2005 12:24:15 -0500,
 John Richard Moser [EMAIL PROTECTED] escribió:
 
 [...]
 
 - Smaller kernel tree
 
 [...]
 
 - Better focused development
 
 [...]
 
 - Faster rebuilding for developers
 
 
 It can be done without UDI.
 
 

Then do so.

 
- UDI supplies SMP safety
 
 
 Well designed drivers don't have SMP issues either...
 

I figured as much but *shrug*

 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCMKqihDd4aOud5P8RAqifAJ0aH26fq/x6AatH5S2ji2PRCsbOrQCgkGht
xKVJ2rbkE+WGxCkGqyqrckM=
=nQYU
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

People are still e-mailing me about this?

Lennart Sorensen wrote:
 On Thu, Mar 10, 2005 at 12:24:15PM -0500, John Richard Moser wrote:
 
I've done more thought, here's a small list of advantages on using
binary drivers, specifically considering UDI.  You can consider a
different implementation for binary drivers as well, with most of the
same advantages.

 - Smaller kernel tree
   The kernel tree would no longer contain all of the drivers; they'd
   slowly have to bleed into UDI until most were there
 
 
 Users would have to go hunting for drivers to add to their kernel to get
 hardware supported.  Making a CD with a kernel and drivers for a wide
 variety of hardware would be a nightmare.
 

/pub/kernel/2.6
/pub/drivers/

 
 - Better focused development
   The kernel's core would be the core.  Driver code would be isolated,
   so work on the kernel would affect the kernel only and not any
   drivers.  UDI is a standard interface that shouldn't be broken.  This
   means that work on the high-level drivers will not need to be sanity
   checked a thousand times against the PCI Bus interface or the USB
   host controler API or whatnot.
 
 
 But anything that runs in kernel memory space can still go trampling on
 memory in the kernel by accident and is very difficult to debug without
 the sources.
 

True, but that only should happen if you code things to access exact
memory locations, rather than asking the kernel for memory or mappings.

 
 - Faster rebuilding for developers
   The isolation between drivers and core would make rebuilding involve
   the particular component (driver, core).  A broken driver would
   just require recoding and rebuilding the driver; a broken kernel
   would require building pretty much a skeletal core
 
 
 That can already be done basicly.  The makefiles work just fine for
 rebuilding only what has changed in general.
 

I don't remember what I was thinking
 
 - UDI supplies SMP safety
   The UDI page brags[1]:

   An advanced scheduling model. Multiple driver instances can be run
in parallel on multiple processors with no lock management performed
by the driver. Free paralllism and scalability!

   Drivers can be considered SMP safe, apparently.  Inside the same
   driver, however, I have my doubts; I can see a driver maintaining a
   linked list that needs to be locked during insertions or deletions,
   which needs lock managment for the driver.  Still, no consideration
   for anything outside the driver need be made, apparently.
 - Vendor drivers and religious issues
   Vendors can supply third party drivers until there are open source
   alternatives, since they have this religious thing where they don't
   want people to see their driver code, which is kind of annoying and
   impedes progress
 
 
 I imagine a driver writer could still easily do something not SMP safe,
 but I don't know that for sure.  It sounds like a very complex thing to
 promise a perfect solution for.
 

internally drivers would need to be smp safe, eh.  Externally I guess
they're safe.

 
Disadvantages:

 - Preemption
   Is it still possible to implement a soft realtime kernel that
   responds to interrupts quickly?
 - Performance
   UDI's developers claim that the performance overhead is negligible.
   It's still added work, but it remains to be seen if it's significant
   enough to degrade performance.
 - Religious battles
   People have this religious thing about binary drivers, which is kind
   of annoying and impedes progress
 
 
 Many of the disadvantages are a good reason why they have these opinions
 on binary drivers.  They do impede getting work done if you have to use
 them on your system and something isn't working right.
 
 
 - Constriction
   This would of course create an abstraction layer that constricts the
   driver developer's ability to do low level complex operations for any
   portable binary driver
 
 
 You forgot the very important:
- Only works on architecture it was compiled for.  So anyone not
  using i386 (and maybe later x86-64) is simply out of luck.  What do
  nvidia users that want accelerated nvidia drivers for X DRI do
  right now if they have a powerpc or a sparc or an alpha?  How about
  porting Linux to a new architecture.  With binary drivers you now
  start out with no drivers on the new architecture except for the
  ones you have source for.  Not very productive.
 
 [snip]
 

yeah, I was thinking the open source drivers would be ubiquitous to all
architectures anyway though.  Closed drivers would be subject to lazy
venders.

 Len Sorensen
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out

Re: binary drivers and development

2005-03-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Peter Chubb wrote:
John == John Richard Moser [EMAIL PROTECTED] writes:
 
 
 
 John I've done more thought, here's a small list of advantages on
 John using binary drivers, specifically considering UDI.  You can
 John consider a different implementation for binary drivers as well,
 John with most of the same advantages.
 
 Almost all these advantages are also present for user-mode drivers...
 and getting drivers out of the kernel, where possible, is a much
 better approach IMHO than trying to maintain a leaky in-kernel
 interface.  The problem with in-kernel interfaces, even if set in
 concrete, is that any binary driver can go outside the interface ---
 there's no encapsulation --- and so break when the kernel changes.
 

CPL=3 scares me; context switches are expensive.  can they have direct
hardware access?  I'm sure a security model to isolate user mode drivers
could be in place. . .

. . . huh.  Xen seems to run Linux at CPL=3 and give direct hardware
access, so I guess user mode drivers are possible *shrug*.  Linux isn't
a microkernel though.

 Peter C
 
 
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be
wasted on re-inventing the wheel when there are so many fascinating
new problems waiting out there.
 -- Eric Steven Raymond
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD4DBQFCMMsGhDd4aOud5P8RAjoOAJ9Owgnd5QOp9MfrQ8PzOLAt/A7k+wCYmxLc
wvXLkQX84Z2PF2J4oEIbVA==
=wi8f
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Filesystem linking protections

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
> * John Richard Moser ([EMAIL PROTECTED]) wrote:
> 
>>Yes, mkdtemp() and mkstemp().
>>
>>Of course we can't always rely on programmers to get it right, so the
>>idea here is to make sure we ask broken code to behave nicely, and stab
>>it in the face if it doesn't.  Please try to examine this in that scope.
> 
> 
> It's fine for hardened distro.  But still inappropriate for mainline.
> 

Perhaps in mainline as an option?  The [*] notations next to things are
really nice, they let you turn kernel stuff on and off :)  It's
appropriate for mainline to support added security isn't it?  I think
following the path of supporting-but-not-forcing is the best route,
because it encourages people to account for systems which may take
advantage of such options, and thus leads to a software base in which
it's quite sane to actually enable those options globally.

That's just how I think though.

> thanks,
> -chris

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCCB+GhDd4aOud5P8RAlD9AJ45JTY20WY6qHe0h0ZIcFasgxJDtACbB1aB
i4hytMAy6Cs1AUNXC296JOk=
=oLVs
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Filesystem linking protections

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
> * John Richard Moser ([EMAIL PROTECTED]) wrote:
> 
>>I've yet to see this break anything on Ubuntu or Gentoo; Brad Spengler
>>claims this breaks nothing on Debian.  On the other hand, this could
>>potentially squash the second most prevalent security bug.
> 
> 
> Yes I know, I've worked on distro with it as well in the past.  And it
> has broken atd and courier in the past.  This is something that also
> can be done in userspace using sane subdirs in +t world writable dirs,
> or O_EXCL so there's work to be done in userspace.
> 

Yes, mkdtemp() and mkstemp().

Of course we can't always rely on programmers to get it right, so the
idea here is to make sure we ask broken code to behave nicely, and stab
it in the face if it doesn't.  Please try to examine this in that scope.

> thanks,
> -chris

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB+vThDd4aOud5P8RAssCAJ9L7Cf5pnvI8GdKs1P4cpM2lJvtYACZAXee
a5kkPkxXm9YK0DFSfvDd6fQ=
=00DK
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Filesystem linking protections

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
> * Lorenzo Hernández García-Hierro ([EMAIL PROTECTED]) wrote:
> 
>>This patch adds two checks to do_follow_link() and sys_link(), for
>>prevent users to follow (untrusted) symlinks owned by other users in
>>world-writable +t directories (i.e. /tmp), unless the owner of the
>>symlink is the owner of the directory, users will also not be able to
>>hardlink to files they do not own.
>>
>>The direct advantage of this pretty simple patch is that /tmp races will
>>be prevented.
> 
> 
> The disadvantage is that it can break things and places policy in the
> kernel.
> 

It can break things, yes.  For example, programs which have and use two
separate FS UIDs at the same time, or which attempt to make hardlinks to
files they don't own without CAP_FOWNER or root (should this just be
CAP_FOWNER?  Is root now irrelavent?).

Hang on, when do any programs have 2 FS UIDs at the same time. . . .

I've yet to see this break anything on Ubuntu or Gentoo; Brad Spengler
claims this breaks nothing on Debian.  On the other hand, this could
potentially squash the second most prevalent security bug.

> thanks,
> -chris

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB8S0hDd4aOud5P8RAvYSAJ9zcGArfbC6i5uM1JW4ZHdELriUzACeOH/q
5ndpSdjporfnFAMK1OrMASE=
=XjWB
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Sabotaged PaXtest

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
> On Mon, 2005-01-31 at 13:57 +0100, Peter Busser wrote:
> 
>>Hi!

[...]

> the paxtest 0.9.6 that John Moser mailed to this list had this gem in
> it:
> @@ -39,8 +42,6 @@
>  */
> int paxtest_mode = 1;
> 
> +   /* Dummy nested function */
> +   void dummy(void) {}
> 
> mode = getenv( "PAXTEST_MODE" );
> if( mode == NULL ) {
> 
> 
> which is clearly there with the only possible function of sabotaging the
> automatic PT_GNU_STACK setting by the toolchain (which btw is not fedora
> specific but happens by all new enough (3.3 or later) gcc compilers on
> all distros) since that requires an executable stack.

I'll say this AGAIN:  I execstack -c'd the directory after the first
borked up test and stuff randomly failed.  After upgrading FC3's kernel
though I got predictable results.


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB7T0hDd4aOud5P8RAk7XAJ4oYgr2ySTdg+y80f6pBasO+x8ttACfdjYx
++dQRSTGzTGP/Vsp2KZ6YHU=
=jp9U
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Sabotaged PaXtest (was: Re: Patch 4/6 randomize the stack pointer)

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Roman Zippel wrote:
> Hi,
> 
> On Thu, 3 Feb 2005, Peter Busser wrote:
> 
> 
>>- What happens when you run existing commercial applications which have not 
>>been compiled using GCC.
> 
> 
>>From http://pax.grsecurity.net/docs/pax.txt:
> 
>The goal of the PaX project is to research various defense mechanisms
>against the exploitation of software bugs that give an attacker arbitrary
>read/write access to the attacked task's address space. 
> 
> Could you please explain how PaX makes such applications secure?
> 

I wrote an easy-to-chew article[1] about PaX on Wikipedia, although
looking back at it I think there may be some erratta in the ASLR
concept; I think the mmap() base is randomized, but I'm not sure now if
the actual base of each mmap() call is individually randomized as shown
in my diagrams.  I'm also no longer sure where I got the notion that the
heap/.bss/data segments are the same entity, and I'll have to check on that.

Nevertheless, it's basically accurate, in the same way that saying you
have a gameboy advance SP when you just have a gameboy advance is
basically accurate.

[1] http://en.wikipedia.org/wiki/PaX

> bye, Roman
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB7PlhDd4aOud5P8RAr+pAKCCcbqLuG7OQzZlJrd5UdsA3NooUgCePXnp
D+xS98fWm9MVEBZpB+pIrTY=
=r+20
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Sabotaged PaXtest (was: Re: Patch 4/6 randomize the stack pointer)

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Roman Zippel wrote:
 Hi,
 
 On Thu, 3 Feb 2005, Peter Busser wrote:
 
 
- What happens when you run existing commercial applications which have not 
been compiled using GCC.
 
 
From http://pax.grsecurity.net/docs/pax.txt:
 
The goal of the PaX project is to research various defense mechanisms
against the exploitation of software bugs that give an attacker arbitrary
read/write access to the attacked task's address space. 
 
 Could you please explain how PaX makes such applications secure?
 

I wrote an easy-to-chew article[1] about PaX on Wikipedia, although
looking back at it I think there may be some erratta in the ASLR
concept; I think the mmap() base is randomized, but I'm not sure now if
the actual base of each mmap() call is individually randomized as shown
in my diagrams.  I'm also no longer sure where I got the notion that the
heap/.bss/data segments are the same entity, and I'll have to check on that.

Nevertheless, it's basically accurate, in the same way that saying you
have a gameboy advance SP when you just have a gameboy advance is
basically accurate.

[1] http://en.wikipedia.org/wiki/PaX

 bye, Roman
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB7PlhDd4aOud5P8RAr+pAKCCcbqLuG7OQzZlJrd5UdsA3NooUgCePXnp
D+xS98fWm9MVEBZpB+pIrTY=
=r+20
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Sabotaged PaXtest

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
 On Mon, 2005-01-31 at 13:57 +0100, Peter Busser wrote:
 
Hi!

[...]

 the paxtest 0.9.6 that John Moser mailed to this list had this gem in
 it:
 @@ -39,8 +42,6 @@
  */
 int paxtest_mode = 1;
 
 +   /* Dummy nested function */
 +   void dummy(void) {}
 
 mode = getenv( PAXTEST_MODE );
 if( mode == NULL ) {
 
 
 which is clearly there with the only possible function of sabotaging the
 automatic PT_GNU_STACK setting by the toolchain (which btw is not fedora
 specific but happens by all new enough (3.3 or later) gcc compilers on
 all distros) since that requires an executable stack.

I'll say this AGAIN:  I execstack -c'd the directory after the first
borked up test and stuff randomly failed.  After upgrading FC3's kernel
though I got predictable results.


- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB7T0hDd4aOud5P8RAk7XAJ4oYgr2ySTdg+y80f6pBasO+x8ttACfdjYx
++dQRSTGzTGP/Vsp2KZ6YHU=
=jp9U
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Filesystem linking protections

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
 * Lorenzo Hernández García-Hierro ([EMAIL PROTECTED]) wrote:
 
This patch adds two checks to do_follow_link() and sys_link(), for
prevent users to follow (untrusted) symlinks owned by other users in
world-writable +t directories (i.e. /tmp), unless the owner of the
symlink is the owner of the directory, users will also not be able to
hardlink to files they do not own.

The direct advantage of this pretty simple patch is that /tmp races will
be prevented.
 
 
 The disadvantage is that it can break things and places policy in the
 kernel.
 

It can break things, yes.  For example, programs which have and use two
separate FS UIDs at the same time, or which attempt to make hardlinks to
files they don't own without CAP_FOWNER or root (should this just be
CAP_FOWNER?  Is root now irrelavent?).

Hang on, when do any programs have 2 FS UIDs at the same time. . . .

I've yet to see this break anything on Ubuntu or Gentoo; Brad Spengler
claims this breaks nothing on Debian.  On the other hand, this could
potentially squash the second most prevalent security bug.

 thanks,
 -chris

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB8S0hDd4aOud5P8RAvYSAJ9zcGArfbC6i5uM1JW4ZHdELriUzACeOH/q
5ndpSdjporfnFAMK1OrMASE=
=XjWB
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Filesystem linking protections

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
 * John Richard Moser ([EMAIL PROTECTED]) wrote:
 
I've yet to see this break anything on Ubuntu or Gentoo; Brad Spengler
claims this breaks nothing on Debian.  On the other hand, this could
potentially squash the second most prevalent security bug.
 
 
 Yes I know, I've worked on distro with it as well in the past.  And it
 has broken atd and courier in the past.  This is something that also
 can be done in userspace using sane subdirs in +t world writable dirs,
 or O_EXCL so there's work to be done in userspace.
 

Yes, mkdtemp() and mkstemp().

Of course we can't always rely on programmers to get it right, so the
idea here is to make sure we ask broken code to behave nicely, and stab
it in the face if it doesn't.  Please try to examine this in that scope.

 thanks,
 -chris

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCB+vThDd4aOud5P8RAssCAJ9L7Cf5pnvI8GdKs1P4cpM2lJvtYACZAXee
a5kkPkxXm9YK0DFSfvDd6fQ=
=00DK
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH] Filesystem linking protections

2005-02-07 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Chris Wright wrote:
 * John Richard Moser ([EMAIL PROTECTED]) wrote:
 
Yes, mkdtemp() and mkstemp().

Of course we can't always rely on programmers to get it right, so the
idea here is to make sure we ask broken code to behave nicely, and stab
it in the face if it doesn't.  Please try to examine this in that scope.
 
 
 It's fine for hardened distro.  But still inappropriate for mainline.
 

Perhaps in mainline as an option?  The [*] notations next to things are
really nice, they let you turn kernel stuff on and off :)  It's
appropriate for mainline to support added security isn't it?  I think
following the path of supporting-but-not-forcing is the best route,
because it encourages people to account for systems which may take
advantage of such options, and thus leads to a software base in which
it's quite sane to actually enable those options globally.

That's just how I think though.

 thanks,
 -chris

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCCB+GhDd4aOud5P8RAlD9AJ45JTY20WY6qHe0h0ZIcFasgxJDtACbB1aB
i4hytMAy6Cs1AUNXC296JOk=
=oLVs
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: msdos/vfat defaults are annoying

2005-02-06 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Christoph Hellwig wrote:
> On Sun, Feb 06, 2005 at 12:33:43AM -0500, John Richard Moser wrote:
> 
>>I dunno.  I can never understand the innards of the kernel devs' minds.
> 
> 
> filesystem detection isn't handled at the kerne level.
> 

o_o

. . . then I shall bug [EMAIL PROTECTED]
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCBkXqhDd4aOud5P8RAt3CAJ4uOZFfhAIDlF8crE4SXfLSoDLDrACfSDWG
LSaHLkMAGdKq8TuNFIHNQXM=
=SbC+
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: msdos/vfat defaults are annoying

2005-02-06 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Christoph Hellwig wrote:
 On Sun, Feb 06, 2005 at 12:33:43AM -0500, John Richard Moser wrote:
 
I dunno.  I can never understand the innards of the kernel devs' minds.
 
 
 filesystem detection isn't handled at the kerne level.
 

o_o

. . . then I shall bug [EMAIL PROTECTED]
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCBkXqhDd4aOud5P8RAt3CAJ4uOZFfhAIDlF8crE4SXfLSoDLDrACfSDWG
LSaHLkMAGdKq8TuNFIHNQXM=
=SbC+
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


msdos/vfat defaults are annoying

2005-02-05 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

So I've noticed, again, much annoyed, that if I rely on -t auto,
horrible horrible things happen.

I have had floppies and compact flash cards that I've done mkfs.vfat to
make fat32 filesystems on (not fat16), and mounting them brings the
thing on as msdos by default (autodetect).  Furthermore, I build msdos
out, and mount says the msdos FS isn't supported.  In either case I need
to use -t vfat.

Vfat is much more common and should be backwards compatible with msdos.
  When there's a ton of foo~1 files around after mounting, something's
wrong.

Shouldn't vfat be the automatic default?  Or at least, if only vfat and
not msdos is available, use vfat.  For that matter, can msdos and vfat
be collapsed?  As I recall, the difference is that vfat makes more
inodes to store long file names, one for each 13 characters (in reverse?)

I dunno.  I can never understand the innards of the kernel devs' minds.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCBaw3hDd4aOud5P8RAtBGAJsE8I1510nLSNqM6MRwPFGnl9l2UQCfSaGy
HPGDuNVPvMZq8nkI34DlfPI=
=uNx9
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


msdos/vfat defaults are annoying

2005-02-05 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

So I've noticed, again, much annoyed, that if I rely on -t auto,
horrible horrible things happen.

I have had floppies and compact flash cards that I've done mkfs.vfat to
make fat32 filesystems on (not fat16), and mounting them brings the
thing on as msdos by default (autodetect).  Furthermore, I build msdos
out, and mount says the msdos FS isn't supported.  In either case I need
to use -t vfat.

Vfat is much more common and should be backwards compatible with msdos.
  When there's a ton of foo~1 files around after mounting, something's
wrong.

Shouldn't vfat be the automatic default?  Or at least, if only vfat and
not msdos is available, use vfat.  For that matter, can msdos and vfat
be collapsed?  As I recall, the difference is that vfat makes more
inodes to store long file names, one for each 13 characters (in reverse?)

I dunno.  I can never understand the innards of the kernel devs' minds.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCBaw3hDd4aOud5P8RAtBGAJsE8I1510nLSNqM6MRwPFGnl9l2UQCfSaGy
HPGDuNVPvMZq8nkI34DlfPI=
=uNx9
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Patch 4/6 randomize the stack pointer

2005-01-31 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



[EMAIL PROTECTED] wrote:
>>Why not compromise, if possible?  256M of randomization, but move the
>>split up to 3.5/0.5 gig, if possible.  I seem to recall seeing an option
>>(though I think it was UML) to do 3.5/0.5 before; and I'm used to "a
>>little worse" meaning "microbenches say it's worse, but you won't notice
>>it," so perhaps this would be a good compromise.  How well tuned can
>>3G/1G be?  Come on, 1G is just a big friggin' even number.
> 
> 
> Ah, grasshopper, so much you have to learn...

I'm always learning, that's why I'm not stupid.  I hate dumb people who
just sit around and go "what?"  "OK it works like--"  "shut up I don't
care!  . . . . wut now?"

> In particular, prople these days are more likely to want to move the
> split DOWN rather than UP.
> 
> First point: it is important that the split happens at an "even" boundary
> for the highest-level page table.  This makes it as simple as possible to
> copy the shared global pages into each process' page tables.
> 
> On typical x86, each table is 1024 entries long, so the top table maps
> 4G/1024 = 4M sections.
> 
> However, with PAE (Physical Address Extensions), a 32-bit page table
> entry is no longer enough to hold the 36-bit physical address.  Instead,
> the entries are 64 bits long, so only 512 fit into a page.  With a
> 4K page and 18 more bits from page tables, two levels will map only
> 30 bits of the 32-bit virtual address space.  So Intel added a small,
> 4-entry third-level page table.
> 

Interesting.

> With PAE, you are indeed limited to 1G boundaries.  (Unless you want to
> seriously overhaul mm setup and teardown.)
> 

Yii x.x

> 
> Secondly, remember that, unless you want to pay a performance penalty
> for enabling one of the highmem options, you have to fit ALL of physical
> memory, PLUS memory-mapped IO (say around 128M) into the kernel's portion
> of the address space.  512M of kernel space isn't enough unless you have
> less than 512M (like 384M) of memory to keep track of.
> 
> That is getting less common, *especially* on servers.  (Which are
> presumably an important target audience for buffer overflow defenses.)
> 

Semantics, you can skip this but here's my understanding about the
situation:

ASLR is a code injection/ret2libc exploit defense, not a buffer overflow
bug defense.  True buffer overflow defense occurs in userspace.

ASLR and proper NX protections provide a more wide-ranged defense which
stops format string and heap/stack buffer overflow based attacks (and
possibly others) from using ret2libc or code injection exploits.

Stack based buffer overflow protection is available in userspace and
actually prevents some things not protected by kernel-level protections.
 ProPolice for example protects local pointers and local variables;
sometimes changing the contents of a variable (say a filename, or an
integer containing an encoded IP address) can give you a quick and dirty
information leak.

All defenses are important everywhere, not just on servers; although
they're probably less important on internal servers behind your
firewall/IDS/security net/DMZ because you need an insider to access
those, which often means someone skilled enough or with enough info
(passwords, etc) to bypass your security anyway (not that you shouldn't
make the best effort possible to stop them).

Some people don't agree with me (in fact I'm probably alone in this line
of thinking), but I'm of the persuasion that the best protection is most
important on the desktop; and second but roughly equal on exposed
servers (web/ftp servers).

This is because attackers can use desktops as drones, and they're very
exposed due to web browsers, ftp clients, IM clients, and P2P apps, not
to mention tons of other stuff that comes in contact with foreign data.
 Web servers have a few choice daemons, so there's a much smaller
codebase touching things that come from outside, meaning less of a
chance of an open bug touching the outside world.

Anyway, that's my rant :)

> 
> Indeed, if you have a lot of RAM and you don't have a big database that
> needs tons of virtual address space, it's usually worth moving the
> split DOWN.
> 

Or you have a big database that wants tons of virtual address space and
so you buy an amd64.  ;)

> 
> Now, what about the case where you have gobs of RAM and need a highmem
> option anyway?  Well, there's a limit to what you can use high mem for.
> Application memory and page cache, yes.  Kernel data structures, no.
> You can't use it for dcache or inodes or network sockets or page
> tables or the struct page array.
> 

o.o

> And the mem_map array of struct pages (on x86, it's 32 bytes per page,
> or 1/128 of physical memory; 32M for a 4G machine) is a fixed overhead
> that's subtracted before you even start.  Fully-populated 64G x86
> machines need 512M of mem_map, and the remaining space isn't enough to
> really run well in.
> 
> If you crunch kernel lowmem too tightly, that 

Re: Patch 4/6 randomize the stack pointer

2005-01-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Christoph Hellwig wrote:
> On Sat, Jan 29, 2005 at 12:49:05PM -0500, John Richard Moser wrote:
> 
>>>The ideas in IBM's ProPolice changes are good and worth
>>>implementing, but the current implementation is bad.
>>>
>>
>>Lies.  I've read the paper on the current implementation, it's
>>definitely good.  It only operates on C/C++ code though, but that's the
>>scope of it.
> 
> 
> Yeah, I guess your extensive compiler internals experience and knowledge
> of gcc internals weights a lot more than the opinion of the gcc team..
> 

I read "implementation" as "the way it's implemented," not as "the
quality of the code."

Did I miss the target?  *Aims in the other direction then?*

> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+9GthDd4aOud5P8RAm/FAJwNvbrxPP8fcJmMM//vcYL10nMXTACggi57
jOKfS4FU1sdPL7AjKRgMmBg=
=igR0
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Patch 4/6 randomize the stack pointer

2005-01-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Jakub Jelinek wrote:
> On Sat, Jan 29, 2005 at 01:31:46AM -0500, John Richard Moser wrote:
> 
>>Finally, although an NX stack is nice, you should probably take into
>>account IBM's stack smash protector, ProPolice.  Any attack that can
>>evade SSP reliably can evade an NX stack; but ProPolice protects from
>>other overflows.  Now I'm sure RH is over there inventing something that
>>detects buffer overflows at compile time and misses or warns about the
>>ones it can't identify:
>>
>>if (strlen(a) > 4)
>>  a[5] = '\0';
>>foo(a);
>>
>>void foo(char *a) {
>>   char b[5];
>>   strcpy(b,a);
>>}
>>
>>This code is safe, but you can't tell from looking at foo().  You don't
>>get a look at every other object being compiled against this one that
>>may call foo() either.  So compile time buffer overflow detection is a
>>best-effort at best.
> 
> 
> If strlen(a) > 4 above, then -D_FORTIFY_SOURCE={1,2} compiled program
> will be terminated in the strcpy call.  At compile time it computes
> that the strcpy call can fill in at most 5 bytes and if it copies more,
> then it terminates.

And somehow you check every operation like this with less overhead than
propolice?

> 
> 
>>ProPolice protects local variables with 0 overhead; passed arguments
>>with a few instructions; and the return pointer and stack frame pointer
>>with a couple instructions.  At runtime.  Want to impress me?  Actually
>>deploy ProPolice instead of showing up 3 years from now waving around
>>your own patch that you wrote that half-impliments half of it.  If you
>>want "something better," it's GPL, so grab it and start hacking.
> 
> 
> __builtin_object_size () checking/-D_FORTIFY_SOURCE=n changes are (partly)
> orthogonal to ProPolice.  There are exploits prevented by
> -D_FORTIFY_SOURCE={1,2} checking and not ProPolice and vice versa.

So a belt-and-suspenders approach is better.

> Things that the former protects and the latter does not are e.g.
> some non-automatic buffer overflows or heap overflows, some format string
> vulnerabilities and for automatic variables e.g. those that don't
> overflow into another function's frame, but just overwrite other local
> variables in the same function.  ProPolice on the other side will detect
> stack overflows that overflow into another function's frame,

or past the top of any buffer by at most 2 ints (I didn't check with 1
int or 1 char when I wrote my regression suite), definitely before it
hits the SFP and return pointer

> even if they
> aren't done through string operations (, s*printf, gets, etc.)
> or if the compiler can't figure out what certain arguments to these
> functions points to (and where) at compile time.
> 
> The ideas in IBM's ProPolice changes are good and worth
> implementing, but the current implementation is bad.
> 

Lies.  I've read the paper on the current implementation, it's
definitely good.  It only operates on C/C++ code though, but that's the
scope of it.

> FYI, you can find some details about -D_FORTIFY_SOURCE=n in
> http://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html
> 
>   Jakub
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+8yPhDd4aOud5P8RAsekAJsGklzrgWi7ymrRmfhXpqv2LjB//gCeNBDy
8sCZBhtzy6l6L/WDjQpMq6A=
=4/Dz
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Patch 4/6 randomize the stack pointer

2005-01-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
> On Sat, 2005-01-29 at 11:21 -0500, John Richard Moser wrote:
> 
>>-BEGIN PGP SIGNED MESSAGE-
>>Hash: SHA1
>>
>>
>>
>>Arjan van de Ven wrote:
>>
>>>>I actually just tried to paxtest a fresh Fedora Core 3, unadultered,
>>>>that I installed, and it FAILED every test.  After a while, spender
>>>>reminded me about PT_GNU_STACK.  It failed everything but the Executable
>>>>Stack test after execstack -c *.  The randomization tests gave
>>>>13(heap-etexec), 16(heap-etdyn), 17(stack), and none for main exec
>>>>(etexec,et_dyn) or shared library randomization.
>>>
>>>
>>>because you ran prelink.
>>>and you did not compile paxtest with -fPIE -pie to make it a PIE
>>>executable.
>>>
> 
> 
> what I get is
> 
> Executable anonymous mapping : Killed
> Executable bss   : Killed
> Executable data  : Vulnerable
> Executable heap  : Killed
> Executable stack : Killed
> Executable anonymous mapping (mprotect)  : Vulnerable
> Executable bss (mprotect): Vulnerable
> Executable data (mprotect)   : Vulnerable
> Executable heap (mprotect)   : Vulnerable
> Executable shared library bss (mprotect) : Vulnerable
> Executable shared library data (mprotect): Vulnerable
> Executable stack (mprotect)  : Vulnerable
> Anonymous mapping randomisation test : No randomisation
> Heap randomisation test (ET_EXEC): 13 bits (guessed)
> Heap randomisation test (ET_DYN) : 13 bits (guessed)
> Main executable randomisation (ET_EXEC)  : 12 bits (guessed)
> Main executable randomisation (ET_DYN)   : 12 bits (guessed)
> Shared library randomisation test: 12 bits (guessed)
> Stack randomisation test (SEGMEXEC)  : 17 bits (guessed)
> Stack randomisation test (PAGEEXEC)  : 17 bits (guessed)
> Return to function (strcpy)  : paxtest: bad luck, try
> different compiler options.
> Return to function (strcpy, RANDEXEC): paxtest: bad luck, try
> different compiler options.
> Return to function (memcpy)  : Vulnerable
> Return to function (memcpy, RANDEXEC): Vulnerable
> Executable shared library bss: Killed
> Executable shared library data   : Killed
> Writable text segments   : Vulnerable
> 
> 
> I'm not entirely happy yet (it shows a bug in mmap randomisation) but
> it's way better than what you get in your tests (this is the desabotaged
> 0.9.6 version fwiw)
> 

I used 0.9.6 too, it had a slight bug in the randomization test
(getmain.c), which I pointed out in another post.

void foo( int unused )
{
printf( "%p\n", __builtin_return_address(0) );
//printf( "0x%08x\n", ((unsigned long*))[-1] );
}

I'm curious as to what the hell you're doing to get these results.  Exec
Shield came with the sysctl sys/kernel/exec-shield = 1 and
sys/kernel/exec-shield-randomize = 1.  I tried exec-shield = 0, 1, 2,
and 3 and couldn't get anything but the stack to kill on a Barton cored
32 bit athlon xp.

The tests I did were on a Fedora Core 3 i net-installed last night, no
adulteration.  Whatever black magic you're doing, it's not working here.
> 
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+8H/hDd4aOud5P8RAlIEAJkBwhIxdrXZ+jNz46oRg1SoSPmOHQCgiWfJ
HxzCBB43i6iLLhli5boKzoM=
=etT7
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Patch 4/6 randomize the stack pointer

2005-01-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1





Arjan van de Ven wrote:
> On Sat, 2005-01-29 at 11:21 -0500, John Richard Moser wrote:
> 
>>-BEGIN PGP SIGNED MESSAGE-
> 
> 
>>These are the only places mprotect() is mentioned; a visual scan
>>confirms no trickery:
>>
>>if( fork() == 0 ) {
>>/* Perform a dirty (but not unrealistic) trick to circumvent
>> * the kernel protection.
>> */
>>if( paxtest_mode == 1 ) {
>>pthread_t thread;
>>pthread_create(, NULL, test_thread, dummy);
>>doit();
>>pthread_kill(thread, SIGTERM);
>>} else {
> 
> 
>>So, there you have it.  These tests do not intentionally kill
>>exec-shield based on its known issue with tracking the upper limit of
>>the code segment.
> 
> 
> 
> here they do.
> dummy is a local NESTED function, which causes the stack to *correctly*
> be marked executable, due to the need of trampolines. 
> That disables execshield for any tests that use dummy.o, which most of
> them are.
> 

Only in "Blackhat" mode.  I ran in Kiddie and Blackhat mode, and my
second batch of tests (tests.txt, my errata) was run after execstack -c.

> 

[EMAIL PROTECTED]:/mnt/redhat/root/paxtest-0.9.6 # strace ./execstack 2>&1 |
grep mprotect
mprotect(0x20822000, 4096, PROT_NONE)   = 0
[EMAIL PROTECTED]:/mnt/redhat/root/paxtest-0.9.6 # strace ./execstack 2>&1 |
grep EXEC
old_mmap(NULL, 64996, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x265d9000
old_mmap(NULL, 1232332, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x265e9000
mmap2(NULL, 8392704, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x26716000

I killed the fork() line and straced it.
0x26716000 is only ~600 megs up,  I find my stack at ~1.5G under
segmexec, I'd guess it'd be at ~3G under normal things like execshield.
 You know what *looks*

getstack1:  0xfefcead7
getstack1:  0xfefe9947
getstack1:  0xfeedd4f7
getstack1:  0xfefe6e37
getstack1:  0xfee412b7
getstack1:  0xfee71737

Yeah it's pretty high under exec shield.

none of these are doing ANYTHING weird (grepping for EXEC and scanning
for PROT_EXEC and related addresses), aside from the normal mapping of
libraries by the system, which is probably what's killing Exec Shield's
anonymous mapping, heap, data, bss, library data, library bss. . .

To put it bluntly, you're wrong.  The tests are valid.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+8DuhDd4aOud5P8RAscYAKCErG3KB5M9gMwZ1BRqTDBKyYYLrACfcYeb
0ptJYTE+uUx0QYSHmKXWlsA=
=PL8+
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Patch 4/6 randomize the stack pointer

2005-01-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1





Arjan van de Ven wrote:
 On Sat, 2005-01-29 at 11:21 -0500, John Richard Moser wrote:
 
-BEGIN PGP SIGNED MESSAGE-
 
 
These are the only places mprotect() is mentioned; a visual scan
confirms no trickery:

if( fork() == 0 ) {
/* Perform a dirty (but not unrealistic) trick to circumvent
 * the kernel protection.
 */
if( paxtest_mode == 1 ) {
pthread_t thread;
pthread_create(thread, NULL, test_thread, dummy);
doit();
pthread_kill(thread, SIGTERM);
} else {
 
 
So, there you have it.  These tests do not intentionally kill
exec-shield based on its known issue with tracking the upper limit of
the code segment.
 
 
 
 here they do.
 dummy is a local NESTED function, which causes the stack to *correctly*
 be marked executable, due to the need of trampolines. 
 That disables execshield for any tests that use dummy.o, which most of
 them are.
 

Only in Blackhat mode.  I ran in Kiddie and Blackhat mode, and my
second batch of tests (tests.txt, my errata) was run after execstack -c.

 

[EMAIL PROTECTED]:/mnt/redhat/root/paxtest-0.9.6 # strace ./execstack 21 |
grep mprotect
mprotect(0x20822000, 4096, PROT_NONE)   = 0
[EMAIL PROTECTED]:/mnt/redhat/root/paxtest-0.9.6 # strace ./execstack 21 |
grep EXEC
old_mmap(NULL, 64996, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x265d9000
old_mmap(NULL, 1232332, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x265e9000
mmap2(NULL, 8392704, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x26716000

I killed the fork() line and straced it.
0x26716000 is only ~600 megs up,  I find my stack at ~1.5G under
segmexec, I'd guess it'd be at ~3G under normal things like execshield.
 You know what *looks*

getstack1:  0xfefcead7
getstack1:  0xfefe9947
getstack1:  0xfeedd4f7
getstack1:  0xfefe6e37
getstack1:  0xfee412b7
getstack1:  0xfee71737

Yeah it's pretty high under exec shield.

none of these are doing ANYTHING weird (grepping for EXEC and scanning
for PROT_EXEC and related addresses), aside from the normal mapping of
libraries by the system, which is probably what's killing Exec Shield's
anonymous mapping, heap, data, bss, library data, library bss. . .

To put it bluntly, you're wrong.  The tests are valid.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+8DuhDd4aOud5P8RAscYAKCErG3KB5M9gMwZ1BRqTDBKyYYLrACfcYeb
0ptJYTE+uUx0QYSHmKXWlsA=
=PL8+
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: Patch 4/6 randomize the stack pointer

2005-01-29 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:
 On Sat, 2005-01-29 at 11:21 -0500, John Richard Moser wrote:
 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Arjan van de Ven wrote:

I actually just tried to paxtest a fresh Fedora Core 3, unadultered,
that I installed, and it FAILED every test.  After a while, spender
reminded me about PT_GNU_STACK.  It failed everything but the Executable
Stack test after execstack -c *.  The randomization tests gave
13(heap-etexec), 16(heap-etdyn), 17(stack), and none for main exec
(etexec,et_dyn) or shared library randomization.


because you ran prelink.
and you did not compile paxtest with -fPIE -pie to make it a PIE
executable.

 
 
 what I get is
 
 Executable anonymous mapping : Killed
 Executable bss   : Killed
 Executable data  : Vulnerable
 Executable heap  : Killed
 Executable stack : Killed
 Executable anonymous mapping (mprotect)  : Vulnerable
 Executable bss (mprotect): Vulnerable
 Executable data (mprotect)   : Vulnerable
 Executable heap (mprotect)   : Vulnerable
 Executable shared library bss (mprotect) : Vulnerable
 Executable shared library data (mprotect): Vulnerable
 Executable stack (mprotect)  : Vulnerable
 Anonymous mapping randomisation test : No randomisation
 Heap randomisation test (ET_EXEC): 13 bits (guessed)
 Heap randomisation test (ET_DYN) : 13 bits (guessed)
 Main executable randomisation (ET_EXEC)  : 12 bits (guessed)
 Main executable randomisation (ET_DYN)   : 12 bits (guessed)
 Shared library randomisation test: 12 bits (guessed)
 Stack randomisation test (SEGMEXEC)  : 17 bits (guessed)
 Stack randomisation test (PAGEEXEC)  : 17 bits (guessed)
 Return to function (strcpy)  : paxtest: bad luck, try
 different compiler options.
 Return to function (strcpy, RANDEXEC): paxtest: bad luck, try
 different compiler options.
 Return to function (memcpy)  : Vulnerable
 Return to function (memcpy, RANDEXEC): Vulnerable
 Executable shared library bss: Killed
 Executable shared library data   : Killed
 Writable text segments   : Vulnerable
 
 
 I'm not entirely happy yet (it shows a bug in mmap randomisation) but
 it's way better than what you get in your tests (this is the desabotaged
 0.9.6 version fwiw)
 

I used 0.9.6 too, it had a slight bug in the randomization test
(getmain.c), which I pointed out in another post.

void foo( int unused )
{
printf( %p\n, __builtin_return_address(0) );
//printf( 0x%08x\n, ((unsigned long*)unused)[-1] );
}

I'm curious as to what the hell you're doing to get these results.  Exec
Shield came with the sysctl sys/kernel/exec-shield = 1 and
sys/kernel/exec-shield-randomize = 1.  I tried exec-shield = 0, 1, 2,
and 3 and couldn't get anything but the stack to kill on a Barton cored
32 bit athlon xp.

The tests I did were on a Fedora Core 3 i net-installed last night, no
adulteration.  Whatever black magic you're doing, it's not working here.
 
 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+8H/hDd4aOud5P8RAlIEAJkBwhIxdrXZ+jNz46oRg1SoSPmOHQCgiWfJ
HxzCBB43i6iLLhli5boKzoM=
=etT7
-END PGP SIGNATURE-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


  1   2   3   >