Re: Integrating "safe" languages into OpenBSD?

[Max Hayden Chiz]

On Sun, Dec 3, 2017 at 7:59 PM, Nicolas Schmidt 
wrote:
> Hi,
>
> I recently watched a recording of Theo's talk on pledge at
> EuroBSDCon 2017, in which the question of memory-safe
> languages and their practical usefulness came up. Specifically,
> someone in the audience criticized the approach taken by
> OpenBSD, which (as I understand) accepts that all software
> is broken and mitigates the damage caused
> by various classes of exploits through techniques like ASLR, and
> suggested that instead one should stick to "memory safe languages"
> to avoid these exploits altogether.
>
> As a response to this, Theo asked rhetorically "Where's ls, where's
> cat, where's grep, and where's sort?", implying that noone so far
> bothered to write implementations of even the basic unix utilities
> in such a language.
>
> This brings me to the question, what if someone actually bothered?
> Under what conditions would you consider replacing one of the
> current C implementations with an implementation written in
> another, "safer" language? Note that with Cgrep and haskell-ls,
> there do in fact exist implementations/analogues of two of the
> mentioned utilities in a memory safe language (Haskell).

Sorry for the thread res, but I wanted to add something to this discussion
and didn't have a chance until now.

There's a big misconception here about the point of "safe" languages. Safe
languages are *not* a security feature.

Let's take Rust as an example. Neither of the "two remote holes" would have
been caught by Rust's features. Rust doesn't protect against integer
overflow errors. *At best*, if SSH had been written in Rust, it would have
turned the remote hole into a failed bounds check and a panic. So instead
of a remote hole, we'd have had a denial of service attack. More
realistically, if the code in question had been part of an "unsafe" block,
the same security breach would have occurred.

Similarly, Rust doesn't prevent SSL from being written with spaghetti code
and obfuscating the fact that it was taking a user provided number to use
as a bounds check on a crappy custom memory structure. The same limitations
holds for most of the security bugs recorded in the CERT C standard. The
ones that can be automatically checked can be detected by C programming
tools. The ones that can't be automatically checked don't magically go away
when you use Rust.

More fundamentally, even if you have an application written in a safe
language, you have no way of examining the binary and knowing that those
safety guarantees still hold. In an embedded system, you can do verified
programming, use a verified compiler, and then make sure that the binary
can't be modified afterward by using read-only memory. (And you could still
have a problem due to a hardware bug.) None of that stuff is available or
practical for OpenBSD. Even worse, if your "safe" system has an elaborate
run-time, like Java or JavaScript, that run-time itself becomes an attack
surface. (Just look at how many security issues these things have had over
the years.)

Moreover, *good* C code is about as safe as it can be. CCured was an
application that (in part) could automatically prove that large aspects of
C programs were safe. In practice about 90% of the code was safe as-is.
Another 9% was safe if it had a bounds check (the system couldn't verify
this). And only 1% needed special handling. If you translate the code into
Rust, you are just going to end up proving that the safe 90% is safe and
the remaining 10% will live in an "unsafe" block because of what it does.

Contra the questioner's assumption, the real point of safe languages is
that they enhance programmer productivity by handling certain repetitive
issues automatically and by allowing for easier use of higher-level
language constructs. Done well, a safe language makes your code more
compact, faster to write, and easier to reason about.

But stated that way, it's obvious that OpenBSD won't benefit from them. For
one, an operating system is inherently low-level and doesn't have much room
for higher-level constructs. For another, productivity enhancements only
count when you are writing code from scratch. Re-writing the 3M+ lines of
OpenBSD's kernel code would be an obvious waste. (Not to mention the rest
of the system.)

Don't get me wrong, I think Rust is worth using in a new project for a
sufficiently complex application. But "use the right tool for the job"
applies here. Most of what an OS does (be it the kernel, ld.so, or the
various state machines inside of priv-sep'ed services) is "low-level" and
benefits from neither the additional abstraction nor from the safety
guarantees that you'd just have to disable or work around.

In contrast to mere memory-safety, how to do *low-level* programming in a
verifiable, bug-free way is still an active area of language research.
There are some promising developments out there, and rather than rewriting
Unix tools in Rust or making yet another 

Re: Reproducible system hang in 4.9 (ral in hostap)

[Max Hayden Chiz]

Update:  I tried a brand new Linksys WMP600N (same chipset) and got
the same error.  So I've now ruled out the card being an issue.

If there is anything else I should try please let me know, but as of
now, I'm all out of ideas.

Also, my offer to send one of these cards to the appropriate develop
stands.  Please let me know if it is needed.

Re: Reproducible system hang in 4.9 (ral in hostap)

[Max Hayden Chiz]

Update2:  It turns out that the problem I had back in 4.6 has not gone
away, but that it just takes much more traffic to cause it.  ~5
minutes of FTP across the bridge while ral is in hostAP will cause the
hang.

On Mon, Jun 20, 2011 at 5:50 PM, Max Hayden Chiz max.c...@gmail.com wrote:
 Update: B I tried a brand new Linksys WMP600N (same chipset) and got
 the same error. B So I've now ruled out the card being an issue.

 If there is anything else I should try please let me know, but as of
 now, I'm all out of ideas.

 Also, my offer to send one of these cards to the appropriate develop
 stands. B Please let me know if it is needed.

Reproducible system hang in 4.9 (ral in hostap)

[Max Hayden Chiz]

I have a Soekris net 5501 and a ral RT2860/2850 PCI card (dmesg
below). B While trying to use the system for an access point, I
discovered a way to consistently cause the system to hang. B (Note:
This is *different* from a similar, now fixed, system hang that I
reported circa 4.6-beta).

To cause the system to hang, put ral0 into HostAP mode, connect a computer
with an FTP server to one of the Ethernet connections, then connect
another computer to the wireless access point and attempt to download
a largish file over FTP. B The system will hang within 15 seconds from
the start of the transfer.

Using FTP is not required, any heavy traffic will cause a similar
result, but the hang only occurs when ral is in hostAP mode.

B/c the previous hang I discovered has since been fixed, I do not
think this is a hardware problem. B I'd like to work with someone who
knows more about this to try and pin down the source of the problem so
that it too can be fixed. B If this problem is specific to this ral
model, I have no problems ordering a different, working one and giving
this one to the appropriate developer so that it can be debugged from
there. B Similarly, if there is a concern that this *is* a hardware
problem, I'll be happy to purchase a known good ral or similar wifi
card and test it under similar conditions (provided someone can direct
me to one).

Also:
*Using or not using wpa doesn't change anything
*Using 11a or 11g does not change anything
*The power supply is not an issue; I'm using the largest one from
Soekris and have tried half a dozen different ones. B All get the same
hang.
*I couldn't get the console to give me ddb access after the hang, so I
tried sending it a break before and then c. B I get that far, but
when the system hangs ddb is unresponsive.

For reference, the hang from 4.6-beta was a caused by sending traffic
between two of the vr ethernet ports while ral was up and in HostAP
mode. B That hang no longer exists, but if someone wants to look at it
for comparison, my emails are
http://marc.info/?l=openbsd-miscm=124685949929721w=2 and
http://marc.info/?l=openbsd-miscm=124697898624989w=2

Thank you for any help you can provide.

--Max H. Chiz

Here is the dmesg:
OpenBSD 4.9 (GENERIC) #671: Wed Mar B 2 07:09:00 MST 2011
B  B dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Geode(TM) Integrated Processor by AMD PCS (AuthenticAMD 586-class) 500
M
Hz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX
real mem B = 536440832 (511MB)
avail mem = 517533696 (493MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 20/80/26, BIOS32 rev. 0 @ 0xfac40
pcibios0 at bios0: rev 2.0 @ 0xf/0x1
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable.
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xc8000/0xa800
cpu0 at mainbus0: (uniprocessor)
amdmsr0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
io address conflict 0x6100/0x100
io address conflict 0x6200/0x200
pchb0 at pci0 dev 1 function 0 AMD Geode LX rev 0x33
glxsb0 at pci0 dev 1 function 2 AMD Geode LX Crypto rev 0x00: RNG AES
vr0 at pci0 dev 6 function 0 VIA VT6105M RhineIII rev 0x96: irq 11, address
00
:00:24:cc:10:7c
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063,
model 0x0034
vr1 at pci0 dev 7 function 0 VIA VT6105M RhineIII rev 0x96: irq 5, address
00:
00:24:cc:10:7d
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063,
model 0x0034
vr2 at pci0 dev 8 function 0 VIA VT6105M RhineIII rev 0x96: irq 9, address
00:
00:24:cc:10:7e
ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063,
model 0x0034
vr3 at pci0 dev 9 function 0 VIA VT6105M RhineIII rev 0x96: irq 12, address
00
:00:24:cc:10:7f
ukphy3 at vr3 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063,
model 0x0034
ral0 at pci0 dev 14 function 0 Ralink RT2860 rev 0x00: irq 10, address
00:1e:e
5:e8:ea:c9
ral0: MAC/BBP RT2860 (rev 0x0103), RF RT2850 (MIMO 2T2R)
glxpcib0 at pci0 dev 20 function 0 AMD CS5536 ISA rev 0x03: rev 3, 32-bit
3579
545Hz timer, watchdog, gpio
gpio0 at glxpcib0: 32 pins
pciide0 at pci0 dev 20 function 2 AMD CS5536 IDE rev 0x01: DMA, channel 0
wire
d to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: SanDisk SDCFH2-002G
wd0: 4-sector PIO, LBA, 1953MB, 4001760 sectors
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pciide0: channel 1 ignored (disabled)
ohci0 at pci0 dev 21 function 0 AMD CS5536 USB rev 0x02: irq 15, version
1.0,
legacy support
ehci0 at pci0 dev 21 function 1 AMD CS5536 USB rev 0x02: irq 15
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 AMD EHCI root hub rev 2.00/1.00 addr 1
isa0 at glxpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: 

Soekris Net 5501 RT2860/2850 hangs in 4.6-beta

[Max Hayden Chiz]

I've been trying to use a Soekris Net5501 with a ral PCI card for a
wireless access point.  I'm running the most recent snapshot but I'm
still getting the weird hangs that people were reporting on misc@ back
in 4.4.

Doing heavy traffic through the Soekris (e.g. ftp a large file) will
consistently and predictably cause the system to hang; whether the
traffic is through ral0 or not doesn't matter.  If ral0 is up and in
hostAP mode, a large ftp transfer or similar traffic load will hang
the system. When the hang happens, even the console becomes
unresponsive (and thus I can't give you ddb output.)

I've tried everything I can to narrow down the problem, but I'm not
getting anywhere.  I would REALLY like to get this problem fixed; any
help would be greatly appreciated.  If the devs need something on my
end (testing, equipment, beer money), I'll be happy to accommodate,
just let me know.

Below is a dmesg and what I've tried so far.

*Using or not using wpa doesn't change anything
*The power supply is not an issue; I've tried half a dozen different
ones and the get the same hang.
*I couldn't make the hang happen in BSS mode or when ral0 was down.
*Making ral be part of a bridge doesn't change anything.
*I couldn't get the console to give me ddb access after the hang, so I
tried sending it a break before and then c.  I get that far, but
when the system hangs ddb is unresponsive.

OpenBSD 4.6-beta (GENERIC) #29: Sat Jun 27 18:37:05 MDT 2009
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Geode(TM) Integrated Processor by AMD PCS (AuthenticAMD
586-class) 500 MHz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX
real mem  = 536440832 (511MB)
avail mem = 509902848 (486MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 20/80/26, BIOS32 rev. 0 @ 0xfac40
pcibios0 at bios0: rev 2.0 @ 0xf/0x1
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable.
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xc8000/0xa800
cpu0 at mainbus0: (uniprocessor)
amdmsr0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
io address conflict 0x6100/0x100
io address conflict 0x6200/0x200
extent `pciio' (0x0 - 0x), flags=0
 0x6000 - 0x7fff
 0xe000 - 0xe00f
 0xe100 - 0xe4ff
 0x1 - 0x
extent `pcimem' (0x0 - 0x), flags=0
 0x0 - 0x9
 0xf - 0x1fff
 0xa000 - 0xa00043ff
 0xa001 - 0xa0021fff
 0xfff0 - 0x
pchb0 at pci0 dev 1 function 0 AMD Geode LX rev 0x33
glxsb0 at pci0 dev 1 function 2 AMD Geode LX Crypto rev 0x00: RNG AES
vr0 at pci0 dev 6 function 0 VIA VT6105M RhineIII rev 0x96: irq 11,
address 00:00:24:cc:10:7c
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
vr1 at pci0 dev 7 function 0 VIA VT6105M RhineIII rev 0x96: irq 5,
address 00:00:24:cc:10:7d
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
vr2 at pci0 dev 8 function 0 VIA VT6105M RhineIII rev 0x96: irq 9,
address 00:00:24:cc:10:7e
ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
vr3 at pci0 dev 9 function 0 VIA VT6105M RhineIII rev 0x96: irq 12,
address 00:00:24:cc:10:7f
ukphy3 at vr3 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
ral0 at pci0 dev 14 function 0 Ralink RT2860 rev 0x00: irq 10,
address 00:1e:e5:e8:ea:c9
ral0: MAC/BBP RT2860 (rev 0x0103), RF RT2850 (MIMO 2T2R)
glxpcib0 at pci0 dev 20 function 0 AMD CS5536 ISA rev 0x03: rev 0,
32-bit 3579545Hz timer, watchdog, gpio
gpio0 at glxpcib0: 32 pins
pciide0 at pci0 dev 20 function 2 AMD CS5536 IDE rev 0x01: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: SanDisk SDCFH2-002G
wd0: 4-sector PIO, LBA, 1953MB, 4001760 sectors
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pciide0: channel 1 ignored (disabled)
ohci0 at pci0 dev 21 function 0 AMD CS5536 USB rev 0x02: irq 15,
version 1.0, legacy support
ehci0 at pci0 dev 21 function 1 AMD CS5536 USB rev 0x02: irq 15
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 AMD EHCI root hub rev 2.00/1.00 addr 1
isa0 at glxpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
spkr0 at pcppi0
nsclpcsio0 at isa0 port 0x2e/2: NSC PC87366 rev 9: GPIO VLM TMS
gpio1 at nsclpcsio0: 29 pins
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 AMD OHCI root hub rev 1.00/1.00 addr 1
biomask e1c5 netmask ffe5 ttymask 
mtrr: K6-family MTRR support (2 registers)
softraid0 at root
root on wd0a swap on wd0b dump on wd0b
WARNING: / was not properly unmounted

Any Hardware Advice for Building an 802.11a AP?

[Max Hayden Chiz]

I'm planning on building an OpenBSD wireless access point.  I am
primarily interested in having 802.11a as there are already close to
30 2.4GHz APs in the vicinity.

I want something small, quiet, and low-power.  There aren't many
people using my network, but I would like something that can handle a
large SCP transfer or stream a video to a laptop without choking.  My
initial plan was to get a Soekris net5501-70 and a PCI ral card.
Because I'm in a high-noise urban area, I want to have the ability to
hook a decent high-gain/directional antenna.  I was specifically
looking at the Linksys WMP600N which Google says uses the rt2870
chipset.

A search of the misc@ archives, however, shows a whole lot of people
having stability issues with ral-based cards.  I couldn't find an
email recommending something better though.  What would list members
suggest I buy for wireless?  Does anyone have any suggestions on where
to buy quality antennas here in the US?

Thanks for any help you can provide.

--MHC

Re: Any Hardware Advice for Building an 802.11a AP?

[Max Hayden Chiz]

Several people have privately suggested that I consider an ath-based
card instead of the ral b/c of the ongoing problems.  Does anyone
disagree?  Does anyone know of a PCI-based ath card?  Or am I going to
have to get a mini-PCI card and a U.FL to coax connector to connect an
external antenna?

On Tue, Jun 16, 2009 at 1:06 AM, Max Hayden Chizmax.c...@gmail.com wrote:
 I'm planning on building an OpenBSD wireless access point. B I am
 primarily interested in having 802.11a as there are already close to
 30 2.4GHz APs in the vicinity.

 I want something small, quiet, and low-power. B There aren't many
 people using my network, but I would like something that can handle a
 large SCP transfer or stream a video to a laptop without choking. B My
 initial plan was to get a Soekris net5501-70 and a PCI ral card.
 Because I'm in a high-noise urban area, I want to have the ability to
 hook a decent high-gain/directional antenna. B I was specifically
 looking at the Linksys WMP600N which Google says uses the rt2870
 chipset.

 A search of the misc@ archives, however, shows a whole lot of people
 having stability issues with ral-based cards. B I couldn't find an
 email recommending something better though. B What would list members
 suggest I buy for wireless? B Does anyone have any suggestions on where
 to buy quality antennas here in the US?

 Thanks for any help you can provide.

 --MHC

dmesg for Samsung N110 with 4.5-current

[Max Hayden Chiz]

Haven't played around with it too much, but things generally seem to
work.  Obviously the Atheros AR5424 doesn't work (b/c it isn't
supported) and I haven't tried sound yet.  If anyone wants/needs me to
try something specific, let me know and I'll try to help.

MP Kernel (AFAIK, cpu1 is just hyperthreading):

OpenBSD 4.5-current (GENERIC.MP) #56: Tue Jun  9 14:51:31 MDT 2009
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Atom(TM) CPU N270 @ 1.60GHz (GenuineIntel 686-class) 1.60 GHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,EST,TM2,xTPR
real mem  = 1063677952 (1014MB)
avail mem = 1020129280 (972MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 04/13/09, BIOS32 rev. 0 @
0xfd5f0, SMBIOS rev. 2.5 @ 0xdf010 (36 entries)
bios0: vendor Phoenix Technologies Ltd. version
04D0.M002.20090413.KTW date 04/13/2009
bios0: SAMSUNG ELECTRONICS CO., LTD. NC10/N110
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP APIC HPET MCFG TCPA TMOR APIC BOOT SLIC SSDT SSDT SSDT
acpi0: wakeup devices HDEF(S4) PXS1(S4) PXS2(S4) PXS3(S4) USB1(S3)
USB2(S3) USB3(S3) USB4(S3) USB7(S3) SLT0(S4) SLT1(S4) SLT2(S4)
SLT3(S4) SLT6(S4) LANC(S4) PWRB(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 132MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Atom(TM) CPU N270 @ 1.60GHz (GenuineIntel 686-class) 1.60 GHz
cpu1: 
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,EST,TM2,xTPR
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins
ioapic0: misconfigured as apic 2, remapped to apid 1
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (RP01)
acpiprt2 at acpi0: bus -1 (RP02)
acpiprt3 at acpi0: bus 3 (RP03)
acpiprt4 at acpi0: bus 4 (PCIB)
acpiec0 at acpi0
acpicpu0 at acpi0: C3, C2
acpicpu1 at acpi0: C3, C2
acpipwrres0 at acpi0: FN00
acpitz0 at acpi0: critical temperature 98 degC
acpibat0 at acpi0: BAT1 type LION oem SAMSUNG Electronics
acpiac0 at acpi0: AC unit online
acpibtn0 at acpi0: LID0
acpibtn1 at acpi0: PWRB
acpibtn2 at acpi0: SLPB
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD01
acpivout1 at acpivideo0: DD02
acpivout2 at acpivideo0: DD03
acpivout3 at acpivideo0: DD04
acpivout4 at acpivideo0: DD05
bios0: ROM list: 0xc/0xec00! 0xdf000/0x1000! 0xe/0x1800!
cpu0: unknown Enhanced SpeedStep CPU, msr 0x060f0c2006000c20
cpu0: using only highest and lowest power states
cpu0: Enhanced SpeedStep 1596 MHz: speeds: 1600, 800 MHz
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 Intel 82945GME Host rev 0x03
vga1 at pci0 dev 2 function 0 Intel 82945GME Video rev 0x03
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
intagp0 at vga1
agp0 at intagp0: aperture at 0xd000, size 0x1000
inteldrm0 at vga1: apic 1 int 16 (irq 11)
drm0 at inteldrm0
Intel 82945GM Video rev 0x03 at pci0 dev 2 function 1 not configured
azalia0 at pci0 dev 27 function 0 Intel 82801GB HD Audio rev 0x02:
apic 1 int 22 (irq 5)
azalia0: codecs: Realtek ALC272
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 Intel 82801GB PCIE rev 0x02: apic 1
int 17 (irq 10)
pci1 at ppb0 bus 2
ath0 at pci1 dev 0 function 0 Atheros AR5424 rev 0x01: apic 1 int 16 (irq 11)
ath0: AR5424 14.2 phy 7.0 rf 0.0, WOR4W, address 00:24:d2:6a:b5:0b
ppb1 at pci0 dev 28 function 2 Intel 82801GB PCIE rev 0x02: apic 1
int 18 (irq 5)
pci2 at ppb1 bus 3
mskc0 at pci2 dev 0 function 0 Marvell Yukon 88E8040 rev 0x13,
Yukon-2 FE+ rev. A0 (0x0): apic 1 int 18 (irq 5)
msk0 at mskc0 port A: address 00:13:77:f7:a2:c0
eephy0 at msk0 phy 0: 88E3016 10/100 PHY, rev. 0
uhci0 at pci0 dev 29 function 0 Intel 82801GB USB rev 0x02: apic 1
int 23 (irq 5)
uhci1 at pci0 dev 29 function 1 Intel 82801GB USB rev 0x02: apic 1
int 19 (irq 5)
uhci2 at pci0 dev 29 function 2 Intel 82801GB USB rev 0x02: apic 1
int 18 (irq 5)
uhci3 at pci0 dev 29 function 3 Intel 82801GB USB rev 0x02: apic 1
int 16 (irq 11)
ehci0 at pci0 dev 29 function 7 Intel 82801GB USB rev 0x02: apic 1
int 23 (irq 5)
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1
ppb2 at pci0 dev 30 function 0 Intel 82801BAM Hub-to-PCI rev 0xe2
pci3 at ppb2 bus 4
ichpcib0 at pci0 dev 31 function 0 Intel 82801GBM LPC rev 0x02: PM disabled
pciide0 at pci0 dev 31 function 2 Intel 82801GBM SATA rev 0x02: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: SAMSUNG HM160HI
wd0: 16-sector PIO, LBA48, 152627MB, 312581808 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
ichiic0 at pci0 dev 31 function 3 Intel 82801GB SMBus rev 0x02: apic
1 int 19 (irq 5)
iic0 at ichiic0
spdmem0 at iic0 addr 

Re: Why do clients running BitTorrent make my router's latency go through the roof?

[Max Hayden Chiz]

On Jan 15, 2008 11:43 AM, Chris Cappuccio [EMAIL PROTECTED] wrote:
 you keep saying that you aren't maxing out your bandwidth, but if you
 only have 512Kbps upstream, it would be very easy to do.  do you have
 any idea how much upstream bandwidth you are using between all of your BT
 connections?

My original test was capped at 384Kbps (i.e. 48KBps). I have tried it
with 256Kbps (32KBps), 128Kbps (16KBps), etc. I have also managed to
sustain HTTP and FTP connections to my server at 500+Kbps for days at
a time with no problems before.  If upload was a problem, I would
expect this usage situation to cause problems as well.  More
importantly,  I do not experience the latency when I am simply
seeding (only uploading).  I only have this problem when BitTorrent
is making large numbers of connections to download.

I had some extra time today, and swapped out the OpenBSD box for a
NetGear router.  I am no longer experiencing the latency problem.  I
will continue monitoring the problem for another 48 hours or so, but
my initial conclusion is that the problem is with the OpenBSD box.
Either this is a hardware problem with my box, or it is a software
bug.  Given the number of posts experiencing difficulty with
BitTorrent I am inclined toward the latter as opposed to the former.

I'm going to see if I can get another OpenBSD box to test this on, but
it won't be a Blade100.  I can also try adding an extra network card
and not using gem0 (the interface with the problem).  But that's the
best I'm going to be able to do from this end.

--MHC

Re: Suggested PF Setup when using BitTorrent?

[Max Hayden Chiz]

Brian,

After your post (and several others), I tried BitTorrent out on my
network (sparc64 router + DOCSIS 2.0 cable connection; see
http://marc.info/?l=openbsd-miscm=120019379210857w=2)

After some experimentation, I was able to determine that running
BitTorrent with a large number of connections causes a huge increase
in latency regardless of bandwidth.  No one seems to know why this is,
but that might just be because my thread got buried by trolls and
other posts.

I'm not having watchdog timeouts but there is an off chance that the
latency increase that I experience and your timeout problem may be
related.  My work around is to use the max-src-states feature of pf to
limit the number of bit torrent connections to a reasonable number (50
seems to be a good trade-off on my machine, YMMV).

Could you modify your pf.conf to do this (or limit your connections at
the client and use pf to confirm) and let us know if that works on
your end as well?

--MHC

On Jan 5, 2008 1:22 PM, Brian [EMAIL PROTECTED] wrote:
 Is there any suggested PF setup when using BitTorrent?

 Right now, the biggest problem I have when using BitTorrent is watchdog
 timeouts.

 Thanks,

 Brian




   
 
 Never miss a thing.  Make Yahoo your home page.
 http://www.yahoo.com/r/hs

Re: Suggested PF Setup when using BitTorrent?

[Max Hayden Chiz]

On Jan 14, 2008 6:30 PM, Chris Kuethe [EMAIL PROTECTED] wrote:
 My theory is that you're using a ... uh... well, not very good
 connection that bogs down easily.

My connection normally works fine; even when I max out my 7Mb/512Kb
line.  Running BitTorrent (even with a fraction of the bandwidth)
makes my latency go through the roof.

 Some time ago, I tossed together a
 little undeadly article on how to use altq to keep bittorrent from
 dragging your network down -
 http://www.undeadly.org/cgi?action=articlesid=20061109202501

See the thread I referenced in my previous email.  The issue I am
experiencing seems to have nothing to do with bandwidth usage.

 I think you can solve this by tuning your service classes a little
 better.

If you have a specific suggestion, I will try it and post the result
here, but as I discussed in the previous thread, altq shows that there
is no backlog of packets, and even with BitTorrent rate limited to a
small fraction of the bandwidth (via CBQ or HFSC) it is still able to
cause the latency issue.  By contrast if I limit the number of
connections, BitTorrent can consume almost all of the bandwidth and
the issue will not appear.

Perhaps this problem is specific to my configuration (or specific to
DOCSIS cable modems).  But if it makes Brian (or someone else's
problem) go away, then it is likely that this problem is not unique.

--MHC

Re: Why do clients running BitTorrent make my router's latency go through the roof?

[Max Hayden Chiz]

Because several people have asked,  my Internet connection is a
business class cable connection with guaranteed 512Kbps up and 7Mbps
down.  I do get those speeds and can sustain them essentially
indefinitely.

On Jan 12, 2008 9:01 PM, Max Hayden Chiz [EMAIL PROTECTED] wrote:
 I noticed that running BitTorrent was making my network go very slow
 and have been trying to fix it.  After spending most of the day
 playing around with it I have concluded that the problem is caused by
 having too many simultaneous BitTorrent connections.  As you increase
 the number of connections, the latency on the external interface
 increases dramatically (e.g. ping times hit 900+ms or time out
 entirely.)  This is true regardless of bandwidth usage, because I can
 rate limit client and still cause the problem.  Running `pfctl -vvsq`
 shows that altq doesn't have a backlog.  Looking at the archives, it
 seems that others on the list have experienced this problem in the
 past, but there hasn't been a final resolution.

 I am at a total loss as to why this would be causing the massive
 increase in latency.  Can someone more experienced explain why this is
 (and possibly tell me what I'm doing wrong)?  For your reference I'm
 running OpenBSD4.2-current (Dec18 snapshot) on a Sun Blade 100.  The
 computer is as it comes from the factory except that I have added a
 gigabit network card (re) and a wifi card (ral).

 Here is my pf.conf:

 ext_if=gem0
 int_if=re0
 wifi=ral0
 vpn=enc0

 bthost=172.16.1.10
 btport=21885

 set skip on lo

 scrub in
 scrub on $vpn max-mss 1400 no-df random-id

 altq on $ext_if priq bandwidth 512Kb queue{ack, main, others, bt}
 queue ack priority 7
 queue main priority 6
 queue others priority 5
 queue bt priority 1 priq(default)


 nat on $ext_if from !($ext_if) - ($ext_if:0)
 rdr on $ext_if proto tcp to port $btport tag BT - $bthost

 block all

 pass on $int_if no state

 pass in on $ext_if proto tcp to port $btport queue bt

 pass out on $ext_if queue (others, ack)
 pass out on $ext_if from $bthost queue bt
 pass out on $ext_if proto tcp to port {ssh, http, https} queue (main,ack)

 pass in proto tcp to port ssh

 ##Rules for WiFi Gateway

 #Allow configuring IPSec
 pass in on $wifi proto udp to port isakmp
 pass in on $wifi proto udp to port domain
 pass in on $wifi proto esp

 #allow authenticated users to do everything
 pass on $vpn no state

 I can send a dmesg or anything else if I need to.  Thanks in advance
 for your help.

 --MHC

 P.S. The obvious way to have pf deal with this is to use
 max-src-states.  I have tested this approach and confirmed that it
 will avoid the problem, but I don't understand why this works, nor do
 I know if this is the correct way to deal with this.

Re: Why do clients running BitTorrent make my router's latency go through the roof?

[Max Hayden Chiz]

On Jan 13, 2008 1:16 PM, Darrin Chandler [EMAIL PROTECTED] wrote:
   altq on $ext_if priq bandwidth 512Kb queue{ack, main, others, bt}

 On my home assymetric connection I noticed that I had to adjust the
 bandwidth down just a little before the ackpriq method worked well. Yes,
 I measured upload speed and tried *that* number, but I still had to set
 it a bit lower. This is purely anecdotal, and I didn't do in-depth
 measurements. However, it's easy to try. :)

Okay,  maybe I wasn't clear what the problem is.  The problem is that
having a high number of bittorrent connections causes high latency on
the external interface.  Using max-src-states fixes this problem, but
I don't understand why it is a problem to begin with.

From extensive experimentation here is what I have been able to determine:

The problem has nothing to do with bandwidth.  I don't experience this
problem with any other protocol (HTTP and FTP for example) and if I am
running a few connections and pulling down a huge chunk of my download
and using almost all of my upload, altq will work fine and I will have
little or no latency.

On the other hand, I can cause this problem even if I am only using a
fraction of the bandwidth -- all I have to do is have the bittorrent
client start a bunch of torrents and make hundreds of connections
each.

Altq isn't useful here because it isn't going to engage until there is
a backlog. Now, it is true that once the latency starts to rise, a
backlog will result, but I am already running a modified ack-priq and
the increase in latency becomes a problem (as in no one can browse the
web) long before it turns into backlog.  I have played with the altq
six ways from Sunday in an attempt to solve this.  It doesn't affect
anything unless I turn the bandwidth down to some ridiculously low
number.

Although the increase in latency seems to generally slow down the time
it takes to process a packet.  It seems to disproportionately impact
TCP handshakes.  At first I thought this was just because the
handshake was experiencing 3x the latency increase, but after playing
with it more, it seems that the increase is more than linear.

My concern is that this is a bug that would allow a malicious user to
perform a DoS attack on any router that allows for BitTorrent.  If you
don't use max-src-states, then the BitTorrent user (even if bandwidth
limited by hfsc or cbq) can make an absurd number of connections and
increase latency to the point that the external interface becomes
unusable.

--MHC

P.S. I am not using pppoe; I am just using an Ethernet cable connected
to a DOCSIS 2.0 cable modem.  There is nothing fancy going on there.

Re: Why do clients running BitTorrent make my router's latency go through the roof?

[Max Hayden Chiz]

On Jan 13, 2008 6:03 PM, Stuart Henderson [EMAIL PROTECTED] wrote:

 I think the upshot is you might well be better off to let the cable
 modem handle all this stuff, so do some measurements and find out...

I have the latency problem no matter what altq does.  Whether it is
off, priq, cbq, or hfsc, I can get this increase in latency as I
increase the number of bit torrent connections.

I don't know of a good way to generate lots of long-term connections
for any other application to see if this issue is bit torrent specific
either.  Nor do I know of a way to see if this is a problem with
DOCSIS or with OpenBSD.

Does anyone have any testing ideas?  I'm all out.

Why do clients running BitTorrent make my router's latency go through the roof?

[Max Hayden Chiz]

I noticed that running BitTorrent was making my network go very slow
and have been trying to fix it.  After spending most of the day
playing around with it I have concluded that the problem is caused by
having too many simultaneous BitTorrent connections.  As you increase
the number of connections, the latency on the external interface
increases dramatically (e.g. ping times hit 900+ms or time out
entirely.)  This is true regardless of bandwidth usage, because I can
rate limit client and still cause the problem.  Running `pfctl -vvsq`
shows that altq doesn't have a backlog.  Looking at the archives, it
seems that others on the list have experienced this problem in the
past, but there hasn't been a final resolution.

I am at a total loss as to why this would be causing the massive
increase in latency.  Can someone more experienced explain why this is
(and possibly tell me what I'm doing wrong)?  For your reference I'm
running OpenBSD4.2-current (Dec18 snapshot) on a Sun Blade 100.  The
computer is as it comes from the factory except that I have added a
gigabit network card (re) and a wifi card (ral).

Here is my pf.conf:

ext_if=gem0
int_if=re0
wifi=ral0
vpn=enc0

bthost=172.16.1.10
btport=21885

set skip on lo

scrub in
scrub on $vpn max-mss 1400 no-df random-id

altq on $ext_if priq bandwidth 512Kb queue{ack, main, others, bt}
queue ack priority 7
queue main priority 6
queue others priority 5
queue bt priority 1 priq(default)


nat on $ext_if from !($ext_if) - ($ext_if:0)
rdr on $ext_if proto tcp to port $btport tag BT - $bthost

block all

pass on $int_if no state

pass in on $ext_if proto tcp to port $btport queue bt

pass out on $ext_if queue (others, ack)
pass out on $ext_if from $bthost queue bt
pass out on $ext_if proto tcp to port {ssh, http, https} queue (main,ack)

pass in proto tcp to port ssh

##Rules for WiFi Gateway

#Allow configuring IPSec
pass in on $wifi proto udp to port isakmp
pass in on $wifi proto udp to port domain
pass in on $wifi proto esp

#allow authenticated users to do everything
pass on $vpn no state

I can send a dmesg or anything else if I need to.  Thanks in advance
for your help.

--MHC

P.S. The obvious way to have pf deal with this is to use
max-src-states.  I have tested this approach and confirmed that it
will avoid the problem, but I don't understand why this works, nor do
I know if this is the correct way to deal with this.

Can I please get help debugging performance issues with my IPSec configuration?

[Max Hayden Chiz]

I have a Sun Blade 100 with OpenBSD 4.2-current (Dec 18).  I'm trying
to configure it as a router/access point for my home network. The
hardware is as shipped from Sun except that I have added an extra
network card and a wireless card (re0 and ral0).  I can send a dmesg
if anyone thinks it would be helpful.

Ultimately my goal is to configure the wireless to use authpf and
IPSec, similar to the configuration presented in:
http://www.openbsd-support.com/jp/en/htm/mgp/pacsec05/index.html

I have IPSec working between the clients and the OpenBSD box, and
almost everything is working with only a slight increase in latency.
But, loading very complex websites (yahoo, YouTube) takes so long that
the HTTP connection will reset before the browser is done.  I can't
figure out why this is happening and didn't find anything similar when
I searched the archives.

I would really appreciate it if someone could take a look at the
minimal configuration below and tell me where I am messing up.  If I
left out a configuration file that you need or if you need me to run
commands for you, please let me know.  I also have an OpenBSD server
on the wired portion of the network that can be used for testing if
necessary. Thanks in advance for any help you can provide.

--MHC

Note: ral0 on the OpenBSD system is 172.16.1.1.

Windows Clients:
I am connecting using a simple vpn.bat:
ipseccmd -u
ipseccmd.exe -f 0=* -n ESP[3DES,SHA] -t 172.16.1.1 -a
PRESHARE:testword -1s 3DES-SHA-2
ipseccmd.exe -f *=0 -n ESP[3DES,SHA] -t %1 -a PRESHARE:testword -1s 3DES-SHA-2

OpenBSD:
# cat /etc/pf.conf
#   $OpenBSD: pf.conf,v 1.34 2007/02/24 19:30:59 millert Exp $
#
# See pf.conf(5) and /usr/share/pf for syntax and examples.
# Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1
# in /etc/sysctl.conf if packets are to be forwarded between interfaces.

ext_if=gem0
int_if=re0
wifi=ral0

set skip on {lo enc0}

scrub in

nat on $ext_if from !($ext_if) - ($ext_if:0)

block in
pass out on $ext_if
pass in proto tcp to port ssh
pass on $wifi no state
pass on $int_if no state

# cat /etc/ipsec.conf
#   $OpenBSD: ipsec.conf,v 1.5 2006/09/14 15:10:43 hshoexer Exp $
#
# See ipsec.conf(5) for syntax and examples.

wifi=172.16.0.0/16
ike passive esp from any to $wifi \
main auth hmac-sha1 enc 3des  group modp1024 \
quick auth hmac-sha1 enc 3des group none \
psk testword

# cat /etc/rc.conf.local
ntpd_flags= # enabled during install
dhcpd_flags=-Ldhcp-leases -Adhcp-abandoned
pf=
named_flags=
isakmpd_flags=-4K
ipsec=YES

Re: Can I please get help debugging performance issues with my IPSec configuration?

[Max Hayden Chiz]

Thank you very much for your swift reply.  Using 'scrub on enc0
max-mss 1310 no-df' immediately solved the problem.

I have two questions though,  since 1310 is smaller than needed, how
do I determine the correct setting to use after max-mss?  I understand
that in theory I want to subtract the length of the extra IP header
and the ESP header from 1500, but I'm not sure what the length of an
ESP header is (since it looks like it is variable because of padding.)

Also, the pf.conf man page recommends using random-id with no-df.  Is
that appropriate here?

Thank you again for all your help.

--MHC



On 1/6/08, Stuart Henderson [EMAIL PROTECTED] wrote:
 On 2008/01/06 03:10, Max Hayden Chiz wrote:
  But, loading very complex websites (yahoo, YouTube) takes so long that
  the HTTP connection will reset before the browser is done.  I can't
  figure out why this is happening and didn't find anything similar when
  I searched the archives.

 Sounds like it could be MTU problems. With IPsec you don't have the
 usual 1500-byte MTU from a normal ethernet interface, it's smaller
 because of the additional headers.

  set skip on {lo enc0}
  scrub in

 I would remove enc0 from 'set skip' (you'll need a pass rule in its
 place) and then try something like 'scrub on enc0 max-mss 1310 no-df'
 (iirc, this comes after the other scrub rule). 1310 is smaller than
 you're actually likely to need but should work.

dhcpd misleading documentation about the interaction between fixed-address statement and -L option

[Max Hayden Chiz]

Unless I am doing something silly, dhcpd(8) may not be accurate
regarding how the fixed-address statement and -L option
inter-operate.

Yesterday I discovered that when you assign a DHCP client an IP
address using the fixed-address command, dhcpd does not create an
entry in dhcpd.leases.  As a result, it will not enter the IP address
into the pf table specified by the -L option.  This seems to be the
intended behavior, but it isn't documented anywhere.  In fact dhcpd(8)
says [e]ach client is assigned a lease, implying that even clients
using fixed-address should be given a lease entry.

Is this something that should be fixed or am I reading the man pages
wrong (or worse have I messed something up on my end)?

Max H. Chiz