Re: Tuning OpenBSD network throughput

[Srebrenko Sehic]

You can try netperf. It works fine. Snippet below shows the results of
the network thruput between 2 Opteron boxes running 3.9 with bge/em
gigabit cards.

[EMAIL PROTECTED]:/home/ssehic$ netperf -H x2100 -l 300 -- -s 64K -m 64K -M
64K -S 64K
TCP STREAM TEST to 192.168.0.61
Recv   SendSend
Socket Socket  Message  Elapsed
Size   SizeSize Time Throughput
bytes  bytes   bytessecs.10^6bits/sec

65536  65536  65536300.01 936.58

Tuning OpenBSD network throughput

[Matthew R. Dempsky]

I have three machines that I'm using for testing network performance:
- 2.0GHz Pentium 4, 256MiB RAM, Ubuntu 6.06, e1000
- 266MHz Pentium II, 192MiB RAM, Debian Unstable, sk98lin
- 600MHz Pentium M, 256MiB RAM, OpenBSD 4.0-current, em(4)
All network settings are still at their respective defaults.

First, I connected the two Linux boxes with an Ethernet cable and ran 
``iperf -s'' on the 2.0GHz machine and ``iperf -c 192.168.10.1'' on the 
266MHz machine, and iperf reported a bandwidth of about 224 Mbits/sec.

Then, I substituted out the 266MHz machine and replaced it with the 
600MHz machine (i.e., faster processor, more ram, and better software), 
but running ``iperf -c 192.168.10.1'' under OpenBSD reported a mere
3.8 Mbits/sec---nearly two orders of magnitude less!

Can anyone explain the huge discrepancy here?  Can I do anything to get 
OpenBSD to achieve at least 150 Mbits/sec?

Thanks.

(I've omitted the Linux dmesgs, but can provide them if they would be 
considered useful and not just line noise.)

OpenBSD 4.0-beta (GENERIC) #1055: Thu Aug  3 11:39:24 MDT 2006
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) M processor 1.10GHz (GenuineIntel 686-class) 
599 MHz
cpu0: 
FPU,V86,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,TM,SBF,EST,TM2
cpu0: Enhanced SpeedStep 600 MHz (812 mV): speeds: 1100, 1000, 900, 800, 
600 MHz
real mem  = 258437120 (252380K)
avail mem = 228171776 (222824K)
using 3180 buffers containing 13025280 bytes (12720K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(77) BIOS, date 06/15/05, BIOS32 rev. 0 @ 
0xfd740, SMBIOS rev. 2.33 @ 0xe0010 (56 entries)
bios0: IBM 2371BMU
apm0 at bios0: Power Management spec V1.2
apm0: battery life expectancy 35%
apm0: AC off, battery charge high, estimated 1:42 hours
apm0: flags 30102 dobusy 0 doidle 1
pcibios0 at bios0: rev 2.1 @ 0xfd6d0/0x930
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdeb0/256 (14 entries)
pcibios0: PCI Interrupt Router at 000:31:0 (Intel 82371FB ISA rev 
0x00)
pcibios0: PCI bus #2 is the last bus
bios0: ROM list: 0xc/0xc800! 0xcc800/0x1000 0xcd800/0x1000 
0xdc000/0x4000! 0xe/0x1
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 Intel 82852GM Hub-PCI rev 0x02
Intel 82852GM Memory rev 0x02 at pci0 dev 0 function 1 not configured
Intel 82852GM Configuration rev 0x02 at pci0 dev 0 function 3 not 
configured
vga1 at pci0 dev 2 function 0 Intel 82852GM AGP rev 0x02: aperture at 
0xe000, size 0x800
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
Intel 82852GM AGP rev 0x02 at pci0 dev 2 function 1 not configured
uhci0 at pci0 dev 29 function 0 Intel 82801DB USB rev 0x01: irq 11
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1 at pci0 dev 29 function 1 Intel 82801DB USB rev 0x01: irq 11
usb1 at uhci1: USB revision 1.0
uhub1 at usb1
uhub1: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhci2 at pci0 dev 29 function 2 Intel 82801DB USB rev 0x01: irq 11
usb2 at uhci2: USB revision 1.0
uhub2 at usb2
uhub2: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
ehci0 at pci0 dev 29 function 7 Intel 82801DB USB rev 0x01: irq 11
usb3 at ehci0: USB revision 2.0
uhub3 at usb3
uhub3: Intel EHCI root hub, rev 2.00/1.00, addr 1
uhub3: 6 ports with 6 removable, self powered
ppb0 at pci0 dev 30 function 0 Intel 82801BAM Hub-to-PCI rev 0x81
pci1 at ppb0 bus 1
cbb0 at pci1 dev 0 function 0 Ricoh 5C476 CardBus rev 0x8d: irq 11
sdhc0 at pci1 dev 0 function 1 Ricoh 5C822 SD/MMC rev 0x13: irq 11
sdmmc0 at sdhc0
em0 at pci1 dev 1 function 0 Intel PRO/1000MT Mobile (82541GI) rev 
0x00: irq 11, address 00:0a:e4:37:61:6a
iwi0 at pci1 dev 2 function 0 Intel PRO/Wireless 2200BG rev 0x05: irq 
11, address 00:13:ce:58:8f:14
cardslot0 at cbb0 slot 0 flags 0
cardbus0 at cardslot0: bus 2 device 0 cacheline 0x0, lattimer 0xb0
pcmcia0 at cardslot0
ichpcib0 at pci0 dev 31 function 0 Intel 82801DBM LPC rev 0x01
pciide0 at pci0 dev 31 function 1 Intel 82801DBM IDE rev 0x01: DMA, 
channel 0 configured to compatibility, channel 1 configured to 
compatibility
wd0 at pciide0 channel 0 drive 0: HTC426030G7AT00
wd0: 16-sector PIO, LBA, 28615MB, 58605120 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
pciide0: channel 1 disabled (no drives)
ichiic0 at pci0 dev 31 function 3 Intel 82801DB SMBus rev 0x01: irq 11
iic0 at ichiic0
auich0 at pci0 dev 31 function 5 Intel 82801DB AC97 rev 0x01: irq 11, 
ICH4 AC97
ac97: codec id 0x41445374 (Analog Devices AD1981B)
ac97: codec features headphone, 20 bit DAC, No 3D Stereo
audio0 at auich0
Intel 82801DB Modem rev 0x01 at pci0 dev 31 function 6 not configured
isa0 at ichpcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5

Re: Tuning OpenBSD network throughput

[knitti]

On 8/8/06, Matthew R. Dempsky [EMAIL PROTECTED] wrote:

First, I connected the two Linux boxes with an Ethernet cable and ran
``iperf -s'' on the 2.0GHz machine and ``iperf -c 192.168.10.1'' on the
266MHz machine, and iperf reported a bandwidth of about 224 Mbits/sec.

Then, I substituted out the 266MHz machine and replaced it with the
600MHz machine (i.e., faster processor, more ram, and better software),
but running ``iperf -c 192.168.10.1'' under OpenBSD reported a mere
3.8 Mbits/sec---nearly two orders of magnitude less!

Can anyone explain the huge discrepancy here?  Can I do anything to get
OpenBSD to achieve at least 150 Mbits/sec?


first look for duplex mismatch, bad cabling etc.
then look for high interrupt load, change hardware etc.
then read about iperf, and think whether it applies to your problem.
then think about your goal. do you want 150 mbit with tiny 40 bytes packets
or with jumbo frames (huge difference)
and, in any case, search the archives about tuning openbsd.

--knitti

Re: Tuning OpenBSD network throughput

[Luiz Souza]

knitti wrote:

On 8/8/06, Matthew R. Dempsky [EMAIL PROTECTED] wrote:

First, I connected the two Linux boxes with an Ethernet cable and ran
``iperf -s'' on the 2.0GHz machine and ``iperf -c 192.168.10.1'' on the
266MHz machine, and iperf reported a bandwidth of about 224 Mbits/sec.

Then, I substituted out the 266MHz machine and replaced it with the
600MHz machine (i.e., faster processor, more ram, and better software),
but running ``iperf -c 192.168.10.1'' under OpenBSD reported a mere
3.8 Mbits/sec---nearly two orders of magnitude less!

Can anyone explain the huge discrepancy here?  Can I do anything to get
OpenBSD to achieve at least 150 Mbits/sec?


first look for duplex mismatch, bad cabling etc.
then look for high interrupt load, change hardware etc.
then read about iperf, and think whether it applies to your problem.
then think about your goal. do you want 150 mbit with tiny 40 bytes packets
or with jumbo frames (huge difference)
and, in any case, search the archives about tuning openbsd.

--knitti


i think iperf doest like openbsd threads.

iperf never reports more than 5mb/s if running on openbsd, but there is no 
problem on troughput of openbsd at all.

Look at some other tool or use iperf to check bandwidth through openbsd (using 
two other machines) and not to openbsd.

luiz

Re: Tuning OpenBSD network throughput

[Antti Harri]

On Tue, 8 Aug 2006, Matthew R. Dempsky wrote:


but running ``iperf -c 192.168.10.1'' under OpenBSD reported a mere
3.8 Mbits/sec---nearly two orders of magnitude less!


The version of iperf in ports is broken for at least i386. It needs a 
patch to run correctly. I wrote one but someone replied 
it doesn't work on other architecture (amd64 iirc) after that. I didn't

bother to look into it as I have only i386 machines.

Use version 1.x or write a patch for 2.x and submit it :-)

--
Antti Harri

Re: Tuning OpenBSD network throughput

[Michal Soltys]

Matthew R. Dempsky wrote:

I have three machines that I'm using for testing network performance:
- 2.0GHz Pentium 4, 256MiB RAM, Ubuntu 6.06, e1000
- 266MHz Pentium II, 192MiB RAM, Debian Unstable, sk98lin
- 600MHz Pentium M, 256MiB RAM, OpenBSD 4.0-current, em(4)


 [cut]


Can anyone explain the huge discrepancy here?  Can I do anything to get 
OpenBSD to achieve at least 150 Mbits/sec?


Thanks.



Besides certain compex cards (wb driver, 3.8, with queuing under PF),
I haven't had any strange problems with bandwidth either (testing plenty
of fxp, xl, rl, vr cards). Well, there were some, but that's different 
subject regarding cbq/hfsc and non-borrowing queues, at certain speeds.


You can try few other methods to measure bandwidth. For example:

- 2 nc, one reading from /dev/zero, the other writing to /dev/null
- out of box openbsd's httpd, wget writing to /dev/null
- some ftp transfer or even scp from dd's premade file (disk shouldn't 
really be a bottleneck at fast ethernet speeds, neither should be encryption 
with not hopelessy old cpu)


Pair these with some simple PF setting using queues, and watch the bandwidth
with pftop, systat, ifstat or pfctl -vvsq to name a few. Time command can be 
helpful too.


Also, remember that queuing works only in outgoing direction, if you decide 
to use it.