Hello. I've been having some serious network throughput issues recently.
Prior to these issues, I was running 7.0-STABLE from a few months back. I
recently build a world/kernel from csup(1) on 2008-Jul-25 and started
noticing the issues. I'm currently on a RELENG_7 from 2008-Aug-02 and am
building another one today.
The server is running samba, dnsmasq, an NFS server, and is the gateway for
a couple of freebsd machines. I first noticed the issue when I was unable
to play mp3s served over samba. I was getting a throughput of about 3
KBytes/sec over my gigabit switched network. After a few weeks of trying
different versions of samba and watching tcpdump/trafshow, I decided to try
another test:
% ssh gateway cat /dev/zero | dd of=/dev/null
load: 0.00 cmd: dd 68286 [runnable] 0.00u 0.00s 0% 844k
3680+0 records in
3680+0 records out
1884160 bytes transferred in 512.856684 secs (3674 bytes/sec)
^C3680+0 records in
3680+0 records out
1884160 bytes transferred in 513.392858 secs (3670 bytes/sec)
Killed by signal 2.
Again, not even 4 KB/s throughput over ssh. What's weird is that not all
network traffic is slow. I'm able to download at almost my full DSL speed
(7168 Kbps) and here's the strange bit: NFS is fast! From the same
machine as above, I have an NFS mountpoint:
% dd if=/nfs/some_large_file of=/dev/null
55980+0 records in
55980+0 records out
28661760 bytes transferred in 0.147159 secs (194767476 bytes/sec)
Which is what I would expect for a gigabit network. Even if I perform the
same test over ssh:
% ssh workstation cat /nfs/some_large_file | dd of=/dev/null
55980+0 records in
55980+0 records out
28661760 bytes transferred in 2.791392 secs (10267909 bytes/sec)
This is reasonable considering ssh overhead. The kernel config for the
server contains:
include GENERIC
ident DDB
options KDB
options KDB_TRACE
options DDB
options DDB_NUMSYM
My internal network is on 172.23.20.x and the DSL modem is connected to the
same NIC on 192.168.0.1:
# ifconfig nfe0
nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=10b<RXCSUM,TXCSUM,VLAN_MTU,TSO4>
ether 00:15:f2:17:0c:20
inet 172.23.20.1 netmask 0xffffff00 broadcast 172.23.20.255
inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (1000baseTX <full-duplex,flag0,flag1>)
status: active
# ipfw show
00100 62968 8170100 allow ip from any to any via lo0
00110 0 0 deny ip from any to 127.0.0.0/8
00120 0 0 deny ip from 127.0.0.0/8 to any
00130 25444031 23419621364 divert 8668 ip from any to any via 192.168.0.3
00140 0 0 deny ip from not 172.23.20.0/24 to 172.23.20.0/24
dst-port 137-149,445
00150 20639141 15067620538 allow tcp from any to any established
00160 21 10683 allow ip from any to any frag
00170 79791 4696696 allow tcp from any to any setup
65530 5713367 8864760207 allow ip from any to any
65535 0 0 deny ip from any to any
Other than that, I'm not doing anything out of the ordinary. Why is NFS
behaving correctly and why are ssh/smbd connections so slow? I've pasted
my dmesg output below. I've used this configuration for years and it
wasn't until a recent RELENG_7 upgrade that I've had any problems. The box
was 99-100% idle during those tests and I don't see an interrupt storm or
anything funny like that. Any ideas?
-- Rick C. Petty
>From /var/run/dmesg.boot:
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-STABLE #5: Sat Aug 2 19:43:25 CDT 2008
[EMAIL PROTECTED]:/usr/obj/usr/src/sys/DDB
module_register: module sk/miibus already exists!
Module sk/miibus failed to register: 17
module_register: module skc/sk already exists!
Module skc/sk failed to register: 17
module_register: module pci/skc already exists!
Module pci/skc failed to register: 17
module_register: module rl/miibus already exists!
Module rl/miibus failed to register: 17
module_register: module cardbus/rl already exists!
Module cardbus/rl failed to register: 17
module_register: module pci/rl already exists!
Module pci/rl failed to register: 17
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ (2211.34-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0x20f32 Stepping = 2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x1<SSE3>
AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!>
AMD Features2=0x3<LAHF,CMP>
Cores per package: 2
real memory = 1073676288 (1023 MB)
avail memory = 1037037568 (988 MB)
ACPI APIC Table: <Nvidia AWRDACPI>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
ioapic0: Changing APIC ID to 2
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
acpi0: <Nvidia AWRDACPI> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 3fef0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory> at device 0.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
atapci0: <nVidia nForce CK804 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 6.0 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
atapci1: <nVidia nForce CK804 SATA300 controller> port
0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xd800-0xd80f mem
0xd0002000-0xd0002fff irq 20 at device 7.0 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
atapci2: <nVidia nForce CK804 SATA300 controller> port
0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xc400-0xc40f mem
0xd0001000-0xd0001fff irq 21 at device 8.0 on pci0
atapci2: [ITHREAD]
ata4: <ATA channel 0> on atapci2
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci2
ata5: [ITHREAD]
pcib1: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci5: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> mem 0xc0000000-0xc7ffffff irq 16 at device
6.0 on pci5
atapci3: <Promise PDC40718 SATA300 controller> port 0xa000-0xa07f,0xa400-0xa4ff
mem 0xc9041000-0xc9041fff,0xc9000000-0xc901ffff irq 17 at device 7.0 on pci5
atapci3: [ITHREAD]
atapci3: [ITHREAD]
ata6: <ATA channel 0> on atapci3
ata6: [ITHREAD]
ata7: <ATA channel 1> on atapci3
ata7: [ITHREAD]
ata8: <ATA channel 2> on atapci3
ata8: [ITHREAD]
ata9: <ATA channel 3> on atapci3
ata9: [ITHREAD]
atapci4: <Promise PDC40718 SATA300 controller> port 0xa800-0xa87f,0xac00-0xacff
mem 0xc9040000-0xc9040fff,0xc9020000-0xc903ffff irq 18 at device 8.0 on pci5
atapci4: [ITHREAD]
atapci4: [ITHREAD]
ata10: <ATA channel 0> on atapci4
ata10: [ITHREAD]
ata11: <ATA channel 1> on atapci4
ata11: [ITHREAD]
ata12: <ATA channel 2> on atapci4
ata12: [ITHREAD]
ata13: <ATA channel 3> on atapci4
ata13: [ITHREAD]
nfe0: <NVIDIA nForce4 CK804 MCP9 Networking Adapter> port 0xb000-0xb007 mem
0xd0000000-0xd0000fff irq 22 at device 10.0 on pci0
miibus0: <MII bus> on nfe0
e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 9 on miibus0
e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
nfe0: Ethernet address: 00:15:f2:17:0c:20
nfe0: [FILTER]
pcib2: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci4: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 12.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 13.0 on pci0
pci2: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 14.0 on pci0
pci1: <ACPI PCI bus> on pcib5
cpu0: <ACPI CPU> on acpi0
powernow0: <Cool`n'Quiet K8> on cpu0
device_attach: powernow0 attach returned 6
cpu1: <ACPI CPU> on acpi0
powernow1: <Cool`n'Quiet K8> on cpu1
device_attach: powernow1 attach returned 6
acpi_button0: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xc0000-0xccfff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 8250 or not responding
sio0: [FILTER]
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based forwarding
disabled, default to deny, logging disabled
ad0: 157066MB <Hitachi HDS722516VLAT80 V34OA6MA> at ata0-master UDMA100
ad12: 715404MB <Seagate ST3750330AS SD04> at ata6-master SATA150
ad14: 715404MB <Seagate ST3750330AS SD04> at ata7-master SATA150
ad16: 715404MB <Seagate ST3750330AS SD04> at ata8-master SATA150
ad18: 715404MB <Seagate ST3750330AS SD04> at ata9-master SATA150
ad20: 476940MB <WDC WD5000AAKS-65TMA0 12.01C01> at ata10-master SATA300
ad22: 476940MB <WDC WD5000AAKS-65TMA0 12.01C01> at ata11-master SATA300
ad24: 476940MB <WDC WD5000AAKS-00TMA0 12.01C01> at ata12-master SATA300
ad26: 476940MB <WDC WD5000AAKS-65TMA0 12.01C01> at ata13-master SATA300
SMP: AP CPU #1 Launched!
GEOM_LABEL: Label for provider ad0s1g is ufs/cache.
Trying to mount root from ufs:/dev/ad0s1a
GEOM_LABEL: Label ufs/cache removed.
GEOM_LABEL: Label for provider ad0s1g is ufs/cache.
GEOM_LABEL: Label ufs/cache removed.
nfe0: link state changed to UP
Additional messages from dmesg -a:
Aug 16 14:02:20 gateway login: ROOT LOGIN (root) ON ttyv3
nfe0: promiscuous mode enabled
nfe0: promiscuous mode disabled
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
