Re: strange network performace
show us the output of ifconfig -a on each box. Yeah ... did you try to force media opt to 100baseTX full duplex w/ ifconfig ? I had once a lot of problems connecting 3C905 (XL) cards to 3COM switches, until i tried to force the negociation ... To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
How many times did you run the test? Could it have been cached in host B, but not C? What about disk performance? Maybe one is ATA66 or 100, and tthe other is not? Host C may only be UDMA33, and possibly have a slow drive, with a lower amount of memory for caching. Did you rebuild kernels on any of them? Eric Danny Braniss wrote: setup: host A: dual pentium III/1GHz (Dell 2450) host B: dual pentium III/950MHz (Intel STL2) host C: Pentium III/1GHz (Dell GX150) all connected at 100Mgb full duplex all three are running FreeBSD 4.4. all three have identical troughput when writing to a NetAPP fileserver ~ 10MBs but: B - A: ~ 6MBs C - A: ~ 2MBs Q: why is C - A so bad? danny To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- - Eric Anderson[EMAIL PROTECTED]Centaur Technology # rm -rf /bin/laden - To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
How many times did you run the test? Could it have been cached in host B, but not C? What about disk performance? Maybe one is ATA66 or 100, and tthe other is not? Host C may only be UDMA33, and possibly have a slow drive, with a lower amount of memory for caching. Did you rebuild kernels on any of them? so many questions :-)! the test were run many times, on host C the port was changed, the cable was changed, the NIC was changed. the disk is not relevant, the test uses a small program i wrote that writes from memory. at the moment all three run the same version of the kernel, the only diff is that A B are smp, and C is not. danny To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
And what happens when you go from A or B to C? Have you been running a top or systat -vmstat while this is happening? I'm thinking it might be a purely IO thing, on the proc box. I have seen similar slowness with the default FreeBSD install on single proc boxes, but a few sysctl's seem to do the trick. Eric Danny Braniss wrote: How many times did you run the test? Could it have been cached in host B, but not C? What about disk performance? Maybe one is ATA66 or 100, and tthe other is not? Host C may only be UDMA33, and possibly have a slow drive, with a lower amount of memory for caching. Did you rebuild kernels on any of them? so many questions :-)! the test were run many times, on host C the port was changed, the cable was changed, the NIC was changed. the disk is not relevant, the test uses a small program i wrote that writes from memory. at the moment all three run the same version of the kernel, the only diff is that A B are smp, and C is not. danny -- - Eric Anderson[EMAIL PROTECTED]Centaur Technology # rm -rf /bin/laden - To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
And what happens when you go from A or B to C? Have you been running a top or systat -vmstat while this is happening? I'm thinking it might be a purely IO thing, on the proc box. I have seen similar slowness with the default FreeBSD install on single proc boxes, but a few sysctl's seem to do the trick. a reminder: A - NetAPP B - NetAPP C - NetAPP is fine, im running the same test, and the results are consitant, ~10MGBs so that should eliminate network/switch/port/cable problems, right? btw, no network errors are reported, neither from the hosts nor the switches. danny To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
Oh, well, I thought you said 10mb/s, not 10MB/s .. that makes it a bit different. I wonder if it could still be a tcp window size or something.. Try these sysctl's on C: vfs.nfs.gatherdelay=0 vfs.nfs.async=1 vfs.vmiodirenable=1 kern.ipc.maxsockbuf=2097152 kern.ipc.somaxconn=8192 kern.ipc.maxsockets=16424 kern.maxfiles=65536 kern.maxfilesperproc=32768 net.inet.tcp.rfc1323=1 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=65535 net.inet.tcp.recvspace=65535 net.inet.udp.recvspace=65535 net.inet.udp.maxdgram=57344 net.local.stream.recvspace=65535 net.local.stream.sendspace=65535 Just a try.. Danny Braniss wrote: And what happens when you go from A or B to C? Have you been running a top or systat -vmstat while this is happening? I'm thinking it might be a purely IO thing, on the proc box. I have seen similar slowness with the default FreeBSD install on single proc boxes, but a few sysctl's seem to do the trick. a reminder: A - NetAPP B - NetAPP C - NetAPP is fine, im running the same test, and the results are consitant, ~10MGBs so that should eliminate network/switch/port/cable problems, right? btw, no network errors are reported, neither from the hosts nor the switches. danny To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- - Eric Anderson[EMAIL PROTECTED]Centaur Technology # rm -rf /bin/laden - To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
Oh, well, I thought you said 10mb/s, not 10MB/s .. that makes it a bit different. I wonder if it could still be a tcp window size or something.. Try these sysctl's on C: ... i will, but im beginning to believe that it's memory related, not the quantity (they all have over 256M) but quality/bus, but that means hardware, and i have my software hat today :-) thanks, danny To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
heheh.. well, that definitely could be it.. Let me know what you find out.. Eric Danny Braniss wrote: Oh, well, I thought you said 10mb/s, not 10MB/s .. that makes it a bit different. I wonder if it could still be a tcp window size or something.. Try these sysctl's on C: ... i will, but im beginning to believe that it's memory related, not the quantity (they all have over 256M) but quality/bus, but that means hardware, and i have my software hat today :-) thanks, danny -- - Eric Anderson[EMAIL PROTECTED]Centaur Technology # rm -rf /bin/laden - To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
Danny Braniss wrote: setup: host A: dual pentium III/1GHz (Dell 2450) host B: dual pentium III/950MHz (Intel STL2) host C: Pentium III/1GHz (Dell GX150) all connected at 100Mgb full duplex all three are running FreeBSD 4.4. all three have identical troughput when writing to a NetAPP fileserver ~ 10MBs but: B - A: ~ 6MBs C - A: ~ 2MBs Q: why is C - A so bad? show us the output of ifconfig -a on each box. -- -- Daniel Frazier [EMAIL PROTECTED] Tel: 302-239-5900 Ext. 231 Systems AdministratorFax: 302-239-3909 MAGPAGE, We Power the Internet WWW: http://www.magpage.com/ They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. - Benjamin Franklin, Historical Review of Pennsylvania, 1759. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: strange network performace
Eric Anderson wrote: Oh, well, I thought you said 10mb/s, not 10MB/s .. that makes it a bit different. I wonder if it could still be a tcp window size or something.. Try these sysctl's on C: vfs.nfs.gatherdelay=0 vfs.nfs.async=1 vfs.vmiodirenable=1 Alfred stated that he had seen some strangeness with this. kern.ipc.maxsockbuf=2097152 kern.ipc.somaxconn=8192 This won't work, unless it's done at boot time, since that's where the zone sizes are initialized (should be read-only). kern.ipc.maxsockets=16424 Also boot time (should be read-only) -- note that these things are strange, as the UDP and TCP allocations don't occur from the same space, so there are some which size per-protocol structures, while others size per-system structures. The distinction between global vs. protocol specific isn't really clearly documented. kern.maxfiles=65536 Also boot time (should be read-only). kern.maxfilesperproc=32768 Just a soft limit -- can be done any time. net.inet.tcp.rfc1323=1 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=65535 net.inet.tcp.recvspace=65535 net.inet.udp.recvspace=65535 Cranking up the space and the connections simultaneously will make you need more than 2G of memory, max, for the sockets for the window data, should the windows be full. Be very careful, since it's possible to run out of memory and crash the box doing this... for it to work, you'd need to have 4G of RAM, and set the KVA space up to 3G (from the default of 1G). THere are some bugs in /sys/i386/machdep.c as regards overallocation with an alomost full memory map (mostly swap related stuff for page tables that will no longer fit in the address space, along with everything else). This is a good suggestion, but is fraught with peril, if the system networking load goes very high (e.g. DDOS or other attack). net.inet.udp.maxdgram=57344 Probably no change for his application. net.local.stream.recvspace=65535 net.local.stream.sendspace=65535 No change (UNIX domain sockets only). The other suggestions are good. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
