Re: Network throughput: Never get more than 112MB/s über two NICs *SOLVED*
Hi Dan, Am Dienstag, den 12.04.2011, 17:42 -0500 schrieb Dan Nelson: > http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/hardware/module/guide/03instal.html#wpxref23495 you saved our week :-) That was it. My Cisco administrator take a closer look and found out, that both NICs was connected to the same backplane (aha! :-) ) and switched them. Now I get, what I have expected :-) Thank you very very much :-) signature.asc Description: This is a digitally signed message part
Re: Network throughput: Never get more than 112MB/s über two NICs
In the last episode (Apr 12), Dan Nelson said: > In the last episode (Apr 12), Denny Schierz said: > > Am Montag, den 11.04.2011, 21:52 +0200 schrieb Denny Schierz: > > > Am 11.04.2011 um 20:06 schrieb Tim Daneliuk: > > > > Are you certain you are not somehow running active-passive instead of > > > > active-active ... just a thought... > > > > > > 150% sure. I used two dedicated NICs WITHOUT any loadbalancing. The sum > > > has to be more than 112MB/s. > > > > it must me the network. I tested two crossover connections and I've got > > 220MB/s :-) > > Check to see whether your switch ports are oversubscribed (common for older > blade switches, or very high-density blades); sometimes there will be > rectangles enclosing groups of 6-8 ports, which means that they are > controlled by a single chip internally. Moving each of your test machines > to a separate group may improve your performance. .. I missed a line in your original post: > > All are connected through a Cisco Catalyst WS-X4515. This is a supervisor module for a 4500 series chassis, but only has two SFP ports on it. Your servers are unlikely to be plugged into it. They're probably plugged into another module. This page lists some gigabit ethernet modules that oversubscribe their ports, and which ports belong to which groups: http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/hardware/module/guide/03instal.html#wpxref23495 -- Dan Nelson dnel...@allantgroup.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Network throughput: Never get more than 112MB/s über two NICs
On 11 April 2011 22:00, Denny Schierz wrote: > hi, > > after testing severals loadbalancing (LACP) types with Cisco, we saw, > that we never get more than 112MB/s with two network cards and iperf. > > So, we tested without loadbalancing, 4 Clients (iperf -f M -c ) and > two target IPs. Every IP has his own 1Gb/s network card. > On the end, two clients had a connection to IP 1 and the second two to > IP 2. > > First we used the two onboard NICs and then, one onboard and one > external NIC, but without success. We never get more then 112MB/s > > All are connected through a Cisco Catalyst WS-X4515. > > The mainboard is a Intel S3420GP. > > any suggestion? Are you doing LACP from the FreeBSD host? (ie. lagg(4) interface). The current hash just uses the mac and IP addresses (not tcp/udp ports) so you need to make sure your multiple streams have different mac/ip numbers in order to load balance over multiple links. Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Network throughput: Never get more than 112MB/s über two NICs
On Mon, 11.04.2011 at 12:00:39 +0200, Denny Schierz wrote: > hi, > > after testing severals loadbalancing (LACP) types with Cisco, we saw, > that we never get more than 112MB/s with two network cards and iperf. > > So, we tested without loadbalancing, 4 Clients (iperf -f M -c ) and > two target IPs. Every IP has his own 1Gb/s network card. > On the end, two clients had a connection to IP 1 and the second two to > IP 2. > > First we used the two onboard NICs and then, one onboard and one > external NIC, but without success. We never get more then 112MB/s > > All are connected through a Cisco Catalyst WS-X4515. > > The mainboard is a Intel S3420GP. Are the NICs PCI or PCIe? If the former, IIRC the PCI bus bandwidth maxes out at 133MB/s so that might explain your numbers. If your NICs are PCIe, I have no helpful clues, sorry. Uli ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Network throughput: Never get more than 112MB/s über two NICs
In the last episode (Apr 12), Denny Schierz said: > Am Montag, den 11.04.2011, 21:52 +0200 schrieb Denny Schierz: > > Am 11.04.2011 um 20:06 schrieb Tim Daneliuk: > > > Are you certain you are not somehow running active-passive instead of > > > active-active ... just a thought... > > > > 150% sure. I used two dedicated NICs WITHOUT any loadbalancing. The sum > > has to be more than 112MB/s. > > it must me the network. I tested two crossover connections and I've got > 220MB/s :-) Check to see whether your switch ports are oversubscribed (common for older blade switches, or very high-density blades); sometimes there will be rectangles enclosing groups of 6-8 ports, which means that they are controlled by a single chip internally. Moving each of your test machines to a separate group may improve your performance. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Network throughput: Never get more than 112MB/s über two NICs
hi, Am Montag, den 11.04.2011, 21:52 +0200 schrieb Denny Schierz: > hi, > > Am 11.04.2011 um 20:06 schrieb Tim Daneliuk: > > > Are you certain you are not somehow running active-passive instead of > > active-active ... > > just a thought... > > 150% sure. I used two dedicated NICs WITHOUT any loadbalancing. The sum has > to be more than 112MB/s. it must me the network. I tested two crossover connections and I've got 220MB/s :-) cu denny signature.asc Description: This is a digitally signed message part
Re: Network throughput: Never get more than 112MB/s über two NICs
hi, Am 11.04.2011 um 20:06 schrieb Tim Daneliuk: > Are you certain you are not somehow running active-passive instead of > active-active ... > just a thought... 150% sure. I used two dedicated NICs WITHOUT any loadbalancing. The sum has to be more than 112MB/s. cu denny ps. I get every answer two times, one from the list, one from the direct reply, so please answer only to the list :-)___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Network throughput: Never get more than 112MB/s über two NICs
On 4/11/2011 12:55 PM, Denny Schierz said this: > > Am 11.04.2011 um 16:20 schrieb Michael Loftis: > >> Most switches load balance based on MAC addresses, not IP, unless it >> is routing the traffic as a Layer 3 switch then you can enable IP >> based load balancing in some of those. Also you might simply be > > that was the reason, why we disabled the loadbalancer and tested with plain > NICs. > >> reaching the limits of your firewall box too you haven't mentioned any >> of it's specs, nor do you seem to have run top while running the iperf >> tests. > > The clients (who running iperf -c ) had a load near zero, they are > powerful machines (Sun sparcs) with 8 cores and more. The machine, with 4 > Cores (Xeon) who is running "iperf -s", had a load round about ~0.8. > > No firewall etc. between the hosts, just plain network :-) > > cu denny___ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > Are you certain you are not somehow running active-passive instead of active-active ... just a thought... -- Tim Daneliuk tun...@tundraware.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Network throughput: Never get more than 112MB/s über two NICs
Am 11.04.2011 um 16:20 schrieb Michael Loftis: > Most switches load balance based on MAC addresses, not IP, unless it > is routing the traffic as a Layer 3 switch then you can enable IP > based load balancing in some of those. Also you might simply be that was the reason, why we disabled the loadbalancer and tested with plain NICs. > reaching the limits of your firewall box too you haven't mentioned any > of it's specs, nor do you seem to have run top while running the iperf > tests. The clients (who running iperf -c ) had a load near zero, they are powerful machines (Sun sparcs) with 8 cores and more. The machine, with 4 Cores (Xeon) who is running "iperf -s", had a load round about ~0.8. No firewall etc. between the hosts, just plain network :-) cu denny___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
