Hi, I am trying to run tcp connection between two thumpers that have pci-x "Sun 10 Gigabit Ethernet PCI-X Adapters (X5544A-4)" connected via a Cisco 4900M switch with 10GigabitE optics. Jumbo Packets are configured on all interfaces.
I cannot seem to get more than 320Mbps across the link when doing on the boxes: hostA$ cat /dev/zero | pv | nc -v hostB 3333 hostB$ nc -l -p 3333 > /dev/null I have attempted to tune the TCP window size and buffers and ring buffers with the following in /etc/system set ndd:tcp_naglim_def=1 set ndd:udp_recv_hiwat=4000000 set ndd:udp_xmit_hiwat=4000000 set ndd:tcp_recv_hiwat=4000000 set ndd:tcp_xmit_hiwat=4000000 set ndd:tcp_max_buf=2097152 set ndd:tcp_cwnd_max=2097152 set ip_squeue_soft_ring=1 set ip:ip_soft_rings_cnt=4 set ip:ip_squeue_fanout=1 set ip:ip_squeue_bind=0 set ip:tcp_squeue_wput=1 set ddi_msix_alloc_limit=8 set pcplusmp:apic_multi_msi_max=8 set pcplusmp:apic_msix_max=8 set pcplusmp:apic_intr_policy=1 Host A is running Sol10u5 and host B is running opensolaris 2008.05 The output of intrstat shows that the ixgb interrupts are always landing on the same cpu. Does anyone have any tips to help tune this link for higher transfer speeds and if/how to tune the ixgb interrupt moderation and/or type as well as the ring buffers for this card? - sjw This message posted from opensolaris.org _______________________________________________ networking-discuss mailing list [email protected]
