[pfSense Support] Re: unknown cause of limited throughput
On Tue, Jul 5, 2011 at 11:52 PM, David Burgess wrote: > I'll probably kick myself when I figure this one out And the answer is... traffic shaper. I'm so embarrassed. ::Off to kick self:: db - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Re: unknown cause of limited throughput
On Thu, Jul 14, 2011 at 4:39 AM, Ermal Luçi wrote: > Try to tune these sysctl: > net.isr.numthreads: 1 > net.isr.bindthreads: 0 > net.isr.direct: 1 > net.isr.direct_force: 1 I tried those in System: Advanced: System Tunables. Throughput is still 17.4 Mbps between vlan240 and any other. Does pfsense require a reboot to make those sysctl effective? db - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Re: unknown cause of limited throughput
On 7/14/11 12:46 PM, David Burgess wrote: On Thu, Jul 14, 2011 at 11:56 AM, Adam Thompson wrote: Are you passing the VLAN tags all the way into the pfSense VM on a single vNIC, or are you splitting the VLANs at the vSwitch level and passing them into multiple vNICs on the pfSense VM? Adam, Thanks for the info. In fact, pfsense is not virtualized here, so in my most recent posting I was able to eliminate virtual machines from the problem altogether by testing from ren to mule, and passes only through pfsense and one vlan switch (twice, on different ports). Ermal, Thanks for the hints. I will test and post back. db - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org I use slower cables out to the Internet and faster ones on my internal network. Mehma - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Re: unknown cause of limited throughput
On Thu, Jul 14, 2011 at 11:56 AM, Adam Thompson wrote: > Are you passing the VLAN tags all the way into the pfSense VM on a single > vNIC, or are you splitting the VLANs at the vSwitch level and passing them > into multiple vNICs on the pfSense VM? Adam, Thanks for the info. In fact, pfsense is not virtualized here, so in my most recent posting I was able to eliminate virtual machines from the problem altogether by testing from ren to mule, and passes only through pfsense and one vlan switch (twice, on different ports). Ermal, Thanks for the hints. I will test and post back. db - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
RE: [pfSense Support] Re: unknown cause of limited throughput
Are you passing the VLAN tags all the way into the pfSense VM on a single vNIC, or are you splitting the VLANs at the vSwitch level and passing them into multiple vNICs on the pfSense VM? I found that every layer of software that inspected VLAN tags diminished my throughput by a factor of 10, so allowing ESXi to split the VLANs into multiple vNICs was much, much faster than allowing the VLAN tags to propagate through to the VM. -Adam Thompson athom...@athompso.net > -Original Message- > From: David Burgess [mailto:apt@gmail.com] > Sent: Thursday, July 14, 2011 01:27 > To: support > Subject: [pfSense Support] Re: unknown cause of limited throughput > > 2.0-RC3 (amd64) > built on Tue Jul 12 21:23:55 EDT 2011 > > On Tue, Jul 5, 2011 at 11:52 PM, David Burgess > wrote: > > > I hope that's not too confusing. To summarize, any two machines, > real > > or virtual, get iperf results near wire speed when on the same L2 > > network. Any two machines on different (routed) networks see > iperf > > speeds between 320 and 550, which is expected due to the > limitations > > of the router. The exception is rip. Of my three virtual hosts, > which > > all live on the same ESXi server, only rip is seeing very slow > iperf > > speeds (and similar nfs speeds) when acting as server to routed > hosts. > > I did some more testing and was surprised by the results. I created > a new virtual server "chunk" running Ubuntu Server 10.10 and > expected that because it was now the same version OS as my other > servers, it would now exhibit normal routed network speeds. But I > was wrong. Chunk consistently serves iperf at 12.8 Mbps to a routed > client. > > Intrigued, I moved chunk to a different local vlan/network and > tested again. The result: > > iperf client vlanserver vlan result > renreal85chunk virtual250 380 Mbps routed > renreal85chunk virtual240 12.8 Mbps routed > mule real85chunk virtual250 380 Mbps routed > mule real85chunk virtual240 12.8 Mbps routed > ren real85 mule real 240 16.8 Mbps routed > > So it's not the server, it's the vlan or something related to it. > vlan85 is my LAN, and the only firewall rule on that interface is a > PASS all rule. There is no floating rule that should touch any of > this as far as I can tell. > > The only thing that distinguishes vlan 240 from the other vlans I'm > testing (besides being slower) is that the hosts on this vlan have > publicly routable IP addresses, while the hosts on every other vlan > are 192.168.x.x addresses. There is no NAT occurring between local > networks. > > I've now ruled out virtualization and OS as being the cause of > this, and that leaves pfsense and the switch. The switch is not > slow where the router is not involved, so unless I've misjudged, > this is a pfsense problem. > > Any ideas? > > db > > --- > -- > To unsubscribe, e-mail: support-unsubscr...@pfsense.com For > additional commands, e-mail: support-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Re: unknown cause of limited throughput
On Thu, Jul 14, 2011 at 8:26 AM, David Burgess wrote: > 2.0-RC3 (amd64) > built on Tue Jul 12 21:23:55 EDT 2011 > > On Tue, Jul 5, 2011 at 11:52 PM, David Burgess wrote: > >> I hope that's not too confusing. To summarize, any two machines, real >> or virtual, get iperf results near wire speed when on the same L2 >> network. Any two machines on different (routed) networks see iperf >> speeds between 320 and 550, which is expected due to the limitations >> of the router. The exception is rip. Of my three virtual hosts, which >> all live on the same ESXi server, only rip is seeing very slow iperf >> speeds (and similar nfs speeds) when acting as server to routed hosts. > > I did some more testing and was surprised by the results. I created a > new virtual server "chunk" running Ubuntu Server 10.10 and expected > that because it was now the same version OS as my other servers, it > would now exhibit normal routed network speeds. But I was wrong. Chunk > consistently serves iperf at 12.8 Mbps to a routed client. > > Intrigued, I moved chunk to a different local vlan/network and tested > again. The result: > > iperf client vlan server vlan result > ren real 85 chunk virtual 250 380 Mbps routed > ren real 85 chunk virtual 240 12.8 Mbps routed > mule real 85 chunk virtual 250 380 Mbps routed > mule real 85 chunk virtual 240 12.8 Mbps routed > ren real 85 mule real 240 16.8 Mbps routed > > So it's not the server, it's the vlan or something related to it. > vlan85 is my LAN, and the only firewall rule on that interface is a > PASS all rule. There is no floating rule that should touch any of this > as far as I can tell. > > The only thing that distinguishes vlan 240 from the other vlans I'm > testing (besides being slower) is that the hosts on this vlan have > publicly routable IP addresses, while the hosts on every other vlan > are 192.168.x.x addresses. There is no NAT occurring between local > networks. > > I've now ruled out virtualization and OS as being the cause of this, > and that leaves pfsense and the switch. The switch is not slow where > the router is not involved, so unless I've misjudged, this is a > pfsense problem. > > Any ideas? > Try to tune these sysctl: net.isr.numthreads: 1 net.isr.bindthreads: 0 net.isr.direct: 1 net.isr.direct_force: 1 latest pfSense ships with net.isr.direct disable and net.isr.bindthreads enabled. It creates isr threads for each cpu it finds. Possibly you can try the above values and see if they improve your problem. > db > > - > To unsubscribe, e-mail: support-unsubscr...@pfsense.com > For additional commands, e-mail: support-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org > > -- Ermal - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
