Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput -SOLVED
On Jun 18, 2016, at 9:13 AM, Jed Clear wrote: On Jun 14, 2016, at 8:50 PM, Jed Clear wrote: > On Jun 9, 2016, at 11:01 PM, Andrew Atrens wrote: >> On 2016-06-09 8:47 PM, Jed Clear wrote: Thanks for the replies so far. Looks like I’ll have to wait until Saturday to test further. Starting with an L2 bridge seems like a good baseline to try. Although will probably take the easier step of just NAT w/o rules first. >>> At it's most basic, an l2 bridge can be created using - >>> >>> ifconfig bridge0 create >>> ifconfig bridge0 addm vr0 addm vr1 up >> >> Had an interesting time getting this working. First no “device if_bridge” >> in my kernel (and nanobsd set to not install any kernel modules). Installed >> a new kernel and configured the bridge. But can’t DHCP across the bridge0. >> Finally had to directly attach the laptop to cable modem, let it DHCP and >> then reinstall the net5501 bridge. At that point I was able to download at >> 83. While directly connected to do the DHCP, the same test got 90. But was >> GbE to the cable modem. So I’m thinking 83 is pretty good for 100BASE-T >> interfaces. >> >> The bridge test didn’t come off until now because I’d forgotten a few real >> life things I had to do. But I did do some more thinking and googling >> during the time away. I don’t think I mentioned that I’m still set up to >> do NAT with natd and ipfw divert. Got to thinking that switching in and out >> of the kernel context a few times a packet might not be too good for >> throughput. So next I’m going to see if I can change that over to ipfw >> kernel NAT. Don’t even recall that there was a kernel nat option when I >> first set this up, many, many moons ago. Probably have to add another >> kernel option…. > > Of course it required a new kernel option. In fact it required two. I will > spare you the tale of figuring the second one out. As many have commented on > other boards, ipfw kernel NAT is not well documented. > > But it was worth it. I now get 82 Mbps download through the 5501, with > essentially the same firewall rule set. I did drop dummynet and the inbound > server NAT rules as I no longer have a static IP and I haven’t decided if I’m > going the dynDNS course or sign up for external hosting/VPS/cloud. And I > believe inbound FTP will no longer be an an option as the “punch” dynamic > rules only work with natd. But FTP is no loss. One loose end, polling. I flipped that back on just now and still tested at 83. With the earlier results, it would imply polling and natd is a very band combination for performance. -Jed ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput -SOLVED
On Jun 14, 2016, at 8:50 PM, Jed Clear wrote: On Jun 9, 2016, at 11:01 PM, Andrew Atrens wrote: > On 2016-06-09 8:47 PM, Jed Clear wrote: >>> Thanks for the replies so far. Looks like I’ll have to wait until Saturday >>> to test further. Starting with an L2 bridge seems like a good baseline to >>> try. Although will probably take the easier step of just NAT w/o rules >>> first. >> At it's most basic, an l2 bridge can be created using - >> >> ifconfig bridge0 create >> ifconfig bridge0 addm vr0 addm vr1 up > > Had an interesting time getting this working. First no “device if_bridge” in > my kernel (and nanobsd set to not install any kernel modules). Installed a > new kernel and configured the bridge. But can’t DHCP across the bridge0. > Finally had to directly attach the laptop to cable modem, let it DHCP and > then reinstall the net5501 bridge. At that point I was able to download at > 83. While directly connected to do the DHCP, the same test got 90. But was > GbE to the cable modem. So I’m thinking 83 is pretty good for 100BASE-T > interfaces. > > The bridge test didn’t come off until now because I’d forgotten a few real > life things I had to do. But I did do some more thinking and googling during > the time away. I don’t think I mentioned that I’m still set up to do NAT > with natd and ipfw divert. Got to thinking that switching in and out of the > kernel context a few times a packet might not be too good for throughput. So > next I’m going to see if I can change that over to ipfw kernel NAT. Don’t > even recall that there was a kernel nat option when I first set this up, > many, many moons ago. Probably have to add another kernel option…. Of course it required a new kernel option. In fact it required two. I will spare you the tale of figuring the second one out. As many have commented on other boards, ipfw kernel NAT is not well documented. But it was worth it. I now get 82 Mbps download through the 5501, with essentially the same firewall rule set. I did drop dummynet and the inbound server NAT rules as I no longer have a static IP and I haven’t decided if I’m going the dynDNS course or sign up for external hosting/VPS/cloud. And I believe inbound FTP will no longer be an an option as the “punch” dynamic rules only work with natd. But FTP is no loss. -Jed ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
On Jun 9, 2016, at 11:01 PM, Andrew Atrens wrote: On 2016-06-09 8:47 PM, Jed Clear wrote: >> Thanks for the replies so far. Looks like I’ll have to wait until Saturday >> to test further. Starting with an L2 bridge seems like a good baseline to >> try. Although will probably take the easier step of just NAT w/o rules >> first. > At it's most basic, an l2 bridge can be created using - > > ifconfig bridge0 create > ifconfig bridge0 addm vr0 addm vr1 up Had an interesting time getting this working. First no “device if_bridge” in my kernel (and nanobsd set to not install any kernel modules). Installed a new kernel and configured the bridge. But can’t DHCP across the bridge0. Finally had to directly attach the laptop to cable modem, let it DHCP and then reinstall the net5501 bridge. At that point I was able to download at 83. While directly connected to do the DHCP, the same test got 90. But was GbE to the cable modem. So I’m thinking 83 is pretty good for 100BASE-T interfaces. The bridge test didn’t come off until now because I’d forgotten a few real life things I had to do. But I did do some more thinking and googling during the time away. I don’t think I mentioned that I’m still set up to do NAT with natd and ipfw divert. Got to thinking that switching in and out of the kernel context a few times a packet might not be too good for throughput. So next I’m going to see if I can change that over to ipfw kernel NAT. Don’t even recall that there was a kernel nat option when I first set this up, many, many moons ago. Probably have to add another kernel option…. -Jed ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
Andrew Atrens [and...@atrens.ca] wrote: > BUGS > The vr driver always copies transmit mbuf chains into longword-aligned > buffers prior to transmission in order to pacify the Rhine chips. If > buffers are not aligned correctly, the chip will round the supplied > buffer address and begin DMAing from the wrong location. This buffer > copying impairs transmit performance on slower systems but cannot be > avoided. On faster machines (e.g. a Pentium II), the performance > impact > is much less noticeable. This is not true for the version of the chip included with Soekris and PC Engines boxes, nor is it true for the OpenBSD or FreeBSD drivers. OpenBSD's vr driver has interrupt mitigation and hw vlan tagging that FreeBSD does (did?) not. Both enable hw checksum capability for IPv4. ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
On 2016-06-10, Jed Clear wrote: >> The vr driver always copies transmit mbuf chains into longword-aligned >> buffers prior to transmission in order to pacify the Rhine chips. If >> buffers are not aligned correctly, the chip will round the supplied >> buffer address and begin DMAing from the wrong location. This buffer >> copying impairs transmit performance on slower systems but cannot be >> avoided. On faster machines (e.g. a Pentium II), the performance >> impact >> is much less noticeable. > > Interesting. You'd think the drivers would align the packets on Rx and I > vaguely recall that FreeBSD managed to go zero copy a few major versions ago. > Although that could be for straight forwarding. IPFW and NAT might behave > differently. Not sure I'd have any control over it. The manpage is outdated, this is only done for the old chips. It's not needed and isn't done for VT610x. ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
On 2016-06-10 7:11 AM, Jed Clear wrote: >> On Jun 9, 2016, at 11:01 PM, Andrew Atrens wrote: >>> On 2016-06-09 8:47 PM, Jed Clear wrote: >>> With the current set up, I ran top during the download. Never got lower >>> than 25% idle time on the CPU. ~30% system and and 40+% interrupt. 384M >>> (of 512M) Free on memory, so no issue there. So doesn’t seem to be pegging >>> the CPU with my full rule set. >> That's interesting. The relationship between cpu use and throughput is >> pretty linear. You should be able to 'peg' your cpu unless something >> (ipfw?) is somehow throttling. You don't have, by chance, 'options >> DUMMYNET' configured? > My regular rules do have dummy net queues on the uplink, which are now OBE. > And I had initially overlooked the bandwidth setting there. In any event, the > built in "simple" rule set doesn't I think. Hmm .. a number of years ago (maybe around 2008) I experimented with ipfw and have a vague memory of seeing some hardcoded macros in use in the code. These would be day-1 things because they were present in the DragonFlyBSD fork back then. > >> It might also be an tcp-ack-prioritization issue. You did mention that >> your uplink speed was kind of crappy. Uplink saturation will affect >> downlink speed if tcp acks aren't getting upstream quickly enough. >> Maybe ipfw is somehow exacerbating that. > 5-6Mbps, crappy only in that $BIG_CABLE_CO disabled 5 of the 8 uplink > channels when they "provisioned" my cable modem. That should be oodles of bandwidth. > > >> If the l2 bridge thing works and you're able to peg your net5501 cpu >> then, notwithstanding vr driver fixes you're probably at the limit. >> >> BUGS >> The vr driver always copies transmit mbuf chains into longword-aligned >> buffers prior to transmission in order to pacify the Rhine chips. If >> buffers are not aligned correctly, the chip will round the supplied >> buffer address and begin DMAing from the wrong location. This buffer >> copying impairs transmit performance on slower systems but cannot be >> avoided. On faster machines (e.g. a Pentium II), the performance >> impact >> is much less noticeable. > Interesting. You'd think the drivers would align the packets on Rx and I > vaguely recall that FreeBSD managed to go zero copy a few major versions ago. Yes, it should be zero copy (or nearly always zero copy). I add that caveat as the ipfw code has been somewhat neglected. > Although that could be for straight forwarding. Yes. In the forwarding case there should be no need to modify the mbuf. > IPFW and NAT might behave differently. Not sure I'd have any control over > it. > > Also recalled that I was running ntpd and named (caching) on the 5501. > Turning off ntpd added 5 to the download speed. Turning both off didn't > increase that. Good to have those datapoints .. not surprised that named caching wouldn't help for raw throughput. But I am surprised that disabling ntpd did help. > Didn't think to unplug the GPS which causes a 1 PPS DCD interrupt as well as > the 4800 bps serial interrupts. Will give that a try, too. I have the PPS > option in the kernel if anyone knows of an interaction. Well, PPS stuff could impact netisr which might affect receive processing. There is a neat little tool called 'netstrain' - http://netstrain.sourceforge.net that would likely be helpful. It's small enough to build statically and run on nanobsd. You could run it directly on the 5501 to isolate tx vs rx performance. I think it uses udp which would help figure out if there's any weirdness around tcp, mtu, etc. > > -Jed > ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
> On Jun 9, 2016, at 11:01 PM, Andrew Atrens wrote: >> On 2016-06-09 8:47 PM, Jed Clear wrote: >> With the current set up, I ran top during the download. Never got lower >> than 25% idle time on the CPU. ~30% system and and 40+% interrupt. 384M >> (of 512M) Free on memory, so no issue there. So doesn’t seem to be pegging >> the CPU with my full rule set. > That's interesting. The relationship between cpu use and throughput is > pretty linear. You should be able to 'peg' your cpu unless something > (ipfw?) is somehow throttling. You don't have, by chance, 'options > DUMMYNET' configured? My regular rules do have dummy net queues on the uplink, which are now OBE. And I had initially overlooked the bandwidth setting there. In any event, the built in "simple" rule set doesn't I think. > It might also be an tcp-ack-prioritization issue. You did mention that > your uplink speed was kind of crappy. Uplink saturation will affect > downlink speed if tcp acks aren't getting upstream quickly enough. > Maybe ipfw is somehow exacerbating that. 5-6Mbps, crappy only in that $BIG_CABLE_CO disabled 5 of the 8 uplink channels when they "provisioned" my cable modem. > If the l2 bridge thing works and you're able to peg your net5501 cpu > then, notwithstanding vr driver fixes you're probably at the limit. > > BUGS > The vr driver always copies transmit mbuf chains into longword-aligned > buffers prior to transmission in order to pacify the Rhine chips. If > buffers are not aligned correctly, the chip will round the supplied > buffer address and begin DMAing from the wrong location. This buffer > copying impairs transmit performance on slower systems but cannot be > avoided. On faster machines (e.g. a Pentium II), the performance > impact > is much less noticeable. Interesting. You'd think the drivers would align the packets on Rx and I vaguely recall that FreeBSD managed to go zero copy a few major versions ago. Although that could be for straight forwarding. IPFW and NAT might behave differently. Not sure I'd have any control over it. Also recalled that I was running ntpd and named (caching) on the 5501. Turning off ntpd added 5 to the download speed. Turning both off didn't increase that. Didn't think to unplug the GPS which causes a 1 PPS DCD interrupt as well as the 4800 bps serial interrupts. Will give that a try, too. I have the PPS option in the kernel if anyone knows of an interaction. -Jed ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
On 2016-06-09 8:47 PM, Jed Clear wrote: > Thanks for the replies so far. Looks like I’ll have to wait until Saturday > to test further. Starting with an L2 bridge seems like a good baseline to > try. Although will probably take the easier step of just NAT w/o rules first. At it's most basic, an l2 bridge can be created using - ifconfig bridge0 create ifconfig bridge0 addm vr0 addm vr1 up There are a number of options and settings but the defaults work pretty well in most setups . You may want to consult ifconfig - https://www.freebsd.org/cgi/man.cgi?query=ifconfig&apropos=0&sektion=0&manpath=FreeBSD+9.0-RELEASE&arch=default&format=html > Switching to pfsense or a 6501 were already further down my option list, so > agree with those. > > Had not considered just a change to pf, but will mull over. Although if I’m > going to learn a new syntax, might as well get the pretty interface to go > with it (pfsense) assuming I can get the performance. pf works well for l3 stuff - I quite like it. > > > With the current set up, I ran top during the download. Never got lower than > 25% idle time on the CPU. ~30% system and and 40+% interrupt. 384M (of > 512M) Free on memory, so no issue there. So doesn’t seem to be pegging the > CPU with my full rule set. That's interesting. The relationship between cpu use and throughput is pretty linear. You should be able to 'peg' your cpu unless something (ipfw?) is somehow throttling. You don't have, by chance, 'options DUMMYNET' configured? It might also be an tcp-ack-prioritization issue. You did mention that your uplink speed was kind of crappy. Uplink saturation will affect downlink speed if tcp acks aren't getting upstream quickly enough. Maybe ipfw is somehow exacerbating that. If the l2 bridge thing works and you're able to peg your net5501 cpu then, notwithstanding vr driver fixes you're probably at the limit. BUGS The vr driver always copies transmit mbuf chains into longword-aligned buffers prior to transmission in order to pacify the Rhine chips. If buffers are not aligned correctly, the chip will round the supplied buffer address and begin DMAing from the wrong location. This buffer copying impairs transmit performance on slower systems but cannot be avoided. On faster machines (e.g. a Pentium II), the performance impact is much less noticeable. > > -Jed > > On Jun 8, 2016, at 8:18 PM, Jed Clear wrote: > >> I just climbed out of the bronze age of home networking (DSL) and now have >> "75Mbps service” from $BIG_CABLE_CO (iron age?). Before the DSL was the >> bottle neck. Now it appears the 5501 is the bottle neck. My net5501-70 has >> long been running nanobsd (FreeBSD 9.3-R) and ipfw as my perimeter >> router-firewall-nat. While I’m not expecting 75, especially in the evening, >> it’s not even close. Note all the speeds mentioned are download speeds in >> Mbps. The upload is much worse, but not bothered by that in this exercise. >> >> When the cable modem was first brought up, a laptop directly on it pulled 56 >> with one of the speed test sites. The cable modem channel power and SNR >> don’t look bad. Putting the 5501 in-line dropped the speed to the 30s. >> Some googling later and I discover FreeBSD’s polling feature. So I added >> options DEVICE_POLLING to the kernel config (HZ was already 1000), baked a >> new image, set all the interfaces to polling and … it dropped like a rock to >> 5 Mbps. Flipping off polling on the three interfaces brought it back to the >> 30s. >> >> I tried the built in “simple” firewall rule set, and that did modestly >> better than my, perhaps overly complicated, rule set. It got around 44. I >> will work that later. >> >> Anyway I’m a bit baffled by the negative results when enabling polling. And >> any other advice on improving the performance through the 5501 would be >> appreciated. I haven’t given up on self help, but need a break from google >> for a bit so will appeal to the collective wisdom of soekris-tech. >> >> Thanks, >> >> -Jed >> >> PS: To add insult to injury, I just repeated the directly connected laptop >> experiment and clocked over 90. :-( >> >> ___ >> Soekris-tech mailing list >> Soekris-tech@lists.soekris.com >> http://lists.soekris.com/mailman/listinfo/soekris-tech >> > ___ > Soekris-tech mailing list > Soekris-tech@lists.soekris.com > http://lists.soekris.com/mailman/listinfo/soekris-tech ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
Thanks for the replies so far. Looks like I’ll have to wait until Saturday to test further. Starting with an L2 bridge seems like a good baseline to try. Although will probably take the easier step of just NAT w/o rules first. Switching to pfsense or a 6501 were already further down my option list, so agree with those. Had not considered just a change to pf, but will mull over. Although if I’m going to learn a new syntax, might as well get the pretty interface to go with it (pfsense) assuming I can get the performance. With the current set up, I ran top during the download. Never got lower than 25% idle time on the CPU. ~30% system and and 40+% interrupt. 384M (of 512M) Free on memory, so no issue there. So doesn’t seem to be pegging the CPU with my full rule set. -Jed On Jun 8, 2016, at 8:18 PM, Jed Clear wrote: > I just climbed out of the bronze age of home networking (DSL) and now have > "75Mbps service” from $BIG_CABLE_CO (iron age?). Before the DSL was the > bottle neck. Now it appears the 5501 is the bottle neck. My net5501-70 has > long been running nanobsd (FreeBSD 9.3-R) and ipfw as my perimeter > router-firewall-nat. While I’m not expecting 75, especially in the evening, > it’s not even close. Note all the speeds mentioned are download speeds in > Mbps. The upload is much worse, but not bothered by that in this exercise. > > When the cable modem was first brought up, a laptop directly on it pulled 56 > with one of the speed test sites. The cable modem channel power and SNR > don’t look bad. Putting the 5501 in-line dropped the speed to the 30s. Some > googling later and I discover FreeBSD’s polling feature. So I added options > DEVICE_POLLING to the kernel config (HZ was already 1000), baked a new image, > set all the interfaces to polling and … it dropped like a rock to 5 Mbps. > Flipping off polling on the three interfaces brought it back to the 30s. > > I tried the built in “simple” firewall rule set, and that did modestly better > than my, perhaps overly complicated, rule set. It got around 44. I will > work that later. > > Anyway I’m a bit baffled by the negative results when enabling polling. And > any other advice on improving the performance through the 5501 would be > appreciated. I haven’t given up on self help, but need a break from google > for a bit so will appeal to the collective wisdom of soekris-tech. > > Thanks, > > -Jed > > PS: To add insult to injury, I just repeated the directly connected laptop > experiment and clocked over 90. :-( > > ___ > Soekris-tech mailing list > Soekris-tech@lists.soekris.com > http://lists.soekris.com/mailman/listinfo/soekris-tech > ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
On 2016-06-09, Christopher Hilton wrote: > At the end of the day I think that 75 ~ 85 MBit's per second is the = > limit on the vr interface in net5501. Bandwidth is only part of the equation; packets-per-second matter too. Even if you get to 75-85Mb/s I'd still consider that as a stopgap, if you get a lot of small packets you run out of steam again. > If you only have one firewall/router I'd replace it with either = > a Net6501 or some other Intel Atom/PCIe/Intel Gigabit based solution. If looking at other vendors, don't restrict yourself to boards with Intel CPUs, there are also some small boxes using AMD GX-412TC and Intel I210AT NICs (em) that work quite nicely. ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
> On Jun 8, 2016, at 8:18 PM, Jed Clear wrote: > > I just climbed out of the bronze age of home networking (DSL) and now have > "75Mbps service” from $BIG_CABLE_CO (iron age?). Before the DSL was the > bottle neck. Now it appears the 5501 is the bottle neck. My net5501-70 has > long been running nanobsd (FreeBSD 9.3-R) and ipfw as my perimeter > router-firewall-nat. While I’m not expecting 75, especially in the evening, > it’s not even close. Note all the speeds mentioned are download speeds in > Mbps. The upload is much worse, but not bothered by that in this exercise. > > When the cable modem was first brought up, a laptop directly on it pulled 56 > with one of the speed test sites. The cable modem channel power and SNR > don’t look bad. Putting the 5501 in-line dropped the speed to the 30s. Some > googling later and I discover FreeBSD’s polling feature. So I added options > DEVICE_POLLING to the kernel config (HZ was already 1000), baked a new image, > set all the interfaces to polling and … it dropped like a rock to 5 Mbps. > Flipping off polling on the three interfaces brought it back to the 30s. > > I tried the built in “simple” firewall rule set, and that did modestly better > than my, perhaps overly complicated, rule set. It got around 44. I will > work that later. > > Anyway I’m a bit baffled by the negative results when enabling polling. And > any other advice on improving the performance through the 5501 would be > appreciated. I haven’t given up on self help, but need a break from google > for a bit so will appeal to the collective wisdom of soekris-tech. > > Thanks, > > -Jed > > PS: To add insult to injury, I just repeated the directly connected laptop > experiment and clocked over 90. :-( > I don’t know about FreeBSD/ipfw but on OpenBSD/pf and the latest performance tweaks to both the vr driver and the pf firewall the best I could do with a Net5501-70, pf, and the vr driver based nics was 85Mbit/s. If I understand correctly, FreeBSD’s vr driver is more performant than OpenBSD’s but that may have changed. In 5.8, OpenBSD’s pf is much more performant than pf in FreeBSD 9-* and less buggy. Again, I don’t know about FreeBSD/ipfw. If you aren’t reaching 75Mbit/s now on the Net5501-70 you might be able to do so either by switching to pfSense or by switching to OpenBSD. SUMMARY At the end of the day I think that 75 ~ 85 MBit’s per second is the limit on the vr interface in net5501. I don’t know what the limit is on the em interface in the Net6501 because between the 1Gbit speed of the NIC and the PCIe bus I can’t afford to buy enough bandwidth to get close. If you only have one firewall/router I’d replace it with either a Net6501 or some other Intel Atom/PCIe/Intel Gigabit based solution. DETAILS: At the end of the day, I solved this by throwing money at the problem in three steps: First I replace my Net5501 with a Net6501. That changed the ethernet driver from vr to em and the em driver is much more performant. I have two firewalls serially so moving to the Net6501 just moved the problem upstream in my network. So, Second, to address that, I put the Net5501-70 into a Soekris Rackmount case for the better power supply and put a dual em interface into a net5501-70 for the second firewall. This worked and was stable under OpenBSD 5.2. It became biweekly unstable when I upgraded the OS From OpenBSD 5.2 to OpenBSD 5.6 and to 5.8 meaning that I never saw an uptime greater than 14 days out of the OpenBSD 5.6/5.8 box with the dual em card in it. This box would spontaneously reboot under heavy traffic and I could never figure out the reason. I speculate that under heavy traffic loads the power supply can’t keep up with the PCI dual em card. Thus the dual em hangs the PCI bus and ultimately triggers the watchdog reboot on the Net5501-70. Finally, I replaced the Net6501 with a 1U SuperMicro D525 Atom and moved the Net6501 upstream replacing the net5501. Since then the only reason that either firewall goes down is because I rebooted it. Hope this helps, -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*).___o..___..o...ooO..._ Christopher Sean Hilton[chris/at/vindaloo/dot/com] signature.asc Description: Message signed with OpenPGP using GPGMail ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
Re: [Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
I would try with a simple l2 bridge set up across two of the interfaces, run your tests and see if the cpu on the soekris is pegged. If it is, then the network driver is likely the bottleneck. Depending on the driver there might be some tunables available via sysctl. If not (if cpu is moderately loaded and throughput is good) then it is the ipfw framework. Have you experimented at all with pf? On 2016-06-08 8:18 PM, Jed Clear wrote: > I just climbed out of the bronze age of home networking (DSL) and now have > "75Mbps service” from $BIG_CABLE_CO (iron age?). Before the DSL was the > bottle neck. Now it appears the 5501 is the bottle neck. My net5501-70 has > long been running nanobsd (FreeBSD 9.3-R) and ipfw as my perimeter > router-firewall-nat. While I’m not expecting 75, especially in the evening, > it’s not even close. Note all the speeds mentioned are download speeds in > Mbps. The upload is much worse, but not bothered by that in this exercise. > > When the cable modem was first brought up, a laptop directly on it pulled 56 > with one of the speed test sites. The cable modem channel power and SNR > don’t look bad. Putting the 5501 in-line dropped the speed to the 30s. Some > googling later and I discover FreeBSD’s polling feature. So I added options > DEVICE_POLLING to the kernel config (HZ was already 1000), baked a new image, > set all the interfaces to polling and … it dropped like a rock to 5 Mbps. > Flipping off polling on the three interfaces brought it back to the 30s. > > I tried the built in “simple” firewall rule set, and that did modestly better > than my, perhaps overly complicated, rule set. It got around 44. I will > work that later. > > Anyway I’m a bit baffled by the negative results when enabling polling. And > any other advice on improving the performance through the 5501 would be > appreciated. I haven’t given up on self help, but need a break from google > for a bit so will appeal to the collective wisdom of soekris-tech. > > Thanks, > > -Jed > > PS: To add insult to injury, I just repeated the directly connected laptop > experiment and clocked over 90. :-( > > ___ > Soekris-tech mailing list > Soekris-tech@lists.soekris.com > http://lists.soekris.com/mailman/listinfo/soekris-tech ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
[Soekris] net5501: FreeBSD ipfw and the elusive 75Mbps throughput
I just climbed out of the bronze age of home networking (DSL) and now have "75Mbps service” from $BIG_CABLE_CO (iron age?). Before the DSL was the bottle neck. Now it appears the 5501 is the bottle neck. My net5501-70 has long been running nanobsd (FreeBSD 9.3-R) and ipfw as my perimeter router-firewall-nat. While I’m not expecting 75, especially in the evening, it’s not even close. Note all the speeds mentioned are download speeds in Mbps. The upload is much worse, but not bothered by that in this exercise. When the cable modem was first brought up, a laptop directly on it pulled 56 with one of the speed test sites. The cable modem channel power and SNR don’t look bad. Putting the 5501 in-line dropped the speed to the 30s. Some googling later and I discover FreeBSD’s polling feature. So I added options DEVICE_POLLING to the kernel config (HZ was already 1000), baked a new image, set all the interfaces to polling and … it dropped like a rock to 5 Mbps. Flipping off polling on the three interfaces brought it back to the 30s. I tried the built in “simple” firewall rule set, and that did modestly better than my, perhaps overly complicated, rule set. It got around 44. I will work that later. Anyway I’m a bit baffled by the negative results when enabling polling. And any other advice on improving the performance through the 5501 would be appreciated. I haven’t given up on self help, but need a break from google for a bit so will appeal to the collective wisdom of soekris-tech. Thanks, -Jed PS: To add insult to injury, I just repeated the directly connected laptop experiment and clocked over 90. :-( ___ Soekris-tech mailing list Soekris-tech@lists.soekris.com http://lists.soekris.com/mailman/listinfo/soekris-tech
