Re: [Cerowrt-devel] bulk packet transmission
The best approach to dealing with locking overhead is to stop thinking that if locks are good, more locking (finer grained locking) is better. OS designers (and Linux designers in particular) are still putting in way too much locking. I deal with this in my day job (we support systems with very large numbers of cpus and because of the fine grained locking obsession, the parallelized capacity is limited). If you do a thoughtful design of your network code, you don't need lots of locking - because TCP/IP streams don't have to interact much - they are quite independent. But instead OS designers spend all their time thinking about doing one thing at a time. There are some really good ideas out there (e.g. RCU) but you have to think about the big picture of networking to understand how to use them. I'm not impressed with the folks who do the Linux networking stacks. On Thursday, October 9, 2014 3:48pm, Dave Taht dave.t...@gmail.com said: I have some hope that the skb-xmit_more API could be used to make aggregating packets in wifi on an AP saner. (my vision for it was that the overlying qdisc would set xmit_more while it still had packets queued up for a given station and then stop and switch to the next. But the rest of the infrastructure ended up pretty closely tied to BQL) Jesper just wrote a nice piece about it also. http://netoptimizer.blogspot.com/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html It was nice to fool around at 10GigE for a while! And netperf-wrapper scales to this speed also! :wow: I do worry that once sch_fq and fq_codel support is added that there will be side effects. I would really like - now that there are al these people profiling things at this level to see profiles including those qdiscs. /me goes grumbling back to thinking about wifi. On Thu, Oct 9, 2014 at 12:40 PM, David Lang da...@lang.hm wrote: lwn.net has an article about a set of new patches that avoid some locking overhead by transmitting multiple packets at once. It doesn't work for things with multiple queues (like fq_codel) in it's current iteration, but it sounds like something that should be looked at and watched for latency related issues. http://lwn.net/Articles/615238/ David Lang ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht https://www.bufferbloat.net/projects/make-wifi-fast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] bulk packet transmission
I've been watching Linux kernel development for a long time and they add locks only when benchmarks show that a lock is causing a bottleneck. They don't just add them because they can. They do also spend a lot of time working to avoid locks. One thing that you are missing is that you are thinking of the TCP/IP system as a single thread of execution, but there's far more going on than that, especially when you have multiple NICs and cores and have lots of interrupts going on. Each TCP/IP stream is not a separate queue of packets in the kernel, instead the details of what threads exist is just a table of information. The packets are all put in a small number of queues to be sent out, and the low-level driver picks the next packet to send from these queues without caring about what TCP/IP stream it's from. David Lang On Fri, 10 Oct 2014, dpr...@reed.com wrote: The best approach to dealing with locking overhead is to stop thinking that if locks are good, more locking (finer grained locking) is better. OS designers (and Linux designers in particular) are still putting in way too much locking. I deal with this in my day job (we support systems with very large numbers of cpus and because of the fine grained locking obsession, the parallelized capacity is limited). If you do a thoughtful design of your network code, you don't need lots of locking - because TCP/IP streams don't have to interact much - they are quite independent. But instead OS designers spend all their time thinking about doing one thing at a time. There are some really good ideas out there (e.g. RCU) but you have to think about the big picture of networking to understand how to use them. I'm not impressed with the folks who do the Linux networking stacks. On Thursday, October 9, 2014 3:48pm, Dave Taht dave.t...@gmail.com said: I have some hope that the skb-xmit_more API could be used to make aggregating packets in wifi on an AP saner. (my vision for it was that the overlying qdisc would set xmit_more while it still had packets queued up for a given station and then stop and switch to the next. But the rest of the infrastructure ended up pretty closely tied to BQL) Jesper just wrote a nice piece about it also. http://netoptimizer.blogspot.com/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html It was nice to fool around at 10GigE for a while! And netperf-wrapper scales to this speed also! :wow: I do worry that once sch_fq and fq_codel support is added that there will be side effects. I would really like - now that there are al these people profiling things at this level to see profiles including those qdiscs. /me goes grumbling back to thinking about wifi. On Thu, Oct 9, 2014 at 12:40 PM, David Lang da...@lang.hm wrote: lwn.net has an article about a set of new patches that avoid some locking overhead by transmitting multiple packets at once. It doesn't work for things with multiple queues (like fq_codel) in it's current iteration, but it sounds like something that should be looked at and watched for latency related issues. http://lwn.net/Articles/615238/ David Lang ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht https://www.bufferbloat.net/projects/make-wifi-fast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
Re: [Cerowrt-devel] bulk packet transmission
I do know that. I would say that benchmarks rarely match real world problems of real systems- they come from sources like academia and technical marketing depts. My job for the last few years has been looking at stems with dozens of processors across 2 and 4 sockets and multiple 10 GigE adapters. There are few benchmarks that look like real workloads. And even smaller systems do very poorly compared to what is possible. Linux is slowly getting better but not so much in the network area at scale. That would take a plan and a rethinking. Beyond incremental tweaks. My opinion ... ymmv. On Oct 10, 2014, David Lang da...@lang.hm wrote: I've been watching Linux kernel development for a long time and they add locks only when benchmarks show that a lock is causing a bottleneck. They don't just add them because they can. They do also spend a lot of time working to avoid locks. One thing that you are missing is that you are thinking of the TCP/IP system as a single thread of execution, but there's far more going on than that, especially when you have multiple NICs and cores and have lots of interrupts going on. Each TCP/IP stream is not a separate queue of packets in the kernel, instead the details of what threads exist is just a table of information. The packets are all put in a small number of queues to be sent out, and the low-level driver picks the next packet to send from these queues without caring about what TCP/IP stream it's from. David Lang On Fri, 10 Oct 2014, dpr...@reed.com wrote: The best approach to dealing with locking overhead is to stop thinking that if locks are good, more locking (finer grained locking) is better. OS designers (and Linux designers in particular) are still putting in way too much locking. I deal with this in my day job (we support systems with very large numbers of cpus and because of the fine grained locking obsession, the parallelized capacity is limited). If you do a thoughtful design of your network code, you don't need lots of locking - because TCP/IP streams don't have to interact much - they are quite independent. But instead OS designers spend all their time thinking about doing one thing at a time. There are some really good ideas out there (e.g. RCU) but you have to think about the big picture of networking to understand how to use them. I'm not impressed with the folks who do the Linux networking stacks. On Thursday, October 9, 2014 3:48pm, Dave Taht dave.t...@gmail.com said: I have some hope that the skb-xmit_more API could be used to make aggregating packets in wifi on an AP saner. (my vision for it was that the overlying qdisc would set xmit_more while it still had packets queued up for a given station and then stop and switch to the next. But the rest of the infrastructure ended up pretty closely tied to BQL) Jesper just wrote a nice piece about it also. http://netoptimizer.blogspot.com/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html It was nice to fool around at 10GigE for a while! And netperf-wrapper scales to this speed also! :wow: I do worry that once sch_fq and fq_codel support is added that there will be side effects. I would really like - now that there are al these people profiling things at this level to see profiles including those qdiscs. /me goes grumbling back to thinking about wifi. On Thu, Oct 9, 2014 at 12:40 PM, David Lang da...@lang.hm wrote: lwn.net has an article about a set of new patches that avoid some locking overhead by transmitting multiple packets at once. It doesn't work for things with multiple queues (like fq_codel) in it's current iteration, but it sounds like something that should be looked at and watched for latency related issues. http://lwn.net/Articles/615238/ David Lang ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Dave Täht https://www.bufferbloat.net/projects/make-wifi-fast ___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel -- Sent from my Android device with K-@ Mail. Please excuse my brevity.___ Cerowrt-devel mailing list Cerowrt-devel@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/cerowrt-devel
