Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Johannes Berg writes: > On Thu, 2018-11-15 at 09:10 -0800, Toke Høiland-Jørgensen wrote: >> Louie Lu writes: >> >> > Hi Rajkumar, Toke, >> > >> > I found the series (v3,4/6) remove the debugfs remove reset station's >> > airtime method, and didn't added at here. >> > >> > Not sure how to help this kind of situation, do I need a separate >> > patch to fix this, or posting the patch here is fine? >> >> This is fine; we can fold it into the next version. Thanks :) > > Just FYI - I'm going to assume, given this comment and the long > discussion, that there will be a next version :) Yes. Got caught up in moving before I managed to get another version out. Will get around to it eventually, promise! :) -Toke ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Johannes Berg writes: > On Thu, 2018-11-15 at 09:10 -0800, Toke Høiland-Jørgensen wrote: >> Louie Lu writes: >> >> > Hi Rajkumar, Toke, >> > >> > I found the series (v3,4/6) remove the debugfs remove reset >> > station's >> > airtime method, and didn't added at here. >> > >> > Not sure how to help this kind of situation, do I need a separate >> > patch to fix this, or posting the patch here is fine? >> >> This is fine; we can fold it into the next version. Thanks :) > > Just FYI - I'm going to assume, given this comment and the long > discussion, that there will be a next version :) I think toke's still sleeping off the post graduation hangover. > > johannes > > ___ > Make-wifi-fast mailing list > make-wifi-f...@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On Thu, 2018-11-15 at 09:10 -0800, Toke Høiland-Jørgensen wrote: > Louie Lu writes: > > > Hi Rajkumar, Toke, > > > > I found the series (v3,4/6) remove the debugfs remove reset station's > > airtime method, and didn't added at here. > > > > Not sure how to help this kind of situation, do I need a separate > > patch to fix this, or posting the patch here is fine? > > This is fine; we can fold it into the next version. Thanks :) Just FYI - I'm going to assume, given this comment and the long discussion, that there will be a next version :) johannes ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Felix Fietkau writes:
>> diff --git a/net/mac80211/status.c b/net/mac80211/status.c
>> index aa4afbf0abaf..a1f1256448f5 100644
>> --- a/net/mac80211/status.c
>> +++ b/net/mac80211/status.c
>> @@ -818,6 +818,12 @@ static void __ieee80211_tx_status(struct ieee80211_hw
>> *hw,
>> ieee80211_sta_tx_notify(sta->sdata, (void *) skb->data,
>> acked, info->status.tx_time);
>>
>> +if (info->status.tx_time &&
>> +wiphy_ext_feature_isset(local->hw.wiphy,
>> +
>> NL80211_EXT_FEATURE_AIRTIME_FAIRNESS))
>> +ieee80211_sta_register_airtime(&sta->sta, tid,
>> + info->status.tx_time, 0);
>> +
>> if (ieee80211_hw_check(&local->hw, REPORTS_TX_ACK_STATUS)) {
>> if (info->flags & IEEE80211_TX_STAT_ACK) {
>> if (sta->status_stats.lost_packets)
> I think the same is needed in ieee80211_tx_status_ext.
So finally circled back to this. In ieee80211_tx_status_ext() we don't
have an skb, so we don't know which TID the packet was sent to; what
airtime information would the driver actually provide in this case? Is
it an aggregate of all ACs, or?
-Toke
___
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On Mon, 19 Nov 2018, Dave Taht wrote: I'm not sure if this was a fluke or not, but at Starbucks recently I sat outside, right next to their window, and could not scan their AP at all. Previously, I sat inside, 3 feet away through the glass, and got great signal. I wonder what that was all about! Maybe special tinting that blocks RF? Or just dumb luck of some sort. Ya know, I could definitely see a market for a material like that! I'd like it for my car, so bluetooth wouldn't escape. That would break your tire pressure sensors (each car is rolling around broadcasting 4 unique bluetooth IDs, not hard to track) David Lang ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On Mon, Nov 19, 2018 at 4:52 PM Simon Barber wrote: > > Low-e glass, it’s a thin metallic film used to reflect infra-red to keep heat > in or out. Totally blocks/reflects RF. Very cool. I imagine it's hell on cell too? I can see this stuff becoming very popular in places where keeping the good wifi in is important. Could cover floors and ceilings with it to. Cars could be tempest rated... /me goes looking for stock to buy > Simon > > On Nov 19, 2018, at 4:20 PM, Ben Greear wrote: > > On 11/19/2018 04:13 PM, Dave Taht wrote: > > On Mon, Nov 19, 2018 at 3:56 PM Ben Greear wrote: > > > On 11/19/2018 03:47 PM, Dave Taht wrote: > > On Mon, Nov 19, 2018 at 3:30 PM Simon Barber wrote: > > > > > On Nov 19, 2018, at 2:44 PM, Toke Høiland-Jørgensen wrote: > > Dave Taht writes: > > Toke Høiland-Jørgensen writes: > > Felix Fietkau writes: > > On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: > > This part doesn't really make much sense to me, but maybe I'm > misunderstanding how the code works. > Let's assume we have a driver like ath9k or mt76, which tries to keep a > > …. > > > Well, there's going to be a BQL-like queue limit (but for airtime) on > top, which drivers can opt-in to if the hardware has too much queueing. > > > Very happy to read this - I first talked to Dave Taht about the need for Time > Queue Limits more than 5 years ago! > > > Michal faked up a dql estimator 3 (?) years ago. it worked. > > http://blog.cerowrt.org/post/dql_on_wifi_2/ > > As a side note, in *any* real world working mu-mimo situation at any > scale, on any equipment, does anyone have any stats on how often the > feature is actually used and useful? > > My personal guess, from looking at the standard, was in home > scenarios, usage would be about... 0, and in a controlled environment > in a football stadium, quite a lot. > > In a office or apartment complex, I figured interference and so forth > would make it a negative benefit due to retransmits. > > I felt when that part of the standard rolled around... that mu-mimo > was an idea that should never have escaped the lab. I can be convinced > by data, that we can aim for a higher goal here. But it would be > comforting to have a measured non-lab, real-world, at real world > rates, result for it, on some platform, of it actually being useful. > > > We're working on building a lab with 20 or 30 mixed 'real' devices > using various different /AC NICs (QCA wave2 on OpenWRT, Fedora, realtek USB > 8812au on OpenWRT, Fedora, > and some Intel NICs in NUCs on Windows, and maybe more). I'm not actually > sure if that realtek > or the NUCs can do MU-MIMO or not, but the QCA NICs will be able to. It > should be at least somewhat similar > to a classroom environment or coffee shop. > > > In the last 3 coffee shops I went to, I could hear over 30 APs on > competing SSIDs, running G, N, and AC, > occupying every available channel. > > > I especially like when someone uses channel 3 because, I guess, they > think it is un-used :) > > I'm not sure if this was a fluke or not, but at Starbucks recently I sat > outside, > right next to their window, and could not scan their AP at all. Previously, > I sat > inside, 3 feet away through the glass, and got great signal. I wonder what > that was > all about! Maybe special tinting that blocks RF? Or just dumb luck of some > sort. > > Thanks, > Ben > > > -- > Ben Greear > Candela Technologies Inc http://www.candelatech.com > > -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On Mon, Nov 19, 2018 at 4:20 PM Ben Greear wrote: > > On 11/19/2018 04:13 PM, Dave Taht wrote: > > On Mon, Nov 19, 2018 at 3:56 PM Ben Greear wrote: > >> > >> On 11/19/2018 03:47 PM, Dave Taht wrote: > >>> On Mon, Nov 19, 2018 at 3:30 PM Simon Barber wrote: > > > > On Nov 19, 2018, at 2:44 PM, Toke Høiland-Jørgensen wrote: > > Dave Taht writes: > > Toke Høiland-Jørgensen writes: > > Felix Fietkau writes: > > On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: > > This part doesn't really make much sense to me, but maybe I'm > misunderstanding how the code works. > Let's assume we have a driver like ath9k or mt76, which tries to keep a > > …. > > > Well, there's going to be a BQL-like queue limit (but for airtime) on > top, which drivers can opt-in to if the hardware has too much queueing. > > > Very happy to read this - I first talked to Dave Taht about the need for > Time Queue Limits more than 5 years ago! > >>> > >>> Michal faked up a dql estimator 3 (?) years ago. it worked. > >>> > >>> http://blog.cerowrt.org/post/dql_on_wifi_2/ > >>> > >>> As a side note, in *any* real world working mu-mimo situation at any > >>> scale, on any equipment, does anyone have any stats on how often the > >>> feature is actually used and useful? > >>> > >>> My personal guess, from looking at the standard, was in home > >>> scenarios, usage would be about... 0, and in a controlled environment > >>> in a football stadium, quite a lot. > >>> > >>> In a office or apartment complex, I figured interference and so forth > >>> would make it a negative benefit due to retransmits. > >>> > >>> I felt when that part of the standard rolled around... that mu-mimo > >>> was an idea that should never have escaped the lab. I can be convinced > >>> by data, that we can aim for a higher goal here. But it would be > >>> comforting to have a measured non-lab, real-world, at real world > >>> rates, result for it, on some platform, of it actually being useful. > >> > >> We're working on building a lab with 20 or 30 mixed 'real' devices > >> using various different /AC NICs (QCA wave2 on OpenWRT, Fedora, realtek > >> USB 8812au on OpenWRT, Fedora, > >> and some Intel NICs in NUCs on Windows, and maybe more). I'm not actually > >> sure if that realtek > >> or the NUCs can do MU-MIMO or not, but the QCA NICs will be able to. It > >> should be at least somewhat similar > >> to a classroom environment or coffee shop. > > > > In the last 3 coffee shops I went to, I could hear over 30 APs on > > competing SSIDs, running G, N, and AC, > > occupying every available channel. > > I especially like when someone uses channel 3 because, I guess, they > think it is un-used :) I think avery actually found a case where that was a benefit, in an apartment building that had each per-apartment AP located at exactly the same place on every floor. I do wish I could go back in time and explain the four colour theorem to whoever allocated the 2.4ghz wifi band, and then explain even that was a planar rather than 3D problem. The 3D coloring problem I visualize as the scene in Aliens 2, where the monsters just waltz in over the ceiling panels. To me it's an indelible image kind of superimposed over the interfering waves in the air, full of gore, goo, and blood... > > I'm not sure if this was a fluke or not, but at Starbucks recently I sat > outside, > right next to their window, and could not scan their AP at all. Previously, > I sat > inside, 3 feet away through the glass, and got great signal. I wonder what > that was > all about! Maybe special tinting that blocks RF? Or just dumb luck of some > sort. Ya know, I could definitely see a market for a material like that! I'd like it for my car, so bluetooth wouldn't escape. anyway, just blowing off steam. :) When v4 of this rolls around + BQL? I can get some results back on it too. One less than perfect option or the other would be better than continuing to have lousy wifi in all cases on all platforms. > Thanks, > Ben > > > -- > Ben Greear > Candela Technologies Inc http://www.candelatech.com > -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On 11/19/2018 04:13 PM, Dave Taht wrote: On Mon, Nov 19, 2018 at 3:56 PM Ben Greear wrote: On 11/19/2018 03:47 PM, Dave Taht wrote: On Mon, Nov 19, 2018 at 3:30 PM Simon Barber wrote: On Nov 19, 2018, at 2:44 PM, Toke Høiland-Jørgensen wrote: Dave Taht writes: Toke Høiland-Jørgensen writes: Felix Fietkau writes: On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: This part doesn't really make much sense to me, but maybe I'm misunderstanding how the code works. Let's assume we have a driver like ath9k or mt76, which tries to keep a …. Well, there's going to be a BQL-like queue limit (but for airtime) on top, which drivers can opt-in to if the hardware has too much queueing. Very happy to read this - I first talked to Dave Taht about the need for Time Queue Limits more than 5 years ago! Michal faked up a dql estimator 3 (?) years ago. it worked. http://blog.cerowrt.org/post/dql_on_wifi_2/ As a side note, in *any* real world working mu-mimo situation at any scale, on any equipment, does anyone have any stats on how often the feature is actually used and useful? My personal guess, from looking at the standard, was in home scenarios, usage would be about... 0, and in a controlled environment in a football stadium, quite a lot. In a office or apartment complex, I figured interference and so forth would make it a negative benefit due to retransmits. I felt when that part of the standard rolled around... that mu-mimo was an idea that should never have escaped the lab. I can be convinced by data, that we can aim for a higher goal here. But it would be comforting to have a measured non-lab, real-world, at real world rates, result for it, on some platform, of it actually being useful. We're working on building a lab with 20 or 30 mixed 'real' devices using various different /AC NICs (QCA wave2 on OpenWRT, Fedora, realtek USB 8812au on OpenWRT, Fedora, and some Intel NICs in NUCs on Windows, and maybe more). I'm not actually sure if that realtek or the NUCs can do MU-MIMO or not, but the QCA NICs will be able to. It should be at least somewhat similar to a classroom environment or coffee shop. In the last 3 coffee shops I went to, I could hear over 30 APs on competing SSIDs, running G, N, and AC, occupying every available channel. I especially like when someone uses channel 3 because, I guess, they think it is un-used :) I'm not sure if this was a fluke or not, but at Starbucks recently I sat outside, right next to their window, and could not scan their AP at all. Previously, I sat inside, 3 feet away through the glass, and got great signal. I wonder what that was all about! Maybe special tinting that blocks RF? Or just dumb luck of some sort. Thanks, Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On Mon, Nov 19, 2018 at 3:56 PM Ben Greear wrote: > > On 11/19/2018 03:47 PM, Dave Taht wrote: > > On Mon, Nov 19, 2018 at 3:30 PM Simon Barber wrote: > >> > >> > >> > >> On Nov 19, 2018, at 2:44 PM, Toke Høiland-Jørgensen wrote: > >> > >> Dave Taht writes: > >> > >> Toke Høiland-Jørgensen writes: > >> > >> Felix Fietkau writes: > >> > >> On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: > >> > >> This part doesn't really make much sense to me, but maybe I'm > >> misunderstanding how the code works. > >> Let's assume we have a driver like ath9k or mt76, which tries to keep a > >> > >> …. > >> > >> > >> Well, there's going to be a BQL-like queue limit (but for airtime) on > >> top, which drivers can opt-in to if the hardware has too much queueing. > >> > >> > >> Very happy to read this - I first talked to Dave Taht about the need for > >> Time Queue Limits more than 5 years ago! > > > > Michal faked up a dql estimator 3 (?) years ago. it worked. > > > > http://blog.cerowrt.org/post/dql_on_wifi_2/ > > > > As a side note, in *any* real world working mu-mimo situation at any > > scale, on any equipment, does anyone have any stats on how often the > > feature is actually used and useful? > > > > My personal guess, from looking at the standard, was in home > > scenarios, usage would be about... 0, and in a controlled environment > > in a football stadium, quite a lot. > > > > In a office or apartment complex, I figured interference and so forth > > would make it a negative benefit due to retransmits. > > > > I felt when that part of the standard rolled around... that mu-mimo > > was an idea that should never have escaped the lab. I can be convinced > > by data, that we can aim for a higher goal here. But it would be > > comforting to have a measured non-lab, real-world, at real world > > rates, result for it, on some platform, of it actually being useful. > > We're working on building a lab with 20 or 30 mixed 'real' devices > using various different /AC NICs (QCA wave2 on OpenWRT, Fedora, realtek USB > 8812au on OpenWRT, Fedora, > and some Intel NICs in NUCs on Windows, and maybe more). I'm not actually > sure if that realtek > or the NUCs can do MU-MIMO or not, but the QCA NICs will be able to. It > should be at least somewhat similar > to a classroom environment or coffee shop. In the last 3 coffee shops I went to, I could hear over 30 APs on competing SSIDs, running G, N, and AC, occupying every available channel. > I'll let you know what we find > as far as how well MU-MIMO improves things or not. Thank you. My lab is shut down and I'm selling off the gear, so all I can do anymore is be grumpy. > > At least in simple test cases (one 1x1 stations, one 2x2 station, with 4x4 > MU-MIMO AP), > it works very well for increased download throughput. Is that a UDP or TCP test? My specific "most important" benchmark for wifi is web PLT with lots of stations. That's the most common wifi use case, IMHO. Some videoconferencing, voip, ssh, etc. One big upload, one big download, going on somewhere, sometimes. Slashdot for me is 78 separate ssl'd tcp flows, a dozen + dns lookups, over 2.2 seconds. bulk downloads rarely figure in except for streaming video, and that peaks out generally at 20Mbit/sec for most people. So while I do like the potential in mu-mimo to, schedule lower service times on a per station basis, I see too many variable rates and interference in the real world, and not enough simultaneity between schedulable stations, and a ton of acks, for me to imagine there's much of a real-world difference from mu-mimo in anything other than a well managed office or stadium. People going for the max download throughput with the max number of stations... PLT, PLT is a way better benchmark. I can certainly be convinced by data, and I look forward to your experiments. > In home setups, I'd guess that the DSL or Cable Modem or other uplink is the > bottleneck Offices too. Wifi - aside from interference and range issues lowering the rate, only becomes the bottleneck at internet speeds > 40Mbits/sec. We are certainly starting to see wifi become more of the bottleneck, while running at rates, generally, far lower than what people bench at. > way more often than the wifi is, even if your are just running /n. But, > maybe that is just > my experience living out at the end of a long skinny phone line all these > years. Fiber networks and newer cable connections now crack 100mbits, and there you do see all that nifty fq_codel-ly chocolatey goodness kicking in but I do tend to think the optimization focus should be at low rates with lots of stations for plt more than bandwidth. > Thanks, > Ben > > > > -- > Ben Greear > Candela Technologies Inc http://www.candelatech.com > -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On 11/19/2018 03:47 PM, Dave Taht wrote: On Mon, Nov 19, 2018 at 3:30 PM Simon Barber wrote: On Nov 19, 2018, at 2:44 PM, Toke Høiland-Jørgensen wrote: Dave Taht writes: Toke Høiland-Jørgensen writes: Felix Fietkau writes: On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: This part doesn't really make much sense to me, but maybe I'm misunderstanding how the code works. Let's assume we have a driver like ath9k or mt76, which tries to keep a …. Well, there's going to be a BQL-like queue limit (but for airtime) on top, which drivers can opt-in to if the hardware has too much queueing. Very happy to read this - I first talked to Dave Taht about the need for Time Queue Limits more than 5 years ago! Michal faked up a dql estimator 3 (?) years ago. it worked. http://blog.cerowrt.org/post/dql_on_wifi_2/ As a side note, in *any* real world working mu-mimo situation at any scale, on any equipment, does anyone have any stats on how often the feature is actually used and useful? My personal guess, from looking at the standard, was in home scenarios, usage would be about... 0, and in a controlled environment in a football stadium, quite a lot. In a office or apartment complex, I figured interference and so forth would make it a negative benefit due to retransmits. I felt when that part of the standard rolled around... that mu-mimo was an idea that should never have escaped the lab. I can be convinced by data, that we can aim for a higher goal here. But it would be comforting to have a measured non-lab, real-world, at real world rates, result for it, on some platform, of it actually being useful. We're working on building a lab with 20 or 30 mixed 'real' devices using various different /AC NICs (QCA wave2 on OpenWRT, Fedora, realtek USB 8812au on OpenWRT, Fedora, and some Intel NICs in NUCs on Windows, and maybe more). I'm not actually sure if that realtek or the NUCs can do MU-MIMO or not, but the QCA NICs will be able to. It should be at least somewhat similar to a classroom environment or coffee shop. I'll let you know what we find as far as how well MU-MIMO improves things or not. At least in simple test cases (one 1x1 stations, one 2x2 station, with 4x4 MU-MIMO AP), it works very well for increased download throughput. In home setups, I'd guess that the DSL or Cable Modem or other uplink is the bottleneck way more often than the wifi is, even if your are just running /n. But, maybe that is just my experience living out at the end of a long skinny phone line all these years. Thanks, Ben -- Ben Greear Candela Technologies Inc http://www.candelatech.com ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On Mon, Nov 19, 2018 at 3:30 PM Simon Barber wrote: > > > > On Nov 19, 2018, at 2:44 PM, Toke Høiland-Jørgensen wrote: > > Dave Taht writes: > > Toke Høiland-Jørgensen writes: > > Felix Fietkau writes: > > On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: > > This part doesn't really make much sense to me, but maybe I'm > misunderstanding how the code works. > Let's assume we have a driver like ath9k or mt76, which tries to keep a > > …. > > > Well, there's going to be a BQL-like queue limit (but for airtime) on > top, which drivers can opt-in to if the hardware has too much queueing. > > > Very happy to read this - I first talked to Dave Taht about the need for Time > Queue Limits more than 5 years ago! Michal faked up a dql estimator 3 (?) years ago. it worked. http://blog.cerowrt.org/post/dql_on_wifi_2/ As a side note, in *any* real world working mu-mimo situation at any scale, on any equipment, does anyone have any stats on how often the feature is actually used and useful? My personal guess, from looking at the standard, was in home scenarios, usage would be about... 0, and in a controlled environment in a football stadium, quite a lot. In a office or apartment complex, I figured interference and so forth would make it a negative benefit due to retransmits. I felt when that part of the standard rolled around... that mu-mimo was an idea that should never have escaped the lab. I can be convinced by data, that we can aim for a higher goal here. But it would be comforting to have a measured non-lab, real-world, at real world rates, result for it, on some platform, of it actually being useful. > Simon > > ___ > Make-wifi-fast mailing list > make-wifi-f...@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast -- Dave Täht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740 ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Toke Høiland-Jørgensen writes: > Dave Taht writes: > >> Toke Høiland-Jørgensen writes: >> >>> Felix Fietkau writes: >>> On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: >> This part doesn't really make much sense to me, but maybe I'm >> misunderstanding how the code works. >> Let's assume we have a driver like ath9k or mt76, which tries to keep a >> number of aggregates in the hardware queue, and the hardware queue is >> currently empty. >> If the current txq entry is kept at the head of the schedule list, >> wouldn't the code just pull from that one over and over again, until >> enough packets are transmitted by the hardware and their tx status >> processed? >> It seems to me that while fairness is still preserved in the long run, >> this could lead to rather bursty scheduling, which may not be >> particularly latency friendly. > > Yes, it'll be a bit more bursty when the hardware queue is completely > empty. However, when a TX completion comes back, that will adjust the > deficit of that sta and cause it to be rotated on the next dequeue. This > obviously relies on the fact that the lower-level hardware queue is > sufficiently shallow to not add a lot of latency. But we want that to be > the case anyway. In practice, it works quite well for ath9k, but not so > well for ath10k because it has a large buffer in firmware. > > If we requeue the TXQ at the end of the list, a station that is taking > up too much airtime will fail to be throttled properly, so the > queue-at-head is kinda needed to ensure fairness... Thanks for the explanation, that makes sense to me. I have an idea on how to mitigate the burstiness within the driver. I'll write it down in pseudocode, please let me know if you think that'll work. >>> >>> I don't think it will, unfortunately. For example, consider the case >>> where there are two stations queued; one with a large negative deficit >>> (say, -10ms), and one with a positive deficit. >> >> Perhaps a flag for one way or the other? >> >> if(driver->has_absurd_hardware_queue_depth) doitthisway(); else >> doitabetterway(); > > Well, there's going to be a BQL-like queue limit (but for airtime) on > top, which drivers can opt-in to if the hardware has too much queueing. > >>> In this case, we really need to throttle the station with a negative >>> deficit. But if the driver loops and caches txqs, we'll get something >>> like the following: >>> >>> - First driver loop iteration: returns TXQ with positive deficit. >>> - Second driver loop iteration: Only the negative-deficit TXQ is in the >>> mac80211 list, so it will loop until that TXQ's deficit turns positive >>> and return it. >>> >>> Because of this, the negative-deficit station won't be throttled, and we >>> won't get fairness. >>> >>> How many frames will mt76 queue up below the driver point? I.e., how >>> much burstiness are you expecting this will introduce on that driver? >>> >>> Taking a step back, it's clear that it would be good to be able to >>> dequeue packets to multiple STAs at once (we need that for MU-MIMO on >>> ath10k as well). However, I don't think we can do that with the >>> round-robin fairness scheduler; so we are going to need a different >>> algorithm. I *think* it may be possible to do this with a virtual-time >>> scheduler, but I haven't sat down and worked out the details yet... >> >> The answer to which did not fit on the margins of your thesis. :) >> >> I too have been trying to come up with a better means of gang >> scheduling... for about 2 years now. In terms of bitmaps it looks a bit >> like QFQ, but honestly... > > It's not the gang scheduling we need, deciding which devices to send to > at once is generally done in firmware anyway. I have a long held dream that one day some firmware will be able to send an interrupt and some information along... "Hi, I'll be done transmitting/receiving in about 1ms, here's who I think I can talk to next, and here's who else I maybe could gang schedule". That would let us get away from 5ms wasted in the "ready to go" portion of the algo, and share the highest likelyhood "groups" with the higher layer. > We just need to be able to > dequeue packets for more than one station when possible. And a huge fantasy is in some future 802.11ZZZ standard the on-board firmware and the linux drivers can be co-designed, even, dare I say, open sourced, to better evolve to meet real world requirements. mbox's per station would be nice, with scatter/gather I/O... I can think of a zillion things I'd want the firmware to handle (other than buffering) > I don't think > we need the fancy bitmap stuff from QFQ since we don't have that many > stations to schedule at once; so we can probably live with O(log(n)) in > the number of active stations. Best of two or three "groups", per above, from the firmware. >> Is there going to be some point where whatever we h
Re: [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Hi Felix
Thinking a bit more about this, I think that rather than having the
driver work around the API as in your example...
> do {
> struct ieee80211_txq *pending_txq[4];
> int n_pending_txq = 0;
> int i;
>
> if (hwq->pending < 4)
> break;p
>
> nframes = 0;
>
> ieee80211_txq_schedule_start(hw, ac)
> do {
> bool requeue = false;
>
> struct ieee80211_txq *txq;
>
> txq = ieee80211_next_txq(hw, ac);
> if (!txq)
> break;
>
> nframes += schedule_txq(txq, &requeue);
> if (requeue)
> pending_txq[n_pending_txq++] = txq;
>
> } while (n_pending_txq < ARRAY_SIZE(pending_txq));
>
> for (i = n_pending_txq; i > 0; i--)
> ieee80211_return_txq(hw, pending_txq[i - 1]);
>
> ieee80211_txq_schedule_end(hw, ac)
> } while (nframes);
... really what we want is that the driver can just do this:
ieee80211_txq_schedule_start(hw, ac);
while ((txq = ieee80211_next_txq(hw, ac)) {
schedule_txq(txq, &requeue);
return_txq(txq);
}
ieee80211_txq_schedule_end(hw, ac);
and expect so get through all eligible TXQs. Note that there will be
cases where there is only a single eligible TXQ (such as the example I
gave in the other email); in which case the current version is fine. But
there is (probably) also going to be cases where more than one TXQ is
eligible at the same time, which we cannot handle with the current RR
scheduler.
However, I think that assuming we can get the scheduler to guarantee
that it will return all eligible TXQs between each pair of calls to
schedule_start()/schedule_end(), we should be fine with the current API.
Do you agree with this?
-Toke
___
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Dave Taht writes: > Toke Høiland-Jørgensen writes: > >> Felix Fietkau writes: >> >>> On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: > This part doesn't really make much sense to me, but maybe I'm > misunderstanding how the code works. > Let's assume we have a driver like ath9k or mt76, which tries to keep a > number of aggregates in the hardware queue, and the hardware queue is > currently empty. > If the current txq entry is kept at the head of the schedule list, > wouldn't the code just pull from that one over and over again, until > enough packets are transmitted by the hardware and their tx status > processed? > It seems to me that while fairness is still preserved in the long run, > this could lead to rather bursty scheduling, which may not be > particularly latency friendly. Yes, it'll be a bit more bursty when the hardware queue is completely empty. However, when a TX completion comes back, that will adjust the deficit of that sta and cause it to be rotated on the next dequeue. This obviously relies on the fact that the lower-level hardware queue is sufficiently shallow to not add a lot of latency. But we want that to be the case anyway. In practice, it works quite well for ath9k, but not so well for ath10k because it has a large buffer in firmware. If we requeue the TXQ at the end of the list, a station that is taking up too much airtime will fail to be throttled properly, so the queue-at-head is kinda needed to ensure fairness... >>> Thanks for the explanation, that makes sense to me. I have an idea on >>> how to mitigate the burstiness within the driver. I'll write it down in >>> pseudocode, please let me know if you think that'll work. >> >> I don't think it will, unfortunately. For example, consider the case >> where there are two stations queued; one with a large negative deficit >> (say, -10ms), and one with a positive deficit. > > Perhaps a flag for one way or the other? > > if(driver->has_absurd_hardware_queue_depth) doitthisway(); else > doitabetterway(); Well, there's going to be a BQL-like queue limit (but for airtime) on top, which drivers can opt-in to if the hardware has too much queueing. >> In this case, we really need to throttle the station with a negative >> deficit. But if the driver loops and caches txqs, we'll get something >> like the following: >> >> - First driver loop iteration: returns TXQ with positive deficit. >> - Second driver loop iteration: Only the negative-deficit TXQ is in the >> mac80211 list, so it will loop until that TXQ's deficit turns positive >> and return it. >> >> Because of this, the negative-deficit station won't be throttled, and we >> won't get fairness. >> >> How many frames will mt76 queue up below the driver point? I.e., how >> much burstiness are you expecting this will introduce on that driver? >> >> Taking a step back, it's clear that it would be good to be able to >> dequeue packets to multiple STAs at once (we need that for MU-MIMO on >> ath10k as well). However, I don't think we can do that with the >> round-robin fairness scheduler; so we are going to need a different >> algorithm. I *think* it may be possible to do this with a virtual-time >> scheduler, but I haven't sat down and worked out the details yet... > > The answer to which did not fit on the margins of your thesis. :) > > I too have been trying to come up with a better means of gang > scheduling... for about 2 years now. In terms of bitmaps it looks a bit > like QFQ, but honestly... It's not the gang scheduling we need, deciding which devices to send to at once is generally done in firmware anyway. We just need to be able to dequeue packets for more than one station when possible. I don't think we need the fancy bitmap stuff from QFQ since we don't have that many stations to schedule at once; so we can probably live with O(log(n)) in the number of active stations. > Is there going to be some point where whatever we have here is > significantly better than what we had? Or not significantly worse? Or > handwavy enough to fix the rest once enlightenment arrives? > > The perfect is the enemy of the good. Well, what we have now works for ath9k, works reasonably well for ath10k in pull mode, not so well for ath10k in push mode, and then there's Felix' comments in this thread... > I'd rather like the intel folk to be weighing in on this stuff, too, > trying to get an API right requires use cases. Johannes has already reviewed a previous version, and I do believe he said he'd review it again once we have converged on something :) -Toke ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Toke Høiland-Jørgensen writes: > Felix Fietkau writes: > >> On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: This part doesn't really make much sense to me, but maybe I'm misunderstanding how the code works. Let's assume we have a driver like ath9k or mt76, which tries to keep a number of aggregates in the hardware queue, and the hardware queue is currently empty. If the current txq entry is kept at the head of the schedule list, wouldn't the code just pull from that one over and over again, until enough packets are transmitted by the hardware and their tx status processed? It seems to me that while fairness is still preserved in the long run, this could lead to rather bursty scheduling, which may not be particularly latency friendly. >>> >>> Yes, it'll be a bit more bursty when the hardware queue is completely >>> empty. However, when a TX completion comes back, that will adjust the >>> deficit of that sta and cause it to be rotated on the next dequeue. This >>> obviously relies on the fact that the lower-level hardware queue is >>> sufficiently shallow to not add a lot of latency. But we want that to be >>> the case anyway. In practice, it works quite well for ath9k, but not so >>> well for ath10k because it has a large buffer in firmware. >>> >>> If we requeue the TXQ at the end of the list, a station that is taking >>> up too much airtime will fail to be throttled properly, so the >>> queue-at-head is kinda needed to ensure fairness... >> Thanks for the explanation, that makes sense to me. I have an idea on >> how to mitigate the burstiness within the driver. I'll write it down in >> pseudocode, please let me know if you think that'll work. > > I don't think it will, unfortunately. For example, consider the case > where there are two stations queued; one with a large negative deficit > (say, -10ms), and one with a positive deficit. Perhaps a flag for one way or the other? if(driver->has_absurd_hardware_queue_depth) doitthisway(); else doitabetterway(); > > In this case, we really need to throttle the station with a negative > deficit. But if the driver loops and caches txqs, we'll get something > like the following: > > - First driver loop iteration: returns TXQ with positive deficit. > - Second driver loop iteration: Only the negative-deficit TXQ is in the > mac80211 list, so it will loop until that TXQ's deficit turns positive > and return it. > > Because of this, the negative-deficit station won't be throttled, and we > won't get fairness. > > How many frames will mt76 queue up below the driver point? I.e., how > much burstiness are you expecting this will introduce on that driver? > > Taking a step back, it's clear that it would be good to be able to > dequeue packets to multiple STAs at once (we need that for MU-MIMO on > ath10k as well). However, I don't think we can do that with the > round-robin fairness scheduler; so we are going to need a different > algorithm. I *think* it may be possible to do this with a virtual-time > scheduler, but I haven't sat down and worked out the details yet... The answer to which did not fit on the margins of your thesis. :) I too have been trying to come up with a better means of gang scheduling... for about 2 years now. In terms of bitmaps it looks a bit like QFQ, but honestly... Is there going to be some point where whatever we have here is significantly better than what we had? Or not significantly worse? Or handwavy enough to fix the rest once enlightenment arrives? The perfect is the enemy of the good. I'd rather like the intel folk to be weighing in on this stuff, too, trying to get an API right requires use cases. > > -Toke > ___ > Make-wifi-fast mailing list > make-wifi-f...@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/make-wifi-fast ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Felix Fietkau writes: > On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote: >>> This part doesn't really make much sense to me, but maybe I'm >>> misunderstanding how the code works. >>> Let's assume we have a driver like ath9k or mt76, which tries to keep a >>> number of aggregates in the hardware queue, and the hardware queue is >>> currently empty. >>> If the current txq entry is kept at the head of the schedule list, >>> wouldn't the code just pull from that one over and over again, until >>> enough packets are transmitted by the hardware and their tx status >>> processed? >>> It seems to me that while fairness is still preserved in the long run, >>> this could lead to rather bursty scheduling, which may not be >>> particularly latency friendly. >> >> Yes, it'll be a bit more bursty when the hardware queue is completely >> empty. However, when a TX completion comes back, that will adjust the >> deficit of that sta and cause it to be rotated on the next dequeue. This >> obviously relies on the fact that the lower-level hardware queue is >> sufficiently shallow to not add a lot of latency. But we want that to be >> the case anyway. In practice, it works quite well for ath9k, but not so >> well for ath10k because it has a large buffer in firmware. >> >> If we requeue the TXQ at the end of the list, a station that is taking >> up too much airtime will fail to be throttled properly, so the >> queue-at-head is kinda needed to ensure fairness... > Thanks for the explanation, that makes sense to me. I have an idea on > how to mitigate the burstiness within the driver. I'll write it down in > pseudocode, please let me know if you think that'll work. I don't think it will, unfortunately. For example, consider the case where there are two stations queued; one with a large negative deficit (say, -10ms), and one with a positive deficit. In this case, we really need to throttle the station with a negative deficit. But if the driver loops and caches txqs, we'll get something like the following: - First driver loop iteration: returns TXQ with positive deficit. - Second driver loop iteration: Only the negative-deficit TXQ is in the mac80211 list, so it will loop until that TXQ's deficit turns positive and return it. Because of this, the negative-deficit station won't be throttled, and we won't get fairness. How many frames will mt76 queue up below the driver point? I.e., how much burstiness are you expecting this will introduce on that driver? Taking a step back, it's clear that it would be good to be able to dequeue packets to multiple STAs at once (we need that for MU-MIMO on ath10k as well). However, I don't think we can do that with the round-robin fairness scheduler; so we are going to need a different algorithm. I *think* it may be possible to do this with a virtual-time scheduler, but I haven't sat down and worked out the details yet... -Toke ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Louie Lu writes: > Hi Rajkumar, Toke, > > I found the series (v3,4/6) remove the debugfs remove reset station's > airtime method, and didn't added at here. > > Not sure how to help this kind of situation, do I need a separate > patch to fix this, or posting the patch here is fine? This is fine; we can fold it into the next version. Thanks :) -Toke ___ ath10k mailing list ath10k@lists.infradead.org http://lists.infradead.org/mailman/listinfo/ath10k
Re: [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On 2018-11-14 18:40, Toke Høiland-Jørgensen wrote:
>> This part doesn't really make much sense to me, but maybe I'm
>> misunderstanding how the code works.
>> Let's assume we have a driver like ath9k or mt76, which tries to keep a
>> number of aggregates in the hardware queue, and the hardware queue is
>> currently empty.
>> If the current txq entry is kept at the head of the schedule list,
>> wouldn't the code just pull from that one over and over again, until
>> enough packets are transmitted by the hardware and their tx status
>> processed?
>> It seems to me that while fairness is still preserved in the long run,
>> this could lead to rather bursty scheduling, which may not be
>> particularly latency friendly.
>
> Yes, it'll be a bit more bursty when the hardware queue is completely
> empty. However, when a TX completion comes back, that will adjust the
> deficit of that sta and cause it to be rotated on the next dequeue. This
> obviously relies on the fact that the lower-level hardware queue is
> sufficiently shallow to not add a lot of latency. But we want that to be
> the case anyway. In practice, it works quite well for ath9k, but not so
> well for ath10k because it has a large buffer in firmware.
>
> If we requeue the TXQ at the end of the list, a station that is taking
> up too much airtime will fail to be throttled properly, so the
> queue-at-head is kinda needed to ensure fairness...
Thanks for the explanation, that makes sense to me. I have an idea on
how to mitigate the burstiness within the driver. I'll write it down in
pseudocode, please let me know if you think that'll work.
do {
struct ieee80211_txq *pending_txq[4];
int n_pending_txq = 0;
int i;
if (hwq->pending < 4)
break;
nframes = 0;
ieee80211_txq_schedule_start(hw, ac)
do {
bool requeue = false;
struct ieee80211_txq *txq;
txq = ieee80211_next_txq(hw, ac);
if (!txq)
break;
nframes += schedule_txq(txq, &requeue);
if (requeue)
pending_txq[n_pending_txq++] = txq;
} while (n_pending_txq < ARRAY_SIZE(pending_txq));
for (i = n_pending_txq; i > 0; i--)
ieee80211_return_txq(hw, pending_txq[i - 1]);
ieee80211_txq_schedule_end(hw, ac)
} while (nframes);
- Felix
___
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k
Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Hi Rajkumar, Toke,
I found the series (v3,4/6) remove the debugfs remove reset station's
airtime method, and didn't added at here.
Not sure how to help this kind of situation, do I need a separate
patch to fix this, or posting the patch here is fine?
From 3a4a856c397345311c9d7f3679828cadc40e6a80 Mon Sep 17 00:00:00 2001
From: Louie Lu
Date: Thu, 15 Nov 2018 16:13:57 +0800
Subject: [PATCH] mac80211: Add reset for station's airtime
Let user can reset station airtime status by debugfs, it will
reset all airtime deficit to `sta->airtime_weight` and reset rx/tx
airtime accumulate to 0.
diff --git a/net/mac80211/debugfs_sta.c b/net/mac80211/debugfs_sta.c
index 446908ab3f5d..d84d2369a76e 100644
--- a/net/mac80211/debugfs_sta.c
+++ b/net/mac80211/debugfs_sta.c
@@ -233,7 +233,28 @@ static ssize_t sta_airtime_read(struct file
*file, char __user *userbuf,
kfree(buf);
return rv;
}
-STA_OPS(airtime);
+
+/*
+ * FIXME: This *only* reset station airtime, didn't accept input
+ */
+static ssize_t sta_airtime_write(struct file *file, const char __user *userbuf,
+ size_t count, loff_t *ppos)
+{
+struct sta_info *sta = file->private_data;
+struct ieee80211_local *local = sta->sdata->local;
+int ac;
+
+for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) {
+spin_lock_bh(&local->active_txq_lock[ac]);
+sta->airtime[ac].rx_airtime = 0;
+sta->airtime[ac].tx_airtime = 0;
+sta->airtime[ac].deficit = sta->airtime_weight;
+spin_unlock_bh(&local->active_txq_lock[ac]);
+}
+
+return count;
+}
+STA_OPS_RW(airtime);
static ssize_t sta_agg_status_read(struct file *file, char __user *userbuf,
size_t count, loff_t *ppos)
--
2.18.0
Rajkumar Manoharan 於 2018年11月13日 週二 上午6:52寫道：
>
> From: Toke Høiland-Jørgensen
>
> This adds airtime accounting and scheduling to the mac80211 TXQ
> scheduler. A new callback, ieee80211_sta_register_airtime(), is added
> that drivers can call to report airtime usage for stations.
>
> When airtime information is present, mac80211 will schedule TXQs
> (through ieee80211_next_txq()) in a way that enforces airtime fairness
> between active stations. This scheduling works the same way as the ath9k
> in-driver airtime fairness scheduling. If no airtime usage is reported
> by the driver, the scheduler will default to round-robin scheduling.
>
> For drivers that don't control TXQ scheduling in software, a new API
> function, ieee80211_txq_may_transmit(), is added which the driver can use
> to check if the TXQ is eligible for transmission, or should be throttled to
> enforce fairness. Calls to this function must also be enclosed in
> ieee80211_txq_schedule_{start,end}() calls to ensure proper locking.
>
> The API ieee80211_txq_may_transmit() also ensures that TXQ list will be
> aligned aginst driver's own round-robin scheduler list. i.e it rotates
> the TXQ list till it makes the requested node becomes the first entry
> in TXQ list. Thus both the TXQ list and driver's list are in sync.
>
> Co-Developed-by: Rajkumar Manoharan
> Signed-off-by: Toke Høiland-Jørgensen
> Signed-off-by: Rajkumar Manoharan
> ---
> include/net/mac80211.h | 59 ++
> net/mac80211/cfg.c | 3 ++
> net/mac80211/debugfs.c | 3 ++
> net/mac80211/debugfs_sta.c | 50 --
> net/mac80211/ieee80211_i.h | 2 ++
> net/mac80211/main.c| 4 +++
> net/mac80211/sta_info.c| 44 +--
> net/mac80211/sta_info.h| 13 +++
> net/mac80211/status.c | 6
> net/mac80211/tx.c | 90
> +++---
> 10 files changed, 264 insertions(+), 10 deletions(-)
>
> diff --git a/include/net/mac80211.h b/include/net/mac80211.h
> index 18b11c119b7e..c43d615ee9b1 100644
> --- a/include/net/mac80211.h
> +++ b/include/net/mac80211.h
> @@ -2357,6 +2357,9 @@ enum ieee80211_hw_flags {
> * @tx_sk_pacing_shift: Pacing shift to set on TCP sockets when frames from
> * them are encountered. The default should typically not be changed,
> * unless the driver has good reasons for needing more buffers.
> + *
> + * @weight_multipler: Driver specific airtime weight multiplier used while
> + * refilling deficit of each TXQ.
> */
> struct ieee80211_hw {
> struct ieee80211_conf conf;
> @@ -2393,6 +2396,7 @@ struct ieee80211_hw {
> const struct ieee80211_cipher_scheme *cipher_schemes;
> u8 max_nan_de_entries;
> u8 tx_sk_pacing_shift;
> + u8 weight_multiplier;
> };
>
> static inline bool _ieee80211_hw_check(struct ieee80211_hw *hw,
> @@ -5393,6 +5397,34 @@ void ieee80211_sta_block_awake(struct ieee80211_hw *hw,
> void ieee80211_send_eosp_nullfunc(struct ieee80211_sta *pubsta, int tid);
>
> /**
> + * ieee80211_sta_register_airtime - register airtime usage for a sta/tid
> + *
> + * Register airtime usage for a given sta on a given tid. The driver can call
Re: [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
Felix Fietkau writes:
> On 2018-11-12 23:51, Rajkumar Manoharan wrote:
>> From: Toke Høiland-Jørgensen
>>
>> This adds airtime accounting and scheduling to the mac80211 TXQ
>> scheduler. A new callback, ieee80211_sta_register_airtime(), is added
>> that drivers can call to report airtime usage for stations.
>>
>> When airtime information is present, mac80211 will schedule TXQs
>> (through ieee80211_next_txq()) in a way that enforces airtime fairness
>> between active stations. This scheduling works the same way as the ath9k
>> in-driver airtime fairness scheduling. If no airtime usage is reported
>> by the driver, the scheduler will default to round-robin scheduling.
>>
>> For drivers that don't control TXQ scheduling in software, a new API
>> function, ieee80211_txq_may_transmit(), is added which the driver can use
>> to check if the TXQ is eligible for transmission, or should be throttled to
>> enforce fairness. Calls to this function must also be enclosed in
>> ieee80211_txq_schedule_{start,end}() calls to ensure proper locking.
>>
>> The API ieee80211_txq_may_transmit() also ensures that TXQ list will be
>> aligned aginst driver's own round-robin scheduler list. i.e it rotates
>> the TXQ list till it makes the requested node becomes the first entry
>> in TXQ list. Thus both the TXQ list and driver's list are in sync.
>>
>> Co-Developed-by: Rajkumar Manoharan
>> Signed-off-by: Toke Høiland-Jørgensen
>> Signed-off-by: Rajkumar Manoharan
>> ---
>> include/net/mac80211.h | 59 ++
>> net/mac80211/cfg.c | 3 ++
>> net/mac80211/debugfs.c | 3 ++
>> net/mac80211/debugfs_sta.c | 50 --
>> net/mac80211/ieee80211_i.h | 2 ++
>> net/mac80211/main.c| 4 +++
>> net/mac80211/sta_info.c| 44 +--
>> net/mac80211/sta_info.h| 13 +++
>> net/mac80211/status.c | 6
>> net/mac80211/tx.c | 90
>> +++---
>> 10 files changed, 264 insertions(+), 10 deletions(-)
>>
>> diff --git a/net/mac80211/status.c b/net/mac80211/status.c
>> index aa4afbf0abaf..a1f1256448f5 100644
>> --- a/net/mac80211/status.c
>> +++ b/net/mac80211/status.c
>> @@ -818,6 +818,12 @@ static void __ieee80211_tx_status(struct ieee80211_hw
>> *hw,
>> ieee80211_sta_tx_notify(sta->sdata, (void *) skb->data,
>> acked, info->status.tx_time);
>>
>> +if (info->status.tx_time &&
>> +wiphy_ext_feature_isset(local->hw.wiphy,
>> +
>> NL80211_EXT_FEATURE_AIRTIME_FAIRNESS))
>> +ieee80211_sta_register_airtime(&sta->sta, tid,
>> + info->status.tx_time, 0);
>> +
>> if (ieee80211_hw_check(&local->hw, REPORTS_TX_ACK_STATUS)) {
>> if (info->flags & IEEE80211_TX_STAT_ACK) {
>> if (sta->status_stats.lost_packets)
> I think the same is needed in ieee80211_tx_status_ext.
Right, good point.
>> diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
>> index 305965283506..3f417e80e041 100644
>> --- a/net/mac80211/tx.c
>> +++ b/net/mac80211/tx.c
>> @@ -3660,12 +3680,74 @@ void ieee80211_return_txq(struct ieee80211_hw *hw,
>> lockdep_assert_held(&local->active_txq_lock[txq->ac]);
>>
>> if (list_empty(&txqi->schedule_order) &&
>> -(!skb_queue_empty(&txqi->frags) || txqi->tin.backlog_packets))
>> -list_add_tail(&txqi->schedule_order,
>> - &local->active_txqs[txq->ac]);
>> +(!skb_queue_empty(&txqi->frags) || txqi->tin.backlog_packets)) {
>> +/* If airtime accounting is active, always enqueue STAs at the
>> + * head of the list to ensure that they only get moved to the
>> + * back by the airtime DRR scheduler once they have a negative
>> + * deficit. A station that already has a negative deficit will
>> + * get immediately moved to the back of the list on the next
>> + * call to ieee80211_next_txq().
>> + */
>> +if (txqi->txq.sta &&
>> +wiphy_ext_feature_isset(local->hw.wiphy,
>> +
>> NL80211_EXT_FEATURE_AIRTIME_FAIRNESS))
>> +list_add(&txqi->schedule_order,
>> + &local->active_txqs[txq->ac]);
>> +else
>> +list_add_tail(&txqi->schedule_order,
>> + &local->active_txqs[txq->ac]);
>> +}
>> }
> This part doesn't really make much sense to me, but maybe I'm
> misunderstanding how the code works.
> Let's assume we have a driver like ath9k or mt76, which tries to keep a
> number of aggregates in the hardware queue, and the hardware queue is
> currently empty.
> If the current txq entry is kept at the head of the schedule list,
Re: [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
On 2018-11-12 23:51, Rajkumar Manoharan wrote:
> From: Toke Høiland-Jørgensen
>
> This adds airtime accounting and scheduling to the mac80211 TXQ
> scheduler. A new callback, ieee80211_sta_register_airtime(), is added
> that drivers can call to report airtime usage for stations.
>
> When airtime information is present, mac80211 will schedule TXQs
> (through ieee80211_next_txq()) in a way that enforces airtime fairness
> between active stations. This scheduling works the same way as the ath9k
> in-driver airtime fairness scheduling. If no airtime usage is reported
> by the driver, the scheduler will default to round-robin scheduling.
>
> For drivers that don't control TXQ scheduling in software, a new API
> function, ieee80211_txq_may_transmit(), is added which the driver can use
> to check if the TXQ is eligible for transmission, or should be throttled to
> enforce fairness. Calls to this function must also be enclosed in
> ieee80211_txq_schedule_{start,end}() calls to ensure proper locking.
>
> The API ieee80211_txq_may_transmit() also ensures that TXQ list will be
> aligned aginst driver's own round-robin scheduler list. i.e it rotates
> the TXQ list till it makes the requested node becomes the first entry
> in TXQ list. Thus both the TXQ list and driver's list are in sync.
>
> Co-Developed-by: Rajkumar Manoharan
> Signed-off-by: Toke Høiland-Jørgensen
> Signed-off-by: Rajkumar Manoharan
> ---
> include/net/mac80211.h | 59 ++
> net/mac80211/cfg.c | 3 ++
> net/mac80211/debugfs.c | 3 ++
> net/mac80211/debugfs_sta.c | 50 --
> net/mac80211/ieee80211_i.h | 2 ++
> net/mac80211/main.c| 4 +++
> net/mac80211/sta_info.c| 44 +--
> net/mac80211/sta_info.h| 13 +++
> net/mac80211/status.c | 6
> net/mac80211/tx.c | 90
> +++---
> 10 files changed, 264 insertions(+), 10 deletions(-)
>
> diff --git a/net/mac80211/status.c b/net/mac80211/status.c
> index aa4afbf0abaf..a1f1256448f5 100644
> --- a/net/mac80211/status.c
> +++ b/net/mac80211/status.c
> @@ -818,6 +818,12 @@ static void __ieee80211_tx_status(struct ieee80211_hw
> *hw,
> ieee80211_sta_tx_notify(sta->sdata, (void *) skb->data,
> acked, info->status.tx_time);
>
> + if (info->status.tx_time &&
> + wiphy_ext_feature_isset(local->hw.wiphy,
> +
> NL80211_EXT_FEATURE_AIRTIME_FAIRNESS))
> + ieee80211_sta_register_airtime(&sta->sta, tid,
> +info->status.tx_time, 0);
> +
> if (ieee80211_hw_check(&local->hw, REPORTS_TX_ACK_STATUS)) {
> if (info->flags & IEEE80211_TX_STAT_ACK) {
> if (sta->status_stats.lost_packets)
I think the same is needed in ieee80211_tx_status_ext.
> diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
> index 305965283506..3f417e80e041 100644
> --- a/net/mac80211/tx.c
> +++ b/net/mac80211/tx.c
> @@ -3660,12 +3680,74 @@ void ieee80211_return_txq(struct ieee80211_hw *hw,
> lockdep_assert_held(&local->active_txq_lock[txq->ac]);
>
> if (list_empty(&txqi->schedule_order) &&
> - (!skb_queue_empty(&txqi->frags) || txqi->tin.backlog_packets))
> - list_add_tail(&txqi->schedule_order,
> - &local->active_txqs[txq->ac]);
> + (!skb_queue_empty(&txqi->frags) || txqi->tin.backlog_packets)) {
> + /* If airtime accounting is active, always enqueue STAs at the
> + * head of the list to ensure that they only get moved to the
> + * back by the airtime DRR scheduler once they have a negative
> + * deficit. A station that already has a negative deficit will
> + * get immediately moved to the back of the list on the next
> + * call to ieee80211_next_txq().
> + */
> + if (txqi->txq.sta &&
> + wiphy_ext_feature_isset(local->hw.wiphy,
> +
> NL80211_EXT_FEATURE_AIRTIME_FAIRNESS))
> + list_add(&txqi->schedule_order,
> + &local->active_txqs[txq->ac]);
> + else
> + list_add_tail(&txqi->schedule_order,
> + &local->active_txqs[txq->ac]);
> + }
> }
This part doesn't really make much sense to me, but maybe I'm
misunderstanding how the code works.
Let's assume we have a driver like ath9k or mt76, which tries to keep a
number of aggregates in the hardware queue, and the hardware queue is
currently empty.
If the current txq entry is kept at the head of the schedule list,
wouldn't the code just pull from that one over and over again, until
enough packets are transmitted by the hardware a
[PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs
From: Toke Høiland-Jørgensen
This adds airtime accounting and scheduling to the mac80211 TXQ
scheduler. A new callback, ieee80211_sta_register_airtime(), is added
that drivers can call to report airtime usage for stations.
When airtime information is present, mac80211 will schedule TXQs
(through ieee80211_next_txq()) in a way that enforces airtime fairness
between active stations. This scheduling works the same way as the ath9k
in-driver airtime fairness scheduling. If no airtime usage is reported
by the driver, the scheduler will default to round-robin scheduling.
For drivers that don't control TXQ scheduling in software, a new API
function, ieee80211_txq_may_transmit(), is added which the driver can use
to check if the TXQ is eligible for transmission, or should be throttled to
enforce fairness. Calls to this function must also be enclosed in
ieee80211_txq_schedule_{start,end}() calls to ensure proper locking.
The API ieee80211_txq_may_transmit() also ensures that TXQ list will be
aligned aginst driver's own round-robin scheduler list. i.e it rotates
the TXQ list till it makes the requested node becomes the first entry
in TXQ list. Thus both the TXQ list and driver's list are in sync.
Co-Developed-by: Rajkumar Manoharan
Signed-off-by: Toke Høiland-Jørgensen
Signed-off-by: Rajkumar Manoharan
---
include/net/mac80211.h | 59 ++
net/mac80211/cfg.c | 3 ++
net/mac80211/debugfs.c | 3 ++
net/mac80211/debugfs_sta.c | 50 --
net/mac80211/ieee80211_i.h | 2 ++
net/mac80211/main.c| 4 +++
net/mac80211/sta_info.c| 44 +--
net/mac80211/sta_info.h| 13 +++
net/mac80211/status.c | 6
net/mac80211/tx.c | 90 +++---
10 files changed, 264 insertions(+), 10 deletions(-)
diff --git a/include/net/mac80211.h b/include/net/mac80211.h
index 18b11c119b7e..c43d615ee9b1 100644
--- a/include/net/mac80211.h
+++ b/include/net/mac80211.h
@@ -2357,6 +2357,9 @@ enum ieee80211_hw_flags {
* @tx_sk_pacing_shift: Pacing shift to set on TCP sockets when frames from
* them are encountered. The default should typically not be changed,
* unless the driver has good reasons for needing more buffers.
+ *
+ * @weight_multipler: Driver specific airtime weight multiplier used while
+ * refilling deficit of each TXQ.
*/
struct ieee80211_hw {
struct ieee80211_conf conf;
@@ -2393,6 +2396,7 @@ struct ieee80211_hw {
const struct ieee80211_cipher_scheme *cipher_schemes;
u8 max_nan_de_entries;
u8 tx_sk_pacing_shift;
+ u8 weight_multiplier;
};
static inline bool _ieee80211_hw_check(struct ieee80211_hw *hw,
@@ -5393,6 +5397,34 @@ void ieee80211_sta_block_awake(struct ieee80211_hw *hw,
void ieee80211_send_eosp_nullfunc(struct ieee80211_sta *pubsta, int tid);
/**
+ * ieee80211_sta_register_airtime - register airtime usage for a sta/tid
+ *
+ * Register airtime usage for a given sta on a given tid. The driver can call
+ * this function to notify mac80211 that a station used a certain amount of
+ * airtime. This information will be used by the TXQ scheduler to schedule
+ * stations in a way that ensures airtime fairness.
+ *
+ * The reported airtime should as a minimum include all time that is spent
+ * transmitting to the remote station, including overhead and padding, but not
+ * including time spent waiting for a TXOP. If the time is not reported by the
+ * hardware it can in some cases be calculated from the rate and known frame
+ * composition. When possible, the time should include any failed transmission
+ * attempts.
+ *
+ * The driver can either call this function synchronously for every packet or
+ * aggregate, or asynchronously as airtime usage information becomes available.
+ * TX and RX airtime can be reported together, or separately by setting one of
+ * them to 0.
+ *
+ * @pubsta: the station
+ * @tid: the TID to register airtime for
+ * @tx_airtime: airtime used during TX (in usec)
+ * @rx_airtime: airtime used during RX (in usec)
+ */
+void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid,
+ u32 tx_airtime, u32 rx_airtime);
+
+/**
* ieee80211_iter_keys - iterate keys programmed into the device
* @hw: pointer obtained from ieee80211_alloc_hw()
* @vif: virtual interface to iterate, may be %NULL for all
@@ -6150,6 +6182,33 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw
*hw,
void ieee80211_txq_schedule_end(struct ieee80211_hw *hw, u8 ac);
/**
+ * ieee80211_txq_may_transmit - check whether TXQ is allowed to transmit
+ *
+ * This function is used to check whether given txq is allowed to transmit by
+ * the airtime scheduler, and can be used by drivers to access the airtime
+ * fairness accounting without going using the scheduling order enfored by
+ * next_txq().
+ *
+ * Returns %true if the airtime scheduler thinks the TXQ shoul
