Re: [Bloat] [Ecn-sane] sce materials from ietf
> there are a multitude of papers posted for the buffer sizing workshop > > http://buffer-workshop.stanford.edu/papers/paper23.pdf was interesting. Would be nice to get them to add ECN(sce) to the mix of there tests. > On Fri, Nov 29, 2019 at 12:08 PM Dave Taht wrote: > > > > there are no minutes posted. > > > > https://datatracker.ietf.org/meeting/106/materials/slides-106-tsvwg-sessa-81-some-congestion-experienced-00 > > > > https://datatracker.ietf.org/meeting/106/materials/slides-106-tcpm-some-congestion-experienced-in-tcp-00 > > -- > > > > Dave T?ht > Dave T?ht -- Rod Grimes rgri...@freebsd.org ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> > there are no minutes posted. > > > > https://datatracker.ietf.org/meeting/106/materials/slides-106-tsvwg-sessa-81-some-congestion-experienced-00 > > > > https://datatracker.ietf.org/meeting/106/materials/slides-106-tcpm-some-congestion-experienced-in-tcp-00 > > The above 2 decks are identical. Jonathan did not get any time > during tsvwg, so I reposted the whole deck to tcpm, in which I > also did not get any time to present. Correction, Jonathan did get 6 minutes, iirc. > > BUT, and that is a all caps BUT, good stuff happened for SCE > forward progress during the meetinhgs none the less. We did > infact get an announcement that we have asked for adoption of > draft-morton-tsvwg-sce, with a 25 hand count on who has read > the draft, which by my rough estimate was more than 1/4 of > the room. > > During the tcpm session the issues around allocation of bit 7 > for AccECN may of been worked out, that draft (AccECN) is becoming > a proposed standard, which can do the IANA allocation, and Mira > at least continues to affirm that bit 7 can be used for other > purposes after an AccECN negotiation failure when it falls back > to RFC3168 ECN, so we (SCE) believe we do have a path forward on > our alternate use for bit 7. > > The tsvwg chairs, and the work group itself now needs to discuss > the 2 experiment problem, the conflicts and compatibilities between > the 2, and just how to deal with the situation. > > YOUR (that being all the list members of ecn-sane, and the larger > bufferbloat community) inputs and helps are highly desired in > this process. > > The SCE teams possition is that L4S is fundementally flawed in > its use of the ECT(1) code point as a "Traffic Classifier" since > that leads to the end nodes telling the network the traffic is > special, aka treat me differently than any other traffic, and > is likely to lead to abuse, which may possibly lead to bleaching > of the code point, which would be bad for everyone. > > It would be much nicer to use this last code point, 1/2 a bit, > for a high fidelity signal from the network to the receiver > of the level of congestion in a fully backwards to RFC3168 > way. > > We (the SCE team) also feel that L4S is overly complex and continues > to grow complexity as problems with it are exposed. (Recently it > has become apparent that protecton from RFC3168 behavior is needed, > and thus a new proposal and a new chunk of code are being > developed to deal with this issue. > > > > Dave T?ht > -- > Rod Grimes rgri...@freebsd.org > ___ > Ecn-sane mailing list > ecn-s...@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/ecn-sane > > -- Rod Grimes rgri...@freebsd.org ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> Hi Jonathan, > > > > On Nov 30, 2019, at 23:23, Jonathan Morton wrote: > > > >> On 1 Dec, 2019, at 12:17 am, Carsten Bormann wrote: > >> > >>> There are unfortunate problems with introducing new TCP options, in that > >>> some overzealous firewalls block traffic which uses them. This would be > >>> a deployment hazard for SCE, which merely using a spare header flag > >>> avoids. So instead we are still planning to use the spare bit - which > >>> happens to be one that AccECN also uses, but AccECN negotiates in such a > >>> way that SCE can safely use it even with an AccECN capable partner. > >> > >> This got me curious: Do you have any evidence that firewalls are > >> friendlier to new flags than to new options? > > > > Mirja Kuhlewind said as much during the TCPM session we attended, and she > > ought to know. There appear to have been several studies performed on this > > subject; reserved TCP flags tend to get ignored pretty well, but unknown > > TCP options tend to get either stripped or blocked. > > > > This influenced the design of AccECN as well; in an early version it would > > have used only a TCP option and left the TCP flags alone. When it was > > found that firewalls would often interfere with this, the three-bit field > > in the TCP flags area was cooked up. > > > Belt and suspenders, eh? But realistically, the idea of using an > accumulating SCE counter to allow for a lossy reverse ACK path seems sort of > okay (after all TCP relies on the same, so there would be a nice symmetry ). > I really wonder whether SCE could not, in addition to its current bit, borrow > the URG pointer field in cases when it is not used, or not fully used (if the > MSS is smaller than 64K there might be a few bits leftover, with an MTU < > 2000 I would expect that ~5 bits might still be usable in that rate case). I > might be completely of to lunch here, but boy a nice rarely used contiguous > 16bit field in the TCP header, what kind of mischief one could arrange with > that ;) Looking at the AccECN draft, I see that my idea is not terribly > original... But, hey for SCE having an additional higher fidelity SCE counter > might be a nice addition, assuming URG(0), urgent pointer > 0 will not > bleached/rejected by uninitiated TCP stacks/middleboxes... We need to fix the ACK issues rather than continue to work around it. Ack thinning is good, as long as it does not cause information loss. There is no draft/RFC on this, one needs to be written that explains you can not just ignore all the bits, you have to preserve the reserve bits, so you can only thin if they are the same. Jonathan already fixed Cake (I think that is the one that has ACK thinning) to not collapse ACK's that have different bit 7 values. Note that I consider the time of the arriving ACKS to also be informaition, RACK for instance uses that, so in the case of RACK any thinning could be considered bad. BUTT I'll settle for not tossing reserved bit changes away as a "good enough" step forward that should be simple to implement (2 gate delay xor/or function). > Sebastian > > - Jonathan Morton -- Rod Grimes rgri...@freebsd.org ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> there are no minutes posted. > > https://datatracker.ietf.org/meeting/106/materials/slides-106-tsvwg-sessa-81-some-congestion-experienced-00 > > https://datatracker.ietf.org/meeting/106/materials/slides-106-tcpm-some-congestion-experienced-in-tcp-00 The above 2 decks are identical. Jonathan did not get any time during tsvwg, so I reposted the whole deck to tcpm, in which I also did not get any time to present. BUT, and that is a all caps BUT, good stuff happened for SCE forward progress during the meetinhgs none the less. We did infact get an announcement that we have asked for adoption of draft-morton-tsvwg-sce, with a 25 hand count on who has read the draft, which by my rough estimate was more than 1/4 of the room. During the tcpm session the issues around allocation of bit 7 for AccECN may of been worked out, that draft (AccECN) is becoming a proposed standard, which can do the IANA allocation, and Mira at least continues to affirm that bit 7 can be used for other purposes after an AccECN negotiation failure when it falls back to RFC3168 ECN, so we (SCE) believe we do have a path forward on our alternate use for bit 7. The tsvwg chairs, and the work group itself now needs to discuss the 2 experiment problem, the conflicts and compatibilities between the 2, and just how to deal with the situation. YOUR (that being all the list members of ecn-sane, and the larger bufferbloat community) inputs and helps are highly desired in this process. The SCE teams possition is that L4S is fundementally flawed in its use of the ECT(1) code point as a "Traffic Classifier" since that leads to the end nodes telling the network the traffic is special, aka treat me differently than any other traffic, and is likely to lead to abuse, which may possibly lead to bleaching of the code point, which would be bad for everyone. It would be much nicer to use this last code point, 1/2 a bit, for a high fidelity signal from the network to the receiver of the level of congestion in a fully backwards to RFC3168 way. We (the SCE team) also feel that L4S is overly complex and continues to grow complexity as problems with it are exposed. (Recently it has become apparent that protecton from RFC3168 behavior is needed, and thus a new proposal and a new chunk of code are being developed to deal with this issue. > Dave T?ht -- Rod Grimes rgri...@freebsd.org ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> On Dec 2, 2019, at 6:38 AM, Dave Taht wrote: > > I do hate watching y'all continually concede the "latency" point and > have to argue on the "chosen ground" of single or dualq about > long-running tcp flows. I don’t think we’ve conceded that. It would be possible to run more tests on a LAN with a shorter Codel interval and show ~1ms TCP RTT, and perhaps we should. I don’t think there's any fundamental reason why one signaling mechanism or the other should ultimately be any better in this regard. ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Dave, On December 2, 2019 6:38:11 AM GMT+01:00, Dave Taht wrote: >Jonathan Morton writes: > >>> On 30 Nov, 2019, at 5:42 pm, Sebastian Moeller >wrote: >>> > I fear that they will come up with something that in reality will a) by opt-out, that is they will assume L4S-style feedback until reluctantly convinced that the bottleneck marker is rfc3160-compliant and hence will b) trigger too late c) trigger to rarely to be actually helpful in reality, but might show a good enough effort to push L4S past issue #16. I'm sure they will, and we will of course point out these >shortcomings as they occur, so as to count them against issue #16. >>> >>> That might be bad position to be in though (if one party only >>> gives negative feed-back no matter how justified it will generate a >>> residual feeling of lack of good faith cooperation), I would have >>> preferred if the requirements would have bee discussed before. >>> Conversely, if they do manage to make it fail-safe, it is highly likely that their scheme will give false positives on real Internet paths and fail to switch into L4S mode, impairing their performance in other ways. >>> >>> Yes, so far they always err on the advantage of L4S, and >>> justify this with "but, latency" and if one buys the latency > >I do hate watching y'all continually concede the "latency" point and Well, they have pretty graphs to wield around hitting their PR target of ~1ms queueing delay. I had a trawl through the bake-off data but all SCE results I found show noticeably larger delays (not bad by any standards, but a much harder sell than the simple <1ms story) I might not have looked to carefully though? >have to argue on the "chosen ground" of single or dualq about >long-running tcp flows. > >"fq" already achieves "ultra-low latency" for nearly all flows, >especially including flows in the presence of bursts, short flows that >never get out of slow start (e.g. most of them), simple malicious >traffic, and so on. > >The need for any additional marking "stuff" is lessened, particularly >as >fq enables RTT based tcps such as BBR to work better in the first >place. > >codel has a target of 5ms where pie is 16ms. In the dual queue draft the reference latency sits at 15ms without a good rationale, I would love to see dual queue results at short RTT with a theoretically justified 5 Ms target, to see whether that counteracts the equitable sharing failure. Neither achieve these, >but both come close, but the second "classic" queue in dualq is 3x >worse >than any given queue in fq_codel. My take on that is: Bob must have realized early on that equal sharing very much is not optimal, if you know more about the relative importance of the flows you can do better. (He ignores that as the other side of the coin you can easily do worse, simply permute the latencies and bandwidth of the optimal asymmetric set, but I degrees) And now he struggles in getting this information in the first place and in distributing it to the hops that are in a position to act on this information. The informational ConEx RFC 6789 is a great example of that quixotic quest, where Bob proposes the punish flows based on their total experienced congestion on the full path. In short FQ is not perfect, but neither is anything goes, and FQ has fewer catastrophic failure modes, and the one it has, sensitivy to flow count can a) be remedied by an appropriate flow definition (maybe just a 2-tuple for backbone routers, or even by AS on peering routers, already recognised in rfc2914) and b) anything goes is equally sensitive to the same condition. I really hate the line of argument, since we can not come up with a perfect solution, let's do nothing: that is not how evolution operates. > >the gross unfairness of spitting l4s marked packets through a rfc3168 >compliant link is made much worse when your flows are short. > >both l4s and sce both seem to have an issue in aborting slow start >too early at this point. > >lastly, overusage of ecn in either system can bloat up a link. According to Koen, PIE is stricter than vodel in bringing down the hammer on flows not reacting fast enough though, so the level of ECN bloat might be a function of the employed AQM, no? > >I'm happy to see the two primary approaches to making that less >disasterous by seeing some code arrive for shortening the MSS or "sub >packet windows", but i'd still like to see cwnd 1 and the other >fallback >methods in rfc3168 implemented, and there are still adversaries to >deal with (see rfc3168 sec 7) > >>> justification cautiously default to rfc3168 becomes obviously >>> sub-optimal, and so far none of the chairs put down the "first, do >>> no harm" hammer (and I doubt they will). > >> I got the impression that failing to close most of L4S' open issues >at >> Singapore is politically damaging to them. This is a substan
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Dave, On December 2, 2019 6:10:51 AM GMT+01:00, Dave Taht wrote: >Sebastian Moeller writes: > >> Hi Rodney, >> >> >>> On Dec 1, 2019, at 18:30, Rodney W. Grimes <4b...@gndrsh.dnsmgr.net> >wrote: >>> Hi Jonathan, > On Nov 30, 2019, at 23:23, Jonathan Morton >wrote: > >> On 1 Dec, 2019, at 12:17 am, Carsten Bormann >wrote: >> >>> There are unfortunate problems with introducing new TCP options, >>> in that some overzealous firewalls block traffic which uses >>> them. This would be a deployment hazard for SCE, which merely >>> using a spare header flag avoids. So instead we are still >>> planning to use the spare bit - which happens to be one that >>> AccECN also uses, but AccECN negotiates in such a way that SCE >>> can safely use it even with an AccECN capable partner. >> >> This got me curious: Do you have any evidence that firewalls are >friendlier to new flags than to new options? > > Mirja Kuhlewind said as much during the TCPM session we attended, > and she ought to know. There appear to have been several studies > performed on this subject; reserved TCP flags tend to get ignored > pretty well, but unknown TCP options tend to get either stripped > or blocked. > > This influenced the design of AccECN as well; in an early version > it would have used only a TCP option and left the TCP flags alone. > When it was found that firewalls would often interfere with this, > the three-bit field in the TCP flags area was cooked up. Belt and suspenders, eh? But realistically, the idea of using an accumulating SCE counter to allow for a lossy reverse ACK path seems sort of okay (after all TCP relies on the same, so there would be a nice symmetry ). I really wonder whether SCE could not, in addition to its current bit, borrow the URG pointer field in cases when it is not used, or not fully used (if the MSS is smaller than 64K there might be a few bits leftover, with an MTU < 2000 I would expect that ~5 bits might still be usable in that rate case). I might be completely of to lunch here, but boy a nice rarely used contiguous 16bit field in the TCP header, what kind of mischief one could arrange with that ;) Looking at the AccECN draft, I see that my idea is not terribly original... But, hey for SCE having an additional higher fidelity SCE counter might be a nice addition, assuming URG(0), urgent pointer > 0 will not bleached/rejected by uninitiated TCP stacks/middleboxes... >>> >>> We need to fix the ACK issues rather than continue to work around >>> it. Ack thinning is good, as long as it does not cause information >>> loss. There is no draft/RFC on this, one needs to be written that >>> explains you can not just ignore all the bits, you have to preserve >>> the reserve bits, so you can only thin if they are the same. >>> Jonathan already fixed Cake (I think that is the one that has ACK >>> thinning) to not collapse ACK's that have different bit 7 values. >> >> Well, I detest ACK thinning and believe that the network >> should not try to second guess the users traffic (dropping/marking on >> reaching capacity is acceptable, but the kind of silent ACK thinning >> some DOCSIS ISPs perform seems actively user-hostile). But thinning >or >> no thinning, the accumulative signaling is how the ACK stream deals >> with (reasonably) lossy paths, and I think any additional signaling >> via pure ACK packets should simply be tolerant to unexpected losses. >I >> fully agree that if ACK thinning is performed it really should be >> careful to not loose information when doing its job, but SCE >hopefully >> can deal with whatever is out in the field today (I am looking at you >> DOCSIS uplinks...), no? > >I happen to not be huge on ack thinning either, but the effect >on highly assymetric networks was pretty convincing, and having >to handle less acks at the sender a potential goodness also. > >http://blog.cerowrt.org/post/ack_filtering/ > >At the time we did I thought it could be made even better, >if we allowed more droppable packets to accumulate on each >round, it would both be "fairer" and be able to "drop more" >over each round. > >https://github.com/dtaht/sch_cake/issues/104 > >Never got around to it. > >I'd much rather have fewer highly assymmetric networks, +1, I will not hold my breath though on getting this anytime soon... GPON by default is asymmetric (2:1) and full duplex DOCSIS requires costly plant changes and got moved from the 3.1 spec to 4, and the DSLs simply have no symmetric Bandplans I know of (well G.fast has, but that only helps if the G.fast uplink is not running over GPON). But then GPON's 2:1 ratio would already be most of the way to symmetry... and the >endpoint tcps do the thinning (which is what more or less happens >with GSO), but Yepp, the end
Re: [Bloat] [Ecn-sane] sce materials from ietf
Jonathan Morton writes: >> On 30 Nov, 2019, at 5:42 pm, Sebastian Moeller wrote: >> I fear that they will come up with something that in reality will >>> a) by opt-out, that is they will assume L4S-style feedback until >>> reluctantly convinced that the bottleneck marker is >>> rfc3160-compliant and hence will b) trigger too late c) trigger to >>> rarely to be actually helpful in reality, but might show a good >>> enough effort to push L4S past issue #16. >>> >>> I'm sure they will, and we will of course point out these shortcomings as >>> they occur, so as to count them against issue #16. >> >> That might be bad position to be in though (if one party only >> gives negative feed-back no matter how justified it will generate a >> residual feeling of lack of good faith cooperation), I would have >> preferred if the requirements would have bee discussed before. >> >>> Conversely, if they do manage to make it fail-safe, it is highly >>> likely that their scheme will give false positives on real Internet >>> paths and fail to switch into L4S mode, impairing their performance >>> in other ways. >> >> Yes, so far they always err on the advantage of L4S, and >> justify this with "but, latency" and if one buys the latency I do hate watching y'all continually concede the "latency" point and have to argue on the "chosen ground" of single or dualq about long-running tcp flows. "fq" already achieves "ultra-low latency" for nearly all flows, especially including flows in the presence of bursts, short flows that never get out of slow start (e.g. most of them), simple malicious traffic, and so on. The need for any additional marking "stuff" is lessened, particularly as fq enables RTT based tcps such as BBR to work better in the first place. codel has a target of 5ms where pie is 16ms. Neither achieve these, but both come close, but the second "classic" queue in dualq is 3x worse than any given queue in fq_codel. the gross unfairness of spitting l4s marked packets through a rfc3168 compliant link is made much worse when your flows are short. both l4s and sce both seem to have an issue in aborting slow start too early at this point. lastly, overusage of ecn in either system can bloat up a link. I'm happy to see the two primary approaches to making that less disasterous by seeing some code arrive for shortening the MSS or "sub packet windows", but i'd still like to see cwnd 1 and the other fallback methods in rfc3168 implemented, and there are still adversaries to deal with (see rfc3168 sec 7) >> justification cautiously default to rfc3168 becomes obviously >> sub-optimal, and so far none of the chairs put down the "first, do >> no harm" hammer (and I doubt they will). > I got the impression that failing to close most of L4S' open issues at > Singapore is politically damaging to them. This is a substantial list > of problems opened at Montreal, as blockers for their WGLC on > publishing L4S drafts as experimental RFCs. They had all the time in well, I'd like to file at least one more so we can get a real rrul test comparison done. > the world to talk about solutions to the major showstopper problems, > but were only able to concede a point that maybe tying RACK to the > ECT(1) codepoint is better written as a SHOULD instead of a MUST. > That lack of progress was noticed at the WG Chair level; I think they > may have been giving them the rope to hang themselves, so to speak. I > think they had a slide up at the side session, showing massive > unfairness between L4S and "classic" flows, for a full half-hour - and > they somehow thought that was *helpful* to their case! I think there is at least a small segment of the audience that thinks that prioritizing the internet for data coming from the ISP's or mobile operator's DC is a very good thing. It doesn't matter how (think vpn backhauls from cell towers as one example, or coast to coast data transfers) long term disadvantagous RTT-unfairness may be to that mindset. Me, I love how FQ brings true RTT fairness to the internet, offering a shot at equal bandwidth to sites near and far, bringing the world closer together. > I'm reasonably sure some industry attendees also noticed this - Stuart > Cheshire (of Apple) in particular. Apple have been on the front lines > of enabling ECN deployment in practice in recent years. He invited > me, one of the ICCRG chairs, and Bob Briscoe - among others - to > dinner, where we discussed some technical distinctions and Bob > demonstrated a fundamental misunderstanding of control theory. I am glad y'all got through dinner alive. > And we will have more ammunition at Vancouver. It remains to be seen how > much progress they'll make… > > - Jonathan Morton > ___ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat ___ Bloat mailing list Bloat@lists.bufferbloat.net h
Re: [Bloat] [Ecn-sane] sce materials from ietf
Sebastian Moeller writes: > Hi Rodney, > > >> On Dec 1, 2019, at 18:30, Rodney W. Grimes <4b...@gndrsh.dnsmgr.net> wrote: >> >>> Hi Jonathan, >>> >>> On Nov 30, 2019, at 23:23, Jonathan Morton wrote: > On 1 Dec, 2019, at 12:17 am, Carsten Bormann wrote: > >> There are unfortunate problems with introducing new TCP options, >> in that some overzealous firewalls block traffic which uses >> them. This would be a deployment hazard for SCE, which merely >> using a spare header flag avoids. So instead we are still >> planning to use the spare bit - which happens to be one that >> AccECN also uses, but AccECN negotiates in such a way that SCE >> can safely use it even with an AccECN capable partner. > > This got me curious: Do you have any evidence that firewalls are > friendlier to new flags than to new options? Mirja Kuhlewind said as much during the TCPM session we attended, and she ought to know. There appear to have been several studies performed on this subject; reserved TCP flags tend to get ignored pretty well, but unknown TCP options tend to get either stripped or blocked. This influenced the design of AccECN as well; in an early version it would have used only a TCP option and left the TCP flags alone. When it was found that firewalls would often interfere with this, the three-bit field in the TCP flags area was cooked up. >>> >>> >>> Belt and suspenders, eh? But realistically, the idea of using >>> an accumulating SCE counter to allow for a lossy reverse ACK path >>> seems sort of okay (after all TCP relies on the same, so there >>> would be a nice symmetry ). >>> I really wonder whether SCE could not, in addition to its current >>> bit, borrow the URG pointer field in cases when it is not used, or >>> not fully used (if the MSS is smaller than 64K there might be a few >>> bits leftover, with an MTU < 2000 I would expect that ~5 bits might >>> still be usable in that rate case). I might be completely of to >>> lunch here, but boy a nice rarely used contiguous 16bit field in >>> the TCP header, what kind of mischief one could arrange with that >>> ;) Looking at the AccECN draft, I see that my idea is not terribly >>> original... But, hey for SCE having an additional higher fidelity >>> SCE counter might be a nice addition, assuming URG(0), urgent >>> pointer > 0 will not bleached/rejected by uninitiated TCP >>> stacks/middleboxes... >> >> We need to fix the ACK issues rather than continue to work around >> it. Ack thinning is good, as long as it does not cause information >> loss. There is no draft/RFC on this, one needs to be written that >> explains you can not just ignore all the bits, you have to preserve >> the reserve bits, so you can only thin if they are the same. >> Jonathan already fixed Cake (I think that is the one that has ACK >> thinning) to not collapse ACK's that have different bit 7 values. > > Well, I detest ACK thinning and believe that the network > should not try to second guess the users traffic (dropping/marking on > reaching capacity is acceptable, but the kind of silent ACK thinning > some DOCSIS ISPs perform seems actively user-hostile). But thinning or > no thinning, the accumulative signaling is how the ACK stream deals > with (reasonably) lossy paths, and I think any additional signaling > via pure ACK packets should simply be tolerant to unexpected losses. I > fully agree that if ACK thinning is performed it really should be > careful to not loose information when doing its job, but SCE hopefully > can deal with whatever is out in the field today (I am looking at you > DOCSIS uplinks...), no? I happen to not be huge on ack thinning either, but the effect on highly assymetric networks was pretty convincing, and having to handle less acks at the sender a potential goodness also. http://blog.cerowrt.org/post/ack_filtering/ At the time we did I thought it could be made even better, if we allowed more droppable packets to accumulate on each round, it would both be "fairer" and be able to "drop more" over each round. https://github.com/dtaht/sch_cake/issues/104 Never got around to it. I'd much rather have fewer highly assymmetric networks, and the endpoint tcps do the thinning (which is what more or less happens with GSO), but secondly, I note that "ack prioritization" is a very common thing in multiple shapers I've looked at, starting with wondershaper and in many others (including dd-wrt). A lot of these are *wrong*, wondershaper, for example, only recognized 64 byte acks. I think more than a few modems do ack prioritization rather than "thinning". thirdly, protocols such as QUIC are already sending less acknowlegements per packet than most TCP do, which is a a good thing. fourthly, I've been meaning to try thinning on wifi for a while. Wifi has a problem in that only a fixed number of packets can fit in a txop and everythi
Re: [Bloat] [Ecn-sane] sce materials from ietf
> On 1 Dec, 2019, at 9:32 pm, Sebastian Moeller wrote: > >> Meanwhile, an ack filter that avoids dropping acks in which the reserved >> flag bits differ from its successor will not lose any information in the >> one-bit scheme. This is what's implemented in Cake (except that not all the >> reserved bits are covered yet, only the one we use). > > So, to show my lack of knowledge, basically a pure change in sequence number > is acceptable, any other differences should trigger ACK conservation instead > of filtering? You are broadly correct, in that a pure advance of acked sequence number effectively obsoletes the earlier ack and it is therefore safe (and even arguably beneficial) to drop it. However a *duplicate* ack should *not* be dropped, because that may be required to trigger Fast Retransmission in the absence of SACK. Cake's ack filter is a bit more sophisticated than that, in that it can also accept certain harmless changes within TCP options. I believe Timestamps and SACK get special handling along these lines; Timestamps can always change, SACK gets equivalent "pure superset" logic to detect when the old ack is completely covered by the new one. Other options not specifically handled are treated as disqualifying. All this only occurs in two consecutive packets which are both acks for the same connection and which are both waiting for a delivery opportunity in the queue. An earlier ack is never delayed just to see if it can be combined with a later one. The result is a better use of limited capacity to carry useful payloads, without having to rely on dropping acks by AQM action (which Codel is actually rather bad at). - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Jonathan, > On Dec 1, 2019, at 20:27, Jonathan Morton wrote: > >> On 1 Dec, 2019, at 9:03 pm, Sebastian Moeller wrote: >> >>> If less feedback is observed by the sender than intended by the AQM, growth >>> will continue and the AQM will increase its marking to compensate, >>> ultimately resorting to a CE mark. >> >> Well, that seems undesirable? > > As a safety valve, getting a CE mark is greatly preferable to losing > congestion control entirely, or incurring a packet loss as the other > alternative congestion signal. Well, yes, I fully agree, I was referring to the "less feedback is observed by the sender than intended" part; I think it is great that SCE is safe by design in this regard. > It would only happen if the SCE signal or feedback were seriously disrupted > or entirely erased - the latter being the *normal* state of affairs when > either endpoint is not SCE aware in the first place. > >> Am I right to assume that the fault tolerance requires a relative steady ACK >> stream though? > > It only needs to be sufficient to keep the TCP stream flowing. If the acks > are bursty, that's a separate problem in which it doesn't really matter if > they're all present or not. And technically, the one-bit feedback mechanism > is capable of precisely reflecting a sparse sequence of SCE marks using just > two acks per mark. > >> I fully agree that if ACK thinning is performed it really should be careful >> to not loose information when doing its job, but SCE hopefully can deal with >> whatever is out in the field today (I am looking at you DOCSIS uplinks...), >> no? > > Right, that's the essence of the above discussion about relative feedback > error, which is the sort of thing that random ack loss or unprincipled ack > thinning is likely to introduce. "unprincipled ack thinning" nice description. > > Meanwhile, an ack filter that avoids dropping acks in which the reserved flag > bits differ from its successor will not lose any information in the one-bit > scheme. This is what's implemented in Cake (except that not all the reserved > bits are covered yet, only the one we use). So, to show my lack of knowledge, basically a pure change in sequence number is acceptable, any other differences should trigger ACK conservation instead of filtering? Best Regards Sebastian > > - Jonathan Morton > ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> On 1 Dec, 2019, at 9:03 pm, Sebastian Moeller wrote: > >> If less feedback is observed by the sender than intended by the AQM, growth >> will continue and the AQM will increase its marking to compensate, >> ultimately resorting to a CE mark. > > Well, that seems undesirable? As a safety valve, getting a CE mark is greatly preferable to losing congestion control entirely, or incurring a packet loss as the other alternative congestion signal. It would only happen if the SCE signal or feedback were seriously disrupted or entirely erased - the latter being the *normal* state of affairs when either endpoint is not SCE aware in the first place. > Am I right to assume that the fault tolerance requires a relative steady ACK > stream though? It only needs to be sufficient to keep the TCP stream flowing. If the acks are bursty, that's a separate problem in which it doesn't really matter if they're all present or not. And technically, the one-bit feedback mechanism is capable of precisely reflecting a sparse sequence of SCE marks using just two acks per mark. > I fully agree that if ACK thinning is performed it really should be careful > to not loose information when doing its job, but SCE hopefully can deal with > whatever is out in the field today (I am looking at you DOCSIS uplinks...), > no? Right, that's the essence of the above discussion about relative feedback error, which is the sort of thing that random ack loss or unprincipled ack thinning is likely to introduce. Meanwhile, an ack filter that avoids dropping acks in which the reserved flag bits differ from its successor will not lose any information in the one-bit scheme. This is what's implemented in Cake (except that not all the reserved bits are covered yet, only the one we use). - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Rodney, > On Dec 1, 2019, at 18:30, Rodney W. Grimes <4b...@gndrsh.dnsmgr.net> wrote: > >> Hi Jonathan, >> >> >>> On Nov 30, 2019, at 23:23, Jonathan Morton wrote: >>> On 1 Dec, 2019, at 12:17 am, Carsten Bormann wrote: > There are unfortunate problems with introducing new TCP options, in that > some overzealous firewalls block traffic which uses them. This would be > a deployment hazard for SCE, which merely using a spare header flag > avoids. So instead we are still planning to use the spare bit - which > happens to be one that AccECN also uses, but AccECN negotiates in such a > way that SCE can safely use it even with an AccECN capable partner. This got me curious: Do you have any evidence that firewalls are friendlier to new flags than to new options? >>> >>> Mirja Kuhlewind said as much during the TCPM session we attended, and she >>> ought to know. There appear to have been several studies performed on this >>> subject; reserved TCP flags tend to get ignored pretty well, but unknown >>> TCP options tend to get either stripped or blocked. >>> >>> This influenced the design of AccECN as well; in an early version it would >>> have used only a TCP option and left the TCP flags alone. When it was >>> found that firewalls would often interfere with this, the three-bit field >>> in the TCP flags area was cooked up. >> >> >> Belt and suspenders, eh? But realistically, the idea of using an >> accumulating SCE counter to allow for a lossy reverse ACK path seems sort of >> okay (after all TCP relies on the same, so there would be a nice symmetry ). >> I really wonder whether SCE could not, in addition to its current bit, >> borrow the URG pointer field in cases when it is not used, or not fully used >> (if the MSS is smaller than 64K there might be a few bits leftover, with an >> MTU < 2000 I would expect that ~5 bits might still be usable in that rate >> case). I might be completely of to lunch here, but boy a nice rarely used >> contiguous 16bit field in the TCP header, what kind of mischief one could >> arrange with that ;) Looking at the AccECN draft, I see that my idea is not >> terribly original... But, hey for SCE having an additional higher fidelity >> SCE counter might be a nice addition, assuming URG(0), urgent pointer > 0 >> will not bleached/rejected by uninitiated TCP stacks/middleboxes... > > We need to fix the ACK issues rather than continue to work around it. Ack > thinning is good, as long as it does not cause information loss. There is no > draft/RFC on this, one needs to be written that explains you can not just > ignore all the bits, you have to preserve the reserve bits, so you can only > thin if they are the same. Jonathan already fixed Cake (I think that is the > one that has ACK thinning) to not collapse ACK's that have different bit 7 > values. Well, I detest ACK thinning and believe that the network should not try to second guess the users traffic (dropping/marking on reaching capacity is acceptable, but the kind of silent ACK thinning some DOCSIS ISPs perform seems actively user-hostile). But thinning or no thinning, the accumulative signaling is how the ACK stream deals with (reasonably) lossy paths, and I think any additional signaling via pure ACK packets should simply be tolerant to unexpected losses. I fully agree that if ACK thinning is performed it really should be careful to not loose information when doing its job, but SCE hopefully can deal with whatever is out in the field today (I am looking at you DOCSIS uplinks...), no? > > Note that I consider the time of the arriving ACKS to also be informaition, > RACK for instance uses that, so in the case of RACK any thinning could be > considered bad. I am with you here, if the end-points decided to exchange packets the network should do its best to deliver these. That is orthogonal to the question whether a every-two-MSS packets ACK rate is ideal for all/most applications. > BUTT I'll settle for not tossing reserved bit changes away as a "good enough" > step forward that should be simple to implement (2 gate delay xor/or > function). Fair enough, question is more, what behavior happens out in the field, and could any other bit be toggled ACK by ACK to reduce the likelihood of an ACK filte to trigger? Best Regards Sebastian > >> Sebastian >>> - Jonathan Morton > -- > Rod Grimes rgri...@freebsd.org ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Jonathan, > On Dec 1, 2019, at 17:54, Jonathan Morton wrote: > >> On 1 Dec, 2019, at 6:35 pm, Sebastian Moeller wrote: >> >> Belt and suspenders, eh? But realistically, the idea of using an >> accumulating SCE counter to allow for a lossy reverse ACK path seems sort of >> okay (after all TCP relies on the same, so there would be a nice symmetry ). > > Sure, we did think of several schemes that used a counter. But when it came > down to actually implementing it, we decided to try the simplest possible > solution first and see how well it worked in practice. +1; simplicity has its own elegance. > It turned out to work very well, and can recover cleanly from as much as 100% > relative feedback error caused by ack loss: > > If less feedback is observed by the sender than intended by the AQM, growth > will continue and the AQM will increase its marking to compensate, ultimately > resorting to a CE mark. Well, that seems undesirable? > This is, incidentally, exactly what happens if the receiver *or* sender are > completely SCE-ignorant, and looks very much like RFC-3168 behaviour, which > is entirely intentional. > > If feedback is systematically doubled by the time it reaches the sender, > perhaps through faulty ack filtering on the return path, it will back off > more than intended, the bottleneck queue will empty, and AQM feedback will > consequently reduce or cease entirely. Only a very serious fault would > re-inject ESCE feedback once SCE marking has completely ceased, so the sender > will then grow back towards the correct cwnd after a relatively small > negative excursion. Am I right to assume that the fault tolerance requires a relative steady ACK stream though? > > The above represents both extremes of 100% relative error in the feedback, > which is shown to be safe and reasonably tolerable. Great that the current simple scheme is safe (and for my pie in the sky "let's high-jack the URG pointer" scheme essential, since there are valid existimg users of the URG mechanism, at least google tells me that both ftp and telnet are candidates; bit seem rare enough though that giving these 16+1 bits something else to do might be fun). > Smaller errors due to random ack loss are more likely, and consequently > easier to tolerate in a closed negative-feedback control loop. Fair enough. Best Regards Sebastian > > - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
I wonder if an inexpensive and credible test of the acceptability of (URG(0) && urgent pointer > 0) by middle boxes might be possible using load-testing/reachability services like NeoLoad or Pingdom? On 2019-12-01 11:35 a.m., Sebastian Moeller wrote: Hi Jonathan, On Nov 30, 2019, at 23:23, Jonathan Morton wrote: On 1 Dec, 2019, at 12:17 am, Carsten Bormann wrote: There are unfortunate problems with introducing new TCP options, in that some overzealous firewalls block traffic which uses them. This would be a deployment hazard for SCE, which merely using a spare header flag avoids. So instead we are still planning to use the spare bit - which happens to be one that AccECN also uses, but AccECN negotiates in such a way that SCE can safely use it even with an AccECN capable partner. This got me curious: Do you have any evidence that firewalls are friendlier to new flags than to new options? This influenced the design of AccECN as well; in an early version it would have used only a TCP option and left the TCP flags alone. When it was found that firewalls would often interfere with this, the three-bit field in the TCP flags area was cooked up. Belt and suspenders, eh? But realistically, the idea of using an accumulating SCE counter to allow for a lossy reverse ACK path seems sort of okay (after all TCP relies on the same, so there would be a nice symmetry ). I really wonder whether SCE could not, in addition to its current bit, borrow the URG pointer field in cases when it is not used, or not fully used (if the MSS is smaller than 64K there might be a few bits leftover, with an MTU < 2000 I would expect that ~5 bits might still be usable in that rate case). I might be completely of to lunch here, but boy a nice rarely used contiguous 16bit field in the TCP header, what kind of mischief one could arrange with that ;) Looking at the AccECN draft, I see that my idea is not terribly original... But, hey for SCE having an additional higher fidelity SCE counter might be a nice addition, assuming URG(0), urgent pointer > 0 will not bleached/rejected by uninitiated TCP stacks/middleboxes... Indeed, do we know if this was what the studies used, that Mirja Kuhlewind referred to? --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dav...@spamcop.net | -- Mark Twain ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> On 1 Dec, 2019, at 6:35 pm, Sebastian Moeller wrote: > > Belt and suspenders, eh? But realistically, the idea of using an accumulating > SCE counter to allow for a lossy reverse ACK path seems sort of okay (after > all TCP relies on the same, so there would be a nice symmetry ). Sure, we did think of several schemes that used a counter. But when it came down to actually implementing it, we decided to try the simplest possible solution first and see how well it worked in practice. It turned out to work very well, and can recover cleanly from as much as 100% relative feedback error caused by ack loss: If less feedback is observed by the sender than intended by the AQM, growth will continue and the AQM will increase its marking to compensate, ultimately resorting to a CE mark. This is, incidentally, exactly what happens if the receiver *or* sender are completely SCE-ignorant, and looks very much like RFC-3168 behaviour, which is entirely intentional. If feedback is systematically doubled by the time it reaches the sender, perhaps through faulty ack filtering on the return path, it will back off more than intended, the bottleneck queue will empty, and AQM feedback will consequently reduce or cease entirely. Only a very serious fault would re-inject ESCE feedback once SCE marking has completely ceased, so the sender will then grow back towards the correct cwnd after a relatively small negative excursion. The above represents both extremes of 100% relative error in the feedback, which is shown to be safe and reasonably tolerable. Smaller errors due to random ack loss are more likely, and consequently easier to tolerate in a closed negative-feedback control loop. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Jonathan, > On Nov 30, 2019, at 23:23, Jonathan Morton wrote: > >> On 1 Dec, 2019, at 12:17 am, Carsten Bormann wrote: >> >>> There are unfortunate problems with introducing new TCP options, in that >>> some overzealous firewalls block traffic which uses them. This would be a >>> deployment hazard for SCE, which merely using a spare header flag avoids. >>> So instead we are still planning to use the spare bit - which happens to be >>> one that AccECN also uses, but AccECN negotiates in such a way that SCE can >>> safely use it even with an AccECN capable partner. >> >> This got me curious: Do you have any evidence that firewalls are friendlier >> to new flags than to new options? > > Mirja Kuhlewind said as much during the TCPM session we attended, and she > ought to know. There appear to have been several studies performed on this > subject; reserved TCP flags tend to get ignored pretty well, but unknown TCP > options tend to get either stripped or blocked. > > This influenced the design of AccECN as well; in an early version it would > have used only a TCP option and left the TCP flags alone. When it was found > that firewalls would often interfere with this, the three-bit field in the > TCP flags area was cooked up. Belt and suspenders, eh? But realistically, the idea of using an accumulating SCE counter to allow for a lossy reverse ACK path seems sort of okay (after all TCP relies on the same, so there would be a nice symmetry ). I really wonder whether SCE could not, in addition to its current bit, borrow the URG pointer field in cases when it is not used, or not fully used (if the MSS is smaller than 64K there might be a few bits leftover, with an MTU < 2000 I would expect that ~5 bits might still be usable in that rate case). I might be completely of to lunch here, but boy a nice rarely used contiguous 16bit field in the TCP header, what kind of mischief one could arrange with that ;) Looking at the AccECN draft, I see that my idea is not terribly original... But, hey for SCE having an additional higher fidelity SCE counter might be a nice addition, assuming URG(0), urgent pointer > 0 will not bleached/rejected by uninitiated TCP stacks/middleboxes... Best Regards Sebastian > > - Jonathan Morton > ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> On 1 Dec, 2019, at 12:17 am, Carsten Bormann wrote: > >> There are unfortunate problems with introducing new TCP options, in that >> some overzealous firewalls block traffic which uses them. This would be a >> deployment hazard for SCE, which merely using a spare header flag avoids. >> So instead we are still planning to use the spare bit - which happens to be >> one that AccECN also uses, but AccECN negotiates in such a way that SCE can >> safely use it even with an AccECN capable partner. > > This got me curious: Do you have any evidence that firewalls are friendlier > to new flags than to new options? Mirja Kuhlewind said as much during the TCPM session we attended, and she ought to know. There appear to have been several studies performed on this subject; reserved TCP flags tend to get ignored pretty well, but unknown TCP options tend to get either stripped or blocked. This influenced the design of AccECN as well; in an early version it would have used only a TCP option and left the TCP flags alone. When it was found that firewalls would often interfere with this, the three-bit field in the TCP flags area was cooked up. - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
On Nov 30, 2019, at 15:32, Jonathan Morton wrote: > > There are unfortunate problems with introducing new TCP options, in that some > overzealous firewalls block traffic which uses them. This would be a > deployment hazard for SCE, which merely using a spare header flag avoids. So > instead we are still planning to use the spare bit - which happens to be one > that AccECN also uses, but AccECN negotiates in such a way that SCE can > safely use it even with an AccECN capable partner. This got me curious: Do you have any evidence that firewalls are friendlier to new flags than to new options? Grüße, Carsten ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
> On 30 Nov, 2019, at 5:42 pm, Sebastian Moeller wrote: > >>> I fear that they will come up with something that in reality will a) by >>> opt-out, that is they will assume L4S-style feedback until reluctantly >>> convinced that the bottleneck marker is rfc3160-compliant and hence will b) >>> trigger too late c) trigger to rarely to be actually helpful in reality, >>> but might show a good enough effort to push L4S past issue #16. >> >> I'm sure they will, and we will of course point out these shortcomings as >> they occur, so as to count them against issue #16. > > That might be bad position to be in though (if one party only gives > negative feed-back no matter how justified it will generate a residual > feeling of lack of good faith cooperation), I would have preferred if the > requirements would have bee discussed before. > >> Conversely, if they do manage to make it fail-safe, it is highly likely that >> their scheme will give false positives on real Internet paths and fail to >> switch into L4S mode, impairing their performance in other ways. > > Yes, so far they always err on the advantage of L4S, and justify this > with "but, latency" and if one buys the latency justification cautiously > default to rfc3168 becomes obviously sub-optimal, and so far none of the > chairs put down the "first, do no harm" hammer (and I doubt they will). We do have a political ally in the form of David Black. As one of the authors of RFC-3168, he has a natural desire to defend his work. At Singapore I believe he mostly spoke from the floor, but he is also advocating for SCE behind the scenes. He's actually quite encouraged by the situation at present, in which L4S were seen to bluster for 2+ hours without actually moving very much forward, while we were able to present some new work in a very limited time. I got the impression that failing to close most of L4S' open issues at Singapore is politically damaging to them. This is a substantial list of problems opened at Montreal, as blockers for their WGLC on publishing L4S drafts as experimental RFCs. They had all the time in the world to talk about solutions to the major showstopper problems, but were only able to concede a point that maybe tying RACK to the ECT(1) codepoint is better written as a SHOULD instead of a MUST. That lack of progress was noticed at the WG Chair level; I think they may have been giving them the rope to hang themselves, so to speak. I think they had a slide up at the side session, showing massive unfairness between L4S and "classic" flows, for a full half-hour - and they somehow thought that was *helpful* to their case! I'm reasonably sure some industry attendees also noticed this - Stuart Cheshire (of Apple) in particular. Apple have been on the front lines of enabling ECN deployment in practice in recent years. He invited me, one of the ICCRG chairs, and Bob Briscoe - among others - to dinner, where we discussed some technical distinctions and Bob demonstrated a fundamental misunderstanding of control theory. And we will have more ammunition at Vancouver. It remains to be seen how much progress they'll make… - Jonathan Morton ___ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Jonathan, thanks, more below. > On Nov 30, 2019, at 15:32, Jonathan Morton wrote: > >>> 2: Accurate ECN Feedback. >>> >>> We use a spare bit in the header of TCP acks to feed back SCE marks, and >>> the existing ECE/CWR mechanism from RFC-3168 unchanged for CE marks. The >>> SCE feedback is "accurate" but not "reliable", because it can tolerate >>> large errors (as much as 100% relative) without departing the control loop. >>> The scheme is very simple and straightforward to implement at the receiver, >>> and interpret at the sender. >>> >>> L4S uses AccECN to give CE mark feedback that is both "accurate" and >>> "reliable". It is a somewhat complex specification which takes over three >>> TCP header bits, including the two used for RFC-3168 feedback. >> >> Question: How feasible would it be for any SCE aware transport protocol to >> evaluate AccECN? This might make sense if not viewed from a technical but >> from a ietf politics perspective? >> I personally believe, that if the ECN feedback woukd e really important it >> should be packeged into TCP data as the payload has some delivery >> guarantees, while ACKs are effectively best effort (tangent: and this is why >> I consider ACK filtering/compression as abominations which should be counted >> against any guarantee the contract with the traffic-carrier entails, not >> that this helps end customers). > > It would be *possible* to use AccECN for SCE feedback, but only because the > distinction between ECT(0) and ECT(1) is fed back in a TCP option. SCE also > has no use for the "accurate" CE feedback for which the ECE/CWR bits are > replaced; if that three-bit field lay somewhere else, it could conceivably > have been used for SCE feedback instead. I guess a "proper" solution would be to get a decently sized counter and just accumulate the SCE marks the receiver side saw, similar to the acknowledgment counter, to gain reasonable robustness against lost/filtered ACK packets. I naively would just try to get access to the 16 bit URG field ;) (ducks and runs) > > There are unfortunate problems with introducing new TCP options, in that some > overzealous firewalls block traffic which uses them. This would be a > deployment hazard for SCE, which merely using a spare header flag avoids. So > instead we are still planning to use the spare bit - which happens to be one > that AccECN also uses, but AccECN negotiates in such a way that SCE can > safely use it even with an AccECN capable partner. Fair enough, I was mainly concerned about politics here. > >>> 4: TCP-friendly response to RFC-3168 CE marking. >>> >>> SCE does this by design, retaining the existing feedback mechanism for CE >>> marks and implementing an RFC-8511 (ABE) compliant response in each of the >>> TCP algorithms presented so far. We can do this easily because CE and SCE >>> information from the network is unambiguous. >>> >>> L4S presently does not do this, largely because CE marks from RFC-3168 AQMs >>> are not easily distinguished vice CE marks from an L4S AQM. They seem to >>> be working on some sort of solution, but it has not yet been demonstrated >>> to work, and their paper describing it leaves a lot of open questions >>> (including tuning constants). That we saw no demonstration of it at >>> IETF-106 (indeed they even skipped over their planned talk on it in a side >>> session dedicated to L4S) suggests to me that they found flaws that were >>> difficult to overcome at short notice, and possibly even managed to look >>> bad next to our demonstration of jitter tolerance at the Hackathon. >> >> I fear that they will come up with something that in reality will a) by >> opt-out, that is they will assume L4S-style feedback until reluctantly >> convinced that the bottleneck marker is rfc3160-compliant and hence will b) >> trigger too late c) trigger to rarely to be actually helpful in reality, but >> might show a good enough effort to push L4S past issue #16. > > I'm sure they will, and we will of course point out these shortcomings as > they occur, so as to count them against issue #16. That might be bad position to be in though (if one party only gives negative feed-back no matter how justified it will generate a residual feeling of lack of good faith cooperation), I would have preferred if the requirements would have bee discussed before. > Conversely, if they do manage to make it fail-safe, it is highly likely that > their scheme will give false positives on real Internet paths and fail to > switch into L4S mode, impairing their performance in other ways. Yes, so far they always err on the advantage of L4S, and justify this with "but, latency" and if one buys the latency justification cautiously default to rfc3168 becomes obviously sub-optimal, and so far none of the chairs put down the "first, do no harm" hammer (and I doubt they will). > >>> 5: Reduced RTT depen
Re: [Bloat] [Ecn-sane] sce materials from ietf
>> 2: Accurate ECN Feedback. >> >> We use a spare bit in the header of TCP acks to feed back SCE marks, and the >> existing ECE/CWR mechanism from RFC-3168 unchanged for CE marks. The SCE >> feedback is "accurate" but not "reliable", because it can tolerate large >> errors (as much as 100% relative) without departing the control loop. The >> scheme is very simple and straightforward to implement at the receiver, and >> interpret at the sender. >> >> L4S uses AccECN to give CE mark feedback that is both "accurate" and >> "reliable". It is a somewhat complex specification which takes over three >> TCP header bits, including the two used for RFC-3168 feedback. > > Question: How feasible would it be for any SCE aware transport protocol to > evaluate AccECN? This might make sense if not viewed from a technical but > from a ietf politics perspective? > I personally believe, that if the ECN feedback woukd e really important it > should be packeged into TCP data as the payload has some delivery guarantees, > while ACKs are effectively best effort (tangent: and this is why I consider > ACK filtering/compression as abominations which should be counted against any > guarantee the contract with the traffic-carrier entails, not that this helps > end customers). It would be *possible* to use AccECN for SCE feedback, but only because the distinction between ECT(0) and ECT(1) is fed back in a TCP option. SCE also has no use for the "accurate" CE feedback for which the ECE/CWR bits are replaced; if that three-bit field lay somewhere else, it could conceivably have been used for SCE feedback instead. There are unfortunate problems with introducing new TCP options, in that some overzealous firewalls block traffic which uses them. This would be a deployment hazard for SCE, which merely using a spare header flag avoids. So instead we are still planning to use the spare bit - which happens to be one that AccECN also uses, but AccECN negotiates in such a way that SCE can safely use it even with an AccECN capable partner. >> 4: TCP-friendly response to RFC-3168 CE marking. >> >> SCE does this by design, retaining the existing feedback mechanism for CE >> marks and implementing an RFC-8511 (ABE) compliant response in each of the >> TCP algorithms presented so far. We can do this easily because CE and SCE >> information from the network is unambiguous. >> >> L4S presently does not do this, largely because CE marks from RFC-3168 AQMs >> are not easily distinguished vice CE marks from an L4S AQM. They seem to be >> working on some sort of solution, but it has not yet been demonstrated to >> work, and their paper describing it leaves a lot of open questions >> (including tuning constants). That we saw no demonstration of it at >> IETF-106 (indeed they even skipped over their planned talk on it in a side >> session dedicated to L4S) suggests to me that they found flaws that were >> difficult to overcome at short notice, and possibly even managed to look bad >> next to our demonstration of jitter tolerance at the Hackathon. > > I fear that they will come up with something that in reality will a) by > opt-out, that is they will assume L4S-style feedback until reluctantly > convinced that the bottleneck marker is rfc3160-compliant and hence will b) > trigger too late c) trigger to rarely to be actually helpful in reality, but > might show a good enough effort to push L4S past issue #16. I'm sure they will, and we will of course point out these shortcomings as they occur, so as to count them against issue #16. Conversely, if they do manage to make it fail-safe, it is highly likely that their scheme will give false positives on real Internet paths and fail to switch into L4S mode, impairing their performance in other ways. >> 5: Reduced RTT dependence >> >> This is a mathematically interesting requirement which, at present, neither >> L4S nor SCE meets. >> >> Fundamentally, any two flows following the same congestion-signal response >> which makes average cwnd dependent solely on marking probability, and which >> share the same bottleneck queue and AQM and therefore experience the same >> marking probability, will converge to the same average cwnd and their >> relative throughputs will therefore be inversely proportional to their RTTs. >> This adequately describes both the pure AIMD response of Reno, and the >> so-called 1/p response of DCTCP (which TCP Prague apes slavishly). >> >> The steady-state cwnd formula for CUBIC, however, is a function of both >> p(CE) and RTT, such that its throughput should be proportional to the >> reciprocal quartic root of RTT, rather than linearly reciprocal. This >> assumes that CUBIC is not in its Reno compatibility regime, of course. So >> CUBIC is the standard to beat, or at least match, for this requirement. > > "Funny" story, looking at figure 6 of Høiland-Jørgensen T, Hurtig P, > Brunstrom A (2015) The Good,
Re: [Bloat] [Ecn-sane] sce materials from ietf
Hi Jonathan, > On Nov 30, 2019, at 02:39, Jonathan Morton wrote: > >> I don't see what you gain by going after the Prague requirements. They're >> internal requirements for a TCP that would fulfill the L4S goals if >> classified into the L4S side of a DualQ AQM: 'Packet Identification' means >> that the L4S AQM can identify L4S supporting flows. This seems like a >> distraction from your main pitch to me. It would seem better to compare >> against the actual goals of L4S (AFAICT, low latency at the 99th percentile, >> in the presence of Reno-compatible flows, with some fairness requirement >> which I increasingly don't understand). > > We're certainly not treating the Prague Requirements as our only goals. We > just looked over them and realised we do sufficiently well on them already to > compare favourably against L4S. They are failing on their own merits. Like > it or not, we are somewhat in competition with them in IETF space, so this > sort of comparison should help to bolster our standing. > > A brief summary of the Prague Requirements: > > > 1: Packet Identifier. > > We ID ourselves as RFC-3168 compliant using ECT(0), because we are. > > L4S has to identify itself more specifically to the network, because it is > *not* RFC-3168 compliant. It additionally relies on AQMs in the network > understanding this distinction, which at present none do. We would much > prefer that they use a DSCP for this purpose, but at present they use ECT(1). > > > 2: Accurate ECN Feedback. > > We use a spare bit in the header of TCP acks to feed back SCE marks, and the > existing ECE/CWR mechanism from RFC-3168 unchanged for CE marks. The SCE > feedback is "accurate" but not "reliable", because it can tolerate large > errors (as much as 100% relative) without departing the control loop. The > scheme is very simple and straightforward to implement at the receiver, and > interpret at the sender. > > L4S uses AccECN to give CE mark feedback that is both "accurate" and > "reliable". It is a somewhat complex specification which takes over three > TCP header bits, including the two used for RFC-3168 feedback. Question: How feasible would it be for any SCE aware transport protocol to evaluate AccECN? This might make sense if not viewed from a technical but from a ietf politics perspective? I personally believe, that if the ECN feedback woukd e really important it should be packeged into TCP data as the payload has some delivery guarantees, while ACKs are effectively best effort (tangent: and this is why I consider ACK filtering/compression as abominations which should be counted against any guarantee the contract with the traffic-carrier entails, not that this helps end customers). > > > 3: TCP-friendly response to packet loss. > > Both SCE and L4S do this without difficulty. > > > 4: TCP-friendly response to RFC-3168 CE marking. > > SCE does this by design, retaining the existing feedback mechanism for CE > marks and implementing an RFC-8511 (ABE) compliant response in each of the > TCP algorithms presented so far. We can do this easily because CE and SCE > information from the network is unambiguous. > > L4S presently does not do this, largely because CE marks from RFC-3168 AQMs > are not easily distinguished vice CE marks from an L4S AQM. They seem to be > working on some sort of solution, but it has not yet been demonstrated to > work, and their paper describing it leaves a lot of open questions (including > tuning constants). That we saw no demonstration of it at IETF-106 (indeed > they even skipped over their planned talk on it in a side session dedicated > to L4S) suggests to me that they found flaws that were difficult to overcome > at short notice, and possibly even managed to look bad next to our > demonstration of jitter tolerance at the Hackathon. I fear that they will come up with something that in reality will a) by opt-out, that is they will assume L4S-style feedback until reluctantly convinced that the bottleneck marker is rfc3160-compliant and hence wib) trigger too late c) trigger to rarely to be actually helpful in reality, but might show a good enough effort to push L4S past issue #16. > > This point has always been the main difference between L4S and SCE. > ll > > 5: Reduced RTT dependence > > This is a mathematically interesting requirement which, at present, neither > L4S nor SCE meets. > > Fundamentally, any two flows following the same congestion-signal response > which makes average cwnd dependent solely on marking probability, and which > share the same bottleneck queue and AQM and therefore experience the same > marking probability, will converge to the same average cwnd and their > relative throughputs will therefore be inversely proportional to their RTTs. > This adequately describes both the pure AIMD response of Reno, and the > so-called 1/p response of DCTCP (which TCP Prague apes slavishly). > >
