@Sebastian: This is a really great list of what the issues were in the EU, and if y'all can repost there, rather than here, perhaps more light will be generated outside our circles.
https://news.ycombinator.com/item?id=37694306#37694307 On Thu, Sep 28, 2023 at 12:31 PM Sebastian Moeller <moell...@gmx.de> wrote: > > > > > On Sep 28, 2023, at 18:38, Dave Taht <dave.t...@gmail.com> wrote: > > > > On Wed, Sep 27, 2023 at 11:25 PM Sebastian Moeller <moell...@gmx.de> wrote: > >> > >> Hi Dave, > >> > >> please excuse a number of tangents below ;) > > > > It would be nice, if as a (dis)organisation... the bufferbloat team > > could focus on somehow getting both sides of the network neutrality > > debate deeplying understanding the technological problem their > > pre-conceptions face, and the (now readily available and inexpensive) > > solutions that could be deployed, by most ISPs, over a weekend. We are > > regularly bringing up a few thousand people a week on libreqos (that > > we know of), and then of course, there are all the home routers and > > CPE that are increasingly capable of doing the right thing. > > > > The time to try and shift the memes in in the coming days, and weeks. > > > > So ya'all being distracting below... aggh... ok, I'll bite. > > [SM] Sorry to drag you into the weeds... > > > > > >> > >> > >>> On Sep 27, 2023, at 20:21, Dave Taht via Rpm <r...@lists.bufferbloat.net> > >>> wrote: > >>> > >>> Jason just did a beautiful thread as to what was the original source > >>> of the network neutrality > >>> bittorrent vs voip bufferbloat blowup. > >>> > >>> https://twitter.com/jlivingood/status/1707078242857849244 > >> > >> But the core issue IMHO really was an economic one, the > >> over-subscription ratios that worked before torrenting simply did not cut > >> it any more in an environment when customers actually tried to use their > >> contracted access rates "quantitatively". In my outsider perspective an > >> ISP owes its customers the contracted rates (with some slack*) and any > >> sort of over-subscription is a valid economic optimization an ISP might > >> take, IFF that ISP is prepared to rapidly increase segment capacity (or > >> down-grade customer plans)) if the deployed over-subscription rate proves > >> to have been too optimistic. Mind you, most ISPs market plans by access > >> speed (and charge more for higher speeds) and hence are somewhat > >> responsible to actually deliver (again with some slack) these speeds. > > > > The average use at peak hours for home broadband is below 5mbits per > > subscriber regardless of on a 25mbit or gbit plan. Remarkably, > > business plans are less (but more bursty). Oversubscription within a > > set of parameters is needed because, in part because upstream > > bandwidth to the internet remains expensive (although I hope that cost > > continues to drop rapidly), and in part, because it is needlessly > > expensive to provision exactly the contracted rate to the customer. As > > one example, I use the inverse square law as a rule of thumb for > > wireless deployments - the range you can get from a 25/10 deployment > > is geometrically better than 100/25, and the average bandwidth usage > > nearly the same. > > [SM] +1; hence my argument is not oversubscription per se is bad, but > that oversubscription needs to be managed and the level of oversubscription > needs to to be adapted ot he actual usage patterns. I am sure however that > ISPs already actively do that (I assume most ISPs want happy customers). > > > > Agreeing on industry standards for what the "slack" should be, might be of > > help. > > [SM] Not being part of that industry at all I would love to see that > as well, but can not contribute to defining that in any way. > > > > > >> > >> > >> *) Claiming "Up to", only carries that far, if you sell me X and I mostly > >> get Y (with Y close to X) and occasionally Z (with Z << X), that is > >> acceptable unless occasionally is "at every late afternoon and evening" > >> prime-time. > > > > We have discussed elsewhere, the idea of a minimum contracted rate > > (down to), which is easier to reason about. > > [SM] Indeed, the German regulator (and I am not saying this is the > only or best option) decided to require ISPs to give 6 numbers pre-contract > signing: 3 for up- and 3 for downstream: a maximal (net) rate, a minimal > (net) rate, and a usually achievable (net) rate, all rates were defined as > IP/TCP goodput to make verification easier on end-users. The regulator also > defined a measurement regime that end-users can follow to check whether the > ISP is fulfilling the contract and the law gives remedies if ISPs do not > deliver (either the right to immediately cancel the contract and/or the right > to adapt the monthly payments to the actually delivered ratio of the > contracted rates*). I think I need to add, that ISPs can set these numbers > freely, but are only allowed to advertise with the maximum rate. > But if we talk about a single number per direction, minimal rate is > probably easiest, or something like a "usually achievable rate" (that needs > to be met or exceeded in say 90% of >= 20 measurements or so). > > > *) In a ironic twist however the regulator so far has not explained how > deviations in each of the 6 numbers above should be aggregated to get one > single contract rate achievement ratio..., most ISPs took measures into their > own hands and simply offer customers typically a permanent rebate of 5 EUR or > immediate cancelation what ever the customer prefers.... > > > > > >> > >>> > >>> Seeing all the political activity tied onto it since (and now again) > >>> reminds of two families at war about an incident that had happened > >>> generations and generations before, where the two sides no longer > >>> remembered why they hated each other so, but just went on hating, and > >>> not forgiving, and not moving on. > >>> > >>> Yes, there are entirely separate and additional NN issues, but the > >>> technical problem of providing common carriage between two very > >>> different network application types (voip/gaming vs file transfer) is > >>> thoroughly solved now, > >> > >> I am not sure this was at the core of the problem, my take is > >> really that "always-on" and relative upload-heavy torrent simply > >> demonstrated painfully for all involved that the old oversubscription > >> ratios were not suitable for the changed traffic profiles. I have some > >> sympathy for the ISPs here, they certainly did not try to sell their > >> customers short (good ISPs try to retain their customers and that works > >> best when customers are happy with the service) and having this problem > >> appear on many segments at the same time is not a fun place to be, and > >> upload was/is often (too) low in DOCSIS segments anyway; but this is why > >> e.g. my bit torrent could affect your VoIP, simply by pushing the whole > >> segment into upload capacity congestion... (ISPs in theory could fix this > >> by plain old QoS engineering, but the issue at hand was with a non-ISP > >> VoIP/SIP service and there QoS becomes tricky if the ISP as these packets > >> need to be identified in a way that is not game-able**) > > [SM] See my later mail to Jason, I will not claim I know Comcast's > issues better than him and will accept that self-congestion also played a > major role in the initial problem. Over here in Europe the net neutrality > debate was kicked of less by an unfortunate confluence of new usage profiles > and traditional oversibscription ratios and less than ideal packet > scheduling, but rather by a series of pretty apparent attempts at restricting > consumer choice, see eg. > https://www.accessnow.org/wp-content/uploads/archive/docs/Net_Neutrality_Ending_Network_Discrimination_in_Europe_Access_v3.pdf > for an admitted slightly biased position: > > " • Blocking of applications and services: In order to maximise > profits, some ISPs – that also offer their own services and applications > online – exclude certain services and applications of competing market > players. The most prominent case of this form of network discrimination is > European mobile providers (like Deutsche Telekom) blocking or restricting the > use of Voice over IP (VoIP) services (like Skype and Viber) for their > customers.20 > > • Slowing or “throttling” internet speeds: Some ISPs slow down > specific services (like YouTube) and applications (like Skype), or ask users > to pay an extra fee to have access to these internet platforms. Given the > high latency (delay) sensitivity of many applications, ISPs are able to > compromise the correct functioning of these services by slowing them down, > preventing the services from running properly. Often ISPs – especially > telecommunication companies – do this to favour their own voice calling > services over VoIP services, thereby crushing competition. > > > • Blocking websites: ISPs often block websites for a number of > reasons – to secure their network, or to avoid competition, and sometimes for > social, public relations or political reasons. In the UK, for instance, > Orange Telecom blocked the French digital rights advocacy group, La > Quadrature du Net’s website on pre-paid mobile accounts.21 > > • Preferential treatment of services and platforms: ISPs can also > impose data caps on internet access contracts while granting data allowance > exceptions to a company’s own proprietary streaming services (like Deutsche > Telekom to its own “T-Entertain”).22 They can (and do) also grant > preferential treatment to select services – such as Orange France with the > popular music streaming service Deezer23 – ahead of other competitors, > effectively imposing anti-competitive limitations on markets such as those > for legal online music. Moreover, generally only large, well-established > companies can afford this preferential treatment, resulting in a further > stifling of innovation." > > These look like clearer attempts to monetize the ISP position as gate-keeper > to his customer's eye-balls (for content providers) as well as gate-keeper to > the internet for for paying customers. > I find it much harder to have sympathy for the listed examples of ISP > behavior than the situation of rapidly changing usage pattern posed where > technical changes needed to be made, but where no attempt at unfair > monetization was evident, but maybe I am overly sensitive. These are all > examples that make me personally applaud the EU for its intervention to make > clear rules what "internet access providers" can and can not do. (The EU also > was quite flexible in applying/interpreting its rules during the covid > isolation periods, when it made it clear that e.g. treating certain traffic > classes e.g. streaming media to lower priority than video conferences was > within the neutrality framework as long as it was based on traffic class and > not on specific service providers IIRC). > > > > > > > Torrent problem thoroughly solved with FQ on the path and backpressure > > from the mac scheduler. > > > > https://perso.telecom-paristech.fr/drossi/paper/rossi14comnet-b.pdf > > [SM] Thanks for the link. This is mainly for the self-congesrion case? > > > > > >> I agree that on a single link we mostly solved the problem (a few > >> corner cases are left on links with very low capacity, where essentially > >> we can only manage the pain, not remove it)... > >> > >> > >> **) Which is not rocket science either, a VoIP stream takes ~100 Kbps, so > >> in theory an ISP might simply allow each customer say 5 VoIP stream > >> equivalents by allowing up to 500Kbps od traffic marked with a specific > >> DSCP as higher priority (with higher access probability for the shared > >> medium). I am not sayng this is my preferred solution, just saying this is > >> a solution that would have been available at the time if I memorize my > >> docsis capabilities correctly. > >> > >> > >>> and if only all sides recognised at least this > >>> much, and made peace over it, and worked together to deploy those > >>> solutions, maybe, just maybe, we could find mutually satisfactory > >>> solutions to the other problems that plague the internet today, like > >>> security, and the ipv6 rollout. > >> > >> +1. IPv6 is IMHO a prime example where the regulators should stop > >> talking softly and start showing the big stick they carry. > > > > I really would like to see a push for IPv6 mandated as a part of the > > BEAD programs. > > [SM] +1; I am not the biggest IPv6 fan, but that's what we have and > it is mostly serviceable so let's get this "party started" finally. > > > > >> Heck in Germany we have ISPs that still only supply CG-NATed IPv4 > >> addresses only.. (most mass market ISPs do much better in that regard, but > >> for the stragglers it would help if the regulator would demand IPv6 with > >> PD***). Regarding security, the easiest way to achieve that would be to > >> put some heavy requirements on IoT manufacturers and vendors (like do what > >> you please as long as you are local LAN only, but once you reach out into > >> the cloud you need to fulfill the following list of requirements, with > >> timely security updates over a reasonable long usage period), however that > >> might not be very attractive for politicians/regulators to tackle (active > >> regulatory acts tends to get bad press unless something bad happened, but > >> even then the press often complains about the acts coming too late, but I > >> digress****) > > > > There is a separate NRPM going on for the new cybersecurity label at > > the FCC. If I had time, and a partner, > > we could rework the doc we did a few years ago. It is due oct 6. > > > >> > >> > >> ***) Strictly speaking IPv6 is required, since "internet access" is > >> defined as reaching all of the internet (as far as in the ISPs power) and > >> IPv6-only sites are not reachable for the CG-NAT-only customers. But so > >> far the local regulator does not seem to enforce that requirement, or > >> hopefully is working on this quietly behind the curtains. > >> > >> ****) This is not to diss the press, they are doing what they are supposed > >> to do, but it just shows that active regulation is a tricky business, and > >> a light touch typically "looks better" (even though I see no real evidence > >> it actually works better). > >> > >>> If anyone here knows anyone more political, still vibrating with 10+ > >>> years of outrage about NN on this fronts, on one side or the other, if > >>> you could sit them down, over a beer, and try to explain that at the > >>> start it was a technical problem nobody understood at the time, maybe > >>> that would help. > >> > >> So in the EU that debate is essentially settled, there is a EU > >> regulation that essentially spills out what ISPs owe their customers and > >> that has become the law of the land. The rationale for required un-biased > >> service and freedom to select terminal devices is well justified by the > >> market ideals of the EU, allowing ISPs to discriminate packets or terminal > >> devices restricts the market and will lead to undesired outcomes. (Fun > >> fact most big players in capitalist societies argue for "free markets" but > >> at the same time act to work-around the market mechanism by trying to move > >> the market into an oligo- or even monopoly condition, which is why strong > >> regulation is required*****). > > > > Governments make safer markets feasible. > > [SM] Yes, I agree, both in markets are a pretty decent tool, but need > constant maintenance. > > Regards & Sorry for the tangent > Sebastian > > > > >> > >> *****) This is akin to professional sports where the audience generally > >> accepts that referees are necessary and occasionally need to make > >> "painful" calls, as the alternative would be anarchy, but I digress. > >> > >> Regards > >> Sebastian > >> > >> > >>> > >>> -- > >>> Oct 30: > >>> https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html > >>> Dave Täht CSO, LibreQos > >>> _______________________________________________ > >>> Rpm mailing list > >>> r...@lists.bufferbloat.net > >>> https://lists.bufferbloat.net/listinfo/rpm > >> > > > > > > -- > > Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html > > Dave Täht CSO, LibreQos > > <2510.jpeg> > -- Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html Dave Täht CSO, LibreQos _______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat
