Hi Thibaut,
> On Nov 17, 2022, at 16:53, Thibaut <ha...@slashdirt.org> wrote:
>
> Hi Sebastian,
>
>> Le 17 nov. 2022 à 16:19, Sebastian Moeller <moell...@gmx.de> a écrit :
>>
>> Hi Thibaut,
>
> [...]
>>>>> Although I must confess that it certainly feels counter-intuitive that
>>>>> for ethernet (and FTTH) we suggest a higher overhead than e.g.
>>>>> VDSL2/cable (which themselves run off an ethernet interface).
>>>>
>>>> That is simply because for (ADSL) DOCSIS VDSL2 we have a much clear
>>>> picture about what we need to account for. And AON can be essentially
>>>> using true ethernet encapsulation so we can expect an unspecified "FTTH"
>>>> class to encompass a broad set of different encapsulations, if we want to
>>>> recommend a single number that should also cover AON (the PONs are much
>>>> less clear*). Why do you assume that FTTH necessarily would have a smaller
>>>> per-packet-overhead than other link technologies?
>>>
>>> I’m not (necessarily) making that assumption for FTTH (although I would
>>> expect it to be the case, from my limited understanding of FTTH
>>> technologies),
>>
>> Well, all I can say about encapsulations and per-packet-overhead is:
>> it's complicated.
>
> I can see that now :)
>
>>> I am however certainly making that assumption for plain ethernet, which is
>>> included in the 44-byte documentation case. I think it’s a reasonable one?
>>
>> Yes, the goal is to give one number that works everywhere so 44 is what
>> we recommend. As I said overestimating the overhead is benign,
>> underestimating it is not. True ethernet has an effective*
>> per-packet-overhead of:
>> 7 Byte Preamble + 1 Byte start of frame delimiter (SFD) + 12 Byte inter
>> frame gap (IFG) + 4 Byte Frame Check Sequence (FCS) + 6 (dest MAC) + 6 (src
>> MAC) + 2 (ethertype)
>> 7 + 1 + 12 + 4 + 6 + 6 + 2 = 38 bytes
>
> And now I’m even more confused: why do we suggest an overhead of 34/26 for
> VDSL or 22 for docsis cable then? I assume the respective modems are
> connected via ethernet, aren’t they?
For DOCSIS you will be correct as there are no DOCSIS modem-routers
that run OpenWrt, for DSL we do have some devices (older xrx200 and in
development newer vrx518 lantiq/maxlinear SoCs) that actually have DSL
interfaces.
This is a bit of a judgement call, we only need to account for those overheads
that actually happen on the bottleneck link of a path.
For DSL/PTM the dsl overhead is smaller than the full ethernet overhead as some
parts of ethernet encapsulation are not transmitted via the DSL link.
So let's run the numbers for a router with gigabit ethernet port connected to a
DSL modem with gigabit ethernet port. At what DSL rate will the added ethernet
overhead actually matter and to save some time let's only look at the worst
case of small packets (as there the relative overhead cost is larger):
Ethernet max payload rate for minimal packet size:
1000 * ((64-18)/(64+20)) = 547.619047619 Mbps (true ethernet adds 20 bytes
equivalent on top of the L2 ethernet frame)
DOCSIS:
(1000 * ((64-18)/(64+20))) * ((64)/(64-18)) = 761.904761905 Mbps
Now use this payload rate to see how much raw DSL rate we would need to
meet/exceed that rate:
(1000 * ((64-18)/(64+20))) * (65/64) * ((64+4)/(64-18)) = 822.172619048 Mbps
(PTM adds 4 bytes on top of the L2 ethernet frame
https://en.wikipedia.org/wiki/Ethernet_frame)
Currently VDSL2 is limited to ~250 Mbps, so we are far off from the true
ethernet overhead becoming relevant for our traffic shaper.
HOWEVER, if the link between router and modem is only 100 Mbps ethernet the
picture changes, and indeed when using a 100 Mbps modem with a 100Mbps VDSL2
link I used `rate 95 overhead 50` (why 50 and not 34+20=54 you might ask?
because on the link between modem and router I did not use a VLAN tag, but had
the modem add this tag later).
As I said, it's complicated.
It is similar for DOCSIS, except there rates in the >= 1 Gbps range are a thing
since some time already and unless rputer and modem use a 2.5/5/10/... Gbps
connection the full ethernet overhead becomes relevant. Time to change the
docsis recommendations for say rates > 500 Mbps.
ADSL/ATM/AAL5 is essentially only used for speeds below 25 Mbps with devices
with at least 100Mbps ethernet ports so let's ignore this for now:
>
> Another confusing point in the current detailed doc (and possibly the one
> that got me started):
> " • Link Layer Adaptation calculates the proper overheads for WAN links
> such as DSL and ADSL. If you use any kind of DSL link, you should review this
> section."
>
> This seems to imply LLA is only necessary for « exotic » links such as
> ADSL/VDSL.
This is historic, LLA is exceptionally important for ATM/AAL5 as there
the effectove overhead depends on the actual packet size, the importance of
proper overhead accounting for all link technologies only became obvious later
(one naively assumes that this can be fixed by setting the shaper somewhat
lower, but as the section about overhead and shaper rate shows it is not as
easy).
> This also brought me to the (naïve) comment that it would be nice if CAKE
> would account for local NIC L2 overhead internally (since that info is
> readily available to it, is it not?), and one would only have to configure
> the actual « overhead » of whatever is connected to the local NIC. There
> would be less margin for error I believe.
You need to account for the relevant overhead on the bottleneck link,
which often is not an L1/L2 interface of the router itself. Cake has a number
of technology specific link layer accounting "sets" prepared you can activate
with a keyword, but that does not solve the hard question, which overhead is
actually to be accounted for (see the example of VDSL connected over ethernet
above with the to be accounted for overhead depending on the ratio od DSL rate
to ethernet rate).
> Then I saw this:
>
> "Please note that as of middle 2018 cake, and cake only, will try to
> interpret any given overhead to be applied on top of IP packets, all other
> qdiscs (and cake if configured with the “raw” keyword) will add the specified
> overhead on top of the overhead the kernel already accounted for. This seems
> confusing, because it is ;) so if in doubt stick to cake."
>
> Well yes, it *is* very confusing :P
Yepp, on ethernet interfaces the kernel will typically only account for
14 bytes of the total 38 bytes of ethernet framing, as the kernel maintain
these fields (MAC addresses and ethertype) itself while the other parts are
mostly handled by the driver of offloaded onto the NIC, so the kernel is not
doing anything wrong by not reporting these other parts.
>
>> Now if a VLAN tag is added you end up with 38+4 = 42 so both pure ethernet
>> and ethernet+VLAN have a er-packet-overhead close to 44 bytes.
>>
>> *) The IFG is not real bytes, but transmission silence for the time it would
>> take to transmit 12 octets.
>>
>>>
>>>> Now, if you have reliable information for say GPON-overhead, by all
>>>> means add it (but to be useful this also should contain an easy identifier
>>>> for other users to figure out whether they use GPON in the first place).
>>>>
>>>> The bigger point however is IMHO, from the perspective of minimizing
>>>> bufferbloat/latency-under-load-increase using a slightly too large
>>>> per-packet-overhead is fully benign, while specifying to small an overhead
>>>> can easily result in measurable bufferbloat increase. So IMHO it is fine
>>>> to err on the side of too large when estimating the per-packet-overhead.
>>>
>>> OK. Although I would think people reading the detailed explanation are
>>> looking for precise info and explanations, not one-stop-shop approximations.
>>
>> Sure, but the problem is we can not give them that "precise
>> information" them, you , me would like to have, so we do the best we can.
>>
>>> Not to mention the kind of users who want to squeeze the maximum
>>> performance out of their setup.
>>
>> Just read the section where I explain how per-packet-overhead and
>> shaper-rate are not orthogonal variables to understand how problematic the
>> whoe thing is, add to it that ISPs often employ traffic shapers with
>> potentially independent overhead-assumptions making the whole problem even
>> harder.
>
> Yes, I see it’s a bit of a headache :)
+1; the whole wiki would read easier if the subject matter was less
complicated ;)
>
>>>> *) The core problem is that we have no straight forward way to empirically
>>>> deduce the effective per-packet-overhead over an arbitrary network path,
>>>> so if a link technology defines/documents the overhead well and
>>>> conclusively (like docsis) we are in luck, but if the details a vague or
>>>> leave many options to the ISP we have to make an educated guess.
>>>>
>>>>
>>>>> I would also like to see some info about ppp vs ethernet interface in
>>>>> there (matching your previous email): unless you beat me to it I will add
>>>>> it.
>>>>
>>>> I will not beat you to it, because for users of cake it does not matter
>>>
>>> You said the exact opposite in your previous email (persistence of
>>> statistics) :)
>>
>> I misunderstood your argument here and thought you wanted a section
>> explaining the issues with getting the overhead numbers for pppoe** and
>> ethernet interfaces set for HTB+fq_codel, sorry. If you want to explain
>> differences between cake on pppoe or on ethN go for it.
>
> OK but I also believe it’s important to clarify that the overhead setting
> needs not be adjusted whether we shape the pppoe interface or its underlying
> ethernet one: that certainly was counter-intuitive for me.
+1; go for it!
>
> Thanks for bearing with my ignorance,
This is how wikis improve if open questions are discussed and text
changed.
> T
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel
