UDP is better for getting actual packet latency, for sure. TCP is
typical-user-experience-latency though,
so it is also useful.
I'm interested in the test and visualization side of this. If there were a
way to give engineers
a good real-time look at a complex real-world network, then they have
something to go on while trying
to tune various knobs in their network to improve it.
I'll let others try to figure out how build and tune the knobs, but the data
acquisition and
visualization is something we might try to accomplish. I have a feeling I'm
not the
first person to think of this, however....probably someone already has done
such
a thing.
Thanks,
Ben
On 7/12/21 1:04 PM, Bob McMahon wrote:
I believe end host's TCP stats are insufficient as seen per the "failed"
congested control mechanisms over the last decades. I think Jaffe pointed
this out in
1979 though he was using what's been deemed on this thread as "spherical
cow queueing theory."
"Flow control in store-and-forward computer networks is appropriate for
decentralized execution. A formal description of a class of "decentralized
flow control
algorithms" is given. The feasibility of maximizing power with such
algorithms is investigated. On the assumption that communication links behave
like M/M/1
servers it is shown that no "decentralized flow control algorithm" can
maximize network power. Power has been suggested in the literature as a
network
performance objective. It is also shown that no objective based only on the
users' throughputs and average delay is decentralizable. Finally, a
restricted class
of algorithms cannot even approximate power."
https://ieeexplore.ieee.org/document/1095152
Did Jaffe make a mistake?
Also, it's been observed that latency is non-parametric in it's
distributions and computing gaussians per the central limit theorem for OWD
feedback loops
aren't effective. How does one design a control loop around things that are
non-parametric? It also begs the question, what are the feed forward knobs
that can
actually help?
Bob
On Mon, Jul 12, 2021 at 12:07 PM Ben Greear <gree...@candelatech.com
<mailto:gree...@candelatech.com>> wrote:
Measuring one or a few links provides a bit of data, but seems like if
someone is trying to understand
a large and real network, then the OWD between point A and B needs to
just be input into something much
more grand. Assuming real-time OWD data exists between 100 to 1000
endpoint pairs, has anyone found a way
to visualize this in a useful manner?
Also, considering something better than ntp may not really scale to
1000+ endpoints, maybe round-trip
time is only viable way to get this type of data. In that case, maybe
clever logic could use things
like trace-route to get some idea of how long it takes to get 'onto'
the internet proper, and so estimate
the last-mile latency. My assumption is that the last-mile latency is
where most of the pervasive
assymetric network latencies would exist (or just ping 8.8.8.8 which is
20ms from everywhere due to
$magic).
Endpoints could also triangulate a bit if needed, using some anchor
points in the network
under test.
Thanks,
Ben
On 7/12/21 11:21 AM, Bob McMahon wrote:
> iperf 2 supports OWD and gives full histograms for TCP write to
read, TCP connect times, latency of packets (with UDP), latency of "frames"
with
> simulated video traffic (TCP and UDP), xfer times of bursts with low
duty cycle traffic, and TCP RTT (sampling based.) It also has support for
sampling (per
> interval reports) down to 100 usecs if configured with
--enable-fastsampling, otherwise the fastest sampling is 5 ms. We've released
all this as open source.
>
> OWD only works if the end realtime clocks are synchronized using a
"machine level" protocol such as IEEE 1588 or PTP. Sadly, *most data centers
don't
provide
> sufficient level of clock accuracy and the GPS pulse per second * to
colo and vm customers.
>
> https://iperf2.sourceforge.io/iperf-manpage.html
>
> Bob
>
> On Mon, Jul 12, 2021 at 10:40 AM David P. Reed <dpr...@deepplum.com
<mailto:dpr...@deepplum.com> <mailto:dpr...@deepplum.com
<mailto:dpr...@deepplum.com>>> wrote:
>
>
> On Monday, July 12, 2021 9:46am, "Livingood, Jason"
<jason_living...@comcast.com <mailto:jason_living...@comcast.com>
<mailto:jason_living...@comcast.com
<mailto:jason_living...@comcast.com>>> said:
>
> > I think latency/delay is becoming seen to be as important
certainly, if not a more direct proxy for end user QoE. This is all still
evolving and I
have
> to say is a super interesting & fun thing to work on. :-)
>
> If I could manage to sell one idea to the management hierarchy
of communications industry CEOs (operators, vendors, ...) it is this one:
>
> "It's the end-to-end latency, stupid!"
>
> And I mean, by end-to-end, latency to complete a task at a
relevant layer of abstraction.
>
> At the link level, it's packet send to packet receive
completion.
>
> But at the transport level including retransmission buffers,
it's datagram (or message) origination until the acknowledgement arrives for
that
message being
> delivered after whatever number of retransmissions, freeing the
retransmission buffer.
>
> At the WWW level, it's mouse click to display update
corresponding to completion of the request.
>
> What should be noted is that lower level latencies don't
directly predict the magnitude of higher-level latencies. But longer lower
level latencies
almost
> always amplfify higher level latencies. Often non-linearly.
>
> Throughput is very, very weakly related to these latencies, in
contrast.
>
> The amplification process has to do with the presence of
queueing. Queueing is ALWAYS bad for latency, and throughput only helps if it
is in exactly the
> right place (the so-called input queue of the bottleneck
process, which is often a link, but not always).
>
> Can we get that slogan into Harvard Business Review? Can we get
it taught in Managerial Accounting at HBS? (which does address
logistics/supply chain
queueing).
>
>
>
>
>
>
>
> This electronic communication and the information and any files
transmitted with it, or attached to it, are confidential and are intended
solely for the
use of
> the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or
otherwise
> restricted from disclosure to anyone else. If you are not the
intended recipient or the person responsible for delivering the e-mail to the
intended
recipient,
> you are hereby notified that any use, copying, distributing,
dissemination, forwarding, printing, or copying of this e-mail is strictly
prohibited. If you
> received this e-mail in error, please return the e-mail to the
sender, delete it from your computer, and destroy any printed copy of it.
--
Ben Greear <gree...@candelatech.com <mailto:gree...@candelatech.com>>
Candela Technologies Inc http://www.candelatech.com
This electronic communication and the information and any files transmitted
with it, or attached to it, are confidential and are intended solely for the
use of
the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or otherwise
restricted from disclosure to anyone else. If you are not the intended
recipient or the person responsible for delivering the e-mail to the intended
recipient,
you are hereby notified that any use, copying, distributing, dissemination,
forwarding, printing, or copying of this e-mail is strictly prohibited. If
you
received this e-mail in error, please return the e-mail to the sender,
delete it from your computer, and destroy any printed copy of it.
