#29370: Measure mode with arbitrary tgen traffic models
---------------------------------------+------------------------------
Reporter: irl | Owner: metrics-team
Type: enhancement | Status: reopened
Priority: Low | Milestone:
Component: Metrics/Onionperf | Version:
Severity: Normal | Resolution:
Keywords: metrics-team-roadmap-2020 | Actual Points: 0.1
Parent ID: #33321 | Points: 1
Reviewer: | Sponsor: Sponsor59
---------------------------------------+------------------------------
Comment (by karsten):
Thank you, robgjansen and acute, for your comments above! The more I think
about this feature, the more I come to the conclusion that we do not need
it.
I'll start with addressing acute's thoughts:
Replying to [comment:9 acute]:
> Having a pass-through feature could be very useful for research. For
example, evaluating Tor performance with clients in mobile or other type
of bandwidth-constrained networks would require a model that minimises the
used bandwidth; or if a user wanted to fill the pipe for a congestion
control experiment a larger file size model would be needed etc.
I can see how different network environments would require different
measurement models. But maybe we can identify how these models should
differ and then add parameters to OnionPerf's command line that feed into
the generated TGen models. For example, the file size of the downloaded
file could easily be a `--filesize` parameter on the command line.
> If there is no appetite for implementing this feature, what we could
have instead is documentation that explains to users how to use their own
model if they want to, and keep our own models (including oneshot)
internally as suggested here.
If researchers need to change parts of a model that cannot be configured
using the OnionPerf command line interface, they will have to change the
OnionPerf sources to do what they want. I'd say that that's still easier
than editing a TGen XML file. If the missing piece is better
documentation, we should provide that.
Now to robgjansen's thoughts:
Replying to [comment:8 robgjansen]:
> The use case was to allow OnionPerf to measure traffic patterns other
than the usual "file download" pattern (the model that OnionPerf generates
for itself internally). So, for example, you could set up a tgen traffic
model to act like a ping utility, sending a few bytes to the server side
and back to the client to measure circuit round-trip times.
The ping model is something that we're considering to implement in #30798,
so let's consider what that would entail:
- We would need a different TGen client model that sends a few bytes
every second or every configurable amount of time.
- We would also need to update the analysis code in OnionPerf. We'd no
longer be interested in elapsed seconds between start and reaching a
certain stage of the download. We'd want to extract the time between
sending a ping and receiving a pong, for every one of them.
- In fact, we might want to parse the TGen server logs as well, to learn
when a ping arrived and the pong was sent back. We have that information,
which is not available to the typical ping application, so why not use it.
- We would have to update the visualization code to extract and display
different metrics than in the bulk download case.
Thinking about different traffic models, what if we wanted to measure
something like an `HTTP POST` rather than the `HTTP GET`? I'd assume that
we'd have to provide a different TGen ''server'' model file as well, but I
don't know for sure. If that's still possible with replacing just the TGen
client model, there's probably another model that requires a custom TGen
server model which we just didn't think of yet.
All in all, it's more than just ''the'' TGen model. We'd have to write a
fair amount of code in order to implement a useful ping model in
OnionPerf.
> My initial idea was not to make OnionPerf generate these models, but
rather to create them externally and have OnionPerf just "pass them
through" to tgen. That means whoever generated the models would need to
correctly set the server addresses and SOCKS ports, etc.
>
> I'm not sure that Tor wants this feature for OnionPerf. An alternative
could just be to wait until you have a specific model in mind that you've
decided you want to start measuring, and then generate that model
internally as we do now with the 5MiB downloads. In that case the pass-
through feature would not be needed, and you wouldn't have to maintain
something that you don't use.
>
> Do you think the pass-through feature is actually useful for Tor, or
does generating models internally make more sense?
>
> (The tgen models in Shadow are generated with the correct
addresses/ports during the phase when we generate the Shadow experiment
configuration. This is currently done using the Shadow-Tor config
generator [https://github.com/shadow/tornetgen here].)
If there had been existing models that we could have plugged in easily,
that would have been a good argument in favor of this feature. But it
seems like we could as well reuse the model-generating code from the
Shadow-Tor config generator if we wanted to support these models in
OnionPerf. And still, we would have to write new analysis and
visualization code to evaluate those new measurements.
The internally generated model also has the advantage that it's easier to
use. All it takes to start a measurement is a (potentially quite long)
command with several parameters. But it doesn't require a (still
potentially long) command plus one or two files. Describing the experiment
would then be a matter of listing all software versions and the OnionPerf
command used to start measurements.
My suggestions are that we:
- make the current bulk transfer model more configurable by adding
parameters like initial pause, transfer count, or filesize as part of
#33432;
- develop a ping model as internal model where OnionPerf generates TGen
files, plus analysis and visualization code, as part of #30798, assuming
there's need for developing such a model; and
- remove the `-m/--traffic-model` parameter from the codebase and close
this ticket as something we considered carefully but decided against.
Oops, this comment was longer than I had expected when starting to write
it. Thanks for making it to the end, and thanks again for sharing your
thoughts above. I'm open to discussing this more if there are aspects that
I didn't acknowledge as much as I should.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29370#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs
