#16843: Add all bwauth measurements (from votes) ------------------------------+------------------------------ Reporter: cypherpunks | Owner: metrics-team Type: enhancement | Status: assigned Priority: Medium | Milestone: Component: Metrics/Onionoo | Version: Severity: Normal | Resolution: Keywords: tor-bwauth-needs | Actual Points: Parent ID: #24834 | Points: Reviewer: | Sponsor: ------------------------------+------------------------------
Comment (by karsten): Replying to [comment:6 teor]: > Do you need more developer time, more disk, more CPU, or more RAM? > Because we can try to make these things happen. So, I wrote a quick hack of an Onionoo that downloads and reads votes in addition to all the other descriptors. Here's what I learned: - The local descriptor cache without votes is 569M. With votes it's 3.3G. That's an increase of 494%! - The time to read 3 days of descriptors in the hourly run without votes is 10 minutes. With votes it's 16 minutes. That's an increase of 60%. (But we typically only read 1 hour of descriptors per hour.) So, I'd say we'd need these things to make this happen: - Some more disk space on the Onionoo hosts, but not really much more RAM or CPUs. (Reading descriptors is still done in a single thread, so reading more descriptors simply takes more time.) - Some developer time to write a specification patch for this new Onionoo feature, an implementation, and tests. - Some review time. '''teor''': Would you want to work on that specification patch, so that we have a better idea what we're supposed to build here? If we want to do this properly, we should also resolve a few related issues that become a bit more important by adding this feature: - Ability to use compression when downloading descriptors from CollecTor using metrics-lib (no ticket yet) - Ability to process descriptors in parallel using metrics-lib (#21365 is related) - Ability to handle large descriptor files in metrics-lib (#20395) > (Another alternative for #24834 is that we build a quick stem script to export the data we need, and import it into Relay Search.) For a short-term thing, sure. But long-term this sounds like a maintenance nightmare. (After all, Onionoo is the tool to export the data we need and import it into Relay Search.) -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16843#comment:7> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs