#24218: Implement new metrics-web module for IPv6 relay statistics --------------------------------+------------------------------ Reporter: karsten | Owner: metrics-team Type: enhancement | Status: needs_review Priority: Medium | Milestone: Component: Metrics/Statistics | Version: Severity: Normal | Resolution: Keywords: | Actual Points: Parent ID: | Points: Reviewer: | Sponsor: --------------------------------+------------------------------ Changes (by karsten):
* status: new => needs_review Comment: So, I rewrote the earlier prototype into a metrics-web module that uses a PostgreSQL database. Please review [https://gitweb.torproject.org/karsten /metrics-web.git/log/?h=task-24218 my task-24218 branch]. Here are some first (meta) statistics on how it performs: - Processed five weeks of descriptors from 2017-11-01 to 2017-12-04, roughly 500M in XZ-compressed form plus recent descriptors from past three days. - Processing took ~12 minutes on my laptop. - The resulting database has a size of ~1G before vacuuming and ~150M afterwards. Remaining tasks: - Add a specification of the CSV file and three new graph pages to Tor Metrics. I'll take care of this. - Import the descriptor archive since 2008 somewhere, though not necessarily on the production system. I can take care of this, but after the first review round when it's clear whether the database schema can stay. - Find a way to test the `Database` class. I briefly tried testing it with an in-memory HSQLDB database and got it working to some extent. But we're using a few features that are specific to PostgreSQL and that we'll have to replace in these tests. The result would be that we're testing something slightly different that is similar to the PostgreSQL database but not quite the same. And the code in `Database` looks trivial enough to not contain the major bugs. I think I'd prefer to test the whole code with real descriptors as input and a real test PostgreSQL database to do the aggregation. Let's try to find a testing approach that we can later apply to other modules. (This shouldn't block either review or deployment.) - Write a specification of the new CSV file according to what we said we'll do for [https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor13 Sponsor 13]. -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24218#comment:4> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs