#30125: Port server's log sanitization to client, broker, and proxy-go -----------------------------------+----------------------------- Reporter: dcf | Owner: cohosh Type: enhancement | Status: merge_ready Priority: Medium | Milestone: Component: Obfuscation/Snowflake | Version: Severity: Normal | Resolution: Keywords: | Actual Points: Parent ID: | Points: Reviewer: | Sponsor: Sponsor19 -----------------------------------+----------------------------- Changes (by dcf):
* status: needs_review => merge_ready Comment: Replying to [comment:6 cohosh]: > Replying to [comment:4 dcf]: > > The refactoring looks good. I have a few ideas about deployment to save us some trouble later. My main goal is that there should be a clean break between the old unsanitized logs and the new sanitized logs, so that we don't later have to trawl through a log file and figure out where the change happened. This is because I'd like us to extract what we need from the old logs and then delete them. > > Thanks! This looks reasonable to me. Do you have something in mind for extracting useful data from the unsanitized logs? I suppose we could write a separate scrubber to sanitize them retroactively. For myself, I just want to make graphs showing the number of client and proxy requests per second, like these from flash proxy: * https://people.torproject.org/~dcf/graphs/fp-facilitator-201601.png * https://people.torproject.org/~dcf/graphs/fp-num-proxies-201601.png I'm fine with keeping/publishing a scrubbed version of the old logs, or e.g. CSV files derived from them. I don't think we should keep the originals indefinitely. I'll add this topic to the agenda for the next check-in meeting. > > For proxy-go, it will be similar, except that there are several /home /snowflake-proxy/*.log.d log directories. Also /home/snowflake-proxy /snowflake-proxy-*.log{,.xz} are unsanitized logs from before we started using runit log directories (happened in #28390). > I've noticed that there are a lot of old logs from different proxy-go instances. I'll set up the tarball to keep the directory structure, but I guess my question is the same as above about what we're planning on using these logs for. Yeah, the log structure changed in the past in order to allow compression and rotation, because we ran out of disk space using single files :/ (That was #28390.) For me personally, I don't have any use in mind for the old proxy-go logs and would be fine with just deleting them. > > For the client, we'll need a Tor Browser ticket to pick up the upgrade. A sample ticket and patch that can serve as a template is #26795. I know you are interested in the reproducible build and this would be a good introduction to [[doc/TorBrowser/Hacking#BuildingOfficialTorBrowserReleaseBinaries|rbm]] if you haven't used it yet. Basically, you just need to edit projects/snowflake/config and update `git_hash`, then run `make testbuild` to make sure it still builds, then open a ticket in the Applications/Tor Browser component. > Cool! I also wanted to ask you about thoughts you have about when to make snowflake client releases. I'm assuming it's just whenever there are changes we think are important to have people start using. But I also don't want to overwhelm the applications team. Yes, so far it's whenever there's a change we want people to start using. Doing it once per alpha release is not too much. As long as you test that the build works across all platforms (that's what `make testbuild` does), it's not so much trouble for the Tor Browser devs--it's when something breaks the build and they have to start backing out changes that it's cumbersome. IMO it's justified in this case and also a good excuse to file a first Tor Browser ticket. -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30125#comment:7> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs