#18910: distributing descriptors accross CollecTor instances -------------------------------+----------------------------------- Reporter: iwakeh | Owner: iwakeh Type: enhancement | Status: needs_information Priority: High | Milestone: CollecTor 1.1.0 Component: Metrics/CollecTor | Version: Severity: Normal | Resolution: Keywords: ctip | Actual Points: Parent ID: | Points: Reviewer: | Sponsor: -------------------------------+-----------------------------------
Comment (by karsten): Alright, I read the wiki page and the comment above. Just two clarifications of what I meant above: - The only code with permission to write to (and delete from) `SyncFolder` should be `DescriptorCollector`, which would also delete files as soon as they disappear remotely. We shouldn't move away or delete files while going through that directory and looking at descriptors, because that would mean that `DescriptorCollector` would have to download them again next time. Every time it runs. - There will now be two cases where we want to write a descriptor and need to check if we already have it: 1) when downloading it locally and 2) when syncing from another CollecTor instance. In the first case, if a file already exists and has different contents, we now overwrite it. It could be a descriptor we synced from another instance in a previous run or obtained earlier by downloading it from somewhere. Note that we're looking at our `out/` directory to decide whether we already have a descriptor, not at our `recent/` directory. In the second case, we simply don't store the descriptor. I think that's fine as initial strategy. If this all makes sense, feel free to work on the code, and I'll take a look once there's something to review. If it doesn't make sense yet, feel free to ask more questions. Thanks! -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:16> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online _______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs