Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread starlight . 2015q3
aAt 20:56 11/5/2015 -0600, you wrote: >On 5 November 2015 at 16:37, wrote: >> By having a single thread handle >> consensus retrieval and sub-division, >> issues of "lost" relays should >> go away entirely. > >So I'm coming around to this idea, after spending >an hour trying to explain why it was

Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread Tom Ritter
On 5 November 2015 at 16:37, wrote: > At 11:47 11/5/2015 -0600, Tom Ritter wrote: >> . . . >>So them falling between the slices would be my >>best guess. . . > > Immediately comes to mind that dealing > with the changing consensus while > scanning might be handled in a different > but nonetheless

Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread starlight . 2015q3
At 17:37 11/5/2015 -0500, you wrote: > >. . .Consensus allocation worker. . The consensus list manger could run as an independent Python process and "message" changes to the scanner processes to avoid complexities of trying to share data (I know very little about Python and whether sharing data i

Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread starlight . 2015q3
At 11:47 11/5/2015 -0600, Tom Ritter wrote: > . . . >So them falling between the slices would be my >best guess. . . Immediately comes to mind that dealing with the changing consensus while scanning might be handled in a different but nonetheless straightforward manner. Why not create a snapshot

Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread starlight . 2015q3
At 11:47 11/5/2015 -0600, Tom Ritter wrote: > . . . >So them falling between the slices would be my >best guess. . . Immediately comes to mind that dealing with the changing consensus while scanning might be handled in a different but nonetheless straightforward manner. Why not create a snapshot

Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread Tom Ritter
Talked with Mike on IRC: 12:12 < tjr:#tor-dev> mikeperry: If you have a moment today, we'd appreciate it if you could peek at the tor-dev thread 'stale entries in bwscan.20151029-1145' 12:14 < mikeperry:#tor-dev> that seems to be one of the mails I lost 12:14 < mikeperry:#tor-dev> (had a mail fail

Re: [tor-dev] stale entries in bwscan.20151029-1145

2015-11-05 Thread Tom Ritter
[+tor-dev] So... weird. I dug into Onyx primarily. No, in scanner.1/scan-data I cannot find any evidence of Onyx being present. I'm not super familiar with the files torflow produces, but I believe the bws- files list what slice each relay is assigned to. I've put those files (concatted) here: h