Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by karsten):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 Closed the last remaining child ticket. Closing this ticket, too.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Replying to [comment:11 karsten]:
 > ...
 > > Last modified time of `status/summary` is not deterministic across
 instances.  What about using the latest time of the two valid after
 timestamps?  This would sort of synchronize Onionoo with 'Tor time'?
 >
 > Yes, sounds good!

 Child ticket #25091.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Replying to [comment:9 iwakeh]:
 > Replying to [comment:8 karsten]:
 > > Replying to [comment:6 iwakeh]:
 > > > Maybe, also use the valid after timestamp for the value in file
 `out/update` or remove the file?
 > >
 > > That might even be part of #16513. We cannot remove that file, because
 we need it for updating the index in the server part. But we could use
 something else as timestamp than the current system time. Not sure if
 valid-after timestamp is the best choice, because we have two such
 timestamps for relays and for bridges. Maybe we could use the last-
 modified time of `status/summary` here. Untested, just an idea.
 >
 > Last modified time of `status/summary` is not deterministic across
 instances.  What about using the latest time of the two valid after
 timestamps?  This would sort of synchronize Onionoo with 'Tor time'?

 Yes, sounds good!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Replying to [comment:7 karsten]:
 > ...
 > > Please find a suggestion for a patch
 [https://gitweb.torproject.org/user/iwakeh/onionoo.git/commit/?h=task-25002
 here].
 >
 > Commit 0b2e5a2 looks good! Can you rebase to master, add a change log
 entry, and provide a metrics-web patch for updating the
 [https://metrics.torproject.org/onionoo.html#parameters_order
 specification]?

 Sure. Isolated this part as child ticket.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Replying to [comment:8 karsten]:
 > Replying to [comment:6 iwakeh]:
 > > Maybe, also use the valid after timestamp for the value in file
 `out/update` or remove the file?
 >
 > That might even be part of #16513. We cannot remove that file, because
 we need it for updating the index in the server part. But we could use
 something else as timestamp than the current system time. Not sure if
 valid-after timestamp is the best choice, because we have two such
 timestamps for relays and for bridges. Maybe we could use the last-
 modified time of `status/summary` here. Untested, just an idea.

 Last modified time of `status/summary` is not deterministic across
 instances.  What about using the latest time of the two valid after
 timestamps?  This would sort of synchronize Onionoo with 'Tor time'?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Replying to [comment:6 iwakeh]:
 > Maybe, also use the valid after timestamp for the value in file
 `out/update` or remove the file?

 That might even be part of #16513. We cannot remove that file, because we
 need it for updating the index in the server part. But we could use
 something else as timestamp than the current system time. Not sure if
 valid-after timestamp is the best choice, because we have two such
 timestamps for relays and for bridges. Maybe we could use the last-
 modified time of `status/summary` here. Untested, just an idea.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Replying to [comment:5 iwakeh]:
 > Replying to [comment:4 karsten]:
 > > Replying to [comment:2 iwakeh]:
 > > > Another non-deterministic element in Onionoo docs is the order of
 entries.  When there is a tie in the used order parameter order is
 arbitrary and will differ.  A simple solution could be to always sort by
 fingerprint in case of a tie.
 > >
 > > True, that sounds easy enough to do.
 >
 > Please find a suggestion for a patch
 [https://gitweb.torproject.org/user/iwakeh/onionoo.git/commit/?h=task-25002
 here].

 Commit 0b2e5a2 looks good! Can you rebase to master, add a change log
 entry, and provide a metrics-web patch for updating the
 [https://metrics.torproject.org/onionoo.html#parameters_order
 specification]?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Maybe, also use the valid after timestamp for the value in file
 `out/update` or remove the file?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Replying to [comment:4 karsten]:
 > Replying to [comment:2 iwakeh]:
 > > Another non-deterministic element in Onionoo docs is the order of
 entries.  When there is a tie in the used order parameter order is
 arbitrary and will differ.  A simple solution could be to always sort by
 fingerprint in case of a tie.
 >
 > True, that sounds easy enough to do.

 Please find a suggestion for a patch
 [https://gitweb.torproject.org/user/iwakeh/onionoo.git/commit/?h=task-25002
 here].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Replying to [comment:2 iwakeh]:
 > Another non-deterministic element in Onionoo docs is the order of
 entries.  When there is a tie in the used order parameter order is
 arbitrary and will differ.  A simple solution could be to always sort by
 fingerprint in case of a tie.

 True, that sounds easy enough to do.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Replying to [comment:1 iwakeh]:
 > `RdnsLookupRequest` leads to many differences between the summary and
 the details documents of two instances even when run from the same machine
 within a short period of time.  These are also the cause of most
 differences between the two main tp.o instances.

 True. We might consider running these lookups in CollecTor and providing
 results using a new data format. Onionoo would then fetch and process
 those files.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 Another non-deterministic element in Onionoo docs is the order of entries.
 When there is a tie in the used order parameter order is arbitrary and
 will differ.  A simple solution could be to always sort by fingerprint in
 case of a tie.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by iwakeh):

 `RdnsLookupRequest` leads to many differences between the summary and the
 details documents even when run from the same machine within a short
 period of time.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #25002 [Metrics/Onionoo]: Make data and results from Onionoo deterministic

[Tor Bug Tracker & Wiki]

#25002: Make data and results from Onionoo deterministic
-+--
 Reporter:  iwakeh   |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   |   Keywords:
Actual Points:   |  Parent ID:
   Points:   |   Reviewer:
  Sponsor:   |
-+--
 As started with #16513 all documents and data generated by Onionoo should
 be deterministic with reasonable effort.

 One other step is discussed
 [https://trac.torproject.org/projects/tor/ticket/16513#comment:38 here].

 More topics will be added as comment, when they appear.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs