Re: [tor-bugs] #27181 [Metrics/Onionoo]: Avoid unnecessary disk writes

[Tor Bug Tracker & Wiki]

#27181: Avoid unnecessary disk writes
-+--
 Reporter:  irl  |  Owner:  metrics-team
 Type:  enhancement  | Status:  closed
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  wontfix
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by karsten):

 * status:  new => closed
 * resolution:   => wontfix


Comment:

 This ticket is mostly obsolete with #32660 where we compare the digest of
 an existing file with the digest of the serialized string that we were
 about to write. We could indeed do more by remembering whether a status
 object has changed before serializing it and comparing checksums. But it's
 unclear whether that development and testing effort would be well spent.
 I'd say as long as I/O remains the critical resource we should keep what
 we have and just spend that extra CPU time on serializing and computing
 digests. I'll close this ticket as something we're very unlikely to do,
 but feel free to reopen it if you object.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #27181 [Metrics/Onionoo]: Avoid unnecessary disk writes

[Tor Bug Tracker & Wiki]

#27181: Avoid unnecessary disk writes
-+--
 Reporter:  irl  |  Owner:  metrics-team
 Type:  enhancement  | Status:  new
 Priority:  Medium   |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   |   Keywords:
Actual Points:   |  Parent ID:
   Points:   |   Reviewer:
  Sponsor:   |
-+--
 Right now if you run the updater, every status document will be touched by
 Onionoo regardless of whether or not the content actually changed.
 Additionally every file in the out directory will be touched, although I'm
 less concerned about this as I've seen that the out directory is
 reproducibly built from the status directory where no changes have
 occurred. My primary motivation for this ticket is enabling consistency in
 backups and the creation of test instances where we'd like to copy out the
 status directory between updater runs, where there is a limited time
 window.

 I propose that we add a field to the Document class that marks a document
 as having changed since it was loaded from disk. Initially this flag would
 be clear but if any field in the document is changed then this would
 trigger a write. There may be conditions where a value is set to the value
 it already has and we should detect these cases and not set the flag to
 perform a write.

 Alternatively, we can store the serialization of the document along with
 the parsed fields. When serializing the document we can compare the
 document with its original serialization to determine if it needs to be
 written to disk.

 We will need to have good test coverage for this as if we miss setting the
 flag to write data we may end up with gaps in data on the Onionoo hosts.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs