Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+-
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  closed
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+-

Comment (by karsten):

 Replying to [comment:19 Vort]:
 > **@karsten**, one more 500 error popped out:
 [https://metrics.torproject.org/userstats-relay-
 country.png?start=2018-05-15=2018-08-13=all=off link].
 Is it related to this report or separate report is needed?

 Oops, fixed. It was indeed unrelated.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+-
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  closed
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:  fixed
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+-
Changes (by irl):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 This is merged and deployed.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+-
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  merge_ready
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+-

Comment (by irl):

 The above in comment:18 is unrelated.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+-
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  merge_ready
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+-

Comment (by Vort):

 **@karsten**, one more 500 error popped out:
 [https://metrics.torproject.org/userstats-relay-
 country.png?start=2018-05-15=2018-08-13=all=off link].
 Is it related to this report or separate report is needed?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+-
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  merge_ready
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+-

Comment (by karsten):

 Sounds like a good plan! Please find #27113 for the release part.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+-
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  merge_ready
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+-
Changes (by irl):

 * status:  needs_review => merge_ready


Comment:

 Ok. I've run this on a local copy of the Onionoo data directory. For the
 deployment, I'll document the steps I took during testing, so we can just
 duplicate these:

 * Upload the JAR file that includes the `--rewrite-only` command.
 * Upload the new release's JAR and WAR files.
 * Wait for hourly updater to end.
 * Stop hourly updater and the web server.
 * Run `java -jar onionoo-6.2-1.16.0-rewrite.jar --rewrite-only`.
 * Remove the intermediate JAR file to avoid confusion.
 * Re-start the web server and the hourly updater.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+--
Changes (by irl):

 * status:  assigned => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  karsten
 Type:  defect   | Status:  assigned
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+--
Changes (by irl):

 * owner:  irl => karsten
 * status:  needs_review => assigned


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+--

Comment (by irl):

 Ok. I want to run this on the real Onionoo data we have so that we're only
 fixing it once, so will need to wait for this to sync locally. I'm also
 going to try and do something clever with ZFS snapshots to make this kind
 of testing faster for me to get set up in the future (e.g. for #27050).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:  irl  |Sponsor:
-+--
Changes (by irl):

 * reviewer:   => irl


Comment:

 Will look at this today hopefully.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 And please review
 
[https://gitweb.torproject.org/user/karsten/onionoo.git/commit/?h=task-27039-2=3be9f20055b0b427e5d8647540dc540330dd4332
 commit 3be9f20 in my task-27039-2 branch] with the actual fix.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  needs_review
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by karsten):

 * status:  accepted => needs_review


Comment:

 Please review
 
[https://gitweb.torproject.org/user/karsten/onionoo.git/commit/?h=task-27039=18e5c04b9cfc9ca4b08ab7509a756d7cca814ac7
 commit 18e5c04 in my task-27039 branch], which is the small tool that
 rewrites broken history documents.

 I'll now work on the actual fix.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  accepted
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 I'll start with the small tool and will let you know here how that works.
 Feel free to start with the actual patch in the meantime, but if you're
 busy with other tickets, I'll do that after finishing the tool.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  accepted
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by irl):

 karsten - I've been looking at other Relay Search tickets and was going to
 loop back round to this one. The option with the small tool looks like the
 best path to take, do you want to take this instead as you'll probably be
 able to do it quicker than me?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  accepted
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--

Comment (by karsten):

 Yes, I think that's the issue. Looks like we did not include the
 `GraphHistory` class when we added Jackson-related annotations in 52494a7.

 The fix is probably as simple as copying those annotations from `Document`
 to `GraphHistory`.

 But how do we fix existing files in `out/` directories?
  - We could wait until new files are being written and accept that old
 files might be broken. Not very nice.
  - We could change the servlet to read all history files and rewrite them
 before including them in a response. Not very efficient.
  - We could write a small tool to run on an existing `out/` directory that
 reads in history files and overwrites them, if they are broken. Might be
 okay.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #27039 [Metrics/Onionoo]: Timestamps in graph history documents are incorrectly formatted (was: Relay Search graphs missing)

[Tor Bug Tracker & Wiki]

#27039: Timestamps in graph history documents are incorrectly formatted
-+--
 Reporter:  starlight|  Owner:  irl
 Type:  defect   | Status:  accepted
 Priority:  High |  Milestone:
Component:  Metrics/Onionoo  |Version:
 Severity:  Normal   | Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+--
Changes (by irl):

 * priority:  Medium => High
 * component:  Metrics/Relay Search => Metrics/Onionoo


Old description:

> maintenance or a problem?  No graphs at time of ticket.

New description:

 Timestamps in graph history documents are being served as ints (maybe unix
 timestamps) instead of -MM-DD hh:mm:ss format.

 This has the result of breaking Relay Search's graphs and also of breaking
 the t-shirt script.

--

Comment:

 I think it's possible that the getter is being used instead of the
 variable, which parses the String to return a long.

 
https://gitweb.torproject.org/onionoo.git/tree/src/main/java/org/torproject/onionoo/docs/GraphHistory.java#n16

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs