#29374: Analysis files sometimes present negative numbers in the payload_progress field -----------------------------------+------------------------------ Reporter: irl | Owner: metrics-team Type: defect | Status: new Priority: Medium | Milestone: Component: Metrics/Onionperf | Version: Severity: Normal | Resolution: Keywords: acute-2019-q1-planned | Actual Points: Parent ID: | Points: Reviewer: | Sponsor: -----------------------------------+------------------------------
Comment (by acute): Replying to [comment:3 karsten]: > Some thoughts on this, since I have been looking at tgen logs for a while now: > > - The tgen log line you mention does not contain anything to identify the transfer. Would it be easier to use the next tgen log line which at least contains the source port? `2019-02-11 00:13:00 1549843980.555482 [info] [shd-tgen-transport.c:203] [_tgentransport_changeState] transport TCP,16,localhost:127.0.0.1:43810,localhost:127.0.0.1:41204,op- ab.onionperf.torproject.net:137.50.19.2:80,state=CONNECT,error=NONE moving from state CONNECT to state INIT`; and would it still be accurate enough? Have had another look at it, the OP parser only looks at lines containing [transfer-status], [transfer-complete] and [transfer-error]. It currently identifies transfers by the time and transfer number only, without including any ports. The problem is that the transfer number only starts appearing in lines onwards after 'moving from state COMMAND to state RESPONSE', and the parser gets all the other information for a transfer from the tgen process summary report... Not sure where to go from here...we could change the way we identify reports (I agree with you that we should use both ports + time) to get timestamp information from the previous lines, and we add rules for parsing these? This would mean a major change to the OP parser. > - What's the reason for overwriting this timestamp with the calculated value for ''complete'' transfers? This log line has usec granularity, too (1554084910'''.250433'''). I think it would be easier to compare partial progress of complete and incomplete transfers if we'd use the same timestamps for calculating them. My fear was that if the values reported by the tgen process are the absoulute source of truth, by changing them we'd report inaccurate information. I agree that calculating everything the same way is more consistent - we should go with this after we figure out what to do for the previous step. > - Do we know of any tools ''relying'' on `payload_progress` numbers being negative for incomplete downloads? I admit that this would be a weird use of the data. Still, maybe worth checking if we're using them anywhere ourselves and probably worth writing something in the next release notes. Excellent point, will have a look at the visualization module. > - You'll find more tgen logs for testing at https://op- ab.onionperf.torproject.net/ . Thanks for this! -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29374#comment:4> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs