Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  (none)
 Type:  defect   | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:  Tor:
 |  unspecified
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328, |
  035-deferred-20180930, 040-deferred-201915 |
Parent ID:   | Points:
 Reviewer:  mikeperry|Sponsor:
-+-
Changes (by neel):

 * status:  assigned => needs_information


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  (none)
 Type:  defect   | Status:
 |  assigned
 Priority:  Medium   |  Milestone:  Tor:
 |  unspecified
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328, |
  035-deferred-20180930, 040-deferred-201915 |
Parent ID:   | Points:
 Reviewer:  mikeperry|Sponsor:
-+-
Changes (by neel):

 * owner:  neel => (none)
 * status:  needs_information => assigned


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  needs_information
 Priority:  Medium   |  Milestone:  Tor:
 |  0.3.5.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:  mikeperry|Sponsor:
-+-
Changes (by dgoulet):

 * status:  needs_review => needs_information


Comment:

 I 100% agree with nickm...

 A while back, when I was working on KIST with pastly, we realized that
 much of the cmux subsystem is quite complex and could be simplify
 drastically. I even did Dia diagram to try to make sense of it all and
 came up with a design to address the complexities.

 So there is an argument to be made to try to revive all my notes in a
 Ticket and address the overall issue instead of patching things here and
 there.

 We have no sponsor or roadmap slots for this afaict but if it is out
 there, great volunteers like neel can go at it :). I'll try to find an
 evening soon and sum up it all up in a ticket. In the meantime, I
 recommend we keep that ticket on hold until we decide if overhauling is
 what should be done or if we should merge this as a temporary band aid
 before bigger work.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.3.5.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:  mikeperry|Sponsor:
-+-

Comment (by nickm):

 I've left a review on the PR; Mike is planning to do so as well.

 In general, I think that the approach is a good start, but that a more
 thorough approach might be called for in order to keep the code clean.
 I'm not sure it's a good idea to proliferate these "hashent" usages
 throughout our codebase, I'd like to take a step back.

 Before we optimize too much we should ask ourselves, "Is this hashtable
 really necessary?"  After all, there are at most two of these hash entries
 for each circuit, and we are looking them up _from_ the circuit. Perhaps a
 better solution would be to give each circuit a couple of pointers to this
 information, to make lookup into an O(1) operation.  Maybe after we were
 done, we might find that the hashtable wasn't necessary at all.

 Of course, we would need to manage lifetime issues carefully here.

 I am not taking a position about whether the change I suggest above should
 be done ''instead of'' or ''after'' this change; I would like a second
 opinion from mike about that -- and ideally from dgoulet too, once he is
 back online.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.3.5.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:  mikeperry|Sponsor:
-+-
Changes (by asn):

 * reviewer:   => mikeperry


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  needs_review
 Priority:  Medium   |  Milestone:  Tor:
 |  0.3.5.x-final
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by nickm):

 * status:  assigned => needs_review
 * milestone:  Tor: unspecified => Tor: 0.3.5.x-final


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  assigned
 Priority:  Medium   |  Milestone:  Tor:
 |  unspecified
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by neel):

 I have a PR here: https://github.com/torproject/tor/pull/243

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  assigned
 Priority:  Medium   |  Milestone:  Tor:
 |  unspecified
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by neel):

 * cc: neel@… (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
-+-
 Reporter:  dgoulet  |  Owner:  neel
 Type:  defect   | Status:
 |  assigned
 Priority:  Medium   |  Milestone:  Tor:
 |  unspecified
Component:  Core Tor/Tor |Version:
 Severity:  Normal   | Resolution:
 Keywords:  performance, tor-relay,  |  Actual Points:
  034-triage-20180328, 034-removed-20180328  |
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by neel):

 * status:  new => assigned
 * owner:  (none) => neel


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #25152 [Core Tor/Tor]: Try to call less circuitmux_find_map_entry()

[Tor Bug Tracker & Wiki]

#25152: Try to call less circuitmux_find_map_entry()
--+-
 Reporter:  dgoulet   |  Owner:  (none)
 Type:  defect| Status:  new
 Priority:  Medium|  Milestone:  Tor: 0.3.4.x-final
Component:  Core Tor/Tor  |Version:
 Severity:  Normal|   Keywords:  performance, tor-relay,
Actual Points:|  Parent ID:
   Points:|   Reviewer:
  Sponsor:|
--+-
 The `circuitmux_find_map_entry()` function is currently taking more than
 3% of the CPU on a busy relay. It is literally the second highest after
 curv25519 crypto stuff (which is expected to be high).

 We can't really optimize that function so much but we can try to call it
 less! For instance, at every single cell we append to a circuit in
 `append_cell_to_circuit_queue()` we call `update_circuit_on_cmux()` which
 calls *4* functions in succession that calls
 `circuitmux_find_map_entry()`:

 * `circuitmux_is_circuit_attached()`
 * `circuitmux_attached_circuit_direction()`
 * `circuitmux_set_num_cells()` and this function can call it again with
 `circuitmux_make_circuit_inactive()` or
 `circuitmux_make_circuit_active()`.

 This is quite a lot of CPU at _each_ cell especially when we do have many
 many circuits.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs