Re: [tor-dev] Raising exceptions in add_event_listener() threads (was Re: HSv3 descriptor work in stem)
Damian Johnson writes: > Thanks George, this is a great question! I've expanded our tutorial to > hopefully cover this a bit better... > > https://stem.torproject.org/tutorials/tortoise_and_the_hare.html#advanced-listeners > Thanks both for this information! It was very useful! I basically followed the tutorial and it now works just fine! ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Raising exceptions in add_event_listener() threads (was Re: HSv3 descriptor work in stem)
Thanks George, this is a great question! I've expanded our tutorial to
hopefully cover this a bit better...
https://stem.torproject.org/tutorials/tortoise_and_the_hare.html#advanced-listeners
You can trivially print exceptions within your listener if that is all
you care about...
import time
import traceback
from stem.control import EventType, Controller
def broken_handler(event):
try:
raise ValueError('boom')
except:
print(traceback.format_exc())
with Controller.from_port() as controller:
controller.authenticate()
controller.add_event_listener(broken_handler, EventType.BW)
time.sleep(2)
% python demo.py
Traceback (most recent call last):
File "demo.py", line 9, in broken_handler
raise ValueError('boom')
ValueError: boom
Traceback (most recent call last):
File "demo.py", line 9, in broken_handler
raise ValueError('boom')
ValueError: boom
... but if your event handler has grown sophisticated enough to make
this a significant issue I'd suggest a producer/consumer model as the
tutorial now demonstrates.
Cheers! -Damian
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Raising exceptions in add_event_listener() threads (was Re: HSv3 descriptor work in stem)
Hi George,
> On 28 Nov 2019, at 02:04, George Kadianakis wrote:
>
> Hello Damian (and list),
>
> here is another question about an issue I have encountered while
> developing onionbalance v3.
>
> In particular, I'm fetching HS descriptors using HSFETCH and then adding
> an add_event_listener() event to a function that does the descriptor
> parsing and handling as follows:
>
>controller.add_event_listener(handle_new_desc_content_event,
> EventType.HS_DESC_CONTENT)
>
> The problem is that the handle_new_desc_content_event() callback has
> grown to a non-trivial size and complexity, since it needs to parse the
> descriptor, put it in the right places, and tick off the right
> checkboxes.
>
> Since its size has increased, so has the number of bugs and errors that
> are appearing during development. The problem is that because the
> callback is running on a separate thread (?) any errors and exceptions
> that get raised in that thread never surface to the my console and hence
> I don't see them. This means that I need to do very tedious printf
> debugging to find the exact place and type of error everytime something
> happens.
>
> What's the proper way to do debugging and development in callbacks like
> that? Is there a way to make the exceptions float back to the main
> thread or something? Or any other tips?
In general, there are two ways to avoid exceptions disappearing in python
threads, a code change, and a design change.
Code Change
The code change catches all exceptions in a thread, you should be able to
do it right now. (Or Damian could implement it in stem, every time a thread
is launched.)
1. Wrap all code called in a thread in a try/catch block
2. Catch every exception
3. Log it
4. Stop the thread/process/...
The code looks a bit like this:
try:
run_thread()
except e:
print("Error in thread")
log_error()
# Pick one of these actions
# Terminate the process immediately
os._exit(1)
# Re-raise the error - does this terminate the thread?
raise
# Stop the thread
return 1
You'll want a log_error() function like this one:
https://github.com/privcount/privcount/blob/master/privcount/log.py#L19
Another alternative is to join() threads when they terminate, and re-raise
the exception in the main thread. But the code is a lot more complex:
https://stackoverflow.com/a/6874161
Design Change
The design change moves all the work to the main thread, and just stores data/
sets a flag from other threads. That's not always possible, but it's usually a
good idea to limit work in threads, because it avoids races, delays, and
deadlocks.
In this case, you could store the data in the fetch thread, and then do the
processing the next time the main thread tries to access that data, or anything
related to it.
T
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Raising exceptions in add_event_listener() threads (was Re: HSv3 descriptor work in stem)
Hello Damian (and list), here is another question about an issue I have encountered while developing onionbalance v3. In particular, I'm fetching HS descriptors using HSFETCH and then adding an add_event_listener() event to a function that does the descriptor parsing and handling as follows: controller.add_event_listener(handle_new_desc_content_event, EventType.HS_DESC_CONTENT) The problem is that the handle_new_desc_content_event() callback has grown to a non-trivial size and complexity, since it needs to parse the descriptor, put it in the right places, and tick off the right checkboxes. Since its size has increased, so has the number of bugs and errors that are appearing during development. The problem is that because the callback is running on a separate thread (?) any errors and exceptions that get raised in that thread never surface to the my console and hence I don't see them. This means that I need to do very tedious printf debugging to find the exact place and type of error everytime something happens. What's the proper way to do debugging and development in callbacks like that? Is there a way to make the exceptions float back to the main thread or something? Or any other tips? Thanks! :) ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
