Re: Interesting Thread Gotcha
"Dan" wrote: > Would it be possible to have pychecker (or some such) warn that there > is an insufficient parameter count to start_new_thread? I guess that > would require knowing the type of thread. . . I think this is the hub of the thing - its not only start_new_thread, but the way that parameters are evaluated before being counted, generally. See my reply to Diez's post - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Hendrik van Rooyen wrote: > "Duncan Booth" wrote: > >> Given that the start_new_thread function never actually got called, what >> code exactly do you expect to complain about the absence of a tuple? > > I don't understand this assertion. > > I thought that start_new_thread was called with a missing comma in > its argument list, which had the effect that I am complaining about. > > Putting the comma in place solved the problem, without any other > changes, so why do you say that start_new_thread was not called? Well, when kbd_driver() is called the kbd_q queue is probably empty, and as kbd_driver() runs in the main thread, who could ever put something into that queue? The function will therefore never terminate. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
"Diez B. Roggisch" wrote: > Hendrik van Rooyen wrote: > > It would have been nice, however, to have gotten something like: > > > > TypeError - This routine needs a tuple. > > > > instead of the silent in line calling of the routine in question, > > while failing actually to start a new thread. > > You can't prevent the silent inline-calling - otherwise, how would you do > this: > > def compute_thread_target(): > def target(): > pass > return target > > thread.start_new_thread(compute_thread_target()) > > > Of course start_new_thread could throw an error if it got nothing callable > as first argument. No idea why it doesn't. Thanks - got it, I think. Doesn't mean I like it, though: >>> a = 42 >>> b = 24 >>> def do_something(c,d): print c print d >>> do_something(a,b) 42 24 >>> def harmless(): return a >>> def evil(): while True: pass >>> do_something(a) Traceback (most recent call last): File "", line 1, in ? do_something(a) TypeError: do_something() takes exactly 2 arguments (1 given) >>> do_something(harmless()) Traceback (most recent call last): File "", line 1, in ? do_something(harmless()) TypeError: do_something() takes exactly 2 arguments (1 given) >>>do_something(evil()) This hangs and needs OS intervention to kill it - and there is also just one argument, not two. Looks like the arguments are handled one by one without validation till the end. Lets see: >>> do_something(a,b,harmless()) Traceback (most recent call last): File "", line 1, in ? do_something(a,b,harmless()) TypeError: do_something() takes exactly 2 arguments (3 given) So far, so good. >>>do_something(a,b,evil()) This also hangs - the third, extra argument is actually called! Are you all sure this is not a buglet? - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Hendrik van Rooyen wrote: > "Bjoern Schliessmann" wrote: > > Hendrik van Rooyen wrote: >> Absolutely! - well spotted! > > This is no threading problem at all; not even a syntax problem. If > you don't know exactly what start_new_thread and kbd_driver > functions do it's impossible to tell if your code does what is > intended. > >> It would have been nice, however, to have gotten something like: >> >> TypeError - This routine needs a tuple. >> >> instead of the silent in line calling of the routine in question, >> while failing actually to start a new thread. > > Exactly which part of the code should give you this warning? > > I am obviously missing something. > > My understanding is that, to start a new thread, one does: > > NewThreadID = thread.start_new_thread(NameOfRoutineToStart, > (ArgumentToCall_it_with,secondArg,Etc)) > > This calls start_new_thread with the name and the arguments to pass. > > If one omits the comma, then start_new_thread is surely stilled called, > but with an argument that is now a call to the routine in question, which > somehow causes the problem. > > So start_new_thread is the code that that is executed, with a bad set of > arguments - one thing, (a call to a routine) instead of two things - > a routine and a tuple of arguments. > > Everywhere else in Python if you give a routine the incorrect number of > arguments, you get an exception. Why not here? Python always evaluates the function's arguments first. The check for the correct number of arguments is part of the call and therefore done afterwards: >>> def f(x): print x ... >>> f(f(1), f(2), f(3)) 1 2 3 Traceback (most recent call last): File "", line 1, in TypeError: f() takes exactly 1 argument (3 given) So if one of the arguments takes forever to calculate you will never see the TypeError: >>> def g(x): ... print x ... import time ... while 1: time.sleep(1) ... >>> f(f(1), g(2), f(3)) 1 2 Traceback (most recent call last): File "", line 1, in File "", line 4, in g KeyboardInterrupt # I hit Ctrl-C Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
"Duncan Booth" wrote: > Given that the start_new_thread function never actually got called, what > code exactly do you expect to complain about the absence of a tuple? I don't understand this assertion. I thought that start_new_thread was called with a missing comma in its argument list, which had the effect that I am complaining about. Putting the comma in place solved the problem, without any other changes, so why do you say that start_new_thread was not called? - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
"Bjoern Schliessmann" wrote: Hendrik van Rooyen wrote: > Absolutely! - well spotted! This is no threading problem at all; not even a syntax problem. If you don't know exactly what start_new_thread and kbd_driver functions do it's impossible to tell if your code does what is intended. > It would have been nice, however, to have gotten something like: > > TypeError - This routine needs a tuple. > > instead of the silent in line calling of the routine in question, > while failing actually to start a new thread. Exactly which part of the code should give you this warning? I am obviously missing something. My understanding is that, to start a new thread, one does: NewThreadID = thread.start_new_thread(NameOfRoutineToStart, (ArgumentToCall_it_with,secondArg,Etc)) This calls start_new_thread with the name and the arguments to pass. If one omits the comma, then start_new_thread is surely stilled called, but with an argument that is now a call to the routine in question, which somehow causes the problem. So start_new_thread is the code that that is executed, with a bad set of arguments - one thing, (a call to a routine) instead of two things - a routine and a tuple of arguments. Everywhere else in Python if you give a routine the incorrect number of arguments, you get an exception. Why not here? - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Diez B. Roggisch <[EMAIL PROTECTED]> wrote: >Of course start_new_thread could throw an error if it got nothing callable >as first argument. No idea why it doesn't. It does: >>> thread.start_new_thread(None, None) Traceback (most recent call last): File "", line 1, in ? TypeError: first arg must be callable -- \S -- [EMAIL PROTECTED] -- http://www.chaos.org.uk/~sion/ "Frankly I have no feelings towards penguins one way or the other" -- Arthur C. Clarke her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Dan schrieb: > On Jan 16, 1:33 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: >> Dan schrieb: >> >> >> >>> On Jan 16, 11:06 am, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: Hendrik van Rooyen wrote: > "Dan" wrote: > keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) >> Needs to be > keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) >> Commas are important! >> -Dan > Absolutely! - well spotted! > As the first correct respondent, you win the freedom to spend a week in > Naboomspruit at your own expense. > It would have been nice, however, to have gotten something like: > TypeError - This routine needs a tuple. > instead of the silent in line calling of the routine in question, > while failing actually to start a new thread. You can't prevent the silent inline-calling - otherwise, how would you do this: def compute_thread_target(): def target(): pass return target thread.start_new_thread(compute_thread_target()) Of course start_new_thread could throw an error if it got nothing callable as first argument. No idea why it doesn't. Diez >>> Of course, in his case, having start_new_thread throw an error >>> wouldn't have helped, since he went into an infinite loop while >>> evaluating the parameters for start_new_thread. >>> Would it be possible to have pychecker (or some such) warn that there >>> is an insufficient parameter count to start_new_thread? I guess that >>> would require knowing the type of thread. . . >> What has this to do with the second argument? It's perfectly legal to >> have a function as thread-target that takes no arguments at all, so >> enforcing a second argument wouldn't be helpful - all it would do is to >> force all developers that don't need an argument tuple to pass the empty >> tuple. So there was no insufficient argument count. >> >> And none of these would solve the underlying problem that in python >> expressions are evaluated eagerly. Changing that would mean that you end >> up with a totally new language. >> >> the only thing that could help to a certain extend would be static >> types. Which we don't want here :) >> >> Diez > > It doesn't seem to be legal in my version of python (or the doc): > import thread def bat(): > print "hello" > > thread.start_new_thread(bat) > > Traceback (most recent call last): > File "", line 1, in > thread.start_new_thread(bat) > TypeError: start_new_thread expected at least 2 arguments, got 1 thread.start_new_thread(bat, ()) > 2256hello Ah, I thought it was optional, as in the threading.Thread(target=..., args=)-version. Sorry for not looking that up. Then you'd might stand a chance that pychecker can find such a situation - but of course not on a general level, as in the above - that would only work with type-annotations. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
On Jan 16, 1:33 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > Dan schrieb: > > > > > On Jan 16, 11:06 am, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > >> Hendrik van Rooyen wrote: > >>> "Dan" wrote: > >>> keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) > Needs to be > >>> keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) > Commas are important! > -Dan > >>> Absolutely! - well spotted! > >>> As the first correct respondent, you win the freedom to spend a week in > >>> Naboomspruit at your own expense. > >>> It would have been nice, however, to have gotten something like: > >>> TypeError - This routine needs a tuple. > >>> instead of the silent in line calling of the routine in question, > >>> while failing actually to start a new thread. > >> You can't prevent the silent inline-calling - otherwise, how would you do > >> this: > > >> def compute_thread_target(): > >> def target(): > >> pass > >> return target > > >> thread.start_new_thread(compute_thread_target()) > > >> Of course start_new_thread could throw an error if it got nothing callable > >> as first argument. No idea why it doesn't. > > >> Diez > > > Of course, in his case, having start_new_thread throw an error > > wouldn't have helped, since he went into an infinite loop while > > evaluating the parameters for start_new_thread. > > > Would it be possible to have pychecker (or some such) warn that there > > is an insufficient parameter count to start_new_thread? I guess that > > would require knowing the type of thread. . . > > What has this to do with the second argument? It's perfectly legal to > have a function as thread-target that takes no arguments at all, so > enforcing a second argument wouldn't be helpful - all it would do is to > force all developers that don't need an argument tuple to pass the empty > tuple. So there was no insufficient argument count. > > And none of these would solve the underlying problem that in python > expressions are evaluated eagerly. Changing that would mean that you end > up with a totally new language. > > the only thing that could help to a certain extend would be static > types. Which we don't want here :) > > Diez It doesn't seem to be legal in my version of python (or the doc): >>> import thread >>> def bat(): print "hello" >>> thread.start_new_thread(bat) Traceback (most recent call last): File "", line 1, in thread.start_new_thread(bat) TypeError: start_new_thread expected at least 2 arguments, got 1 >>> thread.start_new_thread(bat, ()) 2256hello >>> -Dan -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Dan schrieb: > On Jan 16, 11:06 am, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: >> Hendrik van Rooyen wrote: >>> "Dan" wrote: >>> keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) Needs to be >>> keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) Commas are important! -Dan >>> Absolutely! - well spotted! >>> As the first correct respondent, you win the freedom to spend a week in >>> Naboomspruit at your own expense. >>> It would have been nice, however, to have gotten something like: >>> TypeError - This routine needs a tuple. >>> instead of the silent in line calling of the routine in question, >>> while failing actually to start a new thread. >> You can't prevent the silent inline-calling - otherwise, how would you do >> this: >> >> def compute_thread_target(): >> def target(): >> pass >> return target >> >> thread.start_new_thread(compute_thread_target()) >> >> Of course start_new_thread could throw an error if it got nothing callable >> as first argument. No idea why it doesn't. >> >> Diez > > Of course, in his case, having start_new_thread throw an error > wouldn't have helped, since he went into an infinite loop while > evaluating the parameters for start_new_thread. > > Would it be possible to have pychecker (or some such) warn that there > is an insufficient parameter count to start_new_thread? I guess that > would require knowing the type of thread. . . What has this to do with the second argument? It's perfectly legal to have a function as thread-target that takes no arguments at all, so enforcing a second argument wouldn't be helpful - all it would do is to force all developers that don't need an argument tuple to pass the empty tuple. So there was no insufficient argument count. And none of these would solve the underlying problem that in python expressions are evaluated eagerly. Changing that would mean that you end up with a totally new language. the only thing that could help to a certain extend would be static types. Which we don't want here :) Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
On Jan 16, 11:06 am, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > Hendrik van Rooyen wrote: > > "Dan" wrote: > > >> >>> keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) > > >> Needs to be > >> >>> keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) > > >> Commas are important! > > >> -Dan > > > Absolutely! - well spotted! > > > As the first correct respondent, you win the freedom to spend a week in > > Naboomspruit at your own expense. > > > It would have been nice, however, to have gotten something like: > > > TypeError - This routine needs a tuple. > > > instead of the silent in line calling of the routine in question, > > while failing actually to start a new thread. > > You can't prevent the silent inline-calling - otherwise, how would you do > this: > > def compute_thread_target(): > def target(): > pass > return target > > thread.start_new_thread(compute_thread_target()) > > Of course start_new_thread could throw an error if it got nothing callable > as first argument. No idea why it doesn't. > > Diez Of course, in his case, having start_new_thread throw an error wouldn't have helped, since he went into an infinite loop while evaluating the parameters for start_new_thread. Would it be possible to have pychecker (or some such) warn that there is an insufficient parameter count to start_new_thread? I guess that would require knowing the type of thread. . . -Dan -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Hendrik van Rooyen wrote: > "Dan" wrote: > > >> >>> keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) >> >> Needs to be >> >>> keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) >> >> Commas are important! >> >> -Dan > > Absolutely! - well spotted! > > As the first correct respondent, you win the freedom to spend a week in > Naboomspruit at your own expense. > > It would have been nice, however, to have gotten something like: > > TypeError - This routine needs a tuple. > > instead of the silent in line calling of the routine in question, > while failing actually to start a new thread. You can't prevent the silent inline-calling - otherwise, how would you do this: def compute_thread_target(): def target(): pass return target thread.start_new_thread(compute_thread_target()) Of course start_new_thread could throw an error if it got nothing callable as first argument. No idea why it doesn't. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
Hendrik van Rooyen wrote: > Absolutely! - well spotted! This is no threading problem at all; not even a syntax problem. If you don't know exactly what start_new_thread and kbd_driver functions do it's impossible to tell if your code does what is intended. > It would have been nice, however, to have gotten something like: > > TypeError - This routine needs a tuple. > > instead of the silent in line calling of the routine in question, > while failing actually to start a new thread. Exactly which part of the code should give you this warning? > Is it worth the trouble of learning how to submit a bug report? For your problem not, IMHO, as a bug report for it will be closed quickly. Regards, Björn -- BOFH excuse #330: quantum decoherence -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
"Hendrik van Rooyen" <[EMAIL PROTECTED]> wrote: > It would have been nice, however, to have gotten something like: > > TypeError - This routine needs a tuple. > > instead of the silent in line calling of the routine in question, > while failing actually to start a new thread. Given that the start_new_thread function never actually got called, what code exactly do you expect to complain about the absence of a tuple? > > It seems to act no different from plain old: > > kbd_driver (port_q,kbd_q) > > Is it worth the trouble of learning how to submit a bug report? On your own code? There doesn't appear to be a bug in anyone else's code here. -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
"Dan" wrote: > >>> keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) > > Needs to be > >>> keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) > > Commas are important! > > -Dan Absolutely! - well spotted! As the first correct respondent, you win the freedom to spend a week in Naboomspruit at your own expense. It would have been nice, however, to have gotten something like: TypeError - This routine needs a tuple. instead of the silent in line calling of the routine in question, while failing actually to start a new thread. It seems to act no different from plain old: kbd_driver (port_q,kbd_q) Is it worth the trouble of learning how to submit a bug report? - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Interesting Thread Gotcha
On Jan 15, 10:07 am, "Hendrik van Rooyen" <[EMAIL PROTECTED]> wrote: > I thought I would share this nasty little gotcha with the group. > > Consider the following code fragment: > > > print 'starting kbd thread' > keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) > print 'starting main loop' > error = Mainloop(s,port_q,active_q_list) > > > It produces, as output, the following: > > starting kbd thread > we get here - a > > It does not print 'starting main loop', the Mainloop routine > is never executed, and no exceptions are raised. > > Here is the offending routine that seems to capture the control: > > > def kbd_driver(out_q,in_q): > """ > thread to look for keyboard input and to put it on the queue out_q > also looks for replies on in_q and prints them > """ > > kbdname = '/dev/stdin' > > kbd = open(kbdname,'r+',1) # Reading, line buffered > > unblock(kbd) # Call the magic to unblock keyboard > print 'we get here - a' > while True: > > try: > d = kbd.readline() # see if any kbd input > except: > IOError > try: > msg=in_q.get(block=False) > except Queue.Empty: > time.sleep(0.1) > continue > print msg > time.sleep(0.1) > continue > d = d.rstrip()# get rid of line feed > out_q.put([d + '\r',in_q]) # add a carriage return and return q and > send > to port > > > The unblock is a routine that unblocks a port using fcntl - it > is not the problem. In case you don't believe me, here it is: > > def unblock(f): > """Given file 'f', sets its unblock flag to true.""" > > fcntl.fcntl(f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) > > I will post the solution tomorrow when I read my mail, > if no one has spotted it by then. > > - Hendrik >>> keyboard_thread = thread.start_new_thread(kbd_driver (port_q,kbd_q)) Needs to be >>> keyboard_thread = thread.start_new_thread(kbd_driver, (port_q,kbd_q)) Commas are important! -Dan -- http://mail.python.org/mailman/listinfo/python-list
