Re: thread/queue bug
[EMAIL PROTECTED] Note that python-bugs-list is a moderated list for use only by automated reports generated from SourceForge. I'm redirecting the reply to [EMAIL PROTECTED] I have a very strange bug. A thread in a .pyc stops dead. This program has many threads and queues and has worked great for months. One thread listens for UDP messages from other programs, and puts the messages in listenq. Procmsgs gets from listenq and for a certain kind of message creates another mq = Queue(0). I don't wish to distribute the source to pnetx.py so I have a short program pnet.py import it, that is pnetx.pyc. Then the procmsgs thread dies at the Queue statement. But if I run pnetx.py as the main program, all works fine. So if pnetx.py or .pyc is imported this thread dies. ??? I've tried this with 2.3.3 and 2.3.4. Anyone? Sounds like your module spawns a thread as a side effect of being imported. Don't do that -- you'll get in deadlock trouble with the import lock if you do (use Google to learn more about that). -- http://mail.python.org/mailman/listinfo/python-list
thread/queue bug
I have a very strange bug. A thread in a .pyc stops dead. This program has many threads and queues and has worked great for months. One thread listens for UDP messages from other programs, and puts the messages in listenq. Procmsgs gets from listenq and for a certain kind of message creates another mq = Queue(0). I don't wish to distribute the source to pnetx.py so I have a short program pnet.py import it, that is pnetx.pyc. Then the procmsgs thread dies at the Queue statement. But if I run pnetx.py as the main program, all works fine. So if pnetx.py or .pyc is imported this thread dies. ??? I've tried this with 2.3.3 and 2.3.4. Anyone? ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: thread/queue bug
Op 2004-12-14, Steve Holden schreef [EMAIL PROTECTED]: Antoon Pardon wrote: Op 2004-12-13, Tim Peters schreef [EMAIL PROTECTED]: [Antoon Pardon] I don't see why starting a thread as a side effect of importing is bad thread practice. Sure python doesn't cater for it, but IMO that seems to be python failing. Obviously, it's bad practice in Python because it can lead to deadlocks in Python. By that argument any use of locks is bad practice because it can lead to deadlock. Not at all. You mentioned locks, not Tim. That is beside the point. The argument was that starting a thread as a side effect of importing is bad practice because it can lead to deadlock. This suggest that a general condition for something to be bad practice is if that something can lead to deadlock. Locks are in that case. It's nearly tautological. Import is an executable statement in Python, not, e.g., as in many other languages, a declaration directed at the system linker. With that power comes opportunities to shoot yourself, although they're generally easy to avoid. Come up with a practical design that doesn't have this limitation, and then perhaps your characterization of the current design as a failing would be both credible and constructive. If a car model has cranky brakes, I think I can call that a failing even without having the ability to come up with a pratical design that doesn's has those limitations. But in fact your situation is more closely analogous to a customer who's bought a car that can be stopped by pressing on the brake pedal now complaining that sideways pressure on the brake pedal doesn;t affect the car's motion. You will have to explain how you come by that analogy. I judge a language by what it can and cannot do, not by my ability to correct the things I perceive as failings. For all I know python may have taken some design decisions that might have seen perfectly logical but now prohibit a a practical design that doesn't have this limitation. I don't see why something like that would make this any less a failing then when a practical design was easy in the current implemenation. All that Tim was suggesting is that it's MORE SENSIBLE to start a thread as the result of a specific call to programmed functionality rather than as the side effect of an import. The reason for this is due to the semantics of the import statement. If you perceive that as a failing then you'd be better rewarded by an attempt to modify your perceptions. I've always found don't argue with Tim about Python to be a useful rule of thumb. He's wrong much less often than I am. I suspect he's also wrong much less often than you ;-) With all respect I find that lousy advise. I don't mind that it will turn out I'll be wrong most of the time. But I'll probably will have gained a better understanding by the responses to my argument than by merely accepting the word of Tim. [ ... ] Note that the OP's example had a module that, upon the first attempt to import it, ran an infinite loop (even if it hadn't deadlocked), and it's clearly severe abuse of import's purpose.to write a module M such that import M *never* returns. Indeed, that's the other half of how deadlock occurs: not only that the imported module spawn a thread as a side effect of importing, but also that the imported module refuse to allow the import to complete. Well I'll agree here. An import that has as a side effect that the import doesn't return is bad practice. But that's precisely the risk you run when starting up threads! Now you are confusing me. Is it a problem of an import that doesn't return of is it a case of a race condition where the import has to return in a timely fashion? -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
[Antoon Pardon] I don't see why starting a thread as a side effect of importing is bad thread practice. Sure python doesn't cater for it, but IMO that seems to be python failing. Obviously, it's bad practice in Python because it can lead to deadlocks in Python. It's nearly tautological. Import is an executable statement in Python, not, e.g., as in many other languages, a declaration directed at the system linker. With that power comes opportunities to shoot yourself, although they're generally easy to avoid. Come up with a practical design that doesn't have this limitation, and then perhaps your characterization of the current design as a failing would be both credible and constructive. Apart from that, ya, I do think it would *uisually* be poor practice to start a thread as a side effect of importing anyway. It's too mysterious, and IME has caused trouble even when it didn't lead to deadlocks. The fundamental purpose of import in Python is to add useful names to the importer's namespace, and users of a module generally think of it as doing no more than that. Note that the OP's example had a module that, upon the first attempt to import it, ran an infinite loop (even if it hadn't deadlocked), and it's clearly severe abuse of import's purpose.to write a module M such that import M *never* returns. Indeed, that's the other half of how deadlock occurs: not only that the imported module spawn a thread as a side effect of importing, but also that the imported module refuse to allow the import to complete. The current design actually supports spawning all the threads you like as a side effect of importing, provided you ensure also that the import ompletes. The easiest way to avoid trouble remains not to spawn threads as a side effect of importing to begin with, although a programmer determined to demonstrate their bad taste wink can easily enough make it work. -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
Op 2004-12-13, Tim Peters schreef [EMAIL PROTECTED]: [Antoon Pardon] I don't see why starting a thread as a side effect of importing is bad thread practice. Sure python doesn't cater for it, but IMO that seems to be python failing. Obviously, it's bad practice in Python because it can lead to deadlocks in Python. By that argument any use of locks is bad practice because it can lead to deadlock. It's nearly tautological. Import is an executable statement in Python, not, e.g., as in many other languages, a declaration directed at the system linker. With that power comes opportunities to shoot yourself, although they're generally easy to avoid. Come up with a practical design that doesn't have this limitation, and then perhaps your characterization of the current design as a failing would be both credible and constructive. If a car model has cranky brakes, I think I can call that a failing even without having the ability to come up with a pratical design that doesn's has those limitations. I judge a language by what it can and cannot do, not by my ability to correct the things I perceive as failings. For all I know python may have taken some design decisions that might have seen perfectly logical but now prohibit a a practical design that doesn't have this limitation. I don't see why something like that would make this any less a failing then when a practical design was easy in the current implemenation. Apart from that, ya, I do think it would *uisually* be poor practice to start a thread as a side effect of importing anyway. It's too mysterious, and IME has caused trouble even when it didn't lead to deadlocks. The fundamental purpose of import in Python is to add useful names to the importer's namespace, and users of a module generally think of it as doing no more than that. Importing a module in general also does some kind of initialisation. If starting a thread is a logical thing to do in this initialisation fase I see nothing wrong with it. Note that the OP's example had a module that, upon the first attempt to import it, ran an infinite loop (even if it hadn't deadlocked), and it's clearly severe abuse of import's purpose.to write a module M such that import M *never* returns. Indeed, that's the other half of how deadlock occurs: not only that the imported module spawn a thread as a side effect of importing, but also that the imported module refuse to allow the import to complete. Well I'll agree here. An import that has as a side effect that the import doesn't return is bad practice. The current design actually supports spawning all the threads you like as a side effect of importing, provided you ensure also that the import ompletes. Well in that case I don't have any problems with it. The perceived failing was because of only knowing part of it based on what I had read in this thread. The easiest way to avoid trouble remains not to spawn threads as a side effect of importing to begin with, although a programmer determined to demonstrate their bad taste wink can easily enough make it work. Well then probably I have a bad taste. I have though of designs in which it seemed very natural to have a thread started as part of the initialisation in a module. Other limitations of python didn't make it workable but in priciple I saw nothing wrong with doing it. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
phil wrote: [...] 5. Sorry I can't be more help. You don't give anyone much to go on. All that stuff about Queue(0) and listenq is pretty much meaningless to us, you know... You know, I get this all the time on language support groups. This might be a clue. All of my Linux support groups, if they don't understand, say why and ask for elaboration. I tried to explain what was going on, without incuding the source. I have about 200 man hours in the source and I bet it would take longer to understand it. If my explanation was insufficient, I'm sorry. It *was* pretty sketchy. Basically you shouldn't expect people to debug code they can't see. There are some very able programmers active on this newsgroup, but their psychic abilities are limited :-) ALSO, you did not respond to my email, so I didn't know how to reply. There is nothing on the archives page which gives me a clue as to how to respond. Once yo post in the newsgroup (or mailing list) then use of the same channel is advisable. SO, if there is ZERO chance there is some sort of inadvertent lock occuring in saved byte code, I just kludge around and move on. Maybe 2.4 If you suspect the problem is a bug in Python then you might consider searching the bugs database on sourceforge. If nobody's reported a bug then you are being very optimistic assuming that migrating to 2.4 will fix the problem. Thanks. regards Steve -- http://www.holdenweb.com http://pydish.holdenweb.com Holden Web LLC +1 800 494 3119 -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
phil wrote: Well its an anomaly. I sent to bug list. Probably never see it again. I think some sort of semaphore thingy, which I know nothing about, is sneaking in under unreproducible conditions. I'm moving on. If you want to try one more thing, try mucking with a call to sys.setcheckinterval(). This used to be 10 by default but is now 100. In any case, changing it will quite often significantly change the code's behaviour in the presence of race conditions. Try values of 1 and 1000 and see if anything makes a difference. -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
On Fri, 10 Dec 2004 16:18:51 -0600, phil [EMAIL PROTECTED] wrote:
And sorry I got ticked, frustrating week
threading problems can do that ;-)
You are obviusly deeper into this than I can get from a cursory scan,
but I'll make some general debugging comments ;-)
And I could help more, being fairly experienced with
threading issues and race conditions and such, but
as I tried to indicate in the first place, you've
provided next to no useful (IMHO) information to
let anyone help you more than this
This is about 5% of the code.
Uses no locks.
I am mystified, I have written probably 100,000 lines
of Python and never seen a thread just lock up and quit
running. It happens on a Queue() statement so my suspicion
is a bug. ??
Or a once-in-a-blue-moon resource access deadlock of some kind?
I have kludged around it by putting all the thread/queue stuff
in the main program and import the stuff I don't want to
distribute. But mysteries haunt your dreams, sooo...
#!/usr/local/bin/python
# pnetx.py
from threading import *
from time import *
from Queue import Queue
from socket import *
Do you really need to import * ? Though the above should be safe import-wise
you are setting yourself up for name-clash problems by putting so many in
the same space. E.g., if something happened to match a misspelling typo in
your program, you wouldn't get a name error exception. Etc., etc.
One of the first rules of debugging is to eliminate the unnecessary ;-)
At least there doesn't seem to be name clashes between the above (except
starting with '_' which shouldn't get imported with *
d = {}
for imp in 'threading time Queue socket'.split():
...m = __import__(imp)
...for name in m.__dict__.keys():
...d.setdefault(name, []).append(imp)
...
for k,v in d.items():
... if len(v)!=1: print k,v
...
_sleep ['threading', 'Queue']
__file__ ['threading', 'Queue', 'socket']
__all__ ['threading', 'Queue', 'socket']
__builtins__ ['threading', 'Queue', 'socket']
__name__ ['threading', 'time', 'Queue', 'socket']
_time ['threading', 'Queue']
__doc__ ['threading', 'time', 'Queue', 'socket']
Just verifying an assumption ;-)
OTOH, do you really need (ignoring wid and name and the first content of dir()):
dir()
['__builtins__', '__doc__', '__name__']
from threading import *
from time import *
from Queue import *
from socket import *
wid = 0
for name in dir():
... print repr(name),
... wid += len(repr(name))+1
... if wid60:
... print
... wid = 0
...
'AF_APPLETALK' 'AF_INET' 'AF_IPX' 'AF_UNSPEC' 'AI_ADDRCONFIG'
'AI_ALL' 'AI_CANONNAME' 'AI_DEFAULT' 'AI_MASK' 'AI_NUMERICHOST'
'AI_PASSIVE' 'AI_V4MAPPED' 'AI_V4MAPPED_CFG' 'BoundedSemaphore'
'CAPI' 'Condition' 'EAI_ADDRFAMILY' 'EAI_AGAIN' 'EAI_BADFLAGS'
'EAI_BADHINTS' 'EAI_FAIL' 'EAI_FAMILY' 'EAI_MAX' 'EAI_MEMORY'
'EAI_NODATA' 'EAI_NONAME' 'EAI_PROTOCOL' 'EAI_SERVICE' 'EAI_SOCKTYPE'
'EAI_SYSTEM' 'Empty' 'Event' 'Full' 'INADDR_ALLHOSTS_GROUP' 'INADDR_ANY'
'INADDR_BROADCAST' 'INADDR_LOOPBACK' 'INADDR_MAX_LOCAL_GROUP'
'INADDR_NONE' 'INADDR_UNSPEC_GROUP' 'IPPORT_RESERVED' 'IPPORT_USERRESERVED'
'IPPROTO_GGP' 'IPPROTO_ICMP' 'IPPROTO_IDP' 'IPPROTO_IGMP' 'IPPROTO_IP'
'IPPROTO_MAX' 'IPPROTO_ND' 'IPPROTO_PUP' 'IPPROTO_RAW' 'IPPROTO_TCP'
'IPPROTO_UDP' 'IP_ADD_MEMBERSHIP' 'IP_DEFAULT_MULTICAST_LOOP'
'IP_DEFAULT_MULTICAST_TTL' 'IP_DROP_MEMBERSHIP' 'IP_MAX_MEMBERSHIPS'
'IP_MULTICAST_IF' 'IP_MULTICAST_LOOP' 'IP_MULTICAST_TTL' 'IP_OPTIONS'
'IP_TOS' 'IP_TTL' 'Lock' 'MSG_DONTROUTE' 'MSG_OOB' 'MSG_PEEK'
'NI_DGRAM' 'NI_MAXHOST' 'NI_MAXSERV' 'NI_NAMEREQD' 'NI_NOFQDN'
'NI_NUMERICHOST' 'NI_NUMERICSERV' 'Queue' 'RAND_add' 'RAND_egd'
'RAND_status' 'RLock' 'SOCK_DGRAM' 'SOCK_RAW' 'SOCK_RDM' 'SOCK_SEQPACKET'
'SOCK_STREAM' 'SOL_IP' 'SOL_SOCKET' 'SOL_TCP' 'SOL_UDP' 'SOMAXCONN'
'SO_ACCEPTCONN' 'SO_BROADCAST' 'SO_DEBUG' 'SO_DONTROUTE' 'SO_ERROR'
'SO_KEEPALIVE' 'SO_LINGER' 'SO_OOBINLINE' 'SO_RCVBUF' 'SO_RCVLOWAT'
'SO_RCVTIMEO' 'SO_REUSEADDR' 'SO_SNDBUF' 'SO_SNDLOWAT' 'SO_SNDTIMEO'
'SO_TYPE' 'SO_USELOOPBACK' 'SSLType' 'SSL_ERROR_EOF'
'SSL_ERROR_INVALID_ERROR_CODE'
'SSL_ERROR_SSL' 'SSL_ERROR_SYSCALL' 'SSL_ERROR_WANT_CONNECT'
'SSL_ERROR_WANT_READ' 'SSL_ERROR_WANT_WRITE' 'SSL_ERROR_WANT_X509_LOOKUP'
'SSL_ERROR_ZERO_RETURN' 'Semaphore' 'SocketType' 'TCP_NODELAY'
'Thread' 'Timer' '__builtins__' '__doc__' '__name__' 'accept2dyear'
'activeCount' 'altzone' 'asctime' 'clock' 'ctime' 'currentThread'
'daylight' 'enumerate' 'error' 'errorTab' 'gaierror' 'getaddrinfo'
'getdefaulttimeout' 'getfqdn' 'gethostbyaddr' 'gethostbyname'
'gethostbyname_ex' 'gethostname' 'getnameinfo' 'getprotobyname'
'getservbyname' 'gmtime' 'has_ipv6' 'herror' 'htonl' 'htons'
'inet_aton' 'inet_ntoa' 'localtime' 'mktime' 'ntohl' 'ntohs'
'setdefaulttimeout' 'setprofile' 'settrace' 'sleep' 'socket'
'ssl' 'sslerror' 'strftime' 'strptime' 'struct_time' 'time' 'timeout'
'timezone' 'tzname' 'wid'
(Hm, should have pre-tested wid+len(current) to
Re: thread/queue bug
phil wrote: And sorry I got ticked, frustrating week And I could help more, being fairly experienced with threading issues and race conditions and such, but as I tried to indicate in the first place, you've provided next to no useful (IMHO) information to let anyone help you more than this This is about 5% of the code. Uses no locks. A lock is acquired in Queue.__init__(). I am mystified, I have written probably 100,000 lines of Python and never seen a thread just lock up and quit running. It happens on a Queue() statement so my suspicion is a bug. ?? I have kludged around it by putting all the thread/queue stuff in the main program and import the stuff I don't want to distribute. But mysteries haunt your dreams, sooo... What I believe to be a minimal example: freeze.py import Queue import threading import time q = Queue.Queue(4) def proc(): while True: q.get(1) Queue.Queue() print YADDA threading.Thread(target=proc).start() while True: print yadda q.put(None) time.sleep(1) /freeze.py freezemain.py import freeze /freezemain.py Invoking freezemain.py produces yadda yadda yadda yadda yadda yadda i. e. consistently q.maxsize + 2. One item about to be put, one already taken before Queue.Queue(). Deferring execution of the module-level code until after the import nofreeze.py import Queue import threading import time q = Queue.Queue(4) def proc(): while True: q.get(1) Queue.Queue() print YADDA def start(): threading.Thread(target=proc).start() while True: print yadda q.put(None) time.sleep(1) /nofreeze.py nofreezemain.py import nofreeze nofreeze.start() /nofreezemain.py and invoking nofreezemain.py produces yadda YADDA yadda YADDA yadda YADDA yadda YADDA apparently ad infinitum. Conclusion? None so far, though you might welcome the confirmation that this is not a once in a blue moon accident as Bengt Richter surmised. Import _is_ a sensitive phase... As an experiment I moved try: import thread except ImportError: import dummy_thread as thread from the Queue.Queue.__init__() method to the module body -- and now freezemain.py seems to work, too. So that would be an easy remedy, but sure there is a reason why that import statement is in such an unusual place? Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
[Peter Otten] What I believe to be a minimal example: freeze.py import Queue import threading import time q = Queue.Queue(4) def proc(): while True: q.get(1) Queue.Queue() print YADDA threading.Thread(target=proc).start() while True: print yadda q.put(None) time.sleep(1) /freeze.py freezemain.py import freeze /freezemain.py CPython has an internal, reentrant import lock. When a thread does an import, it acquires this lock, and the lock remains held until that import is complete. As a consequence, no *other* thread can do an import (it blocks waiting to obtain the internal import lock) until the original import completes. So until import freeze returns in the main thread, no other thread can do an import. Partly for that reason, it's generally a Horrible Idea to start a thread as a side effect of importing a module. That's what freeze.py does, and you get the expected deadlock as a result. The main thread is hung waiting for import freeze to return, and the spawned thread is hung at an import in Queue.__init__() waiting for the main thread to release the import lock. Invoking freezemain.py produces yadda yadda yadda yadda yadda yadda i. e. consistently q.maxsize + 2. One item about to be put, one already taken before Queue.Queue(). Deferring execution of the module-level code until after the import nofreeze.py import Queue import threading import time q = Queue.Queue(4) def proc(): while True: q.get(1) Queue.Queue() print YADDA def start(): threading.Thread(target=proc).start() while True: print yadda q.put(None) time.sleep(1) /nofreeze.py nofreezemain.py import nofreeze nofreeze.start() /nofreezemain.py and invoking nofreezemain.py produces yadda YADDA yadda YADDA yadda YADDA yadda YADDA apparently ad infinitum. Right. Note that this is the same reason threaded tests in Python's standard regression suite define a 'test_main()' function, called by the regrtest.py driver after import of the test's module completes. It's generally suicidal to start a thread as a side effect of an import. ... Import _is_ a sensitive phase... It's quite easy to avoid thread problems in imports: never start a thread as a side effect of importing, and you'll never get a deadlock due to importing. As an experiment I moved try: import thread except ImportError: import dummy_thread as thread from the Queue.Queue.__init__() method to the module body -- and now freezemain.py seems to work, too. So that would be an easy remedy, but sure there is a reason why that import statement is in such an unusual place? I think you'd have to ask Brett (who did most of the work on dummy_thread and dummy_threading). It doesn't really matter, though: it's a general truth that starting a thread as a side effect of importing is a recipe for deadlock, and hacking specific methods and functions to avoid imports just moves the problem around. It's not a goal that anything in the standard Python library cater to bad thread practice here (the bad thread practice being, again, starting a thread as a side effect of importing). -- http://mail.python.org/mailman/listinfo/python-list
thread/queue bug
I have a very strange bug. A thread in a .pyc stops dead. This program has many threads and queues and has worked great for months. One thread listens for UDP messages from other programs, and puts the messages in listenq. Procmsgs gets from listenq and for a certain kind of message creates another mq = Queue(0). I don't wish to distribute the source to pnetx.py so I have a short program pnet.py import it, that is pnetx.pyc. Then the procmsgs thread dies at the Queue statement. But if I run pnetx.py as the main program, all works fine. So if pnetx.py or .pyc is imported this thread dies. ??? I've tried this with 2.3.3 and 2.3.4. Anyone? -- http://mail.python.org/mailman/listinfo/python-list
thread/queue bug
4. The fact that you have a .pyc file instead of a .py file very likely has *nothing* to do with any threading problem you are facing, so I suggest you get past that mental block and look elsewhere. Well, I tried to make it clear that the ONLY difference between working and not working was the pnetx.pyc when imported did not work when #!/usr/bin/python # pnet.py, exists only to import pnetx.py import pnetx.py Otherwise the program has worked fine for months. I don't think its a mental block, its the ONLY difference. It freezes on the following statement: mq = Queue(0) and I haven't a clue why. All the other threads continue to run. 5. Sorry I can't be more help. You don't give anyone much to go on. All that stuff about Queue(0) and listenq is pretty much meaningless to us, you know... You know, I get this all the time on language support groups. All of my Linux support groups, if they don't understand, say why and ask for elaboration. I tried to explain what was going on, without incuding the source. I have about 200 man hours in the source and I bet it would take longer to understand it. If my explanation was insufficient, I'm sorry. ALSO, you did not respond to my email, so I didn't know how to reply. There is nothing on the archives page which gives me a clue as to how to respond. SO, if there is ZERO chance there is some sort of inadvertent lock occuring in saved byte code, I just kludge around and move on. Maybe 2.4 Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
phil wrote: You know, I get this all the time on language support groups. All of my Linux support groups, if they don't understand, say why and ask for elaboration. Wow, amazing! Imagine that... asking for elaboration when someone posts unclear confusing questions and extraneous information. The noive! I also find it remarkable that so many different people are all doing the same thing to you. It must be a conspiracy. Certainly not a problem with, say, you... -- http://mail.python.org/mailman/listinfo/python-list
thread/queue bug
Wow, amazing! Imagine that... asking for elaboration when someone posts unclear confusing questions and extraneous information. The noive! I would be happy to elaborate. No one asked to me to elaborate. I was simply told I didn't give enough information. I wasn't given an idea of what additional information was needed or some clue as to how to better phrase my question. I also find it remarkable that so many different people are all doing the same thing to you. It must be a conspiracy. Certainly not a problem with, say, you... I didn't say so many people are doing the same thing to me. Some language support groups make me feel like I am asking stupid questions in a stupid way, not all. And the Linux and FreeBSD support groups never do. In fact it seems on those groups the more newbie and inadequate your questions are the more patience and help you get. I've never before on any group seen anyone told they had a mental block, because they were fishing for info. I have a genuine problem here, which I have no clue how to approach, and I am very sorry I started a flame over protocol. THAT has certainly never happened to me before. signing off. -- http://mail.python.org/mailman/listinfo/python-list
thread/queue bug
And sorry I got ticked, frustrating week
And I could help more, being fairly experienced with
threading issues and race conditions and such, but
as I tried to indicate in the first place, you've
provided next to no useful (IMHO) information to
let anyone help you more than this
This is about 5% of the code.
Uses no locks.
I am mystified, I have written probably 100,000 lines
of Python and never seen a thread just lock up and quit
running. It happens on a Queue() statement so my suspicion
is a bug. ??
I have kludged around it by putting all the thread/queue stuff
in the main program and import the stuff I don't want to
distribute. But mysteries haunt your dreams, sooo...
#!/usr/local/bin/python
# pnetx.py
from threading import *
from time import *
from Queue import Queue
from socket import *
import sys
import os
# glob is a DUMMY CLASS
glob.listenq = Queue(1000)
def listener():
while 1:
msg,addrport = listenersocket.recvfrom(BUFSIZE)
addr = addrport[0]
glob.listenq.put (( msg,addr),1)
if msg == 'DIE': break
def procmsgs():
while 1:
msg,addr = glob.listenq.get(1)
if msg == 'DIE': break
wds = msg.split(';')
if wds[0] == 'S': recvsucc(msg); continue
if wds[0] == 'E': recvevent(msg); continue
if wds[0] == 'ACK': recvack(msg[4:]); continue
if wds[0] == 'MONITOR':
if addr not in monitorlist:
print 'This line ALWAYS PRINTS'
queuelist.append( Queue(0) )
## The above fails if this code is imported
## It doesn't matter whether it is imported
##as .py or .pyc
## also mq = Queue(0); queuelist.append(mq) # fails
print 'This line ALWAYS PRINTS if i execute this source'
print 'but NEVER PRINTS if I import '
print 'this source from a 1 line program'
print 'Thats weird'
monitoron.append( 1 )
monitorlist.append( addr )
queuelist = [Queue(0)]
listenersocket = socket(AF_INET,SOCK_DGRAM)
listenersocket.bind(ListenAddr)
procm = Thread(target=procmsgs)
procm.start()
listenr = Thread(target=listener)
listenr.start()
## Then start a bunch of other threads and queuea.
Don't spend a lot of time on this, not worth it.
I just thought someone might have experienced
something similar.
--
http://mail.python.org/mailman/listinfo/python-list
Re: thread/queue bug
phil wrote: Uses no locks. It does use locks implicitly, though, since even just importing threading will do that, and creating a Queue does too. I am mystified, I have written probably 100,000 lines of Python and never seen a thread just lock up and quit running. It happens on a Queue() statement so my suspicion is a bug. ?? You have the source to Queue.py in your standard library folder. Why not throw a few more print statements into its __init__ and see what you learn? I have kludged around it by putting all the thread/queue stuff in the main program and import the stuff I don't want to distribute. But mysteries haunt your dreams, sooo... #!/usr/local/bin/python [snip source] I cobbled together a working version of your code and ran it just fine, whether imported or run directly. No lockups. On Windows XP. Are you by any chance running on a new version of the Linux kernel, where the threading model has changed? (Or was it just RedHat 9.0?) I don't know the details, but I know folks have had trouble with this and Python... For example, I found this reference to the issue: http://groups.google.ca/groups?th=fe9a064ffeb38adcseekm=bpi438%24qta%2408%241%40news.t-online.com#link2 -Peter -- http://mail.python.org/mailman/listinfo/python-list
thread/queue bug
You have the source to Queue.py in your standard library folder. Why not throw a few more print statements into its __init__ and see what you learn? Yeah I put some print statements in init and it seems to complete. Are you by any chance running on a new version of the Linux kernel, where the threading model has changed? (Or was it just RedHat 9.0?) RedHat 7.2, 2.4.20 kernel. Get the same hangup on Win2000 Pro. Which leads me to believe it has nothing to do with kernel threads. Well its an anomaly. I sent to bug list. Probably never see it again. I think some sort of semaphore thingy, which I know nothing about, is sneaking in under unreproducible conditions. I'm moving on. I'm love python for multi-thread, simple socket programming and lots of reasons. This is the first thing that's looked like a bug in 5 years (1.5), I'm not worried. Thanks for your help. -- http://mail.python.org/mailman/listinfo/python-list
