Re: How do i : Python Threads + KeyboardInterrupt exception
I tested this a bit more. My windows example was incorrect. It should have used CTRL_C_EVENT. But even then, the problem is that the process will also close the console window from which it was called because of the 0. Also this requires that the process actually have a console and is not a GUI application. Is there some other method rather than a blocking for line in fin: that i can use for reading from a file like device that plays nicely with other threads asynchronously waking it up? Sorry for all the posts. I am giving updates as i look further into the problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: How do i : Python Threads + KeyboardInterrupt exception
I tested this a bit more. My windows example was incorrect. It should have used CTRL_C_EVENT. But even then, the problem is that the process will also close the console window from which it was called because of the 0. Also this requires that the process actually have a console and is not a GUI application. Is there some other method rather than a blocking for line in fin: that i can use for reading from a file like device that plays nicely with other threads asynchronously waking it up? Sorry for all the posts. I am giving updates as i look further into the problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: How do i : Python Threads + KeyboardInterrupt exception
If only the main thread can receive KeyboardInterrupt, is there any reason why you couldn't move the functionality of the Read thread into the main thread? It looks like it's not doing any work, just waiting for the Proc thread to finish. You could start the Proc thread, do the current Read thread functionality until the interrupt occurs, put the apporpriate message in the queue, and then wait for the Proc thread to finish. It is already doing that. You will notice that the Proc() function is called by a threading.Thread instance so Proc() is running in a thread, but the Read() function is being called by the main thread right after this. It DOES work with the Ctrl + C, but i can find no way at all of closing down the script from within the Proc() thread. The relevant bit of code is: t = MyThread(Proc, queue, sys.stderr, None) Read(queue, sys.stdin, sys.stderr) In the end, the problem is that i am trying to multiplex IO and other operations. In UNIX i would use select with the input file descriptor and an anonymous pipe for the extra commands to achieve this without any threads, but this script needs to run primarily on a windows box and i would like to use it on UNIX too. I thought i could use threads to achieve the IO Multiplexing in python, but it seems not or at least not simply. How do people usually manage IO multiplexing (not just using sockets) cross platform in python? I only need to multiplex two sources really: * Input file or stdin * A input event queue This will have messages injected from various locations: timers, the processing thread itself, and possibly from a single GUI thread at a later point in time. Though i can foresee that at some point in the future i may also need to multiplex those two above and some sockets (For a server with a few clients). I was thinking of looking at some asynchronous IO libraries for python on Windows + UNIX, any suggestions (Must include more than just sockets)? -- http://mail.python.org/mailman/listinfo/python-list
How do i : Python Threads + KeyboardInterrupt exception
Hi all,
I have a small python project i am working on. Basically i always have
two threads. A Read thread that sits in a loop reading a line at a
time from some input (Usually stdin) and then generating events to be
processed and a Proc thread that processes incoming events from a
queue. There will be additional threads as well that asynchronously
insert events into the queue to be processed, but they are not a part
of this so i have omitted them.
What i want to know is: What is the standard/best way of implementing
such a pattern that works in the presence of errors and particularly
with the KeyboardInterrupt exception?
Some sample code is shown below. This code works as is, except in the
case where the Proc thread wants to initiate the exit of the
application.
For example:
* running the code below and pressing Ctrl + C works fine as the Read
thread is initiating the shutdown.
* running the code below and entering:
pquit
some other data
Ctrl + D
will cause oytput:
Processing: pquit
Proc: Initiating quit
and then it HANGS waiting for the Read thread to exit.
Some questions i have that are:
* KeyboardInterrupt exception seems to only be recieved by the main
thread. Is this ALWAYS the case across all UNIX + windows platforms
(not so worried about others)?
* Can i somehow get the Proc thread to force the Read thread to
generate a KeyboardInterrupt or somehow exit its blocking for line in
fin: call?
Thanks,
Brendon
--- SNIP ---
# Two or more threads
#
# proc : Is a processing thread that basically reads events from a
event queue and processes them
# read : Is a thread reading in a loop from stdin and generating
events for proc
# * : Other additional threads that may asynchronously add events to
the queue to be processed
import Queue
import threading
import sys
def Read(queue, fin, fout):
ret = (1, 'Normal Exit')
try:
for line in fin:
queue.put((0, line))
#raise Exception(Blah)
#raise Blah
except KeyboardInterrupt: ret = (1, 'KeyboardInterrupt')
except Exception, e: ret = (1, 'ERROR: ' + str(e))
except:ret = (1, 'UNKNOWN-ERROR')
# Notify Proc thread that we are exiting.
queue.put(ret)
print fout, 'Read: Initiating quit'
def Proc(queue, fout, ignore):
quit = False
while not quit:
(evt_type, evt_data) = queue.get()
if evt_type == 0:
print fout, 'Processing: ' + str(evt_data)
if evt_data.startswith('pquit'):
print fout, 'Proc: Initiating quit'
quit = True
elif evt_type == 1:
print fout, 'Quit: ' + str(evt_data)
quit = True
class MyThread(threading.Thread):
def __init__(self, func, queue, file1, file2, *args, **kwds):
threading.Thread.__init__(self, *args, **kwds)
self.func = func
self.queue = queue
self.file1 = file1
self.file2 = file2
self.start()
def run(self):
return self.func(self.queue, self.file1, self.file2)
if __name__ == '__main__':
queue = Queue.Queue()
# Read thread is the main thread and seems to get the
KeyboardInterrupt exception.
t = MyThread(Proc, queue, sys.stderr, None)
Read(queue, sys.stdin, sys.stderr)
# Read thread is NOT the main thread and never seems to get the
KeyboardInterrupt exception.
# This doesnt work for that reason.
#t = MyThread(Read, queue, sys.stdin, sys.stderr)
#Proc(queue, sys.stderr, None)
# @@@Brendon How do we notify the Read thread that they should
exit?
# If the Read thread initiated the quit then all is fine.
# If the Proc thread initiated the quit then i need to get the
Read
# thread to exit too somehow. But it is currently blocking in a
read
# on an input file.
print sys.stderr, 'Joining thread.'
t.join()
--
http://mail.python.org/mailman/listinfo/python-list
Re: How do i : Python Threads + KeyboardInterrupt exception
I don't know the standard way, but perhaps you can get some ideas from this recent thread:http://groups.google.com/group/comp.lang.python/browse_thread/thread/... I had a quick read through that thread. I think i will need some more time to think about what they are saying in there though. They seem to talking about killing a python thread kind of similar to the C functions TerminateThread() in windows or pthread_cancel() on UNIX which are not suitable for my purpose. You might try using the PyThreadState_SetAsyncExc function (from the Python C API) to inject a KeyboardInterrupt exception into the Read thread - but I don't know if it will work at all, the execution might be blocked waiting for an I/O call to complete, and never return to Python code... Unfortunately that is the problem. It is blocking in a IO system call and i want to force it to exit that with an error, hopefully causing a Python exception. I looked at what you mentioned and it is described a bit here too: http://sebulba.wikispaces.com/recipe+thread2 I really need a python mechanism for interrupting blocking system calls. have dealt with this sort of thing before in C/C++ on windows and UNIX. For UNIX it is really a matter of sending a signal to the process (SIGINTR), the main thread which is the only one in Python to accept signals (others as i understand are masked) will get the signal and and return with an EINTR breaking out of the blocking read hopefully a python exception of Interrupted IO or KeyboardInterrupt. Note that this only works for the one thread , but that is all i need. For windows, it is possible to do a similar thing using: GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, 0) with which behaves a bit like a UNIX signal and i assume is what causes the KeyboardInterrupt in the first place. The problem i have is how do I achieve this in python? -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting _node* to a Code object?
Gabriel Genellina wrote: En Sun, 01 Apr 2007 01:35:59 -0300, Brendon Costa [EMAIL PROTECTED] escribió: How do i convert a _node* object returned from: PyParser_SimpleParseStringFlagsFilename() into a code object i can use as a module to import with: PyImport_ExecCodeModule() Using PyNode_Compile. But why don't you use Py_CompileXXX instead? And look into import.c, maybe there is something handy. Thanks for the pointer. I am not using Py_CompileXXX because i could only find Py_CompileString... i could not find a file version of it (Which i thought should exist). My original email though i copied and pasted the wrong function into. Instead of: PyParser_SimpleParseStringFlagsFilename() i meant to use: PyParser_SimpleParseFileFlags() Basically i will open a FILE* for the file requested, parse it and load it into the module. Using this method i don't have to load its contents first into a string to be compiled, but just get the python library to parse directly from the file. It all seems to work fine now. Thanks for the help. Brendon. -- http://mail.python.org/mailman/listinfo/python-list
Converting _node* to a Code object?
Hi All, I have an application with an embedded python interpreter and i need to get the embedded interpreter to import strangely named files as python modules. Anyhow the simple part of the question is: How do i convert a _node* object returned from: PyParser_SimpleParseStringFlagsFilename() into a code object i can use as a module to import with: PyImport_ExecCodeModule() I cant seem to find any documentation about this. If you are wondering why i want to do this, then you can read on and maybe there is a much better way to achieve what i am after. Thanks, Brendon - What i am trying to achieve - Basically i want to import using the python C API a module from various files which may have names that do not make valid python module names. In particular i have an embedded python interpreter that is used to apply a number of operations on some data based on a user supplied suppressions file (*.eds). This .eds file is just a python source file designed only to be used from within this application. Now the names of these suppression files usually matche that of a library or application to which they will apply to. For example: libstdc++.so.6.0.5 : has eds file: libstdc++.so.6.0.5.eds Now i want to have my code import this eds file as a python module so that i can then use it as any other standard python module. So i need to separate the name of the module from the name of the file. I was thinking for example for the above filename i would give it a module name like: libstdc___so_6_0_5 (The name is not important just that the above one is a valid module name from what i understand and it is based on the original name) So i have some random file: /foo/bar/lib/libstdc++.so.6.0.5.eds and i want to be able to import that as a module using the Python C API so native python code can access the module like: libstdc___so_6_0_5.PostCalculation() If you have any better method of doing this, i would love to know what it is. Thanks in advance for any help, Brendon. -- http://mail.python.org/mailman/listinfo/python-list
