Re: Twisted for non-networking applications
Kottiyath wrote: Is it a good idea to use Twisted inside my application, even though it has no networking part in it? Basically, my application needs lots of parallel processing - but I am rather averse to using threads - With or without threads, the Python interpreter does not do parallel processing. You could use multiple processes, or a thread could call an extension module that releases Python's global interpreter lock, but Python itself does not offer parallel processing. due to myraid issues it can cause. So, I was hoping to use a reactor pattern to avoid the threads. The reactor pattern describes event-driven I/0, not parallel processing. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
Twisted for non-networking applications
Hi all, Is it a good idea to use Twisted inside my application, even though it has no networking part in it? Basically, my application needs lots of parallel processing - but I am rather averse to using threads - due to myraid issues it can cause. So, I was hoping to use a reactor pattern to avoid the threads. I am using twisted in another part of the application for networking, so I was hoping to use the same for the non-networking part for reusing the reactor pattern. If somebody can help me on this, it would be very helpful. Regards K -- http://mail.python.org/mailman/listinfo/python-list
Re: Twisted for non-networking applications
On Mon, Dec 22, 2008 at 4:27 AM, Kottiyath n.kottiy...@gmail.com wrote: Hi all, Is it a good idea to use Twisted inside my application, even though it has no networking part in it? Basically, my application needs lots of parallel processing - but I am rather averse to using threads - due to myraid issues it can cause. So, I was hoping to use a reactor pattern to avoid the threads. I am using twisted in another part of the application for networking, so I was hoping to use the same for the non-networking part for reusing the reactor pattern. If somebody can help me on this, it would be very helpful. Alternatively you could give circuits (1) a go. It _can_ be a nice alternative to Twisted and isn't necessarily focused on Networking applications. cheers James 1. http://trac.softcircuit.com.au/circuits/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Twisted for non-networking applications
On Dec 22, 3:26 am, James Mills prolo...@shortcircuit.net.au wrote: On Mon, Dec 22, 2008 at 4:27 AM, Kottiyath n.kottiy...@gmail.com wrote: Hi all, Is it a good idea to use Twisted inside my application, even though it has no networking part in it? Basically, my application needs lots of parallel processing - but I am rather averse to using threads - due to myraid issues it can cause. So, I was hoping to use a reactor pattern to avoid the threads. I am using twisted in another part of the application for networking, so I was hoping to use the same for the non-networking part for reusing the reactor pattern. If somebody can help me on this, it would be very helpful. Alternatively you could give circuits (1) a go. It _can_ be a nice alternative to Twisted and isn't necessarily focused on Networking applications. cheers James 1.http://trac.softcircuit.com.au/circuits/ I was unable to see documentation explaining this - so asking again. Suppose the event handlers in the component is doing blocking work, how is it handled? I went through ciruits.core, but was unable to understand exactly how blocking mechanisms are handled. My scenario is as follows: I have 4 loops, 1 small and high priority, 3 quite large and blocking (takes upto 3 seconds) and comparatively low priority. The small loops goes through everytime and does some work - and optionally uses the data sent by the other 3 loops. I do not want the smaller loop to get blocked by the other loops. So, if the event handler does blocking work, can that cause the whole loop to block? -- http://mail.python.org/mailman/listinfo/python-list
Re: Twisted for non-networking applications
On Mon, Dec 22, 2008 at 3:25 PM, RajNewbie raj.indian...@gmail.com wrote: I was unable to see documentation explaining this - so asking again. Documentation is available here: http://trac.softcircuit.com.au/circuits/wiki/docs And here: pydoc circuits The code itself is heavily documented. I'm still writing better online references, tutorials and what not ... :) Please see the examples/ Suppose the event handlers in the component is doing blocking work, how is it handled? You have a few options. 1. Do your work in a thread. 2. Do your work in a process. You could utilize the Worker component for such things. I'm also building a Process component that uses the multiprocessing module. I went through ciruits.core, but was unable to understand exactly how blocking mechanisms are handled. They aren't. It's up to you to handle such situations. If your event-handler blocks, everything, it will block every other event handler from being processes. In what situation would you have this ? I'm curious :) My scenario is as follows: I have 4 loops, 1 small and high priority, 3 quite large and blocking (takes upto 3 seconds) and comparatively low priority. The small loops goes through everytime and does some work - and optionally uses the data sent by the other 3 loops. I do not want the smaller loop to get blocked by the other loops. This sounds complex :) What is your application ? What's being processes ? So, if the event handler does blocking work, can that cause the whole loop to block? Yes. If you decide to use circuits, I suggest you restructure your program. Try to do your work (if it's blocking) in Worker components (threads). cheers James -- http://mail.python.org/mailman/listinfo/python-list
