Re: Traceback when using multiprocessing, less than helpful?

2013-11-22 Thread Mark Lawrence
On 22/11/2013 03:57, John Ladasky wrote: ...Richard submits his "hack" (his description) to Python 3.4 which pickles and passes the string. When time permits, I'll try it out. Or maybe I'll wait, since Python 3.4.0 is still in alpha. FTR beta 1 is due this Saturday 24/11/2013. -- Python

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread Chris Angelico
On Fri, Nov 22, 2013 at 2:57 PM, John Ladasky wrote: > or, for that matter, why data needs to be pickled to pass it between > processes. Oh, that part's easy. Let's leave the multiprocessing module out of it for the moment; imagine you spin up two completely separate instances of Python. Create

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread John Ladasky
On Thursday, November 21, 2013 2:32:08 PM UTC-8, Ethan Furman wrote: > Check out bugs.python.org. Search for multiprocessing and tracebacks to see > if anything is already there; if not, create a new issue. And on Thursday, November 21, 2013 2:37:13 PM UTC-8, Terry Reedy wrote: > 1. Use 3.3.3

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread Terry Reedy
On 11/21/2013 12:01 PM, John Ladasky wrote: This is a case where you need to dig into the code (or maybe docs) a bit File ".../evaluate.py", line 81, in evaluate > result = pool.map(evaluate, bundles) File "/usr/lib/python3.3/multiprocessing/pool.py", line 228, in map > return self._map_

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread Ethan Furman
On 11/21/2013 01:49 PM, John Ladasky wrote: So now, for anyone who is still reading this: is it your opinion that the traceback that I obtained through multiprocessing.pool._map_async().get() SHOULD have allowed me to see what the ultimate cause of the exception was? It would certainly be ni

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread John Ladasky
Followup: I didn't need to go as far as Chris Angelico's second suggestion. I haven't looked at certain parts of my own code for a while, but it turns out that I wrote it REASONABLY logically... My evaluate() calls another function through pool.map_async() -- _evaluate(), which actually proce

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread John Ladasky
On Thursday, November 21, 2013 12:53:07 PM UTC-8, Chris Angelico wrote: > What you could try is Suggestion 1: > printing out the __cause__ and __context__ of > the exception, to see if there's anything useful in them; Suggestion 2: > if there's > nothing, the next thing to try would be some

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread Chris Angelico
On Fri, Nov 22, 2013 at 5:25 AM, John Ladasky wrote: > On Thursday, November 21, 2013 9:24:33 AM UTC-8, Chris Angelico wrote: > >> Hmm. This looks like a possible need for the 'raise from' syntax. > > Thank you, Chris, that made me feel like a REAL Python programmer -- I just > did some reading,

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread Chris Angelico
On Fri, Nov 22, 2013 at 4:01 AM, John Ladasky wrote: > Here is the end of the traceback, starting with the last line of my code: > "result = pool.map(evaluate, bundles)". After that, I'm into Python itself. > > File ".../evaluate.py", line 81, in evaluate > result = pool.map(evaluate, bund

Re: Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread John Ladasky
On Thursday, November 21, 2013 9:24:33 AM UTC-8, Chris Angelico wrote: > Hmm. This looks like a possible need for the 'raise from' syntax. Thank you, Chris, that made me feel like a REAL Python programmer -- I just did some reading, and the "raise from" feature was not implemented until Python

Traceback when using multiprocessing, less than helpful?

2013-11-21 Thread John Ladasky
Hi folks, Somewhat over a year ago, I struggled with implementing a routine using multiprocessing.Pool and numpy. I eventually succeeded, but I remember finding it very hard to debug. Now I have managed to provoke an error from that routine again, and once again, I'm struggling. Here is the