Re: [Community] Shapely With Thread Safe GEOS

2009-04-06 Thread Sean Gillies
Aron, I've committed most of your patch, with a few changes, to the Shapely (1.1) trunk. http://trac.gispython.org/lab/changeset/1226 If you try test_threads.py you'll see that the individual threads now get their own GEOS handles. I like the use of functools.partial. Thanks for the help!

Re: [Community] Shapely With Thread Safe GEOS

2009-02-26 Thread Aron Bierbaum
After doing a lot of testing and thinking I came to a conclusion as to what I think was causing by errors, and have come up with a fix. The basic idea behind my issues were that multiple threads could can not in any way change the ctypes function pointers. With the current code it is possible to ha

Re: [Community] Shapely With Thread Safe GEOS

2009-02-26 Thread Sean Gillies
Give it a try. I like the optimization idea too. On Feb 25, 2009, at 1:25 PM, Aron Bierbaum wrote: > It would be part of the function wrapper returned by the LGEOS > object. The idea being that we have to check before every call. In > order to improve performance a little I was also wondering

Re: [Community] Shapely With Thread Safe GEOS

2009-02-25 Thread Aron Bierbaum
It would be part of the function wrapper returned by the LGEOS object. The idea being that we have to check before every call. In order to improve performance a little I was also wondering about storing the function wrappers in the LGEOS object using setattr so that __getattr__ wouldn't get called

Re: [Community] Shapely With Thread Safe GEOS

2009-02-25 Thread Sean Gillies
That would be a method of what class? On Feb 25, 2009, at 10:43 AM, Aron Bierbaum wrote: > I guess I don't see where these discussions would effect what we are > doing. I am sure I am missing something, but in order to allow no > changes by users, can't we just do something like: > >

Re: [Community] Shapely With Thread Safe GEOS

2009-02-25 Thread Aron Bierbaum
I guess I don't see where these discussions would effect what we are doing. I am sure I am missing something, but in order to allow no changes by users, can't we just do something like: def __call__(self, *args): if not hasattr(thread_data, 'geos_handle'):

Re: [Community] Shapely With Thread Safe GEOS

2009-02-25 Thread Sean Gillies
Aron, r1194 is indeed the last stable revision for a while. Importing modules from a thread is fraught with all kinds of issues. See this interesting discussion: http://bugs.python.org/issue1720705 And the conclusion at the end of http://svn.python.org/view/python/trunk/Doc/library/threading

Re: [Community] Shapely With Thread Safe GEOS

2009-02-24 Thread Aron Bierbaum
Well I finally got around to testing these changes. It appears that there are a lot of changes being made currently, which revision should I be testing with? I have tried running test_threads.py with both trunk and r1194 and the test fails. I briefly looked at r1194 and it looks like the thread spe

Re: [Community] Shapely With Thread Safe GEOS

2009-01-29 Thread Aron Bierbaum
Wow, this looks really nice. I should have time to test it either later tonight or tomorrow. It's also interesting that this should allow an easy transition if/when they move to a thread safe only interface. Thanks, Aron On Thu, Jan 29, 2009 at 4:33 PM, Sean Gillies wrote: > Aron, > > I did a bi

Re: [Community] Shapely With Thread Safe GEOS

2009-01-29 Thread Sean Gillies
Aron, I did a bit of work along those lines and made some pretty good progress. Tests are passing except for a couple I had to disable. Updating of coordinate sequences through geos_linestring and geos_linearring is currently broken. I'm not sure whether it's my fault or a bug in the reent

Re: [Community] Shapely With Thread Safe GEOS

2009-01-28 Thread Aron Bierbaum
This is a really long shot but what do people think of something like the following. I have run into a few problems with ctypes while trying to run with it, but I thought it might start people thinking. import threading import functools class LocalStorage(threading.local): def __getattr__(self

Re: [Community] Shapely With Thread Safe GEOS

2009-01-27 Thread Justin Bronn
Aron Bierbaum wrote: > The only idea that I have off the top of my head is to use > threading.local in order to keep track of a handle for each thread > that calls any function in GEOS. I tried implementing something like > this quickly and ran into problems. Any other ideas? Since I'm going to ha

Re: [Community] Shapely With Thread Safe GEOS

2009-01-26 Thread Aron Bierbaum
The only idea that I have off the top of my head is to use threading.local in order to keep track of a handle for each thread that calls any function in GEOS. I tried implementing something like this quickly and ran into problems. Any other ideas? -Aron On Mon, Jan 26, 2009 at 9:31 PM, Sean Gilli

Re: [Community] Shapely With Thread Safe GEOS

2009-01-26 Thread Sean Gillies
Oh, right. I only meant that GEOSDisjoint (for example) calls GEOSDisjoint_r as you can see in http://trac.osgeo.org/geos/browser/trunk/capi/geos_c.cpp#L94 Thread safety hasn't been a problem up to now, and we've hammered on it quite a bit. We're using PyDLL to load libgeos_c, which means the

Re: [Community] Shapely With Thread Safe GEOS

2009-01-26 Thread Justin Bronn
Aron Bierbaum wrote: > I could be wrong, but I thought that the new API was thread safe only > if you used the _r versions of the method so that you can pass in a > handle to the thread specific data. For example you can look at: > badthreadtest.c and threadtest.c I was under the impression that th

Re: [Community] Shapely With Thread Safe GEOS

2009-01-26 Thread Aron Bierbaum
I could be wrong, but I thought that the new API was thread safe only if you used the _r versions of the method so that you can pass in a handle to the thread specific data. For example you can look at: badthreadtest.c and threadtest.c I was under the impression that the old API could be used in or

Re: [Community] Shapely With Thread Safe GEOS

2009-01-26 Thread Sean Gillies
Aron, The GEOS 3.1 C API *is* the new thread safe API, and I've encountered no problems with it yet. Shapely's tests pass. I encourage you to checkout the Shapely trunk and try it. If you're interested, I could introduce you to the zc.buildout configuration I am using to make python, GEOS,

[Community] Shapely With Thread Safe GEOS

2009-01-26 Thread Aron Bierbaum
Has there been any work/testing to get Shapely to work with the new thread safe GEOS API? http://trac.osgeo.org/geos/wiki/RFC3 http://trac.osgeo.org/geos/ticket/210 http://trac.osgeo.org/geos/changeset/2232 I would be very interested in testing any changes that have been made to support this. Th