November 2014 6:00 PM
To: 'Blake Thompson'; Even Rouault
Cc: gdal-dev@lists.osgeo.org
Subject: RE: [gdal-dev] RFC 47 and Threading
Blake,
I tested your proposed change. It is better but sleeps are still visible. BTW:
isn’t it an example of double checked locking?
Even,
I am using Intel VTune
imieniu Blake Thompson [flippm...@gmail.com]
Wysłano: 29 sierpnia 2014 02:35
Do: Even Rouault
DW: gdal-dev@lists.osgeo.org
Temat: Re: [gdal-dev] RFC 47 and Threading
Even and Andre,
I want to start off by saying a big thanks to Blake for taking his time
to tackle what can only be a very difficult
-boun...@lists.osgeo.org] w
imieniu Blake Thompson [flippm...@gmail.com]
Wys³ano: 29 sierpnia 2014 02:35
Do: Even Rouault
DW: gdal-dev@lists.osgeo.org
Temat: Re: [gdal-dev] RFC 47 and Threading
Even and Andre,
I want to start off by saying a big thanks to Blake for taking his time
[gdal-dev-boun...@lists.osgeo.org]
w
imieniu Blake Thompson [flippm...@gmail.com]
Wys³ano: 29 sierpnia 2014 02:35
Do: Even Rouault
DW: gdal-dev@lists.osgeo.org
Temat: Re: [gdal-dev] RFC 47 and Threading
Even and Andre,
I want to start off by saying a big thanks to Blake for taking
Even,
Yeah, that why was I suggested the reverse ;-) But an automatic renaming
should be doable.
Another danger of keeping IReadBlock and making it now responsible of
ensuring
its thread safety is for out-of-tree drivers that wouldn't realize they
need
to change something as code would
Even and Andre,
I want to start off by saying a big thanks to Blake for taking his time
to tackle what can only be a very difficult problem.
From what I can observe, the current discussion seems to be around the
boundary of who should be responsible for ensuring thread safety around
the
I want to start off by saying a big thanks to Blake for taking his time
to tackle what can only be a very difficult problem.
From what I can observe, the current discussion seems to be around the
boundary of who should be responsible for ensuring thread safety around
the block cache. The core
Selon Andre Vautour andre.vaut...@caris.com:
I want to start off by saying a big thanks to Blake for taking his time
to tackle what can only be a very difficult problem.
From what I can observe, the current discussion seems to be around the
boundary of who should be responsible for ensuring
Thanks Blake for the detailed response. I did not realize that I did not do
a reply all in my previous email I sent.
On Thu, Aug 21, 2014 at 3:59 PM, Blake Thompson flippm...@gmail.com wrote:
On Thu, Aug 21, 2014 at 11:36 AM, Jeff Lacoste jefflacosteg...@gmail.com
wrote:
Hi,
Improving
I've got threading issues on my list of things to investigate with GDAL.
I've been running MSAN on GDAL lately and want to get TSAN going too. I'm
out on leave this month, but hope to get more into this stuff (testing,
threading, caching, etc) come October.
On Fri, Aug 22, 2014 at 6:11 AM,
Robert,
I agree - from reading it seems like the major improvement is shifting
away from a global, locking cache to a per-dataset cache. (ah. has been
edited since you read it?)
Yes, I updated the RFC some yesterday after the email from Jeff Lacoste. He
forgot to hit reply all so it ended
Jeff,
Thanks Blake for the detailed response. I did not realize that I did not do
a reply all in my previous email I sent.
Not an issue, glad you guys are interested in my changes.
-- I thought that this was not possible using current trunk GDAL because
of the global cache. At least the
Kurt,
On Fri, Aug 22, 2014 at 9:07 AM, Kurt Schwehr schw...@gmail.com wrote:
I've got threading issues on my list of things to investigate with GDAL.
I've been running MSAN on GDAL lately and want to get TSAN going too. I'm
out on leave this month, but hope to get more into this stuff
Blake,
With a 2 week old baby, my brain has been soggy.
I do see the 3 threads here:
http://lists.osgeo.org/pipermail/gdal-dev/2014-August/thread.html
The asan/msan/tsan tools are
asan - address sanitizer - https://code.google.com/p/address-sanitizer/
msan - memory sanitizer -
Kurt,
With a 2 week old baby, my brain has been soggy.
Congrats!
I do see the 3 threads here:
http://lists.osgeo.org/pipermail/gdal-dev/2014-August/thread.html
The asan/msan/tsan tools are
asan - address sanitizer - https://code.google.com/p/address-sanitizer/
msan - memory
Quick question - presumably for VRT datasets any source images currently
share the global cache and are treated from this proposals' POV as their
own datasets? As well as the VRT being a separate dataset? If so, seems
like this could be quite a major win for users with VRTs for tiled
Le vendredi 22 août 2014 17:53:50, Blake Thompson a écrit :
Jeff,
Thanks Blake for the detailed response. I did not realize that I did not do
a reply all in my previous email I sent.
Not an issue, glad you guys are interested in my changes.
-- I thought that this was not possible
Even,
This might be a problem in practice since the amount of cache might grow
quickly. By default a VRT will open simultaneously up to
GDAL_MAX_DATASET_POOL_SIZE (whose default is 100, but can be changed at
runtime with the config option) source datasets.
If you do a gdal_translate of the
Even,
On Fri, Aug 22, 2014 at 1:33 PM, Even Rouault even.roua...@spatialys.com
wrote:
Note: after re-reading, I realize that I misread your above sentence as it
will work to translate multiple datasets in a parallel way with a *per-
dataset* cache).
So, even if you didn't write it, I'm
Le vendredi 22 août 2014 21:34:49, Blake Thompson a écrit :
Even,
This might be a problem in practice since the amount of cache might grow
quickly. By default a VRT will open simultaneously up to
GDAL_MAX_DATASET_POOL_SIZE (whose default is 100, but can be changed at
runtime with the
Even,
The naming is perhaps not well choosen, but the documentation of the
contract
of this API should make it clear on what an implementation should do and
what
it should not do.
Perhaps it should be reversed? IReadBlock and IReadBlock_not_thread_safe?
(would obviously require lots of
Le vendredi 22 août 2014 22:17:37, Blake Thompson a écrit :
Even,
The naming is perhaps not well choosen, but the documentation of the
contract
of this API should make it clear on what an implementation should do and
what
it should not do.
Perhaps it should be reversed? IReadBlock
On Thu, Aug 21, 2014 at 11:36 AM, Jeff Lacoste jefflacosteg...@gmail.com
wrote:
Hi,
Improving the thread safety of GDAL is a big improvement. I know this
proposal is not claiming to 'fix' gdal thread safety but adress at least
the cache safety. This is said, may be to help
clarify more the
On Thu, Aug 21, 2014 at 8:26 AM, Even Rouault even.roua...@spatialys.com
wrote:
I guess the silence in thread is due to people being impressed by the
austerity of the topic...
Something like that :)
On the other side, I would be very pleased with having just the
preliminary
step of
Hi,
I guess the silence in thread is due to people being impressed by the
austerity of the topic...
For what is worth, I've had a few opportunities to discuss directly with Blake
about his work. I'm very impressed with his energy and enthousiasm to hack in
that difficult core area of GDAL,
Even,
Thank you very much for response to this, your helping me understand all of
this stuff has been very valuable.
I'm not sure the ratio gain/effort to make dataset methods a bit more
thread
safe is high enough. Very few drivers can be made fully thread-safe, and
fully
parallelized (i.e.
I wanted to call to everyone's attention the work I have been doing in an
attempt to make it possible for more portions of GDAL to be thread safe and
improve speed in multi-threaded environments. I have put a RFC up here:
http://trac.osgeo.org/gdal/wiki/rfc47_dataset_caching
Originally my work
27 matches
Mail list logo
