Re: [gdal-dev] Optimizing access to shapefiles

2010-07-23 Thread Yevgen Antymyrov
On 07/23/2010 02:31 PM, Martin Dobias wrote: This is exactly my intent. Vector data providers in QGIS always receive a list of fields that are needed, however until now OGR data provider couldn't make use of it, because currently there's no way in OGR to limit what fields should be fetched. Knowi

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-23 Thread Martin Dobias
On Fri, Jul 23, 2010 at 12:47 PM, Yevgen Antymyrov wrote: > On 07/19/2010 02:08 PM, Martin Dobias wrote: >> >> Hi, >> >> in order to speed up rendering in QGIS as a part of my GSoC project, >> I've took some time to profile reading of shapefiles in OGR. From the >> results I'd like to suggest some

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-23 Thread Yevgen Antymyrov
On 07/19/2010 02:08 PM, Martin Dobias wrote: Hi, in order to speed up rendering in QGIS as a part of my GSoC project, I've took some time to profile reading of shapefiles in OGR. From the results I'd like to suggest some changes that significantly contribute to the speed of data retrieval. On a

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Martin Dobias
On Mon, Jul 19, 2010 at 3:46 PM, Frank Warmerdam wrote: > > Would GetFeature() still return a feature with a full vector of > fields, but those not desired just being left in the null state? > If so, I think such an approach would be reasonable.  However, it will > require an RFC process to update

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Martin Dobias
On Mon, Jul 19, 2010 at 7:54 PM, Frank Warmerdam wrote: > Martin Dobias wrote: >> One note to avoid confusion: the suggestion I've made above relates >> only to shapefile driver in OGR and doesn't impose any changes to the >> API. The suggested patch reuses OGRShape instances which are passed >> b

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Ragi Burhum
On Mon, Jul 19, 2010 at 10:54 AM, Frank Warmerdam wrote: > Ragi Burhum wrote: > >> Would it make sense instead of implementing a SetDesiredFields(..) to >> implement a SetSubFields(string fieldnames) where the function >> takes a comma delimited list of subfields and then those are parsed by the >

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Frank Warmerdam
Ragi Burhum wrote: Would it make sense instead of implementing a SetDesiredFields(..) to implement a SetSubFields(string fieldnames) where the function takes a comma delimited list of subfields and then those are parsed by the shapefile driver to find out which field values to fetch? That way,

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Martin Dobias
On Mon, Jul 19, 2010 at 6:50 PM, Ragi Burhum wrote: >> >> 1. allow users of OGR library set which fields they really need. Most >> >> of time is wasted by fetching all the attributes, but typically none >> >> or just one attribute is necessary when rendering. For that, I've >> >> added the followi

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Ragi Burhum
> > Date: Mon, 19 Jul 2010 16:34:40 +0200 > From: Martin Dobias > Subject: Re: [gdal-dev] Optimizing access to shapefiles > To: Frank Warmerdam > Cc: gdal-dev@lists.osgeo.org > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Frank &g

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Martin Dobias
Hi Frank On Mon, Jul 19, 2010 at 3:46 PM, Frank Warmerdam wrote: >> 1. allow users of OGR library set which fields they really need. Most >> of time is wasted by fetching all the attributes, but typically none >> or just one attribute is necessary when rendering. For that, I've >> added the follo

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Frank Warmerdam
Martin Dobias wrote: Hi, in order to speed up rendering in QGIS as a part of my GSoC project, I've took some time to profile reading of shapefiles in OGR. From the results I'd like to suggest some changes that significantly contribute to the speed of data retrieval. On a test shapefile of a road

Re: [gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Chaitanya kumar CH
Martin, These changes look quite promising. Please add two tickets at http://trac.osgeo.org/gdal/newticket , one for each topic, and attach the patch. On Mon, Jul 19, 2010 at 5:38 PM, Martin Dobias wrote: > Hi, > > in order to speed up rendering in QGIS as a part of my GSoC project, > I've took

[gdal-dev] Optimizing access to shapefiles

2010-07-19 Thread Martin Dobias
Hi, in order to speed up rendering in QGIS as a part of my GSoC project, I've took some time to profile reading of shapefiles in OGR. From the results I'd like to suggest some changes that significantly contribute to the speed of data retrieval. On a test shapefile of a road network (about 100 tho