Re: [gdal-dev] Fast Pixel Access

2014-02-10 Thread David Baker (Geoscience)
) Cc: 'Brian Case'; 'gdal-dev@lists.osgeo.org' Subject: Re: [gdal-dev] Fast Pixel Access Le samedi 01 février 2014 15:04:46, David Baker (Geoscience) a écrit : Evan, I am not sure how to profile as I do not have access to the code to profile. I did do a timing test... vrt file = 22,970 KB

Re: [gdal-dev] Fast Pixel Access

2014-02-10 Thread Even Rouault
, 2014 6:36 AM To: David Baker (Geoscience) Cc: 'Brian Case'; 'gdal-dev@lists.osgeo.org' Subject: Re: [gdal-dev] Fast Pixel Access Le samedi 01 février 2014 15:04:46, David Baker (Geoscience) a écrit : Evan, I am not sure how to profile as I do not have access to the code to profile. I

Re: [gdal-dev] Fast Pixel Access

2014-02-10 Thread David Baker (Geoscience)
: [gdal-dev] Fast Pixel Access Selon David Baker (Geoscience) david.m.ba...@chk.com: Even, No not an i386... A Dell Precision T3500 w/Intel W3680 @ 3.33GHhz 6x2 cores with 12.0GB. Thought the data is on the network, not local, with 1Gbps access. The GDAL_DISABLE_READDIR_ON_OPEN = TRUE did

Re: [gdal-dev] Fast Pixel Access

2014-02-10 Thread Even Rouault
a usefull stack trace. David -Original Message- From: Even Rouault [mailto:even.roua...@mines-paris.org] Sent: Monday, February 10, 2014 8:54 AM To: David Baker (Geoscience) Cc: 'Even Rouault'; 'Brian Case'; 'gdal-dev@lists.osgeo.org' Subject: RE: [gdal-dev] Fast Pixel Access Selon

Re: [gdal-dev] Fast Pixel Access

2014-02-05 Thread David Baker (Geoscience)
be okay. David From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Luke Roth Sent: Monday, February 03, 2014 8:11 AM To: Jukka Rahkonen Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access Another thing that might speed up access

Re: [gdal-dev] Fast Pixel Access

2014-02-04 Thread Jukka Rahkonen
David Baker (Geoscience david.m.baker at chk.com writes: Jukka, No matter the endpoint the user uses to access the data, behind the scenes, there must be fast pixel access, correct? Or are you saying that at WFS would do it quickly out of the box? Hi, Fast access to data, yes, but using

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread Luke Roth
Another thing that might speed up access is setting the config option GDAL_DISABLE_READDIR_ON_OPEN = TRUE, either as an environment variable or on the command line. That should help with GDAL reading the directory each time it opens a dataset. I have an application which reads one value from

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread David Baker (Geoscience)
: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Daniel Clewley Sent: Saturday, February 01, 2014 1:45 PM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access Hi David, Following on from the VRT / Bigtiff comparison Jukka posted. Have you

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread Jukka Rahkonen
Luke Roth roth.luke at gmail.com writes: Another thing that might speed up access is setting the config option GDAL_DISABLE_READDIR_ON_OPEN = TRUE, either as an environment variable or on the command line.  That should help with GDAL reading the directory each time it opens a dataset.  I have

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread Brian Case
-Jukka tileindex, mapserver, and the gdal wms driver On Mon, 2014-02-03 at 17:20 +, Jukka Rahkonen wrote: Luke Roth roth.luke at gmail.com writes: Another thing that might speed up access is setting the config option GDAL_DISABLE_READDIR_ON_OPEN = TRUE, either as an environment

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread Rahkonen Jukka (Tike)
Hi, Perhaps, but in this game the rule was not to have any GIS servers. Myself I would rather consider WFS. It could send heights from single points but also a profile along a line or all values within a polygon. -Jukka- Brian Case [r...@winkey.org] wrote: -Jukka tileindex, mapserver, and

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread David Baker (Geoscience)
...@lists.osgeo.org] On Behalf Of Rahkonen Jukka (Tike) Sent: Monday, February 03, 2014 11:37 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access Hi, Perhaps, but in this game the rule was not to have any GIS servers. Myself I would rather consider WFS. It could send heights from

Re: [gdal-dev] Fast Pixel Access

2014-02-03 Thread Klokan Petr Přidal
-Original Message- From: gdal-dev-boun...@lists.osgeo.org [mailto: gdal-dev-boun...@lists.osgeo.org] On Behalf Of Rahkonen Jukka (Tike) Sent: Monday, February 03, 2014 11:37 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access Hi, Perhaps, but in this game the rule

Re: [gdal-dev] Fast Pixel Access

2014-02-02 Thread Jukka Rahkonen
Hi, I made a few tests and here comes my conclusions. Hypothesis is that someone wants to make a DEM query service which is using gdallocationinfo for queries and DEM data is to be accessed as files from a standard web site. I compared three alternatives: 1) There are thousands of DEM files on

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread Jukka Rahkonen
David Baker (Geoscience david.m.baker at chk.com writes: Dev’s,   I have a set of 55,501 bil files in a single directory.  They are DEMS data that cover the US in 7.5 minute tiles.  I would like to randomly access elevations at a given lat/lon’s from the whole dataset.  I created a vrt

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread David Baker (Geoscience)
window. Does this give any insight? David -Original Message- From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Even Rouault Sent: Saturday, February 01, 2014 1:28 AM To: Brian Case Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread David Baker (Geoscience)
Of Jukka Rahkonen Sent: Saturday, February 01, 2014 7:09 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access David Baker (Geoscience david.m.baker at chk.com writes: Dev’s, I have a set of 55,501 bil files in a single directory. They are DEMS data that cover the US

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread Norman Vine
Of Jukka Rahkonen Sent: Saturday, February 01, 2014 7:09 AM To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access David Baker (Geoscience david.m.baker at chk.com writes: Dev’s, I have a set of 55,501 bil files in a single directory. They are DEMS data that cover

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread David Baker (Geoscience)
...@cape.com] Sent: Saturday, February 01, 2014 8:23 AM To: David Baker (Geoscience) Cc: gdal-dev Subject: Re: [gdal-dev] Fast Pixel Access This is an application that is just screaming for a spatial index For starters you could build a spatialite db of the individual file extents that returned

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread Jukka Rahkonen
David Baker (Geoscience david.m.baker at chk.com writes: Jukka, Jukka wrote: I was experimenting with something like a GIS service without a GIS server) and I have some examples online but... I am looking to do as you have, a RESTful service to query the elevation at a given location.

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread Joaquim Luis
Now what to do instead? A WMS/WCS service can send a piece of DEM with thousands of pixels as GeoTIFF in a second and a heavy client like QGIS could continue the analysis. Or you can use the existing vrt file and read the region of interest with gdal_translate. This request is not very fast

Re: [gdal-dev] Fast Pixel Access

2014-02-01 Thread Daniel Clewley
(Geoscience) david.m.ba...@chk.com To: 'gdal-dev@lists.osgeo.org' gdal-dev@lists.osgeo.org Subject: [gdal-dev] Fast Pixel Access Message-ID: 2a18a4344312134b937df938d992264a0508f...@okcexhprd122.chkenergy.net Content-Type: text/plain; charset=us-ascii Dev's, I have a set of 55,501 bil files

[gdal-dev] Fast Pixel Access

2014-01-31 Thread David Baker (Geoscience)
Dev's, I have a set of 55,501 bil files in a single directory. They are DEMS data that cover the US in 7.5 minute tiles. I would like to randomly access elevations at a given lat/lon's from the whole dataset. I created a vrt file from the directory of bil files, and have been able to access

Re: [gdal-dev] Fast Pixel Access

2014-01-31 Thread Even Rouault
Le vendredi 31 janvier 2014 17:15:53, David Baker (Geoscience) a écrit : Dev's, I have a set of 55,501 bil files in a single directory. They are DEMS data that cover the US in 7.5 minute tiles. I would like to randomly access elevations at a given lat/lon's from the whole dataset. I

Re: [gdal-dev] Fast Pixel Access

2014-01-31 Thread Brian Case
evenr what about the use of a tileindex? seems an intersection with a set of polys first would be quick brian On Fri, 2014-01-31 at 19:30 +0100, Even Rouault wrote: Le vendredi 31 janvier 2014 17:15:53, David Baker (Geoscience) a écrit : Dev's, I have a set of 55,501 bil files in a

Re: [gdal-dev] Fast Pixel Access

2014-01-31 Thread Even Rouault
Le samedi 01 février 2014 00:23:13, Brian Case a écrit : evenr what about the use of a tileindex? You really mean a tileindex as produced by gdaltindex ? Well, that's not exactly the same beast as a VRT, but yes if it was recognized as a GDAL dataset then you could potentially save the