Evan, I am not sure how to profile as I do not have access to the code to profile. I did do a timing test...
vrt file = 22,970 KB bil file = 35,180 KB * 55,501 I piped five locations from the loc.txt file: -96.0 36.0 -98.0 37.0 -100.0 38.0 -99.0 39.0 -101.0 35.0 gdallocationinfo -valonly -geoloc intermap.vrt < loc.txt 189.841857910156 25.5 sec 384.857452392578 22.6 sec 762.015930175781 22.9 sec 550.719116210938 23.6 sec 883.637023925781 22.9 sec Note: I used a lap timer on my iPhone to capture the split times as the results appeared in the console window. Does this give any insight? David -----Original Message----- From: gdal-dev-boun...@lists.osgeo.org [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Even Rouault Sent: Saturday, February 01, 2014 1:28 AM To: Brian Case Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Fast Pixel Access Le samedi 01 février 2014 00:23:13, Brian Case a écrit : > evenr > > > what about the use of a tileindex? You really mean a tileindex as produced by gdaltindex ? Well, that's not exactly the same beast as a VRT, but yes if it was recognized as a GDAL dataset then you could potentially save the cost of XML parsing. One could imagine that the VRT driver would accept a tileindex as an altenate connection string. Anyway it would be interesting to first profile where the time is spent in David use case. If it's in the XML parsing, then I can't see what could be easily improved in that area. If it's the intersection, then there's potential for improvement. > seems an intersection with a set of > polys first would be quick > > brian > > On Fri, 2014-01-31 at 19:30 +0100, Even Rouault wrote: > > Le vendredi 31 janvier 2014 17:15:53, David Baker (Geoscience) a écrit : > > > Dev's, > > > > > > I have a set of 55,501 bil files in a single directory. They are DEMS > > > data that cover the US in 7.5 minute tiles. I would like to randomly > > > access elevations at a given lat/lon's from the whole dataset. I > > > created a vrt file from the directory of bil files, and have been able > > > to access the elevation at a given lat/lon using gdallocationinfo, but > > > because of the size of the dataset, this operation is somewhat slow. > > > Can the vrt be indexed? > > > > No, it isn't currently, although I think it could be improved to have a > > in- memory index with moderate effort. > > > > But are you sure the slowness is due to the lack of index ? 55,000 is a > > big number, but not that big. Maybe the slowness just comes from the > > opening time (XML parsing) of such a big VRT. That would need to be > > profiled to be sure where the bottleneck is. > > > > > Or, is there a faster, better way to access the pixels? I would > > > first like to do this with the utilities before diving into code (C#). > > > The files are regularly named base on their location within a 1 > > > arc-second grid. > > > > > > Thanks, > > > David > > > > > > David M. Baker > > > Senior Advisor - Geoscience Technology > > > Chesapeake Energy Corporation > > > david.m.ba...@chk.com<mailto:david.m.ba...@chk.com> > > > > > > > > > ________________________________ > > > > > > This email (and attachments if any) is intended only for the use of the > > > individual or entity to which it is addressed, and may contain > > > information that is confidential or privileged and exempt from > > > disclosure under applicable law. If the reader of this email is not > > > the intended recipient, or the employee or agent responsible for > > > delivering this message to the intended recipient, you are hereby > > > notified that any dissemination, distribution or copying of this > > > communication is strictly prohibited. If you have received this > > > communication in error, please notify the sender immediately by return > > > email and destroy all copies of the email (and attachments if any). -- Geospatial professional services http://even.rouault.free.fr/services.html _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev ________________________________ This email (and attachments if any) is intended only for the use of the individual or entity to which it is addressed, and may contain information that is confidential or privileged and exempt from disclosure under applicable law. If the reader of this email is not the intended recipient, or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by return email and destroy all copies of the email (and attachments if any). _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev