As usual Even, spot on! Thank you for precise diagnosis and the quick solution. My project is back on track and the product can be delivered soon. I am guessing that I could bump the pool_size up somewhat. I plan to play around with that value and maybe get a little more performance (on my old Mac laptop...).
Thanks again, David. On Tue, May 20, 2014 at 3:37 AM, Even Rouault <even.roua...@mines-paris.org>wrote: > Selon David Fawcett <david.fawc...@gmail.com>: > > > I am using gdal_translate and a .vrt to take 115 .jp2 files from the > USGS, > > pull them together and then export a lower resolution .jp2. > > > > This is with GDAL 1.10 on OSX. > > gdalinfo --version > > GDAL 1.10.1, released 2013/08/26 > > > > When I run gdal_translate, it gets about 30% of the way through the > process > > and errors out saying that it can't open one of the .j2w files. When I > run > > gdalinfo on the file, I can open it, and if I run gdal_translate on just > > that file, I can successfully export a reduced resolution copy. > > > > The output of gdalinfo is here: > > https://gist.github.com/fawcett/beead31741e24394b38e > > > > I can also load the .vrt image into QGIS 2.2 and it loads (slowly of > > course). > > Even when you zoom out at 10% like your gdal_translate command ? > > > > > gdal_translate img_index.vrt mpls34.jp2 -outsize 10% 10% -of JP2OpenJPEG > > -co QUALITY=25 -epo > > Input file size is 45000, 65000 > > 0...10...20.ERROR 4: > > > > CSLLoad2("/Users/david/projects/mpls_watersheds/data/15TVK830815_201203_0x3000m_CL_1/15TVK830815_201203_0x3000m_CL_1.j2w") > > failed: unable to open file. > > ERROR 4: > > > > `/Users/david/projects/mpls_watersheds/data/15TVK830830_201203_0x3000m_CL_1/15TVK830830_201203_0x3000m_CL_1.jp2' > > not recognised as a supported file format. > > > > gdalinfo 15TVK830815_201203_0x3000m_CL_1.jp2 > > Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library > > Files: 15TVK830815_201203_0x3000m_CL_1.jp2 > > 15TVK830815_201203_0x3000m_CL_1.j2w > > Size is 5000, 5000 > > Coordinate System is: > > PROJCS["NAD83 / UTM zone 15N", > > GEOGCS["NAD83", > > DATUM["North_American_Datum_1983", > > SPHEROID["GRS 1980",6378137,298.2572221010002, > > AUTHORITY["EPSG","7019"]], > > AUTHORITY["EPSG","6269"]], > > PRIMEM["Greenwich",0], > > UNIT["degree",0.0174532925199433], > > AUTHORITY["EPSG","4269"]], > > PROJECTION["Transverse_Mercator"], > > PARAMETER["latitude_of_origin",0], > > PARAMETER["central_meridian",-93], > > PARAMETER["scale_factor",0.9996], > > PARAMETER["false_easting",500000], > > PARAMETER["false_northing",0], > > UNIT["metre",1, > > AUTHORITY["EPSG","9001"]], > > AUTHORITY["EPSG","26915"]] > > Origin = (483000.000000000000000,4983000.000000000000000) > > Pixel Size = (0.300000000000012,-0.300000000000186) > > Image Structure Metadata: > > INTERLEAVE=PIXEL > > Corner Coordinates: > > Upper Left ( 483000.000, 4983000.000) ( 93d12'56.50"W, 45d 0' 0.87"N) > > Lower Left ( 483000.000, 4981500.000) ( 93d12'56.32"W, 44d59'12.26"N) > > Upper Right ( 484500.000, 4983000.000) ( 93d11'47.99"W, 45d 0' 1.00"N) > > Lower Right ( 484500.000, 4981500.000) ( 93d11'47.82"W, 44d59'12.39"N) > > Center ( 483750.000, 4982250.000) ( 93d12'22.16"W, 44d59'36.63"N) > > Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red > > Overviews: 2500x2500, 1250x1250, 625x625, 312x312, 156x156 > > Overviews: arbitrary > > Band 2 Block=1024x1024 Type=Byte, ColorInterp=Green > > Overviews: 2500x2500, 1250x1250, 625x625, 312x312, 156x156 > > Overviews: arbitrary > > Band 3 Block=1024x1024 Type=Byte, ColorInterp=Blue > > Overviews: 2500x2500, 1250x1250, 625x625, 312x312, 156x156 > > Overviews: arbitrary > > > > The .vrt file contents are here: > > https://gist.github.com/fawcett/c8cd5d271b0a89e67779 > > > > Does anyone have a suggestion on how to troubleshoot this? I am > wondering > > if I am just running out of memory. > > I guess you might rather run out of file descriptors. What is the default > limit > on Mac : 1024 as on Linux ? > For each opened JP2 by the OpenJPEG driver, it maintains a file descriptor > per > overview level (and for the main level). So in your above case, 6 file > descriptors. > The VRT mechanism shouldn't open simultaneously more than > GDAL_MAX_DATASET_POOL_SIZE=100 datasets > ( > http://trac.osgeo.org/gdal/browser/trunk/gdal/gcore/gdalproxypool.cpp#L313 > ). > 100 * 6 = 600... So if you have JP2 images bigger than the one you > sampled, you > might have more overview levels. Or possibly the limit on Mac is lower than > 1024. > You might want to try setting the GDAL_MAX_DATASET_POOL_SIZE configuration > option/environmenet variable to a lower value, let's say 10, and see if it > works > better. > > Even > > >
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev