This should now work if you upgrade to raster_2.1-12. It will still use as much memory as is contained in the file, just to open it deep inside the raster() handling of the connection - but it's not permanent and if you can get past this step, try running gc() and it should settle down (I'm on Windows with ncdf_1.6.6):
library(raster) ## 2.1-12 or greater bat <- raster(file="gebco_08_-82_37_63_80") gc() (I've only tested on gebco_08.nc, the original large file itself). Do not set varname = "z", it should choose that anyway. I'm hoping that a change in ncdf4 will be published to avoid the memory consumption, but I need to put the case for change in ncdf, ncdf4 and raster which potentially involves 3 separate package maintainers. They have each been very patient with me so far. If you want to shortcut to a solution, just use gdal_translate from OSGeo4W, and write to GeoTIFF with (optionally) lossless compression and tiling: gdal_translate gebco_08_-82_37_63_80 gebco_08_-82_37_63_80.tif -co COMPRESS=LZW -co TILED=YES BTW, It's a strange coincidence that this was asked this week just when I was investigating why it did not already work! The GDAL workaround is good, but raster/ncdf/4 is a very nice general fix. And, for what it is worth - GDAL has long supported this NetCDF variant, so a build of rgdal with the GMT driver should work well too with the extra caveats of subsetting with readGDAL() or via a GDALMajorObject. (CRAN's Windows rgdal does not have GMT) Cheers, Mike. On Sat, Mar 9, 2013 at 3:24 PM, Michael Sumner <[email protected]> wrote: > The raster package isn't able to read the data from this "GMT-style" > NetCDF, since the structure of the variable is one-dimensional internally > and you are supposed to deal with that yourself. I would stick with the > ncdf4 package to read it, you'll have to scan into the "z" variable > manually to reconstruct it as a matrix, or subset it. > > Note that you should have a large object size with "d", since the nc_open > function returns an index vector for each dimension (so one of them is > length 89784000, you must have sufficient memory to deal with that - I > would close/delete the "d" object once you read the "z", "dimension" and > other values). > > I have code to read from these files, but I can't access it today - it's > not that difficult though, you can probably do something like matrix(z, > dimension[1], dimension[2]) but you'll have to fiddle to get the > orientation right. > > > > > On Fri, Mar 8, 2013 at 9:18 PM, Julian Burgos <[email protected]> wrote: > >> Dear list, >> >> I am having some troubles opening accessing a raster in the netCDF format. >> The file is bathimetric data from GEBCO: >> http://www.gebco.net/data_and_products/gridded_bathymetry_data/ >> >> If I use the ncdf4 package to open and access the data, I have no issues: >> >> library(ncdf4) >> > d <- nc_open("gebco_08_-82_37_63_80.nc") >> > z <- as.vector(ncvar_get(d,varid="z")) >> > head(z) >> [1] 19 40 61 83 106 128 >> >> >> But if I try to use the raster package to access the file and create a >> raster object, I get the following error: >> >> bat <- raster(file="gebco_08_-82_37_63_80",varname="z") >> Error in .varName(nc, varname, warn = warn) : >> varname: z does not exist in the file. Select one from: >> >> ... and no varnames are listed. >> >> >> If I do not define a variable name and let the raster package choose it, I >> get this: >> >> > bat <- raster(file="gebco_08_-82_37_63_80",varname="z") >> Error in .varName(nc, varname, warn = warn) : >> >> The description of the netCDF file is as following: >> >> > d >> [1] "File gebco_08_-82_37_63_80.nc (NC_FORMAT_CLASSIC):" >> [1] "" >> [1] " 6 variables:" >> [1] " double x_range[side] " >> [1] " units: user_x_unit" >> [1] " double y_range[side] " >> [1] " units: user_y_unit" >> [1] " short z_range[side] " >> [1] " units: user_z_unit" >> [1] " double spacing[side] " >> [1] " int dimension[side] " >> [1] " short z[xysize] " >> [1] " scale_factor: 1" >> [1] " add_offset: 0" >> [1] " node_offset: 1" >> [1] "" >> [1] " 2 dimensions:" >> [1] " side Size:2" >> [1] " xysize Size:89784000" >> [1] "" >> [1] " 2 global attributes:" >> [1] " title: GEBCO_08 Grid" >> [1] " source: 20100927" >> >> I am using R 15.2 on Linux (Fedora), raster 2.1-8, and ncdf4 version >> ncdf4_1.6.1_20120411. >> Any help will be welcomed! >> >> Julian >> >> -- >> Julian Mariano Burgos, PhD >> Hafrannsóknastofnunin/Marine Research Institute >> Skúlagata 4, 121 ReykjavÃk, Iceland >> SÃmi/Telephone : +354-5752037 >> BréfsÃmi/Telefax: +354-5752001 >> Netfang/Email: [email protected] >> >> _______________________________________________ >> R-sig-Geo mailing list >> [email protected] >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo >> > > > > -- > Michael Sumner > Hobart, Australia > e-mail: [email protected] > -- Michael Sumner Hobart, Australia e-mail: [email protected] [[alternative HTML version deleted]]
_______________________________________________ R-sig-Geo mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-geo
