Hi Roger, I did run with a 90m resolution, with the same error: > srtm <- readRAST("mozambique_srtm_90m",cat=FALSE) Creating BIL support files... Exporting raster as integer values (bytes=4) 100% Warning messages: 1: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : Discarded ellps unknown in CRS definition: +proj=merc +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +R=6378137 +units=m +no_defs 2: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : Discarded datum unknown in CRS definition > image(srtm) Error in rasterImage(m, bb[1, 1], bb[2, 1], bb[1, 2], bb[2, 2], interpolate = interpolate, : cannot allocate memory block of size 16777216 Tb
However, with a 250m resolution the import and visualization worked fine: > srtm <- readRAST("mozambique_srtm_250m",cat=FALSE) Creating BIL support files... Exporting raster as integer values (bytes=4) 100% Warning messages: 1: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : Discarded ellps unknown in CRS definition: +proj=merc +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +R=6378137 +units=m +no_defs 2: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : Discarded datum unknown in CRS definition > image(srtm,col = terrain.colors(20)) Thank you for your help! Best regards, Tom On Sun, Jul 12, 2020 at 9:19 AM Roger Bivand <roger.biv...@nhh.no> wrote: > On Sun, 12 Jul 2020, Thomas Adams wrote: > > > Hi Roger, > > > > I believe I may have been correct about the memory issue... it's > possible, > > in GRASS, that I did not restrict my REGION sufficiently, using g.region. > > Additionally, I applied a MASK for my country of interest, Mozambique, > > before starting R, which yielded: > > > > srtm <- readRAST("mozambique_srtm_patch",cat=FALSE) > > Creating BIL support files... > > Exporting raster as integer values (bytes=4) > > 100% > > Warning messages: > > 1: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : > > Discarded ellps unknown in CRS definition: +proj=merc +lat_ts=0 +lon_0=0 > > +x_0=0 +y_0=0 +R=6378137 +units=m +no_defs > > 2: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : > > Discarded datum unknown in CRS definition > >> image(srtm) > > Error in rasterImage(m, bb[1, 1], bb[2, 1], bb[1, 2], bb[2, 2], > interpolate > > = interpolate, : > > cannot allocate memory block of size 16777216 Tb > > I'd got to an errant g.region before r.out.bin too. > > graphics::rasterImage() is reporting the error. The metric is wrong, but > the problem is real. The object is about 650M x 4 bytes, which image() > needs to manipulate too, creating several copies (in the imported object > it is a vector, but is cast to a matrix and transposed). > > > > > Still, a huge amount of memory needed (is 16777216 Tb correct??) > > > > My thought was to resample the SRTM data in GRASS to 1km resolution, then > > start R, and proceed, which quite honestly, for my purposes, really is > > sufficient. By doing this, everything worked perfectly! I was able to > read > > the srtm data quickly and generate an image. I am puzzled by the "cannot > > allocate memory block of size 16777216 Tb" message with my GRASS Region > > settings in place. > > Please try r.out.gdal in 90m, then stars::read_stars(..., proxy=TRUE) > (outside rgrass7). We need to try to get stars to work in rgrass7, but we > had not gone into that yet. In stars and gdalcubes, the idea is to do work > on the back end, and only bring data to (say) display in the feasible > resolution. So I think something like: > > library(stars) > big <- read_stars("big.tif", proxy=TRUE) > plot(big) > > automates the downsampling. The default arguments to readRAST() assume > that uncompressed binary is the optimal transfer strategy, and that all > the data should be in R's workspace. > > I can speculate that if GDAL used the GRASS plugin, it might be possibble > to proxy the actual GRASS CELL or DCELL. However, someone else would need > to take responsibility for implementing prototypes for that. The same > applies to support for stars, alternatively for terra in addition (raster > is used by many, but with limited resources, stars and terra should be > preferred). > > Roger > > > > > Thank you for all your help! > > > > Best, > > Tom > > > > > > > > On Sat, Jul 11, 2020 at 11:02 PM Thomas Adams <tea...@gmail.com> wrote: > > > >> Hi Roger, > >> > >> r.out.bin from GRASS did succeed... r.out.bin > >> input=mozambique_srtm_patch@teaiii output=mozambique_srtm_patch > null=-999 > >> Exporting raster as integer values (bytes=2) > >> > >> A very large file!! > >> -rw-r--r-- 1 teaiii teaiii 1045368225220 Jul 11 13:26 > >> mozambique_srtm_patch > >> > >> Tom > >> > >> > >> On Sat, Jul 11, 2020 at 12:12 PM Roger Bivand <roger.biv...@nhh.no> > wrote: > >> > >>> Briefly, please try to run r.out.bin manually, to the same output or > >>> another. This is what is failing on theGRASS side. > >>> > >>> Roger > >>> > >>> ------- > >>> Roger Bivand > >>> Norwegian School of Economics, Bergen, Norway > >>> roger.biv...@nhh.no > >>> > >>> ------------------------------ > >>> *Fra:* Thomas Adams <tea...@gmail.com> > >>> *Sendt:* lørdag 11. juli 2020, 17:59 > >>> *Til:* Roger Bivand > >>> *Kopi:* r-sig-Geo@r-project.org > >>> *Emne:* Re: [R-sig-Geo] Issue with rgrass7 and readRAST > >>> > >>> Hi Roger! > >>> > >>> Here is what was written in > >>> > /media/teaiii/development/grass/greensboro_mercator/teaiii/.tmp/teaiii-Leopard-WS: > >>> > >>> drwxr-xr-x 3 teaiii teaiii 4096 Sep 23 2019 .. > >>> -rw-r--r-- 1 teaiii teaiii 131072 Jul 9 10:02 801.0.gpkg > >>> -rw-r--r-- 1 teaiii teaiii 131072 Jul 9 10:04 466.0.gpkg > >>> -rw-r--r-- 1 teaiii teaiii 0 Jul 9 20:32 801.0 > >>> -rw-r--r-- 1 teaiii teaiii 0 Jul 9 22:40 188.0 > >>> -rw-r--r-- 1 teaiii teaiii 0 Jul 9 22:51 222.0 > >>> > >>> And there is plenty of space on the disk drive... > >>> > >>> Tom > >>> > >>> On Sat, Jul 11, 2020 at 9:26 AM Roger Bivand <roger.biv...@nhh.no> > wrote: > >>> > >>>> Tom > >>>> > >>>> On Fri, 10 Jul 2020, Thomas Adams wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> I'm using R in combination with GRASS GIS 7.8.3, attempting to read a > >>>> GRASS > >>>>> raster file into R with the rgrass7 interface. I have done this kind > of > >>>>> thing countless times previously without problems. It's quite > possible > >>>> I am > >>>>> running into memory issues, but my Ubuntu 18.04 Linux computer has 32 > >>>> GB of > >>>>> RAM. > >>>>> > >>>> > >>>> I have not succeeded in reproducing this, I'm afraid. I created a > >>>> location > >>>> with cell counts to match, and ran r.out.bin first to check whether > >>>> there > >>>> were any issues writing the data to a temporary file. Then I ran the > >>>> equivalent of your command (readRAST()), again without problems. I'm > on > >>>> a > >>>> 16 GB PC, so the problem is puzzling. I'm curious about the free space > >>>> on > >>>> your > >>>> > >>>> /media/teaiii/development/grass/greensboro_mercator/teaiii/.tmp/ > >>>> teaiii-Leopard-WS > >>>> > >>>> directory. Did you check whether any output arrived there, since the > >>>> error > >>>> message is coming from r.out.bin? > >>>> > >>>> The error is on line 53 in raster/r.out.bin/main.c, when the system > >>>> function fwrite() returns an error. > >>>> > >>>> Thanks for your patience. > >>>> > >>>> Roger > >>>> > >>>> > >>>>> My GRASS raster file: > >>>>> > >>>>> Type of Map: raster Number of Categories: 0 > >>>>> Data Type: CELL > >>>>> Rows: 22663 > >>>>> Columns: 28794 > >>>>> Total Cells: 652558422 > >>>>> ~90m resolution > >>>>> > >>>>> Using... > >>>>> > >>>>> srtm <- readRAST("mozambique_srtm_patch",cat=FALSE) > >>>>> > >>>>> After quite some time, I get... > >>>>> > >>>>> Error in execGRASS("r.out.bin", flags = rOutBinFlags, input = > >>>> vname[i], : > >>>>> The command: > >>>>> r.out.bin -b -i input=mozambique_srtm_patch > >>>>> > >>>> > output=/media/teaiii/development/grass/greensboro_mercator/teaiii/.tmp/teaiii-Leopard-WS/mozambique_srtm_patch > >>>>> bytes=4 null=-81 > >>>>> produced an error (1) during execution: > >>>>> Creating BIL support files... > >>>>> Exporting raster as integer values (bytes=4) > >>>>> ERROR: Error writing data > >>>>> In addition: Warning messages: > >>>>> 1: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : > >>>>> Discarded ellps unknown in CRS definition: +proj=merc +lat_ts=0 > >>>> +lon_0=0 > >>>>> +x_0=0 +y_0=0 +R=6378137 +units=m +no_defs > >>>>> 2: In showSRID(uprojargs, format = "PROJ", multiline = "NO") : > >>>>> Discarded datum unknown in CRS definition > >>>>> > >>>>> Libraries loaded include: sp, rgrass7, rgdal, raster... > >>>>> > >>>>> Any thoughts? > >>>>> > >>>>> Best, > >>>>> Tom > >>>>> > >>>>> > >>>> > >>>> -- > >>>> Roger Bivand > >>>> Department of Economics, Norwegian School of Economics, > >>>> Helleveien 30, N-5045 Bergen, Norway. > >>>> voice: +47 55 95 93 55; e-mail: roger.biv...@nhh.no > >>>> https://orcid.org/0000-0003-2392-6140 > >>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en > >>>> > >>> > >>> > >>> -- > >>> Thomas E Adams, III > >>> 1724 Sage Lane > >>> Blacksburg, VA 24060 > >>> tea...@gmail.com (personal) > >>> t...@terrapredictions.org (work) > >>> > >>> 1 (513) 739-9512 (cell) > >>> > >>> > >>> > >> > >> -- > >> Thomas E Adams, III > >> 1724 Sage Lane > >> Blacksburg, VA 24060 > >> tea...@gmail.com (personal) > >> t...@terrapredictions.org (work) > >> > >> 1 (513) 739-9512 (cell) > >> > >> > > > > > > -- > Roger Bivand > Department of Economics, Norwegian School of Economics, > Helleveien 30, N-5045 Bergen, Norway. > voice: +47 55 95 93 55; e-mail: roger.biv...@nhh.no > https://orcid.org/0000-0003-2392-6140 > https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en -- Thomas E Adams, III 1724 Sage Lane Blacksburg, VA 24060 tea...@gmail.com (personal) t...@terrapredictions.org (work) 1 (513) 739-9512 (cell) [[alternative HTML version deleted]] _______________________________________________ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo