Hi Jonathan,
I think it is time to wrap up this thread.
After I restarted the computer(by killing several other applications), I
used the suggestions you provided:
cl <- makeCluster(spec=4,type="PSOCK")
registerDoParallel(cl)
I succeeded with the parallel processing, with the files that I did be
Responses below:
On Thu, Apr 10, 2014 at 4:49 PM, ping yang wrote:
> Hi Jonathan,
>
> I am using GDAL 1.9.2 and -wo NUM_THREADS=ALL_CPUS didn't work. Actually I
> have several thousands file to process, so I am thinking of the foreach, I
> changed the code as follows:
GDAL 1.9.2 is pretty old (O
Hi Jonathan,
I am using GDAL 1.9.2 and -wo NUM_THREADS=ALL_CPUS didn't work. Actually I
have several thousands file to process, so I am thinking of the foreach, I
changed the code as follows:
ProjectANDSlope<- function(zipfile)
{
require(raster)
require(rgdal)
require(gdalUtils)
require(f
Ping:
Re: raster:::projectRaster I'm not sure how much you can speed that up
(Robert?) -- I think it is native R code which is generally a lot
slower than compiled code like the GDAL utilities. It has to use
"chunking", but for reasonable chunk sizes this is not likely to cause
any noticeable ove
Hi Jonathan,
I am following up with the previous thread, Now I successfully installed
the gdalUtils from the source and ran successfully the task (both
projection and slope calculation) using the following script:
rasterOptions(chunksize = 3e+07, maxmemory = 1e+08) #suggestions according
to Jonan
For gdalUtils, you don't need to upgrade to 3.2 -- the latest CRAN
version only requires R 2.14 or later. It should find your cygwin
version, but let me know if it doesn't.
The key with the rasterOptions you used is that, previously, you were
forcing the ENTIRE RASTER to load into memory with the
Hi Alex and Jonathan,
The reason I used the rasterOptions is because when previously I did the
reproject for 30 meter DEM, it worked after I raised the
chunksize(otherwise it ran out of memory), so I added here just to avoid
that issue(probably it not work for this case)
I am going to install gd
Ping:
Re: maxmemory -- you really shouldn't adjust that, definitely not to
what you set it to -- you've set it to (by my quick calculations) to
be around 7500 GB of needed memory (this is assuming each cell takes
64 bits of space) -- this is probably why you are crashing. Chunksize
also is usuall
On Mon, Apr 7, 2014 at 4:05 PM, ping yang wrote:
> Dear r-sig-geo,
>
> I plan to reproject from lonlat to UTM for a bunch of 10 meter NED files
> (large) using a R script(I did it successfully for the 30 meter DEM), I
> used the following code to run:
>
> rasterOptions(chunksize = 1e+10, maxmemor
Dear r-sig-geo,
I plan to reproject from lonlat to UTM for a bunch of 10 meter NED files
(large) using a R script(I did it successfully for the 30 meter DEM), I
used the following code to run:
rasterOptions(chunksize = 1e+10, maxmemory = 1e+12) #need to allow bigger
chunksize for the processing!!
10 matches
Mail list logo
