On Wed, Mar 12, 2014 at 11:29 PM, Boulanger, Yan <yan.boulan...@rncan-nrcan.gc.ca> wrote: > Actually, I have several rasters of more than 440 000 000 pixels (MODIS > covering all Canada) and I have a 32-cores machine so I would like to take > advantage of it! ;-) > > Time is money (really?!!)
As mentioned earlier, I would be careful about using rasterEngine for this kind of task. It may actually slow you down. I would recommend testing on smaller subsets to determine your gains (or losses) from doing this type of calculation in parallel versus sequentially. While I have seen great speed increases for CPU intensive calculations from using rasterEngine, it sounds like your processing is heavily IO intensive. I am not sure 32 cores will help you unless you have a very fast disk or RAID array. Alex > > Thanks again! > yan > > Yan Boulanger, Chercheur scientifique / Research scientist > Ressources Naturelles Canada, Canadian Forest Service > Centre de Foresterie des Laurentides > 1055, rue du P.E.P.S. > C.P. 10380, succ. Sainte-Foy > Québec (Québec) Canada > G1V 4C7 > Tel. : +1 418 649-6859 > > From: Forrest Stevens [mailto:forr...@ufl.edu] > Sent: 12 mars 2014 22:25 > To: Boulanger, Yan > Cc: r-sig-geo@r-project.org > Subject: Re: [R-sig-Geo] loops in rasterEngine > > Hi Yan, I guess I would be surprised for such a simple process if > rasterEngine() would be worth the overhead? Though, admittedly, Jonathan > Greenberg might have more information on the topic. To do such an operation > this is the approach I would take without using rasterEngine(): > > > for (i in 1:5) { > assign(paste("Safranyik_zones_1961_1990b_",i, sep=""), > Safranyik_zones_1961_1990b == i) > } > > > To do it using rasterEngine() this is the function definition that I would > use. This of course requires that you've already created a cluster using one > of the various supported parallel backends otherwise you'll gain nothing from > the parallel processing. > > > require("spatial.tools") > > ## Begin a parallel cluster and register it with foreach: > ## The number of nodes/cores to use in the cluster > cpus = 2 > cl <- makeCluster(spec = cpus, type = "PSOCK", methods = FALSE) > ## Register the cluster with foreach: > registerDoParallel(cl) > > ## Or use the following, quick and dirty way: > #sfQuickInit(cpus=2) > > fun_zone <- function( zones, i, ...) { > return(zones == i) > } > > for (j in 1:5){ > assign(paste("Safranyik_zones_1961_1990b_",j, sep=""), rasterEngine( > zones=Safranyik_zones_1961_1990b, args=list("i"=j), fun=fun_zone) ) > } > > stopCluster(cl) > #sfQuickStop() > > > Hope this helps, > Forrest > > -- > Forrest R. Stevens > Ph.D. Candidate, QSE3 IGERT Fellow > Department of Geography > Land Use and Environmental Change Institute > University of Florida > www.clas.ufl.edu/users/forrest<http://www.clas.ufl.edu/users/forrest> > > On Wed, Mar 12, 2014 at 8:51 PM, Boulanger, Yan > <yan.boulan...@rncan-nrcan.gc.ca<mailto:yan.boulan...@rncan-nrcan.gc.ca>> > wrote: > Hi folks, > > I guess I have a lot to learn to write functions but I'm stuck when using > rasterEngine. It seems that it should be very easy to do but I'm missing > something, apparently... I have a raster, Safranyik_zones_1961_1990, with > values (integer) from 1 to 5. I would like to create five rasters for which > value will be 1 when the raster Safranyik_zones_1961_1990 is equal to "i", > and NA otherwise. I would like to run everything in a loop . Here's what I > thought would be ok. > > fun_zone <- function(Safranyik_zones,i,...) { > Safranyik_zonesb <- Safranyik_zones > Safranyik_zonesb[] <- NA > Safranyik_zonesb[Safranyik_zones == i] <- 1 > return(Safranyik_zonesb) > } > > for (i in 1:5){ > Safranyik_zones_1961_1990b <- > rasterEngine(Safranyik_zones=Safranyik_zones_1961_1990,i=i, fun=fun_zone) > assign(paste("Safranyik_zones_1961_1990b_",i, > sep=""),Safranyik_zones_1961_1990b[[1]]) > } > > Of course, it says that « i » is missing...: > >>Erreur dans Safranyik_zones == i : 'i' est manquant > > Any help? > > Thanks in advance, > > Yan > > > Yan Boulanger, Chercheur scientifique / Research scientist > Ressources Naturelles Canada, Canadian Forest Service > Centre de Foresterie des Laurentides > 1055, rue du P.E.P.S. > C.P. 10380, succ. Sainte-Foy > Québec (Québec) Canada > G1V 4C7 > Tel. : +1 418 649-6859 > > > > > [[alternative HTML version deleted]] > > > _______________________________________________ > R-sig-Geo mailing list > R-sig-Geo@r-project.org<mailto:R-sig-Geo@r-project.org> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > > > [[alternative HTML version deleted]] > > > _______________________________________________ > R-sig-Geo mailing list > R-sig-Geo@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > -- Alex Zvoleff Postdoctoral Associate Tropical Ecology Assessment and Monitoring (TEAM) Network Conservation International 2011 Crystal Dr. Suite 500, Arlington, Virginia 22202, USA Tel: +1-703-341-2749, Fax: +1-703-979-0953, Skype: azvoleff http://www.teamnetwork.org | http://www.conservation.org _______________________________________________ R-sig-Geo mailing list R-sig-Geo@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-geo