I would like to store a big spatial weight matrix in R memory to do more
calculation. I know there are memory issue for 32 bit computer and I have tried
increasing the memory to maximum without success. I am using R version 2.15.2
and window vista. The data has about 15,000 observations and I would like to
create and store a distance based Spatial Weight Matrix in R memory. A
reproducible codes are as follows and for both codes produce out of memory
error. The codes works to up 5000 observations.
Based on these codes, (for those with experiece in big data)is it possible to
use ffbase or bigmemory packages to achieve the objective (keep the matrix in
memory)? I will appreciate any help.
#Remove all objects
rm(list=ls())
library(spdep)
library(maptools)
library(GeoXp)
library(ffbase)
library(bigmemory)
##########################################################################
#Testing using fake data
n = 15000
data = data.frame(n1=1:n)
data$SEQN <- seq(1, n,1)
data$LAT <- runif(n, 29.0000, 32.0000)
data$LON <- runif(n, -100.0000, -89.0000)
######################################################################3
#Create distance based spatial weight matrix using dist function
#using euclidean method
coords <- cbind(data$LON, data$LAT)
SWM <- as.matrix(dist(coords, method = "euclidean", upper=TRUE))
#Other data management
data1 <- cbind(data, SWM)
#Alternatively
#Create spatial weight matrix based on distance between Points
#using the spDistsN1 function and produce the same results as above
#Create coordinates
coordinates(data) <- ~LON+LAT
#Function to calculate the distance matrix
DistMatrix = function(obj, longlat = FALSE) {
return(sapply(1:length(obj[[1]]), function(x) spDistsN1(coordinates(obj),
coordinates(obj[x,]), longlat)))
}
SWMD <- big.matrix(DistMatrix(data))
data2 <- cbind(data, SWMD)
With thanks
Peter Maclean
Department of Economics
UDSM
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
