Hi Roy,
R (usually) makes a copy if the dimensionality of an array is modified,
even if you use this syntax:
x <- array(1:24, c(2, 3, 4))
dim(x) <- c(6, 4)
See also ?tracemem, ?data.table::address, ?pryr::address and other tools
to trace if an internal copy is done.
Workaround: use data.table::setattr or bit::setattr to modify the
dimensions in place (i.e., without making a copy). Risk: if you modify
an object by reference, all other objects which point to the same memory
address will be modified silently, too.
HTH,
Denes
On 03/18/2016 10:28 PM, Roy Mendelssohn - NOAA Federal wrote:
Hi All:
I am working with a very large array. if noLat is the number of latitudes,
noLon the number of longitudes and noTime the number of time periods, the
array is of the form:
myData[noLat, no Lon, noTime].
It is read in this way because that is how it is stored in a (series) of netcdf
files. For the analysis I need to do, I need instead the array:
myData[noLat*noLon, noTime]. Normally this would be easy:
myData<- array(myData,dim=c(noLat*noLon,noTime))
My question is how does this command work in R - does it make a copy of the
existing array, with different indices for the dimensions, or does it just redo
the indices and leave the given array as is? The reason for this question is
my array is 30GB in memory, and I don’t have enough space to have a copy of the
array in memory. If the latter I will have to figure out a work around to
bring in only part of the data at a time and put it into the proper locations.
Thanks,
-Roy
**********************
"The contents of this message do not reflect any position of the U.S. Government or
NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new address and phone***
110 Shaffer Road
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: roy.mendelss...@noaa.gov www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
