My apologies, I forgot to CC: to the list on my previous communication with Daniel.
Jesse ----- Forwarded by Jesse Albert Canchola/EMVL/DIAG/US/BAYER on 08/21/2006 10:50 AM ----- Jesse Albert Canchola/EMVL/DIAG/US/BAYER 08/21/2006 09:36 AM To "Daniel Nordlund" <[EMAIL PROTECTED]> cc Subject RE: [R] Permutations with replacement Thanks, Daniel. I need to enumerate all possibilities of 8^8 and take a random sample of 10,000 from there. Then I will use the sampled possibilities to do a combination of data frames/files then do some math on those files and develop the probability distribution from resulting sampling statistics (I couldn't get the available bootstrap packages to do what I want). BTW, the preferred solution (however inelegant) is reprinted below. I did have a memory problem with the hypercube for 8^8 so I did an 8^7 hypercube that worked and concatenated a 1-8 to the 8^7 matrix that resulted in 8 large matrices which I attempted rbind together to create the 8^8 but ran into more memory problems (on matrix number 7 of 8- it's a Windows problem - I used the --max-mem-size=2G - to no avail). The final solution was to take a random sample of 10,000/8=1250 from each of the files (doing four of the 8 files at a time, and for which I permuted the rows to make it more random), removed the heavy-laden files then rbind 'ed the smaller sampled files together to make the 10,000. Here is the final final code: # IDEA: We cannot simply do a permutation of the 8 classes/file id's since this will not allow/simulate repeats of numbers # as in a bootstrap (e.g., for 3 items - 1,2,3 - we would also want the possibility 1,1,1 or 2,2,2 or 3,3,3 etc. # The 8!=40,320 permutations with replacement would become an 8^8=16,777,216 so we would want to take a random # sample of 10,000 from the posibilities library(combinat) # THIS WILL NOT WORK DUE TO THE LIMITATIONS OF WINDOWS MEMORY (PROBLEMATIC AS IN THE FAQs) SO WE WILL # USE A WORKAROUND # WORKAROUND: 1) Construct the 8 to the 7 power hypercube. 2) For these data, create eight additional data sets that include # the last position to finish the construction of an 8^8 hypercube #x <- rep(8,8) # for partitions of 8 units into classes {1,2,3,4,5,6,7,8} #hcube8 <- hcube(x, scale=1, transl=0) #hcube8 #step 1: 8^7 = 2,097,152 x1 <- rep(8,7) x1 hcube87 <- hcube(x1, scale=1, transl=0) #this will generate 2,097,152 results from 1-8 but for only 7 positions #step 2: column bind each file with 1-8 in the 8th position #x1a <- cbind(hcube87,1) #x2a <- cbind(hcube87,2) #x3a <- cbind(hcube87,3) #x4a <- cbind(hcube87,4) #x5a <- cbind(hcube87,5) #x6a <- cbind(hcube87,6) #x7a <- cbind(hcube87,7) #x8a <- cbind(hcube87,8) #turns out this method also chokes with the memory limitations # Step 2 will be modified as follows: # Step 2a: as before; Step2b: Sample 1/8 from each piece and after every 4 processes, delete the objects to allow for the rest. #Step 2a x1a <- cbind(hcube87,1) x1a <- x1a[sample(1:2097152),] #randomly permute the rows for more randomness x2a <- cbind(hcube87,2) x2a <- x2a[sample(1:2097152),] #randomly permute the rows for more randomness x3a <- cbind(hcube87,3) x3a <- x3a[sample(1:2097152),] #randomly permute the rows for more randomness x4a <- cbind(hcube87,4) x4a <- x4a[sample(1:2097152),] #randomly permute the rows for more randomness x1b <- x1a[sample(1:1250,replace=FALSE),] x2b <- x2a[sample(1:1250,replace=FALSE),] x3b <- x3a[sample(1:1250,replace=FALSE),] x4b <- x4a[sample(1:1250,replace=FALSE),] rm(x1a,x2a,x3a,x4a) #remove the big files x5a <- cbind(hcube87,5) x5a <- x5a[sample(1:2097152),] #randomly permute the rows for more randomness x6a <- cbind(hcube87,6) x6a <- x6a[sample(1:2097152),] #randomly permute the rows for more randomness x7a <- cbind(hcube87,7) x7a <- x7a[sample(1:2097152),] #randomly permute the rows for more randomness x8a <- cbind(hcube87,8) x8a <- x8a[sample(1:2097152),] #randomly permute the rows for more randomness x5b <- x5a[sample(1:1250,replace=FALSE),] x6b <- x6a[sample(1:1250,replace=FALSE),] x7b <- x7a[sample(1:1250,replace=FALSE),] x8b <- x8a[sample(1:1250,replace=FALSE),] rm(x5a,x6a,x7a,x8a) #remove the big files #Step 3: combine all the randomly sampled files m <- rbind(x1b,x2b,x3b,x4b,x5b,x6b,x7b,x8b) # NOTE: each number in the matrix represents a file "name" from 1-8. # the first pointer should be numeric and then subsequent as character # since the first time you assign a number to a character in a matrix # the rest of the numbers in the matrix are coerced to character m[m==1]='a'; m[m=='2']='b'; m[m=='3']='c' ; m[m=='4']='d'; m[m=='5']='e' ; m[m=='6']='f'; m[m=='7']='g' ; m[m=='8']='h' m ########### end R code ############ Thanks, David. That worked fabulously! Here is the R code for the hypercube test example: ########## begin R code ############ library(combinat) x <- rep(3,3) # for partitions of 3 units into the three classes {1,2,3} hcube(x, scale=1, transl=0) ########### end R code ############ For the larger one I want (i.e., 8^8), I will take a random sample of 10,000 from the 16,777,216 possibilities. Regards, Jesse Canchola <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 08/18/2006 01:33 PM To "Jesse Albert Canchola" <[EMAIL PROTECTED]>, "r-help" <r-help@stat.math.ethz.ch> cc Subject Re: [R] Permutations with replacement If you also want 1,1,1 and so on, the number of these is n^n, (n choices for each of n slots.) In that case, you could use hcube from combinat. David L. Reiner Rho Trading Securities, LLC Chicago IL 60605 "Daniel Nordlund" <[EMAIL PROTECTED]> 08/18/2006 05:16 PM To "'Jesse Albert Canchola'" <[EMAIL PROTECTED]>, "'r-help'" <r-help@stat.math.ethz.ch> cc Subject RE: [R] Permutations with replacement > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > On Behalf Of Jesse Albert Canchola > Sent: Friday, August 18, 2006 1:02 PM > To: r-help > Subject: [R] Permutations with replacement > > Is there a simple function or process that will create permutations with > replacement? > > I know that using the combinat package > > ###### begin R code ###### > > library(combinat) > > m <- t(array(unlist(permn(3)), dim = c(3, 6))) > > # we can get the permutations, for example 3!=6 > # gives us > > > m > [,1] [,2] [,3] > [1,] 1 2 3 > [2,] 1 3 2 > [3,] 3 1 2 > [4,] 3 2 1 > [5,] 2 3 1 > [6,] 2 1 3 > ###### end R code ########## > > I'd like to include the "with replacement possibilities" such as > > 1,1,3 > 1,1,2 > 2,3,3 > Isn't what you want just sampling with replacement? x <- c(1,2,3) sample(x,3,replace=TRUE) Hope this is helpful, Dan Dan Nordlund Bothell, WA USA _______________________________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.