[R] replicate matrix blocks different numbers of times into new matrix
Hi, I am trying to replicate blocks of a matrix (defined by factors) into another matrix, but an unequal, consecutive number of times for each factor. I need to find an elegant and fast way to do this, so loops will not work. An example of what I am trying to do is the following: # the data - first column entries are both data and the two factors x<-matrix(c(1,2,3,4),2,2) > x [,1] [,2] [1,]13 [2,]24 # the number of repetitions of the first and second factor n<-c(1,3) This is what I want as output: [,1] [,2] [1,]13 [2,]24 [3,]24 [4,]24 Any ideas how to get there? I have tried using tapply with combination of rep, but this does not work (I need 1 and then 3 replications). Any help would be great! -Ralph _ With Windows Live for mobile, your contacts travel with you. 072008 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] replicate matrix blocks different numbers of times into new matrix
Actually not quite - my mistake, since I oversimplified the problem I have. Here is a more realistic x matrix (plus some additional information): # the data x<-matrix(c(1,1,2,1,3,4),3,2) # number of factors n_f<-2 # number of rows taken by each factor f_length <- c(2,1) # number of repetitions I want for the first and second factor # actually, always the same factor of expansion: both factors are to be replicated n times n<-c(2,2) I want something like [,1] [,2] [1,] 1 1 [2,] 1 3 [3,] 1 1 [4,] 1 3 [5,] 2 4 [6,] 2 4 but it is only easy to get [,1] [,2] [1,] 1 1 [2,] 1 1 [3,] 1 3 [4,] 1 3 [5,] 2 4 [6,] 2 4 I am not sure about the first target matrix. I could loop through each level of the factor, use a "which(x[,1]==f[k]" to get the row indices for each factor f[k] and then replicate those indices n times and append them to the result for the previous level of the factor. This does not seem efficient, given that I actually have a large matrix with more than 600 factors. Sorry for the initial misspecification - any ideas how I could solve my problem? Best, Ralph > Date: Sat, 19 Jul 2008 21:39:25 +0200 > From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > CC: r-help@r-project.org > Subject: Re: [R] replicate matrix blocks different numbers of times into new > matrix > > Ralph S. wrote: >> Hi, >> >> I am trying to replicate blocks of a matrix (defined by factors) into >> another matrix, but an unequal, consecutive number of times for each factor. >> >> I need to find an elegant and fast way to do this, so loops will not work. >> >> An example of what I am trying to do is the following: >> >> # the data - first column entries are both data and the two factors >> x<-matrix(c(1,2,3,4),2,2) >> >>> x >>> >> [,1] [,2] >> [1,] 1 3 >> [2,] 2 4 >> >> # the number of repetitions of the first and second factor >> n<-c(1,3) >> >> This is what I want as output: >> >> [,1] [,2] >> [1,] 1 3 >> [2,] 2 4 >> [3,] 2 4 >> [4,] 2 4 >> >> >> Any ideas how to get there? I have tried using tapply with combination of >> rep, but this does not work (I need 1 and then 3 replications). >> [[elided Hotmail spam]] >> > Will this do? > > x[rep(1:2,n),] > > -- > O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B > c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K > (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 > ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 > _ _ family_safety_072008 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] replicate matrix blocks different numbers of times into new matrix
yes, this works, thank yo. very smart solution! -R > Date: Sat, 19 Jul 2008 19:55:58 -0400 > From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > Subject: Re: [R] replicate matrix blocks different numbers of times into new > matrix > CC: r-help@r-project.org > > Will this do it: > >> # determine the row numbers of each of the factors >> x.row <- split(seq(nrow(x)), x[,1]) >> # process the data and replicate the rows >> result <- lapply(seq_along(x.row), function(.fact){ > + x[rep(x.row[[.fact]], n[.fact]),] > + }) >> do.call(rbind, result) > [,1] [,2] > [1,]11 > [2,]13 > [3,]11 > [4,]13 > [5,]24 > [6,]24 >> > > > On Sat, Jul 19, 2008 at 7:17 PM, Ralph S. wrote: >> >> Actually not quite - my mistake, since I oversimplified the problem I have. >> >> Here is a more realistic x matrix (plus some additional information): >> >> # the data >> x<-matrix(c(1,1,2,1,3,4),3,2) >> >> # number of factors >> n_f<-2 >> >> # number of rows taken by each factor >> f_length <- c(2,1) >> >> # number of repetitions I want for the first and second factor >> # actually, always the same factor of expansion: both factors are to be >> replicated n times >> n<-c(2,2) >> >> I want something like >> >> [,1] [,2] >> [1,] 1 1 >> [2,] 1 3 >> [3,] 1 1 >> [4,] 1 3 >> [5,] 2 4 >> [6,] 2 4 >> >> but it is only easy to get >> [,1] [,2] >> [1,] 1 1 >> [2,] 1 1 >> [3,] 1 3 >> [4,] 1 3 >> [5,] 2 4 >> [6,] 2 4 >> >> I am not sure about the first target matrix. >> >> I could loop through each level of the factor, use a "which(x[,1]==f[k]" to >> get the row indices for each factor f[k] and then replicate those indices n >> times and append them to the result for the previous level of the factor. >> This does not seem efficient, given that I actually have a large matrix with >> more than 600 factors. >> >> Sorry for the initial misspecification - any ideas how I could solve my >> problem? >> >> Best, >> >> Ralph >> >> >>> Date: Sat, 19 Jul 2008 21:39:25 +0200 >>> From: [EMAIL PROTECTED] >>> To: [EMAIL PROTECTED] >>> CC: r-help@r-project.org >>> Subject: Re: [R] replicate matrix blocks different numbers of times into >>> new matrix >>> >>> Ralph S. wrote: >>>> Hi, >>>> >>>> I am trying to replicate blocks of a matrix (defined by factors) into >>>> another matrix, but an unequal, consecutive number of times for each >>>> factor. >>>> >>>> I need to find an elegant and fast way to do this, so loops will not work. >>>> >>>> An example of what I am trying to do is the following: >>>> >>>> # the data - first column entries are both data and the two factors >>>> x<-matrix(c(1,2,3,4),2,2) >>>> >>>>> x >>>>> >>>> [,1] [,2] >>>> [1,] 1 3 >>>> [2,] 2 4 >>>> >>>> # the number of repetitions of the first and second factor >>>> n<-c(1,3) >>>> >>>> This is what I want as output: >>>> >>>> [,1] [,2] >>>> [1,] 1 3 >>>> [2,] 2 4 >>>> [3,] 2 4 >>>> [4,] 2 4 >>>> >>>> >>>> Any ideas how to get there? I have tried using tapply with combination of >>>> rep, but this does not work (I need 1 and then 3 replications). >>>> >> [[elided Hotmail spam]] >>>> >>> Will this do? >>> >>> x[rep(1:2,n),] >>> >>> -- >>> O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B >>> c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K >>> (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 >>> ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 >>> >> >> _ >> _ >> >> >> family_safety_072008 >> __ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > > > -- > Jim Holtman > Cincinnati, OH > +1 513 646 9390 > > What is the problem you are trying to solve? _ enger2_072008 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Sum efficiently from large matrix according to re-occuring levels of factor?
Hi, I am trying to calculate the sum for each occurrence of the level of a factor in a very large matrix. In addition, I want to save that sum together with the information of the level of the factor and the level of a second factor. My matrix looks like this: x<-matrix(c(1,1,1,2,2,3,3,1,1,7,7,7,4,4,2,2,7,7,1,1,1,1,1,1,2,5,5),9,3) I want to sum according to the levels in the first column and save the sum with the information of the level in the first and the second column in a new matrix. That is, I want output in the matrix of form: 1 7 3 2 4 2 3 2 3 1 7 10 The important thing is, that a factor level such as "1" in the example can re-occur many times. There are no regularities on the number of re-occurences etc. How could I do this efficiently (the matrix is large:>10^6 rows)? Many thanks!! -Ralph _ [[elided Hotmail spam]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Sum efficiently from large matrix according to re-occuring levels of factor?
The first and second column are actually indices of another matrix (my example may make this not sufficiently clear). I want to compare the sum with that corresponding entry, and then record the result of that. Any idea? Best, Ralph > Date: Sun, 20 Jul 2008 16:50:41 -0700 > From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > Subject: Re: [R] Sum efficiently from large matrix according to re-occuring > levels of factor? > CC: r-help@r-project.org > > On Sun, Jul 20, 2008 at 4:47 PM, hadley wickham wrote: >> On Sun, Jul 20, 2008 at 4:16 PM, Ralph S. wrote: >>> >>> Hi, >>> >>> I am trying to calculate the sum for each occurrence of the level of a >>> factor in a very large matrix. In addition, I want to save that sum >>> together with the information of the level of the factor and the level of a >>> second factor. >>> >>> My matrix looks like this: >>> >>> x<-matrix(c(1,1,1,2,2,3,3,1,1,7,7,7,4,4,2,2,7,7,1,1,1,1,1,1,2,5,5),9,3) >>> >>> I want to sum according to the levels in the first column and save the sum >>> with the information of the level in the first and the second column in a >>> new matrix. >>> >>> That is, I want output in the matrix of form: >>> >>> 1 7 3 >>> 2 4 2 >>> 3 2 3 >>> 1 7 10 >>> >> >> Why that and not: >> >> 1 7 13 >> 2 4 2 >> 3 2 3 >> >> ? > > Here's a solution for that case: > > index <- x[, 2] + x[, 1] * max(x[, 2]) > cbind(x[!duplicated(index), 1:2], tapply(x[, 3], index, sum)) > > It takes about half a second for a million row matrix. > > Hadley > > > > -- > http://had.co.nz/ _ With Windows Live for mobile, your contacts travel with you. 072008 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Sum efficiently from large matrix according to re-occuring levels of factor?
yes - thank you very much! slowly getting to the full power of R . . . > Date: Sun, 20 Jul 2008 21:21:35 -0400 > From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > Subject: Re: [R] Sum efficiently from large matrix according to re-occuring > levels of factor? > CC: [EMAIL PROTECTED]; r-help@r-project.org > > Does this do what you want: > >> # following up on another idea that was presented >> # where are the breaks >> dataBreaks <- cumsum(c(0, (diff(x[, 2] + x[, 1] * max(x[, 2])) != 0))) >> # sum up column 3 and output the first two columns with the indices >> result <- lapply(split(seq(nrow(x)), dataBreaks), function(.sect){ > + c(x[.sect[1], 1:2], sum(x[.sect, 3])) > + }) >> do.call(rbind, result) > [,1] [,2] [,3] > 0173 > 124 2 > 2323 > 317 10 > > > On Sun, Jul 20, 2008 at 7:57 PM, Ralph S. wrote: >> >> The first and second column are actually indices of another matrix (my >> example may make this not sufficiently clear). I want to compare the sum >> with that corresponding entry, and then record the result of that. >> >> Any idea? >> >> Best, >> >> Ralph >> >> >> >> >>> Date: Sun, 20 Jul 2008 16:50:41 -0700 >>> From: [EMAIL PROTECTED] >>> To: [EMAIL PROTECTED] >>> Subject: Re: [R] Sum efficiently from large matrix according to re-occuring >>> levels of factor? >>> CC: r-help@r-project.org >>> >>> On Sun, Jul 20, 2008 at 4:47 PM, hadley wickham wrote: >>>> On Sun, Jul 20, 2008 at 4:16 PM, Ralph S. wrote: >>>>> >>>>> Hi, >>>>> >>>>> I am trying to calculate the sum for each occurrence of the level of a >>>>> factor in a very large matrix. In addition, I want to save that sum >>>>> together with the information of the level of the factor and the level of >>>>> a second factor. >>>>> >>>>> My matrix looks like this: >>>>> >>>>> x<-matrix(c(1,1,1,2,2,3,3,1,1,7,7,7,4,4,2,2,7,7,1,1,1,1,1,1,2,5,5),9,3) >>>>> >>>>> I want to sum according to the levels in the first column and save the >>>>> sum with the information of the level in the first and the second column >>>>> in a new matrix. >>>>> >>>>> That is, I want output in the matrix of form: >>>>> >>>>> 1 7 3 >>>>> 2 4 2 >>>>> 3 2 3 >>>>> 1 7 10 >>>>> >>>> >>>> Why that and not: >>>> >>>> 1 7 13 >>>> 2 4 2 >>>> 3 2 3 >>>> >>>> ? >>> >>> Here's a solution for that case: >>> >>> index <- x[, 2] + x[, 1] * max(x[, 2]) >>> cbind(x[!duplicated(index), 1:2], tapply(x[, 3], index, sum)) >>> >>> It takes about half a second for a million row matrix. >>> >>> Hadley >>> >>> >>> >>> -- >>> http://had.co.nz/ >> >> _ >> With Windows Live for mobile, your contacts travel with you. >> >> 072008 >> __ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > > > > -- > Jim Holtman > Cincinnati, OH > +1 513 646 9390 > > What is the problem you are trying to solve? _ _WL_Refresh_messenger_video_072008 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Cannot re-start R after bus error
Hi, I ran a program that was exhausting the (unix) server's available memory. I got a bus error and then chose to save and quit workspace (option 4). That may not have been a smart idea - when I try to use R now, I get the error message: Error in load(name, envir = .GlobalEnv) : ReadItem: unknown type 0, perhaps written by later version of R Fatal error: unable to restore saved data in .RData I deleted my temporary R directories but still get the same message. What can I do to change this? Best, Ralph _ family_safety_072008 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Cannot re-start R after bus error
thank you for the prompt help - found the file it was in a different directory R > Date: Tue, 22 Jul 2008 16:13:19 -0500 > From: [EMAIL PROTECTED] > Subject: RE: [R] Cannot re-start R after bus error > To: [EMAIL PROTECTED] > > i realize you said that deleted your temporary R directories but did > you delete that .RData file. It's trying to load it during startup and > it can't > so it's probably there somewhere I would think ? Hopefully someone more > knowledgable than me on these things will reply. > > > > On Tue, Jul 22, 2008 at 4:22 PM, Ralph S. wrote: > >> Hi, >> >> I ran a program that was exhausting the (unix) server's available >> memory. I got a bus error and then chose to save and quit workspace >> (option 4). >> >> That may not have been a smart idea - when I try to use R now, I get >> the error message: >> >> Error in load(name, envir = .GlobalEnv) : ReadItem: unknown type 0, >> perhaps written by later version of R >> Fatal error: unable to restore saved data in .RData >> >> I deleted my temporary R directories but still get the same message. >> >> What can I do to change this? >> >> Best, >> >> Ralph >> >> _ >> >> >> family_safety_072008 >> [[alternative HTML version deleted]] >> >> __ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. _ _WL_Refresh_messenger_video_072008 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Eaxct position of specific elements in array
Hi, I am trying to get the positions in array coordinates (needed later) of certain elements in an array but I am not sure how to get them. My array is Q and the condition is dt>dV, where dt and dV are arrays of exactly the same dimensions as Q. I know that I can extract the elements of Q by using Q[dt>dV] but I need the array position of where this happens. I don't see how to use the which() command since some elements given by that the condition are not unique ( I tried which(Q==Q[dt>dV] ,arr.ind=TRUE) but that gives me too many positions plus Warning message: In Q == Q[dt > dV] : longer object length is not a multiple of shorter object length) Any help would be really appreciated! -Ralph _ [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] subsetting matrix according to columns with character index
Hi, I have a long matrix of the following form which I would like to subset according to the third column: [x y z]: a1 c1 1 a1 c1 2 a2 c1 1 a1 c2 1 a1 c2 2 . . . The first two columns a characters ai and cj. I would like to keep all the rows where there are two entries for z, 1 and 2. That is, I want: a1 c1 1 a1 c1 2 a1 c2 1 a1 c2 2 . . . I try to use something like df[by(df,c(df$x,df$y),sum(z)==3),] but that only gives me one line of data per x y combination. Is there an easy way of coding to keep all rows for a and c combinations where z has entries both 1 and 2? Many thanks, Ralph _ LM_WLYIA_whichathlete_us __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] subsetting matrix according to columns with character index
I tried this - I get an empty set: <0 rows> (or 0-length row.names) I guess this happens because the z variable takes only one value per row?? What works is: DFsub<-DF[DF$z == 1 | DF$z == 2,] but then, I do not eliminate the entries where there is only one entry for z given an a and c combination. Any idea what to do? -Ralph > Date: Wed, 13 Aug 2008 13:05:25 -0500 > From: [EMAIL PROTECTED] > Subject: RE: [R] subsetting matrix according to columns with character index > To: [EMAIL PROTECTED] > > it must be a dataframe so, if it was DF, then, assuming i understand > what you want then either of the following should work: > > DFsub<-DF[DF$z == 1 & DF$z == 2,] > > or > > DFsub<-subset(DF, z == 1 & z == 2 ) > > > On Wed, Aug 13, 2008 at 2:00 PM, Ralph S. wrote: > > > Hi, > > > > I have a long matrix of the following form which I would like to > > subset according to the third column: > > > > [x y z]: > > > > a1 c1 1 > > a1 c1 2 > > a2 c1 1 > > a1 c2 1 > > a1 c2 2 > > . . . > > > > > > The first two columns a characters ai and cj. > > > > I would like to keep all the rows where there are two entries for z, 1 > > and 2. > > > > That is, I want: > > a1 c1 1 > > a1 c1 2 > > a1 c2 1 > > a1 c2 2 > > . . . > > > > I try to use something like df[by(df,c(df$x,df$y),sum(z)==3),] but > > that only gives me one line of data per x y combination. > > > > Is there an easy way of coding to keep all rows for a and c > > combinations where z has entries both 1 and 2? > > Many thanks, > > > > Ralph > > > > _ > > > > > > LM_WLYIA_whichathlete_us > > __ > > R-help@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. _ [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] algorithm to create unique identifiers
Hi all, I am trying to create a unique identifier for each row, combining numbers from three columns. Do you know if there is a general formula to do this (or some manual where I can read about this)? I figure I can use the numeric entries of the columns as "coordinates" and multiply them with different coefficients (different magnitudes) to get the unique ID - but it would be nice to read about such algorithms in general. Any links/input would be great - Ralph _ e. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Cointegration no constant
Hi, I am trying to estimate a VECM without constant using the following code: data(finland) sjf <- finland sjf.reg<-ca.jo(sjf, type = c("eigen"), ecdet = c("none"), K = 2,spec=c("transitory"), season = NULL, dumvar = NULL) cajools(sjf.reg) While the cointegration test does not use a constant, it is used in the cajools which I do not want. I am sure I am doing something wrong - what should I change? Any help very much appreciated! Ralph _ Need to know the score, the latest news, or you need your Hotmail®-get your "fix". __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.