Re: [R] Using apply

[Bert Gunter]

Indeed.

But perhaps it's also worth noting that if such statistics are calculated
as implementations of (e.g. anova) formulae still found (sadly) in many
statistics texts, then they shouldn't be calculated at all. Rather, the
appropriate matrix methods (e.g. QR decompositions ) built into R -- many
of which are already incorporated into R's statistical corpus -- should be
used. To say more would of course be far O/T.

Cheers,
Bert

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Oct 30, 2018 at 8:44 PM Peter Langfelder 
wrote:

> It should be said that for many basic statistics, there are faster
> functions than apply, for example here you want
>
> sum = colSums(x)
>
> As already said, for sum of squares you would do colSums(x^2).
>
> Many useful functions of this kind are implemented in package
> matrixStats. Once you install it, either look at the package manual or
> type ls("package:matrixStats") to see a list of functions. Most if not
> all have self-explanatory names.
>
> HTH,
>
> Peter
> On Tue, Oct 30, 2018 at 7:28 PM Steven Yen  wrote:
> >
> > I need help with "apply". Below, I have no problem getting the column
> sums.
> > 1. How do I get the sum of squares?
> > 2. In general, where do I look up these functions?
> > Thanks.
> >
> > x<-matrix(1:10,nrow=5); x
> > sum <- apply(x,2,sum); sum
> >
> >
> >
> >
> > [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply

[Peter Langfelder]

It should be said that for many basic statistics, there are faster
functions than apply, for example here you want

sum = colSums(x)

As already said, for sum of squares you would do colSums(x^2).

Many useful functions of this kind are implemented in package
matrixStats. Once you install it, either look at the package manual or
type ls("package:matrixStats") to see a list of functions. Most if not
all have self-explanatory names.

HTH,

Peter
On Tue, Oct 30, 2018 at 7:28 PM Steven Yen  wrote:
>
> I need help with "apply". Below, I have no problem getting the column sums.
> 1. How do I get the sum of squares?
> 2. In general, where do I look up these functions?
> Thanks.
>
> x<-matrix(1:10,nrow=5); x
> sum <- apply(x,2,sum); sum
>
>
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply

[jim holtman]

> s2 <- apply(x*x, 2, sum)
> s2
[1]  55 330

Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Tue, Oct 30, 2018 at 10:28 PM Steven Yen  wrote:
>
> I need help with "apply". Below, I have no problem getting the column sums.
> 1. How do I get the sum of squares?
> 2. In general, where do I look up these functions?
> Thanks.
>
> x<-matrix(1:10,nrow=5); x
> sum <- apply(x,2,sum); sum
>
>
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply

[Steven Yen]

I need help with "apply". Below, I have no problem getting the column sums.
1. How do I get the sum of squares?
2. In general, where do I look up these functions?
Thanks.

x<-matrix(1:10,nrow=5); x
sum <- apply(x,2,sum); sum




[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function to merge list of data frames

[Jeff Newmiller]

Er, rbind is not merge... do.call expects the function you specify to handle 
all the elements of the list in a single invocation... Reduce will work with a 
two-argument function.

Reduce(merge, df.list, accumulate=TRUE, by='date')

For clarity: apply and the like have for loops inside them, so the primary 
benefit is a compact and easy to read invocation.

Do not assume that this syntax will have an appreciably-different performance 
behavior than the for loop solution. In particular, merge is a potentially very 
slow operation so if your real data frames have the identical key like your 
example does, using cbind could help performance significantly. Also, both your 
for loop and Reduce allocate memory as needed, leading to potential memory 
thrashing that could be a problem for large data sets. If this is an issue for 
you then you might want to roll your own preallocating for loop or use a 
function like bind_cols that has that feature [1][2]

[1] http://r4ds.had.co.nz/iteration.html
[2] https://dplyr.tidyverse.org/reference/bind.html


On July 27, 2018 4:45:31 AM PDT, S Ellison  wrote:
>Short answer: do.call()
>
>do.call("rbind", df.list)
>will rbind all of the data frames in df.list.
>
>You may have to tidy up row names afterwards, and you will need to make
>sure that the data frames all have the same column names and each
>column has the same class, or you'll get unexpected results.
>
>S Ellison
>
>> -Original Message-
>> From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of
>Naresh
>> Gurbuxani
>> Sent: 25 July 2018 07:17
>> To: R-help@r-project.org
>> Subject: [R] Using apply function to merge list of data frames
>> 
>> I have a list whose components are data frames.  My goal is to
>construct a
>> data frame by merging all the list components.  Is it possible to
>achieve this
>> using apply and without a for loop, as used below?
>> 
>> Thanks,
>> Naresh
>> 
>> mylist <- list(A = data.frame(date = seq.Date(as.Date('2018-01-01'),
>by =
>> 'week',
>>   length.out = 5), ret = rnorm(5)),
>>B = data.frame(date = seq.Date(as.Date('2018-01-01'),
>by = 'week',
>>   length.out = 5), ret = rnorm(5)))
>> 
>> mydf <- data.frame(date = seq.Date(as.Date('2018-01-01'), by =
>'week',
>> length.out = 5))
>> 
>> for(ch in names(mylist)){
>> tempdf <- mylist[[ch]]
>> names(tempdf)[2] <- paste(names(tempdf)[2], ch, sep = '.')
>> mydf <- merge(mydf, tempdf, by = c('date'))}
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-
>> guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
>
>***
>This email and any attachments are confidential. Any
>use...{{dropped:8}}
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

-- 
Sent from my phone. Please excuse my brevity.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function to merge list of data frames

[S Ellison]

Short answer: do.call()

do.call("rbind", df.list)
will rbind all of the data frames in df.list.

You may have to tidy up row names afterwards, and you will need to make sure 
that the data frames all have the same column names and each column has the 
same class, or you'll get unexpected results.

S Ellison

> -Original Message-
> From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Naresh
> Gurbuxani
> Sent: 25 July 2018 07:17
> To: R-help@r-project.org
> Subject: [R] Using apply function to merge list of data frames
> 
> I have a list whose components are data frames.  My goal is to construct a
> data frame by merging all the list components.  Is it possible to achieve this
> using apply and without a for loop, as used below?
> 
> Thanks,
> Naresh
> 
> mylist <- list(A = data.frame(date = seq.Date(as.Date('2018-01-01'), by =
> 'week',
>   length.out = 5), ret = rnorm(5)),
>B = data.frame(date = seq.Date(as.Date('2018-01-01'), by = 
> 'week',
>   length.out = 5), ret = rnorm(5)))
> 
> mydf <- data.frame(date = seq.Date(as.Date('2018-01-01'), by = 'week',
> length.out = 5))
> 
> for(ch in names(mylist)){
> tempdf <- mylist[[ch]]
> names(tempdf)[2] <- paste(names(tempdf)[2], ch, sep = '.')
> mydf <- merge(mydf, tempdf, by = c('date'))}
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.


***
This email and any attachments are confidential. Any use...{{dropped:8}}

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function to merge list of data frames

[Berend Hasselman]

> On 25 Jul 2018, at 08:17, Naresh Gurbuxani  
> wrote:
> 
> I have a list whose components are data frames.  My goal is to construct a 
> data frame by merging all the list components.  Is it possible to achieve 
> this using apply and without a for loop, as used below?
> 
> Thanks,
> Naresh
> 
> mylist <- list(A = data.frame(date = seq.Date(as.Date('2018-01-01'), by = 
> 'week',
>  length.out = 5), ret = rnorm(5)),
>   B = data.frame(date = seq.Date(as.Date('2018-01-01'), by = 
> 'week',
>  length.out = 5), ret = rnorm(5)))
> 
> mydf <- data.frame(date = seq.Date(as.Date('2018-01-01'), by = 'week', 
> length.out = 5))
> 
> for(ch in names(mylist)){
>tempdf <- mylist[[ch]]
>names(tempdf)[2] <- paste(names(tempdf)[2], ch, sep = '.')
>mydf <- merge(mydf, tempdf, by = c('date'))}
> _

See if these would help:

on R-help the thread

https://stat.ethz.ch/pipermail/r-help/2018-May/454249.html

and 

https://stackoverflow.com/questions/4512465/what-is-the-most-efficient-way-to-cast-a-list-as-a-data-frame?rq=1

Berend

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply

[David L Carlson]

This might be faster. It uses apply() which is just another way of constructing 
a loop so it is not necessarily faster. We can vectorize the logical 
comparisons, but all() is not vectorized. Use dput() to share your data. That 
makes it easier to replicate your example:

a <- structure(list(V1. = c(1, 1, 1, 0), V2.x = c(1, 0, 1, 0), V3.x = c(0, 
 1, 1, 0), V1.y = c(1, 1, 1, 1), V2.y = c(0, 0, 0, 0), V3.y = c(1, 1, 
 1, 1)), .Names = c("V1.x", "V2.x", "V3.x", "V1.y", "V2.y", "V3.y"),
 row.names = c(NA, -4L), class = "data.frame")

b <- structure(list(V1 = c(1, 1), V2 = c(0, 0), V3 = c(1, 1), V4 = c(1, 
 0), V5 = c(0, 0), V6 = c(0, 0)), .Names = c("V1", "V2", "V3", 
 "V4", "V5", "V6"), row.names = c(NA, -2L), class = "data.frame")

# Generate the row indices that we need so we can vectorize the logical 
operation:
idxa <- rep(1:4, each=2)
idxb <- rep(1:2, 4)
ab <- (a[idxa, ] & b[idxb, ]) == b[idxb, ]
c <- cbind(idxa, idxb)[apply(ab, 1, all), ]
c
#  idxa idxb
# [1,]21
# [2,]22
# [3,]31
# [4,]32


David L Carlson
Department of Anthropology
Texas A University
College Station, TX 77843-4352




-Original Message-
From: R-help <r-help-boun...@r-project.org> On Behalf Of Ulrik Stervbo
Sent: Wednesday, May 2, 2018 3:49 PM
To: Neha Aggarwal <aggarwalneha2...@gmail.com>
Cc: r-help@r-project.org
Subject: Re: [R] using apply

Hi Neha,

Perhaps merge() from base or join from dplyr is what you are looking for.
data. table could also be interesting.

Hth
Ulrik

On Wed, 2 May 2018, 21:28 Neha Aggarwal, <aggarwalneha2...@gmail.com> wrote:

>  Hi
>
> I have 3 dataframes, a,b,c with 0/1 values...i have to check a 
> condition for dataframe a and b and then input the rows ids to 
> datframe c . In the if condition, I AND the 2 rows of from a and b and 
> then see if the result is equal to one of them.
> I have done this using a for loop, however, it takes a long time to 
> execute with larger dataset..Can you help me do it using apply 
> function so that i can do it faster?
>
> a
>   V1.x V2.x V3.x V1.y V2.y V3.y
> 1110101
> 2101101
> 3111101
> 4000101
>
> b
>   V1 V2 V3 V4 V5 V6
> 1  1  0  1  1  0  0
> 2  1  0  1  0  0  0
>
> c
>  xy
> 1  21
> 2  22
> 3  31
> 4  32
>
> for(i in 1:nrow(a)){
>   for(j in 1:nrow(b)){
> if(all((a[i,][j,])==b[j,]))
> { c[nrow(c)+1, ]<-c(paste(i,j)
>   }
>   }
> }
>
>
> Thanks,
> Neha
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see 
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see 
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply

[Ulrik Stervbo]

Hi Neha,

Perhaps merge() from base or join from dplyr is what you are looking for.
data. table could also be interesting.

Hth
Ulrik

On Wed, 2 May 2018, 21:28 Neha Aggarwal,  wrote:

>  Hi
>
> I have 3 dataframes, a,b,c with 0/1 values...i have to check a condition
> for dataframe a and b and then input the rows ids to datframe c . In the if
> condition, I AND the 2 rows of from a and b and then see if the result is
> equal to one of them.
> I have done this using a for loop, however, it takes a long time to execute
> with larger dataset..Can you help me do it using apply function so that i
> can do it faster?
>
> a
>   V1.x V2.x V3.x V1.y V2.y V3.y
> 1110101
> 2101101
> 3111101
> 4000101
>
> b
>   V1 V2 V3 V4 V5 V6
> 1  1  0  1  1  0  0
> 2  1  0  1  0  0  0
>
> c
>  xy
> 1  21
> 2  22
> 3  31
> 4  32
>
> for(i in 1:nrow(a)){
>   for(j in 1:nrow(b)){
> if(all((a[i,][j,])==b[j,]))
> { c[nrow(c)+1, ]<-c(paste(i,j)
>   }
>   }
> }
>
>
> Thanks,
> Neha
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] using apply

[Neha Aggarwal]

 Hi

I have 3 dataframes, a,b,c with 0/1 values...i have to check a condition
for dataframe a and b and then input the rows ids to datframe c . In the if
condition, I AND the 2 rows of from a and b and then see if the result is
equal to one of them.
I have done this using a for loop, however, it takes a long time to execute
with larger dataset..Can you help me do it using apply function so that i
can do it faster?

a
  V1.x V2.x V3.x V1.y V2.y V3.y
1110101
2101101
3111101
4000101

b
  V1 V2 V3 V4 V5 V6
1  1  0  1  1  0  0
2  1  0  1  0  0  0

c
 xy
1  21
2  22
3  31
4  32

for(i in 1:nrow(a)){
  for(j in 1:nrow(b)){
if(all((a[i,][j,])==b[j,]))
{ c[nrow(c)+1, ]<-c(paste(i,j)
  }
  }
}


Thanks,
Neha

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply on a three dimensional matrix and passing multiple arguments to user defined function

[Justin Peter]

Dear Liusfo and Jean,

Thank you both for your help and suggestions, which both work.

As Jean mentioned, there is no real speed up using the apply family, which I 
thought there would be, so I will stick with the for loop for clarity.

Liusfo, the reason I used c(1,2) in the apply function (i.e. mask_data <- 
apply(data,c(1,2),mask,y=lsmask)) was because this is what you would do if you 
wanted to sum all the two-dimensional vectors over time (for instance to 
produce an average of a field over a year).

i.e. to get the sum you would do sum_data <- apply(data,c(1,2),sum)

I thought it would extend to using my mask function.

Anyway, thanks again for both of your help.

Cheers,
Justin
--
Justin Peter
Research Fellow
International Centre for Applied Climate Sciences,
University of Southern Queensland
West St, Toowoomba, QLD, 4350
Australia

Email: justin.pe...@usq.edu.au<mailto:justin.pe...@usq.edu.au>
Ph: +61 (0) 7 4631 1181
Fax: +61 (0) 7 4631 5581
Mob: +61 (0)474 774 107


-Original Message-
From: Luisfo <luisf...@yahoo.es<mailto:luisfo%20%3cluisf...@yahoo.es%3e>>
To: "Adams, Jean" 
<jvad...@usgs.gov<mailto:%22Adams,%20jean%22%20%3cjvad...@usgs.gov%3e>>, Justin 
Peter 
<justin.pe...@usq.edu.au<mailto:justin%20peter%20%3cjustin.pe...@usq.edu.au%3e>>
CC: r-help@r-project.org 
<r-help@r-project.org<mailto:%22r-h...@r-project.org%22%20%3cr-h...@r-project.org%3e>>
Subject: Re: [R] Using apply on a three dimensional matrix and passing multiple 
arguments to user defined function
Date: Wed, 7 Sep 2016 16:05:48 +0200

Hi,

Jean's example with lapply works fine.

However, if you still want to use apply, I think this works.
One observation first. You were passing c(1,2) as second argument to apply, in 
your code. And that is what makes you have lots of NAs as a result, since your 
function is being applied twice, by rows and columns (first and second 
dimensions) respectively.
Use:
masked_data <- apply(data,3,mask,y=lsmask)
# but now masked_data has dim(nlon*nlat,ntime), so change it
dim(masked_data) <- dim(data)

The apply goes over the third dimension (second parameter '3'), so it takes 
every nlot*nlat matrix as first argument for function mask.
I think it should work.

Regards,

Luisfo Chiroque
PhD Student | PhD Candidate
IMDEA Networks Institute
http://fourier.networks.imdea.org/people/~luis_nunez/<http://fourier.networks.imdea.org/people/%7Eluis_nunez/>


On 09/07/2016 03:17 PM, Adams, Jean wrote:



Justin,

I don't think you can get the apply() function to return an array.  You
could use lapply() instead, and then use simplify2array() to convert the
list of matrices to an array.  Also, in your mask() function you don't need
the which() and you should return the x.  See my example with toy data
below.

# toy data
nlon <- 2
nlat <- 4
ntime <- 3
data <- array(1:(nlon*nlat*ntime), dim=c(nlon, nlat, ntime))
lsmask <- array(sample(0:1, size=nlon*nlat, replace=TRUE), dim=c(nlon,
nlat))

# newly defined function
mask <- function(x, y) {
  x[y==0] <- NA
  x
}

# doit
data2 <- simplify2array(lapply(1:ntime, function(i) mask(data[, , i],
lsmask)))


You may prefer to stick with the for() loop approach (for clarity or
simplicity or ...)  When I ramped up the toy data to much larger
dimensions, the lapply() approach was only slightly faster than the for()
loop approach on my PC.

data3 <- data
data3[ , , i] <- mask(data3[ , , i], lsmask)

Jean




On Tue, Sep 6, 2016 at 11:33 PM, Justin Peter 
<justin.pe...@usq.edu.au><mailto:justin.pe...@usq.edu.au>
wrote:




Dear R-user,

I have a three-dimensional matrix of atmospheric data. The first two
dimensions are spatial (lon and lat) and the third is time, such that

dim(data) <- c(nlon,nlat,ntime)

I wish to apply a land sea mask data which is a matrix of "0" and "1" if
dim(nlon,nlat)

dim(lsmask) <- c(nlon,nlat)

I wish to set all of the elements in the two-dimensional array of
data[,,ntime] for every 1:length(ntime).

I could do this in a loop:

for (i in 1:ntime){
data[,,i][which(lsmask == 0)] <- NA
}

I would like to do this using apply, but I need to pass two variables to
the function in apply (data and lsmask), where data is a two-dimensional
array.

I tried:

mask <- function(x,y) {x[which(y==0)] <- NA}

masked_data <- apply(data,c(1,2),mask,y=lsmask)

but I get back a vector of dim(nlon,nlat) populated with NA.

Any clues as to what I am missing?

Thanks in advance for you help.

Kind regards,
Justin



--
Justin Peter
Research Fellow
International Centre for Applied Climate Sciences,
University of Southern Queensland
West St, Toowoomba, QLD, 4350
Australia

Email: 
justin.pe...@usq.edu.au<mailto:justin.pe...@usq.edu.au><mailto:justin.pe...@usq.edu.au><mailto:justin.pe...@usq.edu.au>
Ph: +61 (0) 7 4631 1181
Fax: +61 (0) 7 4631 5581
Mob: +61 (0)474 774 107

Re: [R] Using apply on a three dimensional matrix and passing multiple arguments to user defined function

[Luisfo via R-help]

Hi,

Jean's example with lapply works fine.

However, if you still want to use apply, I think this works.
One observation first. You were passing c(1,2) as second argument to 
apply, in your code. And that is what makes you have lots of NAs as a 
result, since your function is being applied twice, by rows and columns 
(first and second dimensions) respectively.
Use:
 masked_data <- apply(data,3,mask,y=lsmask)
 # but now masked_data has dim(nlon*nlat,ntime), so change it
 dim(masked_data) <- dim(data)

The apply goes over the third dimension (second parameter '3'), so it 
takes every nlot*nlat matrix as first argument for function mask.
I think it should work.

Regards,

*Luisfo Chiroque*
/PhD Student | PhD Candidate
IMDEA Networks Institute/
http://fourier.networks.imdea.org/people/~luis_nunez/ 


On 09/07/2016 03:17 PM, Adams, Jean wrote:
> Justin,
>
> I don't think you can get the apply() function to return an array.  You
> could use lapply() instead, and then use simplify2array() to convert the
> list of matrices to an array.  Also, in your mask() function you don't need
> the which() and you should return the x.  See my example with toy data
> below.
>
> # toy data
> nlon <- 2
> nlat <- 4
> ntime <- 3
> data <- array(1:(nlon*nlat*ntime), dim=c(nlon, nlat, ntime))
> lsmask <- array(sample(0:1, size=nlon*nlat, replace=TRUE), dim=c(nlon,
> nlat))
>
> # newly defined function
> mask <- function(x, y) {
>x[y==0] <- NA
>x
> }
>
> # doit
> data2 <- simplify2array(lapply(1:ntime, function(i) mask(data[, , i],
> lsmask)))
>
>
> You may prefer to stick with the for() loop approach (for clarity or
> simplicity or ...)  When I ramped up the toy data to much larger
> dimensions, the lapply() approach was only slightly faster than the for()
> loop approach on my PC.
>
> data3 <- data
> data3[ , , i] <- mask(data3[ , , i], lsmask)
>
> Jean
>
>
>
>
> On Tue, Sep 6, 2016 at 11:33 PM, Justin Peter 
> wrote:
>
>> Dear R-user,
>>
>> I have a three-dimensional matrix of atmospheric data. The first two
>> dimensions are spatial (lon and lat) and the third is time, such that
>>
>> dim(data) <- c(nlon,nlat,ntime)
>>
>> I wish to apply a land sea mask data which is a matrix of "0" and "1" if
>> dim(nlon,nlat)
>>
>> dim(lsmask) <- c(nlon,nlat)
>>
>> I wish to set all of the elements in the two-dimensional array of
>> data[,,ntime] for every 1:length(ntime).
>>
>> I could do this in a loop:
>>
>> for (i in 1:ntime){
>>  data[,,i][which(lsmask == 0)] <- NA
>> }
>>
>> I would like to do this using apply, but I need to pass two variables to
>> the function in apply (data and lsmask), where data is a two-dimensional
>> array.
>>
>> I tried:
>>
>> mask <- function(x,y) {x[which(y==0)] <- NA}
>>
>> masked_data <- apply(data,c(1,2),mask,y=lsmask)
>>
>> but I get back a vector of dim(nlon,nlat) populated with NA.
>>
>> Any clues as to what I am missing?
>>
>> Thanks in advance for you help.
>>
>> Kind regards,
>> Justin
>>
>>
>>
>> --
>> Justin Peter
>> Research Fellow
>> International Centre for Applied Climate Sciences,
>> University of Southern Queensland
>> West St, Toowoomba, QLD, 4350
>> Australia
>>
>> Email: justin.pe...@usq.edu.au
>> Ph: +61 (0) 7 4631 1181
>> Fax: +61 (0) 7 4631 5581
>> Mob: +61 (0)474 774 107
>>
>>
>>
>>
>> _
>> This email (including any attached files) is confidential and is for the
>> intended recipient(s) only. If you received this email by mistake, please,
>> as a courtesy, tell the sender, then delete this email.
>>
>> The views and opinions are the originator's and do not necessarily reflect
>> those of the University of Southern Queensland. Although all reasonable
>> precautions were taken to ensure that this email contained no viruses at
>> the time it was sent we accept no liability for any losses arising from its
>> receipt.
>>
>> The University of Southern Queensland is a registered provider of
>> education with the Australian Government.
>> (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>>
>>
>>  [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/
>> posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>   [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

Re: [R] Using apply on a three dimensional matrix and passing multiple arguments to user defined function

[Jeff Newmiller]

Should be working, so the devil is in the details you are not showing. Please 
provide a reproducible (self-contained) example [1] and post in plain text 
rather than HTML formatted email to avoid code corruption. 

[1] http://adv-r.had.co.nz/Reproducibility.html
-- 
Sent from my phone. Please excuse my brevity.

On September 6, 2016 9:33:35 PM PDT, Justin Peter  
wrote:
>Dear R-user,
>
>I have a three-dimensional matrix of atmospheric data. The first two
>dimensions are spatial (lon and lat) and the third is time, such that
>
>dim(data) <- c(nlon,nlat,ntime)
>
>I wish to apply a land sea mask data which is a matrix of "0" and "1"
>if dim(nlon,nlat)
>
>dim(lsmask) <- c(nlon,nlat)
>
>I wish to set all of the elements in the two-dimensional array of
>data[,,ntime] for every 1:length(ntime).
>
>I could do this in a loop:
>
>for (i in 1:ntime){
>data[,,i][which(lsmask == 0)] <- NA
>}
>
>I would like to do this using apply, but I need to pass two variables
>to the function in apply (data and lsmask), where data is a
>two-dimensional array.
>
>I tried:
>
>mask <- function(x,y) {x[which(y==0)] <- NA}
>
>masked_data <- apply(data,c(1,2),mask,y=lsmask)
>
>but I get back a vector of dim(nlon,nlat) populated with NA.
>
>Any clues as to what I am missing?
>
>Thanks in advance for you help.
>
>Kind regards,
>Justin
>
>
>
>--
>Justin Peter
>Research Fellow
>International Centre for Applied Climate Sciences,
>University of Southern Queensland
>West St, Toowoomba, QLD, 4350
>Australia
>
>Email: justin.pe...@usq.edu.au
>Ph: +61 (0) 7 4631 1181
>Fax: +61 (0) 7 4631 5581
>Mob: +61 (0)474 774 107
>
>
>
>
>_
>This email (including any attached files) is confidential and is for
>the intended recipient(s) only. If you received this email by mistake,
>please, as a courtesy, tell the sender, then delete this email.
>
>The views and opinions are the originator's and do not necessarily
>reflect those of the University of Southern Queensland. Although all
>reasonable precautions were taken to ensure that this email contained
>no viruses at the time it was sent we accept no liability for any
>losses arising from its receipt.
>
>The University of Southern Queensland is a registered provider of
>education with the Australian Government.
>(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>
>
>   [[alternative HTML version deleted]]
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply on a three dimensional matrix and passing multiple arguments to user defined function

[Adams, Jean]

Justin,

I don't think you can get the apply() function to return an array.  You
could use lapply() instead, and then use simplify2array() to convert the
list of matrices to an array.  Also, in your mask() function you don't need
the which() and you should return the x.  See my example with toy data
below.

# toy data
nlon <- 2
nlat <- 4
ntime <- 3
data <- array(1:(nlon*nlat*ntime), dim=c(nlon, nlat, ntime))
lsmask <- array(sample(0:1, size=nlon*nlat, replace=TRUE), dim=c(nlon,
nlat))

# newly defined function
mask <- function(x, y) {
  x[y==0] <- NA
  x
}

# doit
data2 <- simplify2array(lapply(1:ntime, function(i) mask(data[, , i],
lsmask)))


You may prefer to stick with the for() loop approach (for clarity or
simplicity or ...)  When I ramped up the toy data to much larger
dimensions, the lapply() approach was only slightly faster than the for()
loop approach on my PC.

data3 <- data
data3[ , , i] <- mask(data3[ , , i], lsmask)

Jean




On Tue, Sep 6, 2016 at 11:33 PM, Justin Peter 
wrote:

> Dear R-user,
>
> I have a three-dimensional matrix of atmospheric data. The first two
> dimensions are spatial (lon and lat) and the third is time, such that
>
> dim(data) <- c(nlon,nlat,ntime)
>
> I wish to apply a land sea mask data which is a matrix of "0" and "1" if
> dim(nlon,nlat)
>
> dim(lsmask) <- c(nlon,nlat)
>
> I wish to set all of the elements in the two-dimensional array of
> data[,,ntime] for every 1:length(ntime).
>
> I could do this in a loop:
>
> for (i in 1:ntime){
> data[,,i][which(lsmask == 0)] <- NA
> }
>
> I would like to do this using apply, but I need to pass two variables to
> the function in apply (data and lsmask), where data is a two-dimensional
> array.
>
> I tried:
>
> mask <- function(x,y) {x[which(y==0)] <- NA}
>
> masked_data <- apply(data,c(1,2),mask,y=lsmask)
>
> but I get back a vector of dim(nlon,nlat) populated with NA.
>
> Any clues as to what I am missing?
>
> Thanks in advance for you help.
>
> Kind regards,
> Justin
>
>
>
> --
> Justin Peter
> Research Fellow
> International Centre for Applied Climate Sciences,
> University of Southern Queensland
> West St, Toowoomba, QLD, 4350
> Australia
>
> Email: justin.pe...@usq.edu.au
> Ph: +61 (0) 7 4631 1181
> Fax: +61 (0) 7 4631 5581
> Mob: +61 (0)474 774 107
>
>
>
>
> _
> This email (including any attached files) is confidential and is for the
> intended recipient(s) only. If you received this email by mistake, please,
> as a courtesy, tell the sender, then delete this email.
>
> The views and opinions are the originator's and do not necessarily reflect
> those of the University of Southern Queensland. Although all reasonable
> precautions were taken to ensure that this email contained no viruses at
> the time it was sent we accept no liability for any losses arising from its
> receipt.
>
> The University of Southern Queensland is a registered provider of
> education with the Australian Government.
> (CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply on a three dimensional matrix and passing multiple arguments to user defined function

[Justin Peter]

Dear R-user,

I have a three-dimensional matrix of atmospheric data. The first two dimensions 
are spatial (lon and lat) and the third is time, such that

dim(data) <- c(nlon,nlat,ntime)

I wish to apply a land sea mask data which is a matrix of "0" and "1" if 
dim(nlon,nlat)

dim(lsmask) <- c(nlon,nlat)

I wish to set all of the elements in the two-dimensional array of data[,,ntime] 
for every 1:length(ntime).

I could do this in a loop:

for (i in 1:ntime){
data[,,i][which(lsmask == 0)] <- NA
}

I would like to do this using apply, but I need to pass two variables to the 
function in apply (data and lsmask), where data is a two-dimensional array.

I tried:

mask <- function(x,y) {x[which(y==0)] <- NA}

masked_data <- apply(data,c(1,2),mask,y=lsmask)

but I get back a vector of dim(nlon,nlat) populated with NA.

Any clues as to what I am missing?

Thanks in advance for you help.

Kind regards,
Justin



--
Justin Peter
Research Fellow
International Centre for Applied Climate Sciences,
University of Southern Queensland
West St, Toowoomba, QLD, 4350
Australia

Email: justin.pe...@usq.edu.au
Ph: +61 (0) 7 4631 1181
Fax: +61 (0) 7 4631 5581
Mob: +61 (0)474 774 107




_
This email (including any attached files) is confidential and is for the 
intended recipient(s) only. If you received this email by mistake, please, as a 
courtesy, tell the sender, then delete this email.

The views and opinions are the originator's and do not necessarily reflect 
those of the University of Southern Queensland. Although all reasonable 
precautions were taken to ensure that this email contained no viruses at the 
time it was sent we accept no liability for any losses arising from its receipt.

The University of Southern Queensland is a registered provider of education 
with the Australian Government.
(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081 )


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply to a data frame

[David Winsemius]

> On Apr 7, 2016, at 1:25 PM, John Sorkin  wrote:
> 
> 
> ‪‪I would like to apply a function, fract, to the columns of a
> dataframe. I tried the following 
> apply(data5NonEventEpochs,2,fract)
> but, no surprise it did not work as apply works on matrices not data
> frames.

As Bert pointed out your analsysis is incorrect. If `fract` is  function that 
takes a single vector then you should have gotten back a list or a matrix with 
as many entries or columns as the data5NonEventEpochs had columns.

So to do anything further, please post the output of dput(fract)

-- 
David.

> How can I apply a fuction to the columns of a data frame? (I
> can't covert data5NonEventsEpochs to a matrix as it contains character
> data).
> Thank you,
> John
> John David Sorkin M.D., Ph.D.
> Professor of Medicine
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology and
> Geriatric Medicine
> Baltimore VA Medical Center
> 10 North Greene Street
> GRECC (BT/18/GR)
> Baltimore, MD 21201-1524
> (Phone) 410-605-7119
> (Fax) 410-605-7913 (Please call phone number above prior to faxing) 
> 
> Confidentiality Statement:
> This email message, including any attachments, is for ...{{dropped:16}}

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply to a data frame

[Peter Langfelder]

Use lapply or sapply. A data frame is also a list with each component
representing one column; lapply/sapply will apply the function to each
column.

Peter

On Thu, Apr 7, 2016 at 1:25 PM, John Sorkin  wrote:
>
> ‪‪I would like to apply a function, fract, to the columns of a
> dataframe. I tried the following
> apply(data5NonEventEpochs,2,fract)
> but, no surprise it did not work as apply works on matrices not data
> frames. How can I apply a fuction to the columns of a data frame? (I
> can't covert data5NonEventsEpochs to a matrix as it contains character
> data).
> Thank you,
> John
> John David Sorkin M.D., Ph.D.
> Professor of Medicine
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology and
> Geriatric Medicine
> Baltimore VA Medical Center
> 10 North Greene Street
> GRECC (BT/18/GR)
> Baltimore, MD 21201-1524
> (Phone) 410-605-7119
> (Fax) 410-605-7913 (Please call phone number above prior to faxing)
>
> Confidentiality Statement:
> This email message, including any attachments, is for ...{{dropped:12}}

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply to a data frame

[Bert Gunter]

Inline.

-- Bert
Bert Gunter




On Thu, Apr 7, 2016 at 1:25 PM, John Sorkin  wrote:
>
> ‪‪I would like to apply a function, fract, to the columns of a
> dataframe. I tried the following
> apply(data5NonEventEpochs,2,fract)
> but, no surprise it did not work as apply works on matrices not data
> frames.

That is false! From ?apply:

"If X is not an array but an object of a class with a non-null dim
value (such as a data frame), apply attempts to coerce it to an array
via as.matrix if it is two-dimensional (e.g., a data frame) or via
as.array."

Your apply() call would not have worked with a matrix either, as your
syntax was wrong.  Here is a corrected example:

> X <- data.frame(a=1:5,b=6:10)

> apply(X,2,function(x)mean(sqrt(x)))
   ab
1.676466 2.817189






 How can I apply a fuction to the columns of a data frame? (I
> can't covert data5NonEventsEpochs to a matrix as it contains character
> data).
> Thank you,
> John
> John David Sorkin M.D., Ph.D.
> Professor of Medicine
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology and
> Geriatric Medicine
> Baltimore VA Medical Center
> 10 North Greene Street
> GRECC (BT/18/GR)
> Baltimore, MD 21201-1524
> (Phone) 410-605-7119
> (Fax) 410-605-7913 (Please call phone number above prior to faxing)
>
> Confidentiality Statement:
> This email message, including any attachments, is for ...{{dropped:12}}

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply to a data frame

[Luisfo Chiroque via R-help]

Dear John,

Try using
sapply(data5NonEventEpochs, fract)

HTH

Best,
Luisfo Chiroque
PhD Student
IMDEA Networks Institute
http://fourier.networks.imdea.org/people/~luis_nunez/ 

> El 7 abr 2016, a las 23:25, John Sorkin  
> escribió:
> 
> 
> ‪‪I would like to apply a function, fract, to the columns of a
> dataframe. I tried the following 
> apply(data5NonEventEpochs,2,fract)
> but, no surprise it did not work as apply works on matrices not data
> frames. How can I apply a fuction to the columns of a data frame? (I
> can't covert data5NonEventsEpochs to a matrix as it contains character
> data).
> Thank you,
> John
> John David Sorkin M.D., Ph.D.
> Professor of Medicine
> Chief, Biostatistics and Informatics
> University of Maryland School of Medicine Division of Gerontology and
> Geriatric Medicine
> Baltimore VA Medical Center
> 10 North Greene Street
> GRECC (BT/18/GR)
> Baltimore, MD 21201-1524
> (Phone) 410-605-7119
> (Fax) 410-605-7913 (Please call phone number above prior to faxing) 
> 
> Confidentiality Statement:
> This email message, including any attachments, is for ...{{dropped:16}}

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] using apply to a data frame

[John Sorkin]

‪‪I would like to apply a function, fract, to the columns of a
dataframe. I tried the following 
apply(data5NonEventEpochs,2,fract)
but, no surprise it did not work as apply works on matrices not data
frames. How can I apply a fuction to the columns of a data frame? (I
can't covert data5NonEventsEpochs to a matrix as it contains character
data).
Thank you,
John
John David Sorkin M.D., Ph.D.
Professor of Medicine
Chief, Biostatistics and Informatics
University of Maryland School of Medicine Division of Gerontology and
Geriatric Medicine
Baltimore VA Medical Center
10 North Greene Street
GRECC (BT/18/GR)
Baltimore, MD 21201-1524
(Phone) 410-605-7119
(Fax) 410-605-7913 (Please call phone number above prior to faxing) 

Confidentiality Statement:
This email message, including any attachments, is for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. Any unauthorized use, disclosure or distribution is
prohibited. If you are not the intended recipient, please contact the
sender by reply email and destroy all copies of the original message. 
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply() with functions I wrote

[Pfeiffer, Steven]

Hello!

Does apply() not work with customized functions?  Here is a simple example:

 AddSeven-function(n){n+7}
 AddSeven(3)
   [1] 10
 M-matrix(nrow=2,ncol=2,data=c(1,2,3,4),byrow=TRUE)
 M
[,1] [,2]
   [1,]12
   [2,]34
 apply(x=M,margin=c(1,2),fun=AddSeven)
   Error in match.fun(FUN) : argument FUN is missing, with no default

Thanks for your help!
-Steve Pfeiffer

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply() with functions I wrote

[Bert Gunter]

ummm R is case sensitive! fun != FUN

(Have you gone through any R tutorials yet? If not, please do so
before posting further).

Cheers,
Bert

Bert Gunter
Genentech Nonclinical Biostatistics
(650) 467-7374

Data is not information. Information is not knowledge. And knowledge
is certainly not wisdom.
Clifford Stoll




On Thu, Jul 24, 2014 at 10:07 AM, Pfeiffer, Steven pfeif...@miamioh.edu wrote:
 Hello!

 Does apply() not work with customized functions?  Here is a simple example:

  AddSeven-function(n){n+7}
  AddSeven(3)
[1] 10
  M-matrix(nrow=2,ncol=2,data=c(1,2,3,4),byrow=TRUE)
  M
 [,1] [,2]
[1,]12
[2,]34
  apply(x=M,margin=c(1,2),fun=AddSeven)
Error in match.fun(FUN) : argument FUN is missing, with no default

 Thanks for your help!
 -Steve Pfeiffer

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply() with functions I wrote

[Pfeiffer, Steven]

Ugghh...  Sorry for bothering you all!!


(Yes, I analyzed the data for my ecology M.S. project with R, but I wasted
countless hours committing silly mistakes like this.)


On Thu, Jul 24, 2014 at 4:11 PM, Bert Gunter gunter.ber...@gene.com wrote:

 ummm R is case sensitive! fun != FUN

 (Have you gone through any R tutorials yet? If not, please do so
 before posting further).

 Cheers,
 Bert

 Bert Gunter
 Genentech Nonclinical Biostatistics
 (650) 467-7374

 Data is not information. Information is not knowledge. And knowledge
 is certainly not wisdom.
 Clifford Stoll




 On Thu, Jul 24, 2014 at 10:07 AM, Pfeiffer, Steven pfeif...@miamioh.edu
 wrote:
  Hello!
 
  Does apply() not work with customized functions?  Here is a simple
 example:
 
   AddSeven-function(n){n+7}
   AddSeven(3)
 [1] 10
   M-matrix(nrow=2,ncol=2,data=c(1,2,3,4),byrow=TRUE)
   M
  [,1] [,2]
 [1,]12
 [2,]34
   apply(x=M,margin=c(1,2),fun=AddSeven)
 Error in match.fun(FUN) : argument FUN is missing, with no
 default
 
  Thanks for your help!
  -Steve Pfeiffer
 
  [[alternative HTML version deleted]]
 
  __
  R-help@r-project.org mailing list
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply() with functions I wrote

[Peter Alspach]

Tena koe Steven

R is case-sensitive.  FUN is missing (you have supplied fun - and × and margin) 
...

HTH 

Peter Alspach

-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
Behalf Of Pfeiffer, Steven
Sent: Friday, 25 July 2014 5:08 a.m.
To: r-help@r-project.org
Subject: [R] Using apply() with functions I wrote

Hello!

Does apply() not work with customized functions?  Here is a simple example:

 AddSeven-function(n){n+7}
 AddSeven(3)
   [1] 10
 M-matrix(nrow=2,ncol=2,data=c(1,2,3,4),byrow=TRUE)
 M
[,1] [,2]
   [1,]12
   [2,]34
 apply(x=M,margin=c(1,2),fun=AddSeven)
   Error in match.fun(FUN) : argument FUN is missing, with no default

Thanks for your help!
-Steve Pfeiffer

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
The contents of this e-mail are confidential and may be ...{{dropped:14}}

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[David Winsemius]

On Apr 30, 2014, at 6:03 PM, Waichler, Scott R wrote:

 Here is a working example with no random parts.  Thanks for your patience and 
 if I'm still off the mark with my presentation I'll drop the matter.  
 
 v - c(NA, 1.5, NA, NA,
   NA, 1.1, 0.5, NA,
   NA, 1.3, 0.4, 0.9)
 a1 - array(v, dim=c(2,2,3))
 m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
 m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
 condition1 - !is.na(m1) m1  m2
 
 ans - matrix(NA, ncol=2, nrow=2) # initialize
 for(i in 1:2) {
  for(j in 1:2) {
ind.not.na - which(!is.na(a1[i,j,]))
if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] - 
 a1[i,j,ind.not.na[1]] + m2[i,j]
  }
 }
 ans
 [,1] [,2]
 [1,]   NA 1.66
 [2,] 3.14   NA

I would ask you to look at this loop-free approach and ask if this is not 
equally valid?

ans - matrix(NA, ncol=2, nrow=2) 
ind.not.na - which(!is.na(a1))
ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal 
dimensions, one logical.
 ans
 [,1] [,2]
[1,]   NA 1.66
[2,] 2.74   NA
 
 Let me try asking again in words.  If I have multiple matrices or slices of 
 3d arrays that are the same dimension, is there a way to pass them all to 
 apply, and have apply take care of looping through i,j?

I don't think `apply` is the correct function for this. Either `mapply` or 
basic matrix operation seem more likely to deliver correct results:


 I understand that apply has just one input object x.  I want to work on more 
 than one array object at once using a custom function that has this 
 characteristic:  in order to compute the answer at i,j I need a result from 
 higher order array at the same i,j.  

If you want to iterate over matrix indices you can either use the vector 
version e.g. m2[3] or the matrix version, m2[2,1[.

vec - sapply(seq(length(m2) , function(idx) m2[idx]*condition1[idx] )



 This is what I tried to demonstrate in my example above.
 
 Thanks,
 Scott

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Waichler, Scott R]

 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
  ans
  [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA

Thanks, I am learning something.  I didn't know you could multiply a logical 
object by a numerical one.  But notice the answer is not the same as mine, 
because I am doing an operation on the vector of values a1[i,j,] first.  

I tried a modification on sapply below, but it doesn't work  because I haven't 
referenced the 3d array a1 properly.  So I guess I must try to get a 2d result 
from a1 first, then use that in matrix arithmetic.

Sapply or mapply may work, I haven't used these much and will try to learn 
better how to use them.  Your use of sapply looks good; but I'm trying to 
understand if and how I can bring in the operation on a1.  This doesn't work:

evaluate - function(idx) {
  ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper indexing 
for a1
  if(length(ind.not.na)  0) {
return(condition1*(a1[idx,ind.not.na[1]] + m2[idx]))  # doesn't work; 
improper indexing for a1
  }
}
vec - sapply(seq(length(m2)), evaluate)

Scott Waichler

 -Original Message-
 From: David Winsemius [mailto:dwinsem...@comcast.net]
 Sent: Wednesday, April 30, 2014 8:46 PM
 To: Waichler, Scott R
 Cc: Bert Gunter; r-help@r-project.org
 Subject: Re: [R] Using apply with more than one matrix
 
 
 On Apr 30, 2014, at 6:03 PM, Waichler, Scott R wrote:
 
  Here is a working example with no random parts.  Thanks for your
 patience and if I'm still off the mark with my presentation I'll drop the
 matter.
 
  v - c(NA, 1.5, NA, NA,
NA, 1.1, 0.5, NA,
NA, 1.3, 0.4, 0.9)
  a1 - array(v, dim=c(2,2,3))
  m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
  m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
  condition1 - !is.na(m1) m1  m2
 
  ans - matrix(NA, ncol=2, nrow=2) # initialize for(i in 1:2) {  for(j
  in 1:2) {
 ind.not.na - which(!is.na(a1[i,j,]))
 if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] -
  a1[i,j,ind.not.na[1]] + m2[i,j]  } } ans
  [,1] [,2]
  [1,]   NA 1.66
  [2,] 3.14   NA
 
 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
  ans
  [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA
 
  Let me try asking again in words.  If I have multiple matrices or slices
 of 3d arrays that are the same dimension, is there a way to pass them all
 to apply, and have apply take care of looping through i,j?
 
 I don't think `apply` is the correct function for this. Either `mapply` or
 basic matrix operation seem more likely to deliver correct results:
 
 
  I understand that apply has just one input object x.  I want to work on
 more than one array object at once using a custom function that has this
 characteristic:  in order to compute the answer at i,j I need a result
 from higher order array at the same i,j.
 
 If you want to iterate over matrix indices you can either use the vector
 version e.g. m2[3] or the matrix version, m2[2,1[.
 
 vec - sapply(seq(length(m2) , function(idx) m2[idx]*condition1[idx] )
 
 
 
  This is what I tried to demonstrate in my example above.
 
  Thanks,
  Scott
 
 David Winsemius
 Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[David Winsemius]

On Apr 30, 2014, at 9:32 PM, Waichler, Scott R wrote:

 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
 ans
 [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA
 
 Thanks, I am learning something.  I didn't know you could multiply a logical 
 object by a numerical one.  But notice the answer is not the same as mine, 
 because I am doing an operation on the vector of values a1[i,j,] first.  
 
 I tried a modification on sapply below, but it doesn't work  because I 
 haven't referenced the 3d array a1 properly.  So I guess I must try to get a 
 2d result from a1 first, then use that in matrix arithmetic.
 
 Sapply or mapply may work, I haven't used these much and will try to learn 
 better how to use them.  Your use of sapply looks good; but I'm trying to 
 understand if and how I can bring in the operation on a1.  This doesn't work:
 
 evaluate - function(idx) {
  ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper indexing 
 for a1
  if(length(ind.not.na)  0) {
return(condition1*(a1[idx,ind.not.na[1]] + m2[idx]))  # doesn't work; 
 improper indexing for a1
  }
 }
 vec - sapply(seq(length(m2)), evaluate)

Are we to assume that the length of `which(!is.na(a1[idx,])) ]))` is guaranteed 
to equal the length of the two other matrices? If not then what sort of 
relationships should be assumed?

-- 
David.

 
 Scott Waichler
 
 -Original Message-
 From: David Winsemius [mailto:dwinsem...@comcast.net]
 Sent: Wednesday, April 30, 2014 8:46 PM
 To: Waichler, Scott R
 Cc: Bert Gunter; r-help@r-project.org
 Subject: Re: [R] Using apply with more than one matrix
 
 
 On Apr 30, 2014, at 6:03 PM, Waichler, Scott R wrote:
 
 Here is a working example with no random parts.  Thanks for your
 patience and if I'm still off the mark with my presentation I'll drop the
 matter.
 
 v - c(NA, 1.5, NA, NA,
  NA, 1.1, 0.5, NA,
  NA, 1.3, 0.4, 0.9)
 a1 - array(v, dim=c(2,2,3))
 m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
 m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
 condition1 - !is.na(m1) m1  m2
 
 ans - matrix(NA, ncol=2, nrow=2) # initialize for(i in 1:2) {  for(j
 in 1:2) {
   ind.not.na - which(!is.na(a1[i,j,]))
   if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] -
 a1[i,j,ind.not.na[1]] + m2[i,j]  } } ans
[,1] [,2]
 [1,]   NA 1.66
 [2,] 3.14   NA
 
 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
 ans
 [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA
 
 Let me try asking again in words.  If I have multiple matrices or slices
 of 3d arrays that are the same dimension, is there a way to pass them all
 to apply, and have apply take care of looping through i,j?
 
 I don't think `apply` is the correct function for this. Either `mapply` or
 basic matrix operation seem more likely to deliver correct results:
 
 
 I understand that apply has just one input object x.  I want to work on
 more than one array object at once using a custom function that has this
 characteristic:  in order to compute the answer at i,j I need a result
 from higher order array at the same i,j.
 
 If you want to iterate over matrix indices you can either use the vector
 version e.g. m2[3] or the matrix version, m2[2,1[.
 
 vec - sapply(seq(length(m2) , function(idx) m2[idx]*condition1[idx] )
 
 
 
 This is what I tried to demonstrate in my example above.
 
 Thanks,
 Scott
 
 David Winsemius
 Alameda, CA, USA
 

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[arun]

Hi,

You may try:
evaluate - function(arr, mat1, mat2) {
    if (!all(dim(mat1) == dim(mat2))) {
    stop(both matrices should have equal dimensions)
    }
    indx1 - as.matrix(do.call(expand.grid, lapply(dim(arr), sequence)))
    indx2 - paste0(indx1[, 1], indx1[, 2])
    condition1 - !is.na(mat1)  mat1  mat2
    ans - sapply(seq_along(unique(indx2)), function(i) {
    x1 - indx1[indx2 %in% unique(indx2)[i], ]
    indx3 - which(!is.na(arr[x1]))[1]
    if (condition1[i]  !is.na(indx3)) {
    arr[x1][indx3] + m2[i]
    } else NA
    })
    dim(ans) - dim(mat1)
    ans
}
evaluate(a1,m1,m2)
# [,1] [,2]
#[1,]   NA 1.66
#[2,] 3.14   NA

A.K.


On Thursday, May 1, 2014 2:53 AM, Waichler, Scott R scott.waich...@pnnl.gov 
wrote:
 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
  ans
      [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA

Thanks, I am learning something.  I didn't know you could multiply a logical 
object by a numerical one.  But notice the answer is not the same as mine, 
because I am doing an operation on the vector of values a1[i,j,] first.  

I tried a modification on sapply below, but it doesn't work  because I haven't 
referenced the 3d array a1 properly.  So I guess I must try to get a 2d result 
from a1 first, then use that in matrix arithmetic.

Sapply or mapply may work, I haven't used these much and will try to learn 
better how to use them.  Your use of sapply looks good; but I'm trying to 
understand if and how I can bring in the operation on a1.  This doesn't work:

evaluate - function(idx) {
  ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper indexing 
for a1
  if(length(ind.not.na)  0) {
    return(condition1*(a1[idx,ind.not.na[1]] + m2[idx]))  # doesn't work; 
improper indexing for a1
  }
}
vec - sapply(seq(length(m2)), evaluate)

Scott Waichler


 -Original Message-
 From: David Winsemius [mailto:dwinsem...@comcast.net]
 Sent: Wednesday, April 30, 2014 8:46 PM
 To: Waichler, Scott R
 Cc: Bert Gunter; r-help@r-project.org
 Subject: Re: [R] Using apply with more than one matrix
 
 
 On Apr 30, 2014, at 6:03 PM, Waichler, Scott R wrote:
 
  Here is a working example with no random parts.  Thanks for your
 patience and if I'm still off the mark with my presentation I'll drop the
 matter.
 
  v - c(NA, 1.5, NA, NA,
        NA, 1.1, 0.5, NA,
        NA, 1.3, 0.4, 0.9)
  a1 - array(v, dim=c(2,2,3))
  m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
  m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
  condition1 - !is.na(m1) m1  m2
 
  ans - matrix(NA, ncol=2, nrow=2) # initialize for(i in 1:2) {  for(j
  in 1:2) {
     ind.not.na - which(!is.na(a1[i,j,]))
     if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] -
  a1[i,j,ind.not.na[1]] + m2[i,j]  } } ans
      [,1] [,2]
  [1,]   NA 1.66
  [2,] 3.14   NA
 
 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
  ans
      [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA
 
  Let me try asking again in words.  If I have multiple matrices or slices
 of 3d arrays that are the same dimension, is there a way to pass them all
 to apply, and have apply take care of looping through i,j?
 
 I don't think `apply` is the correct function for this. Either `mapply` or
 basic matrix operation seem more likely to deliver correct results:
 
 
  I understand that apply has just one input object x.  I want to work on
 more than one array object at once using a custom function that has this
 characteristic:  in order to compute the answer at i,j I need a result
 from higher order array at the same i,j.
 
 If you want to iterate over matrix indices you can either use the vector
 version e.g. m2[3] or the matrix version, m2[2,1[.
 
 vec - sapply(seq(length(m2) , function(idx) m2[idx]*condition1[idx] )
 
 
 
  This is what I tried to demonstrate in my example above.
 
  Thanks,
  Scott
 
 David Winsemius
 Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[arun]

Hi,
Sorry, a typo in the previous function:
-
if (condition1[i]  !is.na(indx3)) {
    arr[x1][indx3] + m2[i]  ## should be mat2[i]
    } else NA
 
---

Also, you can try:
library(plyr)
evaluateNew - function(arr, mat1, mat2){
if (!all(dim(mat1) == dim(mat2))) {
    stop(both matrices should have equal dimensions)
    }
 indx1 - unlist(alply(arr, c(1,2), function(x) {x[!is.na(x)][1]}))
condition1 - !is.na(mat1)  mat1  mat2
ifelse(condition1, indx1+mat2, NA)
}
evaluateNew(a1, m1, m2)

#    [,1] [,2]
#[1,]   NA 1.66
#[2,] 3.14   NA
A.K.




On Thursday, May 1, 2014 5:49 AM, arun smartpink...@yahoo.com wrote:


Hi,

You may try:
evaluate - function(arr, mat1, mat2) {
    if (!all(dim(mat1) == dim(mat2))) {
    stop(both matrices should have equal dimensions)
    }
    indx1 - as.matrix(do.call(expand.grid, lapply(dim(arr), sequence)))
    indx2 - paste0(indx1[, 1], indx1[, 2])
    condition1 - !is.na(mat1)  mat1  mat2
    ans - sapply(seq_along(unique(indx2)), function(i) {
    x1 - indx1[indx2 %in% unique(indx2)[i], ]
    indx3 - which(!is.na(arr[x1]))[1]
    if (condition1[i]  !is.na(indx3)) {
    arr[x1][indx3] + m2[i]
    } else NA
    })
    dim(ans) - dim(mat1)
    ans
}
evaluate(a1,m1,m2)
# [,1] [,2]
#[1,]   NA 1.66
#[2,] 3.14   NA

A.K.



On Thursday, May 1, 2014 2:53 AM, Waichler, Scott R scott.waich...@pnnl.gov 
wrote:
 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
  ans
      [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA

Thanks, I am learning something.  I didn't know you could multiply a logical 
object by a numerical one.  But notice the answer is not the same as mine, 
because I am doing an operation on the vector of values a1[i,j,] first.  

I tried a modification on sapply below, but it doesn't work  because I haven't 
referenced the 3d array a1 properly.  So I guess I must try to get a 2d result 
from a1 first, then use that in matrix arithmetic.

Sapply or mapply may work, I haven't used these much and will try to learn 
better how to use them.  Your use of sapply looks good; but I'm trying to 
understand if and how I can bring in the operation on a1.  This doesn't work:

evaluate - function(idx) {
  ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper indexing 
for a1
  if(length(ind.not.na)  0) {
    return(condition1*(a1[idx,ind.not.na[1]] + m2[idx]))  # doesn't work; 
improper indexing for a1
  }
}
vec - sapply(seq(length(m2)), evaluate)

Scott Waichler


 -Original Message-
 From: David Winsemius [mailto:dwinsem...@comcast.net]
 Sent: Wednesday, April 30, 2014 8:46 PM
 To: Waichler, Scott R
 Cc: Bert Gunter; r-help@r-project.org
 Subject: Re: [R] Using apply with more than one matrix
 
 
 On Apr 30, 2014, at 6:03 PM, Waichler, Scott R wrote:
 
  Here is a working example with no random parts.  Thanks for your
 patience and if I'm still off the mark with my presentation I'll drop the
 matter.
 
  v - c(NA, 1.5, NA, NA,
        NA, 1.1, 0.5, NA,
        NA, 1.3, 0.4, 0.9)
  a1 - array(v, dim=c(2,2,3))
  m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
  m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
  condition1 - !is.na(m1) m1  m2
 
  ans - matrix(NA, ncol=2, nrow=2) # initialize for(i in 1:2) {  for(j
  in 1:2) {
     ind.not.na - which(!is.na(a1[i,j,]))
     if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] -
  a1[i,j,ind.not.na[1]] + m2[i,j]  } } ans
      [,1] [,2]
  [1,]   NA 1.66
  [2,] 3.14   NA
 
 I would ask you to look at this loop-free approach and ask if this is not
 equally valid?
 
 ans - matrix(NA, ncol=2, nrow=2)
 ind.not.na - which(!is.na(a1))
 ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
  ans
      [,1] [,2]
 [1,]   NA 1.66
 [2,] 2.74   NA
 
  Let me try asking again in words.  If I have multiple matrices or slices
 of 3d arrays that are the same dimension, is there a way to pass them all
 to apply, and have apply take care of looping through i,j?
 
 I don't think `apply` is the correct function for this. Either `mapply` or
 basic matrix operation seem more likely to deliver correct results:
 
 
  I understand that apply has just one input object x.  I want to work on
 more than one array object at once using a custom function that has this
 characteristic:  in order to compute the answer at i,j I need a result
 from higher order array at the same i,j.
 
 If you want to iterate over matrix indices you can either use the vector
 version e.g. m2[3] or the matrix version, m2[2,1[.
 
 vec - sapply(seq(length(m2) , function(idx) m2[idx]*condition1[idx] )
 
 
 
  This is what I tried to demonstrate in my example above.
 
  Thanks,
  Scott
 
 David Winsemius
 Alameda, CA, USA

Re: [R] Using apply with more than one matrix

[Waichler, Scott R]

  Sapply or mapply may work, I haven't used these much and will try to
 learn better how to use them.  Your use of sapply looks good; but I'm
 trying to understand if and how I can bring in the operation on a1.  This
 doesn't work:
 
  evaluate - function(idx) {
   ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper
  indexing for a1
   if(length(ind.not.na)  0) {
 return(condition1*(a1[idx,ind.not.na[1]] + m2[idx]))  # doesn't
  work; improper indexing for a1  } } vec - sapply(seq(length(m2)),
  evaluate)
 
 Are we to assume that the length of `which(!is.na(a1[idx,])) ]))` is
 guaranteed to equal the length of the two other matrices? If not then what
 sort of relationships should be assumed?

ind.not.na is just a vector that could be any length.  It is a selection on the 
vector a1[i,j,].  I want to get the first element of that selection.  The key 
relationship is that the dimensions of the matrices and the first two 
dimensions of the 3d arrays are the same.  That is, i and j have the same range 
for all of them.  

Scott

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Waichler, Scott R]

Thank you, A.K.  I learned from both of your solutions.  I find the one that 
uses alply easier to follow and understand intuitively.  I guess I'll want to 
learn more about what plyr can do.  I've been using R for years but hadn't 
pushed vectorization far enough in my code.  Now my computing will be more 
efficient.  

Thanks also to David and the others who responded to my question.  It all 
helped.  --Scott Waichler

 -Original Message-
 From: arun [mailto:smartpink...@yahoo.com]
 Sent: Thursday, May 01, 2014 6:24 AM
 To: R. Help
 Cc: Waichler, Scott R
 Subject: Re: [R] Using apply with more than one matrix
 
 Hi,
 Sorry, a typo in the previous function:
 -
 if (condition1[i]  !is.na(indx3)) {
     arr[x1][indx3] + m2[i]  ## should be mat2[i]
     } else NA
 
 ---
 
 Also, you can try:
 library(plyr)
 evaluateNew - function(arr, mat1, mat2){ if (!all(dim(mat1) ==
 dim(mat2))) {
     stop(both matrices should have equal dimensions)
     }
  indx1 - unlist(alply(arr, c(1,2), function(x) {x[!is.na(x)][1]}))
 condition1 - !is.na(mat1)  mat1  mat2 ifelse(condition1, indx1+mat2,
 NA) } evaluateNew(a1, m1, m2)
 
 #    [,1] [,2]
 #[1,]   NA 1.66
 #[2,] 3.14   NA
 A.K.
 
 
 
 
 On Thursday, May 1, 2014 5:49 AM, arun smartpink...@yahoo.com wrote:
 
 
 Hi,
 
 You may try:
 evaluate - function(arr, mat1, mat2) {
     if (!all(dim(mat1) == dim(mat2))) {
     stop(both matrices should have equal dimensions)
     }
     indx1 - as.matrix(do.call(expand.grid, lapply(dim(arr), sequence)))
     indx2 - paste0(indx1[, 1], indx1[, 2])
     condition1 - !is.na(mat1)  mat1  mat2
     ans - sapply(seq_along(unique(indx2)), function(i) {
     x1 - indx1[indx2 %in% unique(indx2)[i], ]
     indx3 - which(!is.na(arr[x1]))[1]
     if (condition1[i]  !is.na(indx3)) {
     arr[x1][indx3] + m2[i]
     } else NA
     })
     dim(ans) - dim(mat1)
     ans
 }
 evaluate(a1,m1,m2)
 # [,1] [,2]
 #[1,]   NA 1.66
 #[2,] 3.14   NA
 
 A.K.
 
 
 
 On Thursday, May 1, 2014 2:53 AM, Waichler, Scott R
 scott.waich...@pnnl.gov wrote:
  I would ask you to look at this loop-free approach and ask if this is
  not equally valid?
 
  ans - matrix(NA, ncol=2, nrow=2)
  ind.not.na - which(!is.na(a1))
  ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
   ans
       [,1] [,2]
  [1,]   NA 1.66
  [2,] 2.74   NA
 
 Thanks, I am learning something.  I didn't know you could multiply a
 logical object by a numerical one.  But notice the answer is not the same
 as mine, because I am doing an operation on the vector of values a1[i,j,]
 first.
 
 I tried a modification on sapply below, but it doesn't work  because I
 haven't referenced the 3d array a1 properly.  So I guess I must try to get
 a 2d result from a1 first, then use that in matrix arithmetic.
 
 Sapply or mapply may work, I haven't used these much and will try to learn
 better how to use them.  Your use of sapply looks good; but I'm trying to
 understand if and how I can bring in the operation on a1.  This doesn't
 work:
 
 evaluate - function(idx) {
   ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper
 indexing for a1
   if(length(ind.not.na)  0) {
     return(condition1*(a1[idx,ind.not.na[1]] + m2[idx]))  # doesn't work;
 improper indexing for a1
   }
 }
 vec - sapply(seq(length(m2)), evaluate)
 
 Scott Waichler
 
 
  -Original Message-
  From: David Winsemius [mailto:dwinsem...@comcast.net]
  Sent: Wednesday, April 30, 2014 8:46 PM
  To: Waichler, Scott R
  Cc: Bert Gunter; r-help@r-project.org
  Subject: Re: [R] Using apply with more than one matrix
 
 
  On Apr 30, 2014, at 6:03 PM, Waichler, Scott R wrote:
 
   Here is a working example with no random parts.  Thanks for your
  patience and if I'm still off the mark with my presentation I'll drop
  the matter.
  
   v - c(NA, 1.5, NA, NA,
         NA, 1.1, 0.5, NA,
         NA, 1.3, 0.4, 0.9)
   a1 - array(v, dim=c(2,2,3))
   m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
   m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
   condition1 - !is.na(m1) m1  m2
  
   ans - matrix(NA, ncol=2, nrow=2) # initialize for(i in 1:2) {
  for(j  in 1:2) {
      ind.not.na - which(!is.na(a1[i,j,]))
      if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] -
  a1[i,j,ind.not.na[1]] + m2[i,j]  } } ans
       [,1] [,2]
   [1,]   NA 1.66
   [2,] 3.14   NA
 
  I would ask you to look at this loop-free approach and ask if this is
  not equally valid?
 
  ans - matrix(NA, ncol=2, nrow=2)
  ind.not.na - which(!is.na(a1))
  ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
   ans
       [,1] [,2]
  [1,]   NA 1.66
  [2,] 2.74   NA
  
   Let me try asking again in words.  If I have multiple matrices or
   slices
  of 3d arrays that are the same dimension, is there a way to pass them
  all to apply, and have apply take care of looping through i,j?
 
  I don't

Re: [R] Using apply with more than one matrix

[William Dunlap]

 Thank you, A.K.  I learned from both of your solutions.  I find the one that 
 uses alply easier to follow and understand intuitively.

Another approach is to only loop over the 3rd dimensional slices of
a1.  Your original code, converted to a function so it is easier to
test and think about is

f0 - function(a1, m1, m2) {
stopifnot(length(dim(a1))==3, length(dim(m1))==2, length(dim(m2))==2,
  all(dim(m1)==dim(m2)), all(dim(m1)==dim(a1)[1:2]),
  dim(a1)[3]  1)
condition1 - !is.na(m1) m1  m2
ans - array(NA_real_, dim(m1)) # initialize
for(i in seq_len(dim(m1)[1])) {
for(j in seq_len(dim(m1)[2])) {
ind.not.na - which(!is.na(a1[i,j,]))
if(condition1[i,j]  length(ind.not.na)  0) {
ans[i,j] - a1[i,j,ind.not.na[1]] + m2[i,j]
}
}
}
ans
}

I believe the following is an equivalent function, but I haven't
checked it much.
Note that the only loop is over the third dimension of a1, which I assume is not
too big.  That updating loop could probably be replaced by a call to Reduce.

f1 - function(a1, m1, m2) {
stopifnot(length(dim(a1))==3, length(dim(m1))==2, length(dim(m2))==2,
  all(dim(m1)==dim(m2)), all(dim(m1)==dim(a1)[1:2]),
  dim(a1)[3]  1)

firstGoodInA1 - array(NA_real_, dim(a1)[1:2])
for(k in seq_len(dim(a1)[3])) {
isStillBad - is.na(firstGoodInA1)
firstGoodInA1[isStillBad] - a1[, ,k][isStillBad]
}
condition1 - !is.na(firstGoodInA1)  !is.na(m1)  (m1  m2)
ans - array(NA_real_, dim(m1))
ans[condition1] - firstGoodInA1[condition1] + m2[condition1]
ans
}


Bill Dunlap
TIBCO Software
wdunlap tibco.com


On Thu, May 1, 2014 at 8:30 AM, Waichler, Scott R
scott.waich...@pnnl.gov wrote:
 Thank you, A.K.  I learned from both of your solutions.  I find the one that 
 uses alply easier to follow and understand intuitively.  I guess I'll want to 
 learn more about what plyr can do.  I've been using R for years but hadn't 
 pushed vectorization far enough in my code.  Now my computing will be more 
 efficient.

 Thanks also to David and the others who responded to my question.  It all 
 helped.  --Scott Waichler

 -Original Message-
 From: arun [mailto:smartpink...@yahoo.com]
 Sent: Thursday, May 01, 2014 6:24 AM
 To: R. Help
 Cc: Waichler, Scott R
 Subject: Re: [R] Using apply with more than one matrix

 Hi,
 Sorry, a typo in the previous function:
 -
 if (condition1[i]  !is.na(indx3)) {
 arr[x1][indx3] + m2[i]  ## should be mat2[i]
 } else NA

 ---

 Also, you can try:
 library(plyr)
 evaluateNew - function(arr, mat1, mat2){ if (!all(dim(mat1) ==
 dim(mat2))) {
 stop(both matrices should have equal dimensions)
 }
  indx1 - unlist(alply(arr, c(1,2), function(x) {x[!is.na(x)][1]}))
 condition1 - !is.na(mat1)  mat1  mat2 ifelse(condition1, indx1+mat2,
 NA) } evaluateNew(a1, m1, m2)

 #[,1] [,2]
 #[1,]   NA 1.66
 #[2,] 3.14   NA
 A.K.




 On Thursday, May 1, 2014 5:49 AM, arun smartpink...@yahoo.com wrote:


 Hi,

 You may try:
 evaluate - function(arr, mat1, mat2) {
 if (!all(dim(mat1) == dim(mat2))) {
 stop(both matrices should have equal dimensions)
 }
 indx1 - as.matrix(do.call(expand.grid, lapply(dim(arr), sequence)))
 indx2 - paste0(indx1[, 1], indx1[, 2])
 condition1 - !is.na(mat1)  mat1  mat2
 ans - sapply(seq_along(unique(indx2)), function(i) {
 x1 - indx1[indx2 %in% unique(indx2)[i], ]
 indx3 - which(!is.na(arr[x1]))[1]
 if (condition1[i]  !is.na(indx3)) {
 arr[x1][indx3] + m2[i]
 } else NA
 })
 dim(ans) - dim(mat1)
 ans
 }
 evaluate(a1,m1,m2)
 # [,1] [,2]
 #[1,]   NA 1.66
 #[2,] 3.14   NA

 A.K.



 On Thursday, May 1, 2014 2:53 AM, Waichler, Scott R
 scott.waich...@pnnl.gov wrote:
  I would ask you to look at this loop-free approach and ask if this is
  not equally valid?
 
  ans - matrix(NA, ncol=2, nrow=2)
  ind.not.na - which(!is.na(a1))
  ans[] - condition1*a1[,,ind.not.na[1]]+ m2  # two matrices of equal
 dimensions, one logical.
   ans
   [,1] [,2]
  [1,]   NA 1.66
  [2,] 2.74   NA

 Thanks, I am learning something.  I didn't know you could multiply a
 logical object by a numerical one.  But notice the answer is not the same
 as mine, because I am doing an operation on the vector of values a1[i,j,]
 first.

 I tried a modification on sapply below, but it doesn't work  because I
 haven't referenced the 3d array a1 properly.  So I guess I must try to get
 a 2d result from a1 first, then use that in matrix arithmetic.

 Sapply or mapply may work, I haven't used these much and will try to learn
 better how to use them.  Your use of sapply looks good; but I'm trying to
 understand if and how I can bring in the operation on a1.  This doesn't
 work:

 evaluate - function(idx) {
   ind.not.na - which(!is.na(a1[idx,])) ]))  # doesn't work; improper

[R] Using apply with more than one matrix

[Waichler, Scott R]

Hi,

I want to apply a custom function to all the elements of one matrix.  The 
function uses the value in the same i,j in a second matrix.  How can I use 
apply() or similar function to avoid nested loops in i and j?

Thanks,
Scott Waichler
Pacific Northwest National Laboratory
Richland, WA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Ranjan Maitra]

Hi Scott,

You could set up a three-dimensional array and then use apply on the
desired dimension (MARGIN in apply language).

HTH!

Many thanks,
Ranjan

On Wed, 30 Apr 2014 17:54:48 + Waichler, Scott R
scott.waich...@pnnl.gov wrote:

 Hi,
 
 I want to apply a custom function to all the elements of one matrix.  The 
 function uses the value in the same i,j in a second matrix.  How can I use 
 apply() or similar function to avoid nested loops in i and j?
 
 Thanks,
 Scott Waichler
 Pacific Northwest National Laboratory
 Richland, WA, USA
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 


-- 
Important Notice: This mailbox is ignored: e-mails are set to be
deleted on receipt. Please respond to the mailing list if appropriate.
For those needing to send personal or professional e-mail, please use
appropriate addresses.


FREE 3D EARTH SCREENSAVER - Watch the Earth right on your desktop!

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Bert Gunter]

Scott:

Your problem specification is rather vague: What do you mean by use?

In general, matrices are merely vectors with a dim attribute, so if
you can do what you want with them as vectors, then that will work for
them as matrices. For example:

 m1- matrix(1:6, nr=3)
 m2 - matrix(11:16, nr=2)
 m2*m2
 [,1] [,2] [,3]
[1,]  121  169  225
[2,]  144  196  256

If this does not meet your needs, you will have to follow the posting
guide and provide both code and a minimal reproducible example.

Cheers,
Bert


Bert Gunter
Genentech Nonclinical Biostatistics
(650) 467-7374

Data is not information. Information is not knowledge. And knowledge
is certainly not wisdom.
H. Gilbert Welch




On Wed, Apr 30, 2014 at 10:54 AM, Waichler, Scott R
scott.waich...@pnnl.gov wrote:
 Hi,

 I want to apply a custom function to all the elements of one matrix.  The 
 function uses the value in the same i,j in a second matrix.  How can I use 
 apply() or similar function to avoid nested loops in i and j?

 Thanks,
 Scott Waichler
 Pacific Northwest National Laboratory
 Richland, WA, USA

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Waichler, Scott R]

Ok, here is a toy example.  I want to use a custom function that depends on 
more than one matrix/array as input, and I can't figure out how to do that with 
apply. 

v - c(NA, 1.5, NA, NA,
   NA, 1.1, 0.5, NA,
   NA, 1.3, 0.4, 0.9)
a1 - array(v, dim=c(2,2,3))
m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
m2 - matrix(runif(n=4, min=1, max=3), ncol=2)
condition1 - ifelse(!is.na(m1)  m1  m2, T, F)

ans - matrix(NA, ncol=2, nrow=2) # initialize
for(i in 1:2) {
  for(j in 1:2) {
ind.not.na - which(!is.na(a1[i,j,]))
if(condition1[i,j]  length(ind.not.na)  0) {
  ans[i,j] - a1[i,j,ind.not.na[1]] + m2[i,j]
}
  }
}

Scott Waichler



 -Original Message-
 From: Bert Gunter [mailto:gunter.ber...@gene.com]
 Sent: Wednesday, April 30, 2014 12:18 PM
 To: Waichler, Scott R
 Cc: r-help@r-project.org
 Subject: Re: [R] Using apply with more than one matrix
 
 Scott:
 
 Your problem specification is rather vague: What do you mean by use?
 
 In general, matrices are merely vectors with a dim attribute, so if you
 can do what you want with them as vectors, then that will work for them as
 matrices. For example:
 
  m1- matrix(1:6, nr=3)
  m2 - matrix(11:16, nr=2)
  m2*m2
  [,1] [,2] [,3]
 [1,]  121  169  225
 [2,]  144  196  256
 
 If this does not meet your needs, you will have to follow the posting
 guide and provide both code and a minimal reproducible example.
 
 Cheers,
 Bert
 
 
 Bert Gunter
 Genentech Nonclinical Biostatistics
 (650) 467-7374
 
 Data is not information. Information is not knowledge. And knowledge is
 certainly not wisdom.
 H. Gilbert Welch
 
 
 
 
 On Wed, Apr 30, 2014 at 10:54 AM, Waichler, Scott R
 scott.waich...@pnnl.gov wrote:
  Hi,
 
  I want to apply a custom function to all the elements of one matrix.
 The function uses the value in the same i,j in a second matrix.  How can I
 use apply() or similar function to avoid nested loops in i and j?
 
  Thanks,
  Scott Waichler
  Pacific Northwest National Laboratory
  Richland, WA, USA
 
  __
  R-help@r-project.org mailing list
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide
  http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[David Winsemius]

On Apr 30, 2014, at 1:21 PM, Waichler, Scott R wrote:

 Ok, here is a toy example.  I want to use a custom function that depends on 
 more than one matrix/array as input, and I can't figure out how to do that 
 with apply. 
 
 v - c(NA, 1.5, NA, NA,
   NA, 1.1, 0.5, NA,
   NA, 1.3, 0.4, 0.9)
 a1 - array(v, dim=c(2,2,3))
 m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
 m2 - matrix(runif(n=4, min=1, max=3), ncol=2)
 condition1 - ifelse(!is.na(m1)  m1  m2, T, F)

Just use:

condition1 - !is.na(m1)  m1  m2

For the next set of operation you should heed Jim Holtman's tagline:

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.


 
 ans - matrix(NA, ncol=2, nrow=2) # initialize
 for(i in 1:2) {
  for(j in 1:2) {
ind.not.na - which(!is.na(a1[i,j,]))
if(condition1[i,j]  length(ind.not.na)  0) {
  ans[i,j] - a1[i,j,ind.not.na[1]] + m2[i,j]
}
  }
 }

At least in the realizatiuon I used all of the entries for conditon1 were 
FALSE, so a better test might be to specify set.seed() to give a reproducible 
example.

-- 

David.


 
 Scott Waichler
 
 
 
 -Original Message-
 From: Bert Gunter [mailto:gunter.ber...@gene.com]
 Sent: Wednesday, April 30, 2014 12:18 PM
 To: Waichler, Scott R
 Cc: r-help@r-project.org
 Subject: Re: [R] Using apply with more than one matrix
 
 Scott:
 
 Your problem specification is rather vague: What do you mean by use?
 
 In general, matrices are merely vectors with a dim attribute, so if you
 can do what you want with them as vectors, then that will work for them as
 matrices. For example:
 
 m1- matrix(1:6, nr=3)
 m2 - matrix(11:16, nr=2)
 m2*m2
 [,1] [,2] [,3]
 [1,]  121  169  225
 [2,]  144  196  256
 
 If this does not meet your needs, you will have to follow the posting
 guide and provide both code and a minimal reproducible example.
 
 Cheers,
 Bert
 
 
 Bert Gunter
 Genentech Nonclinical Biostatistics
 (650) 467-7374
 
 Data is not information. Information is not knowledge. And knowledge is
 certainly not wisdom.
 H. Gilbert Welch
 
 
 
 
 On Wed, Apr 30, 2014 at 10:54 AM, Waichler, Scott R
 scott.waich...@pnnl.gov wrote:
 Hi,
 
 I want to apply a custom function to all the elements of one matrix.
 The function uses the value in the same i,j in a second matrix.  How can I
 use apply() or similar function to avoid nested loops in i and j?
 
 Thanks,
 Scott Waichler
 Pacific Northwest National Laboratory
 Richland, WA, USA
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Waichler, Scott R]

Here is a working example with no random parts.  Thanks for your patience and 
if I'm still off the mark with my presentation I'll drop the matter.  

v - c(NA, 1.5, NA, NA,
   NA, 1.1, 0.5, NA,
   NA, 1.3, 0.4, 0.9)
a1 - array(v, dim=c(2,2,3))
m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
condition1 - !is.na(m1) m1  m2

ans - matrix(NA, ncol=2, nrow=2) # initialize
for(i in 1:2) {
  for(j in 1:2) {
ind.not.na - which(!is.na(a1[i,j,]))
if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] - 
a1[i,j,ind.not.na[1]] + m2[i,j]
  }
}
ans
 [,1] [,2]
[1,]   NA 1.66
[2,] 3.14   NA

Let me try asking again in words.  If I have multiple matrices or slices of 3d 
arrays that are the same dimension, is there a way to pass them all to apply, 
and have apply take care of looping through i,j?  I understand that apply has 
just one input object x.  I want to work on more than one array object at once 
using a custom function that has this characteristic:  in order to compute the 
answer at i,j I need a result from higher order array at the same i,j.  This is 
what I tried to demonstrate in my example above.

Thanks,
Scott

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply with more than one matrix

[Richard M. Heiberger]

Try
?mapply

Description:

 'mapply' is a multivariate version of 'sapply'.  'mapply' applies
 'FUN' to the first elements of each ...  argument, the second
 elements, the third elements, and so on.  Arguments are recycled
 if necessary.

Rich

On Wed, Apr 30, 2014 at 9:03 PM, Waichler, Scott R
scott.waich...@pnnl.gov wrote:
 Here is a working example with no random parts.  Thanks for your patience and 
 if I'm still off the mark with my presentation I'll drop the matter.

 v - c(NA, 1.5, NA, NA,
NA, 1.1, 0.5, NA,
NA, 1.3, 0.4, 0.9)
 a1 - array(v, dim=c(2,2,3))
 m1 - matrix(c(NA, 1.5, 2.1, NA), ncol=2, byrow=T)
 m2 - matrix(c(1.56, 1.64, 1.16, 2.92), ncol=2)
 condition1 - !is.na(m1) m1  m2

 ans - matrix(NA, ncol=2, nrow=2) # initialize
 for(i in 1:2) {
   for(j in 1:2) {
 ind.not.na - which(!is.na(a1[i,j,]))
 if(condition1[i,j]  length(ind.not.na)  0) ans[i,j] - 
 a1[i,j,ind.not.na[1]] + m2[i,j]
   }
 }
 ans
  [,1] [,2]
 [1,]   NA 1.66
 [2,] 3.14   NA

 Let me try asking again in words.  If I have multiple matrices or slices of 
 3d arrays that are the same dimension, is there a way to pass them all to 
 apply, and have apply take care of looping through i,j?  I understand that 
 apply has just one input object x.  I want to work on more than one array 
 object at once using a custom function that has this characteristic:  in 
 order to compute the answer at i,j I need a result from higher order array at 
 the same i,j.  This is what I tried to demonstrate in my example above.

 Thanks,
 Scott

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply function

[Yen Lee]

Hi all R-users,

I'm trying to using apply function to input a range of values into a 
function I wrote.
I wrote a function with 4 information needed. I would like to make 2 of 
them fixed and the other 2 random (but with specified values).

I would like to replicate the function 1 times.
I was thinking about using loop function but which is really slow, 
therefore I transfer to apply function.

But I got stucked. Could anyone help me?

The question could be illustrated as follows,
Target function: fun(F1,F2,R1,R2)
R1 has values 1, 2, 3, 4, 5
R2 has values -1, -2, -3, -4, -5
F1=10
F2=100
There would be 25 conditions. I would like to avoid using loop to get 
the result.

Could anyone give me some precious suggestion?

Thank you

Best,
Yen

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function

[jim holtman]

Is this what you want:

 random -  expand.grid(R1 = 1:5, R2 = -(1:5))
 result - cbind(F1 = 10, F2 = 100, random)

 result
   F1  F2 R1 R2
1  10 100  1 -1
2  10 100  2 -1
3  10 100  3 -1
4  10 100  4 -1
5  10 100  5 -1
6  10 100  1 -2
7  10 100  2 -2
8  10 100  3 -2
9  10 100  4 -2
10 10 100  5 -2
11 10 100  1 -3
12 10 100  2 -3
13 10 100  3 -3
14 10 100  4 -3
15 10 100  5 -3
16 10 100  1 -4
17 10 100  2 -4
18 10 100  3 -4
19 10 100  4 -4
20 10 100  5 -4
21 10 100  1 -5
22 10 100  2 -5
23 10 100  3 -5
24 10 100  4 -5
25 10 100  5 -5

Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.


On Mon, Jan 27, 2014 at 8:56 AM, Yen Lee b88207...@ntu.edu.tw wrote:
 Hi all R-users,

 I'm trying to using apply function to input a range of values into a
 function I wrote.
 I wrote a function with 4 information needed. I would like to make 2 of them
 fixed and the other 2 random (but with specified values).
 I would like to replicate the function 1 times.
 I was thinking about using loop function but which is really slow, therefore
 I transfer to apply function.
 But I got stucked. Could anyone help me?

 The question could be illustrated as follows,
 Target function: fun(F1,F2,R1,R2)
 R1 has values 1, 2, 3, 4, 5
 R2 has values -1, -2, -3, -4, -5
 F1=10
 F2=100
 There would be 25 conditions. I would like to avoid using loop to get the
 result.
 Could anyone give me some precious suggestion?

 Thank you

 Best,
 Yen

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] using 'apply' to apply princomp to an array of datasets

[David Romano]

Hi everyone,

Suppose I have a 3D array of datasets, where say dimension 1 corresponds to
cases, dimension 2 to datasets, and dimension 3 to observations within a
dataset.  As an example, suppose I do the following:

 x - sample(1:20, 48, replace=TRUE)
 datasets - array(x, dim=c(4,3,2))

Here, for each j=1,2,3, I'd like to think of datasets[,j,] as a single data
matrix with four cases and two observations.  Now, I'd like to be able to
do the following: apply pca to each dataset, and create a matrix of the
first principal component scores.

In this example, I could do:

 pcl-apply(datasets,2,princomp)

which yields a list of princomp output, one for each dataset, so that the
vector of first principal component scores for dataset 1 is obtained by

 score1set1 - pcl[[1]]$scores[,1]

and I could then obtain the desired matrix by

 score1matrix - cbind( score1set1, score1set2, score1set3)


So my first question is: 1) how could I use *apply to do this?  I'm having
trouble because pcl is a list of lists, so I can't use, say, do.call(cbind,
...) without first having a list of the first component score vectors,
which I'm not sure how to produce.

My second question is: 2) Having answered question 1), now suppose there
may be datasets containing NA value -- how could I select the subset of
values from dimension 2 corresponding to the datasets for which this is
true (again using *apply?)?

Thanks in advance for any light you might be able to shed on these
questions!

David Romano

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using 'apply' to apply princomp to an array of datasets

[David Romano]

Sorry, I just realized I didn't send the message below in plain text.
-David Romano

On Wed, Dec 12, 2012 at 9:14 AM, David Romano drom...@stanford.edu wrote:

 Hi everyone,

 Suppose I have a 3D array of datasets, where say dimension 1 corresponds
 to cases, dimension 2 to datasets, and dimension 3 to observations within a
 dataset.  As an example, suppose I do the following:

  x - sample(1:20, 48, replace=TRUE)
  datasets - array(x, dim=c(4,3,2))

 Here, for each j=1,2,3, I'd like to think of datasets[,j,] as a single
 data matrix with four cases and two observations.  Now, I'd like to be able
 to do the following: apply pca to each dataset, and create a matrix of the
 first principal component scores.

 In this example, I could do:

  pcl-apply(datasets,2,princomp)

 which yields a list of princomp output, one for each dataset, so that the
 vector of first principal component scores for dataset 1 is obtained by

  score1set1 - pcl[[1]]$scores[,1]

 and I could then obtain the desired matrix by

  score1matrix - cbind( score1set1, score1set2, score1set3)


 So my first question is: 1) how could I use *apply to do this?  I'm having
 trouble because pcl is a list of lists, so I can't use, say, do.call(cbind,
 ...) without first having a list of the first component score vectors, which
 I'm not sure how to produce.

 My second question is: 2) Having answered question 1), now suppose there
 may be datasets containing NA value -- how could I select the subset of
 values from dimension 2 corresponding to the datasets for which this is true
 (again using *apply?)?

 Thanks in advance for any light you might be able to shed on these
 questions!

 David Romano

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using 'apply' to apply princomp to an array of datasets

[Rui Barradas]

Hello,

As for the first question try

scoreset - lapply(pcl, function(x) x$scores[, 1])
do.call(cbind, scoreset)


As for the second question, you want to know which columns in 'datasets' 
have NA's?


colidx - apply(datasets, 2, function(x) any(is.na(x)))
datasets[, colidx]  # These have NA's


For the column numbers you can do

colnums - which(colidx)

Hope this helps,

Rui Barradas

Em 12-12-2012 17:14, David Romano escreveu:

Hi everyone,

Suppose I have a 3D array of datasets, where say dimension 1 corresponds to
cases, dimension 2 to datasets, and dimension 3 to observations within a
dataset.  As an example, suppose I do the following:


x - sample(1:20, 48, replace=TRUE)
datasets - array(x, dim=c(4,3,2))

Here, for each j=1,2,3, I'd like to think of datasets[,j,] as a single data
matrix with four cases and two observations.  Now, I'd like to be able to
do the following: apply pca to each dataset, and create a matrix of the
first principal component scores.

In this example, I could do:


pcl-apply(datasets,2,princomp)

which yields a list of princomp output, one for each dataset, so that the
vector of first principal component scores for dataset 1 is obtained by


score1set1 - pcl[[1]]$scores[,1]

and I could then obtain the desired matrix by


score1matrix - cbind( score1set1, score1set2, score1set3)


So my first question is: 1) how could I use *apply to do this?  I'm having
trouble because pcl is a list of lists, so I can't use, say, do.call(cbind,
...) without first having a list of the first component score vectors,
which I'm not sure how to produce.

My second question is: 2) Having answered question 1), now suppose there
may be datasets containing NA value -- how could I select the subset of
values from dimension 2 corresponding to the datasets for which this is
true (again using *apply?)?

Thanks in advance for any light you might be able to shed on these
questions!

David Romano

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using 'apply' to apply princomp to an array of datasets

[David Romano]

Thank you, Rui!   This is incredibly helpful -- anonymous functions
are new to me, and I appreciate being shown how useful they are.

Best regards,
David

On Wed, Dec 12, 2012 at 10:12 AM, Rui Barradas ruipbarra...@sapo.pt wrote:
 Hello,

 As for the first question try

 scoreset - lapply(pcl, function(x) x$scores[, 1])
 do.call(cbind, scoreset)


 As for the second question, you want to know which columns in 'datasets'
 have NA's?

 colidx - apply(datasets, 2, function(x) any(is.na(x)))
 datasets[, colidx]  # These have NA's


 For the column numbers you can do

 colnums - which(colidx)

 Hope this helps,

 Rui Barradas

 Em 12-12-2012 17:14, David Romano escreveu:

 Hi everyone,

 Suppose I have a 3D array of datasets, where say dimension 1 corresponds
 to
 cases, dimension 2 to datasets, and dimension 3 to observations within a
 dataset.  As an example, suppose I do the following:

 x - sample(1:20, 48, replace=TRUE)
 datasets - array(x, dim=c(4,3,2))

 Here, for each j=1,2,3, I'd like to think of datasets[,j,] as a single
 data
 matrix with four cases and two observations.  Now, I'd like to be able to
 do the following: apply pca to each dataset, and create a matrix of the
 first principal component scores.

 In this example, I could do:

 pcl-apply(datasets,2,princomp)

 which yields a list of princomp output, one for each dataset, so that the
 vector of first principal component scores for dataset 1 is obtained by

 score1set1 - pcl[[1]]$scores[,1]

 and I could then obtain the desired matrix by

 score1matrix - cbind( score1set1, score1set2, score1set3)


 So my first question is: 1) how could I use *apply to do this?  I'm having
 trouble because pcl is a list of lists, so I can't use, say,
 do.call(cbind,
 ...) without first having a list of the first component score vectors,
 which I'm not sure how to produce.

 My second question is: 2) Having answered question 1), now suppose there
 may be datasets containing NA value -- how could I select the subset of
 values from dimension 2 corresponding to the datasets for which this is
 true (again using *apply?)?

 Thanks in advance for any light you might be able to shed on these
 questions!

 David Romano

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply instead of for loop / multithreading

[Flavio Barros]

That worked better because of vectorization, but isn't multithreaded.

To have this resource look at plyr package.

On Mon, Nov 12, 2012 at 9:08 PM, Charles D. charlybeg...@live.fr wrote:

 it works really faster !
 thank you



 --
 View this message in context:
 http://r.789695.n4.nabble.com/Using-apply-instead-of-for-loop-multithreading-tp4649326p4649346.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Att,

Flávio Barros

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply instead of for loop / multithreading

[Charles D.]

Hello ,

I'm new to R and don't really understand how to use the function apply
instead of a for loop, particularly for a function with multiple entries.
I have a big data file and would like to apply a function in multi thread to
accelerate the processus.

I have a data frame containing values of* CO2 in ppm (resp[i,6])* that I
want to convert in umol of CO2 emitted by stem volume biomass (CO2v) and
stem area (CO2s). (tree respiration) 

The loop that I have is calculating the CO2 fluxes for each row.

*Script :*
for (i in 1:nrow(resp)) {
k=resp[i,5]
CO2umol[i]-resp[i,6]*((Press*infoch[k,11]*1e-6)/(R*(resp[i,7]+273.15)))
CO2v[i]-CO2umol[i]/(infoch[k,10])
CO2s[i]-CO2umol[i]/(infoch[k,9])
}

For that, I have two data frames :
- *infoch :* variables used to calculate CO2 fluxes (16 rows with the
characteristics of 16 analysis chambers)

http://r.789695.n4.nabble.com/file/n4649326/infoch.jpg 

- *resp: *time series containing CO2 values and temperatures by chamber
analysed (more than 500.000 rows)

http://r.789695.n4.nabble.com/file/n4649326/resp.jpg 

other variables :
Press=93000
R=8.31

The loop is working good, but taking quite a long time to process, if
someone could explain me how to use the apply function in this case, it
would be really helpful.

Thanks
Charles



--
View this message in context: 
http://r.789695.n4.nabble.com/Using-apply-instead-of-for-loop-multithreading-tp4649326.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply instead of for loop / multithreading

[jim holtman]

Since you did not provide sample data (use dput instead of posting
links to 'jpg' files), you can probably do it without loops:

k=resp[,5]
CO2umol - resp[,6]*((Press*infoch[k,11]*1e-6)/(R*(resp[,7]+273.15)))
CO2v - CO2umol/(infoch[k,10])
CO2s - CO2umol/(infoch[k,9]

On Mon, Nov 12, 2012 at 12:39 PM, Charles D. charlybeg...@live.fr wrote:
 Hello ,

 I'm new to R and don't really understand how to use the function apply
 instead of a for loop, particularly for a function with multiple entries.
 I have a big data file and would like to apply a function in multi thread to
 accelerate the processus.

 I have a data frame containing values of* CO2 in ppm (resp[i,6])* that I
 want to convert in umol of CO2 emitted by stem volume biomass (CO2v) and
 stem area (CO2s). (tree respiration)

 The loop that I have is calculating the CO2 fluxes for each row.

 *Script :*
 for (i in 1:nrow(resp)) {
 k=resp[i,5]
 CO2umol[i]-resp[i,6]*((Press*infoch[k,11]*1e-6)/(R*(resp[i,7]+273.15)))
 CO2v[i]-CO2umol[i]/(infoch[k,10])
 CO2s[i]-CO2umol[i]/(infoch[k,9])
 }

 For that, I have two data frames :
 - *infoch :* variables used to calculate CO2 fluxes (16 rows with the
 characteristics of 16 analysis chambers)

 http://r.789695.n4.nabble.com/file/n4649326/infoch.jpg

 - *resp: *time series containing CO2 values and temperatures by chamber
 analysed (more than 500.000 rows)

 http://r.789695.n4.nabble.com/file/n4649326/resp.jpg

 other variables :
 Press=93000
 R=8.31

 The loop is working good, but taking quite a long time to process, if
 someone could explain me how to use the apply function in this case, it
 would be really helpful.

 Thanks
 Charles



 --
 View this message in context: 
 http://r.789695.n4.nabble.com/Using-apply-instead-of-for-loop-multithreading-tp4649326.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply instead of for loop / multithreading

[Charles D.]

it works really faster ! 
thank you



--
View this message in context: 
http://r.789695.n4.nabble.com/Using-apply-instead-of-for-loop-multithreading-tp4649326p4649346.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply with sparse matrix from package Matrix

[Martin Maechler]

 Jennifer Lyon jennifer.s.l...@gmail.com
 on Fri, 31 Aug 2012 17:22:57 -0600 writes:

 Hi:
 I was trying to use apply on a sparse matrix from package Matrix,
 and I get the error:

 Error in asMethod(object) :
 Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 
106

 Is there a way to apply a function to all the rows without bumping
 into this problem?

 Here is a simplified example:

 dim(sm)
 [1] 72913 43052

 class(sm)
 [1] dgCMatrix
 attr(,package)
 [1] Matrix

 str(sm)
 Formal class 'dgCMatrix' [package Matrix] with 6 slots
 ..@ i   : int [1:6590004] 789 801 802 1231 1236 11739 17817
 17943 18148 18676 ...
 ..@ p   : int [1:43053] 0 147 303 450 596 751 908 1053 1188 1347 ...
 ..@ Dim : int [1:2] 72913 43052
 ..@ Dimnames:List of 2
 .. ..$ : NULL
 .. ..$ : NULL
 ..@ x   : num [1:6590004] 0.601 0.527 0.562 0.641 0.684 ...
 ..@ factors : list()

 my.sum-apply(sm, 1, sum)
 Error in asMethod(object) :
 Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 
106

So, actually it would have worked (though not efficiently) if
your sm matrix would have been much smaller.

However,  we provide  rowSums(), rowMeans(), colSums(), colMeans() 
for all of our matrices, including the sparse ones.

So your present problem can be solved using

my.sum - rowSums(sm)

Best regards,
Martin Maechler, ETH Zurich

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply with sparse matrix from package Matrix

[Jennifer Lyon]

On Tue, Sep 4, 2012 at 10:58 AM, Martin Maechler
maech...@stat.math.ethz.ch wrote:
 Jennifer Lyon jennifer.s.l...@gmail.com
 on Fri, 31 Aug 2012 17:22:57 -0600 writes:

  Hi:
  I was trying to use apply on a sparse matrix from package Matrix,
  and I get the error:

  Error in asMethod(object) :
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 
 106

  Is there a way to apply a function to all the rows without bumping
  into this problem?

  Here is a simplified example:

  dim(sm)
  [1] 72913 43052

  class(sm)
  [1] dgCMatrix
  attr(,package)
  [1] Matrix

  str(sm)
  Formal class 'dgCMatrix' [package Matrix] with 6 slots
  ..@ i   : int [1:6590004] 789 801 802 1231 1236 11739 17817
  17943 18148 18676 ...
  ..@ p   : int [1:43053] 0 147 303 450 596 751 908 1053 1188 1347 ...
  ..@ Dim : int [1:2] 72913 43052
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x   : num [1:6590004] 0.601 0.527 0.562 0.641 0.684 ...
  ..@ factors : list()

  my.sum-apply(sm, 1, sum)
  Error in asMethod(object) :
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 
 106

 So, actually it would have worked (though not efficiently) if
 your sm matrix would have been much smaller.

 However,  we provide  rowSums(), rowMeans(), colSums(), colMeans()
 for all of our matrices, including the sparse ones.

 So your present problem can be solved using

 my.sum - rowSums(sm)

 Best regards,
 Martin Maechler, ETH Zurich

Thank you for letting me know about rowSums(). Two points.  First,
sadly, I was unclear in my posting, and using sum was just an
example. In the real case I am using my own function on each row. I
guess the answer for this problem is that iteration is my friend. Good
to know.

Second, since I'm embarrassed to say I hadn't remembered rowSums(), for
cases when I needed the sum of the rows, I had just been postmultiplying
by a vector of 1's.  Just FYI, I thought I should try rowSums(), so did
a small timing trial, and it appears postmultiplying is faster than
rowSums. Run is as follows:

 str(sm)
Formal class 'dgCMatrix' [package Matrix] with 6 slots
  ..@ i   : int [1:6590004] 721 926 1275 1791 2370 2755 3393 4638
5363 5566 ...
  ..@ p   : int [1:43053] 0 147 303 450 596 751 908 1053 1188 1347 ...
  ..@ Dim : int [1:2] 72913 43052
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x   : num [1:6590004] 0.0735 0.3206 0.1861 0.1604 0.197 ...
  ..@ factors : list()

 library(rbenchmark)

#Just checking how expensive building a vector of 1's is - not very
#at least for matrix of the size I'm interested in
 benchmark(i1-rep(1, ncol(sm)))
test replications elapsed relative user.self sys.self
1 i1 - rep(1, ncol(sm))  100   0.1191  0.120
  user.child sys.child
1  0 0

#Postmultiplying by 1's timing
 benchmark(la-sm %*% i1)
 test replications elapsed relative user.self sys.self user.child
1 la - sm %*% i1  100   5.9931 5.9930  0
  sys.child
1 0

#rowSums timing
 benchmark(la1-rowSums(sm))
test replications elapsed relative user.self sys.self
1 la1 - rowSums(sm)  100  28.117128.1140.004
  user.child sys.child
1  0 0

#Make sure the results are the same
  all(la==la1)
[1] TRUE

The Matrix package is awesome, and I appreciate you taking the
time to answer my questions.

Jen


 sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-unknown-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=C LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

other attached packages:
[1] rbenchmark_0.3.1 Matrix_1.0-6 lattice_0.20-6

loaded via a namespace (and not attached):
[1] grid_2.15.1

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] using apply with sparse matrix from package Matrix

[Jennifer Lyon]

Hi:

I was trying to use apply on a sparse matrix from package Matrix,
and I get the error:

Error in asMethod(object) :
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 106

Is there a way to apply a function to all the rows without bumping
into this problem?

Here is a simplified example:

 dim(sm)
[1] 72913 43052

 class(sm)
[1] dgCMatrix
attr(,package)
[1] Matrix

 str(sm)
Formal class 'dgCMatrix' [package Matrix] with 6 slots
  ..@ i   : int [1:6590004] 789 801 802 1231 1236 11739 17817
17943 18148 18676 ...
  ..@ p   : int [1:43053] 0 147 303 450 596 751 908 1053 1188 1347 ...
  ..@ Dim : int [1:2] 72913 43052
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x   : num [1:6590004] 0.601 0.527 0.562 0.641 0.684 ...
  ..@ factors : list()

 my.sum-apply(sm, 1, sum)
Error in asMethod(object) :
  Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 106

 sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-unknown-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=C LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

other attached packages:
[1] Matrix_1.0-6   lattice_0.20-6 stringr_0.6.1

loaded via a namespace (and not attached):
[1] grid_2.15.1 plyr_1.7.1

Thanks.

Jen

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply() with a function involving ode()

[Adam Zeilinger]

Hello,

I am trying to get the output from the numerical simulation of a system 
of ordinary differential equations for a range of values for three 
parameters.  I am using the ode() function (deSolve package) to run the 
numerical simulation and apply() to run the simulation function for each 
set of parameter values.  I am having trouble getting the apply() 
function to work.


Here is an example.  Consider an epidemiology model of West Nile Virus:

wnv.model - function(Time, State, Pars){
  with(as.list(c(State, Pars)), {
dSB - -(alphaB*betaB*SB*IM)/(SB + IB)
dIB - (alphaB*betaB*SB*IM)/(SB + IB) - deltaB*IB
dSM - bM*(SM + EM + IM) - (alphaM*betaB*IB*SM)/(SB 
+ IB) - dM*SM
dEM - (alphaM*betaB*SM*IB)/(SB + IB) - kappaM*EM - 
dM*EM

dIM - kappaM*EM - dM*IM
return(list(c(dSB, dIB, dSM, dEM, dIM)))
  })
}

# I would like to run a numerical simulation of this model for each 
combination of values for the parameters alphaB, betaB, # and kappaM:


av - seq(0, 1, by = 0.2) # vector of values for alphaB
bv - seq(0, 1, by = 0.2) # vector of values for betaB
kv - seq(0, 1, by = 0.2) # vector of values for kappaM

# Here is my function with ode() for the numerical simulation.  The 
function returns the last value of the simulation for the # IB state 
variable (Infected Birds).


library(deSolve)
wnv.sim - function(x){
 Pars - c(alphaB = x[1], betaB = x[2], deltaB = 0.2, bM = 
0.03, dM = 0.03, alphaM = 0.69, kappaM = x[3])

 State - c(SB = 100, IB = 0, SM = 1500, EM = 0, IM = 0.0001)
 Time - seq(0, 60, by = 1)
 model.out - as.data.frame(ode(func = wnv.incubation,
   y = State,
   parms = Pars,
   times = Time))
 model.out[nrow(model.out),]$IB
}

# Finally, here is my apply() function:
dat - apply(expand.grid(av, bv, kv), 1, wnv.sim)

However, the apply() function returns an error:
Error in eval(expr, envir, enclos) : object 'alphaB' not found

I am not sure what is wrong.  Any help would be much appreciated.

Sincerely,
Adam

--
Adam Zeilinger
Post Doctoral Scholar
Department of Entomology
University of California Riverside
www.linkedin.com/in/adamzeilinger

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply() with a function involving ode()

[Berend Hasselman]

See inline comments.

On 25-04-2012, at 08:40, Adam Zeilinger wrote:

 Hello,
 
 I am trying to get the output from the numerical simulation of a system of 
 ordinary differential equations for a range of values for three parameters.  
 I am using the ode() function (deSolve package) to run the numerical 
 simulation and apply() to run the simulation function for each set of 
 parameter values.  I am having trouble getting the apply() function to work.
 
 Here is an example.  Consider an epidemiology model of West Nile Virus:
 
 wnv.model - function(Time, State, Pars){
  with(as.list(c(State, Pars)), {
dSB - -(alphaB*betaB*SB*IM)/(SB + IB)
dIB - (alphaB*betaB*SB*IM)/(SB + IB) - deltaB*IB
dSM - bM*(SM + EM + IM) - (alphaM*betaB*IB*SM)/(SB + IB) 
 - dM*SM
dEM - (alphaM*betaB*SM*IB)/(SB + IB) - kappaM*EM - dM*EM
dIM - kappaM*EM - dM*IM
return(list(c(dSB, dIB, dSM, dEM, dIM)))
  })
 }
 
 # I would like to run a numerical simulation of this model for each 
 combination of values for the parameters alphaB, betaB, # and kappaM:
 
 av - seq(0, 1, by = 0.2) # vector of values for alphaB
 bv - seq(0, 1, by = 0.2) # vector of values for betaB
 kv - seq(0, 1, by = 0.2) # vector of values for kappaM
 
 # Here is my function with ode() for the numerical simulation.  The function 
 returns the last value of the simulation for the # IB state variable 
 (Infected Birds).
 
 library(deSolve)
 wnv.sim - function(x){
 Pars - c(alphaB = x[1], betaB = x[2], deltaB = 0.2, bM = 0.03, dM = 
 0.03, alphaM = 0.69, kappaM = x[3])
 State - c(SB = 100, IB = 0, SM = 1500, EM = 0, IM = 0.0001)
 Time - seq(0, 60, by = 1)
 model.out - as.data.frame(ode(func = wnv.incubation,

There is no function wnv.incubation.
Assuming you mean wnv.model

   y = State,
   parms = Pars,
   times = Time))
 model.out[nrow(model.out),]$IB
 }
 
 # Finally, here is my apply() function:
 dat - apply(expand.grid(av, bv, kv), 1, wnv.sim)
 
 However, the apply() function returns an error:
 Error in eval(expr, envir, enclos) : object 'alphaB' not found

Have you inspected the object returned by expand.grid?
Do head(expand.grid(av,bv,kv))
and you will see that the columns have names which confuse the rest of your 
code.

I was able to repair by doing

pars.grid - expand.grid(av, bv, kv)
names(pars.grid) - NULL
dat - apply(pars.grid, 1, wnv.sim)

HTH,

Berend
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply() with a function involving ode()

[Adam Zeilinger]

Dear Berend,

Yes, the wnv.sim function should have included wnv.model instead of 
wnv.incubation.  Sorry for the typo.


Although I had inspected the expand.grid() object, I had not connected 
the error with a naming problem.  Thanks so much for the help!  The 
function works perfectly.


Sincerely,
Adam


On 4/25/2012 2:02 AM, Berend Hasselman wrote:

See inline comments.

On 25-04-2012, at 08:40, Adam Zeilinger wrote:


Hello,

I am trying to get the output from the numerical simulation of a system of 
ordinary differential equations for a range of values for three parameters.  I 
am using the ode() function (deSolve package) to run the numerical simulation 
and apply() to run the simulation function for each set of parameter values.  I 
am having trouble getting the apply() function to work.

Here is an example.  Consider an epidemiology model of West Nile Virus:

wnv.model- function(Time, State, Pars){
  with(as.list(c(State, Pars)), {
dSB- -(alphaB*betaB*SB*IM)/(SB + IB)
dIB- (alphaB*betaB*SB*IM)/(SB + IB) - deltaB*IB
dSM- bM*(SM + EM + IM) - (alphaM*betaB*IB*SM)/(SB + IB) - 
dM*SM
dEM- (alphaM*betaB*SM*IB)/(SB + IB) - kappaM*EM - dM*EM
dIM- kappaM*EM - dM*IM
return(list(c(dSB, dIB, dSM, dEM, dIM)))
  })
}

# I would like to run a numerical simulation of this model for each combination 
of values for the parameters alphaB, betaB, # and kappaM:

av- seq(0, 1, by = 0.2) # vector of values for alphaB
bv- seq(0, 1, by = 0.2) # vector of values for betaB
kv- seq(0, 1, by = 0.2) # vector of values for kappaM

# Here is my function with ode() for the numerical simulation.  The function returns the last value 
of the simulation for the # IB state variable (Infected Birds).

library(deSolve)
wnv.sim- function(x){
 Pars- c(alphaB = x[1], betaB = x[2], deltaB = 0.2, bM = 0.03, dM = 
0.03, alphaM = 0.69, kappaM = x[3])
 State- c(SB = 100, IB = 0, SM = 1500, EM = 0, IM = 0.0001)
 Time- seq(0, 60, by = 1)
 model.out- as.data.frame(ode(func = wnv.incubation,

There is no function wnv.incubation.
Assuming you mean wnv.model


   y = State,
   parms = Pars,
   times = Time))
 model.out[nrow(model.out),]$IB
}

# Finally, here is my apply() function:
dat- apply(expand.grid(av, bv, kv), 1, wnv.sim)

However, the apply() function returns an error:
Error in eval(expr, envir, enclos) : object 'alphaB' not found

Have you inspected the object returned by expand.grid?
Do head(expand.grid(av,bv,kv))
and you will see that the columns have names which confuse the rest of your 
code.

I was able to repair by doing

pars.grid- expand.grid(av, bv, kv)
names(pars.grid)- NULL
dat- apply(pars.grid, 1, wnv.sim)

HTH,

Berend


--
Adam Zeilinger
Post Doctoral Scholar
Department of Entomology
University of California Riverside
www.linkedin.com/in/adamzeilinger

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply function and storing output

[Dennis Murphy]

Hi:

You need to give a function for rollapply() to apply :)

Here's my toy example:

d - as.data.frame(matrix(rpois(30, 5), nrow = 10))
library(zoo)
d1 - zoo(d)   # uses row numbers as index

# rolling means of 3 in each subseries (columns)
 rollmean(d1, 3)
V1   V2   V3
2 3.00 5.33 4.33
3 3.33 4.33 4.00
4 2.67 3.67 3.33
5 4.33 3.67 3.33
6 5.00 5.00 5.00
7 4.67 4.67 4.00
8 5.67 3.67 4.00
9 5.00 3.00 2.67

# create new function
 cv - function(x) sd(x)/mean(x)

# use new function in rollapply():
 rollapply(d1, 3, FUN = cv)
 V1V2V3
2 0.333 0.1082532 0.5329387
3 0.1732051 0.4803845 0.6614378
4 0.2165064 0.5677271 0.4582576
5 0.7418193 0.5677271 0.4582576
6 0.600 0.3464102 0.400
7 0.7525467 0.4948717 0.6614378
8 0.8882158 0.5677271 0.6614378
9 1.0583005 0.333 0.2165064

This is a simplistic example, but it should get you started.

HTH,
Dennis
On Fri, Oct 15, 2010 at 2:00 AM, David A. dasol...@hotmail.com wrote:

  Hi Dennis and Dimitris,

 thanks for your answers. I am trying the rollmean() function and also the
 rollapply() function because I also want to calculate CV for the values.
 For this I created a co.var() function. I am having problems using them.

 co.var-function(x)(
 +sd(x)/mean(x)
 +)


  dim(mydata)
 [1] 1710  244

 xxx-rollmean(mydata,3,by=3)

 works fine and creates a vector which I will transform into a matrix. I
 still have to find out how to store the output in my previously created
 570x244 0's matrix in an ordered way.

 but, since the examples in the help page says it´s the same, I tried

  xxx-rollapply(mydata,3,mean,by=3)
 Error in UseMethod(rollapply) :
   no applicable method for 'rollapply' applied to an object of class
 c('matrix', 'integer', 'numeric')

 and, with my created function...

  xxx-rollapply(ord_raw_filt.df,3,FUN=co.var,by=3)
 Error in UseMethod(rollapply) :
   no applicable method for 'rollapply' applied to an object of class
 c('matrix', 'integer', 'numeric')

 Can you help me with the error?

 Dave

 --
 Date: Fri, 15 Oct 2010 00:45:08 -0700
 Subject: Re: [R] using apply function and storing output
 From: djmu...@gmail.com
 To: dasol...@hotmail.com
 CC: r-help@r-project.org


 Hi:

 Look into the rollmean() function in package zoo.

 HTH,
 Dennis

 On Fri, Oct 15, 2010 at 12:34 AM, David A. dasol...@hotmail.com wrote:


 Hi list,

 I have a 1710x244 matrix of numerical values and I would like to calculate
 the mean of every group of three consecutive values per column to obtain a
 new matrix of 570x244.  I could get it done using a for loop but how can I
 do that using apply functions?
 In addition to this, do I have to initizalize a 570x244 matrix with 0's to
 store the calculated values or can the output matrix be generated while
 calculating the mean values?

 Cheers,

 Dave

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply function and storing output

[David A.]

Thanks Dennis,

I don't think it was a problem of not feeding in a function for rollapply(), 
because I was using mean() and my co.var() function in the FUN argument. 
The key part seems to be the transformation that zoo() does to the matrix. If I 
do the same transformation to my original matrix, the rollapply() function now 
works fine.

Thanks again

David

Date: Fri, 15 Oct 2010 03:00:27 -0700
Subject: Re: [R] using apply function and storing output
From: djmu...@gmail.com
To: dasol...@hotmail.com
CC: r-help@r-project.org

Hi:

You need to give a function for rollapply() to apply :)

Here's my toy example:

d - as.data.frame(matrix(rpois(30, 5), nrow = 10))
library(zoo)
d1 - zoo(d)   # uses row numbers as index


# rolling means of 3 in each subseries (columns)
 rollmean(d1, 3)
V1   V2   V3
2 3.00 5.33 4.33
3 3.33 4.33 4.00
4 2.67 3.67 3.33
5 4.33 3.67 3.33

6 5.00 5.00 5.00
7 4.67 4.67 4.00
8 5.67 3.67 4.00
9 5.00 3.00 2.67

# create new function
 cv - function(x) sd(x)/mean(x)

# use new function in rollapply():

 rollapply(d1, 3, FUN = cv)
 V1V2V3
2 0.333 0.1082532 0.5329387
3 0.1732051 0.4803845 0.6614378
4 0.2165064 0.5677271 0.4582576
5 0.7418193 0.5677271 0.4582576
6 0.600 0.3464102 0.400

7 0.7525467 0.4948717 0.6614378
8 0.8882158 0.5677271 0.6614378
9 1.0583005 0.333 0.2165064

This is a simplistic example, but it should get you started.

HTH,
Dennis
On Fri, Oct 15, 2010 at 2:00 AM, David A. dasol...@hotmail.com wrote:






Hi Dennis and Dimitris,

thanks for your answers. I am trying the rollmean() function and also the 
rollapply() function because I also want to calculate CV for the values.
For this I created a co.var() function. I am having problems using them.


co.var-function(x)(
+sd(x)/mean(x)
+)


 dim(mydata)
[1] 1710  244

xxx-rollmean(mydata,3,by=3) 

works fine and creates a vector which I will transform into a matrix. I still 
have to find out how to store the output in my previously created 570x244 0's 
matrix in an ordered way.


but, since the examples in the help page says it´s the same, I tried

 xxx-rollapply(mydata,3,mean,by=3)
Error in UseMethod(rollapply) : 
  no applicable method for 'rollapply' applied to an object of class 
c('matrix', 'integer', 'numeric')


and, with my created function...

 xxx-rollapply(ord_raw_filt.df,3,FUN=co.var,by=3)
Error in UseMethod(rollapply) : 
  no applicable method for 'rollapply' applied to an object of class 
c('matrix', 'integer', 'numeric')


Can you help me with the error?

Dave

Date: Fri, 15 Oct 2010 00:45:08 -0700
Subject: Re: [R] using apply function and storing output
From: djmu...@gmail.com

To: dasol...@hotmail.com
CC: r-help@r-project.org

Hi:


Look into the rollmean() function in package zoo.

HTH,
Dennis

On Fri, Oct 15, 2010 at 12:34 AM, David A. dasol...@hotmail.com wrote:




Hi list,



I have a 1710x244 matrix of numerical values and I would like to calculate the 
mean of every group of three consecutive values per column to obtain a new 
matrix of 570x244.  I could get it done using a for loop but how can I do that 
using apply functions?



In addition to this, do I have to initizalize a 570x244 matrix with 0's to 
store the calculated values or can the output matrix be generated while 
calculating the mean values?



Cheers,



Dave



[[alternative HTML version deleted]]



__

R-help@r-project.org mailing list



PLEASE do read the posting guide http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.


  

  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply function and storing output

[Gabor Grothendieck]

On Fri, Oct 15, 2010 at 6:46 AM, David A. dasol...@hotmail.com wrote:

 Thanks Dennis,

 I don't think it was a problem of not feeding in a function for rollapply(), 
 because I was using mean() and my co.var() function in the FUN argument.
 The key part seems to be the transformation that zoo() does to the matrix. If 
 I do the same transformation to my original matrix, the rollapply() function 
 now works fine.


rollapply has ts and zoo methods so you have to pass it an object of
one those classes:

 methods(rollapply)
[1] rollapply.ts*  rollapply.zoo*

   Non-visible functions are asterisked


-- 
Statistics  Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] using apply function and storing output

[David A.]

Hi list,

I have a 1710x244 matrix of numerical values and I would like to calculate the 
mean of every group of three consecutive values per column to obtain a new 
matrix of 570x244.  I could get it done using a for loop but how can I do that 
using apply functions?
In addition to this, do I have to initizalize a 570x244 matrix with 0's to 
store the calculated values or can the output matrix be generated while 
calculating the mean values?

Cheers,

Dave
  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply function and storing output

[Dennis Murphy]

Hi:

Look into the rollmean() function in package zoo.

HTH,
Dennis

On Fri, Oct 15, 2010 at 12:34 AM, David A. dasol...@hotmail.com wrote:


 Hi list,

 I have a 1710x244 matrix of numerical values and I would like to calculate
 the mean of every group of three consecutive values per column to obtain a
 new matrix of 570x244.  I could get it done using a for loop but how can I
 do that using apply functions?
 In addition to this, do I have to initizalize a 570x244 matrix with 0's to
 store the calculated values or can the output matrix be generated while
 calculating the mean values?

 Cheers,

 Dave

[[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply function and storing output

[Dimitris Rizopoulos]

one efficient way to do this, avoiding loops, is using the rowsum() 
function, e.g.,


mat - matrix(rnorm(1710*244), 1710, 244)

id - rep(seq_len(570), each = 3)
means - rowsum(mat, id, FALSE) / 3

Regarding the second part of your question, indeed a gold rule of 
efficient R programming when it comes to loops is to pre-allocate the 
output object. Check for instance,


system.time({
x - NULL
for (i in 1:5e04) x - c(x, rnorm(1))
})

versus

system.time({
x - numeric(5e04)
for (i in 1:5e04) x[i] - rnorm(1)
})


I hope it helps.

Best,
Dimitris


On 10/15/2010 9:34 AM, David A. wrote:


Hi list,

I have a 1710x244 matrix of numerical values and I would like to calculate the 
mean of every group of three consecutive values per column to obtain a new 
matrix of 570x244.  I could get it done using a for loop but how can I do that 
using apply functions?
In addition to this, do I have to initizalize a 570x244 matrix with 0's to 
store the calculated values or can the output matrix be generated while 
calculating the mean values?

Cheers,

Dave

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



--
Dimitris Rizopoulos
Assistant Professor
Department of Biostatistics
Erasmus University Medical Center

Address: PO Box 2040, 3000 CA Rotterdam, the Netherlands
Tel: +31/(0)10/7043478
Fax: +31/(0)10/7043014
Web: http://www.erasmusmc.nl/biostatistiek/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply function and storing output

[David A.]

Hi Dennis and Dimitris,

thanks for your answers. I am trying the rollmean() function and also the 
rollapply() function because I also want to calculate CV for the values.
For this I created a co.var() function. I am having problems using them.

co.var-function(x)(
+sd(x)/mean(x)
+)


 dim(mydata)
[1] 1710  244

xxx-rollmean(mydata,3,by=3) 

works fine and creates a vector which I will transform into a matrix. I still 
have to find out how to store the output in my previously created 570x244 0's 
matrix in an ordered way.

but, since the examples in the help page says it´s the same, I tried

 xxx-rollapply(mydata,3,mean,by=3)
Error in UseMethod(rollapply) : 
  no applicable method for 'rollapply' applied to an object of class 
c('matrix', 'integer', 'numeric')

and, with my created function...

 xxx-rollapply(ord_raw_filt.df,3,FUN=co.var,by=3)
Error in UseMethod(rollapply) : 
  no applicable method for 'rollapply' applied to an object of class 
c('matrix', 'integer', 'numeric')

Can you help me with the error?

Dave

Date: Fri, 15 Oct 2010 00:45:08 -0700
Subject: Re: [R] using apply function and storing output
From: djmu...@gmail.com
To: dasol...@hotmail.com
CC: r-help@r-project.org

Hi:

Look into the rollmean() function in package zoo.

HTH,
Dennis

On Fri, Oct 15, 2010 at 12:34 AM, David A. dasol...@hotmail.com wrote:



Hi list,



I have a 1710x244 matrix of numerical values and I would like to calculate the 
mean of every group of three consecutive values per column to obtain a new 
matrix of 570x244.  I could get it done using a for loop but how can I do that 
using apply functions?


In addition to this, do I have to initizalize a 570x244 matrix with 0's to 
store the calculated values or can the output matrix be generated while 
calculating the mean values?



Cheers,



Dave



[[alternative HTML version deleted]]



__

R-help@r-project.org mailing list



PLEASE do read the posting guide http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.


  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply for logical conditions

[Alastair]

Hi,

I've got some boolean data in a data.frame in the form:
  XYZA   B   C
[1]  T TFT   F   F
[2]  F TTF   F   F
.
.
.


What I want to do is create a new column which is the logical disjunction of
several of the columns.
Just like:

new.column - data$X | data$Y | data$Z

However I don't want to hard code the particular columns into the expression
like that. I've tried using apply row wise with `|` as the function:

columns - c(X,Y,Z)
apply(data[,columns], 1,`|`)

This doesn't seem to do what I would have expected, does anyone have any
advice how to use the the apply or similar function to perform a boolean
operation on each row (and a specific subset of the columns) in a data
frame? 

Thanks,
Alastair


-- 
View this message in context: 
http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2310929.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Allan Engelhardt]

`|` is a binary operator which is why the apply will not work.  See

help(Reduce)

For example,

set.seed(1)
data - data.frame(A = runif(10)  0.5, B = runif(10)  0.5, C = 
runif(10)  0.5)

Reduce(`|`, data)
#  [1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE

Hope this helps.

Allan

On 02/08/10 21:35, Alastair wrote:

Hi,

I've got some boolean data in a data.frame in the form:
   XYZA   B   C
[1]  T TFT   F   F
[2]  F TTF   F   F
.
.
.


What I want to do is create a new column which is the logical disjunction of
several of the columns.
Just like:

new.column- data$X | data$Y | data$Z

However I don't want to hard code the particular columns into the expression
like that. I've tried using apply row wise with `|` as the function:

columns- c(X,Y,Z)
apply(data[,columns], 1,`|`)

This doesn't seem to do what I would have expected, does anyone have any
advice how to use the the apply or similar function to perform a boolean
operation on each row (and a specific subset of the columns) in a data
frame?

Thanks,
Alastair





__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Erik Iverson]

Alastair wrote:

Hi,

I've got some boolean data in a data.frame in the form:
  XYZA   B   C
[1]  T TFT   F   F
[2]  F TTF   F   F
.
.
.


What I want to do is create a new column which is the logical disjunction of
several of the columns.
Just like:

new.column - data$X | data$Y | data$Z

However I don't want to hard code the particular columns into the expression
like that. I've tried using apply row wise with `|` as the function:

columns - c(X,Y,Z)
apply(data[,columns], 1,`|`)



Please provide *reproducible* examples.  I cannot run any of your code 
since you don't give us the objects X, Y, or Z.  An easy way to do this 
is to use ?dput on the objects we need to run your code, e.g., your 
data.frame.


Does this do what you want?

df1 - data.frame(x = sample(c(TRUE, FALSE), 10, replace = TRUE),
  y = sample(c(TRUE, FALSE), 10, replace = TRUE),
  z = sample(c(TRUE, FALSE), 10, replace = TRUE))

columns - c(x, y, z)

apply(df1[columns], 1, any)

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Joshua Wiley]

In addition to Reduce(), you can take a look at ?any for '|' and ?all for ''.

Josh

On Mon, Aug 2, 2010 at 1:43 PM, Allan Engelhardt all...@cybaea.com wrote:
 `|` is a binary operator which is why the apply will not work.  See

 help(Reduce)

 For example,

 set.seed(1)
 data - data.frame(A = runif(10)  0.5, B = runif(10)  0.5, C = runif(10) 
 0.5)
 Reduce(`|`, data)
 #  [1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE

 Hope this helps.

 Allan

 On 02/08/10 21:35, Alastair wrote:

 Hi,

 I've got some boolean data in a data.frame in the form:
       X    Y    Z    A   B   C
 [1]  T     T    F    T   F   F
 [2]  F     T    T    F   F   F
 .
 .
 .


 What I want to do is create a new column which is the logical disjunction
 of
 several of the columns.
 Just like:

 new.column- data$X | data$Y | data$Z

 However I don't want to hard code the particular columns into the
 expression
 like that. I've tried using apply row wise with `|` as the function:

 columns- c(X,Y,Z)
 apply(data[,columns], 1,`|`)

 This doesn't seem to do what I would have expected, does anyone have any
 advice how to use the the apply or similar function to perform a boolean
 operation on each row (and a specific subset of the columns) in a data
 frame?

 Thanks,
 Alastair




 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Joshua Wiley
Ph.D. Student, Health Psychology
University of California, Los Angeles
http://www.joshuawiley.com/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Michael Lachmann]

Reduce() is much nicer, but I usually use

rowSums(A)  0 for 'or', and
rowSums(A) == ncols for 'and'.

Which works slightly faster.

I noticed, though, that Reduce() doesn't work on matrices. Is there an
alternative for matrices, or do you have to convert the matrix first to a
data.frame, and then use Reduce?


-- 
View this message in context: 
http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2310991.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Bert Gunter]

Yes, you must do the conversion. The reason is that Reduce requires
its argument x, to be a vector; and a matrix is seen a vector obtained
by columnwise concatenation. e.g.

 Reduce(+,matrix(1:6,nr=3))
[1] 21
 Reduce(+,1:6)
[1] 21

The data frame is seen as a list with elements the columns of the
frame. Whence one concludes that the f argument must be vectorized for
the Reduce to work on the columns of the data frame as you expect.
e.g.

 Reduce(min,data.frame(a=1:3,b=4:6))
[1] 1

but

 Reduce(pmin,data.frame(a=1:3,b=4:6))
[1] 1 2 3


Cheers,

Bert Gunter
Genentech Nonclinical Biostatistics


On Mon, Aug 2, 2010 at 2:08 PM, Michael Lachmann lachm...@eva.mpg.de wrote:

 Reduce() is much nicer, but I usually use

 rowSums(A)  0 for 'or', and
 rowSums(A) == ncols for 'and'.

 Which works slightly faster.

 I noticed, though, that Reduce() doesn't work on matrices. Is there an
 alternative for matrices, or do you have to convert the matrix first to a
 data.frame, and then use Reduce?


 --
 View this message in context: 
 http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2310991.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Joshua Wiley]

On Mon, Aug 2, 2010 at 2:08 PM, Michael Lachmann lachm...@eva.mpg.de wrote:

 Reduce() is much nicer, but I usually use

 rowSums(A)  0 for 'or', and
 rowSums(A) == ncols for 'and'.

 Which works slightly faster.

For the sake of my own curiosity, I compared several of these options,
but in case others are interested.

 boolean - c(TRUE, FALSE, FALSE)

 set.seed(1)
 mydata - data.frame(X = sample(boolean, 10^7, replace = TRUE),
+  Y = sample(boolean, 10^7, replace = TRUE),
+  Z = sample(boolean, 10^7, replace = TRUE))

 system.time(opt1 - apply(mydata, 1, any))
   user  system elapsed
 147.260.42  148.56
 system.time(opt2 - Reduce('|', mydata))
   user  system elapsed
   0.330.000.35
 system.time(opt3 - as.logical(rowSums(mydata, na.rm = TRUE)))
   user  system elapsed
   0.250.000.27
 system.time(opt4 - rowSums(mydata, na.rm = TRUE)  0)
   user  system elapsed
   0.250.000.25

 identical(opt1, opt2)
[1] TRUE
 identical(opt1, opt3)
[1] TRUE
 identical(opt1, opt4)
[1] TRUE

 rm(boolean, mydata, opt1, opt2, opt3, opt4)




 I noticed, though, that Reduce() doesn't work on matrices. Is there an
 alternative for matrices, or do you have to convert the matrix first to a
 data.frame, and then use Reduce?


 --
 View this message in context: 
 http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2310991.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Joshua Wiley
Ph.D. Student, Health Psychology
University of California, Los Angeles
http://www.joshuawiley.com/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Michael Lachmann]

Reduce() is really amazingly fast!

Even with a much larger number of columns, it is still in the same ballpark
(and much more readable):

 boolean - c(TRUE, rep(FALSE,10^3))
 a-matrix(sample(boolean, 10^7, replace = TRUE),10^4,10^3)
 b-data.frame(a)
 system.time({opt4 - rowSums(a, na.rm = TRUE)  0})
   user  system elapsed 
  0.129   0.001   0.131 
 system.time({opt2 - Reduce('|',b)})
   user  system elapsed 
  0.190   0.109   0.303 

and:
 boolean - c(TRUE, rep(FALSE,10^4))
 a-matrix(sample(boolean, 10^7, replace = TRUE),10^3,10^4)
 b-data.frame(a)
 system.time({opt4 - rowSums(a, na.rm = TRUE)  0})
   user  system elapsed 
  0.082   0.001   0.083 
 system.time({opt2 - Reduce('|',b)})
   user  system elapsed 
  0.205   0.001   0.209 

It seems to pretty much make rowSums obsolete, vs. Reduce('+'), except that
it works on lists, and converting a matrix to a data.frame takes ages.

-- 
View this message in context: 
http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2311042.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Bert Gunter]

Just for fun, here are another couple of versions that work for data frames.

For Reduce with |

do.call(pmax,c(mydata,na.rm=TRUE)) 0

and for 

do.call(pmin,c(mydata,na.rm=TRUE)) 0


 Cheers,

Bert Gunter
Genentech Nonclinical Biostatistics


On Mon, Aug 2, 2010 at 2:28 PM, Joshua Wiley jwiley.ps...@gmail.com wrote:
 On Mon, Aug 2, 2010 at 2:08 PM, Michael Lachmann lachm...@eva.mpg.de wrote:

 Reduce() is much nicer, but I usually use

 rowSums(A)  0 for 'or', and
 rowSums(A) == ncols for 'and'.

 Which works slightly faster.

 For the sake of my own curiosity, I compared several of these options,
 but in case others are interested.

 boolean - c(TRUE, FALSE, FALSE)

 set.seed(1)
 mydata - data.frame(X = sample(boolean, 10^7, replace = TRUE),
 +                      Y = sample(boolean, 10^7, replace = TRUE),
 +                      Z = sample(boolean, 10^7, replace = TRUE))

 system.time(opt1 - apply(mydata, 1, any))
   user  system elapsed
  147.26    0.42  148.56
 system.time(opt2 - Reduce('|', mydata))
   user  system elapsed
   0.33    0.00    0.35
 system.time(opt3 - as.logical(rowSums(mydata, na.rm = TRUE)))
   user  system elapsed
   0.25    0.00    0.27
 system.time(opt4 - rowSums(mydata, na.rm = TRUE)  0)
   user  system elapsed
   0.25    0.00    0.25

 identical(opt1, opt2)
 [1] TRUE
 identical(opt1, opt3)
 [1] TRUE
 identical(opt1, opt4)
 [1] TRUE

 rm(boolean, mydata, opt1, opt2, opt3, opt4)




 I noticed, though, that Reduce() doesn't work on matrices. Is there an
 alternative for matrices, or do you have to convert the matrix first to a
 data.frame, and then use Reduce?


 --
 View this message in context: 
 http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2310991.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




 --
 Joshua Wiley
 Ph.D. Student, Health Psychology
 University of California, Los Angeles
 http://www.joshuawiley.com/

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for logical conditions

[Alastair]

Wow,

Thanks for all the excellent (and fast) responses. That's really helped.
Sorry I didn't supply a cut and paste-able example (noted for future
reference) but your examples caught the essence of my problem.  

I ended up opting for the apply any solution. But I'll bear the Reduce
function in mind. 

Thanks,
Alastair
-- 
View this message in context: 
http://r.789695.n4.nabble.com/Using-apply-for-logical-conditions-tp2310929p2311079.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply function on duplicates in a data.frame

[Sunny Srivastava]

Dear R-Helpers,
I have a data.frame (df) and the head of data.frame looks like

 ProbeUID ControlType ProbeName GeneName SystematicName
1665 1577   0 pSysX_50_22_1 pSysX_50   pSysX_50
5422 5147   0  pSysX_49_8_1 pSysX_49   pSysX_49
4042 3843   0 pSysX_51_18_1 pSysX_51   pSysX_51
3646 3466   0   sll1514_0_2  sll1514sll1514
2946 2807   0   sll1514_0_1  sll1514sll1514
624   582   0  pSysX_49_8_2 pSysX_49   pSysX_49

 Description   logFC AveExpr   tP.Value  adj.P.Val
1665 Unknown  4.3887  9.5662  61.038 1.0938e-08 9.4449e-05
5422 Unknown -3.5251  6.9103 -35.908 1.7596e-07 3.5912e-04
4042 Unknown  2.5302  8.7497  35.112 1.9786e-07 3.5912e-04
3646 Unknown  2.3457 11.1678  33.962 2.3549e-07 3.5912e-04
2946 Unknown  2.3151 11.3153  32.689 2.8751e-07 3.5912e-04
624  Unknown -3.6256  6.8986 -31.777 3.e-07 3.5912e-04
  B
1665 9.8342
5422 8.1650
4042 8.0758
3646 7.9408
2946 7.7822
624  7.6622

I want to collapse this data frame into a new data.frame so that the
df$GeneName contains no duplicate GeneNames (for eg: sll1514) AND the
df$logFC contains the average of df$logFC corresponding to these GeneNames
(which had duplicate genenames).

I am aware of an inefficient strategy using loops, but I believe that there
should be a way using Apply functions or may be plyr?

I am not able to think of one at the moment.  Can you please help me?

Any help is appreciated !


Thanks and Best Regards,
S.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function on duplicates in a data.frame

[David Winsemius]

On Jan 31, 2010, at 6:05 PM, Sunny Srivastava wrote:


Dear R-Helpers,
I have a data.frame (df) and the head of data.frame looks like

ProbeUID ControlType ProbeName GeneName SystematicName
1665 1577   0 pSysX_50_22_1 pSysX_50   pSysX_50
5422 5147   0  pSysX_49_8_1 pSysX_49   pSysX_49
4042 3843   0 pSysX_51_18_1 pSysX_51   pSysX_51
3646 3466   0   sll1514_0_2  sll1514sll1514
2946 2807   0   sll1514_0_1  sll1514sll1514
624   582   0  pSysX_49_8_2 pSysX_49   pSysX_49

Description   logFC AveExpr   tP.Value  adj.P.Val
1665 Unknown  4.3887  9.5662  61.038 1.0938e-08 9.4449e-05
5422 Unknown -3.5251  6.9103 -35.908 1.7596e-07 3.5912e-04
4042 Unknown  2.5302  8.7497  35.112 1.9786e-07 3.5912e-04
3646 Unknown  2.3457 11.1678  33.962 2.3549e-07 3.5912e-04
2946 Unknown  2.3151 11.3153  32.689 2.8751e-07 3.5912e-04
624  Unknown -3.6256  6.8986 -31.777 3.e-07 3.5912e-04
 B
1665 9.8342
5422 8.1650
4042 8.0758
3646 7.9408
2946 7.7822
624  7.6622



tdf - tapply(df$logFC, df$GeneName, mean)
ndf - dataframe(Gnames = names(tdf), mn.logFC= tdf)


I want to collapse this data frame into a new data.frame so that the
df$GeneName contains no duplicate GeneNames (for eg: sll1514) AND the
df$logFC contains the average of df$logFC corresponding to these  
GeneNames

(which had duplicate genenames).

I am aware of an inefficient strategy using loops, but I believe  
that there

should be a way using Apply functions or may be plyr?

I am not able to think of one at the moment.  Can you please help me?


David Winsemius, MD
Heritage Laboratories
West Hartford, CT

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function on duplicates in a data.frame

[hadley wickham]

On Sun, Jan 31, 2010 at 5:05 PM, Sunny Srivastava
research.b...@gmail.com wrote:
 Dear R-Helpers,
 I have a data.frame (df) and the head of data.frame looks like

     ProbeUID ControlType     ProbeName GeneName SystematicName
 1665     1577           0 pSysX_50_22_1 pSysX_50       pSysX_50
 5422     5147           0  pSysX_49_8_1 pSysX_49       pSysX_49
 4042     3843           0 pSysX_51_18_1 pSysX_51       pSysX_51
 3646     3466           0   sll1514_0_2  sll1514        sll1514
 2946     2807           0   sll1514_0_1  sll1514        sll1514
 624       582           0  pSysX_49_8_2 pSysX_49       pSysX_49

     Description   logFC AveExpr       t    P.Value  adj.P.Val
 1665     Unknown  4.3887  9.5662  61.038 1.0938e-08 9.4449e-05
 5422     Unknown -3.5251  6.9103 -35.908 1.7596e-07 3.5912e-04
 4042     Unknown  2.5302  8.7497  35.112 1.9786e-07 3.5912e-04
 3646     Unknown  2.3457 11.1678  33.962 2.3549e-07 3.5912e-04
 2946     Unknown  2.3151 11.3153  32.689 2.8751e-07 3.5912e-04
 624      Unknown -3.6256  6.8986 -31.777 3.e-07 3.5912e-04
          B
 1665 9.8342
 5422 8.1650
 4042 8.0758
 3646 7.9408
 2946 7.7822
 624  7.6622

 I want to collapse this data frame into a new data.frame so that the
 df$GeneName contains no duplicate GeneNames (for eg: sll1514) AND the
 df$logFC contains the average of df$logFC corresponding to these GeneNames
 (which had duplicate genenames).

library(plyr)
ddply(df, GeneName, summarise, logFC = mean(logFC)

Hadley


-- 
http://had.co.nz/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply function on duplicates in a data.frame

[Sunny Srivastava]

Thanks a lot.

All the three approaches work for me!

On Sun, Jan 31, 2010 at 10:43 PM, hadley wickham h.wick...@gmail.comwrote:

 On Sun, Jan 31, 2010 at 5:05 PM, Sunny Srivastava
 research.b...@gmail.com wrote:
  Dear R-Helpers,
  I have a data.frame (df) and the head of data.frame looks like
 
  ProbeUID ControlType ProbeName GeneName SystematicName
  1665 1577   0 pSysX_50_22_1 pSysX_50   pSysX_50
  5422 5147   0  pSysX_49_8_1 pSysX_49   pSysX_49
  4042 3843   0 pSysX_51_18_1 pSysX_51   pSysX_51
  3646 3466   0   sll1514_0_2  sll1514sll1514
  2946 2807   0   sll1514_0_1  sll1514sll1514
  624   582   0  pSysX_49_8_2 pSysX_49   pSysX_49
 
  Description   logFC AveExpr   tP.Value  adj.P.Val
  1665 Unknown  4.3887  9.5662  61.038 1.0938e-08 9.4449e-05
  5422 Unknown -3.5251  6.9103 -35.908 1.7596e-07 3.5912e-04
  4042 Unknown  2.5302  8.7497  35.112 1.9786e-07 3.5912e-04
  3646 Unknown  2.3457 11.1678  33.962 2.3549e-07 3.5912e-04
  2946 Unknown  2.3151 11.3153  32.689 2.8751e-07 3.5912e-04
  624  Unknown -3.6256  6.8986 -31.777 3.e-07 3.5912e-04
   B
  1665 9.8342
  5422 8.1650
  4042 8.0758
  3646 7.9408
  2946 7.7822
  624  7.6622
 
  I want to collapse this data frame into a new data.frame so that the
  df$GeneName contains no duplicate GeneNames (for eg: sll1514) AND the
  df$logFC contains the average of df$logFC corresponding to these
 GeneNames
  (which had duplicate genenames).

 library(plyr)
 ddply(df, GeneName, summarise, logFC = mean(logFC)

 Hadley


 --
 http://had.co.nz/


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply() and scale() in combination

[Armin Raznahan]

Hello

I am an R newbie, and am coming against a couple of problems when I try and
apply the scale function across all the rows of a  matrix.

-
#I have a matrix dt_l.

 str(dt_l)
 num [1:40962, 1:885] 3.04 4.1 3.4 3.58 3.77 ...

#I want to convert the values in each row of this matrix into standardised
values (mean 0, sd 1). I believe the command for doing this is scale(),
with the defaults for this function being to set mean at 0 and sd at 1.
Therefore, my desired end-point is an object with the same dimensions as
dt_l.

So, I use this command..which I think should applying scale to every row of
matrix dt_l:

 dt.stand-apply(dt_l, 1, scale)

#This has led to two different sorts of problem:

(1) On some occasions when I have run the command I get an error warning
which includes the phrase

error: can't allocate region...

(2) On other occasions (when as far as I know I have entered the same
command), it runs. BUT, when I look at the object produced the dimensions
are different from dt_l 

 str(dt.stand_l)
 num [1:885, 1:40962] -0.679 -0.0882 0.1204 1.8571 0.8854 ...



I would be very grateful for any explanations/fixes that can be offered for
the above, or any suggestions of alternative ways of reaching my desired
end-point.

Yours hopefully

Armin

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply() and scale() in combination

[Peter Alspach]

Tena koe Armin

From the help file of apply():

 If each call to 'FUN' returns a vector of length 'n', then 'apply'
 returns an array of dimension 'c(n, dim(X)[MARGIN])' if 'n  1'. 
  
So, one should expect what you got in the second case.  You can, of
course, transpose the result using t().

HTH .

Peter Alspach

 -Original Message-
 From: r-help-boun...@r-project.org 
 [mailto:r-help-boun...@r-project.org] On Behalf Of Armin Raznahan
 Sent: Tuesday, 19 May 2009 9:44 a.m.
 To: r-help@r-project.org
 Subject: [R] Using apply() and scale() in combination
 
 Hello
 
 I am an R newbie, and am coming against a couple of problems 
 when I try and apply the scale function across all the rows 
 of a  matrix.
 
 -
 #I have a matrix dt_l.
 
  str(dt_l)
  num [1:40962, 1:885] 3.04 4.1 3.4 3.58 3.77 ...
 
 #I want to convert the values in each row of this matrix into 
 standardised values (mean 0, sd 1). I believe the command for 
 doing this is scale(), with the defaults for this function 
 being to set mean at 0 and sd at 1.
 Therefore, my desired end-point is an object with the same 
 dimensions as dt_l.
 
 So, I use this command..which I think should applying scale 
 to every row of matrix dt_l:
 
  dt.stand-apply(dt_l, 1, scale)
 
 #This has led to two different sorts of problem:
 
 (1) On some occasions when I have run the command I get an 
 error warning which includes the phrase
 
 error: can't allocate region...
 
 (2) On other occasions (when as far as I know I have entered 
 the same command), it runs. BUT, when I look at the object 
 produced the dimensions are different from dt_l 
 
  str(dt.stand_l)
  num [1:885, 1:40962] -0.679 -0.0882 0.1204 1.8571 0.8854 ...
 
 
 
 I would be very grateful for any explanations/fixes that can 
 be offered for the above, or any suggestions of alternative 
 ways of reaching my desired end-point.
 
 Yours hopefully
 
 Armin
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide 
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 

The contents of this e-mail are confidential and may be subject to legal 
privilege.
 If you are not the intended recipient you must not use, disseminate, 
distribute or
 reproduce all or any part of this e-mail or attachments.  If you have received 
this
 e-mail in error, please notify the sender and delete all material pertaining 
to this
 e-mail.  Any opinion or views expressed in this e-mail are those of the 
individual
 sender and may not represent those of The New Zealand Institute for Plant and
 Food Research Limited.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply to get group means

[Alan Cohen]

Hi all,

I'm trying to improve my R skills and make my programming more efficient and 
succinct.  I can solve the following question, but wonder if there's a better 
way to do it:

I'm trying to calculate mean by several variables and then put this back into 
the original data set as a new variable.  For example, if I were measuring 
weight, I might want to have each individual's weight, and also the group mean 
by, say, race, sex, and geographic region.  The following code works:

 x1-rep(c(A,B,C),3)
 x2-c(rep(1,3),rep(2,3),1,2,1)
 x3-c(1,2,3,4,5,6,2,6,4)
 x-as.data.frame(cbind(x1,x2,x3))
 x3.mean-rep(0,nrow(x))
 for (i in 1:nrow(x)){
+   x3.mean[i]-mean(as.numeric(x[,3][x[,1]==x[,1][i]x[,2]==x[,2][i]]))
+   }  
 cbind(x,x3.mean)
  x1 x2 x3 x3.mean
1  A  1  1 1.5
2  B  1  2 2.0
3  C  1  3 3.5
4  A  2  4 4.0
5  B  2  5 5.5
6  C  2  6 6.0
7  A  1  2 1.5
8  B  2  6 5.5
9  C  1  4 3.5

However, I'd love to be able to do this with apply rather than a for-loop.  
Or is there a built-in function? Any suggestions?

Also, any way to avoid the hassles with having to convert to a data frame and 
then again to numeric when one variable is character?

Cheers,
Alan Cohen

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to get group means

[baptiste auguie]

Not exactly the output you asked for, but perhaps you can consider,

library(doBy)
 summaryBy(x3~x2+x1,data=x,FUN=mean)

  x2 x1 x3.mean
1  1  A 1.5
2  1  B 2.0
3  1  C 3.5
4  2  A 4.0
5  2  B 5.5
6  2  C 6.0



the plyr package also provides similar functionality, as do the ?by, ? 
ave, and ?tapply base functions.


HTH,

baptiste


On 31 Mar 2009, at 17:09, Alan Cohen wrote:


Hi all,

I'm trying to improve my R skills and make my programming more  
efficient and succinct.  I can solve the following question, but  
wonder if there's a better way to do it:


I'm trying to calculate mean by several variables and then put this  
back into the original data set as a new variable.  For example, if  
I were measuring weight, I might want to have each individual's  
weight, and also the group mean by, say, race, sex, and geographic  
region.  The following code works:



x1-rep(c(A,B,C),3)
x2-c(rep(1,3),rep(2,3),1,2,1)
x3-c(1,2,3,4,5,6,2,6,4)
x-as.data.frame(cbind(x1,x2,x3))
x3.mean-rep(0,nrow(x))
for (i in 1:nrow(x)){
+   x3.mean[i]-mean(as.numeric(x[,3][x[,1]==x[,1][i]x[,2]==x[,2] 
[i]]))

+   }

cbind(x,x3.mean)

 x1 x2 x3 x3.mean
1  A  1  1 1.5
2  B  1  2 2.0
3  C  1  3 3.5
4  A  2  4 4.0
5  B  2  5 5.5
6  C  2  6 6.0
7  A  1  2 1.5
8  B  2  6 5.5
9  C  1  4 3.5

However, I'd love to be able to do this with apply rather than a  
for-loop.  Or is there a built-in function? Any suggestions?


Also, any way to avoid the hassles with having to convert to a data  
frame and then again to numeric when one variable is character?


Cheers,
Alan Cohen

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


_

Baptiste Auguié

School of Physics
University of Exeter
Stocker Road,
Exeter, Devon,
EX4 4QL, UK

Phone: +44 1392 264187

http://newton.ex.ac.uk/research/emag

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to get group means

[hadley wickham]

On Tue, Mar 31, 2009 at 11:31 AM, baptiste auguie ba...@exeter.ac.uk wrote:
 Not exactly the output you asked for, but perhaps you can consider,

 library(doBy)
 summaryBy(x3~x2+x1,data=x,FUN=mean)

  x2 x1 x3.mean
 1  1  A     1.5
 2  1  B     2.0
 3  1  C     3.5
 4  2  A     4.0
 5  2  B     5.5
 6  2  C     6.0


 the plyr package also provides similar functionality, as do the ?by, ?ave,
 and ?tapply base functions.

In plyr it would look like:

x1 - rep(c(A, B, C), 3)
x2 - c(rep(1, 3), rep(2, 3), 1, 2, 1)
x3 - c(1, 2, 3, 4, 5, 6, 2, 6, 4)
df - data.frame(x1, x2, x3)

ddply(df, .(x1, x2), transform, x3.mean = mean(x3))

Note how I created the data frame - only use cbind if you want a
matrix (i.e. all the columns have the same type)

Hadley


-- 
http://had.co.nz/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to get group means

[Domenico Vistocco]

A different solution (using aggregate for the table of means and merge 
for  adding it to the dataframe):


x1-rep(c(A,B,C),3)
x2-c(rep(1,3),rep(2,3),1,2,1)
x3-c(1,2,3,4,5,6,2,6,4)
x-data.frame(x1,x2,x3) #here using data.frame the x1 variable is directly 
converted to factor


x3means - aggregate(x$x3, by=list(x$x1), FUN=mean)
merge(x, x3means, by.x=x1, by.y=Group.1)


Ciao,
domenico

Alan Cohen wrote:

Hi all,

I'm trying to improve my R skills and make my programming more efficient and 
succinct.  I can solve the following question, but wonder if there's a better 
way to do it:

I'm trying to calculate mean by several variables and then put this back into 
the original data set as a new variable.  For example, if I were measuring 
weight, I might want to have each individual's weight, and also the group mean 
by, say, race, sex, and geographic region.  The following code works:

  

x1-rep(c(A,B,C),3)
x2-c(rep(1,3),rep(2,3),1,2,1)
x3-c(1,2,3,4,5,6,2,6,4)
x-as.data.frame(cbind(x1,x2,x3))
x3.mean-rep(0,nrow(x))
for (i in 1:nrow(x)){


+   x3.mean[i]-mean(as.numeric(x[,3][x[,1]==x[,1][i]x[,2]==x[,2][i]]))
+   }  
  

cbind(x,x3.mean)


  x1 x2 x3 x3.mean
1  A  1  1 1.5
2  B  1  2 2.0
3  C  1  3 3.5
4  A  2  4 4.0
5  B  2  5 5.5
6  C  2  6 6.0
7  A  1  2 1.5
8  B  2  6 5.5
9  C  1  4 3.5

However, I'd love to be able to do this with apply rather than a for-loop.  
Or is there a built-in function? Any suggestions?

Also, any way to avoid the hassles with having to convert to a data frame and 
then again to numeric when one variable is character?

Cheers,
Alan Cohen

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.




__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to get group means

[Domenico Vistocco]

Sorry, there was a mistake in the previous mail:

Domenico Vistocco wrote:
A different solution (using aggregate for the table of means and merge 
for  adding it to the dataframe):


x1-rep(c(A,B,C),3)
x2-c(rep(1,3),rep(2,3),1,2,1)
x3-c(1,2,3,4,5,6,2,6,4)
x-data.frame(x1,x2,x3) #here using data.frame the x1 variable is 
directly converted to factor



x3means - aggregate(x$x3, by=list(x$x1), FUN=mean)
merge(x, x3means, by.x=x1, by.y=Group.1)
#I forgot the second variable in the by argument (both for aggregate and 
by):

x3means - aggregate(x$x3, by=list(x$x1, x$x2), FUN=mean)
merge(x, x3means, by.x=c(x1,x2), by.y=c(Group.1, Group.2))




Ciao,
domenico

Alan Cohen wrote:

Hi all,

I'm trying to improve my R skills and make my programming more 
efficient and succinct.  I can solve the following question, but 
wonder if there's a better way to do it:


I'm trying to calculate mean by several variables and then put this 
back into the original data set as a new variable.  For example, if I 
were measuring weight, I might want to have each individual's weight, 
and also the group mean by, say, race, sex, and geographic region.  
The following code works:


 

x1-rep(c(A,B,C),3)
x2-c(rep(1,3),rep(2,3),1,2,1)
x3-c(1,2,3,4,5,6,2,6,4)
x-as.data.frame(cbind(x1,x2,x3))
x3.mean-rep(0,nrow(x))
for (i in 1:nrow(x)){


+   x3.mean[i]-mean(as.numeric(x[,3][x[,1]==x[,1][i]x[,2]==x[,2][i]]))
+   }   

cbind(x,x3.mean)


  x1 x2 x3 x3.mean
1  A  1  1 1.5
2  B  1  2 2.0
3  C  1  3 3.5
4  A  2  4 4.0
5  B  2  5 5.5
6  C  2  6 6.0
7  A  1  2 1.5
8  B  2  6 5.5
9  C  1  4 3.5

However, I'd love to be able to do this with apply rather than a 
for-loop.  Or is there a built-in function? Any suggestions?


Also, any way to avoid the hassles with having to convert to a data 
frame and then again to numeric when one variable is character?


Cheers,
Alan Cohen

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.




__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to get group means

[David Winsemius]

That is precisely the reason for the existence of the ave function.  
Using Wickham's example:


 x1 - rep(c(A, B, C), 3)
 x2 - c(rep(1, 3), rep(2, 3), 1, 2, 1)
 x3 - c(1, 2, 3, 4, 5, 6, 2, 6, 4)
 df - data.frame(x1, x2, x3)
 df$grpx3 - ave(df$x3, list(x1,x2))
 df
  x1 x2 x3 grpx3
1  A  1  1   1.5
2  B  1  2   2.0
3  C  1  3   3.5
4  A  2  4   4.0
5  B  2  5   5.5
6  C  2  6   6.0
7  A  1  2   1.5
8  B  2  6   5.5
9  C  1  4   3.5

Note that the default function is mean() but other functions could be  
specified.



--
David Winsemius

On Mar 31, 2009, at 12:09 PM, Alan Cohen wrote:


Hi all,

I'm trying to improve my R skills and make my programming more  
efficient and succinct.  I can solve the following question, but  
wonder if there's a better way to do it:


I'm trying to calculate mean by several variables and then put this  
back into the original data set as a new variable.  For example, if  
I were measuring weight, I might want to have each individual's  
weight, and also the group mean by, say, race, sex, and geographic  
region.  The following code works:



x1-rep(c(A,B,C),3)
x2-c(rep(1,3),rep(2,3),1,2,1)
x3-c(1,2,3,4,5,6,2,6,4)
x-as.data.frame(cbind(x1,x2,x3))
x3.mean-rep(0,nrow(x))
for (i in 1:nrow(x)){
+   x3.mean[i]-mean(as.numeric(x[,3][x[,1]==x[,1][i]x[,2]==x[,2] 
[i]]))

+   }

cbind(x,x3.mean)

 x1 x2 x3 x3.mean
1  A  1  1 1.5
2  B  1  2 2.0
3  C  1  3 3.5
4  A  2  4 4.0
5  B  2  5 5.5
6  C  2  6 6.0
7  A  1  2 1.5
8  B  2  6 5.5
9  C  1  4 3.5

However, I'd love to be able to do this with apply rather than a  
for-loop.  Or is there a built-in function? Any suggestions?


Also, any way to avoid the hassles with having to convert to a data  
frame and then again to numeric when one variable is character?


Cheers,
Alan Cohen


David Winsemius, MD
Heritage Laboratories
West Hartford, CT

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply to generate matrix from rows?

[Stephan Lindner]

Dear all,


I have a simple question which I unfortunately do not seem to be able
to solve myself. I have a (NxK) matrix and want to generate a new
matrix by multiplying each row with itself such that the new matrix
has dimension ((N*K)xK) (or better, generate an array with dimension
(K,K,N)). I tried apply, but that did not work. Any suggestions? 

Thanks! 


Stephan



## Here is a simple example: 

u - matrix(c(1,2,3,4,5,6,7,8,9,10),nrow=2)

## What I want to obtain 

u[1,]%*%t(u[1,])
u[2,]%*%t(u[2,])

## stacked together -- 10x5 matrix


## This does not work 

sq - function(x)x%*%t(x)
apply(u,1,function(y)sq(y))






-- 
---
Stephan Lindner
University of Michigan

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to generate matrix from rows?

[Henrique Dallazuanna]

Try this:

matrix(apply(u, 1, tcrossprod), nr = nrow(u)*ncol(u), byrow = T)

On Mon, Jan 19, 2009 at 3:39 PM, Stephan Lindner lindn...@umich.edu wrote:

 Dear all,


 I have a simple question which I unfortunately do not seem to be able
 to solve myself. I have a (NxK) matrix and want to generate a new
 matrix by multiplying each row with itself such that the new matrix
 has dimension ((N*K)xK) (or better, generate an array with dimension
 (K,K,N)). I tried apply, but that did not work. Any suggestions?

 Thanks!


Stephan



 ## Here is a simple example:

 u - matrix(c(1,2,3,4,5,6,7,8,9,10),nrow=2)

 ## What I want to obtain

 u[1,]%*%t(u[1,])
 u[2,]%*%t(u[2,])

 ## stacked together -- 10x5 matrix


 ## This does not work

 sq - function(x)x%*%t(x)
 apply(u,1,function(y)sq(y))






 --
 ---
 Stephan Lindner
 University of Michigan

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Henrique Dallazuanna
Curitiba-Paraná-Brasil
25° 25' 40 S 49° 16' 22 O

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply to generate matrix from rows?

[Jorge Ivan Velez]

Dear Stephan,
Try this:

do.call(rbind,lapply(1:2,function(x) matrix(u[x,]%*%t(u[x,]),ncol=ncol(u

HTH,

Jorge


On Mon, Jan 19, 2009 at 12:39 PM, Stephan Lindner lindn...@umich.eduwrote:

 Dear all,


 I have a simple question which I unfortunately do not seem to be able
 to solve myself. I have a (NxK) matrix and want to generate a new
 matrix by multiplying each row with itself such that the new matrix
 has dimension ((N*K)xK) (or better, generate an array with dimension
 (K,K,N)). I tried apply, but that did not work. Any suggestions?

 Thanks!


Stephan



 ## Here is a simple example:

 u - matrix(c(1,2,3,4,5,6,7,8,9,10),nrow=2)

 ## What I want to obtain

 u[1,]%*%t(u[1,])
 u[2,]%*%t(u[2,])

 ## stacked together -- 10x5 matrix


 ## This does not work

 sq - function(x)x%*%t(x)
 apply(u,1,function(y)sq(y))






 --
 ---
 Stephan Lindner
 University of Michigan

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply for two datasets

[Christos Argyropoulos]

It depends on how the data are arranged

##
x-matrix(c(1,2,3,2,8,2,4,5,6),nrow=3)
y-matrix(c(10,2,13,0,8,4,4.2,5.2,6.2),nrow=3)

q-mapply(t.test,as.data.frame(x),as.data.frame(y))

q

## The ith column of q contain the results of applying t.test to 
## the ith column of x and the jth column of y
##

Since the t.test returns a list, you can wrap it in your own function if 
you want to process the data in an assembly line fashion.

Continuing the previous example:

my.t-function(x,y,...) { c(t.test(x,y,...))[1:3] }

q2-mapply(my.t,as.data.frame(x),as.data.frame(y))

q2


Good luck!

Christos Argyropoulos
University of Pittsburgh Medical Center

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Using apply for two datasets

[Gang Chen]

I can run one-sample t-test on an array, for example a matrix myData1,
with the following

apply(myData1, 2, t.test)

Is there a similar fashion using apply() or something else to run
2-sample t-test with datasets from two groups, myData1 and myData2,
without looping?

TIA,
Gang

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for two datasets

[Henrique Dallazuanna]

I think that you can use mapply for this.

On Tue, Jan 6, 2009 at 3:24 PM, Gang Chen gangch...@gmail.com wrote:

 I can run one-sample t-test on an array, for example a matrix myData1,
 with the following

 apply(myData1, 2, t.test)

 Is there a similar fashion using apply() or something else to run
 2-sample t-test with datasets from two groups, myData1 and myData2,
 without looping?

 TIA,
 Gang

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Henrique Dallazuanna
Curitiba-Paraná-Brasil
25° 25' 40 S 49° 16' 22 O

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for two datasets

[Jorge Ivan Velez]

Hi Gang,
Perhaps this post might be useful in this case. Please take a special
lookat Gábor
Csárdi's reply.

http://www.nabble.com/apply,-t-test-and-p-values-to20012292.html#a20012292

HTH,

Jorge


On Tue, Jan 6, 2009 at 1:10 PM, Gang Chen gangch...@gmail.com wrote:

 Thanks a lot for the quick help!

 mapply() seems promising. However, mapply(t.test, myData1, myData2)
 would not work, so how can I specify the margin in mapply() which
 function t.test() will be applied over? For example, I specify the 2nd
 dimension (column) in apply(myData1, 2, t.test) to run one-sample
 t-test. Is there a way I can achieve the same with mapply()?

 Thanks again,
 Gang


 On Tue, Jan 6, 2009 at 12:34 PM, Henrique Dallazuanna www...@gmail.com
 wrote:
  I think that you can use mapply for this.
 
  On Tue, Jan 6, 2009 at 3:24 PM, Gang Chen gangch...@gmail.com wrote:
 
  I can run one-sample t-test on an array, for example a matrix myData1,
  with the following
 
  apply(myData1, 2, t.test)
 
  Is there a similar fashion using apply() or something else to run
  2-sample t-test with datasets from two groups, myData1 and myData2,
  without looping?
 
  TIA,
  Gang

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for two datasets

[Satoshi Takahama]

Perhaps you can convert your matrices to data frames as in:

mapply(t.test,as.data.frame(myData1),as.data.frame(myData2))
to test by column and

mapply(t.test,as.data.frame(t(myData1)),as.data.frame(t(myData2)))


to test by row?


- Original Message 
From: Gang Chen gangch...@gmail.com
To: Henrique Dallazuanna www...@gmail.com
Cc: r-h...@stat.math.ethz.ch
Sent: Tuesday, January 6, 2009 10:10:44 AM
Subject: Re: [R] Using apply for two datasets

Thanks a lot for the quick help!

mapply() seems promising. However, mapply(t.test, myData1, myData2)
would not work, so how can I specify the margin in mapply() which
function t.test() will be applied over? For example, I specify the 2nd
dimension (column) in apply(myData1, 2, t.test) to run one-sample
t-test. Is there a way I can achieve the same with mapply()?

Thanks again,
Gang


On Tue, Jan 6, 2009 at 12:34 PM, Henrique Dallazuanna www...@gmail.com wrote:
 I think that you can use mapply for this.

 On Tue, Jan 6, 2009 at 3:24 PM, Gang Chen gangch...@gmail.com wrote:

 I can run one-sample t-test on an array, for example a matrix myData1,
 with the following

 apply(myData1, 2, t.test)

 Is there a similar fashion using apply() or something else to run
 2-sample t-test with datasets from two groups, myData1 and myData2,
 without looping?

 TIA,
 Gang

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Using apply for two datasets

[Gang Chen]

Thanks a lot for the suggestions, Jorge and Satoshi Takahama! Both
approaches work well...

Gang

On Tue, Jan 6, 2009 at 2:12 PM, Satoshi Takahama s.takah...@yahoo.com wrote:
 Perhaps you can convert your matrices to data frames as in:

 mapply(t.test,as.data.frame(myData1),as.data.frame(myData2))
 to test by column and

 mapply(t.test,as.data.frame(t(myData1)),as.data.frame(t(myData2)))


 to test by row?


 - Original Message 
 From: Gang Chen gangch...@gmail.com
 To: Henrique Dallazuanna www...@gmail.com
 Cc: r-h...@stat.math.ethz.ch
 Sent: Tuesday, January 6, 2009 10:10:44 AM
 Subject: Re: [R] Using apply for two datasets

 Thanks a lot for the quick help!

 mapply() seems promising. However, mapply(t.test, myData1, myData2)
 would not work, so how can I specify the margin in mapply() which
 function t.test() will be applied over? For example, I specify the 2nd
 dimension (column) in apply(myData1, 2, t.test) to run one-sample
 t-test. Is there a way I can achieve the same with mapply()?

 Thanks again,
 Gang


 On Tue, Jan 6, 2009 at 12:34 PM, Henrique Dallazuanna www...@gmail.com 
 wrote:
 I think that you can use mapply for this.

 On Tue, Jan 6, 2009 at 3:24 PM, Gang Chen gangch...@gmail.com wrote:

 I can run one-sample t-test on an array, for example a matrix myData1,
 with the following

 apply(myData1, 2, t.test)

 Is there a similar fashion using apply() or something else to run
 2-sample t-test with datasets from two groups, myData1 and myData2,
 without looping?

 TIA,
 Gang

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] using apply to loop

[Louis Martin]

Hi,

I am running the following loop, but it takes hours to run as n is big. Is 
there any way apply can be used? Thanks.
### Start
nclass - dim(data)[[2]] - 1
z - matrix(0, ncol = nclass, nrow = nclass)
n - dim(data)[[1]]
x - c(1:nclass)
# loop starts
for(loop in 1:n) {
r - data[loop, 1:nclass]
classified - x[r == max(r)]
   
truth - data[loop, nclass + 1]
z[classified, truth] - z[classified, truth] + 1
}
# loop ends


 
-

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply to loop [SEC=UNCLASSIFIED]

[Crombie, Joe]

Hi Louis,

You could try this:

# find the index of the maximum value in each row of _data_, #
disregarding the last column classified -
apply(data[,-(nclass+1)],1,which.max)

## or, if the maximum may be repeated:
classified - apply(data[,-(nclass+1)], 1, FUN = function(x) which(x ==
max(x)))

# the variable _truth_ is just the last column of _data_ ?
truth - data[,nclass + 1]

?table
z - table(classified, truth)


HTH  Joe

 
Joe Crombie
 
Biosecurity and Information Sciences
Bureau of Rural Science
Canberra  Australia
 
e: [EMAIL PROTECTED]
 

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
On Behalf Of Louis Martin
Sent: Friday, 21 December 2007 11:37 AM
To: R-help@r-project.org
Subject: [R] using apply to loop

Hi,

I am running the following loop, but it takes hours to run as n is big.
Is there any way apply can be used? Thanks.
### Start
nclass - dim(data)[[2]] - 1
z - matrix(0, ncol = nclass, nrow = nclass)
n - dim(data)[[1]]
x - c(1:nclass)
# loop starts
for(loop in 1:n) {
r - data[loop, 1:nclass]
classified - x[r == max(r)]
   
truth - data[loop, nclass + 1]
z[classified, truth] - z[classified, truth] + 1
}
# loop ends


 
-

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

--IMPORTANT - This message has been issued by The Department of 
Agriculture, Fisheries and Forestry (DAFF). The information transmitted is for 
the use of the intended recipient only and may contain confidential and/or 
legally privileged material. It is your responsibility to check any attachments 
for viruses and defects before opening or sending them on. 

Any reproduction, publication, communication, re-transmission, disclosure, 
dissemination or other use of the information contained in this e-mail by 
persons or entities other than the intended recipient is prohibited. The taking 
of any action in reliance upon this information by persons or entities other 
than the intended recipient is prohibited. If you have received this e-mail in 
error please notify the sender and delete all copies of this transmission 
together with any attachments. If you have received this e-mail as part of a 
valid mailing list and no longer want to receive a message such as this one 
advise the sender by return e-mail accordingly. Only e-mail correspondence 
which includes this footer, has been authorised by DAFF 
--

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] using apply to loop

[Tyler Smith]

On 2007-12-21, Louis Martin [EMAIL PROTECTED] wrote:
 Hi,

 I am running the following loop, but it takes hours to run as n is big. Is 
 there any way apply can be used? Thanks.
 ### Start
 nclass - dim(data)[[2]] - 1
 z - matrix(0, ncol = nclass, nrow = nclass)
 n - dim(data)[[1]]
 x - c(1:nclass)
 # loop starts
 for(loop in 1:n) { # looping over rows in data
 r - data[loop, 1:nclass] # vector of length(nclass)
 classified - x[r == max(r)] # index of rows == max(r)

 truth - data[loop, nclass + 1] # next column, single value
 z[classified, truth] - z[classified, truth] + 1 # increment
 the values of 
 }
 # loop ends


Off the top, data is a bad choice for your dataframe, as it conflicts
with a standard function. Also, including some actual data would make
this easier to work with. I think you're using dim(data)[[1]] to get
the number of rows of data? That can be more clearly expressed as
nrow(data), and dim(data)[[2]] == ncol(data).

Anyways, this might be helpful:

add.mat - apply(data[,1:nclass], MAR = 1, FUN = function(x) 
 ifelse(x == max(x), 1, 0))

for(i in 1:n)
z[ , data[i, ncol(data)]] - z[ , data[i, ncol(data)]] + add.mat[,i]

There's still a loop, but it might not be needed depending on what
values 'truth' holds. Most of the calculations are shifted into the
apply() call, so the one line loop should run at least a little faster
than what you started with.

HTH,

Tyler

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.