Re: [R] Multivariate regression
Thank you Robin (also thanks to Andris and Bernhard). I did get answers to
my problem using lm as you had suggested. But my main concern is about
getting the appropriate standard errors for \hat{beta}.
My response is actually preference data, where each individual ranks a list
of k items, assigning them unique ranks from 1 to k. Since the ranks for
each item are negatively-correlated within an individual, I would like to
take this into consideration. Although lm gives me correct parameter
estimates, I think that the standard errors are overestimated.
Best,
Ravi.
---
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: [EMAIL PROTECTED]
Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Robin Hankin
Sent: Monday, October 30, 2006 4:17 AM
To: Andris Jankevics
Cc: r-help@stat.math.ethz.ch
Subject: Re: [R] Multivariate regression
Hi
I discovered the other day that lm() does some of the work for
you:
library(mvtnorm)
X - matrix(rnorm(60),ncol=3)
beta - matrix(1:6,ncol=2)
sig - matrix(c(1,0.7,0.7,1),2,2)
Y - X %*% beta + rmvnorm(n=20,sigma=sig)
lm(Y ~ X-1)
Call:
lm(formula = Y ~ X - 1)
Coefficients:
[,1] [,2]
X1 1.015 4.065
X2 2.483 5.366
X3 2.762 5.727
This gives an estimate for beta.
But I don't know of a ready-made R solution for estimating
the covariance of the elements of beta, or the sig matrix
for the covariance matrix of the observation errors.
Anyone?
On 30 Oct 2006, at 09:01, Andris Jankevics wrote:
Also you can take a look on Partial Least Squares (PLS) regression.
http://www.statsoft.com/textbook/stpls.html
R-package: http://mevik.net/work/software/pls.html
Andris Jankevics
On Sestdiena, 28. Oktobris 2006 06:04, Ritwik Sinha wrote:
You can use gee (
http://finzi.psych.upenn.edu/R/library/geepack/html/00Index.html)
or maybe
the function gls in nlme.
Ritwik.
On 10/27/06, Ravi Varadhan [EMAIL PROTECTED] wrote:
Hi,
Suppose I have a multivariate response Y (n x k) obtained at a
set of
predictors X (n x p). I would like to perform a linear
regression taking
into consideration the covariance structure of Y within each unit
- this
would be represented by a specified matrix V (k x k), assumed to
be the
same
across units. How do I use lm to do this?
One approach that I was thinking of is as follows:
Flatten Y to a vector, say, Yvec (n*k x 1). Create Xvec (n*k,
p*k) such
that it is made up of block matrices Bij (k x k), where Bij is a
diagonal
matrix with X_ij as the diagonal (i = 1,.n, and j = 1,.,p). Now
I can
use lm in a univariate mode to regress Yvec against Xvec, with
covariance matrix Vvec (n*k x n*k). Vvec is a block-diagonal
matrix with
blocks of V along the diagonal. This seems like a valid
approach, but I
still don't know how to specify the covariance structure to do
weighted
least squares.
Any help is appreciated.
Best,
Ravi.
-
--- ---
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: [EMAIL PROTECTED]
Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
-
---
[[alternative HTML version deleted]]
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-
guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Robin Hankin
Uncertainty Analyst
National Oceanography Centre, Southampton
European Way, Southampton SO14 3ZH, UK
tel 023-8059-7743
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
Re: [R] Multivariate regression
Hello Ravi,
have you considered the SUR method proposed by Zellner? An
implementation of it is provided in CRAN-package 'systemfit' (see
?systemfit for more information).
Best,
Bernhard
Suppose I have a multivariate response Y (n x k) obtained at a set of
predictors X (n x p). I would like to perform a linear
regression taking
into consideration the covariance structure of Y within each
unit - this
would be represented by a specified matrix V (k x k), assumed
to be the same
across units. How do I use lm to do this?
One approach that I was thinking of is as follows:
Flatten Y to a vector, say, Yvec (n*k x 1). Create Xvec (n*k,
p*k) such
that it is made up of block matrices Bij (k x k), where Bij is
a diagonal
matrix with X_ij as the diagonal (i = 1,.n, and j = 1,.,p).
Now I can use
lm in a univariate mode to regress Yvec against Xvec, with covariance
matrix Vvec (n*k x n*k). Vvec is a block-diagonal matrix with
blocks of V
along the diagonal. This seems like a valid approach, but I
still don't
know how to specify the covariance structure to do weighted
least squares.
Any help is appreciated.
Best,
Ravi.
---
-
---
Ravi Varadhan, Ph.D.
Assistant Professor, The Center on Aging and Health
Division of Geriatric Medicine and Gerontology
Johns Hopkins University
Ph: (410) 502-2619
Fax: (410) 614-9625
Email: [EMAIL PROTECTED]
Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
---
-
[[alternative HTML version deleted]]
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
*
Confidentiality Note: The information contained in this mess...{{dropped}}
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multivariate regression
Also you can take a look on Partial Least Squares (PLS) regression. http://www.statsoft.com/textbook/stpls.html R-package: http://mevik.net/work/software/pls.html Andris Jankevics On Sestdiena, 28. Oktobris 2006 06:04, Ritwik Sinha wrote: You can use gee ( http://finzi.psych.upenn.edu/R/library/geepack/html/00Index.html) or maybe the function gls in nlme. Ritwik. On 10/27/06, Ravi Varadhan [EMAIL PROTECTED] wrote: Hi, Suppose I have a multivariate response Y (n x k) obtained at a set of predictors X (n x p). I would like to perform a linear regression taking into consideration the covariance structure of Y within each unit - this would be represented by a specified matrix V (k x k), assumed to be the same across units. How do I use lm to do this? One approach that I was thinking of is as follows: Flatten Y to a vector, say, Yvec (n*k x 1). Create Xvec (n*k, p*k) such that it is made up of block matrices Bij (k x k), where Bij is a diagonal matrix with X_ij as the diagonal (i = 1,.n, and j = 1,.,p). Now I can use lm in a univariate mode to regress Yvec against Xvec, with covariance matrix Vvec (n*k x n*k). Vvec is a block-diagonal matrix with blocks of V along the diagonal. This seems like a valid approach, but I still don't know how to specify the covariance structure to do weighted least squares. Any help is appreciated. Best, Ravi. - --- --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: [EMAIL PROTECTED] Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html - --- [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multivariate regression
Hi I discovered the other day that lm() does some of the work for you: library(mvtnorm) X - matrix(rnorm(60),ncol=3) beta - matrix(1:6,ncol=2) sig - matrix(c(1,0.7,0.7,1),2,2) Y - X %*% beta + rmvnorm(n=20,sigma=sig) lm(Y ~ X-1) Call: lm(formula = Y ~ X - 1) Coefficients: [,1] [,2] X1 1.015 4.065 X2 2.483 5.366 X3 2.762 5.727 This gives an estimate for beta. But I don't know of a ready-made R solution for estimating the covariance of the elements of beta, or the sig matrix for the covariance matrix of the observation errors. Anyone? On 30 Oct 2006, at 09:01, Andris Jankevics wrote: Also you can take a look on Partial Least Squares (PLS) regression. http://www.statsoft.com/textbook/stpls.html R-package: http://mevik.net/work/software/pls.html Andris Jankevics On Sestdiena, 28. Oktobris 2006 06:04, Ritwik Sinha wrote: You can use gee ( http://finzi.psych.upenn.edu/R/library/geepack/html/00Index.html) or maybe the function gls in nlme. Ritwik. On 10/27/06, Ravi Varadhan [EMAIL PROTECTED] wrote: Hi, Suppose I have a multivariate response Y (n x k) obtained at a set of predictors X (n x p). I would like to perform a linear regression taking into consideration the covariance structure of Y within each unit - this would be represented by a specified matrix V (k x k), assumed to be the same across units. How do I use lm to do this? One approach that I was thinking of is as follows: Flatten Y to a vector, say, Yvec (n*k x 1). Create Xvec (n*k, p*k) such that it is made up of block matrices Bij (k x k), where Bij is a diagonal matrix with X_ij as the diagonal (i = 1,.n, and j = 1,.,p). Now I can use lm in a univariate mode to regress Yvec against Xvec, with covariance matrix Vvec (n*k x n*k). Vvec is a block-diagonal matrix with blocks of V along the diagonal. This seems like a valid approach, but I still don't know how to specify the covariance structure to do weighted least squares. Any help is appreciated. Best, Ravi. - --- --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: [EMAIL PROTECTED] Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html - --- [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting- guide.html and provide commented, minimal, self-contained, reproducible code. -- Robin Hankin Uncertainty Analyst National Oceanography Centre, Southampton European Way, Southampton SO14 3ZH, UK tel 023-8059-7743 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Multivariate regression
Hi, Suppose I have a multivariate response Y (n x k) obtained at a set of predictors X (n x p). I would like to perform a linear regression taking into consideration the covariance structure of Y within each unit - this would be represented by a specified matrix V (k x k), assumed to be the same across units. How do I use lm to do this? One approach that I was thinking of is as follows: Flatten Y to a vector, say, Yvec (n*k x 1). Create Xvec (n*k, p*k) such that it is made up of block matrices Bij (k x k), where Bij is a diagonal matrix with X_ij as the diagonal (i = 1,.n, and j = 1,.,p). Now I can use lm in a univariate mode to regress Yvec against Xvec, with covariance matrix Vvec (n*k x n*k). Vvec is a block-diagonal matrix with blocks of V along the diagonal. This seems like a valid approach, but I still don't know how to specify the covariance structure to do weighted least squares. Any help is appreciated. Best, Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: [EMAIL PROTECTED] Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multivariate regression
You can use gee ( http://finzi.psych.upenn.edu/R/library/geepack/html/00Index.html) or maybe the function gls in nlme. Ritwik. On 10/27/06, Ravi Varadhan [EMAIL PROTECTED] wrote: Hi, Suppose I have a multivariate response Y (n x k) obtained at a set of predictors X (n x p). I would like to perform a linear regression taking into consideration the covariance structure of Y within each unit - this would be represented by a specified matrix V (k x k), assumed to be the same across units. How do I use lm to do this? One approach that I was thinking of is as follows: Flatten Y to a vector, say, Yvec (n*k x 1). Create Xvec (n*k, p*k) such that it is made up of block matrices Bij (k x k), where Bij is a diagonal matrix with X_ij as the diagonal (i = 1,.n, and j = 1,.,p). Now I can use lm in a univariate mode to regress Yvec against Xvec, with covariance matrix Vvec (n*k x n*k). Vvec is a block-diagonal matrix with blocks of V along the diagonal. This seems like a valid approach, but I still don't know how to specify the covariance structure to do weighted least squares. Any help is appreciated. Best, Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: [EMAIL PROTECTED] Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Ritwik Sinha Graduate Student Epidemiology and Biostatistics Case Western Reserve University [EMAIL PROTECTED] | +12163682366 | http://darwin.cwru.edu/~rsinha [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] multivariate regression using R
Does anyone know if there is a way to run multivariate linear regression in R? I tried using the lm function (e.g., lm(dv1, dv2~iv1+iv2+iv3), but got error messages. Is my syntax wrong, or do I need a particular package? Thanks, Jeff-- Jeffrey J. Lusk, Ph.D. Postdoctoral Research Associate Department of Forestry Natural Resources and Purdue and Climate Change Research Center 715 West State Street; Pfendler Hall Purdue University West Lafayette, IN 47907 Phone: 765-494-9701 E-mail: [EMAIL PROTECTED] Web: http://www.geocities.com/jefflusk2002 [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] multivariate regression using R
Lusk, Jeffrey J [EMAIL PROTECTED] writes: Does anyone know if there is a way to run multivariate linear regression in R? I tried using the lm function (e.g., lm(dv1, dv2~iv1+iv2+iv3), but got error messages. Is my syntax wrong, or do I need a particular package? You need a matrix response: lm(cbind(dv1,dv2)~iv1+iv2+iv3) should do. There is an anova() method for comparing the resulting mlm objects. And yes, your syntax is wrong: you're calling lm with 2 arguments, dv1 and dv2~iv1+iv2+iv3. -- O__ Peter Dalgaard Øster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] Multivariate regression method
Hi Folks,
Thanks to several people's suggestions and clarifications,
I think I have implemented a function which computes the
conditional mean and covariance matrix of a subset of the
dimensions of an MV-normal variable, given the values on the
other dimensions (these conditioning value can be presented
as a matrix, to deal with several cases at once).
The code is below, for anyone who would like to use it.
Comments will be welcome.
Two auxiliary functions .+. and ixX are defined as well as the main
function MV.regn
Example:
U1-rnorm(10);U2-rnorm(10);U3-rnorm(10);
X-cbind(U2+U3.U1+U3,U1+U2); mu-matrix(c(0,0,0),nrow=1);
S-matrix(c(2,1,1, 1,2,1, 1,1,2),nrow=3);
#Ex 1
MV.regn(S,mu,X[,1,drop=FALSE],1)
#Ex 2
MV.regn(S,mu,X[c(1,3,5,7),1:2],1,2)
==
%.+%-function(X,x){sweep(X,2,x,+)} ## Adds x to rows of X
ixX-function(X,...){(1:ncol(X))%in%c( ... )}
MV.regn-function(S,mu,x1,...){
### NB NB The k-variate MV variables etc are ROW vectors throughout
### (as in nxk matrix of data on n cases with k variables observed).
### NB NB S,mu,x1 MUST be arrays (matrices): create with drop=FALSE
### or specify when entering arguments, e.g.
### MV.regn(S,mu,X[,1,drop=FALSE],1)
### Arguments: S is the covariance matrix of MV X
###mu is the ROW (1xk matrix) expectation(X)
###x1 is matrix: rows are conditioning values for selected
### columns of X (NB if a single column make sure it's
### a matrix).
###... is an indexing vector or comma-list of numbers
### selecting the conditioning columns of X for the
### conditioning variable X1 (implies complementary set of
### columns of X for the variable X2 whose conditional
### distribution (X2 | X1=x1) is to be found).
iX1-ixX(S, ... ); iX2-!iX1;
s11-solve(S[iX1,iX1,drop=FALSE]); s12-S[iX1,iX2,drop=FALSE];
s21-S[iX2,iX1,drop=FALSE]; s22-S[iX2,iX2,drop=FALSE];
mu1-mu[,iX1,drop=FALSE]; mu2-mu[,iX2,drop=FALSE];
Cmu - (x1%.+%(-mu1))%*%s11%*%s12 %.+% mu2;
Cvar - s22 - s21%*%s11%*%s12;
list(Cmu=Cmu,Cvar=Cvar,iX1=iX1,iX2=iX2)
}
=
Best wishes to all,
Ted.
E-Mail: (Ted Harding) [EMAIL PROTECTED]
Fax-to-email: +44 (0)870 167 1972
Date: 15-Jul-03 Time: 13:23:25
-- XFMail --
__
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
RE: [R] Multivariate regression in R [followup]
On 16-Jan-03 Ted Harding wrote: Hence the multivariate regression model for the data could be written in matrix form as Y = X*B + w1*W1 + w2*W2 + w3*W3 + e [ Y Nxp ; X Nxk ; W1 W2 W3 Nxp matices of factor level indicators; B kxp ; w1, w2, w3 scalars ] where e is 3-dim N(0,S), and B, w1, w2, w3 and S are to be estimated. What, in R, I can't make out how to do is to give some function (which function?) a model specification of the form Y ~ X + W1 + W2 + W3 but in such a way that it will fit a 2x3 matrix B of coefficients for X, but scalar coefficients w1, w2, w3 for W1, W2, W3 I think the thought underlying my query was that, if R would accept designating a _matrix_ of factor levels as a factor while preserving its matrix structure, then the above could fit into the model specification scheme. However, factor(W1), for instance, returns a linear structure. Apologies for the typo originally in the formula below (now corrected): Analytically, the log-likelihood can be written (summing over r) (-N/2)*log(det(S)) - 0.5*SUM[ e_r * S^(-1) * e_r' ] (e_r = rth row) where e = Y - B*X - w1*W1 - w2*W2 - w3*W3. After differentiation and algebra, one could implement the resulting matrix equations in octave (or matlab) and proceed to a solution. One could even do this, as a numerical procedure, in R -- but I'd rather not! Indeed, R's richness in model-fitting resources tempts one to think that this problem may be solvable using these -- it's just that I can't seem to put my hand on what's needed. E-Mail: (Ted Harding) [EMAIL PROTECTED] Fax-to-email: +44 (0)870 167 1972 Date: 17-Jan-03 Time: 09:09:46 -- XFMail -- __ [EMAIL PROTECTED] mailing list http://www.stat.math.ethz.ch/mailman/listinfo/r-help
