[Rd] Large discrepancies in the same object being saved to .RData
Hi developers,
After some investigation I have found there can be large discrepancies in the
same object being saved as an external xx.RData file. The immediate
repercussion of this is the possible increased size of your .RData workspace
for no apparent reason.
The function and its three scenarios below highlight these discrepancies. Note
that the object being returned is exactly the same in each circumstance. The
first scenario simply loops over a set of lm() models from a simulated set of
data. The second adds a reasonably large matrix calculation within the loop.
The third highlights exactly where the discrepancy lies. It appears that when
the object is saved to an xx.RData it is still burdened, in some capacity,
with the objects created in the function. Only deleting these objects at the
end of the function ensures the realistic size of the returned object.
Performing gc() after each of these short simulations shows that the Vcells
that are accumulated in the function environment appear to remain after the
function returns. These cached remains are then transferred to the .RData upon
saving of the object(s). This is occurring quite broadly across the Windows 7
(R 2.10.1) and 64 Bit Ubuntu Linux (R 2.9.0) systems that I use.
A similar problem was partially pointed out four years ago
http://tolstoy.newcastle.edu.au/R/help/06/03/24060.html
and has been made more obvious in the scenarios given below.
Admittedly I have had many problems with workspace .RData sizes over the years
and it has taken me some time to realise what is actually occurring. Can
someone enlighten myself and my colleagues as to why the objects created and
evaluated in a function call stack are saved, in some capacity, with the
returned object?
Cheers,
Julian
### small simulation from a clean directory
lmfunc - function(loop = 20, add = FALSE, gr = FALSE){
lmlist - rmlist - list()
set.seed(100)
dat - data.frame(matrix(rnorm(100*100), ncol = 100))
rm - matrix(rnorm(10), ncol = 1000)
names(dat)[1] - y
i - 1
for(i in 1:loop) {
lmlist[[i]] - lm(y ~ ., data = dat)
if(add)
rmlist[[i]] - rm
}
fm - lmlist[[loop]]
if(gr) {
print(what - ls(envir = sys.frame(which = 1)))
remove(list = setdiff(what, fm))
}
fm
}
# baseline gc()
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 153325 4.1 35 9.4 35 9.4
Vcells 99228 0.8 786432 6.0 386446 3.0
## 1. simple lm() simulation
lmtest1 - lmfunc()
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 184470 5.0 407500 10.9 35 9.4
Vcells 842169 6.51300721 10.0 1162577 8.9
save(lmtest1, file = lm1.RData)
system(ls -s lm1.RData)
4312 lm1.RData
## A moderate increase in Vcells; .RData object around 4.5 Mb
## 2. add matrix calculation to loop
lmtest2 - lmfunc(add = TRUE)
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 209316 5.6 407500 10.9 405340 10.9
Vcells 3584244 27.44175939 31.9 3900869 29.8
save(lmtest2, file = lm2.RData)
system(ls -s lm2.RData)
19324 lm2.RData
## A enormous increase in Vcells; .RData object is now 19Mb+
## 3. delete all objects in function call stack
lmtest3 - lmfunc(add = TRUE, gr = TRUE)
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 210766 5.7 467875 12.5 467875 12.5
Vcells 3615863 27.66933688 52.9 6898609 52.7
save(lmtest3, file = lm3.RData)
system(ls -s lm3.RData)
320 lm3.RData
## A minimal increase in Vcells; .RData object is now 320Kb
sapply(ls(pattern = lmtest*), function(x) object.size(get(x, envir =
.GlobalEnv)))
lmtest1 lmtest2 lmtest3
358428 358428 358428
## all objects are deemed the same size by object.size()
# End sim
--
---
Dr. Julian Taylor phone: +61 8 8303 8792
Postdoctoral Fellow fax: +61 8 8303 8763
CMIS, CSIRO mobile: +61 4 1638 8180
Private Mail Bag 2email: julian.tay...@csiro.au
Glen Osmond, SA, 5064
---
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] constrained optimization
Dear list, The task view on optimization does not reference a package for non linear constrained optimization problems. Stefan Theussl told me to look at the Rsolnp package, but unfortunately it is not very clear what method is R ported. (The authors ported the matlab code of Yinyu Ye http://www.stanford.edu/~yyye/ http://www.stanford.edu/%7Eyyye/) Currently I'm looking for an implementation of sequential quadratic programming to replicate SNOPT*. A good reference I found on the web is this booklet http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/5456/pdf/imm5456.pdf . Does anyone know an implementation of such algorithms? Is there any fortran implementation available useful if I have to implement it? Thanks in advance Christophe * SNOPT: An SQP Algorithm For Large-Scale Constrained Optimization (1997) by Philip E. Gill , Walter Murray , Michael , Michael A. Saunders -- Christophe DUTANG Ph. D. student at ISFA, Lyon, France [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Large discrepancies in the same object being saved to .RData
On 06/07/2010 9:04 PM, julian.tay...@csiro.au wrote:
Hi developers,
After some investigation I have found there can be large discrepancies in the same object
being saved as an external xx.RData file. The immediate repercussion of this
is the possible increased size of your .RData workspace for no apparent reason.
The function and its three scenarios below highlight these discrepancies. Note that the object
being returned is exactly the same in each circumstance. The first scenario simply loops over a set
of lm() models from a simulated set of data. The second adds a reasonably large matrix calculation
within the loop. The third highlights exactly where the discrepancy lies. It appears that when the
object is saved to an xx.RData it is still burdened, in some capacity, with the objects
created in the function. Only deleting these objects at the end of the function ensures the
realistic size of the returned object. Performing gc() after each of these short simulations shows
that the Vcells that are accumulated in the function environment appear to remain after
the function returns. These cached remains are then transferred to the .RData upon saving of the
object(s). This is occurring quite broadly across the Windows 7 (R 2.10.1) and 64 Bit Ubuntu Linux
(R 2.9.0) systems that I us!
e.
A similar problem was partially pointed out four years ago
http://tolstoy.newcastle.edu.au/R/help/06/03/24060.html
and has been made more obvious in the scenarios given below.
Admittedly I have had many problems with workspace .RData sizes over the years
and it has taken me some time to realise what is actually occurring. Can
someone enlighten myself and my colleagues as to why the objects created and
evaluated in a function call stack are saved, in some capacity, with the
returned object?
I haven't worked through your example, but in general the way that local
objects get captured is when part of the return value includes an
environment. Examples of things that include an environment are locally
created functions and formulas. It's probably the latter that you're
seeing. When R computes the result of y ~ . or a similar formula, it
attaches a pointer to the environment in which the calculation took
place, so that later when the formula is used, it can look up y there.
For example, in your line
lm(y ~ ., data = dat)
from your code, the formula y ~ . needs to be computed before R knows
that you've explicitly listed a dataframe holding the data, and before
it knows whether the variable y is in that dataframe or is just a local
variable in the current function.
Since these are just pointers to the environment, this doesn't take up
much space in memory, but when you save the object to disk, a copy of
the whole environment will be made, and that can end up wasting up a lot
of space if the environment contains a lot of things that aren't needed
by the formula.
Duncan Murdoch
Cheers,
Julian
### small simulation from a clean directory
lmfunc - function(loop = 20, add = FALSE, gr = FALSE){
lmlist - rmlist - list()
set.seed(100)
dat - data.frame(matrix(rnorm(100*100), ncol = 100))
rm - matrix(rnorm(10), ncol = 1000)
names(dat)[1] - y
i - 1
for(i in 1:loop) {
lmlist[[i]] - lm(y ~ ., data = dat)
if(add)
rmlist[[i]] - rm
}
fm - lmlist[[loop]]
if(gr) {
print(what - ls(envir = sys.frame(which = 1)))
remove(list = setdiff(what, fm))
}
fm
}
# baseline gc()
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 153325 4.1 35 9.4 35 9.4
Vcells 99228 0.8 786432 6.0 386446 3.0
## 1. simple lm() simulation
lmtest1 - lmfunc()
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 184470 5.0 407500 10.9 35 9.4
Vcells 842169 6.51300721 10.0 1162577 8.9
save(lmtest1, file = lm1.RData)
system(ls -s lm1.RData)
4312 lm1.RData
## A moderate increase in Vcells; .RData object around 4.5 Mb
## 2. add matrix calculation to loop
lmtest2 - lmfunc(add = TRUE)
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 209316 5.6 407500 10.9 405340 10.9
Vcells 3584244 27.44175939 31.9 3900869 29.8
save(lmtest2, file = lm2.RData)
system(ls -s lm2.RData)
19324 lm2.RData
## A enormous increase in Vcells; .RData object is now 19Mb+
## 3. delete all objects in function call stack
lmtest3 - lmfunc(add = TRUE, gr = TRUE)
gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 210766 5.7 467875 12.5 467875 12.5
Vcells 3615863 27.66933688 52.9 6898609 52.7
save(lmtest3, file = lm3.RData)
system(ls -s lm3.RData)
320 lm3.RData
## A minimal increase in Vcells; .RData object is now 320Kb
sapply(ls(pattern =
Re: [Rd] constrained optimization
Hi Christophe, I have an algorithm for solving nonlinearly constrained optimization. It is a combination of an interior point (for inequalities) algorithm with an augmented Lagrangian (for equalities). It is coded entirely in R, and hence is a bit slow, but it seems to do the job quite robustly in terms of handling poor starting values. I can send this to you, if you are interested. Ravi. -Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r-project.org] On Behalf Of Christophe Dutang Sent: Wednesday, July 07, 2010 8:01 AM To: r-devel@r-project.org Subject: [Rd] constrained optimization Dear list, The task view on optimization does not reference a package for non linear constrained optimization problems. Stefan Theussl told me to look at the Rsolnp package, but unfortunately it is not very clear what method is R ported. (The authors ported the matlab code of Yinyu Ye http://www.stanford.edu/~yyye/ http://www.stanford.edu/%7Eyyye/) Currently I'm looking for an implementation of sequential quadratic programming to replicate SNOPT*. A good reference I found on the web is this booklet http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/5456/pdf/imm5456.pdf . Does anyone know an implementation of such algorithms? Is there any fortran implementation available useful if I have to implement it? Thanks in advance Christophe * SNOPT: An SQP Algorithm For Large-Scale Constrained Optimization (1997) by Philip E. Gill , Walter Murray , Michael , Michael A. Saunders -- Christophe DUTANG Ph. D. student at ISFA, Lyon, France [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] how to define method for + function in a new class
Dear R developers,
I have a new class, which I called Molecule, and have tried to define =
a + operation for 2 objects of this class.
This is what I have written so far, although the method is not complete =
(I'm trying to look at it at intermediate stages):
setMethod(
f=3D+,
signature(x=3DMolecule,y=3DMolecule),
definition=3Dfunction(x,y,...)
{
# Check both objects are correct
checkMolecule(x)
checkMolecule(y)
# Extract chains information
ch1 - getMoleculeChains(x)
ch2 - getMoleculeChains(y)
union_ch - unique(c(ch1,ch2))
not_used - .ALPHABET
for (i in seq(along =3D union_ch))
{
idx - which(not_used !=3D union_ch[i])
if (length(idx) !=3D 0) not_used - not_used[idx]
}
return(not_used)
}
)
The definition of class Molecule is included earlier in the same file. =
When I source it, I get the following error message:
Error in match.call(fun, fcall) :=20
unused argument(s) (x =3D Molecule, y =3D Molecule)
I can't see what's wrong in my method definition. Can anyone help me =
with this?
J
Dr James Foadi PhD
Membrane Protein Laboratory (MPL)
Diamond Light Source Ltd
Diamond House
Harewell Science and Innovation Campus
Chilton, Didcot
Oxfordshire OX11 0DE
Email: james.fo...@diamond.ac.uk
Alt Email: j.fo...@imperial.ac.uk
--
This e-mail and any attachments may contain confidential...{{dropped:8}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] how to define method for + function in a new class
I don't know why a 3D appears instead of a (.
Please, interpret a ( whenever you see a 3D.
J
Dr James Foadi PhD
Membrane Protein Laboratory (MPL)
Diamond Light Source Ltd
Diamond House
Harewell Science and Innovation Campus
Chilton, Didcot
Oxfordshire OX11 0DE
Email: james.fo...@diamond.ac.uk
Alt Email: j.fo...@imperial.ac.uk
-Original Message-
From: r-devel-boun...@r-project.org on behalf of james.fo...@diamond.ac.uk
Sent: Wed 07/07/2010 16:09
To: r-devel@r-project.org
Subject: [Rd] how to define method for + function in a new class
Dear R developers,
I have a new class, which I called Molecule, and have tried to define =
a + operation for 2 objects of this class.
This is what I have written so far, although the method is not complete =
(I'm trying to look at it at intermediate stages):
setMethod(
f=3D+,
signature(x=3DMolecule,y=3DMolecule),
definition=3Dfunction(x,y,...)
{
# Check both objects are correct
checkMolecule(x)
checkMolecule(y)
# Extract chains information
ch1 - getMoleculeChains(x)
ch2 - getMoleculeChains(y)
union_ch - unique(c(ch1,ch2))
not_used - .ALPHABET
for (i in seq(along =3D union_ch))
{
idx - which(not_used !=3D union_ch[i])
if (length(idx) !=3D 0) not_used - not_used[idx]
}
return(not_used)
}
)
The definition of class Molecule is included earlier in the same file. =
When I source it, I get the following error message:
Error in match.call(fun, fcall) :=20
unused argument(s) (x =3D Molecule, y =3D Molecule)
I can't see what's wrong in my method definition. Can anyone help me =
with this?
J
Dr James Foadi PhD
Membrane Protein Laboratory (MPL)
Diamond Light Source Ltd
Diamond House
Harewell Science and Innovation Campus
Chilton, Didcot
Oxfordshire OX11 0DE
Email: james.fo...@diamond.ac.uk
Alt Email: j.fo...@imperial.ac.uk
--
This e-mail and any attachments may contain confidential...{{dropped:16}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] read, write generics?
R-devel list, I'm working on a wrapper for R to the POSIX terminal interface. I'd like to make this code available as a package (as well as a connection patch). The 'seek', 'truncate', 'flush', 'open', and 'close' generic functions are a great help for the packaged interface. The only two missing are 'read' and 'write' generics, but I realize write is currently a wrapper to 'cat'. What are your thoughts on adding 'read', and converting 'write' to generics in a manner similar to 'seek' and friends? Also, I'd like to use R's 'iconv' function to re-encode text from tty data, but this is apparently restricted to character encodings that do not permit embedded NUL characters. I wrote to R-devel about this earlier [1], with potential solution. Any thoughts on this? -Matt [1] http://tolstoy.newcastle.edu.au/R/e10/devel/10/06/0648.html -- Matthew S. Shotwell Graduate Student Division of Biostatistics and Epidemiology Medical University of South Carolina __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] how to define method for + function in a new class
On 07/07/2010 08:09 AM, james.fo...@diamond.ac.uk wrote:
Dear R developers,
I have a new class, which I called Molecule, and have tried to define =
a + operation for 2 objects of this class.
This is what I have written so far, although the method is not complete =
(I'm trying to look at it at intermediate stages):
setMethod(
f=3D+,
signature(x=3DMolecule,y=3DMolecule),
definition=3Dfunction(x,y,...)
{
# Check both objects are correct
checkMolecule(x)
checkMolecule(y)
# Extract chains information
ch1 - getMoleculeChains(x)
ch2 - getMoleculeChains(y)
union_ch - unique(c(ch1,ch2))
not_used - .ALPHABET
for (i in seq(along =3D union_ch))
{
idx - which(not_used !=3D union_ch[i])
if (length(idx) !=3D 0) not_used - not_used[idx]
}
return(not_used)
}
)
The definition of class Molecule is included earlier in the same file. =
When I source it, I get the following error message:
Error in match.call(fun, fcall) :=20
unused argument(s) (x =3D Molecule, y =3D Molecule)
If I
getGeneric(+)
standardGeneric for + defined from package base
belonging to group(s): Arith
function (e1, e2)
standardGeneric(+, .Primitive(+))
environment: 0xb432f8
Methods may be defined for arguments: e1, e2
Use showMethods(+) for currently available ones.
I see that the generic is defined to take two arguments e1 and e2. So
setMethod(+, c(Molecule, Molecule), function(e1, e2) {
## ...
})
but actually here it might often pay to discover ?GroupGenericFunctions
and end up with something like
setClass(A, representation=representation(x=numeric))
setMethod(Arith, c(A, A), function(e1, e2) {
new(class(e1), x=callGeneric(e1...@x, e2...@x))
})
and then
new(A, x=1:5) + new(A, x=5:1)
An object of class A
Slot x:
[1] 6 6 6 6 6
but also
new(A, x=1:5) * new(A, x=5:1)
An object of class A
Slot x:
[1] 5 8 9 8 5
Martin
I can't see what's wrong in my method definition. Can anyone help me =
with this?
J
Dr James Foadi PhD
Membrane Protein Laboratory (MPL)
Diamond Light Source Ltd
Diamond House
Harewell Science and Innovation Campus
Chilton, Didcot
Oxfordshire OX11 0DE
Email: james.fo...@diamond.ac.uk
Alt Email: j.fo...@imperial.ac.uk
--
Martin Morgan
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109
Location: Arnold Building M1 B861
Phone: (206) 667-2793
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] how to define method for + function in a new class
Thank you, Martin! Very clear.
J
Dr James Foadi PhD
Membrane Protein Laboratory (MPL)
Diamond Light Source Ltd
Diamond House
Harewell Science and Innovation Campus
Chilton, Didcot
Oxfordshire OX11 0DE
Email: james.fo...@diamond.ac.uk
Alt Email: j.fo...@imperial.ac.uk
-Original Message-
From: Martin Morgan [mailto:mtmor...@fhcrc.org]
Sent: Wed 07/07/2010 16:43
To: Foadi, James (Imperial Coll.,RAL,DIA)
Cc: r-devel@r-project.org
Subject: Re: [Rd] how to define method for + function in a new class
On 07/07/2010 08:09 AM, james.fo...@diamond.ac.uk wrote:
Dear R developers,
I have a new class, which I called Molecule, and have tried to define =
a + operation for 2 objects of this class.
This is what I have written so far, although the method is not complete =
(I'm trying to look at it at intermediate stages):
setMethod(
f=3D+,
signature(x=3DMolecule,y=3DMolecule),
definition=3Dfunction(x,y,...)
{
# Check both objects are correct
checkMolecule(x)
checkMolecule(y)
# Extract chains information
ch1 - getMoleculeChains(x)
ch2 - getMoleculeChains(y)
union_ch - unique(c(ch1,ch2))
not_used - .ALPHABET
for (i in seq(along =3D union_ch))
{
idx - which(not_used !=3D union_ch[i])
if (length(idx) !=3D 0) not_used - not_used[idx]
}
return(not_used)
}
)
The definition of class Molecule is included earlier in the same file. =
When I source it, I get the following error message:
Error in match.call(fun, fcall) :=20
unused argument(s) (x =3D Molecule, y =3D Molecule)
If I
getGeneric(+)
standardGeneric for + defined from package base
belonging to group(s): Arith
function (e1, e2)
standardGeneric(+, .Primitive(+))
environment: 0xb432f8
Methods may be defined for arguments: e1, e2
Use showMethods(+) for currently available ones.
I see that the generic is defined to take two arguments e1 and e2. So
setMethod(+, c(Molecule, Molecule), function(e1, e2) {
## ...
})
but actually here it might often pay to discover ?GroupGenericFunctions
and end up with something like
setClass(A, representation=representation(x=numeric))
setMethod(Arith, c(A, A), function(e1, e2) {
new(class(e1), x=callGeneric(e1...@x, e2...@x))
})
and then
new(A, x=1:5) + new(A, x=5:1)
An object of class A
Slot x:
[1] 6 6 6 6 6
but also
new(A, x=1:5) * new(A, x=5:1)
An object of class A
Slot x:
[1] 5 8 9 8 5
Martin
I can't see what's wrong in my method definition. Can anyone help me =
with this?
J
Dr James Foadi PhD
Membrane Protein Laboratory (MPL)
Diamond Light Source Ltd
Diamond House
Harewell Science and Innovation Campus
Chilton, Didcot
Oxfordshire OX11 0DE
Email: james.fo...@diamond.ac.uk
Alt Email: j.fo...@imperial.ac.uk
--
Martin Morgan
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109
Location: Arnold Building M1 B861
Phone: (206) 667-2793
--
This e-mail and any attachments may contain confidential...{{dropped:8}}
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] constrained optimization
Thank you Jelmer, I have just spent five minutes on the website, but I'm not sure to understand which method is implemented. Is it *this method? On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming* Do you plan to put lpopt on CRAN? I'm willing to test it! Christophe 2010/7/7 Jelmer Ypma jelmery...@gmail.com Hi Christophe, I wrote an R interface to Ipopt ( https://projects.coin-or.org/Ipopt ), which does non-linear constrained optimization for very large problems. Would this be of interest to you? It took some time to find out which license I could use to distribute it, but I'm planning to make it publicly available this weekend. Jelmer On Wed, Jul 7, 2010 at 13:01, Christophe Dutang duta...@gmail.com wrote: Dear list, The task view on optimization does not reference a package for non linear constrained optimization problems. Stefan Theussl told me to look at the Rsolnp package, but unfortunately it is not very clear what method is R ported. (The authors ported the matlab code of Yinyu Ye http://www.stanford.edu/~yyye/ http://www.stanford.edu/%7Eyyye/ http://www.stanford.edu/%7Eyyye/) Currently I'm looking for an implementation of sequential quadratic programming to replicate SNOPT*. A good reference I found on the web is this booklet http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/5456/pdf/imm5456.pdf. Does anyone know an implementation of such algorithms? Is there any fortran implementation available useful if I have to implement it? Thanks in advance Christophe * SNOPT: An SQP Algorithm For Large-Scale Constrained Optimization (1997) by Philip E. Gill , Walter Murray , Michael , Michael A. Saunders -- Christophe DUTANG Ph. D. student at ISFA, Lyon, France [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Christophe DUTANG Ph. D. student at ISFA, Lyon, France [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] constrained optimization
Hello Ravi, Yes I'm willing to test it. Tell me when there is an R version of it. Christophe 2010/7/7 Ravi Varadhan rvarad...@jhmi.edu Paul, SQUAREM is not an optimization package. It is an accelerator of fixed-point iteration such as the EM, MM, and other algorithms. I am referring to a set of functions that I have written, which I call ALABAMA for Augmented Lagrangian Adaptive Barrier Minimization Algorithm. I think it should be package up pretty soon as there seems to be a need for nonlinearly constrained optimization in R. Best, Ravi. -Original Message- From: Paul Gilbert [mailto:pgilb...@bank-banque-canada.ca] Sent: Wednesday, July 07, 2010 9:18 AM To: Ravi Varadhan; Christophe Dutang Subject: RE: [Rd] constrained optimization Ravi I think you are referring to the SQUAREM package. The development version is available at https://r-forge.r-project.org/R/?group_id=395 and can be installed with install.packages(SQUAREM, repos=http://R-Forge.R-project.org;) . Paul -Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r- project.org] On Behalf Of Ravi Varadhan Sent: July 7, 2010 9:10 AM To: 'Christophe Dutang'; r-devel@r-project.org Subject: Re: [Rd] constrained optimization Hi Christophe, I have an algorithm for solving nonlinearly constrained optimization. It is a combination of an interior point (for inequalities) algorithm with an augmented Lagrangian (for equalities). It is coded entirely in R, and hence is a bit slow, but it seems to do the job quite robustly in terms of handling poor starting values. I can send this to you, if you are interested. Ravi. -Original Message- From: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r- project.org] On Behalf Of Christophe Dutang Sent: Wednesday, July 07, 2010 8:01 AM To: r-devel@r-project.org Subject: [Rd] constrained optimization Dear list, The task view on optimization does not reference a package for non linear constrained optimization problems. Stefan Theussl told me to look at the Rsolnp package, but unfortunately it is not very clear what method is R ported. (The authors ported the matlab code of Yinyu Ye http://www.stanford.edu/~yyye/ http://www.stanford.edu/%7Eyyye/ http://www.stanford.edu/%7Eyyye/) Currently I'm looking for an implementation of sequential quadratic programming to replicate SNOPT*. A good reference I found on the web is this booklet http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/5456/pdf/imm5456.p d f . Does anyone know an implementation of such algorithms? Is there any fortran implementation available useful if I have to implement it? Thanks in advance Christophe * SNOPT: An SQP Algorithm For Large-Scale Constrained Optimization (1997) by Philip E. Gill , Walter Murray , Michael , Michael A. Saunders -- Christophe DUTANG Ph. D. student at ISFA, Lyon, France [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel La version française suit le texte anglais. This email may contain privileged and/or confidential information, and the Bank of Canada does not waive any related rights. Any distribution, use, or copying of this email or the information it contains by other than the intended recipient is unauthorized. If you received this email in error please delete it immediately from your system and notify the sender promptly by email that you have done so. Le présent courriel peut contenir de l'information privilégiée ou confidentielle. La Banque du Canada ne renonce pas aux droits qui s'y rapportent. Toute diffusion, utilisation ou copie de ce courriel ou des renseignements qu'il contient par une personne autre que le ou les destinataires désignés est interdite. Si vous recevez ce courriel par erreur, veuillez le supprimer immédiatement et envoyer sans délai à l'expéditeur un message électronique pour l'aviser que vous avez éliminé de votre ordinateur toute copie du courriel reçu. -- Christophe DUTANG Ph. D. student at ISFA, Lyon, France [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] constrained optimization
Have you considered using the optim function with L-BFGS-B for bounded optimization. Obviously, you will have to do changes of variables so that everything is in terms of a rectangle (which is the type of bounding that it accepts). I believe it is based on A LIMITED MEMORY ALGORITHM FOR BOUND CONSTRAINED OPTIMIZATION By Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. Technical Report NAM-08, May 1994. They also have a paper on the exact implementation. I also believe that R uses the code from Nocedal et al. -- View this message in context: http://r.789695.n4.nabble.com/constrained-optimization-tp2280809p2281282.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] constrained optimization
On Wed, Jul 7, 2010 at 1:49 PM, Paul Bailey pdbai...@umd.edu wrote: Have you considered using the optim function with L-BFGS-B for bounded optimization. Obviously, you will have to do changes of variables so that everything is in terms of a rectangle (which is the type of bounding that it accepts). I believe it is based on A LIMITED MEMORY ALGORITHM FOR BOUND CONSTRAINED OPTIMIZATION By Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. Technical Report NAM-08, May 1994. They also have a paper on the exact implementation. I also believe that R uses the code from Nocedal et al. Yes, the code is in R/src/appl/lbfgsb.c. There is a C++ interface in the cxxPack package (cxxPack/src/optimize.cpp, class ConstrainedMinimizer). I haven't yet included an example. If there is interest I will. Dominick [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Accessing options of L-BFGS-B
Hi, I use L-BFGS-B heavily and I want access to some of the options in the code that R calls but does not provide access to. There are some knobs on the convergence criteria that are not accessible via optim, specifically, I want to require a maximum gradient (obviously, it is actually some norm of the gradient vector so that the requirement is 1 dimensional). I want to do this because rerunning from various starting points gives me poor convergence and my own (pure-R) L-BFGS has a poor line search algorithm that sometimes fails. Is there a simple way to do this? As an alternative, the BFGS type papers tend to be long on the BFGS explanations and short on the line search explanations. If someone pointed me to a good line search paper that uses the Wolfe conditions... Thanks, Paul Bailey -- View this message in context: http://r.789695.n4.nabble.com/Accessing-options-of-L-BFGS-B-tp2281324p2281324.html Sent from the R devel mailing list archive at Nabble.com. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] pass-by-reference
I'm working with a large object that I want to modify slightly in a function.
Pass-by-reference would make a lot of sense, but I don't know how to do it.
I've searched this archive and thought that I can do something like
f - function(x) {
v1 - list(a=x,b=3)
g(x)
v1
}
g - function(x) {
frame - parent.frame()
assign(v1,list(a=x,b=x),frame)
}
f(4)
returns list(a=4,b=4)
but what if I wanted to make v1[[1]] = v1[[1]] + v1[[2]] without creating a
copy of v1?
f2 - function(x) {
v1 - list(a=x,b=3)
g2(x)
v1
}
g2 - function(x) {
frame - parent.frame()
v1 - get(v1,envir=frame)
v1[[1]] - v1[[1]] + v1[[2]]
}
f2(4)
but this fails. (it returns list(a=4,b=3) because v1 was copied into g2, not
passed by reference) Is there a way to do this?
--
View this message in context:
http://r.789695.n4.nabble.com/pass-by-reference-tp2281802p2281802.html
Sent from the R devel mailing list archive at Nabble.com.
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
