[R] variable/model selction (step/stepAIC) for biglm ?

2009-02-21 Thread Tal Galili
Hello dear R mailing list members. I have recently became curious of the possibility applying model selection algorithms (even as simple as AIC) to regressions of large datasets. I searched as best as I could, but couldn't find any reference or wrapper for using step or stepAIC to packages such as

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-21 Thread Charles C. Berry
On Sat, 21 Feb 2009, Tal Galili wrote: Hello dear R mailing list members. I have recently became curious of the possibility applying model selection algorithms (even as simple as AIC) to regressions of large datasets. Large in the sense of many observations, one assumes. But how large in te

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-21 Thread Tal Galili
Hi Chuck, Thanks for the guidelines. I was hoping someone in the group already experienced handling this type of task and have some handy code to share. I'll wait another day or two to see if someone responds with any more ideas or experience, and if nothing will come up, I might try my hand in y

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-22 Thread Thomas Lumley
On Sat, 21 Feb 2009, Charles C. Berry wrote: On Sat, 21 Feb 2009, Tal Galili wrote: Hello dear R mailing list members. I have recently became curious of the possibility applying model selection algorithms (even as simple as AIC) to regressions of large datasets. Large in the sense of many

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-22 Thread Tal Galili
Hi Thomas, What you just wrote is very interesting to me - do you have any suggestion then as to how to implement leaps (or any other package/code) to iterate on the final lm model produced by biglm ? Any advice would be very welcomed! p.s: My purpose is to use a different algorithm than that of

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-22 Thread Thomas Lumley
On Sun, 22 Feb 2009, Tal Galili wrote: Hi Thomas, What you just wrote is very interesting to me - do you have any suggestion then as to how to implement leaps (or any other package/code) to iterate on the final lm model produced by biglm ? Any advice would be very welcomed! If you look at th

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-22 Thread Tal Galili
Thanks Thomas! I'll give it a go and will send updates as to how I am doing. Cheers, Tal On Sun, Feb 22, 2009 at 1:52 PM, Thomas Lumley wrote: > On Sun, 22 Feb 2009, Tal Galili wrote: > > Hi Thomas, >> >> What you just wrote is very interesting to me - do you have any suggestion >> then as t

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-22 Thread Tal Galili
Hi Thomas, On second thought - I will need some more help: What I only now noticed is that leaps is "an exhaustive search for the best subsets of the variables in x for predicting y in linear regression", while what I am aiming for (for now) is a method to implement forward selection (on biglm) - a

Re: [R] variable/model selction (step/stepAIC) for biglm ?

2009-02-22 Thread Thomas Lumley
On Sun, 22 Feb 2009, Tal Galili wrote: Hi Thomas, On second thought - I will need some more help: What I only now noticed is that leaps is "an exhaustive search for the best subsets of the variables in x for predicting y in linear regression", while what I am aiming for (for now) is a method to