Re: [Scikit-learn-general] centering of sparse data for elastic net

2013-10-14 Thread Tadej Štajner
Hi James, I had a similar problem - my approach was to wrap the sparse matrix into another sparse matrix interface that applies the centering on the spot when computing dot products. It builds on the same rationale as the scipy.linalg.LinearOperator that's often used in optimization. I used it for

Re: [Scikit-learn-general] HMM with von Mises Emmissions

2013-10-14 Thread Fred Mailhot
On 14 October 2013 20:48, Robert McGibbon wrote: [...] > > p.s. core devs: pretty please don't remove the HMM code from the scikit :) > +1E6 -- October Webinars: Code for Performance Free Intel webinars can help you acc

[Scikit-learn-general] HMM with von Mises Emmissions

2013-10-14 Thread Robert McGibbon
Hey, I presume this is too specialized for inclusion in the scikit, but I've written a von Mises HMM, suitable for processes on angles/directions where you have periodic boundary conditions. The code is on github, BSD-licensed, etc. I'd be happy to make a PR if there's interest, but it's not nece

Re: [Scikit-learn-general] choice of regularization parameter grid for elastic net

2013-10-14 Thread James Jensen
John, you're right about the difference in nomenclature. I've been using scikit-learn's names for the parameters, so the alpha I've referred to is the regularization strength and corresponds to lambda in glmnet. The mixing parameter, referred to in glmnet as alpha, is the L1-ratio in scikit-lea

Re: [Scikit-learn-general] choice of regularization parameter grid for elastic net

2013-10-14 Thread Olivier Grisel
I meant "l1_ratio" instead of "l1_ration". -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the

Re: [Scikit-learn-general] choice of regularization parameter grid for elastic net

2013-10-14 Thread Olivier Grisel
alpha is the strength of the regularizer and l1_ration is the mixing weight. "lambda" is a reserved keyword in python, hence the use of alpha instead. But this is very confusing and I which we had used a common English name like "penalty_strength" instead. http://scikit-learn.org/stable/modules/ge

Re: [Scikit-learn-general] choice of regularization parameter grid for elastic net

2013-10-14 Thread John Collins
Hi James, In R speak: The reason you see the advice to choose a higher alpha if nobs < nvars and a lower alpha if the comparison is that alpha is the mixing weight between L1 and L2 penalties (whereas lambda is the regularization level) and because the L1 penalty tends to set more coefficients to

Re: [Scikit-learn-general] Contributing to scikit-learn

2013-10-14 Thread Olivier Grisel
Please have a look at the contributors guide: http://scikit-learn.org/stable/developers/#contributing-code In particular this doc mentions [Easy] tagged issues: https://github.com/scikit-learn/scikit-learn/issues?labels=Easy But in general the best way to contribute is to actually use the libra

[Scikit-learn-general] Contributing to scikit-learn

2013-10-14 Thread Ankit Agrawal
Hi, I am Ankit Agrawal, a 4th year undergrad majoring in EE with specialization in Communications and Signal Processing at IIT Bombay. I completed my GSoC with scikit-image this year and have a good grasp with Python(and a little bit with Cython). I have completed a course in ML, and have tak

Re: [Scikit-learn-general] choice of regularization parameter grid for elastic net

2013-10-14 Thread Nicholas Dronen
Hi, James: If by 'alpha' you mean what the lasso literature refers to as 'lambda', my recollection is that the maximum lambda is determined simply by the L1 norm of the coefficients of the ordinary least squares solution, because any value greater than that provides no constraint for the lasso sol

Re: [Scikit-learn-general] centering of sparse data for elastic net

2013-10-14 Thread Lars Buitinck
2013/10/14 James Jensen : > I've noticed people use the term "normalize" in different ways. In the case > of the `normalize=True` flag of the linear models, does it mean both scaling > samples to have unit norm and centering them to have mean zero? If so, this > is inconsistent with the usage in, s

Re: [Scikit-learn-general] choice of regularization parameter grid for elastic net

2013-10-14 Thread James Jensen
Thanks, Alex. That is helpful. Looks like the glmnet documentation says that this is how they do it as well. What they don't explain is how to find alpha_max in the first place. The only thing I've thought of is doing something like a binary search until you find the smallest alpha yielding the

Re: [Scikit-learn-general] centering of sparse data for elastic net

2013-10-14 Thread James Jensen
Thank you, Olivier. Just to clarify: you say You can control the centering with `normalize=True` flag of the ElasticNet class (or any other linear regression model). I've noticed people use the term "normalize" in different ways. In the case of the `normalize=True` flag of the linear mod

Re: [Scikit-learn-general] recommendation systems

2013-10-14 Thread Olivier Grisel
Actually the mrec implementation is not the original SLIM algorithm but a variant demonstrated by the lib author here: http://slideshare.net/MarkLevy/efficient-slides Thanks @larsmans for the tweet :) -- Olivier -- Oct

Re: [Scikit-learn-general] recommendation systems

2013-10-14 Thread Olivier Grisel
2013/10/14 Nick Pentreath : > Mendeley have also recently open-sourced their recommender framework, which > relies on SGD to train models using scikit-learn, and seems to try to fit > into the sklearn API. > > https://github.com/Mendeley/mrec/ Thanks for the hint. Has anybody some feedback to shar