Hi,
My library lightning (https://github.com/mblondel/lightning) supports
L1-regularized logistic regression with warm start.
from lightning.primal_cd import CDClassifier
clf = CDClassifier(loss="log", penalty="l1", warm_start=True,
C=1.0/X.shape[0])
clf.tol = 1e-3
clf.alpha = 1e-3
clf.fit(X, y)
clf.tol = 1e-6
clf.alpha = 1e-4
clf.fit(X, y)
The reason you need to change the tol after the first call to fit is
because, at each outer iteration, CDClassifier compares the optimality
violations with the 1st outer iteration (this is intended as a
normalization so that tol is not dataset dependent) and decides whether to
stop or not based on the tol constant. However, when you use warm-start,
the first outer iteration is already pretty good so you need to use a
stricter tolerance value. In short, lightning uses a relative stopping
criterion rather than an absolute stopping criterion. This makes warm-start
a little less convenient to use.
HTH,
Mathieu
On Fri, May 17, 2013 at 7:13 AM, Patrick Mineault <
[email protected]> wrote:
> Hi all,
>
> I'm performing logistic regression with an L1 penalty with
> sklearn.linear_model.LogisticRegression. I'd like to get the entire
> regularization path (from highly regularized to not regularized), similar
> to lasso_path or lars_path.
>
> However, I've verified that if you call train a second time on the same
> object/same data, it takes the same amount of time to perform the training,
> which mains that the class doesn't use previous estimates of coef_ - it
> doesn't use a warm-start strategy.
>
> Warm-start would be a lot faster - is there any way of doing warm-start
> with LogisticRegression, or some workaround to get the entire
> regularization path for logistic regression with an L1 penalty?
>
> Patrick Mineault
>
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
