2013/5/2 Mathieu Blondel <[email protected]>
>
>
> On Thu, May 2, 2013 at 5:21 PM, Peter Prettenhofer <
> [email protected]> wrote:
>
>> this looks pretty awesome - especially the dataset abstraction is pretty
>> neat - would be great if we could merge this into scikit-learn.
>>
>
> Merging the dataset abstraction would be nice. We could port some of
> scikit-learn's code to it, including SGD and mini-batch k-means. The neural
> network PR by Lars could also benefit it.
>
totally agree - I can raise this issue and work on it at the sprint -
shouldn't take too long - we would need to port SGD first anyways.
>
> BTW, do you think we should keep the weight vector abstraction which is in
> scikit-learn?
>
The idea behind the abstraction was to implement averaged SGD/Perceptron
easily - I didn't finish the PR though...
So I guess the answer is: no
>
>
>> btw: what kind of truncated gradient algorithm does lightning use for L1
>> penalized SGD? As far as I can see its not the one that's currently used in
>> SGDClassifier...
>>
>
> It's the regular truncated SGD by Jonh Langford, which is identical to the
> method described in the FOBOS paper. Compared to the one in scikit-learn,
> it is more theoretically correct. The one in scikit-learn obtains sparser
> weight vectors in practice but has no theoretical justification (it's an
> heuristic). My goal was to compare coordinate descent with regular
> truncated/projected SGD so I didn't implement this heuristic.
>
ok - probably better to use this one (or the projection based method by
Duchi) - on the other hand, the Tsuruoka et al method served me quite well
in the past
thx,
Peter
>
>
> Mathieu
>
>
> ------------------------------------------------------------------------------
> Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
> Get 100% visibility into your production application - at no cost.
> Code-level diagnostics for performance bottlenecks with <2% overhead
> Download for free and get started troubleshooting in minutes.
> http://p.sf.net/sfu/appdyn_d2d_ap1
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Peter Prettenhofer
------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
