On Mon, Mar 19, 2012 at 2:03 PM, Andreas <[email protected]> wrote: > Thanks for your email. I would appreciate it if we could keep the > discussion on the list for as long as possible, > as other people might have something to say or are interested.
Ok, my mistake. I wanted to get some general insight before embarrassing myself. :-) > I recently posted a gist: https://gist.github.com/2061456 > And there is also a branch by me: > https://github.com/amueller/scikit-learn/tree/multilayer_perceptron I'll look into it. One thing I noticed is that you assume only one hidden layer. Is it feature? Another thing is that I forgot there are different loss-functions in my proposal. > These would be good places to start. > > This implements SGD. > My biggest wish would be to do the SGD in Cython, which > should result in a major speedup. This can probably > be done using the recently used helper classes from > the linear SGD, which might need to be extended. I've skipped after http://wiki.cython.org/tutorials/numpy and Cython tutorial ( http://conference.scipy.org/proceedings/SciPy2009/paper_1/ ) and it looks that it shouldn't be that hard to implement the algorithm in Python and then add types and additional info to improve speed of slow parts (main loop and propagations). > As far as deep learning goes: I agree, this is to much > and I think it is at the moment not in the scope of the > scikit. > > Apart from writing the SGD, there should be support for > differnt loss-functions. In my current implementation, the > hinge-loss support is not working yet. I found loss functions in sgd_fast.pyx. Shouldn't they be used? > I feel it hard to judge the amount of work necessary > since I am not so experienced in Cython myself, but > I think it should be possible to extent SGD to other optimization > methods. Levenberg-Marquardt is definitely interesting > and would be good to have. > Maybe you can compare it with resilient backpropagation > and also include that if it proves helpful. > I would like to hear from the "SGD people" what they > think about this project and what they think is doable in the time. So do I. :-) David ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
