Hi I think I was a little confused, I'll try to summarize what I understand is needed:
* the goal is to have multilayer perceptron with stochastic gradient descent and maybe other learning algorithms * basic implementation of sgd already exists * existing implementation is missing some loss-functions, we need to use some from sgd-fast and implement others * existing implementation is slow, the final implementation should be written in Cython Am I correct? I have experimented with Cython and realized that there will be work needed to make the mlp faster. The implementation as it is now is not very Cython friendly. PS: the python builtin function id shouldn't be used as linear output layer ;-) David On Tue, Mar 20, 2012 at 9:05 PM, Andreas <[email protected]> wrote: > >>> I recently posted a gist: https://gist.github.com/2061456 >>> And there is also a branch by me: >>> https://github.com/amueller/scikit-learn/tree/multilayer_perceptron >>> >> I'll look into it. One thing I noticed is that you assume only one >> hidden layer. Is it feature? > I think having one hidden layer makes the implementation > more straight forward and is quite good in practice. Also it makes > model-selection a lot easier. > If there are strong opinions (and good reasons) for supporting > multiple hidden layers, than we can also do that. > >>> These would be good places to start. >>> >>> This implements SGD. >>> My biggest wish would be to do the SGD in Cython, which >>> should result in a major speedup. This can probably >>> be done using the recently used helper classes from >>> the linear SGD, which might need to be extended. >>> >> I've skipped after http://wiki.cython.org/tutorials/numpy and Cython >> tutorial ( http://conference.scipy.org/proceedings/SciPy2009/paper_1/ >> ) and it looks that it shouldn't be that hard to implement the >> algorithm in Python and then add types and additional info to improve >> speed of slow parts (main loop and propagations). >> >> > Well the algorithm is implemented in the code I linked to ;) > It is not fully tested but it should return reasonable results > on the datasets included in the scikit. >>> As far as deep learning goes: I agree, this is to much >>> and I think it is at the moment not in the scope of the >>> scikit. >>> >>> Apart from writing the SGD, there should be support for >>> differnt loss-functions. In my current implementation, the >>> hinge-loss support is not working yet. >>> >> I found loss functions in sgd_fast.pyx. Shouldn't they be used? >> >> > These are only for two classes. Depending on how the > two-class case will be handled by the network, > they could be used there. > > > Cheers, > Andy ------------------------------------------------------------------------------ This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
