Thanks, will do.
On Mon, Jan 6, 2014 at 6:21 PM, Reynold Xin <[email protected]> wrote: > Thanks. Why don't you submit a pr and then we can work on it? > > > On Jan 6, 2014, at 6:15 PM, Michael Kun Yang <[email protected]> > wrote: > > > > Hi Hossein, > > > > I can still use LabeledPoint with little modification. Currently I > convert > > the category into {0, 1} sequence, but I can do the conversion in the > body > > of methods or functions. > > > > In order to make the code run faster, I try not to use DoubleMatrix > > abstraction to avoid memory allocation; another reason is that jblas has > no > > data structure to handle symmetric matrix addition efficiently. > > > > My code is not very pretty because I handle matrix operations manually > (by > > indexing). > > > > If you think it is ok, I will make a pull request. > > > > > >> On Mon, Jan 6, 2014 at 5:34 PM, Hossein <[email protected]> wrote: > >> > >> Hi Michael, > >> > >> This sounds great. Would you please send these as a pull request. > >> Especially if you can make your Newtown method implementation generic > such > >> that it can later be used by other algorithms, it would be very helpful. > >> For example, you could add it as another optimization method under > >> mllib/optimization. > >> > >> Was there a particular reason you chose not use LabeledPoint? > >> > >> We have some instructions for contributions here: < > >> https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark > > > >> > >> Thanks, > >> > >> --Hossein > >> > >> > >> On Mon, Jan 6, 2014 at 11:33 AM, Michael Kun Yang <[email protected] > >>> wrote: > >> > >>> I actually have two versions: > >>> one is based on gradient descent like the logistic regression on mllib. > >>> the other is based on Newtown iteration, it is not as fast as SGD, but > we > >>> can get all the statistics from it like deviance, p-values and fisher > >> info. > >>> > >>> we can get confusion matrix in both versions > >>> > >>> the gradient descent version is just a modification of logistic > >> regression > >>> with my own implementation. I did not use LabeledPoints class. > >>> > >>> > >>> On Mon, Jan 6, 2014 at 11:13 AM, Evan Sparks <[email protected]> > >>> wrote: > >>> > >>>> Hi Michael, > >>>> > >>>> What strategy are you using to train the multinomial classifier? > >>>> One-vs-all? I've got an optimized version of that method that I've > been > >>>> meaning to clean up and commit for a while. In particular, rather than > >>>> shipping a (potentially very big) model with each map task, I ship it > >>> once > >>>> before each iteration with a broadcast variable. Perhaps we can > compare > >>>> versions and incorporate some of my optimizations into your code? > >>>> > >>>> Thanks, > >>>> Evan > >>>> > >>>>>> On Jan 6, 2014, at 10:57 AM, Michael Kun Yang <[email protected] > > > >>>>> wrote: > >>>>> > >>>>> Hi Spark-ers, > >>>>> > >>>>> I implemented a SGD version of multinomial logistic regression based > >> on > >>>>> mllib's optimization package. If this classifier is in the future > >> plan > >>> of > >>>>> mllib, I will be happy to contribute my code. > >>>>> > >>>>> Cheers > >> >
