Re: [Spark MLlib] about linear regression issue

2015-11-04 Thread Zhiliang Zhu
Hi DB Tsai, Firstly I must show my deep appreciation towards your kind help. Did you just mean like this, currently there is no way for users to deal with constrains like all weights >= 0 in spark, though spark also has LBFGS ... Moreover, I did not know whether spark SVD will help some for that

[Spark MLlib] about linear regression issue

2015-11-01 Thread Zhiliang Zhu
Dear All, As for N dimension linear regression, while the labeled training points number (or the rank of the labeled point space) is less than N, then from perspective of math, the weight of the trained linear model may be not unique.  However, the output of model.weight() by spark may be with

Re: [Spark MLlib] about linear regression issue

2015-11-01 Thread DB Tsai
For the constrains like all weights >=0, people do LBFGS-B which is supported in our optimization library, Breeze. https://github.com/scalanlp/breeze/issues/323 However, in Spark's LiR, our implementation doesn't have constrain implementation. I do see this is useful given we're experimenting