Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-26 Thread Zhiliang Zhu
Hi Meihua, DB  Tsai, Thanks very much for your all kind help.While I add some more LabeledPoint  in the training data, then the output result also seems much better. I will also try setFitIntercept(false) way . Currently I encounted some problem about algorithm optimization issue: f(x1, x2,

Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-26 Thread Zhiliang Zhu
Hi Meihua, I just found that setFitIntercept(false) is introduced since Spark 1.5.0, my current version is 1.4.0 . I shall also try that after update the version . Since you said brezee is probably used, I knew brezee is used under the bottom of spark ml.Would you help comment some more how to

Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-25 Thread DB Tsai
Column 4 is always constant, so no predictive power resulting zero weight. On Sunday, October 25, 2015, Zhiliang Zhu wrote: > Hi DB Tsai, > > Thanks very much for your kind reply help. > > As for your comment, I just modified and tested the key part of the codes: > >

Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-25 Thread Zhiliang Zhu
On Monday, October 26, 2015 11:26 AM, Zhiliang Zhu wrote: Hi DB Tsai, Thanks very much for your kind help. I  get it now. I am sorry that there is another issue, the weight/coefficient result is perfect while A is triangular matrix, however, while A

Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-25 Thread Meihua Wu
please add "setFitIntercept(false)" to your LinearRegression. LinearRegression by default includes an intercept in the model, e.g. label = intercept + features dot weight To get the result you want, you need to force the intercept to be zero. Just curious, are you trying to solve systems of

Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-25 Thread Zhiliang Zhu
Hi DB Tsai, Thanks very much for your kind reply help. As for your comment, I just modified and tested the key part of the codes:  LinearRegression lr = new LinearRegression()    .setMaxIter(1)    .setRegParam(0)    .setElasticNetParam(0);  //the number could be reset  final

[SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-25 Thread Zhiliang Zhu
Dear All, I have some program as below which makes me very much confused and inscrutable, it is about multiple dimension linear regression mode, the weight / coefficient is always perfect while the dimension is smaller than 4, otherwise it is wrong all the time.Or, whether the

Re: [SPARK MLLIB] could not understand the wrong and inscrutable result of Linear Regression codes

2015-10-25 Thread DB Tsai
LinearRegressionWithSGD is not stable. Please use linear regression in ML package instead. http://spark.apache.org/docs/latest/ml-linear-methods.html Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0xAF08DF8D On Sun, Oct 25,