Hi DB Tsai,
Thanks very much for your kind reply help.
As for your comment, I just modified and tested the key part of the codes:
LinearRegression lr = new LinearRegression()
.setMaxIter(10000)
.setRegParam(0)
.setElasticNetParam(0); //the number could be reset
final LinearRegressionModel model = lr.fit(training);
Now the output is much reasonable, however, x4 is always 0 while repeatedly
reset those parameters in lr , would you help some about it how to properly set
the parameters ...
Final w: [1.000000127825909,1.999999979185054,2.999999993307136,0.0]
Thank you,Zhiliang
On Monday, October 26, 2015 5:14 AM, DB Tsai <dbt...@dbtsai.com> wrote:
LinearRegressionWithSGD is not stable. Please use linear regression in
ML package instead.
http://spark.apache.org/docs/latest/ml-linear-methods.html
Sincerely,
DB Tsai
----------------------------------------------------------
Web: https://www.dbtsai.com
PGP Key ID: 0xAF08DF8D
On Sun, Oct 25, 2015 at 10:14 AM, Zhiliang Zhu
<zchl.j...@yahoo.com.invalid> wrote:
> Dear All,
>
> I have some program as below which makes me very much confused and
> inscrutable, it is about multiple dimension linear regression mode, the
> weight / coefficient is always perfect while the dimension is smaller than
> 4, otherwise it is wrong all the time.
> Or, whether the LinearRegressionWithSGD would be selected for another one?
>
> public class JavaLinearRegression {
> public static void main(String[] args) {
> SparkConf conf = new SparkConf().setAppName("Linear Regression
> Example");
> JavaSparkContext sc = new JavaSparkContext(conf);
> SQLContext jsql = new SQLContext(sc);
>
> //Ax = b, x = [1, 2, 3, 4] would be the only one output about weight
> //x1 + 2 * x2 + 3 * x3 + 4 * x4 = y would be the multiple linear mode
> List<LabeledPoint> localTraining = Lists.newArrayList(
> new LabeledPoint(30.0, Vectors.dense(1.0, 2.0, 3.0, 4.0)),
> new LabeledPoint(29.0, Vectors.dense(0.0, 2.0, 3.0, 4.0)),
> new LabeledPoint(25.0, Vectors.dense(0.0, 0.0, 3.0, 4.0)),
> new LabeledPoint(16.0, Vectors.dense(0.0, 0.0, 0.0, 4.0)));
>
> JavaRDD<LabeledPoint> training = sc.parallelize(localTraining).cache();
>
> // Building the model
> int numIterations = 1000; //the number could be reset large
> final LinearRegressionModel model =
> LinearRegressionWithSGD.train(JavaRDD.toRDD(training), numIterations);
>
> //the coefficient weights are perfect while dimension of LabeledPoint is
> SMALLER than 4.
> //otherwise the output is always wrong and inscrutable.
> //for instance, one output is
> //Final w:
> [2.537341836047772E25,-7.744333206289736E24,6.697875883454909E23,-2.6704705246777624E22]
> System.out.print("Final w: " + model.weights() + "\n\n");
> }
> }
>
> I would appreciate your kind help or guidance very much~~
>
> Thank you!
> Zhiliang
>
>
