LinearRegression and model prediction threshold
Hi All,
I am using LinearRegression and have a question about the details on
model.predict method. Basically it is predicting variable y given an input
vector x. However, can someone point me to the documentation about what is the
threshold used in the predict method? Can that be changed ? I am assuming that
i/p vector essentially gets mapped to a number and is compared against a
threshold value and then y is either set to 0 or 1 based on those two numbers.
Another question I have is if I want to save the model to hdfs for later reuse
is there a recommended way for doing that?
// Building the model
val numIterations = 100
val model = LinearRegressionWithSGD.train(parsedData, numIterations)
// Evaluate model on training examples and compute training error
val valuesAndPreds = parsedData.map { point =
val prediction = model.predict(point.features)
(point.label, prediction)
}
Re: LinearRegression and model prediction threshold
It sounds like you are asking about logistic regression, not linear
regression. If so, yes that's just what it does. The default would be
0.5 in logistic regression. If you 'clear' the threshold you get the
raw margin out of this and other linear classifiers.
On Fri, Oct 31, 2014 at 7:18 PM, Sameer Tilak ssti...@live.com wrote:
Hi All,
I am using LinearRegression and have a question about the details on
model.predict method. Basically it is predicting variable y given an input
vector x. However, can someone point me to the documentation about what is
the threshold used in the predict method? Can that be changed ? I am
assuming that i/p vector essentially gets mapped to a number and is compared
against a threshold value and then y is either set to 0 or 1 based on those
two numbers.
Another question I have is if I want to save the model to hdfs for later
reuse is there a recommended way for doing that?
// Building the model
val numIterations = 100
val model = LinearRegressionWithSGD.train(parsedData, numIterations)
// Evaluate model on training examples and compute training error
val valuesAndPreds = parsedData.map { point =
val prediction = model.predict(point.features)
(point.label, prediction)
}
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
Re: LinearRegression and model prediction threshold
You can serialize the model to a local/hdfs file system and use it later
when you want.
Best Regards,
Sonal
Nube Technologies http://www.nubetech.co
http://in.linkedin.com/in/sonalgoyal
On Sat, Nov 1, 2014 at 12:02 AM, Sean Owen so...@cloudera.com wrote:
It sounds like you are asking about logistic regression, not linear
regression. If so, yes that's just what it does. The default would be
0.5 in logistic regression. If you 'clear' the threshold you get the
raw margin out of this and other linear classifiers.
On Fri, Oct 31, 2014 at 7:18 PM, Sameer Tilak ssti...@live.com wrote:
Hi All,
I am using LinearRegression and have a question about the details on
model.predict method. Basically it is predicting variable y given an
input
vector x. However, can someone point me to the documentation about what
is
the threshold used in the predict method? Can that be changed ? I am
assuming that i/p vector essentially gets mapped to a number and is
compared
against a threshold value and then y is either set to 0 or 1 based on
those
two numbers.
Another question I have is if I want to save the model to hdfs for later
reuse is there a recommended way for doing that?
// Building the model
val numIterations = 100
val model = LinearRegressionWithSGD.train(parsedData, numIterations)
// Evaluate model on training examples and compute training error
val valuesAndPreds = parsedData.map { point =
val prediction = model.predict(point.features)
(point.label, prediction)
}
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
