[
https://issues.apache.org/jira/browse/SPARK-11439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003384#comment-15003384
]
Kai Sasaki edited comment on SPARK-11439 at 11/13/15 1:50 AM:
--------------------------------------------------------------
[~nakul02]
It seems to indicate the model in SparkR here not gmlnet. According to this
documentation, you can create SparkR linear model with {{glm}} function.
https://spark.apache.org/docs/latest/sparkr.html#machine-learning
This will call {{SparkRWrapper#fitRModelFormula}}. It returns
LinearRegressionModel with Pipeline when it receives "gaussian" as second
argument. So in summary we can write the code like this to use
{{LinearRegressionModel}} in SparkR.
{code}
df <- createDataFrame(sqlContext, iris) // You should replace with generated
data
fit <- glm(Sepal_Length ~ Sepal_Width + Species, data = df, family = "gaussian")
summary(fit)
$devianceResiduals
Min Max
-1.307112 1.412532
$coefficients
Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.251393 0.3697543 6.08889 9.568102e-09
Sepal_Width 0.8035609 0.106339 7.556598 4.187317e-12
Species_versicolor 1.458743 0.1121079 13.01195 0
Species_virginica 1.946817 0.100015 19.46525 0
{code}
In my environment, it seems to work.
was (Author: lewuathe):
[~nakul02]
It seems to indicate the model in SparkR here not gmlnet. According to this
documentation, you can create SparkR linear model with {{glm}} function.
https://spark.apache.org/docs/latest/sparkr.html#machine-learning
This will call {{SparkRWrapper#fitRModelFormula}}. It returns
LinearRegressionModel with Pipeline when it receives "gaussian" as second
argument. So in summary we can write the code like this to use
{{LinearRegressionModel}} in SparkR.
{code}
df <- createDataFrame(sqlContext, iris)
fit <- glm(Sepal_Length ~ Sepal_Width + Species, data = df, family = "gaussian")
summary(fit)
{code}
In my environment, it seems to work.
> Optimization of creating sparse feature without dense one
> ---------------------------------------------------------
>
> Key: SPARK-11439
> URL: https://issues.apache.org/jira/browse/SPARK-11439
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Kai Sasaki
> Priority: Minor
>
> Currently, sparse feature generated in {{LinearDataGenerator}} needs to
> create dense vectors once. It is cost efficient to prevent from generating
> dense feature when creating sparse features.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
