[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
--------------------------------
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" at present. Since currently
we can not well support users' own learning algorithm using
Estimator/Transformer by Java, it may mislead users.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends Params
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" temporary since it may
mislead users.
> ML JavaDeveloperApiExample is broken
> ------------------------------------
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
> Reporter: Yanbo Liang
> Priority: Minor
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
> {code}
> The shared params such as "featuresCol" should also have Java-friendly
> wrappers.
> Look through the code base, I found we only implement JavaParams who is the
> wrappers of Scala Params.
> We still need Java-friendly wrappers for other traits who extends from Scala
> Params.
> For example, in Scala we have:
> {code}
> trait HasLabelCol extends Params
> {code}
> We should have the Java-friendly wrappers as follows:
> {code}
> class JavaHasLabelCol extends Params
> {code}
> Otherwise, "Param.parent" will be null for each param and it will throw
> exceptions when calling "Param.hasParam()".
> I think the Java compatibility for Params/Param should be further defined,
> and it's better to remove the "JavaDeveloperApiExample" at present. Since
> currently we can not well support users' own learning algorithm using
> Estimator/Transformer by Java, it may mislead users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
