[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
--------------------------------
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined in
the next release cycle, and it's better to remove the "JavaDeveloperApiExample"
at present. Since currently we can not well support users to implement their
own algorithm using Estimator/Transformer by Java, it may mislead users.
> ML JavaDeveloperApiExample is broken
> ------------------------------------
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
> Affects Versions: 2.0.0
> Reporter: Yanbo Liang
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
