[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-15605: - Labels: bulk-closed (was: ) > ML JavaDeveloperApiExample is broken > > > Key: SPARK-15605 > URL: https://issues.apache.org/jira/browse/SPARK-15605 > Project: Spark > Issue Type: Bug > Components: Examples, ML >Affects Versions: 1.4.1, 1.5.2, 1.6.1, 2.0.0 >Reporter: Yanbo Liang >Priority: Major > Labels: bulk-closed > > This bug is reported by > http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html > . -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15605: -- Affects Version/s: 1.4.1 > ML JavaDeveloperApiExample is broken > > > Key: SPARK-15605 > URL: https://issues.apache.org/jira/browse/SPARK-15605 > Project: Spark > Issue Type: Bug > Components: Examples, ML >Affects Versions: 1.4.1, 1.5.2, 1.6.1, 2.0.0 >Reporter: Yanbo Liang > > This bug is reported by > http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html > . -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15605: -- Affects Version/s: 1.5.2 1.6.1 > ML JavaDeveloperApiExample is broken > > > Key: SPARK-15605 > URL: https://issues.apache.org/jira/browse/SPARK-15605 > Project: Spark > Issue Type: Bug > Components: Examples, ML >Affects Versions: 1.5.2, 1.6.1, 2.0.0 >Reporter: Yanbo Liang > > This bug is reported by > http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html > . -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined in
the next release cycle, and it's better to remove the "JavaDeveloperApiExample"
at present. Since currently we can not well support users to implement their
own algorithm using Estimator/Transformer by Java, it may mislead users.
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Affects Versions: 2.0.0
>Reporter: Yanbo Liang
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley updated SPARK-15605:
--
Priority: Major (was: Minor)
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Affects Versions: 2.0.0
>Reporter: Yanbo Liang
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
> {code}
> The shared params such as "featuresCol" should also have Java-friendly
> wrappers.
> Look through the code base, I found we only implement JavaParams who is the
> wrappers of Scala Params.
> We still need Java-friendly wrappers for other traits who extends from Scala
> Params.
> For example, in Scala we have:
> {code}
> trait HasLabelCol extends Params
> {code}
> We should have the Java-friendly wrappers as follows:
> {code}
> class JavaHasLabelCol extends Params
> {code}
> Otherwise, "Param.parent" will be null for each param and it will throw
> exceptions when calling "Param.hasParam()".
> I think the Java compatibility for Params/Param should be further defined in
> the next release cycle, and it's better to remove the
> "JavaDeveloperApiExample" at present. Since currently we can not well support
> users to implement their own algorithm using Estimator/Transformer by Java,
> it may mislead users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph K. Bradley updated SPARK-15605:
--
Affects Version/s: 2.0.0
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Affects Versions: 2.0.0
>Reporter: Yanbo Liang
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
> {code}
> The shared params such as "featuresCol" should also have Java-friendly
> wrappers.
> Look through the code base, I found we only implement JavaParams who is the
> wrappers of Scala Params.
> We still need Java-friendly wrappers for other traits who extends from Scala
> Params.
> For example, in Scala we have:
> {code}
> trait HasLabelCol extends Params
> {code}
> We should have the Java-friendly wrappers as follows:
> {code}
> class JavaHasLabelCol extends Params
> {code}
> Otherwise, "Param.parent" will be null for each param and it will throw
> exceptions when calling "Param.hasParam()".
> I think the Java compatibility for Params/Param should be further defined in
> the next release cycle, and it's better to remove the
> "JavaDeveloperApiExample" at present. Since currently we can not well support
> users to implement their own algorithm using Estimator/Transformer by Java,
> it may mislead users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined in
the next release cycle, and it's better to remove the "JavaDeveloperApiExample"
at present. Since currently we can not well support users to implement their
own algorithm using Estimator/Transformer by Java, it may mislead users.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" at present. Since currently
we can not well support users to implement their own algorithm using
Estimator/Transformer by Java, it may mislead users.
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Reporter: Yanbo Liang
>Priority: Minor
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" at present. Since currently
we can not well support users to implement their own algorithm using
Estimator/Transformer by Java, it may mislead users.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" at present. Since currently
we can not well support users' own learning algorithm using
Estimator/Transformer by Java, it may mislead users.
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Reporter: Yanbo Liang
>Priority: Minor
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param null__featuresCol does not belong
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
{code}
trait HasLabelCol extends Params
{code}
We should have the Java-friendly wrappers as follows:
{code}
class JavaHasLabelCol extends Params
{code}
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" at present. Since currently
we can not well support users' own learning algorithm using
Estimator/Transformer by Java, it may mislead users.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends Params
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" temporary since it may
mislead users.
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Reporter: Yanbo Liang
>Priority: Minor
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
> {code}
> The shared params such as "featuresCol" should also have Java-friendly
> wrappers.
> Look
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
But this does not solve all problems, it will throw new exceptions:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: requirement
failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
{code}
The shared params such as "featuresCol" should also have Java-friendly wrappers.
Look through the code base, I found we only implement JavaParams who is the
wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends Params
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" temporary since it may
mislead users.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
I found we only implement JavaParams who is the wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends Params
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" temporary since it may
mislead users.
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Reporter: Yanbo Liang
>Priority: Minor
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> But this does not solve all problems, it will throw new exceptions:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: requirement
> failed: Param null__featuresCol does not belong to myJavaLogReg_fbcf0d015036.
> {code}
> The shared params such as "featuresCol" should also have Java-friendly
> wrappers.
> Look through the code base, I found we only implement JavaParams who is the
> wrappers of Scala Params.
> We still need Java-friendly wrappers for other traits who extends from Scala
> Params.
> For example, in Scala we have:
> trait HasLabelCol extends Params
> We should have the Java-friendly wrappers as follows:
> class JavaHasLabelCol extends Params
> Otherwise, "Param.parent" will be null for each param and it will throw
> exceptions when calling
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[
https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanbo Liang updated SPARK-15605:
Description:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
To fix the issue mentioned in the mailing list, we can implement maxIter as
following:
{code}
private IntParam maxIter_;
public IntParam maxIter() {
return maxIter_;
}
public int getMaxIter() {
return (Integer) getOrDefault(maxIter_);
}
public MyJavaLogisticRegression setMaxIter(int value) {
return (MyJavaLogisticRegression) set(maxIter_, value);
}
private void init() {
maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
setDefault(maxIter(), 100);
}
{code}
I found we only implement JavaParams who is the wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends Params
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" temporary since it may
mislead users.
was:
This bug is reported by
http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
.
I found we only implement JavaParams who is the wrappers of Scala Params.
We still need Java-friendly wrappers for other traits who extends from Scala
Params.
For example, in Scala we have:
trait HasLabelCol extends Params
We should have the Java-friendly wrappers as follows:
class JavaHasLabelCol extends Params
Otherwise, "Param.parent" will be null for each param and it will throw
exceptions when calling "Param.hasParam()".
I think the Java compatibility for Params/Param should be further defined, and
it's better to remove the "JavaDeveloperApiExample" temporary since it may
mislead users.
> ML JavaDeveloperApiExample is broken
>
>
> Key: SPARK-15605
> URL: https://issues.apache.org/jira/browse/SPARK-15605
> Project: Spark
> Issue Type: Bug
> Components: Examples, ML
>Reporter: Yanbo Liang
>Priority: Minor
>
> This bug is reported by
> http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html
> .
> To fix the issue mentioned in the mailing list, we can implement maxIter as
> following:
> {code}
> private IntParam maxIter_;
> public IntParam maxIter() {
> return maxIter_;
> }
> public int getMaxIter() {
> return (Integer) getOrDefault(maxIter_);
> }
> public MyJavaLogisticRegression setMaxIter(int value) {
> return (MyJavaLogisticRegression) set(maxIter_, value);
> }
> private void init() {
> maxIter_ = new IntParam(this, "maxIter", "max number of iterations");
> setDefault(maxIter(), 100);
> }
> {code}
> I found we only implement JavaParams who is the wrappers of Scala Params.
> We still need Java-friendly wrappers for other traits who extends from Scala
> Params.
> For example, in Scala we have:
> trait HasLabelCol extends Params
> We should have the Java-friendly wrappers as follows:
> class JavaHasLabelCol extends Params
> Otherwise, "Param.parent" will be null for each param and it will throw
> exceptions when calling "Param.hasParam()".
> I think the Java compatibility for Params/Param should be further defined,
> and it's better to remove the "JavaDeveloperApiExample" temporary since it
> may mislead users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-15605: Description: This bug is reported by http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html . I found we only implement JavaParams who is the wrappers of Scala Params. We still need Java-friendly wrappers for other traits who extends from Scala Params. For example, in Scala we have: trait HasLabelCol extends Params We should have the Java-friendly wrappers as follows: class JavaHasLabelCol extends Params Otherwise, "Param.parent" will be null for each param and it will throw exceptions when calling "Param.hasParam()". I think the Java compatibility for Params/Param should be further defined, and it's better to remove the "JavaDeveloperApiExample" temporary since it may mislead users. was: This bug is reported by http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html . I found we only implement JavaParams who is the wrappers of Scala Params. We still need Java-friendly wrappers for other traits who extends from Scala Params. For example, in Scala we have: trait HasLabelCol extends Params We should have the Java-friendly wrappers as follows: class JavaHasLabelCol extends Params Otherwise, "Param.parent" will be null for each param and it will throw exceptions when calling "Param.hasParam()". I think the Java compatibility for Params/Param should be further defined, and it's better to remove the "JavaDeveloperApiExample" which will mislead users. > ML JavaDeveloperApiExample is broken > > > Key: SPARK-15605 > URL: https://issues.apache.org/jira/browse/SPARK-15605 > Project: Spark > Issue Type: Bug > Components: Examples, ML >Reporter: Yanbo Liang >Priority: Minor > > This bug is reported by > http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html > . > I found we only implement JavaParams who is the wrappers of Scala Params. > We still need Java-friendly wrappers for other traits who extends from Scala > Params. > For example, in Scala we have: > trait HasLabelCol extends Params > We should have the Java-friendly wrappers as follows: > class JavaHasLabelCol extends Params > Otherwise, "Param.parent" will be null for each param and it will throw > exceptions when calling "Param.hasParam()". > I think the Java compatibility for Params/Param should be further defined, > and it's better to remove the "JavaDeveloperApiExample" temporary since it > may mislead users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-15605: Description: This bug is reported by http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html . I found we only implement JavaParams who is the wrappers of Scala Params. We still need Java-friendly wrappers for other traits who extends from Scala Params. For example, in Scala we have: trait HasLabelCol extends Params We should have the Java-friendly wrappers as follows: class JavaHasLabelCol extends Params Otherwise, "Param.parent" will be null for each param and it will throw exceptions when calling "Param.hasParam()". I think the Java compatibility for Params/Param should be further defined, and it's better to remove the "JavaDeveloperApiExample" which will mislead users. was: This bug is reported by http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html . I found we only implement JavaParams who is the wrappers of Scala Params. We still need Java-friendly wrappers for other traits who extends from Scala Params. For example, in Scala we have: trait HasLabelCol extends Params We should have the Java-friendly wrappers as follows: class JavaHasLabelCol extends Params Otherwise, "Param.parent" will be null for each param and it will throw exceptions when calling "Param.hasParam()". I think the Java compatibility for Params/Param should be further defined, and it's better to remove the "" > ML JavaDeveloperApiExample is broken > > > Key: SPARK-15605 > URL: https://issues.apache.org/jira/browse/SPARK-15605 > Project: Spark > Issue Type: Bug > Components: Examples, ML >Reporter: Yanbo Liang >Priority: Minor > > This bug is reported by > http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html > . > I found we only implement JavaParams who is the wrappers of Scala Params. > We still need Java-friendly wrappers for other traits who extends from Scala > Params. > For example, in Scala we have: > trait HasLabelCol extends Params > We should have the Java-friendly wrappers as follows: > class JavaHasLabelCol extends Params > Otherwise, "Param.parent" will be null for each param and it will throw > exceptions when calling "Param.hasParam()". > I think the Java compatibility for Params/Param should be further defined, > and it's better to remove the "JavaDeveloperApiExample" which will mislead > users. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15605) ML JavaDeveloperApiExample is broken
[ https://issues.apache.org/jira/browse/SPARK-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-15605: Description: This bug is reported by http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html . I found we only implement JavaParams who is the wrappers of Scala Params. We still need Java-friendly wrappers for other traits who extends from Scala Params. For example, in Scala we have: trait HasLabelCol extends Params We should have the Java-friendly wrappers as follows: class JavaHasLabelCol extends Params Otherwise, "Param.parent" will be null for each param and it will throw exceptions when calling "Param.hasParam()". I think the Java compatibility for Params/Param should be further defined, and it's better to remove the "" was: This bug is reported by http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html . This is because we do not have excellent coverage for Java-friendly wrappers. I found we only implement JavaParams who is the wrappers of Scala Params. We still need Java-friendly wrappers for other traits who extends from Scala Params. For example, in Scala we have: trait HasLabelCol extends Params We should have the Java-friendly wrappers as follows: class JavaHasLabelCol extends JavaParams Then each params of the Estimator will be generated correctly and param validation will success. > ML JavaDeveloperApiExample is broken > > > Key: SPARK-15605 > URL: https://issues.apache.org/jira/browse/SPARK-15605 > Project: Spark > Issue Type: Bug > Components: Examples, ML >Reporter: Yanbo Liang >Priority: Minor > > This bug is reported by > http://apache-spark-developers-list.1001551.n3.nabble.com/Creation-of-SparkML-Estimators-in-Java-broken-td17710.html > . > I found we only implement JavaParams who is the wrappers of Scala Params. > We still need Java-friendly wrappers for other traits who extends from Scala > Params. > For example, in Scala we have: > trait HasLabelCol extends Params > We should have the Java-friendly wrappers as follows: > class JavaHasLabelCol extends Params > Otherwise, "Param.parent" will be null for each param and it will throw > exceptions when calling "Param.hasParam()". > I think the Java compatibility for Params/Param should be further defined, > and it's better to remove the "" -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
