[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108103752 [Test build #872 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/872/consoleFull) for PR 6562 at commit [`c16821b`](https://github.com/apache/spark/commit/c16821b9861e60a2998ad99c9f6ee64405b73f7d). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108070264 [Test build #872 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/872/consoleFull) for PR 6562 at commit [`c16821b`](https://github.com/apache/spark/commit/c16821b9861e60a2998ad99c9f6ee64405b73f7d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108070098 streaming failure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108065016 [Test build #34005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/34005/consoleFull) for PR 6562 at commit [`c16821b`](https://github.com/apache/spark/commit/c16821b9861e60a2998ad99c9f6ee64405b73f7d). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108065046 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108058452 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31558805
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala ---
@@ -345,6 +346,12 @@ class DistributedLDAModel private (
}
}
+ /** Java-friendly version of [[topicDistributions]] */
+ def javaTopicDistributions: JavaPairRDD[java.lang.Long, Vector] = {
+new JavaPairRDD[java.lang.Long, Vector](
--- End diff --
They are the same. Java compiler can infer the type in `fromRDD` so you
don't have to write `[java.lang.Long, Vector]` three times.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108038589 [Test build #34005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/34005/consoleFull) for PR 6562 at commit [`c16821b`](https://github.com/apache/spark/commit/c16821b9861e60a2998ad99c9f6ee64405b73f7d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108037911 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-108037814 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31552762
--- Diff:
mllib/src/test/java/org/apache/spark/mllib/stat/JavaStatisticsSuite.java ---
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.mllib.stat;
+
+import java.io.Serializable;
+
+import com.google.common.collect.Lists;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.api.java.JavaSparkContext;
+
+public class JavaStatisticsSuite implements Serializable {
+ private transient JavaSparkContext sc;
+
+ @Before
+ public void setUp() {
+sc = new JavaSparkContext("local", "JavaStatistics");
+ }
+
+ @After
+ public void tearDown() {
+sc.stop();
+sc = null;
+ }
+
+ @Test
+ public void runStatisticsUsingStaticMethods() {
--- End diff --
Oops, too much code copying!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31552623
--- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java
---
@@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {
set(myStringParam_, value); return this;
}
+ private DoubleArrayParam myDoubleArrayParam_;
+ public DoubleArrayParam myDoubleArrayParam() { return
myDoubleArrayParam_; }
+
+ public double[] getMyDoubleArrayParam() { return
getOrDefault(myDoubleArrayParam_); }
+
+ public JavaTestParams setMyDoubleArrayParam(double[] value) {
+set(myDoubleArrayParam_, value); return this;
+ }
+
private void init() {
-myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
ParamValidators.gt(0));
+myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
+ ParamValidators.gt(0));
--- End diff --
I think I was confused & thinking 80 chars for Java, but I'll revert it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31552366
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala ---
@@ -345,6 +346,12 @@ class DistributedLDAModel private (
}
}
+ /** Java-friendly version of [[topicDistributions]] */
+ def javaTopicDistributions: JavaPairRDD[java.lang.Long, Vector] = {
+new JavaPairRDD[java.lang.Long, Vector](
--- End diff --
How is that different? Looking at the definition of JavaPairRDD.fromRDD,
it looks like exactly the same call.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541414 --- Diff: mllib/src/test/java/org/apache/spark/mllib/classification/JavaStreamingLogisticRegressionSuite.java --- @@ -15,7 +15,7 @@ * limitations under the License. */ -package org.apache.spark.ml.classification; +package org.apache.spark.mllib.classification; --- End diff -- Thanks for catching this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541392
--- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java
---
@@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {
set(myStringParam_, value); return this;
}
+ private DoubleArrayParam myDoubleArrayParam_;
+ public DoubleArrayParam myDoubleArrayParam() { return
myDoubleArrayParam_; }
+
+ public double[] getMyDoubleArrayParam() { return
getOrDefault(myDoubleArrayParam_); }
+
+ public JavaTestParams setMyDoubleArrayParam(double[] value) {
+set(myDoubleArrayParam_, value); return this;
--- End diff --
move `return this;` to next line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541375 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -190,6 +186,7 @@ class DoubleParam(parent: String, name: String, doc: String, isValid: Double => def this(parent: Identifiable, name: String, doc: String) = this(parent.uid, name, doc) + /** Creates a param pair with the given value (for Java). */ --- End diff -- Interesting, it doesn't inherit the parent doc. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541385
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/Statistics.scala
---
@@ -96,6 +101,9 @@ object Statistics {
*/
def corr(x: RDD[Double], y: RDD[Double], method: String): Double =
Correlations.corr(x, y, method)
+ def corr(x: JavaRDD[java.lang.Double], y: JavaRDD[java.lang.Double],
method: String): Double =
--- End diff --
missing doc
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541432
--- Diff:
mllib/src/test/java/org/apache/spark/mllib/stat/JavaStatisticsSuite.java ---
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.mllib.stat;
+
+import java.io.Serializable;
+
+import com.google.common.collect.Lists;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+
+import org.apache.spark.api.java.JavaRDD;
+import org.apache.spark.api.java.JavaSparkContext;
+
+public class JavaStatisticsSuite implements Serializable {
+ private transient JavaSparkContext sc;
+
+ @Before
+ public void setUp() {
+sc = new JavaSparkContext("local", "JavaStatistics");
+ }
+
+ @After
+ public void tearDown() {
+sc.stop();
+sc = null;
+ }
+
+ @Test
+ public void runStatisticsUsingStaticMethods() {
--- End diff --
The test name is a little confusing. Based on the content, it should be
`testCorr`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541382
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala ---
@@ -345,6 +346,12 @@ class DistributedLDAModel private (
}
}
+ /** Java-friendly version of [[topicDistributions]] */
+ def javaTopicDistributions: JavaPairRDD[java.lang.Long, Vector] = {
+new JavaPairRDD[java.lang.Long, Vector](
--- End diff --
nit: `return
JavaPairRDD.fromRDD(topicDistributions.asInstanceOf[RDD[(java.lang.Long,
Vector)]])` (saving `[java.lang.Long, Vector]`)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541405
--- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java
---
@@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {
set(myStringParam_, value); return this;
}
+ private DoubleArrayParam myDoubleArrayParam_;
+ public DoubleArrayParam myDoubleArrayParam() { return
myDoubleArrayParam_; }
+
+ public double[] getMyDoubleArrayParam() { return
getOrDefault(myDoubleArrayParam_); }
+
+ public JavaTestParams setMyDoubleArrayParam(double[] value) {
+set(myDoubleArrayParam_, value); return this;
+ }
+
private void init() {
-myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
ParamValidators.gt(0));
+myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
+ ParamValidators.gt(0));
myDoubleParam_ = new DoubleParam(this, "myDoubleParam", "this is a
double param",
ParamValidators.inRange(0.0, 1.0));
List validStrings = Lists.newArrayList("a", "b");
myStringParam_ = new Param(this, "myStringParam", "this is a
string param",
ParamValidators.inArray(validStrings));
+myDoubleArrayParam_ =
+ new DoubleArrayParam(this, "myDoubleArrayParam", "this is a double
param");
+
setDefault(myIntParam_, 1);
+setDefault(myIntParam_.w(1));
setDefault(myDoubleParam_, 0.5);
setDefault(myIntParam().w(1), myDoubleParam().w(0.5));
+setDefault(myDoubleArrayParam_, new double[]{1.0, 2.0});
--- End diff --
space before `{`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541402
--- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java
---
@@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {
set(myStringParam_, value); return this;
}
+ private DoubleArrayParam myDoubleArrayParam_;
+ public DoubleArrayParam myDoubleArrayParam() { return
myDoubleArrayParam_; }
+
+ public double[] getMyDoubleArrayParam() { return
getOrDefault(myDoubleArrayParam_); }
+
+ public JavaTestParams setMyDoubleArrayParam(double[] value) {
+set(myDoubleArrayParam_, value); return this;
+ }
+
private void init() {
-myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
ParamValidators.gt(0));
+myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
+ ParamValidators.gt(0));
myDoubleParam_ = new DoubleParam(this, "myDoubleParam", "this is a
double param",
ParamValidators.inRange(0.0, 1.0));
List validStrings = Lists.newArrayList("a", "b");
myStringParam_ = new Param(this, "myStringParam", "this is a
string param",
ParamValidators.inArray(validStrings));
+myDoubleArrayParam_ =
+ new DoubleArrayParam(this, "myDoubleArrayParam", "this is a double
param");
+
setDefault(myIntParam_, 1);
+setDefault(myIntParam_.w(1));
--- End diff --
minor: We should test `myIntParam()` instead of `myIntParam_`. The latter
is private.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541397
--- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java
---
@@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {
set(myStringParam_, value); return this;
}
+ private DoubleArrayParam myDoubleArrayParam_;
+ public DoubleArrayParam myDoubleArrayParam() { return
myDoubleArrayParam_; }
+
+ public double[] getMyDoubleArrayParam() { return
getOrDefault(myDoubleArrayParam_); }
+
+ public JavaTestParams setMyDoubleArrayParam(double[] value) {
+set(myDoubleArrayParam_, value); return this;
+ }
+
private void init() {
-myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
ParamValidators.gt(0));
+myIntParam_ = new IntParam(this, "myIntParam", "this is an int param",
+ ParamValidators.gt(0));
--- End diff --
Was the line too wide?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/6562#discussion_r31541390
--- Diff:
mllib/src/test/java/org/apache/spark/ml/param/JavaParamsSuite.java ---
@@ -50,6 +50,7 @@ public void testParams() {
testParams.setMyIntParam(2).setMyDoubleParam(0.4).setMyStringParam("a");
Assert.assertEquals(testParams.getMyDoubleParam(), 0.4, 0.0);
Assert.assertEquals(testParams.getMyStringParam(), "a");
+Assert.assertArrayEquals(testParams.getMyDoubleArrayParam(), new
double[]{1.0, 2.0}, 0.0);
--- End diff --
space before `{`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107811443 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107811431 [Test build #33952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33952/consoleFull) for PR 6562 at commit [`d955581`](https://github.com/apache/spark/commit/d9555817975389c453b18c67f9b654e866248328). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107786411 [Test build #33952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33952/consoleFull) for PR 6562 at commit [`d955581`](https://github.com/apache/spark/commit/d9555817975389c453b18c67f9b654e866248328). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107786370 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107786364 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107786245 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107785965 [Test build #33941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33941/consoleFull) for PR 6562 at commit [`d955581`](https://github.com/apache/spark/commit/d9555817975389c453b18c67f9b654e866248328). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107785970 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107766847 [Test build #33941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33941/consoleFull) for PR 6562 at commit [`d955581`](https://github.com/apache/spark/commit/d9555817975389c453b18c67f9b654e866248328). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107766646 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107766591 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107747547 [Test build #33926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33926/consoleFull) for PR 6562 at commit [`29b6b0d`](https://github.com/apache/spark/commit/29b6b0df8c9bb591b128bce9129fe6b792837ff6). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107747555 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107734757 [Test build #33926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33926/consoleFull) for PR 6562 at commit [`29b6b0d`](https://github.com/apache/spark/commit/29b6b0df8c9bb591b128bce9129fe6b792837ff6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/6562 [SPARK-7529] [mllib] Added several Java-friendly APIs + unit tests Java-friendly APIs added: * GaussianMixture.run() * GaussianMixtureModel.predict() * DistributedLDAModel.javaTopicDistributions() * StreamingKMeans: trainOn, predictOn, predictOnValues * Statistics.corr * params * added doc to w() since Java docs do not inherit doc * removed non-Java-friendly w() from StringArrayParam and DoubleArrayParam * made DoubleArrayParam Java-friendly w() actually Java-friendly I generated the doc and verified all changes. CC: @mengxr You can merge this pull request into a Git repository by running: $ git pull https://github.com/jkbradley/spark java-api-1.4 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/6562.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6562 commit fe6dcfeb0694ea33b52ddd233ad3b12c9bb0a04b Author: Joseph K. Bradley Date: 2015-06-01T22:04:45Z Added several Java-friendly APIs + unit tests: NaiveBayes, GaussianMixture, LDA, StreamingKMeans, Statistics.corr, params commit 29b6b0df8c9bb591b128bce9129fe6b792837ff6 Author: Joseph K. Bradley Date: 2015-06-01T22:18:49Z small fixes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107734670 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6562#issuecomment-107734653 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
