Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/10911
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175381894
I'm going to merge this. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175362788
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175362791
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175362482
**[Test build #50146 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50146/consoleFull)**
for PR 10911 at commit
[`3ff902a`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175324830
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175324834
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175324264
**[Test build #50126 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50126/consoleFull)**
for PR 10911 at commit
[`4a40802`](https://g
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175322730
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175322727
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175319396
**[Test build #50146 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50146/consoleFull)**
for PR 10911 at commit
[`3ff902a`](https://gi
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50926992
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketch.java ---
@@ -59,16 +59,17 @@
public enum Version {
/**
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50926614
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1270,4 +1270,37 @@ class DataFrameSuite extends QueryTest with
SharedS
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50926598
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -325,27 +321,43 @@ public void writeTo(OutputStream out)
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50925871
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1270,4 +1270,37 @@ class DataFrameSuite extends QueryTest with
SharedSQLCon
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50925607
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -325,27 +321,43 @@ public void writeTo(OutputStream out) thro
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50925566
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1270,4 +1270,37 @@ class DataFrameSuite extends QueryTest with
SharedS
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50925449
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -1270,4 +1270,37 @@ class DataFrameSuite extends QueryTest with
SharedS
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50925247
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -325,27 +321,43 @@ public void writeTo(OutputStream out)
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50925316
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -325,27 +321,43 @@ public void writeTo(OutputStream out)
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175305092
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175305090
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50919784
--- Diff: sql/core/pom.xml ---
@@ -44,6 +44,11 @@
org.apache.spark
+ spark-sketch_2.10
--- End diff --
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50919715
--- Diff: sql/core/pom.xml ---
@@ -44,6 +44,11 @@
org.apache.spark
+ spark-sketch_2.10
--- End diff --
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50918206
--- Diff: sql/core/pom.xml ---
@@ -44,6 +44,11 @@
org.apache.spark
+ spark-sketch_2.10
--- End diff --
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50914282
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -348,4 +374,30 @@ public static CountMinSketchImpl readF
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175275779
Josh is looking into the PySpark test failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175256282
**[Test build #50126 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50126/consoleFull)**
for PR 10911 at commit
[`4a40802`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175254946
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175254943
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175254410
**[Test build #50117 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50117/consoleFull)**
for PR 10911 at commit
[`fb23a24`](https://g
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50907571
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -348,4 +374,30 @@ public static CountMinSketchImpl readFrom(I
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50907599
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -348,4 +374,30 @@ public static CountMinSketchImpl readFrom(I
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50906771
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -17,17 +17,37 @@
package org.apache.spark.uti
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50906324
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -348,4 +374,30 @@ public static CountMinSketchImpl readF
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50893099
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -348,4 +374,30 @@ public static CountMinSketchImpl readF
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-175176766
**[Test build #50117 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50117/consoleFull)**
for PR 10911 at commit
[`fb23a24`](https://gi
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50880387
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -17,17 +17,37 @@
package org.apache.spark.uti
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50878077
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,84 @@ final class DataFrameStatFunctions private[sql](df:
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50877569
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,84 @@ final class DataFrameStatFunctions private[sql
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50802122
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,84 @@ final class DataFrameStatFunctions private[sql
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50801910
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,84 @@ final class DataFrameStatFunctions private[sql
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174848875
cc @JoshRosen is the python tests broken?
```
Running PySpark tests. Output is in
/home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log
Err
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174847369
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174847368
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174847279
**[Test build #50061 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50061/consoleFull)**
for PR 10911 at commit
[`32a9860`](https://g
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50797694
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -368,4 +379,30 @@ public static CountMinSketchImpl readFrom(I
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174816049
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174816044
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174815506
**[Test build #50055 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50055/consoleFull)**
for PR 10911 at commit
[`4e5d1af`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174785197
**[Test build #50061 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50061/consoleFull)**
for PR 10911 at commit
[`32a9860`](https://gi
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174783482
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174783481
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50787971
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -52,6 +57,10 @@
private double eps;
privat
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786975
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,88 @@ final class DataFrameStatFunctions private[sql](df:
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786852
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,88 @@ final class DataFrameStatFunctions private[sql](df:
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786646
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,88 @@ final class DataFrameStatFunctions private[sql](df:
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786602
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -309,4 +311,88 @@ final class DataFrameStatFunctions private[sql](df:
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786480
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -347,8 +356,10 @@ public void writeTo(OutputStream out) throw
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786439
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -41,7 +46,7 @@
* - ...
* - Row depth - 1 (wid
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50786449
--- Diff:
common/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java
---
@@ -52,6 +57,10 @@
private double eps;
private dou
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174761421
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174761419
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174760913
**[Test build #50055 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50055/consoleFull)**
for PR 10911 at commit
[`4e5d1af`](https://gi
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/10911#discussion_r50784455
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala ---
@@ -45,7 +48,6 @@ final class DataFrameStatFunctions private[sql](d
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/10911#issuecomment-174756777
cc @cloud-fan @rxin @yhuai
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/10911
[SPARK-12935][SQL] DataFrame API for Count-Min Sketch
This PR integrates Count-Min Sketch from spark-sketch into DataFrame. This
version resorts to `RDD.aggregate` for building the sketch. A more
67 matches
Mail list logo
