[GitHub] spark issue #21152: [SPARK-23688][SS] Refactor tests away from rate source

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21152 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89877/ Test PASSed. ---

[GitHub] spark issue #21152: [SPARK-23688][SS] Refactor tests away from rate source

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21152 **[Test build #89877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89877/testReport)** for PR 21152 at commit [`3eee5c9`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-04-26 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r184379580 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -288,6 +288,114 @@ case class ArrayConta

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21147 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21147 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89884/ Test FAILed. ---

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21147 **[Test build #89884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89884/testReport)** for PR 21147 at commit [`9672f92`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.ev...

2018-04-26 Thread mshtelma
Github user mshtelma commented on a diff in the pull request: https://github.com/apache/spark/pull/21147#discussion_r184376159 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -392,13 +392,13 @@ case cl

[GitHub] spark issue #21167: [SPARK-24100][PYSPARK][DStreams]Add the CompressionCodec...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21167 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21167: [SPARK-24100][PYSPARK][DStreams]Add the CompressionCodec...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21167 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2693/ Tes

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21166: [SPARK-11334][CORE] clear idle executors in executorIdTo...

2018-04-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21166 Can you please check again with latest master code, I doubt the issue is not valid any more in the latest code. --- - To unsub

[GitHub] spark pull request #21167: [SPARK-24100][PYSPARK][DStreams]Add the Compressi...

2018-04-26 Thread WzRaCai
GitHub user WzRaCai opened a pull request: https://github.com/apache/spark/pull/21167 [SPARK-24100][PYSPARK][DStreams]Add the CompressionCodec to the saveAsTextFiles interface. ## What changes were proposed in this pull request? Add the CompressionCodec to the saveAsTextFil

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #89886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89886/testReport)** for PR 20146 at commit [`a6551b0`](https://github.com/apache/spark/commit/a6

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-26 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20930 No wonder I can't understand the issue for a long time since I've thought it happened on Spark2.3 . And now it makes sense. Thanks @jiangxb1987 --- ---

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-26 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/20930 > Have you applied this patch: #17955 ? No, this happened on Spark 2.1. Thanks xingbo & wenchen, I'll port back this patch to our internal Spark 2.1. > That PR seems to be addr

[GitHub] spark pull request #20936: [SPARK-23503][SS] Enforce sequencing of committed...

2018-04-26 Thread efimpoberezkin
Github user efimpoberezkin commented on a diff in the pull request: https://github.com/apache/spark/pull/20936#discussion_r184361843 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochCoordinator.scala --- @@ -137,30 +137,65 @@ private[conti

[GitHub] spark issue #20936: [SPARK-23503][SS] Enforce sequencing of committed epochs...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20936 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89876/ Test PASSed. ---

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21156 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21156 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89875/ Test PASSed. ---

[GitHub] spark issue #20936: [SPARK-23503][SS] Enforce sequencing of committed epochs...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20936 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20936: [SPARK-23503][SS] Enforce sequencing of committed epochs...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20936 **[Test build #89876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89876/testReport)** for PR 20936 at commit [`b1b9985`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #89875 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89875/testReport)** for PR 21156 at commit [`a59c94f`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21106 I think `UnsafeProjectionCreator` is only useful for tests, because in production we should always try codegen first, and fallback to interpretation if codegen fails. There is no need to disable t

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89873/ Test PASSed. ---

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #89873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89873/testReport)** for PR 21021 at commit [`f2798f9`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21136 **[Test build #89885 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89885/testReport)** for PR 21136 at commit [`fed913d`](https://github.com/apache/spark/commit/fe

[GitHub] spark issue #20140: [SPARK-19228][SQL] Introduce tryParseDate method to proc...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20140 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r184351841 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -288,6 +288,114 @@ case class ArrayCont

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89883/ Test FAILed. ---

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #89883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89883/testReport)** for PR 21157 at commit [`c67ce29`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.ev...

2018-04-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21147 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21147 somehow I thought it has passed tests and I has merged it to master... Anyway this is a pretty safe change and I don't think it will break any tests. Let's see the test result later. --- --

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89872/ Test FAILed. ---

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #89872 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89872/testReport)** for PR 20146 at commit [`c33cff6`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21147 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2692/ Tes

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21147 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21147 **[Test build #89884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89884/testReport)** for PR 21147 at commit [`9672f92`](https://github.com/apache/spark/commit/96

[GitHub] spark issue #21166: [SPARK-11334][CORE] clear idle executors in executorIdTo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21166 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21166: [SPARK-11334][CORE] clear idle executors in executorIdTo...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21166 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21133: [SPARK-24013][SQL] Remove unneeded compress in Ap...

2018-04-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/21133#discussion_r184343803 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ApproximatePercentileQuerySuite.scala --- @@ -279,4 +282,10 @@ class ApproximatePercentileQ

[GitHub] spark pull request #21133: [SPARK-24013][SQL] Remove unneeded compress in Ap...

2018-04-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/21133#discussion_r184347998 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproximatePercentile.scala --- @@ -238,12 +238,6 @@ obj

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/21147 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark pull request #21166: [SPARK-11334][CORE] clear idle executors in execu...

2018-04-26 Thread sadhen
GitHub user sadhen opened a pull request: https://github.com/apache/spark/pull/21166 [SPARK-11334][CORE] clear idle executors in executorIdToTaskIds keySet ## What changes were proposed in this pull request? quote from #11205 > Executors may never be idle. Currentl

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #89883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89883/testReport)** for PR 21157 at commit [`c67ce29`](https://github.com/apache/spark/commit/c6

[GitHub] spark pull request #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API ...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184342231 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,201 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLa

[GitHub] spark pull request #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API ...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184346287 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,201 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLa

[GitHub] spark pull request #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API ...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184343934 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,201 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLa

[GitHub] spark pull request #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API ...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184344777 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,201 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLa

[GitHub] spark pull request #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API ...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184344901 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,201 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLa

[GitHub] spark pull request #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API ...

2018-04-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21119#discussion_r184345688 --- Diff: python/pyspark/ml/clustering.py --- @@ -1156,6 +1156,201 @@ def getKeepLastCheckpoint(self): return self.getOrDefault(self.keepLa

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/21157 Yes, we can backport some of the cloudpickle code to make the patch less fragile. This would be a nontrivial change in an already complex code, but I'd be happy to sketch this if there's a consen

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2691/ Tes

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #89882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89882/testReport)** for PR 21106 at commit [`68213fb`](https://github.com/apache/spark/commit/68

[GitHub] spark pull request #21063: [SPARK-23886][Structured Streaming] Update query ...

2018-04-26 Thread efimpoberezkin
Github user efimpoberezkin commented on a diff in the pull request: https://github.com/apache/spark/pull/21063#discussion_r184342675 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala --- @@ -111,7 +112,12 @@ trait ProgressReporter ex

[GitHub] spark issue #11205: [SPARK-11334][Core] Handle maximum task failure situatio...

2018-04-26 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/11205 @jerryshao I think the 2nd bullet has not been fixed in SPARK-13054. I use spark 2.1.1, and I still find that finished tasks remain in `private val executorIdToTaskIds = new mutable.HashM

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21136 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21136 **[Test build #89880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89880/testReport)** for PR 21136 at commit [`0f3bc0b`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21136 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89880/ Test FAILed. ---

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89878/ Test FAILed. ---

[GitHub] spark issue #20946: [SPARK-23565] [SS] New error message for structured stre...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20946 **[Test build #89881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89881/testReport)** for PR 20946 at commit [`b35acf5`](https://github.com/apache/spark/commit/b3

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #89878 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89878/testReport)** for PR 21157 at commit [`67c4f67`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21136 **[Test build #89880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89880/testReport)** for PR 21136 at commit [`0f3bc0b`](https://github.com/apache/spark/commit/0f

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21157 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21136 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark

[GitHub] spark issue #20946: [SPARK-23565] [SS] New error message for structured stre...

2018-04-26 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20946 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21157 I don't like the hack too but the complete removal just basically means we are going to drop namedtuple supports in RDD without, for example, any deprecation warnings. Spark is being super conse

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21157 **[Test build #89878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89878/testReport)** for PR 21157 at commit [`67c4f67`](https://github.com/apache/spark/commit/67

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89879/testReport)** for PR 20959 at commit [`d4d9d65`](https://github.com/apache/spark/commit/d4

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-04-26 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/21157 > Does the test even pass? The tests should pass module the tests specifically checking the behaviour being removed. I think the failing RDD test is in this group as well. > Why

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-26 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21136 okey to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spa

[GitHub] spark issue #20946: [SPARK-23565] [SS] New error message for structured stre...

2018-04-26 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20946 okey to test LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #21165: Spark 20087: Attach accumulators / metrics to 'TaskKille...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21165 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21152: [SPARK-23688][SS] Refactor tests away from rate source

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21152 **[Test build #89877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89877/testReport)** for PR 21152 at commit [`3eee5c9`](https://github.com/apache/spark/commit/3e

[GitHub] spark issue #21165: Spark 20087: Attach accumulators / metrics to 'TaskKille...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21165 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21152: [SPARK-23688][SS] Refactor tests away from rate source

2018-04-26 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21152 Thanks fore reviewing. I have addressed review comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org F

[GitHub] spark issue #21165: Spark 20087: Attach accumulators / metrics to 'TaskKille...

2018-04-26 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21165 cc @squito @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark pull request #21165: Spark 20087: Attach accumulators / metrics to 'Ta...

2018-04-26 Thread advancedxy
GitHub user advancedxy opened a pull request: https://github.com/apache/spark/pull/21165 Spark 20087: Attach accumulators / metrics to 'TaskKilled' end reason ## What changes were proposed in this pull request? The ultimate goal is for listeners to onTaskEnd to receive metrics wh

[GitHub] spark pull request #21152: [SPARK-23688][SS] Refactor tests away from rate s...

2018-04-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21152#discussion_r184322699 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -66,157 +66,115 @@ class ContinuousSuite exten

[GitHub] spark pull request #21152: [SPARK-23688][SS] Refactor tests away from rate s...

2018-04-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21152#discussion_r184321762 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -66,157 +66,115 @@ class ContinuousSuite exten

[GitHub] spark issue #21164: [SPARK-24098][SQL] ScriptTransformationExec should wait ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21164 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21164: [SPARK-24098][SQL] ScriptTransformationExec should wait ...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21164 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21164: [SPARK-24098][SQL] ScriptTransformationExec should wait ...

2018-04-26 Thread liutang123
Github user liutang123 commented on the issue: https://github.com/apache/spark/pull/21164 Bug Reappearance: 1. Add Thread.sleep(1000 * 600) before assign for _exception. 2. structure a python script witch will throw exception like follow: test.py ```import sys for l

[GitHub] spark pull request #21152: [SPARK-23688][SS] Refactor tests away from rate s...

2018-04-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21152#discussion_r184316769 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -66,157 +66,115 @@ class ContinuousSuite extends

[GitHub] spark pull request #21164: [SPARK-24098][SQL] ScriptTransformationExec shoul...

2018-04-26 Thread liutang123
GitHub user liutang123 opened a pull request: https://github.com/apache/spark/pull/21164 [SPARK-24098][SQL] ScriptTransformationExec should wait process exiting before output iterator finish When feed thread doesn't set its _exception variable and the progress doesn't exit complete

[GitHub] spark pull request #21152: [SPARK-23688][SS] Refactor tests away from rate s...

2018-04-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21152#discussion_r184315510 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -66,157 +66,115 @@ class ContinuousSuite extends

[GitHub] spark issue #21163: [SPARK-24097][ML] Instruments improvements - RandomFores...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21163 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89874/ Test PASSed. ---

[GitHub] spark issue #21163: [SPARK-24097][ML] Instruments improvements - RandomFores...

2018-04-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21163 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21163: [SPARK-24097][ML] Instruments improvements - RandomFores...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21163 **[Test build #89874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89874/testReport)** for PR 21163 at commit [`cd5ea88`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-04-26 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r184310311 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -288,6 +288,114 @@ case class ArrayContain

[GitHub] spark issue #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.evaluateI...

2018-04-26 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/21147 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

[GitHub] spark pull request #21147: [SPARK-23799][SQL][FOLLOW-UP] FilterEstimation.ev...

2018-04-26 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/21147#discussion_r184308017 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala --- @@ -392,13 +392,13 @@ case class

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-04-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r184305294 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -288,6 +288,114 @@ case class ArrayCont

[GitHub] spark issue #20936: [SPARK-23503][Structured Streaming] Enforcing sequencing...

2018-04-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20936 **[Test build #89876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89876/testReport)** for PR 20936 at commit [`b1b9985`](https://github.com/apache/spark/commit/b1

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-04-26 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20930 Have you applied this patch: https://github.com/apache/spark/pull/17955 ? --- - To unsubscribe, e-mail: reviews-unsubscr...@s

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-04-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21021 Based on the recent trend, I will support codegen to exploit the opportunity of WholeStageCodegen. --- - To unsubscribe, e-mail: r

<    1   2   3   4   5   >