[GitHub] spark issue #22274: [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes for R sql...

2018-09-01 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22274 @felixcheung From system preference - Time zone : Pacific Daylight Time closest city :Cupertino, CA, United States --- --

[GitHub] spark issue #22274: [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes for R sql...

2018-09-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22274 maybe also your laptop's system time zone? could you also check that? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark

[GitHub] spark pull request #22274: [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes fo...

2018-09-01 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22274#discussion_r214530330 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3633,7 +3633,8 @@ test_that("catalog APIs, currentDatabase, setCurrentDatabase, listDatabases",

[GitHub] spark pull request #22295: [SPARK-25255][PYTHON]Add getActiveSession to Spar...

2018-09-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22295#discussion_r214530177 --- Diff: python/pyspark/sql/session.py --- @@ -252,6 +252,16 @@ def newSession(self): """ return self.__class__(self._sc, self

[GitHub] spark pull request #22298: [SPARK-25021][K8S] Add spark.executor.pyspark.mem...

2018-09-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22298#discussion_r214530079 --- Diff: examples/src/main/python/worker_memory_check.py --- @@ -0,0 +1,47 @@ +# --- End diff -- shouldn't this be in python tests (and

[GitHub] spark issue #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRelation ...

2018-09-01 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22270 @ueshin Opened (https://github.com/apache/spark/pull/22314) and (https://github.com/apache/spark/pull/22315). Thank you. --- ---

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214529571 --- Diff: python/pyspark/sql/functions.py --- @@ -1669,20 +1669,36 @@ def repeat(col, n): return Column(sc._jvm.functions.repeat(_to_java_column

[GitHub] spark issue #22314: [SPARK-25307][SQL] ArraySort function may return an erro...

2018-09-01 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22314 @kiszk Thanks. Can you please help look at [22315](https://github.com/apache/spark/pull/22315) as well. --- - To unsubscribe

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22315 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22315 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2764/

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22315 **[Test build #95586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95586/testReport)** for PR 22315 at commit [`84b135c`](https://github.com/apache/spark/commit/84

[GitHub] spark pull request #22315: [SPARK-25308][SQL] ArrayContains function may ret...

2018-09-01 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22315 [SPARK-25308][SQL] ArrayContains function may return a error in the code generation phase. ## What changes were proposed in this pull request? Invoking ArrayContains function with non nulla

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22313 Thank you for review, @kiszk . First, I don't want to hold the memory up after query completion. If we do, it will be a regression. So, I wanted `time` first. Second, It's dif

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22179 Thanks, @wangyum ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-01 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22179 @dongjoon-hyun I'm trying to add test cases. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22313 General question: Why do we use `time` instead of `entry size` to control cache? I am neutral on this decision. I would like to hear the reason of this decision. --- ---

[GitHub] spark issue #22314: [SPARK-25307] ArraySort function may return an error in ...

2018-09-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22314 Good catch, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark pull request #22313: [SPARK-25306][SQL] Use cache to speed up `createF...

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22313#discussion_r214528778 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFilters.scala --- @@ -55,19 +59,52 @@ import org.apache.spark.sql.types._ * k

[GitHub] spark pull request #22313: [SPARK-25306][SQL] Use cache to speed up `createF...

2018-09-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22313#discussion_r214528435 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFilters.scala --- @@ -55,19 +59,52 @@ import org.apache.spark.sql.types._ * kn

[GitHub] spark issue #22314: [SPARK-25307] ArraySort function may return an error in ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22314 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2763/

[GitHub] spark issue #22314: [SPARK-25307] ArraySort function may return an error in ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22314 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22314: [SPARK-25307] ArraySort function may return an error in ...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22314 **[Test build #95585 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95585/testReport)** for PR 22314 at commit [`7dec581`](https://github.com/apache/spark/commit/7d

[GitHub] spark pull request #22314: [SPARK-25307] ArraySort function may return an er...

2018-09-01 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22314 [SPARK-25307] ArraySort function may return an error in the code generation phase ## What changes were proposed in this pull request? Sorting array of booleans (not nullable) returns a comp

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22308 @cloud-fan @srowen This PR will make our jenkins build green. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22308 see https://github.com/apache/spark/pull/12924 and https://github.com/apache/spark/pull/11744 --- - To unsubscribe, e-mail: revie

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22310 @srowen It is about the sbt convention. see my demo project: https://github.com/sadhen/spark-25298 --- - To unsubscribe,

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22308 Remove the deprecated 2.10 jar. And build a 2.12 jar. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22262: [SPARK-25175][SQL] Field resolution should fail if there...

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22262 @seancxmao . Could you explain why we need this PR? Apache Spark 2.3.1 already shows exceptions like the following for the case (for both ORC and Parquet), doesn't it? ```scala sc

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22313 Could you review this PR, @gatorsmile and @cloud-fan ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org F

[GitHub] spark issue #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRelation ...

2018-09-01 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22270 @ueshin Thank you.. will do. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-01 Thread NiharS
Github user NiharS commented on the issue: https://github.com/apache/spark/pull/22192 Tried on a different machine and those tests pass, here's hoping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22313 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95583/ Test PASSed. ---

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22313 **[Test build #95583 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95583/testReport)** for PR 22313 at commit [`ac06b0c`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95584/testReport)** for PR 22192 at commit [`5a2852f`](https://github.com/apache/spark/commit/5a

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214526544 --- Diff: common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java --- @@ -394,12 +394,14 @@ public void substringSQL() {

[GitHub] spark issue #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRelation ...

2018-09-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22270 @dilipbiswal Let's fix the two functions in 2.4. Could you open a pr to fix the two functions? Thanks! --- - To unsubscribe, e-ma

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95582/ Test PASSed. ---

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22312 **[Test build #95582 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95582/testReport)** for PR 22312 at commit [`5772404`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95581/ Test PASSed. ---

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22312 **[Test build #95581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95581/testReport)** for PR 22312 at commit [`ad3a11d`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #22236: [SPARK-10697][ML] Add lift to Association rules

2018-09-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22236 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-09-01 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21721 I spent more hours to take a look at how SQL UI can update the metrics information before task ends, and now I guess I may understand what was the concern from @cloud-fan here. This is

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-09-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22236 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@

[GitHub] spark issue #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22179 Ping, @wangyum . Could you add the test cases for SPARK-20389, SPARK-23131, and SPARK-25176? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22179: [SPARK-23131][BUILD] Upgrade Kryo to 4.0.2

2018-09-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22179#discussion_r214524736 --- Diff: pom.xml --- @@ -1770,6 +1770,10 @@ org.apache.hive hive-storage-api + +

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter`

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22313 **[Test build #95583 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95583/testReport)** for PR 22313 at commit [`ac06b0c`](https://github.com/apache/spark/commit/ac

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter`

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2762/

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter`

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22313 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22313: [SPARK-25306][SQL] Use cache to speed up `createF...

2018-09-01 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/22313 [SPARK-25306][SQL] Use cache to speed up `createFilter` ## What changes were proposed in this pull request? In ORC data source, `createFilter` function has exponential time complexity

[GitHub] spark issue #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-09-01 Thread LucaCanali
Github user LucaCanali commented on the issue: https://github.com/apache/spark/pull/22218 Thanks @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22310 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22310 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95579/ Test PASSed. ---

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22310 **[Test build #95579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95579/testReport)** for PR 22310 at commit [`7c0b9b9`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/7 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95578/ Test FAILed. ---

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22312 **[Test build #95582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95582/testReport)** for PR 22312 at commit [`5772404`](https://github.com/apache/spark/commit/57

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/7 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2761/

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95578/testReport)** for PR 7 at commit [`d17d2df`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-09-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r214522065 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +76,28 @@ class ExecutorSource(threadPool: ThreadPoolExecutor, exe

[GitHub] spark issue #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-09-01 Thread LucaCanali
Github user LucaCanali commented on the issue: https://github.com/apache/spark/pull/22218 I have implemented the changes as from the latest comments by @maropu and @srowen --- - To unsubscribe, e-mail: reviews-unsu

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2760/

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22312 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21273: [SPARK-17916][SQL] Fix empty string being parsed as null...

2018-09-01 Thread koertkuipers
Github user koertkuipers commented on the issue: https://github.com/apache/spark/pull/21273 @HyukjinKwon see https://github.com/apache/spark/pull/22312 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o

[GitHub] spark issue #22312: [SPARK-17916][SQL] Fix new behavior when quote is set an...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22312 **[Test build #95581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95581/testReport)** for PR 22312 at commit [`ad3a11d`](https://github.com/apache/spark/commit/ad

[GitHub] spark pull request #22312: [SPARK-17916][SQL] Fix new behavior when quote is...

2018-09-01 Thread koertkuipers
GitHub user koertkuipers opened a pull request: https://github.com/apache/spark/pull/22312 [SPARK-17916][SQL] Fix new behavior when quote is set and fix old behavior when quote is unset ## What changes were proposed in this pull request? 1) Set nullValue to quoted empty str

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-09-01 Thread LucaCanali
Github user LucaCanali commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r214521537 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,29 @@ class ExecutorSource(threadPool: ThreadPoolExecutor,

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95577/ Test PASSed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #95577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95577/testReport)** for PR 22112 at commit [`63b3eb6`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22310 Why the move of the source code in the 2.11 tree? looks like it's not in the right package dir now. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22308 Oh what's the change here in the .jar file? just copied the jar file for 2.11/2.10? --- - To unsubscribe, e-mail: reviews-unsubs

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22311 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95580/ Test FAILed. ---

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22311 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22311 **[Test build #95580 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95580/testReport)** for PR 22311 at commit [`f94fdf7`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22308 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22308 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95576/ Test PASSed. ---

[GitHub] spark issue #22308: [SPARK-25304][SQL][TEST] Fix HiveSparkSubmitSuite SPARK-...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22308 **[Test build #95576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95576/testReport)** for PR 22308 at commit [`221fcac`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22311 @cloud-fan @gatorsmile @wangyum --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22311: [SPARK-25305][SQL] Respect attribute name in Coll...

2018-09-01 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22311#discussion_r214518048 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -649,17 +648,20 @@ object CollapseProject extend

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22311 Thanks @wangyum for finding the issue. Before this PR, he propose https://github.com/apache/spark/pull/22124 and https://github.com/apache/spark/pull/22287 . But there is no description f

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22311 **[Test build #95580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95580/testReport)** for PR 22311 at commit [`f94fdf7`](https://github.com/apache/spark/commit/f9

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22311 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2759/

[GitHub] spark issue #22311: [SPARK-25305][SQL] Respect attribute name in CollapsePro...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22311 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22311: [SPARK-25305][SQL] Respect attribute name in Coll...

2018-09-01 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/22311 [SPARK-25305][SQL] Respect attribute name in CollapseProject and ColumnPruning ## What changes were proposed in this pull request? Currently in optimizer rule `CollapseProject`, the

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22310 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22310 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22310 @srowen This is a better solution for #22280 No hard-coded 2.11 or 2.12 any more. --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22310: [Spark-25298][Build] Improve build definition for Scala ...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22310 **[Test build #95579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95579/testReport)** for PR 22310 at commit [`7c0b9b9`](https://github.com/apache/spark/commit/7c

[GitHub] spark pull request #22310: [Spark-25298][Build] Improve build definition for...

2018-09-01 Thread sadhen
GitHub user sadhen opened a pull request: https://github.com/apache/spark/pull/22310 [Spark-25298][Build] Improve build definition for Scala 2.12 ## What changes were proposed in this pull request? Improve build for Scala 2.12. ## How was this patch tested?

[GitHub] spark issue #22309: [SPARK-20384][CORE] Support value class in schema of Dat...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22309 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22309: [SPARK-20384][CORE] Support value class in schema of Dat...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22309 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22309: [SPARK-20384][core] Support value class in schema of Dat...

2018-09-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22309 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22309: [SPARK-20384][core] Support value class in schema...

2018-09-01 Thread mt40
GitHub user mt40 opened a pull request: https://github.com/apache/spark/pull/22309 [SPARK-20384][core] Support value class in schema of Dataset ## What changes were proposed in this pull request? This PR adds support for [Scala value class][1] in schema of Datasets (as both top

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-09-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95578/testReport)** for PR 7 at commit [`d17d2df`](https://github.com/apache/spark/commit/d1

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-01 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214516705 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,36 +229,74 @@ case class RLike(left:

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-09-01 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214516644 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,36 +229,74 @@ case class RLike(left:

  1   2   >