[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205397802 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion {

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205397226 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion {

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205395435 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion {

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21103 Added few minor comments, but nothing serious to be solved right now. LGTM --- - To unsubscribe, e-mail: reviews-unsub

[GitHub] spark issue #21879: [SPARK-24927][BUILD][BRANCH-2.3] The scope of snappy-jav...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21879 **[Test build #93585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93585/testReport)** for PR 21879 at commit [`93c34da`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21879: [SPARK-24927][BUILD][BRANCH-2.3] The scope of snappy-jav...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21879 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21879: [SPARK-24927][BUILD][BRANCH-2.3] The scope of snappy-jav...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21879 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93585/ Test FAILed. ---

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21103 **[Test build #93589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93589/testReport)** for PR 21103 at commit [`cf76c1f`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93589/ Test PASSed. ---

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16476 **[Test build #93594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93594/testReport)** for PR 16476 at commit [`c75d786`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16476 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #16476: [SPARK-19084][SQL] Implement expression field

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93594/ Test FAILed. ---

[GitHub] spark issue #21826: [SPARK-24872] Remove the symbol β€œ||” of the β€œOR”...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21826 **[Test build #93596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93596/testReport)** for PR 21826 at commit [`7ab449f`](https://github.com/apache/spark/commit/7a

[GitHub] spark issue #21857: [SPARK-21274][SQL] Implement EXCEPT ALL clause.

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21857 **[Test build #93595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93595/testReport)** for PR 21857 at commit [`93abd51`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21857: [SPARK-21274][SQL] Implement EXCEPT ALL clause.

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21857 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93595/ Test FAILed. ---

[GitHub] spark issue #21857: [SPARK-21274][SQL] Implement EXCEPT ALL clause.

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21857 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21758 **[Test build #93590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93590/testReport)** for PR 21758 at commit [`c7600c2`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93590/ Test PASSed. ---

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21758 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21880 **[Test build #93593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93593/testReport)** for PR 21880 at commit [`ef2303d`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93593/ Test FAILed. ---

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21826: [SPARK-24872] Replace the symbol '||' of Or operator wit...

2018-07-26 Thread httfighter
Github user httfighter commented on the issue: https://github.com/apache/spark/pull/21826 I have submitted a new code. Could you help me review the code? Thank you! @HyukjinKwon @viirya @gatorsmile @rxin @hvanhovell In Hive, "||" performs the function of STRING concat, the symbol o

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21880 **[Test build #93591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93591/testReport)** for PR 21880 at commit [`e87c268`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93591/ Test FAILed. ---

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-07-26 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r205445392 --- Diff: python/pyspark/sql/tests.py --- @@ -5060,6 +5049,147 @@ def test_type_annotation(self): df = self.spark.range(1).select(pandas_udf(

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-07-26 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r205448677 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -94,36 +95,94 @@ object ExtractPythonUDFFromAggre

[GitHub] spark pull request #21882: [SPARK-24934][SQL] Handle missing upper/lower bou...

2018-07-26 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/21882 [SPARK-24934][SQL] Handle missing upper/lower bounds case in in-memory partition pruning ## What changes were proposed in this pull request? Looks we intentionally set `null` for lower

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21882 cc @cloud-fan, mind taking a look please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional c

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r205450947 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21882 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1354/

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21882 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21873: [SPARK-24919][BUILD] New linter rule for sparkContext.ha...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21873 **[Test build #93592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93592/testReport)** for PR 21873 at commit [`a20206d`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21880 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21882 **[Test build #93597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93597/testReport)** for PR 21882 at commit [`ea38b56`](https://github.com/apache/spark/commit/ea

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21880 **[Test build #93598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93598/testReport)** for PR 21880 at commit [`ef2303d`](https://github.com/apache/spark/commit/ef

[GitHub] spark issue #21873: [SPARK-24919][BUILD] New linter rule for sparkContext.ha...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21873 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93592/ Test PASSed. ---

[GitHub] spark issue #21873: [SPARK-24919][BUILD] New linter rule for sparkContext.ha...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21873 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1355/

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1356/

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21403 **[Test build #93599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93599/testReport)** for PR 21403 at commit [`f5fa2c4`](https://github.com/apache/spark/commit/f5

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21653 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21758 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #21802: [SPARK-23928][SQL] Add shuffle collection function.

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21802 @rxin Generally I agree with you, but currently the whole-stage-codegen doesn't support `CodegenFallback`, which means, if we don't implement codegen, this expression will stop WSC and hurt perf a

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21474 just to clarify here, so the failure case if you are using a shuffle server < 2.2, does it throw an obvious error to the user? --- ---

[GitHub] spark pull request #21880: [SPARK-24929][INFRA] Make merge script don't swal...

2018-07-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21880#discussion_r205470335 --- Diff: dev/merge_spark_pr.py --- @@ -331,6 +331,9 @@ def choose_jira_assignee(issue, asf_jira): assignee = asf_jira.user(raw_assig

[GitHub] spark pull request #21880: [SPARK-24929][INFRA] Make merge script don't swal...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21880#discussion_r205474983 --- Diff: dev/merge_spark_pr.py --- @@ -331,6 +331,9 @@ def choose_jira_assignee(issue, asf_jira): assignee = asf_jira.user(raw_

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1357/

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21880 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21880: [SPARK-24929][INFRA] Make merge script don't swallow Key...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21880 **[Test build #93600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93600/testReport)** for PR 21880 at commit [`8b6ea00`](https://github.com/apache/spark/commit/8b

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21474 oh no, good point Tom. it'll fail with ``` 18/07/26 07:15:02 WARN scheduler.TaskSetManager: Lost task 1.0 in stage 1.3 (TID 15, irashid-large-2.gce.cloudera.com, executor 2): FetchFaile

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

2018-07-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21474 https://issues.apache.org/jira/browse/SPARK-24936 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additiona

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-07-26 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r205478496 --- Diff: core/src/main/java/org/apache/hadoop/fs/SparkGlobber.java --- @@ -0,0 +1,293 @@ +/** + * Licensed to the Apache Software Foundation (A

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread pwoody
Github user pwoody commented on the issue: https://github.com/apache/spark/pull/21882 This is the same problem as #20935 , yeah? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21156: [SPARK-24087][SQL] Avoid shuffle when join keys a...

2018-07-26 Thread yucai
Github user yucai closed the pull request at: https://github.com/apache/spark/pull/21156 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-26 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/21156 closed by mistake, reopen it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark pull request #21156: [SPARK-24087][SQL] Avoid shuffle when join keys a...

2018-07-26 Thread yucai
GitHub user yucai reopened a pull request: https://github.com/apache/spark/pull/21156 [SPARK-24087][SQL] Avoid shuffle when join keys are a super-set of bucket keys ## What changes were proposed in this pull request? To improve the bucket join, when join keys are a super-se

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21882 oops it is. I didn't know. But this PR can target lower branches too since it's a correctness issue :-) --- - To unsubscribe

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #93601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93601/testReport)** for PR 21156 at commit [`f406062`](https://github.com/apache/spark/commit/f4

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r205483502 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1184,6 +1184,110 @@ case class ArraySo

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21882 Also, I believe this issue still can happen even after your PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apach

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r205483672 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RandomIndicesGenerator.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to th

[GitHub] spark issue #21802: [SPARK-23928][SQL] Add shuffle collection function.

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21802 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21156 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205484762 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205486499 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205488729 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread pwoody
Github user pwoody commented on the issue: https://github.com/apache/spark/pull/21882 The test case here works fine at least. The linked PR focuses on accurately collecting stats, so null bounds should be correct if they occur. --- ---

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21653 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205495042 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion {

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21882 @pwoody I am not sure I 100% agree on your last sentence. I agree that we should correct null bounds, but letting the users facing bugs returning wrong results meanwhile we find all the possible ca

[GitHub] spark issue #21826: [SPARK-24872] Replace the symbol '||' of Or operator wit...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21826 **[Test build #93596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93596/testReport)** for PR 21826 at commit [`7ab449f`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21653 I kicked off the test manually at https://spark-prs.appspot.com/users/hthuynh2. I dunno why the test triggering via comments stops workign on some prs ---

[GitHub] spark issue #21826: [SPARK-24872] Replace the symbol '||' of Or operator wit...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21826 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21653 **[Test build #4222 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4222/testReport)** for PR 21653 at commit [`b6585da`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #21826: [SPARK-24872] Replace the symbol '||' of Or operator wit...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21826 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93596/ Test PASSed. ---

[GitHub] spark issue #21882: [SPARK-24934][SQL] Handle missing upper/lower bounds cas...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21882 We should backport this one anyway. Actually the stats are logged in DEBUG level. So, I think we are fine. I guess no harm to add this check and get rid of this hole found, and this doesn't bloc

[GitHub] spark pull request #21883: [SPARK-24937][SQL] Datasource partition table sho...

2018-07-26 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21883 [SPARK-24937][SQL] Datasource partition table should load empty partitions ## What changes were proposed in this pull request? How to reproduce: ```sql spark-sql> CREATE TABLE tbl A

[GitHub] spark pull request #21882: [SPARK-24934][SQL] Handle missing upper/lower bou...

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21882#discussion_r205504149 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -183,6 +183,13 @@ case class InMemoryTableSc

[GitHub] spark issue #21883: [SPARK-24937][SQL] Datasource partition table should loa...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21883 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21883: [SPARK-24937][SQL] Datasource partition table should loa...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21883 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1358/

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #93603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93603/testReport)** for PR 21847 at commit [`be037bd`](https://github.com/apache/spark/commit/be

[GitHub] spark pull request #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO ...

2018-07-26 Thread lindblombr
Github user lindblombr commented on a diff in the pull request: https://github.com/apache/spark/pull/21847#discussion_r205505006 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -41,7 +42,7 @@ class AvroSerializer(rootCatalystType: DataTy

[GitHub] spark issue #21883: [SPARK-24937][SQL] Datasource partition table should loa...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21883 **[Test build #93602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93602/testReport)** for PR 21883 at commit [`90f99fc`](https://github.com/apache/spark/commit/90

[GitHub] spark pull request #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` w...

2018-07-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21850#discussion_r205505495 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -414,6 +414,9 @@ object SimplifyConditionals exten

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #93604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93604/testReport)** for PR 21847 at commit [`6f686d6`](https://github.com/apache/spark/commit/6f

[GitHub] spark pull request #21882: [SPARK-24934][SQL] Handle missing upper/lower bou...

2018-07-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21882#discussion_r205508221 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -183,6 +183,13 @@ case class InMemoryTable

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1359/

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21306 **[Test build #93605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93605/testReport)** for PR 21306 at commit [`1ca9115`](https://github.com/apache/spark/commit/1c

[GitHub] spark pull request #21882: [SPARK-24934][SQL] Handle missing upper/lower bou...

2018-07-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21882#discussion_r205504738 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/PartitionBatchPruningSuite.scala --- @@ -95,6 +103,12 @@ class PartitionBatchPru

[GitHub] spark pull request #21882: [SPARK-24934][SQL] Handle missing upper/lower bou...

2018-07-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21882#discussion_r205511472 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -183,6 +183,13 @@ case class InMemoryTableSca

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #93604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93604/testReport)** for PR 21847 at commit [`6f686d6`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93604/ Test FAILed. ---

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #4: SPARK-1137: Make ZK PersistenceEngine not crash for wrong se...

2018-07-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/4 **[Test build #42 has started](https://amplab.cs.berkeley.edu/jenkins/job/ubuntuSparkPRB/42/testReport)** for PR 4 at commit [`414d267`](https://github.com/apache/spark/commit/414d2673b31a72d8a9edb4f5d

<    1   2   3   4   5   6   7   >