[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20352 **[Test build #86494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86494/testReport)** for PR 20352 at commit [`f8ae698`](https://github.com/apache/spark/commit/f8

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread neilalex
Github user neilalex commented on the issue: https://github.com/apache/spark/pull/20352 sure --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apac

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20201 **[Test build #86489 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86489/testReport)** for PR 20201 at commit [`48b9fda`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #20292: [SPARK-23129][CORE] Make deserializeStream of Dis...

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20292#discussion_r163074869 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -463,21 +463,21 @@ class ExternalAppendOnlyMap[K, V,

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86489/ Test PASSed. ---

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20201 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20353: [SPARK-20664][core] Follow-up: remove workaround ...

2018-01-22 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/20353 [SPARK-20664][core] Follow-up: remove workaround for SPARK-23103. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vanzin/spark SPARK-20664 Alte

[GitHub] spark issue #20353: [SPARK-20664][core] Follow-up: remove workaround for SPA...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/112/ Test

[GitHub] spark issue #20353: [SPARK-20664][core] Follow-up: remove workaround for SPA...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20353 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20353: [SPARK-20664][core] Follow-up: remove workaround for SPA...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20353 **[Test build #86497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86497/testReport)** for PR 20353 at commit [`390cf8b`](https://github.com/apache/spark/commit/39

[GitHub] spark issue #20091: [SPARK-22465][FOLLOWUP] Update the number of partitions ...

2018-01-22 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/20091 Thanks @jiangxb1987 for the great work ! I will merge this in when the build successfully completes. --- - To unsubscribe, e-

[GitHub] spark pull request #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManage...

2018-01-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19671#discussion_r163081760 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1322,33 +1322,55 @@ class BlockManagerSuite extends SparkFunSuite with

[GitHub] spark pull request #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManage...

2018-01-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19671#discussion_r163082224 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1322,33 +1322,55 @@ class BlockManagerSuite extends SparkFunSuite with

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20352 **[Test build #86494 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86494/testReport)** for PR 20352 at commit [`f8ae698`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86494/ Test PASSed. ---

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20352 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20295: [WIP][SPARK-23011] Support alternative function f...

2018-01-22 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20295#discussion_r163081467 --- Diff: python/pyspark/serializers.py --- @@ -267,13 +267,13 @@ def load_stream(self, stream): """ Deserialize ArrowRecordBat

[GitHub] spark pull request #19992: [SPARK-22805][CORE] Use StorageLevel aliases in e...

2018-01-22 Thread superbobry
Github user superbobry closed the pull request at: https://github.com/apache/spark/pull/19992 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-22 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20203 I tried this out on a cluster and seemed fine. I also tried with a bad app, where all tasks fail, its not just because of the host, and all the executors show up as blacklisted, but I guess that is

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-22 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20203 @attilapiros can you please update the PR description to also mention node blacklisting (at least briefly) and file a jira for the followup work, and ping me & tom on it? lgtm --- ---

[GitHub] spark pull request #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API wit...

2018-01-22 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/20354 [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala ## What changes were proposed in this pull request? This syncs the ML Python API with Scala for differences found after the 2.3

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20354 @holdenk @MLnick , just found some minor doc differences between python and scala during the audit for 2.3 --- - To unsubscr

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/113/ Test

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20354 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20354 **[Test build #86498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86498/testReport)** for PR 20354 at commit [`46f354b`](https://github.com/apache/spark/commit/46

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #86490 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86490/testReport)** for PR 20203 at commit [`882bb9e`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86490/ Test PASSed. ---

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86498/ Test PASSed. ---

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20354 **[Test build #86498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86498/testReport)** for PR 20354 at commit [`46f354b`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #20354: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20354 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20345: [SPARK-23172][SQL] Respect Project nodes in ReorderJoin

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #20344: [MINOR] Typo fixes

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20344 **[Test build #4071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4071/testReport)** for PR 20344 at commit [`78c5ef5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20345: [SPARK-23172][SQL] Respect Project nodes in ReorderJoin

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20345: [SPARK-23172][SQL] Respect Project nodes in ReorderJoin

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/114/ Test

[GitHub] spark issue #20345: [SPARK-23172][SQL] Respect Project nodes in ReorderJoin

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #86499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86499/testReport)** for PR 20345 at commit [`ca65b9d`](https://github.com/apache/spark/commit/ca

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20350 **[Test build #86495 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86495/testReport)** for PR 20350 at commit [`fcd665e`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86495/ Test FAILed. ---

[GitHub] spark issue #20350: [SPARK-23179][SQL] Support option to throw exception if ...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20350 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20201 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark pull request #20201: [SPARK-22389][SQL] data source v2 partitioning re...

2018-01-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20201 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20351: [SPARK-23014][SS] Fully remove V1 memory sink.

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20351 **[Test build #86493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86493/testReport)** for PR 20351 at commit [`c75190b`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20351: [SPARK-23014][SS] Fully remove V1 memory sink.

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20351 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86493/ Test FAILed. ---

[GitHub] spark issue #20351: [SPARK-23014][SS] Fully remove V1 memory sink.

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20351 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20201 @cloud-fan, please ping me to review PRs for DataSourceV2. Our new table format uses it and we're preparing some changes, so I want to make sure we're heading in the same direction for this. --- -

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19872 **[Test build #86492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86492/testReport)** for PR 19872 at commit [`cc659bc`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86492/ Test PASSed. ---

[GitHub] spark issue #19872: [SPARK-22274][PYTHON][SQL] User-defined aggregation func...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19872 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2018-01-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19861 @jiangxb1987, @cloud-fan, what was the use case you needed to add this for? --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #20342: [SPARK-23170][SQL] Dump the statistics of effective runs...

2018-01-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20342 ah, ok. I wait for the work! Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-01-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20343 ok, I'll try and check. Just s sec. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark pull request #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCD...

2018-01-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20343#discussion_r163112223 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -339,6 +340,24 @@ class TPCDSQuerySuite extends BenchmarkQueryTest {

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19861 With `SessionConfigSupport`, you can use the datasource session configs more easily for that: 1. All configs with the name prefix `spark.datasource.$keyPrefix` will be imported in to the dat

[GitHub] spark issue #20345: [SPARK-23172][SQL] Respect Project nodes in ReorderJoin

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20345 This is not `respect project nodes`, this actually expand the `ReorderJoin` rule to allow handle the project-over-join nodes. --- --

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Respect Project nodes in Reord...

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r163113728 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -84,19 +84,46 @@ object ReorderJoin extends Rule[Logic

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Respect Project nodes in Reord...

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r163114625 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/JoinOptimizationSuite.scala --- @@ -46,6 +46,19 @@ class JoinOptimization

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Respect Project nodes in Reord...

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r163115184 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/JoinOptimizationSuite.scala --- @@ -46,6 +46,19 @@ class JoinOptimization

[GitHub] spark pull request #20345: [SPARK-23172][SQL] Respect Project nodes in Reord...

2018-01-22 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20345#discussion_r163115407 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/JoinOptimizationSuite.scala --- @@ -116,7 +124,12 @@ class JoinOptimizati

[GitHub] spark issue #20347: [SPARK-20129][Core] JavaSparkContext should use SparkCon...

2018-01-22 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20347 Using `getOrCreate` in constructor seems change the semantics. Maybe we can add a new static method for such usage in `JavaSparkContext`. ---

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20352 **[Test build #86500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86500/testReport)** for PR 20352 at commit [`01fc9e1`](https://github.com/apache/spark/commit/01

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20201 ah sorry I missed this, but it's not too late to do post-hoc reviews, any comments are welcome! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCD...

2018-01-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20343#discussion_r163119029 --- Diff: sql/core/src/test/resources/tpcds-v2.7.0/q11.sql --- @@ -0,0 +1,78 @@ +with year_total as ( + select c_customer_id customer_id + ,

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2018-01-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19861 @jiangxb1987, I understand what this does. I just wanted an example use case where it was necessary. What was the motivating *use case*? ---

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2018-01-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19861 @rdblue This is also what we already have for built-in data sources, e.g. `spark.sql.parquet.compression.codec`, `spark.sql.parquet.filterPushdown`, etc. ---

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread neilalex
Github user neilalex commented on the issue: https://github.com/apache/spark/pull/20352 @shivaram alright, should be good with the tests now -- let me know how it seems --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2018-01-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19861 Basically we want per-query options, which can be specified via `DataFrameReader/Writer.option`, and we also want per-session options, which can save users from typing same options again and again

[GitHub] spark pull request #20297: [SPARK-23020][CORE] Fix races in launcher code, t...

2018-01-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20297#discussion_r163119635 --- Diff: launcher/src/main/java/org/apache/spark/launcher/ChildProcAppHandle.java --- @@ -48,14 +48,16 @@ public synchronized void disconnect() {

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20343 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/115/ Test

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20343 **[Test build #86501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86501/testReport)** for PR 20343 at commit [`fff88d2`](https://github.com/apache/spark/commit/ff

[GitHub] spark issue #20091: [SPARK-22465][FOLLOWUP] Update the number of partitions ...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20091 **[Test build #86496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86496/testReport)** for PR 20091 at commit [`ccdc0e6`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20091: [SPARK-22465][FOLLOWUP] Update the number of partitions ...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20091 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20091: [SPARK-22465][FOLLOWUP] Update the number of partitions ...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20091 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86496/ Test PASSed. ---

[GitHub] spark pull request #20355: SPARK-23148: [SQL] Allow pathnames with special c...

2018-01-22 Thread henryr
GitHub user henryr opened a pull request: https://github.com/apache/spark/pull/20355 SPARK-23148: [SQL] Allow pathnames with special characters for CSV / … …JSON / text ## What changes were proposed in this pull request? Fix for JSON and CSV data sources when fi

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20355 **[Test build #86502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86502/testReport)** for PR 20355 at commit [`98da4cd`](https://github.com/apache/spark/commit/98

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20355 **[Test build #86502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86502/testReport)** for PR 20355 at commit [`98da4cd`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86502/ Test FAILed. ---

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20353: [SPARK-20664][core] Follow-up: remove workaround for SPA...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20353 **[Test build #86497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86497/testReport)** for PR 20353 at commit [`390cf8b`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/116/ Test

[GitHub] spark issue #20353: [SPARK-20664][core] Follow-up: remove workaround for SPA...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20353 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20353: [SPARK-20664][core] Follow-up: remove workaround for SPA...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20353 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86497/ Test PASSed. ---

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2018-01-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19861 Thanks for the example, @cloud-fan. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/117/ Test

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #86504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86504/testReport)** for PR 20345 at commit [`f1a6558`](https://github.com/apache/spark/commit/f1

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/118/ Test

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20355 **[Test build #86503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86503/testReport)** for PR 20355 at commit [`535234c`](https://github.com/apache/spark/commit/53

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20355 **[Test build #86503 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86503/testReport)** for PR 20355 at commit [`535234c`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86503/ Test FAILed. ---

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-01-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 Thanks! @jiangxb1987 I'll address your comments and check again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/119/ Test

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20355 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20355: SPARK-23148: [SQL] Allow pathnames with special characte...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20355 **[Test build #86505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86505/testReport)** for PR 20355 at commit [`9c56736`](https://github.com/apache/spark/commit/9c

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20352 **[Test build #86500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86500/testReport)** for PR 20352 at commit [`01fc9e1`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20352 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20352: [SPARK-21727][R] Allow multi-element atomic vector as co...

2018-01-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86500/ Test PASSed. ---

<    1   2   3   4   5   6   >