[GitHub] spark issue #20676: [SPARK-23516][CORE] It is unnecessary to transfer unroll...

2018-02-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20676 I don't think unroll memory is storage memory from the interface. It's only true for unified memory manager. I'm -1 on this change unless you can convince the community to remove the static

[GitHub] spark issue #20687: [SPARK-25000][SQL] Fix complex type simplification rules...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87739/ Test FAILed. ---

[GitHub] spark issue #20687: [SPARK-25000][SQL] Fix complex type simplification rules...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20687: [SPARK-25000][SQL] Fix complex type simplification rules...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20687 **[Test build #87739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87739/testReport)** for PR 20687 at commit

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #87743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87743/testReport)** for PR 18906 at commit

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87738/ Test FAILed. ---

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20679 **[Test build #87738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87738/testReport)** for PR 20679 at commit

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87737/ Test PASSed. ---

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20681 **[Test build #87737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87737/testReport)** for PR 20681 at commit

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #87742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87742/testReport)** for PR 18906 at commit

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87742/ Test FAILed. ---

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #87742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87742/testReport)** for PR 18906 at commit

[GitHub] spark issue #20639: [SPARK-23288][SS] Fix output metrics with parquet sink

2018-02-27 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20639 this is ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20685: [SPARK-23524] Big local shuffle blocks should not be che...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20685 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20639: [SPARK-23288][SS] Fix output metrics with parquet sink

2018-02-27 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20639 @zsxwing as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20685: [SPARK-23524] Big local shuffle blocks should not be che...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20685 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1123/

[GitHub] spark issue #20685: [SPARK-23524] Big local shuffle blocks should not be che...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20685 **[Test build #87741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87741/testReport)** for PR 20685 at commit

[GitHub] spark pull request #20676: [SPARK-23516][CORE] It is unnecessary to transfer...

2018-02-27 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20676#discussion_r171128021 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -246,18 +246,18 @@ private[spark] class MemoryStore(

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20666 Thanks @HyukjinKwon @cloud-fan! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20688: [SPARK-23096][SS] Migrate rate source to V2

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20688 **[Test build #87740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87740/testReport)** for PR 20688 at commit

[GitHub] spark issue #20688: [SPARK-23096][SS] Migrate rate source to V2

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1122/

[GitHub] spark issue #20688: [SPARK-23096][SS] Migrate rate source to V2

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20688 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20688: [SPARK-23096][SS] Migrate rate source to V2

2018-02-27 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/20688 [SPARK-23096][SS] Migrate rate source to V2 ## What changes were proposed in this pull request? This PR migrate micro batch rate source to V2 API and rewrite UTs to suite V2 test.

[GitHub] spark issue #20676: [SPARK-23516][CORE] It is unnecessary to transfer unroll...

2018-02-27 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20676 Hi, @ConeyLiu. I don't think it's about compatibility. Because both ```StaticMemoryManager``` and ```UnifiedMemoryManager``` call the same function for release unroll memory, which is

[GitHub] spark issue #20676: [SPARK-23516][CORE] It is unnecessary to transfer unroll...

2018-02-27 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/20676 Yeah, I see that. I'm not sure it's OK to change. But I think we should follow the interface design, not the underlying implementation. ---

[GitHub] spark pull request #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser be...

2018-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20666 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19108: [SPARK-21898][ML] Feature parity for KolmogorovSmirnovTe...

2018-02-27 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19108 @sethah Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20666 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20676: [SPARK-23516][CORE] It is unnecessary to transfer...

2018-02-27 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20676#discussion_r171124468 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -246,18 +246,18 @@ private[spark] class MemoryStore(

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-02-27 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/13599 That would be awesome. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-02-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13599 Totally reasonable, how about if we did a group hangout or similar and then summarized our discussions back to the PR? It could be after Strata and I can get up early or go to bed late as needed

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87736/ Test PASSed. ---

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #87736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87736/testReport)** for PR 20295 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-02-27 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/13599 I am afraid I would not be present in Strata SJ, I live in Shanghai China, and may not be able to travel at time. --- - To

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87735/ Test PASSed. ---

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20666 **[Test build #87735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87735/testReport)** for PR 20666 at commit

[GitHub] spark pull request #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case bl...

2018-02-27 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/20667#discussion_r171121778 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerId.scala --- @@ -132,10 +133,15 @@ private[spark] object BlockManagerId {

[GitHub] spark pull request #20676: [SPARK-23516][CORE] It is unnecessary to transfer...

2018-02-27 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/20676#discussion_r171116974 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -246,18 +246,18 @@ private[spark] class MemoryStore(

[GitHub] spark issue #20676: [SPARK-23516][CORE] It is unnecessary to transfer unroll...

2018-02-27 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/20676 In `StaticMemoryManager`, the storage memory and unroll memory is managed separately, but, unroll memory is also storage memory, so we do not need release unroll memory really,Just need to

[GitHub] spark issue #20687: [SPARK-25000][SQL] Fix complex type simplification rules...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20687 **[Test build #87739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87739/testReport)** for PR 20687 at commit

[GitHub] spark issue #20687: [SPARK-25000][SQL] Fix complex type simplification rules...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20687: [SPARK-25000][SQL] Fix complex type simplification rules...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1121/

[GitHub] spark pull request #20687: [SPARK-25000][SQL] Fix complex type simplificatio...

2018-02-27 Thread henryr
GitHub user henryr opened a pull request: https://github.com/apache/spark/pull/20687 [SPARK-25000][SQL] Fix complex type simplification rules to apply to entire plan ## What changes were proposed in this pull request? Complex type simplification optimizer rules were not

[GitHub] spark pull request #20676: [SPARK-23516][CORE] It is unnecessary to transfer...

2018-02-27 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/20676#discussion_r171115245 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -246,18 +246,18 @@ private[spark] class MemoryStore(

[GitHub] spark pull request #20676: [SPARK-23516][CORE] It is unnecessary to transfer...

2018-02-27 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/20676#discussion_r171115071 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -246,18 +246,18 @@ private[spark] class MemoryStore(

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2018-02-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18339 Lets see what @BryanCutler thinks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-02-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19876 So now that it looks like 2.3 is pretty much wrapped up do folks have any thoughts? @mlnick @jkbradley @sethah ? --- - To

[GitHub] spark issue #20676: [SPARK-23516][CORE] It is unnecessary to transfer unroll...

2018-02-27 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/20676 This is for compatibility reasons. The memory management also support legacy memory management (`StaticMemoryManager`). In `StaticMemoryManager`, the storage memory and unroll memory is managed

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-02-27 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 cc @gatorsmile @hvanhovell @jiangxb1987 your review comments are applied, is there something else I should work on regarding this PR? ---

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-02-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15670 So yeah if anyone has the spoons to take this over it _should_ just be fixing the merge conflicts. Feel free to ping me on the PR. ---

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-02-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20522 Sure. I'll try, @gatorsmile . It'll take some time for me. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-27 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20638 @HyukjinKwon Oh, thanks. I guess I learned what "approved these changes" means. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20638: [SPARK-23417][python] Fix the build instructions ...

2018-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20638 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallb...

2018-02-27 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20678#discussion_r17018 --- Diff: docs/sql-programming-guide.md --- @@ -1689,6 +1689,10 @@ using the call `toPandas()` and when creating a Spark DataFrame from a Pandas Da

[GitHub] spark pull request #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallb...

2018-02-27 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20678#discussion_r17186 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1986,55 +1986,89 @@ def toPandas(self): timezone = None if

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20638 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-27 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20638 Do we want to merge this or does it need more eyes? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallb...

2018-02-27 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20678#discussion_r171110887 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1986,55 +1986,89 @@ def toPandas(self): timezone = None if

[GitHub] spark pull request #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallb...

2018-02-27 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20678#discussion_r171110674 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1986,55 +1986,89 @@ def toPandas(self): timezone = None if

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1120/

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20522: [SPARK-23355][SQL] convertMetastore should not ignore ta...

2018-02-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20522 @dongjoon-hyun Could you submit the PR to resolve all the related issues? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20679 **[Test build #87738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87738/testReport)** for PR 20679 at commit

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20679 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20295 cc @BryanCutler --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20658: [SPARK-23488][python] Add missing catalog methods to pyt...

2018-02-27 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20658 So it looks like adding the database field has been backed out for this change, which is probably the right thing to do, if we want to make that change we should do that in a separate JIRA & commit

[GitHub] spark pull request #20670: [SPARK-23405] Generate additional constraints for...

2018-02-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20670#discussion_r171102033 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -23,20 +23,23 @@ import

[GitHub] spark issue #20668: [SPARK-23510][SQL] Support Hive 2.2 and Hive 2.3 metasto...

2018-02-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20668 Why my PR https://github.com/apache/spark/pull/20671 does not fail? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20681 **[Test build #87737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87737/testReport)** for PR 20681 at commit

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1119/

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87733/ Test FAILed. ---

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20679 **[Test build #87733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87733/testReport)** for PR 20679 at commit

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #87736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87736/testReport)** for PR 20295 at commit

[GitHub] spark pull request #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallb...

2018-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20678#discussion_r171089207 --- Diff: python/pyspark/sql/tests.py --- @@ -3493,19 +3514,30 @@ def create_pandas_data_frame(self): data_dict["4_float_t"] =

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1118/

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20295 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20686 **[Test build #87734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87734/testReport)** for PR 20686 at commit

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87734/ Test PASSed. ---

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20686 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20666 **[Test build #87735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87735/testReport)** for PR 20666 at commit

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1117/

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20666 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20666: [SPARK-23448][SQL] Clarify JSON and CSV parser behavior ...

2018-02-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20666 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r171071499 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -266,15 +265,24 @@ private[tree] class LearningNode( var isLeaf: Boolean,

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r171071028 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -92,6 +92,7 @@ private[spark] object RandomForest extends Logging {

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r171070831 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -631,10 +634,99 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20686 **[Test build #87734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87734/testReport)** for PR 20686 at commit

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87731/ Test FAILed. ---

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20681 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20681: [SPARK-23518][SQL] Avoid metastore access when the users...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20681 **[Test build #87731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87731/testReport)** for PR 20681 at commit

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20679 **[Test build #87733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87733/testReport)** for PR 20679 at commit

[GitHub] spark issue #20679: [SPARK-23514] Use SessionState.newHadoopConf() to propag...

2018-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20679 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1116/

<    1   2   3   4   5   >