[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14104 **[Test build #61979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61979/consoleFull)** for PR 14104 at commit

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14104 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61980/ Test PASSed. ---

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61979/ Test PASSed. ---

[GitHub] spark issue #14105: [SPARK-16376] [WEBUI] [Spark web UI]:HTTP ERROR 500 when...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14105 **[Test build #61981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61981/consoleFull)** for PR 14105 at commit

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70074963 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70075370 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14088 **[Test build #61982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61982/consoleFull)** for PR 14088 at commit

[GitHub] spark pull request #13831: [SPARK-16119][sql] Support PURGE option to drop t...

2016-07-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13831#discussion_r70079878 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -179,7 +179,8 @@ case class DescribeDatabaseCommand( case

[GitHub] spark issue #13831: [SPARK-16119][sql] Support PURGE option to drop table / ...

2016-07-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/13831 it would be nice to test using purge actually does a purge, ie nothing is in the trash afterwards. I don't see any easy way to do that, though -- maybe using

[GitHub] spark issue #1306: [SPARK-2373]RDD add span function (split an RDD to two RD...

2016-07-08 Thread k0ala
Github user k0ala commented on the issue: https://github.com/apache/spark/pull/1306 FWIW, I frequently have this use case and would love a `span` on RDD. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix should aw...

2016-07-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14049 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #14065: [SPARK-14743][YARN][WIP] Add a configurable token...

2016-07-08 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/14065#discussion_r70074645 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/token/ServiceTokenProvider.scala --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70075273 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14088 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61982/ Test PASSed. ---

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14088 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14008 **[Test build #61983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61983/consoleFull)** for PR 14008 at commit

[GitHub] spark pull request #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix sh...

2016-07-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14049 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get us...

2016-07-08 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14088#discussion_r70073453 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -743,6 +735,14 @@ object ApplicationMaster extends Logging {

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-08 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14088 minor comment change otherwise looks good. thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14008 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13969 @petermaxlee are you sure that we shouldn't implement this using `RuntimeReplaceable`, using NewInstance -> Invoke or a StaticInvoke? The added value is that this support code generation, and

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13680 yea LGTM. And please make sure the benchmark results are up to date, as the code has changed a lot of times. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #14105: [SPARK-16376] [WEBUI] [Spark web UI]:HTTP ERROR 5...

2016-07-08 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/14105 [SPARK-16376] [WEBUI] [Spark web UI]:HTTP ERROR 500 when using rest api "/applications/[app-id]/jobs" if array "stageIds" is empty ## What changes were proposed in this pull request? Avoid

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread janplus
Github user janplus commented on the issue: https://github.com/apache/spark/pull/14008 cc @cloud-fan Thank you. I have resolved conflicts with master and done some code style fixes as you suggested. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get us...

2016-07-08 Thread sharkdtu
Github user sharkdtu commented on a diff in the pull request: https://github.com/apache/spark/pull/14088#discussion_r70076189 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -743,6 +735,14 @@ object ApplicationMaster extends Logging {

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14088 **[Test build #61982 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61982/consoleFull)** for PR 14088 at commit

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61976/ Test PASSed. ---

[GitHub] spark pull request #14068: enhanced simulate multiply

2016-07-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14068#discussion_r70059298 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala --- @@ -426,16 +426,23 @@ class BlockMatrix @Since("1.3.0") (

[GitHub] spark issue #14097: [MINOR][Streaming][Docs] Minor changes on kinesis integr...

2016-07-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14097 Seems reasonable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #11748: [SPARK-13921] Store serialized blocks as multiple chunks...

2016-07-08 Thread bonitao
Github user bonitao commented on the issue: https://github.com/apache/spark/pull/11748 Amazing. Thanks a lot @JoshRosen . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14087 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14087 **[Test build #61976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61976/consoleFull)** for PR 14087 at commit

[GitHub] spark pull request #13831: [SPARK-16119][sql] Support PURGE option to drop t...

2016-07-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13831#discussion_r70065994 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -249,7 +249,18 @@ class VersionsSuite extends SparkFunSuite

[GitHub] spark pull request #13831: [SPARK-16119][sql] Support PURGE option to drop t...

2016-07-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13831#discussion_r70066014 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/VersionsSuite.scala --- @@ -366,7 +377,20 @@ class VersionsSuite extends SparkFunSuite

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13680 @cloud-fan , thank you for a lot of great review comments. This PR looks much better than the first commit. I updated the description of this PR. Is it fine with you? --- If your project is set

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14104 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14104 @phalodi it's a bot, not a person. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14104 **[Test build #61980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61980/consoleFull)** for PR 14104 at commit

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14104 **[Test build #61979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61979/consoleFull)** for PR 14104 at commit

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14104 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @cloud-fan I would appreciate it if you could review this again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries should not be executed ...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14048 **[Test build #61978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61978/consoleFull)** for PR 14048 at commit

[GitHub] spark pull request #14012: [SPARK-16343][SQL] Improve the PushDownPredicate ...

2016-07-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14012#discussion_r70055094 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1135,11 +1146,16 @@ object PushDownPredicate

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries should not be executed ...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61978/ Test FAILed. ---

[GitHub] spark issue #13374: [SPARK-13638][SQL] Add escapeAll option to CSV DataFrame...

2016-07-08 Thread jurriaan
Github user jurriaan commented on the issue: https://github.com/apache/spark/pull/13374 @rxin Sorry for the confusion, fixed it :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13680 **[Test build #61977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61977/consoleFull)** for PR 13680 at commit

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14104 **[Test build #61980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61980/consoleFull)** for PR 14104 at commit

[GitHub] spark issue #14068: enhanced simulate multiply

2016-07-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14068 It does make sense. Please make a JIRA and connect this though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13680 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61977/ Test PASSed. ---

[GitHub] spark pull request #13831: [SPARK-16119][sql] Support PURGE option to drop t...

2016-07-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13831#discussion_r70062869 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLCommandSuite.scala --- @@ -776,21 +778,29 @@ class DDLCommandSuite extends

[GitHub] spark issue #13831: [SPARK-16119][sql] Support PURGE option to drop table / ...

2016-07-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/13831 you could avoid changing all the callsites by adding a default of `purge = false`, but I'm guessing that is discouraged in the sql code (since eg. there is not default for `ignoreIfNotExists`).

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14008 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14008 **[Test build #61983 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61983/consoleFull)** for PR 14008 at commit

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14008 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61983/ Test FAILed. ---

[GitHub] spark pull request #14012: [SPARK-16343][SQL] Improve the PushDownPredicate ...

2016-07-08 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14012#discussion_r70097418 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1135,11 +1146,16 @@ object PushDownPredicate

[GitHub] spark pull request #14075: [SPARK-16401] [SQL] Data Source API: Enable Exten...

2016-07-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14075#discussion_r70095195 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -82,6 +82,34 @@ class DefaultSource } }

[GitHub] spark pull request #14071: [SPARK-16397][SQL] make CatalogTable more general...

2016-07-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14071#discussion_r70096412 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -45,35 +45,28 @@ case class CatalogFunction( */

[GitHub] spark issue #14105: [SPARK-16376] [WEBUI] [Spark web UI]:HTTP ERROR 500 when...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14105 **[Test build #61981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61981/consoleFull)** for PR 14105 at commit

[GitHub] spark issue #14075: [SPARK-16401] [SQL] Data Source API: Enable Extending Re...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14075 **[Test build #61984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61984/consoleFull)** for PR 14075 at commit

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14012 **[Test build #61985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61985/consoleFull)** for PR 14012 at commit

[GitHub] spark issue #14105: [SPARK-16376] [WEBUI] [Spark web UI]:HTTP ERROR 500 when...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14105 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14105: [SPARK-16376] [WEBUI] [Spark web UI]:HTTP ERROR 500 when...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14105 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61981/ Test PASSed. ---

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread janplus
Github user janplus commented on the issue: https://github.com/apache/spark/pull/14008 It seems failed the `org.apache.spark.sql.sources.CreateTableAsSelectSuite.create a table, drop it and create another one with the same name` test which is irrelevant with this PR. Maybe we

[GitHub] spark issue #13374: [SPARK-13638][SQL] Add quoteAll option to CSV DataFrameW...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13374 **[Test build #3171 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3171/consoleFull)** for PR 13374 at commit

[GitHub] spark issue #14106: [SPARK-16448] RemoveAliasOnlyProject should not remove a...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14106 **[Test build #61986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61986/consoleFull)** for PR 14106 at commit

[GitHub] spark issue #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13969 If you mean the following mention, what @rxin said was not about the return type. It's about `function` and `method` name, isn't it? Is there any other mentions about `String`? > Majority

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries should not be executed ...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14048 **[Test build #61989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61989/consoleFull)** for PR 14048 at commit

[GitHub] spark issue #14075: [SPARK-16401] [SQL] Data Source API: Enable Extending Re...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14075 **[Test build #61990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61990/consoleFull)** for PR 14075 at commit

[GitHub] spark pull request #14080: [SPARK-16405] Add metrics and source for external...

2016-07-08 Thread lovexi
Github user lovexi commented on a diff in the pull request: https://github.com/apache/spark/pull/14080#discussion_r70109683 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -143,4 +179,26 @@ private void

[GitHub] spark issue #14075: [SPARK-16401] [SQL] Data Source API: Enable Extending Re...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14075 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14075: [SPARK-16401] [SQL] Data Source API: Enable Extending Re...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14075 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61984/ Test PASSed. ---

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14008 **[Test build #61987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61987/consoleFull)** for PR 14008 at commit

[GitHub] spark pull request #14093: SPARK-16420: Ensure compression streams are close...

2016-07-08 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/14093#discussion_r70105908 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java --- @@ -349,12 +349,19 @@ void forceSorterToSpill() throws IOException {

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70106457 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries should not be executed ...

2016-07-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14048 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14093: SPARK-16420: Ensure compression streams are closed.

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14093 **[Test build #61988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61988/consoleFull)** for PR 14093 at commit

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14088 The comment can be easily replaced by a unit test; then the PR builder will break is someone ignores the comment... --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #14075: [SPARK-16401] [SQL] Data Source API: Enable Exten...

2016-07-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14075#discussion_r70108301 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -82,6 +82,32 @@ class DefaultSource } }

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r70109475 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,21 @@ trait CheckAnalysis extends

[GitHub] spark issue #14075: [SPARK-16401] [SQL] Data Source API: Enable Extending Re...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14075 **[Test build #61984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61984/consoleFull)** for PR 14075 at commit

[GitHub] spark issue #13831: [SPARK-16119][sql] Support PURGE option to drop table / ...

2016-07-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/13831 minor comments andrequest for some extra tests, but overall looks good --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/13969 @hvanhovell in its current form we'd need some refactoring to StaticInvoke to work with this, due to Hive allowing both static invocation and dynamic invocation. Also - does

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14008 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14075: [SPARK-16401] [SQL] Data Source API: Enable Exten...

2016-07-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14075#discussion_r70106028 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -82,6 +82,32 @@ class DefaultSource } }

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70106044 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70106490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark pull request #14106: [SPARK-16448] RemoveAliasOnlyProject should not r...

2016-07-08 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/14106 [SPARK-16448] RemoveAliasOnlyProject should not remove alias with metadata ## What changes were proposed in this pull request? `Alias` with metadata is not a no-op and we should not

[GitHub] spark issue #14106: [SPARK-16448] RemoveAliasOnlyProject should not remove a...

2016-07-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14106 cc @viirya @yhuai @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13969: [SPARK-16284][SQL] Implement reflect SQL function

2016-07-08 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/13969#discussion_r70106095 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Reflect.scala --- @@ -0,0 +1,170 @@ +/* + * Licensed to the

[GitHub] spark issue #14078: [SPARK-11857] [Mesos] [WIP] Deprecate fine grained

2016-07-08 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/14078 I'll get back to you before end of day --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14080: [SPARK-16405] Add metrics and source for external...

2016-07-08 Thread lovexi
Github user lovexi commented on a diff in the pull request: https://github.com/apache/spark/pull/14080#discussion_r70110638 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -93,18 +113,34 @@ protected void

[GitHub] spark pull request #14093: SPARK-16420: Ensure compression streams are close...

2016-07-08 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/14093#discussion_r70111252 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java --- @@ -349,12 +349,19 @@ void forceSorterToSpill() throws IOException

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14012 **[Test build #61985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61985/consoleFull)** for PR 14012 at commit

[GitHub] spark pull request #14080: [SPARK-16405] Add metrics and source for external...

2016-07-08 Thread lovexi
Github user lovexi commented on a diff in the pull request: https://github.com/apache/spark/pull/14080#discussion_r70111868 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -64,6 +75,10 @@ public

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61985/ Test PASSed. ---

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14012 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13831: [SPARK-16119][sql] Support PURGE option to drop table / ...

2016-07-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13831 > it would be nice to test using purge actually does a purge None of the tests run on actual HDFS cluster, so I doubt that would be possible without a lot more work... --- If your project

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r70118783 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,21 @@ trait CheckAnalysis extends

  1   2   3   4   5   >