[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14136 **[Test build #62088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62088/consoleFull)** for PR 14136 at commit [`1ae3df7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14136 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62088/ Test FAILed. ---

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-11 Thread phalodi
Github user phalodi commented on the issue: https://github.com/apache/spark/pull/14104 @srowen @rxin ![screenshot from 2016-07-11 16-51-53](https://cloud.githubusercontent.com/assets/8075390/16729090/e860ff50-4787-11e6-915e-7e39da6558e3.png) --- If your project is set up for it

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70240831 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to t

[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...

2016-07-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14104#discussion_r70240934 --- Diff: docs/programming-guide.md --- @@ -1099,6 +1099,9 @@ for details. +Spark RDD API also exposes asynchronous versions of some acti

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14136 **[Test build #62089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62089/consoleFull)** for PR 14136 at commit [`d5d4fa9`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #13494: [SPARK-15752] [SQL] Optimize metadata only query that ha...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13494 @lianhuiwang I left a few smallish comments. Overall LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14116#discussion_r70241185 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/systemcatalog/InformationSchema.scala --- @@ -0,0 +1,336 @@ +/* + * Licensed to t

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70241201 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/14012 @liancheng please find some time to review the latest updates, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14136 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62089/ Test FAILed. ---

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14136 **[Test build #62089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62089/consoleFull)** for PR 14136 at commit [`d5d4fa9`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70241490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14136 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-11 Thread wesolowskim
GitHub user wesolowskim opened a pull request: https://github.com/apache/spark/pull/14137 SPARK-16478 graphX (added graph caching in strongly connected components) ## What changes were proposed in this pull request? I added caching in every iteration for sccGraph that is ret

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-11 Thread phalodi
Github user phalodi commented on the issue: https://github.com/apache/spark/pull/14104 @srowen Thanks man i know i am not giving correct language but my intention is also same i hope its final commit :) if its done can you merge it. --- If your project is set up for it, you can reply

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70241660 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70241891 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14137 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70242134 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70242099 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark pull request #14104: [SPARK-16438] Add Asynchronous Actions documentat...

2016-07-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14104#discussion_r70242186 --- Diff: docs/programming-guide.md --- @@ -1099,6 +1099,9 @@ for details. +The Spark RDD API also exposes asynchronous versions of some

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70242502 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70242570 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14137 You have introduced a new problem though -- you're not unpersisting the RDDs you cache, and, you're doing a needless count (minor). Of course it's faster to operate on the final RDD at the end: you d

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70242786 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * Licensed

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70243206 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -536,6 +536,25 @@ object functions { def min(columnName: String): Colu

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70243163 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70243309 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70243463 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-11 Thread phalodi
Github user phalodi commented on the issue: https://github.com/apache/spark/pull/14104 @srowen I put back-tick-quote for foreach, foreachAsync and FutureAction. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70243630 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70243723 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...

2016-07-11 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/14116#discussion_r70243938 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/systemcatalog/InformationSchema.scala --- @@ -0,0 +1,336 @@ +/* + * Licensed to the

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70244013 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark pull request #14012: [SPARK-16343][SQL] Improve the PushDownPredicate ...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14012#discussion_r70244000 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1086,6 +1086,28 @@ object PruneFilters extends Rule[

[GitHub] spark issue #14129: [SPARK-16280][SQL][WIP] Implement histogram_numeric SQL ...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14129 **[Test build #3178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3178/consoleFull)** for PR 14129 at commit [`08065d8`](https://github.com/apache/spark/commit

[GitHub] spark issue #14082: [SPARK-16381][SQL][SparkR] Update SQL examples and progr...

2016-07-11 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14082 Since both @shivaram and @felixcheung signed this off, I'm merging this to master and branch-2.0. Thanks @keypointt for working on this and @shivaram and @felixcheung for the review! --

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70245059 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,148 @@ +/* + * License

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14082 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70245302 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -536,6 +536,25 @@ object functions { def min(columnName: String): Col

[GitHub] spark issue #14086: [SPARK-16463][SQL] Support `truncate` option in Overwrit...

2016-07-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14086 Shouldn't truncate function like overwrite in instances where the distinction doesn't matter -- rather than be ignored? I think the point of the issue was that DROP/CREATE _isn't_ the best s

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70245522 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -41,43 +35,47 @@ object HiveFromSpark { // in

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70245539 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -41,43 +35,47 @@ object HiveFromSpark { // in

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70245656 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SqlDataSourceExample.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache So

[GitHub] spark issue #14104: [SPARK-16438] Add Asynchronous Actions documentation

2016-07-11 Thread phalodi
Github user phalodi commented on the issue: https://github.com/apache/spark/pull/14104 @srowen Below is final screen shot as you suggested i hope its looks good please review it and merge it. ![screenshot from 2016-07-11 17-45-24](https://cloud.githubusercontent.com/assets/8075390

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14136 @jiangxb1987 Thanks for working on this. I did a quick pass, and it is a good start. I have a few issues: - I am a bit concerned about the memory characteristics. The worst case scenario

[GitHub] spark issue #14131: [SPARK-16318][SQL] Implement all remaining xpath functio...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14131 **[Test build #62085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62085/consoleFull)** for PR 14131 at commit [`4d6f654`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14131: [SPARK-16318][SQL] Implement all remaining xpath functio...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14131 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62085/ Test FAILed. ---

[GitHub] spark issue #14131: [SPARK-16318][SQL] Implement all remaining xpath functio...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14131 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14136 A more performant way of this would be to plan this using a combination of count grouped by the percentile key, this percentile function. I am not sure if we should pursue that for this PR. ---

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70247856 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the A

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70247911 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the A

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70248137 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the A

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70248199 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the A

[GitHub] spark issue #14115: [SPARK-16459][SQL] Prevent dropping current database

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14115 **[Test build #3177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3177/consoleFull)** for PR 14115 at commit [`19a3160`](https://github.com/apache/spark/commit

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70249015 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to

[GitHub] spark issue #13704: [SPARK-15985][SQL] Reduce runtime overhead of a program ...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13704 **[Test build #62084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62084/consoleFull)** for PR 13704 at commit [`c31729f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14133: [SPARK-15889] [STREAMING] Follow-up fix to erroneous con...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14133 **[Test build #62083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62083/consoleFull)** for PR 14133 at commit [`719012f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13704: [SPARK-15985][SQL] Reduce runtime overhead of a program ...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14133: [SPARK-15889] [STREAMING] Follow-up fix to erroneous con...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14133 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62083/ Test PASSed. ---

[GitHub] spark issue #13704: [SPARK-15985][SQL] Reduce runtime overhead of a program ...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62084/ Test PASSed. ---

[GitHub] spark issue #14133: [SPARK-15889] [STREAMING] Follow-up fix to erroneous con...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14133 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-11 Thread wesolowskim
Github user wesolowskim commented on the issue: https://github.com/apache/spark/pull/14137 Let me introduce some data first: 1. SCC run computed on randomly generated graph just like one provided by me on databrics notebook takes about 120s 2. When doing sccGraph.vertices.count

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #62090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62090/consoleFull)** for PR 13990 at commit [`cc96fea`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #13704: [SPARK-15985][SQL] Reduce runtime overhead of a program ...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13704 **[Test build #62087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62087/consoleFull)** for PR 13704 at commit [`1bbe859`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62090/ Test FAILed. ---

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #62090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62090/consoleFull)** for PR 13990 at commit [`cc96fea`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13990 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13704: [SPARK-15985][SQL] Reduce runtime overhead of a program ...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62087/ Test PASSed. ---

[GitHub] spark issue #13704: [SPARK-15985][SQL] Reduce runtime overhead of a program ...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14115: [SPARK-16459][SQL] Prevent dropping current database

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14115 **[Test build #62086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62086/consoleFull)** for PR 14115 at commit [`19a3160`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14115: [SPARK-16459][SQL] Prevent dropping current database

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14115 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14115: [SPARK-16459][SQL] Prevent dropping current database

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14115 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62086/ Test PASSed. ---

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70252043 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to t

[GitHub] spark issue #13494: [SPARK-15752] [SQL] Optimize metadata only query that ha...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on the issue: https://github.com/apache/spark/pull/13494 @hvanhovell I have addressed your comments. Thanks. If I missed something, please tell me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] Optimize metadata only query ...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r70253642 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala --- @@ -0,0 +1,143 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHE...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14116#discussion_r70253855 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/systemcatalog/InformationSchema.scala --- @@ -0,0 +1,336 @@ +/* + * Licensed to

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-07-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r70254020 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -536,6 +536,25 @@ object functions { def min(columnName: String): Colu

[GitHub] spark pull request #14132: [SPARK-16475][SQL][WIP] Broadcast Hint for SQL Qu...

2016-07-11 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/14132#discussion_r70254654 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -339,8 +339,24 @@ class AstBuilder extends SqlBaseBas

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #62091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62091/consoleFull)** for PR 13990 at commit [`0b4419f`](https://github.com/apache/spark/commit/0

[GitHub] spark pull request #14115: [SPARK-16459][SQL] Prevent dropping current datab...

2016-07-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14115 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14119 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14119 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70256340 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala --- @@ -41,43 +35,47 @@ object HiveFromSpark { // in

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #62093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62093/consoleFull)** for PR 14036 at commit [`a461e35`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14119 **[Test build #62092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62092/consoleFull)** for PR 14119 at commit [`95f0f41`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14119 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14119 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62092/ Test FAILed. ---

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14119 **[Test build #62092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62092/consoleFull)** for PR 14119 at commit [`95f0f41`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL programmi...

2016-07-11 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14119 Since you've added `JavaSparkSqlExample.scala`, we can remove `JavaSparkSQL.scala` now. (I guess that file was from my original WIP branch?) --- If your project is set up for it, you can reply to

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70256850 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSqlDataSourceExample.java --- @@ -0,0 +1,192 @@ +package org.apache.spark.examples.

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70257720 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/SparkSqlExample.scala --- @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #14135: [Spark-16479] Add Example for asynchronous action

2016-07-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14135 This doesn't actually show use of the API other than calling it. I honestly don't think this is something worth much more mention. --- If your project is set up for it, you can reply to this email a

[GitHub] spark issue #14088: [SPARK-16414] [YARN] Fix bugs for "Can not get user conf...

2016-07-11 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14088 ah, thanks @vanzin for the reminder. @sharkdtu could you add a small unit test for this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70261011 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSqlExample.java --- @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70262964 --- Diff: docs/sql-programming-guide.md --- @@ -732,62 +452,7 @@ a `Dataset` can be created programmatically with three steps. by `SparkSession`.

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70262914 --- Diff: docs/sql-programming-guide.md --- @@ -679,43 +435,7 @@ a `DataFrame` can be created programmatically with three steps. by `SparkSession`.

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70263511 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSqlExample.java --- @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark pull request #14119: [SPARK-16303][DOCS][EXAMPLES][WIP] Updated SQL pr...

2016-07-11 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14119#discussion_r70263602 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSqlExample.java --- @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Softw

<    1   2   3   4   5   6   7   >