[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70660445 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -207,20 +207,12 @@ case class Multiply(left: Exp

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70659855 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -957,7 +957,7 @@ class AstBuilder extends SqlBaseBase

[GitHub] spark issue #14106: [SPARK-16448] RemoveAliasOnlyProject should not remove a...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14106 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14106: [SPARK-16448] RemoveAliasOnlyProject should not remove a...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62248/ Test PASSed. ---

[GitHub] spark issue #14106: [SPARK-16448] RemoveAliasOnlyProject should not remove a...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14106 **[Test build #62248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62248/consoleFull)** for PR 14106 at commit [`d337e8e`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14012: [SPARK-16343][SQL] Improve the PushDownPredicate ...

2016-07-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14012 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-13 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14012 Thanks! Merged this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark issue #14140: [SPARK-16426][MLlib] Fix bug that caused NaNs in Isotoni...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14140 **[Test build #62252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62252/consoleFull)** for PR 14140 at commit [`599a868`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14140: [SPARK-16426][MLlib] Fix bug that caused NaNs in Isotoni...

2016-07-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14140 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14140: [SPARK-16426][MLlib] Fix bug that caused NaNs in Isotoni...

2016-07-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14140 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70656424 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -957,7 +957,7 @@ class AstBuilder extends SqlBaseBase

[GitHub] spark issue #14092: [SPARK-16419][SQL] EnsureRequirements adds extra Sort to...

2016-07-13 Thread MasterDDT
Github user MasterDDT commented on the issue: https://github.com/apache/spark/pull/14092 cc @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14148: [SPARK-16482] [SQL] Describe Table Command for Tables Re...

2016-07-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14148 @rxin @cloud-fan @yhuai Will do more investigation and submit a separate PR for solution review. Thanks! --- If your project is set up for it, you can reply to this email and have your reply app

[GitHub] spark issue #12983: [SPARK-15213][PySpark] Unify 'range' usages

2016-07-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/12983 Yes, @zhengruifeng what's the difference here? could you update or close? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark issue #14181: [SPARK-15382][SQL] Fix a rule to push down projects bene...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14181 **[Test build #62251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62251/consoleFull)** for PR 14181 at commit [`9a5f975`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #14181: [SPARK-15382][SQL] Fix a rule to push down projec...

2016-07-13 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/14181 [SPARK-15382][SQL] Fix a rule to push down projects beneath Sample ## What changes were proposed in this pull request? When `X` > 1.0 in `Dataset#sample`, `sample(true, X).withColumn("x", monot

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #62250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62250/consoleFull)** for PR 13990 at commit [`cbc8798`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70642695 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +108,55 @@ class RandomForestRegressorSuite

[GitHub] spark issue #14159: [SQL][PARQUET] Fix for Vectorized Parquet filter pushdow...

2016-07-13 Thread andreweduffy
Github user andreweduffy commented on the issue: https://github.com/apache/spark/pull/14159 Yep, looks like the other one was closed by the committer. I saw Sean commented that this might need to be tested against 2.2, is that going to be necessary? --- If your project is set up for

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14071 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62249/ Test FAILed. ---

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14071 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14071 **[Test build #62249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62249/consoleFull)** for PR 14071 at commit [`d18ac59`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14045: [SPARK-16362][SQL] Support ArrayType and StructType in v...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #62244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62244/consoleFull)** for PR 14045 at commit [`17f3b82`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14045: [SPARK-16362][SQL] Support ArrayType and StructType in v...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14045 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62244/ Test PASSed. ---

[GitHub] spark issue #14045: [SPARK-16362][SQL] Support ArrayType and StructType in v...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14045 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-13 Thread wesolowskim
Github user wesolowskim commented on a diff in the pull request: https://github.com/apache/spark/pull/14137#discussion_r70643061 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala --- @@ -44,6 +44,11 @@ object StronglyConnectedComponents

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-07-13 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/13650 This PR looks good overall, I left some inline comments. Thanks for working on this. @MechCoder --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14137 Is it now not necessary to cache the 'work graph' as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70640862 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/RandomForestRegressor.scala --- @@ -168,15 +173,37 @@ class RandomForestRegressionModel pri

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14137 I was thinking it should be perfectly possible to unpersist scWorkGraph RDDs that are persisted along the way in the same way? it should be the same pattern. It's possible to leave the final result p

[GitHub] spark issue #13650: [SPARK-9623] [ML] Provide variance for RandomForestRegre...

2016-07-13 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/13650 @MechCoder It seems there is a typo in the PR's description: ``` The variance is calculated by summing up the mean of the variance of each decision tree + mean of the prediction^2 of each

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-13 Thread wesolowskim
Github user wesolowskim commented on the issue: https://github.com/apache/spark/pull/14137 I don't know if I understand you correctly. Last work RDD is explicitly materialized because iteration's depend on number of vertices left in graph. My aim with latest solution was to leave

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #62243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62243/consoleFull)** for PR 14116 at commit [`11031e4`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70639783 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -234,6 +234,7 @@ object FunctionRegistry {

[GitHub] spark pull request #13670: [SPARK-15951] Change Executors Page to use datata...

2016-07-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/13670#discussion_r70636564 --- Diff: core/src/main/resources/org/apache/spark/ui/static/executorspage.js --- @@ -0,0 +1,410 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62243/ Test PASSed. ---

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14137 Ack right I mean does it need to be explicitly materialized? I suppose the new code also has the effect of materializing the last work RDD, but then I imagine we need to unpersist it too. --- If yo

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70639350 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-13 Thread wesolowskim
Github user wesolowskim commented on the issue: https://github.com/apache/spark/pull/14137 'work graph' is already cached and it is indeed necessary. To summarize - do you propose to replace sccGraph.triplets.take(1) with sccGraph.vertices.count() and sccGraph.edges.count()?

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-13 Thread wesolowskim
Github user wesolowskim commented on a diff in the pull request: https://github.com/apache/spark/pull/14137#discussion_r70637549 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala --- @@ -64,11 +69,20 @@ object StronglyConnectedComponents

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14137#discussion_r70638869 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala --- @@ -44,6 +44,11 @@ object StronglyConnectedComponents {

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70638775 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70638644 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-13 Thread wesolowskim
Github user wesolowskim commented on a diff in the pull request: https://github.com/apache/spark/pull/14137#discussion_r70638560 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala --- @@ -44,6 +44,11 @@ object StronglyConnectedComponents

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70637087 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70637181 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70638276 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70637680 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #13650: [SPARK-9623] [ML] Provide variance for RandomFore...

2016-07-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/13650#discussion_r70637268 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/RandomForestRegressorSuite.scala --- @@ -105,6 +109,54 @@ class RandomForestRegressorSuite

[GitHub] spark pull request #13670: [SPARK-15951] Change Executors Page to use datata...

2016-07-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/13670#discussion_r70636570 --- Diff: core/src/main/resources/org/apache/spark/ui/static/executorspage.js --- @@ -0,0 +1,410 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14137#discussion_r70636044 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala --- @@ -44,6 +44,11 @@ object StronglyConnectedComponents {

[GitHub] spark pull request #14137: SPARK-16478 graphX (added graph caching in strong...

2016-07-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14137#discussion_r70635683 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/lib/StronglyConnectedComponents.scala --- @@ -64,11 +69,20 @@ object StronglyConnectedComponents {

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14071 **[Test build #62249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62249/consoleFull)** for PR 14071 at commit [`d18ac59`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14071 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14071 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62247/ Test FAILed. ---

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14071 **[Test build #62247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62247/consoleFull)** for PR 14071 at commit [`632d9a7`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14071: [SPARK-16397][SQL] make CatalogTable more general...

2016-07-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14071#discussion_r70634823 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -42,38 +42,54 @@ case class CatalogFunction(

[GitHub] spark issue #14106: [SPARK-16448] RemoveAliasOnlyProject should not remove a...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14106 **[Test build #62248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62248/consoleFull)** for PR 14106 at commit [`d337e8e`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14149: [SPARK-16435][YARN][MINOR] Add warning log if initialExe...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14149 **[Test build #62246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62246/consoleFull)** for PR 14149 at commit [`6dd2a86`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14071: [SPARK-16397][SQL] make CatalogTable more general and le...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14071 **[Test build #62247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62247/consoleFull)** for PR 14071 at commit [`632d9a7`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62242/ Test PASSed. ---

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14137: SPARK-16478 graphX (added graph caching in strongly conn...

2016-07-13 Thread wesolowskim
Github user wesolowskim commented on the issue: https://github.com/apache/spark/pull/14137 I added unpersists to additionally created caches and checked performance. scc.run is slightly longer, but returned graph is cached (both vertices and edges). To fully optimize it and remov

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14036 last 2 comments. @liancheng @clockfly can you also take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #62242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62242/consoleFull)** for PR 14116 at commit [`66fab70`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70631504 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -285,6 +278,26 @@ case class Divide(left: Expressi

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #62245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62245/consoleFull)** for PR 14079 at commit [`338db65`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70631462 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -285,6 +278,26 @@ case class Divide(left: Expressi

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-13 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r70628892 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -234,6 +234,7 @@ object FunctionRegistry {

[GitHub] spark pull request #14149: [SPARK-16435][YARN][MINOR] Add warning log if ini...

2016-07-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/14149#discussion_r70628729 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2342,6 +2342,18 @@ private[spark] object Utils extends Logging { * Return th

[GitHub] spark pull request #14149: [SPARK-16435][YARN][MINOR] Add warning log if ini...

2016-07-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14149#discussion_r70628081 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2342,6 +2342,18 @@ private[spark] object Utils extends Logging { * Return th

[GitHub] spark pull request #14068: [SPARK-16469] enhanced simulate multiply

2016-07-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14068 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14180: Wheelhouse and VirtualEnv support

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14180 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #14149: [SPARK-16435][YARN][MINOR] Add warning log if ini...

2016-07-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/14149#discussion_r70625909 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2342,6 +2342,18 @@ private[spark] object Utils extends Logging { * Return th

[GitHub] spark pull request #14180: Wheelhouse and VirtualEnv support

2016-07-13 Thread Stibbons
GitHub user Stibbons opened a pull request: https://github.com/apache/spark/pull/14180 Wheelhouse and VirtualEnv support ## What changes were proposed in this pull request? Support virtualenv and wheel in PySpark, based on SPARK-13587. Full description in [SPARK-16367]

[GitHub] spark pull request #14149: [SPARK-16435][YARN][MINOR] Add warning log if ini...

2016-07-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14149#discussion_r70625165 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2342,6 +2342,18 @@ private[spark] object Utils extends Logging { * Return th

[GitHub] spark pull request #14149: [SPARK-16435][YARN][MINOR] Add warning log if ini...

2016-07-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/14149#discussion_r70625202 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2342,6 +2342,18 @@ private[spark] object Utils extends Logging { * Return th

[GitHub] spark issue #14045: [SPARK-16362][SQL] Support ArrayType and StructType in v...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #62244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62244/consoleFull)** for PR 14045 at commit [`17f3b82`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2016-07-13 Thread ScrapCodes
Github user ScrapCodes commented on the issue: https://github.com/apache/spark/pull/14151 I have a question, should we keep a column with filenames ? in current approach we ignore key column. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #13704: [SPARK-15985][SQL] Eliminate redundant cast from an arra...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13704: [SPARK-15985][SQL] Eliminate redundant cast from an arra...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62239/ Test PASSed. ---

[GitHub] spark issue #13704: [SPARK-15985][SQL] Eliminate redundant cast from an arra...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13704 **[Test build #62239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62239/consoleFull)** for PR 13704 at commit [`355f5a5`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14169: [WIP][SPARK-16515][SQL]set default record reader and wri...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14169 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14169: [WIP][SPARK-16515][SQL]set default record reader and wri...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14169 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62237/ Test PASSed. ---

[GitHub] spark issue #14169: [WIP][SPARK-16515][SQL]set default record reader and wri...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14169 **[Test build #62237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62237/consoleFull)** for PR 14169 at commit [`e4c7e02`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #62243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62243/consoleFull)** for PR 14116 at commit [`11031e4`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14165: [SPARK-16503] SparkSession should provide Spark version

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62236/ Test PASSed. ---

[GitHub] spark issue #14165: [SPARK-16503] SparkSession should provide Spark version

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14165 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14165: [SPARK-16503] SparkSession should provide Spark version

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14165 **[Test build #62236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62236/consoleFull)** for PR 14165 at commit [`b4372f7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #62242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62242/consoleFull)** for PR 14116 at commit [`66fab70`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14116 Jenkins becomes unstable more and more: Fail -> MiMa -> RAT (with same commit.) I rebased and update the commit id to see the result. --- If your project is set up for it, you can reply t

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62233/ Test PASSed. ---

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62241/ Test FAILed. ---

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #62241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62241/consoleFull)** for PR 14116 at commit [`eb302c4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2016-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #62233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62233/consoleFull)** for PR 14151 at commit [`6e83f46`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #62241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62241/consoleFull)** for PR 14116 at commit [`eb302c4`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14116: [SPARK-16452][SQL][WIP] Support basic INFORMATION_SCHEMA

2016-07-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14116 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

<    1   2   3   4   5   6   7   >