[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15609 Hm, does this cause a actual problem? I thought this is covered in type-widening. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15606 This bug only affects the complex type, (`ArrayType`, `StructType`, and `MapType`)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13891 **[Test build #67435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67435/consoleFull)** for PR 13891 at commit [`a6b5a16`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13891 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13891: [SPARK-6685][MLLIB]Use DSYRK to compute AtA in ALS

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67435/ Test PASSed. ---

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15607 **[Test build #67443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67443/consoleFull)** for PR 15607 at commit [`0b9fa4f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15607: [SPARK-16137][SPARKR] randomForest for R

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15607 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67443/ Test PASSed. ---

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15606 **[Test build #67437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67437/consoleFull)** for PR 15606 at commit [`ba880d8`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15606 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15606: [SPARK-18070][SQL] binary operator should not consider n...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15606 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67437/ Test PASSed. ---

[GitHub] spark issue #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread jaceklaskowski
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/15603 The build has just proved that my thinking was correct. I think I'll propose few other (more agressive) changes to clean the code up a little bit more. I'd appreciate any comments (or accepta

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread priyankagargnitk
Github user priyankagargnitk commented on the issue: https://github.com/apache/spark/pull/15609 actually, its not covered in type widening... So, if any of my expression is calling if expression ( in nested) .. its failing because of this issue. --- If your project is set up for it,

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15553 **[Test build #67436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67436/consoleFull)** for PR 15553 at commit [`ec49e8b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15553 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67436/ Test PASSed. ---

[GitHub] spark issue #15568: [SPARK-18028][SQL] simplify TableFileCatalog

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15568 **[Test build #67444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67444/consoleFull)** for PR 15568 at commit [`c4a906f`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15566 **[Test build #67439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67439/consoleFull)** for PR 15566 at commit [`d4fbeb2`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15588: [SPARK-18039][Scheduler] fix bug maxRegisteredWaitingTim...

2016-10-24 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/15588 I think this fix cannot really handle this imbalance receiver allocation problem, also blindly waste the CPU time. What @lw-lin mentioned is a feasible solution to wait for executors to be

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15566 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67439/ Test PASSed. ---

[GitHub] spark issue #15566: [SPARK-18026][SQL] should not always lowercase partition...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15566 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r84640346 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -93,15 +91,15 @@ private[spark] class SparkUI private ( /** Stop the server behin

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r84640931 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -135,7 +133,7 @@ private[spark] class SparkUI private ( private[spark] abstract cla

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15513 @HyukjinKwon @gatorsmile on the question just above, I'm neutral. I don't think either style is particularly better. All else equal, leave it as is, as we've got a ton of changes in flight here.

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15609 @priyankagargnitk Could I please ask some codes which I can reproduce this issue? It seems fine the sql below: ```sql spark-sql> SELECT if(true, cast('2014-01-01' as date), cast('201

[GitHub] spark issue #10162: [SPARK-11250] [SQL] Generate different alias for columns...

2016-10-24 Thread NarineK
Github user NarineK commented on the issue: https://github.com/apache/spark/pull/10162 I am trying different ways to solve the problem without renaming the columns and it seems that a better place to change the column names would be here: https://github.com/apache/spark/blob/master/s

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67445/consoleFull)** for PR 15377 at commit [`b57f009`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM...

2016-10-24 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/15598#discussion_r84643826 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1059,9 +1059,11 @@ private[spark] class Client( } catch {

[GitHub] spark pull request #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15377#discussion_r84643899 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2508,6 +2508,27 @@ private[spark] object Utils extends Logging { } }

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread priyankagargnitk
Github user priyankagargnitk commented on the issue: https://github.com/apache/spark/pull/15609 Actually our use case is little different, we are not invoking it with the Select queries.. We are using JS to let user type expressions and then we have created out own layer on top of spa

[GitHub] spark pull request #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM...

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15598#discussion_r84644113 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1059,9 +1059,11 @@ private[spark] class Client( } catch {

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15609 Then, wouldn't the responsibility of type coercion be your own layer? I guess this PR virtually mean every expression handled in `TypeCoercion` should be fixed too. For example, I guess

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15432 I assume this will proceed after https://github.com/apache/spark/pull/15513 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark issue #15605: [WIP] [SPARK-18067] [SQL] SortMergeJoin adds shuffle if ...

2016-10-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15605 @tejasapatil this didn't work? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #15432: [SPARK-17854][SQL] rand/randn allows null/long as input ...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15432 I definitely will. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes s

[GitHub] spark issue #15579: Added support for extra command in front of spark.

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15579 What's another use case? I can't think of one. We wouldn't do this with env variables anyway. (PS you need to fix up the title/description in any event) --- If your project is set up for it, you can

[GitHub] spark issue #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for truePos...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15585 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for ...

2016-10-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15585 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15450: [SPARK-3261] [MLLIB] KMeans clusterer can return duplica...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15450 @sethah let me know how you feel about it at this stage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84646984 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84646230 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84649236 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -120,4 +123,80 @@ class In

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648687 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648063 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648114 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84649225 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -120,4 +123,80 @@ class In

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84644855 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648544 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648531 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84649904 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2678,4 +2678,45 @@ class SQLQuerySuite extends QueryTest with Share

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648767 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -120,4 +123,80 @@ class In

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84649180 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -120,4 +123,80 @@ class In

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84643784 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648796 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -120,4 +123,80 @@ class In

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84649130 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -120,4 +123,80 @@ class In

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84646519 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84643941 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84646767 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648636 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15319: [SPARK-17733][SQL] InferFiltersFromConstraints ru...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15319#discussion_r84648323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -74,20 +74,98 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in...

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15441#discussion_r84650620 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala --- @@ -586,12 +592,28 @@ private[ui] class JobPagedTable( override def row(job

[GitHub] spark pull request #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in...

2016-10-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15441#discussion_r84650695 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -59,21 +59,25 @@ private[spark] class SparkUI private ( val killEnabled = sc.map(

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15609 @priyankagargnitk this should be covered in type widening. The `trueValue` and `falseValue` expressions should have the same type by the time we reach the execution phase. The `IfCoercion` rule s

[GitHub] spark pull request #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.d...

2016-10-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15382 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r84653856 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -93,15 +91,15 @@ private[spark] class SparkUI private ( /** Stop the serv

[GitHub] spark pull request #15603: [WEBUI][MINOR] Return types in methods + cleanup

2016-10-24 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/15603#discussion_r84653954 --- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala --- @@ -135,7 +133,7 @@ private[spark] class SparkUI private ( private[spark] abst

[GitHub] spark pull request #15597: [SPARK-18063][SQL] Failed to infer constraints ov...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15597#discussion_r84655429 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -293,15 +293,19 @@ abstract class UnaryNode

[GitHub] spark pull request #15597: [SPARK-18063][SQL] Failed to infer constraints ov...

2016-10-24 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15597#discussion_r84655214 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -293,15 +293,19 @@ abstract class UnaryNode

[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...

2016-10-24 Thread vijoshi
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @ajbozarth @tgravescs @andrewor14 - updated the implementation to always display "Last Updated Date-Time" when we have atleast one completed scan of the event log directory. I have changed the imp

[GitHub] spark issue #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.dir is r...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15382 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15436: [SPARK-17875] [BUILD] Remove unneeded direct dependence ...

2016-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15436 @zsxwing I may have to shelve this; I can't figure this out. It passes locally with Python2/3 but always times out with Python3 on Jenkins. No idea ... --- If your project is set up for it, you can

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67445/consoleFull)** for PR 15377 at commit [`b57f009`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67445/ Test FAILed. ---

[GitHub] spark issue #15568: [SPARK-18028][SQL] simplify TableFileCatalog

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15568 **[Test build #67444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67444/consoleFull)** for PR 15568 at commit [`c4a906f`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15568: [SPARK-18028][SQL] simplify TableFileCatalog

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67444/ Test PASSed. ---

[GitHub] spark issue #15568: [SPARK-18028][SQL] simplify TableFileCatalog

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15568 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15053 **[Test build #3371 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3371/consoleFull)** for PR 15053 at commit [`d384a95`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15413 **[Test build #67448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67448/consoleFull)** for PR 15413 at commit [`15deb72`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #15579: Added support for extra command in front of spark.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15579 **[Test build #3370 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3370/consoleFull)** for PR 15579 at commit [`a24aff9`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67447/consoleFull)** for PR 15377 at commit [`5acefad`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM Applic...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15598 **[Test build #67446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67446/consoleFull)** for PR 15598 at commit [`3ee6120`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15579: Added support for extra command in front of spark.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15579 **[Test build #3370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3370/consoleFull)** for PR 15579 at commit [`a24aff9`](https://github.com/apache/spark/commit

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15413 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15413 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67448/ Test PASSed. ---

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15413 **[Test build #67448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67448/consoleFull)** for PR 15413 at commit [`15deb72`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15053 **[Test build #3371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3371/consoleFull)** for PR 15053 at commit [`d384a95`](https://github.com/apache/spark/commit

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67447/consoleFull)** for PR 15377 at commit [`5acefad`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67447/ Test FAILed. ---

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15513 **[Test build #67449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67449/consoleFull)** for PR 15513 at commit [`15c02ca`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15053#discussion_r84680715 --- Diff: python/pyspark/sql/column.py --- @@ -320,10 +323,13 @@ def alias(self, *alias): def cast(self, dataType): """ Convert the

[GitHub] spark pull request #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15053#discussion_r84680654 --- Diff: python/pyspark/sql/column.py --- @@ -320,10 +323,13 @@ def alias(self, *alias): def cast(self, dataType): """ Convert the

[GitHub] spark pull request #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15053#discussion_r84681021 --- Diff: python/pyspark/sql/context.py --- @@ -553,34 +560,16 @@ def register(self, name, f, returnType=StringType()): def _test(): import

[GitHub] spark pull request #15053: [SPARK-18069][Doc] improve python API docstrings

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15053#discussion_r84684716 --- Diff: python/pyspark/sql/streaming.py --- @@ -612,9 +616,12 @@ def load(self, path=None, format=None, schema=None, **options): :param sc

[GitHub] spark issue #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM Applic...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15598 **[Test build #67446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67446/consoleFull)** for PR 15598 at commit [`3ee6120`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM Applic...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15598 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15598: [SPARK-18027][YARN] .sparkStaging not clean on RM Applic...

2016-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67446/ Test PASSed. ---

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15513 **[Test build #67450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67450/consoleFull)** for PR 15513 at commit [`035baef`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #15513: [SPARK-17963][SQL][Documentation] Add examples (extend) ...

2016-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15513 I think it is ready for a final look. I will take another look tomorrow though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

  1   2   3   4   5   6   >