[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17166 The task killed messages should be informative, and I don't think we should sacrifice informative messages just so they can be shown concisely in the stage summary view. I think it's much

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/17166 Drilling down into the detail view is kind of cumbersome -- I think it's most useful to have a good summary at the progress bar, and then the user can refer to logs for detailed per-task debugging.

[GitHub] spark pull request #15363: [SPARK-17791][SQL] Join reordering using star sch...

2017-03-14 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/15363#discussion_r106088842 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -51,6 +51,11 @@ case class

[GitHub] spark pull request #17299: [SPARK-19817][SS] Make it clear that `timeZone` i...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17299 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17299 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17186: [SPARK-19846][SQL] Add a flag to disable constraint prop...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17186 **[Test build #74581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74581/testReport)** for PR 17186 at commit

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17166 Why not just just "X killed" in the stage summary? It seems like overkill to put the reasons for all of the killings there, now that I'm seeing the screenshot, since they're already in the

[GitHub] spark pull request #15363: [SPARK-17791][SQL] Join reordering using star sch...

2017-03-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15363#discussion_r106086454 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -51,6 +51,11 @@ case class

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #74580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74580/testReport)** for PR 15945 at commit

[GitHub] spark pull request #15363: [SPARK-17791][SQL] Join reordering using star sch...

2017-03-14 Thread ron8hu
Github user ron8hu commented on a diff in the pull request: https://github.com/apache/spark/pull/15363#discussion_r106084556 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -51,6 +51,11 @@ case class

[GitHub] spark pull request #17274: [SPARK-19925][SPARKR] Fix SparkR spark.getSparkFi...

2017-03-14 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/17274#discussion_r106083228 --- Diff: R/pkg/inst/tests/testthat/test_context.R --- @@ -177,6 +177,13 @@ test_that("add and get file to be downloaded with Spark job on every node",

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74578/ Test PASSed. ---

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17299 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17299 **[Test build #74578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74578/testReport)** for PR 17299 at commit

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74576/ Test PASSed. ---

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74576/testReport)** for PR 17166 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #74579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74579/testReport)** for PR 15009 at commit

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74577/ Test PASSed. ---

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #74577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74577/testReport)** for PR 15945 at commit

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15604 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74575/ Test PASSed. ---

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15604 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15604 **[Test build #74575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74575/testReport)** for PR 15604 at commit

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74572/ Test PASSed. ---

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74572/testReport)** for PR 17166 at commit

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2017-03-14 Thread kishorvpatil
Github user kishorvpatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r106080005 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -252,20 +307,55 @@ class YarnClusterSuite

[GitHub] spark issue #17267: [SPARK-19926][PYSPARK] Make pyspark exception more reada...

2017-03-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17267 LGTM cc @davies @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17299 LGTM, thank you for your follow-up! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17297 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17297 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74566/ Test FAILed. ---

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17297 **[Test build #74566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74566/testReport)** for PR 17297 at commit

[GitHub] spark issue #17300: [SPARK-19956][Core]Optimize a location order of blocks w...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17300 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17300: [SPARK-19956][Core]Optimize a location order of b...

2017-03-14 Thread ConeyLiu
GitHub user ConeyLiu opened a pull request: https://github.com/apache/spark/pull/17300 [SPARK-19956][Core]Optimize a location order of blocks with topology information ## What changes were proposed in this pull request? When call the method getLocations of BlockManager, we

[GitHub] spark issue #17267: [SPARK-19926][PYSPARK] Make pyspark exception more reada...

2017-03-14 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17267 ping @viirya --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74570/ Test PASSed. ---

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74570/testReport)** for PR 17166 at commit

[GitHub] spark pull request #17232: [SPARK-18112] [SQL] Support reading data from Hiv...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17232 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17232: [SPARK-18112] [SQL] Support reading data from Hive 2.1 m...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17232 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17178: [SPARK-19828][R] Support array type in from_json ...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17178 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17178: [SPARK-19828][R] Support array type in from_json in R

2017-03-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17178 thanks! merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17268: [SPARK-19932][SS] Also save event time into StateStore f...

2017-03-14 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/17268 Thank you @marmbrus for the detailed explanation! > For that reason, I think its safest to require the user to explicitly include the timestamp. Yea, let me update this in this

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-03-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15125 @jkbradley @ankurdave would you like to review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17297 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17297 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74562/ Test FAILed. ---

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17297 **[Test build #74562 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74562/testReport)** for PR 17297 at commit

[GitHub] spark issue #17251: [SPARK-19910][SQL] `stack` should not reject NULL values...

2017-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17251 @cloud-fan . Sorry, but could you review this `stack` PR once again? :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15363: [SPARK-17791][SQL] Join reordering using star sch...

2017-03-14 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/15363#discussion_r106074890 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -51,6 +51,11 @@ case class

[GitHub] spark issue #17266: [SPARK-19912][SQL] String literals should be escaped for...

2017-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17266 For non-error message cases, incorrect result is also a problem in this issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #17241: [SPARK-19877][SQL] Restrict the nested level of a view

2017-03-14 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/17241 ping @hvanhovell @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17232: [SPARK-18112] [SQL] Support reading data from Hive 2.1 m...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17232 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74574/ Test PASSed. ---

[GitHub] spark issue #17266: [SPARK-19912][SQL] String literals should be escaped for...

2017-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17266 The following is the error message. Since we are not escaping in the spark master, the behavior (incorrect filtering or the error message) is the same from the master branch Spark. ```

[GitHub] spark issue #17232: [SPARK-18112] [SQL] Support reading data from Hive 2.1 m...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17232 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17232: [SPARK-18112] [SQL] Support reading data from Hive 2.1 m...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17232 **[Test build #74574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74574/testReport)** for PR 17232 at commit

[GitHub] spark pull request #17255: [SPARK-19918][SQL] Use TextFileFormat in implemen...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17255 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17277: [SPARK-19887][SQL] dynamic partition keys can be ...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17277 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17255: [SPARK-19918][SQL] Use TextFileFormat in implementation ...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17255 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17255: [SPARK-19918][SQL] Use TextFileFormat in implemen...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17255#discussion_r106072933 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonInferSchema.scala --- @@ -40,18 +40,11 @@ private[sql] object

[GitHub] spark issue #17232: [SPARK-18112] [SQL] Support reading data from Hive 2.1 m...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17232 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17299 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17299 **[Test build #74578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74578/testReport)** for PR 17299 at commit

[GitHub] spark issue #17299: [SPARK-19817][SS] Make it clear that `timeZone` is a gen...

2017-03-14 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/17299 This is the fix for the streaming counter-part (i.e. Structured Streaming), @ueshin @gatorsmile would you take a look? Thanks! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #17299: [SPARK-19817][SS] Make it clear that `timeZone` i...

2017-03-14 Thread lw-lin
GitHub user lw-lin opened a pull request: https://github.com/apache/spark/pull/17299 [SPARK-19817][SS] Make it clear that `timeZone` is a general option in DataStreamReader/Writer ## What changes were proposed in this pull request? As timezone setting can also affect

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74568/ Test FAILed. ---

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17298 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17164: [SPARK-16844][SQL] Support codegen for sort-based aggrea...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #74568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74568/testReport)** for PR 17298 at commit

[GitHub] spark issue #17164: [SPARK-16844][SQL] Support codegen for sort-based aggrea...

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74569/ Test PASSed. ---

[GitHub] spark issue #17164: [SPARK-16844][SQL] Support codegen for sort-based aggrea...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #74569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74569/testReport)** for PR 17164 at commit

[GitHub] spark issue #17237: [SPARK-19852][PYSPARK][ML] Update Python API setHandleIn...

2017-03-14 Thread VinceShieh
Github user VinceShieh commented on the issue: https://github.com/apache/spark/pull/17237 Sure. No problem! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17223: [SPARK-19881][SQL] Support Dynamic Partition Inse...

2017-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/17223 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17223: [SPARK-19881][SQL] Support Dynamic Partition Inserts par...

2017-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17223 I'll close this PR and JIRA issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17223: [SPARK-19881][SQL] Support Dynamic Partition Inserts par...

2017-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17223 I see. That's the reason why not to support that. Thank you, @cloud-fan. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17266: [SPARK-19912][SQL] String literals should be escaped for...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17266 can we say something more in the error message? We should explain that it's a hive bug. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #74577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74577/testReport)** for PR 15945 at commit

[GitHub] spark issue #12436: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-14 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/12436 @jisookim0513 - created a new PR - https://github.com/apache/spark/pull/17297 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74576/testReport)** for PR 17166 at commit

[GitHub] spark issue #17177: [SPARK-19834][SQL] csv escape of quote escape

2017-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17177 @ep1804 @jbax Thank you. I will cc and inform you both when I happen to see a PR bumping up the version to 2.4.0 (or probably I guess I will). --- If your project is set up for it, you can

[GitHub] spark pull request #17177: [SPARK-19834][SQL] csv escape of quote escape

2017-03-14 Thread ep1804
Github user ep1804 closed the pull request at: https://github.com/apache/spark/pull/17177 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17177: [SPARK-19834][SQL] csv escape of quote escape

2017-03-14 Thread ep1804
Github user ep1804 commented on the issue: https://github.com/apache/spark/pull/17177 I agree with you @HyukjinKwon , this PR will be closed for now and re-open. And, thank you for the notice @jbox ! --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-14 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r106066177 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -710,7 +710,11 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #17287: [SPARK-19945][SQL]add test suite for SessionCatal...

2017-03-14 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/17287#discussion_r106065119 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -76,468 +118,499 @@ class

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15604 **[Test build #74575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74575/testReport)** for PR 15604 at commit

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-03-14 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15604 @erenavsarogullari please file a JIRA when you see test failures instead of ignoring them. I updated https://issues.apache.org/jira/browse/SPARK-19803 for the first failure, but please file

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-03-14 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15604 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #74573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74573/testReport)** for PR 15945 at commit

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74573/ Test FAILed. ---

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17232: [SPARK-18112] [SQL] Support reading data from Hive 2.1 m...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17232 **[Test build #74574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74574/testReport)** for PR 17232 at commit

[GitHub] spark issue #17223: [SPARK-19881][SQL] Support Dynamic Partition Inserts par...

2017-03-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17223 Since hive client is shared among all sessions, we can't set hive conf dynamically, to keep session isolation. I think we should treat hive conf as static sql conf, and throw exception when users

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #74573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74573/testReport)** for PR 15945 at commit

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15945 **[Test build #74571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74571/testReport)** for PR 15945 at commit

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2017-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15945 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74571/ Test FAILed. ---

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74572/testReport)** for PR 17166 at commit

[GitHub] spark issue #16788: [SPARK-16742] Kerberos impersonation support

2017-03-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/16788 >Trying to put it differently: if Spark had its own, secure method for distributing the initial set of delegation tokens needed by the executors (+ AM in case of YARN), then the YARN backend

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-14 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r106063119 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -710,7 +710,11 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-14 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r106063061 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -239,8 +244,9 @@ private[spark] class Executor( */

  1   2   3   4   5   6   7   >