[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution serializable

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16272 **[Test build #70103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70103/consoleFull)** for PR 16272 at commit

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution serializable

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16272 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70103/ Test FAILed. ---

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16271 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70100/ Test PASSed. ---

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70100/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16270: [SPARK-18846][Scheduler] Fix flakiness in SchedulerInteg...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16270 **[Test build #70104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70104/consoleFull)** for PR 16270 at commit

[GitHub] spark issue #16266: [SPARK-18842][TESTS][LAUNCHER] De-duplicate paths in cla...

2016-12-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16266 Let me try to fix the tests accordingly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16272: [SPARK-18850][SS]Make StreamExecution serializabl...

2016-12-13 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16272#discussion_r92279982 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -38,7 +38,7 @@ import

[GitHub] spark issue #16272: [SPARK-18850][SS]Make StreamExecution serializable

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16272 **[Test build #70103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70103/consoleFull)** for PR 16272 at commit

[GitHub] spark pull request #16258: [SPARK-18834][SS] Expose event time stats through...

2016-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16258 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16258: [SPARK-18834][SS] Expose event time stats through Stream...

2016-12-13 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16258 Merging to master and 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16272: [SPARK-18850][SS]Make StreamExecution serializabl...

2016-12-13 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16272#discussion_r92279247 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala --- @@ -71,7 +71,7 @@ trait ProgressReporter extends

[GitHub] spark pull request #16272: [SPARK-18850][SS]Make StreamExecution serializabl...

2016-12-13 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16272 [SPARK-18850][SS]Make StreamExecution serializable ## What changes were proposed in this pull request? This PR makes StreamExecution serializable because it is too easy for it to get

[GitHub] spark issue #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #70102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70102/consoleFull)** for PR 14079 at commit

[GitHub] spark pull request #16258: [SPARK-18834][SS] Expose event time stats through...

2016-12-13 Thread tdas
GitHub user tdas reopened a pull request: https://github.com/apache/spark/pull/16258 [SPARK-18834][SS] Expose event time stats through StreamingQueryProgress ## What changes were proposed in this pull request? - Changed `StreamingQueryProgress.watermark` to

[GitHub] spark pull request #16258: [SPARK-18834][SS] Expose event time stats through...

2016-12-13 Thread tdas
Github user tdas closed the pull request at: https://github.com/apache/spark/pull/16258 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14079 thanks for the review @kayousterhout. I also added a testcase to BlacklistTrackerSuite, "task failure timeout works as expected for long-running tasksets" to cover your point about the long running

[GitHub] spark pull request #16268: [SPARK-18843][Core]Fix timeout in awaitResultInFo...

2016-12-13 Thread zsxwing
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/16268 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16258: [SPARK-18834][SS] Expose event time stats through Stream...

2016-12-13 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/16258 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16268 Thanks! Merging to 2.1 and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/16268 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16270: [SPARK-18846][Scheduler] Fix flakiness in Schedul...

2016-12-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16270#discussion_r92276597 --- Diff: core/src/test/scala/org/apache/spark/scheduler/SchedulerIntegrationSuite.scala --- @@ -27,6 +27,9 @@ import scala.language.existentials

[GitHub] spark pull request #16270: [SPARK-18846][Scheduler] Fix flakiness in Schedul...

2016-12-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16270#discussion_r92276753 --- Diff: core/src/test/scala/org/apache/spark/scheduler/SchedulerIntegrationSuite.scala --- @@ -157,8 +160,16 @@ abstract class

[GitHub] spark pull request #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14079#discussion_r92275871 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -408,6 +411,96 @@ class TaskSchedulerImplSuite extends

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70101/consoleFull)** for PR 16271 at commit

[GitHub] spark issue #16271: [SPARK-18845][GraphX] PageRank has incorrect initializat...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16271 **[Test build #70100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70100/consoleFull)** for PR 16271 at commit

[GitHub] spark pull request #16271: [SPARK-18845][GraphX] PageRank has incorrect init...

2016-12-13 Thread aray
GitHub user aray opened a pull request: https://github.com/apache/spark/pull/16271 [SPARK-18845][GraphX] PageRank has incorrect initialization value that leads to slow convergence ## What changes were proposed in this pull request? Change the initial value in all PageRank

[GitHub] spark pull request #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/14079#discussion_r92274093 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -678,4 +716,13 @@ private[spark] object TaskSchedulerImpl {

[GitHub] spark pull request #16256: [SPARK-18816] [Web UI] Executors Logs column only...

2016-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16256 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16270: [SPARK-18846][Scheduler] Fix flakiness in SchedulerInteg...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16270 **[Test build #70099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70099/consoleFull)** for PR 16270 at commit

[GitHub] spark issue #16256: [SPARK-18816] [Web UI] Executors Logs column only ran vi...

2016-12-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16256 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14079#discussion_r92271406 --- Diff: core/src/test/scala/org/apache/spark/scheduler/SchedulerIntegrationSuite.scala --- @@ -157,8 +160,16 @@ abstract class SchedulerIntegrationSuite[T

[GitHub] spark pull request #16037: [SPARK-18471][MLLIB] In LBFGS, avoid sending huge...

2016-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16037 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16270: [SPARK-18846][Scheduler] Fix flakiness in SchedulerInteg...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16270 **[Test build #70098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70098/consoleFull)** for PR 16270 at commit

[GitHub] spark issue #16037: [SPARK-18471][MLLIB] In LBFGS, avoid sending huge vector...

2016-12-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16037 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #16270: [SPARK-18846][Scheduler] Fix flakiness in Schedul...

2016-12-13 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/16270 [SPARK-18846][Scheduler] Fix flakiness in SchedulerIntegrationSuite ## What changes were proposed in this pull request? There is a small race in SchedulerIntegrationSuite. The test

[GitHub] spark pull request #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial...

2016-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16149 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16158 **[Test build #70097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70097/consoleFull)** for PR 16158 at commit

[GitHub] spark issue #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial GLM

2016-12-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16149 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16268 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70091/ Test PASSed. ---

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16268 **[Test build #70091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70091/consoleFull)** for PR 16268 at commit

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16268 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14079#discussion_r92267462 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -678,4 +716,13 @@ private[spark] object TaskSchedulerImpl {

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-13 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/16122 @wangyum Thank you for this important bug fix! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16269: [WIP] simplify data source analysis

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16269 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70092/ Test FAILed. ---

[GitHub] spark issue #16269: [WIP] simplify data source analysis

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16269 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16269: [WIP] simplify data source analysis

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16269 **[Test build #70092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70092/consoleFull)** for PR 16269 at commit

[GitHub] spark issue #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomForest/s...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16264 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomForest/s...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16264 **[Test build #70095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70095/consoleFull)** for PR 16264 at commit

[GitHub] spark issue #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomForest/s...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16264 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70095/ Test PASSed. ---

[GitHub] spark pull request #16134: [SPARK-18703] [SQL] Drop Staging Directories and ...

2016-12-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16134#discussion_r92264626 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -328,6 +332,15 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #16037: [SPARK-18471][MLLIB] In LBFGS, avoid sending huge vector...

2016-12-13 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/16037 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16194: [SPARK-18767][ML] Unify Models' toString methods

2016-12-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16194 Oh that's a good point, you can enumerate the params generically? that makes a lot more sense. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-13 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/16135 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16189: [SPARK-18761][CORE] Introduce "task reaper" to oversee t...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16189 **[Test build #70096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70096/consoleFull)** for PR 16189 at commit

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16268 Yep. I want to backport this obvious fix without the entire patch for master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16268 I see, so there is no point in applying this to master, because it's as if it were already superseded by a bigger fix? but we 'back port' this stop-gap to earlier branches. --- If your project is

[GitHub] spark issue #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomForest/s...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16264 **[Test build #70095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70095/consoleFull)** for PR 16264 at commit

[GitHub] spark issue #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomForest/s...

2016-12-13 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/16264 Agree that it is not necessary to mention the version added here. I will send a follow-up PR after this one to rearrange the ordering of the ML algorithms. There is no logical ordering now. --- If

[GitHub] spark pull request #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomF...

2016-12-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/16264#discussion_r92257313 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -526,6 +530,34 @@ gaussianFitted <- predict(gaussianGLM, carsDF) head(select(gaussianFitted,

[GitHub] spark pull request #16264: [SPARK-18793] [SPARK-18794] [R] add spark.randomF...

2016-12-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/16264#discussion_r92257291 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -539,7 +539,7 @@ In the following example, we use the `longley` dataset to train a random forest

[GitHub] spark issue #16176: [SPARK-18746][SQL] Add implicit encoder for BigDecimal, ...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16176 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70090/ Test PASSed. ---

[GitHub] spark issue #16176: [SPARK-18746][SQL] Add implicit encoder for BigDecimal, ...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16176 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-13 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r92254841 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -161,12 +163,7 @@ private[spark] class Executor( * @param

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r92254810 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -363,48 +364,120 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16176: [SPARK-18746][SQL] Add implicit encoder for BigDecimal, ...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16176 **[Test build #70090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70090/consoleFull)** for PR 16176 at commit

[GitHub] spark pull request #14079: [SPARK-8425][CORE] Application Level Blacklisting

2016-12-13 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14079#discussion_r92254218 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -17,10 +17,254 @@ package org.apache.spark.scheduler

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16247 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70093/ Test PASSed. ---

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16247 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16247 **[Test build #70093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70093/consoleFull)** for PR 16247 at commit

[GitHub] spark issue #16259: [Minor][SparkR]:fix kstest example error and add unit te...

2016-12-13 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16259 windows check passed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16250: [CORE][MINOR] Stylistic changes in DAGScheduler (to ease...

2016-12-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16250 The question is not whether there are downsides; there almost always some. It's whether the upsides are bigger. We typically can't evaluate either of them perfectly, but can take good guesses. The

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15717 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70089/ Test PASSed. ---

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15717 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15717 **[Test build #70089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70089/consoleFull)** for PR 15717 at commit

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-13 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r92252007 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -161,12 +163,7 @@ private[spark] class Executor( * @param

[GitHub] spark issue #16246: [SPARK-18814][SQL] CheckAnalysis rejects TPCDS query 32

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16246 **[Test build #70094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70094/consoleFull)** for PR 16246 at commit

[GitHub] spark issue #16250: [CORE][MINOR] Stylistic changes in DAGScheduler (to ease...

2016-12-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16250 You missed the fact that people don't even think this is a big improvement in usability. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #16247: [SPARK-18817][SparkR] set default spark-warehouse...

2016-12-13 Thread aloknsingh
Github user aloknsingh commented on a diff in the pull request: https://github.com/apache/spark/pull/16247#discussion_r92250724 --- Diff: R/pkg/R/sparkR.R --- @@ -362,6 +362,10 @@ sparkR.session <- function( enableHiveSupport = TRUE, ...) { + if

[GitHub] spark issue #16194: [SPARK-18767][ML] Unify Models' toString methods

2016-12-13 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/16194 Sorry but I would still strongly prefer to override the toString() in Model or Params. The default implementation can provide more information about the params. And only in a few cases you need to

[GitHub] spark pull request #16255: [SPARK-18609][SQL]Fix when CTE with Join between ...

2016-12-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16255#discussion_r92245640 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -200,6 +200,8 @@ object RemoveAliasOnlyProject

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16247 **[Test build #70093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70093/consoleFull)** for PR 16247 at commit

[GitHub] spark issue #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial GLM

2016-12-13 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/16149 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

2016-12-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16247 Yeah disabling Hive for the test is fine. @bdwyer2 Can you add the new config flag as well ? We can do one final pass of review after that --- If your project is set up for it, you can reply to

[GitHub] spark pull request #16134: [SPARK-18703] [SQL] Drop Staging Directories and ...

2016-12-13 Thread merlintang
Github user merlintang commented on a diff in the pull request: https://github.com/apache/spark/pull/16134#discussion_r92244682 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -328,6 +332,15 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #16247: [SPARK-18817][SparkR] set default spark-warehouse path t...

2016-12-13 Thread bdwyer2
Github user bdwyer2 commented on the issue: https://github.com/apache/spark/pull/16247 Would calling the test with this be an acceptable solution? ```R sparkR.session(enableHiveSupport = FALSE) ``` --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #16258: [SPARK-18834][SS] Expose event time stats through Stream...

2016-12-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16258 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16269: [WIP] simplify data source analysis

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16269 **[Test build #70092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70092/consoleFull)** for PR 16269 at commit

[GitHub] spark pull request #16269: [WIP] simplify data source analysis

2016-12-13 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/16269 [WIP] simplify data source analysis ## What changes were proposed in this pull request? The current way of resolving `InsertIntoTable` and `CreateTable` is convoluted: sometimes we

[GitHub] spark pull request #16265: [SPARK-18840][YARN] Avoid throw exception when ge...

2016-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16265 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16268: [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinS...

2016-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16268 **[Test build #70091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70091/consoleFull)** for PR 16268 at commit

[GitHub] spark issue #16265: [SPARK-18840][YARN] Avoid throw exception when getting t...

2016-12-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16265 LGTM. It looks like 2.0 has the same issue as 1.6? Merging to master / 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #16268: [SPARK-18843][Core]Fix timeout in awaitResultInFo...

2016-12-13 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16268 [SPARK-18843][Core]Fix timeout in awaitResultInForkJoinSafely (branch 2.1, 2.0) ## What changes were proposed in this pull request? This PR fixes the timeout value in

[GitHub] spark issue #16259: [Minor][SparkR]:fix kstest example error and add unit te...

2016-12-13 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16259 open to trigger test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16259: [Minor][SparkR]:fix kstest example error and add ...

2016-12-13 Thread wangmiao1981
Github user wangmiao1981 closed the pull request at: https://github.com/apache/spark/pull/16259 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16259: [Minor][SparkR]:fix kstest example error and add unit te...

2016-12-13 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16259 close to trigger appveyor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16259: [Minor][SparkR]:fix kstest example error and add ...

2016-12-13 Thread wangmiao1981
GitHub user wangmiao1981 reopened a pull request: https://github.com/apache/spark/pull/16259 [Minor][SparkR]:fix kstest example error and add unit test ## What changes were proposed in this pull request? While adding vignettes for kstest, I found some errors in the example:

[GitHub] spark pull request #15018: [SPARK-17455][MLlib] Improve PAVA implementation ...

2016-12-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15018#discussion_r92232757 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/IsotonicRegression.scala --- @@ -344,27 +344,30 @@ class IsotonicRegression private

[GitHub] spark pull request #15018: [SPARK-17455][MLlib] Improve PAVA implementation ...

2016-12-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15018#discussion_r92230452 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/IsotonicRegression.scala --- @@ -344,27 +344,30 @@ class IsotonicRegression private

[GitHub] spark pull request #15018: [SPARK-17455][MLlib] Improve PAVA implementation ...

2016-12-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15018#discussion_r92231712 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/IsotonicRegression.scala --- @@ -344,27 +344,30 @@ class IsotonicRegression private

<    1   2   3   4   5   >