[GitHub] spark issue #15229: [SPARK-17654] [SQL] Propagate bucketing information for ...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15229 **[Test build #65862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65862/consoleFull)** for PR 15229 at commit

[GitHub] spark pull request #15229: [SPARK-17654] [SQL] Propagate bucketing informati...

2016-09-23 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/15229 [SPARK-17654] [SQL] Propagate bucketing information for Hive tables to / from Catalog ## What changes were proposed in this pull request? Currently Spark does not respect bucketing

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65858/ Test PASSed. ---

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65861/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15168 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12601 Mostly LGTM, except three minor comments. Thank you for your hard work, @JustinPihony ! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80353253 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -420,62 +420,11 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80353203 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -208,4 +210,84 @@ class JDBCWriteSuite extends SharedSQLContext

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15168 The failure seems to be irrelevant. Retest this please. ``` [info] - Naive Bayes Multinomial *** FAILED *** (137 milliseconds) [info] Expected 0.7 and 0.6494565217391305 to be

[GitHub] spark issue #15228: [SPARK-17654] [SQL] Propagate bucketing information for ...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15228 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15228: [SPARK-17654] [SQL] Propagate bucketing information for ...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15228 **[Test build #65857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65857/consoleFull)** for PR 15228 at commit

[GitHub] spark issue #15228: [SPARK-17654] [SQL] Propagate bucketing information for ...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15228 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65857/ Test FAILed. ---

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80353010 --- Diff: docs/sql-programming-guide.md --- @@ -1096,13 +1096,17 @@ the Data Sources API. The following options are supported: {% highlight

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65859/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65859/ Test FAILed. ---

[GitHub] spark issue #15217: [SPARK-17577][Core] Update SparkContext.addFile to make ...

2016-09-23 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15217 Close this PR. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15217: [SPARK-17577][Core] Update SparkContext.addFile t...

2016-09-23 Thread yanboliang
Github user yanboliang closed the pull request at: https://github.com/apache/spark/pull/15217 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #15228: [SPARK-17654] [SQL] Propagate bucketing informati...

2016-09-23 Thread tejasapatil
Github user tejasapatil closed the pull request at: https://github.com/apache/spark/pull/15228 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread JustinPihony
Github user JustinPihony commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80352586 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java --- @@ -21,6 +21,7 @@ import java.util.ArrayList;

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65860/consoleFull)** for PR 12601 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65859/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/12601 Thanks for mentioning me. It looks good to me in my personal view. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80352317 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java --- @@ -21,6 +21,7 @@ import java.util.ArrayList;

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65858/consoleFull)** for PR 12601 at commit

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread JustinPihony
Github user JustinPihony commented on the issue: https://github.com/apache/spark/pull/12601 @gatorsmile I added the R and SQL documentation. I took the SQL portion from https://github.com/apache/spark/pull/6121/files --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #15228: [SPARK-17654] [SQL] Propagate bucketing information for ...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15228 **[Test build #65857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65857/consoleFull)** for PR 15228 at commit

[GitHub] spark issue #15071: [SPARK-17517][SQL]Improve generated Code for BroadcastHa...

2016-09-23 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/15071 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15228: [SPARK-17654] [SQL] Propagate bucketing informati...

2016-09-23 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/15228 [SPARK-17654] [SQL] Propagate bucketing information for Hive tables to / from Catalog ## What changes were proposed in this pull request? Currently Spark does not respect bucketing

[GitHub] spark issue #15227: [SPARK-17655][SQL]Remove unused variables declarations a...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15227 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15227: [SPARK-17655][SQL]Remove unused variables declara...

2016-09-23 Thread yaooqinn
GitHub user yaooqinn opened a pull request: https://github.com/apache/spark/pull/15227 [SPARK-17655][SQL]Remove unused variables declarations and definations in a WholeStageCodeGened stage ## What changes were proposed in this pull request? A WholeStageCodeGened stage with

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15218 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65856/ Test PASSed. ---

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15218 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15218 **[Test build #65856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65856/consoleFull)** for PR 15218 at commit

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80350919 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java --- @@ -23,6 +23,8 @@ import java.util.List; //

[GitHub] spark issue #15226: [SPARK-17649][CORE] Log how many Spark events got droppe...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65855/ Test PASSed. ---

[GitHub] spark issue #15226: [SPARK-17649][CORE] Log how many Spark events got droppe...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15226 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15226: [SPARK-17649][CORE] Log how many Spark events got droppe...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15226 **[Test build #65855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65855/consoleFull)** for PR 15226 at commit

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread JustinPihony
Github user JustinPihony commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80350755 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java --- @@ -23,6 +23,8 @@ import java.util.List; //

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80350458 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java --- @@ -23,6 +23,8 @@ import java.util.List; //

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15224 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65854/ Test PASSed. ---

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15224 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15224 **[Test build #65854 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65854/consoleFull)** for PR 15224 at commit

[GitHub] spark pull request #15226: [SPARK-17649][CORE] Log how many Spark events got...

2016-09-23 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15226#discussion_r80350179 --- Diff: core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala --- @@ -117,6 +124,24 @@ private[spark] abstract class

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Do not add failedStages when abortS...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15213 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Do not add failedStages when abortS...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15213 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65853/ Test PASSed. ---

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Do not add failedStages when abortS...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15213 **[Test build #65853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65853/consoleFull)** for PR 15213 at commit

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15220 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65851/ Test PASSed. ---

[GitHub] spark pull request #15226: [SPARK-17649][CORE] Log how many Spark events got...

2016-09-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15226#discussion_r80347195 --- Diff: core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala --- @@ -117,6 +124,24 @@ private[spark] abstract class

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15220 **[Test build #65851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65851/consoleFull)** for PR 15220 at commit

[GitHub] spark issue #15226: [SPARK-17649][CORE] Log how many Spark events got droppe...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15226 **[Test build #65855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65855/consoleFull)** for PR 15226 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15218 **[Test build #65856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65856/consoleFull)** for PR 15218 at commit

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/15213 > actual problem is not in abortStage but rather in improper additions to failedStages correct, i think a more accurate description for this issue is "do not add `failedStages` when

[GitHub] spark pull request #15226: [SPARK-17649][CORE] Log how many Spark events got...

2016-09-23 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15226 [SPARK-17649][CORE] Log how many Spark events got dropped in AsynchronousListenerBus ## What changes were proposed in this pull request? Backport #15220 to 1.6. ## How was this

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-09-23 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/15218 @gatorsmile Thanks. #65832 is the latest one which does not have the same failure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15220 Thanks! Merging to master / 2.0. I will submit a patch for 1.6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15223: [SPARKR][SPARK-17651] Set R package version number along...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15223 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15223: [SPARKR][SPARK-17651] Set R package version number along...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65849/ Test PASSed. ---

[GitHub] spark issue #15223: [SPARKR][SPARK-17651] Set R package version number along...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15223 **[Test build #65849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65849/consoleFull)** for PR 15223 at commit

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/15213 Right, but `abortStage` occurs elsewhere. "When abort stage" seems to imply that this fix is necessary for all usages of `abortStage` when the actual problem is not in `abortStage` but rather

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/15213 Actually the failedStages only added here in spark. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/15213 @scwf That description would actually be at least as bad since there are multiple routes to `abortStage` and this issue of adding to `failedStages` only applies to these two. I'll take another

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15089 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15089 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65850/ Test PASSed. ---

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15089 **[Test build #65850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65850/consoleFull)** for PR 15089 at commit

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/15213 Ok, that makes better sense. The `disallowStageRetryForTest` case doesn't worry me too much since it is only used in tests. If we can fix this case, great; else if it remains possible

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/15213 Thanks @zsxwing to explain this. @markhamstra the issue happens in the case of my PR description. It usually depends on muti-thread submitting jobs cases and the order of fetch failure, so i said

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15224 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65846/ Test PASSed. ---

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15224 **[Test build #65846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65846/consoleFull)** for PR 15224 at commit

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65847/ Test PASSed. ---

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15220 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15220 **[Test build #65847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65847/consoleFull)** for PR 15220 at commit

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15089 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65848/ Test PASSed. ---

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15089 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15213 @markhamstra I agreed this is not a race condition since there is only one single thread. This issue is the code doesn't handle the following two corner cases: -

[GitHub] spark issue #15089: [SPARK-15621] [SQL] Support spilling for Python UDF

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15089 **[Test build #65848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65848/consoleFull)** for PR 15089 at commit

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15224 **[Test build #65854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65854/consoleFull)** for PR 15224 at commit

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-23 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15224 @zsxwing Thanks for the review. Addressed the nit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/15213 This doesn't make sense to me. The DAGSchedulerEventProcessLoop runs on a single thread and processes a single event from its queue at a time. When the first CompletionEvent is run as

[GitHub] spark issue #15223: [SPARKR][SPARK-17651] Set R package version number along...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15223 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15223: [SPARKR][SPARK-17651] Set R package version number along...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65844/ Test PASSed. ---

[GitHub] spark issue #15223: [SPARKR][SPARK-17651] Set R package version number along...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15223 **[Test build #65844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65844/consoleFull)** for PR 15223 at commit

[GitHub] spark issue #14808: [SPARK-17156][ML][EXAMPLE] Add multiclass logistic regre...

2016-09-23 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14808 I think we should close this. The new example and the user guide should be updated against [SPARK-17239](https://issues.apache.org/jira/browse/SPARK-17239). @jaceklaskowski If you'd still like to

[GitHub] spark issue #15213: [SPARK-17644] [CORE] Fix the race condition when DAGSche...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15213 **[Test build #65853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65853/consoleFull)** for PR 15213 at commit

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15200 if it's part of the `-Psparkr` profile of the build it will be regenerated by default. If it's changed and not in .gitignore it should be flagged for commit.. --- If your project is set up

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65843/ Test PASSed. ---

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15220 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15220: [SPARK-17649][Core]Log how many Spark events got dropped...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15220 **[Test build #65843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65843/consoleFull)** for PR 15220 at commit

[GitHub] spark pull request #15225: [SPARK-17652] Fix confusing exception message whi...

2016-09-23 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15225#discussion_r80340526 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -285,19 +285,19 @@ public void reserve(int

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12601 Not sure you already knew it. Just want to share the commands how to build the doc. ```Scala SKIP_API=1 jekyll build SKIP_API=1 jekyll serve ``` After the second

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15200 Yeah - so I'm thinking we should just auto-generate this and check in the html file in git. Its not that big. When somebody updates the vignette we need to remind them to regenerate it though as a

[GitHub] spark pull request #15225: [SPARK-17652] Fix confusing exception message whi...

2016-09-23 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/15225#discussion_r80339515 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -285,19 +285,19 @@ public void reserve(int

[GitHub] spark issue #15225: [SPARK-17652] Fix confusing exception message while rese...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15225 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65852/ Test FAILed. ---

[GitHub] spark pull request #15224: [SPARK-17650] malformed url's throw exceptions be...

2016-09-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15224#discussion_r80337786 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -698,6 +698,28 @@ private[spark] object Utils extends Logging { }

[GitHub] spark issue #15225: [SPARK-17652] Fix confusing exception message while rese...

2016-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15225 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15225: [SPARK-17652] Fix confusing exception message while rese...

2016-09-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15225 **[Test build #65852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65852/consoleFull)** for PR 15225 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-09-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15218 See https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65832/testReport/org.apache.spark.streaming.kafka010/DirectKafkaStreamSuite/pattern_based_subscription/history/ --- If

[GitHub] spark pull request #15225: [SPARK-17652] Fix confusing exception message whi...

2016-09-23 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15225#discussion_r80337683 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -285,19 +285,19 @@ public void reserve(int

  1   2   3   4   >