[GitHub] spark pull request: [SPARK-13742][Core] Add non-iterator interface...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11578#issuecomment-201357380 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11961#issuecomment-201358136 **[Test build #54186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54186/consoleFull)** for PR 11961 at commit [`080a506`](https://gi

[GitHub] spark pull request: [SPARK-14134] [core] [test-maven] Change the p...

2016-03-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11941#issuecomment-201359154 this is the same flaky test that's failing in other places with maven... let me see if it's easy to fix. --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/11961#discussion_r57462635 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/RandomForestClassifier.scala --- @@ -222,16 +222,10 @@ final class RandomForestClassificati

[GitHub] spark pull request: [SPARK-14061][SQL] implement CreateMap

2016-03-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11879 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/11961#discussion_r57462916 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -119,6 +119,31 @@ class GBTClassifierSuite extends SparkFun

[GitHub] spark pull request: [SPARK-14158][SQL] implement buildReader for j...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11960#discussion_r57462957 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JSONRelation.scala --- @@ -120,6 +122,39 @@ class DefaultSource extends File

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread sethah
Github user sethah commented on the pull request: https://github.com/apache/spark/pull/11961#issuecomment-201361949 cc @jkbradley whenever you get a chance, could you take a look? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: [SPARK-14158][SQL] implement buildReader for j...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11960#discussion_r57464019 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -58,7 +58,8 @@ private[sql] object FileSourceStra

[GitHub] spark pull request: [SPARK-14134] [core] [test-maven] Change the p...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11941#issuecomment-201368662 @vanzio I fixed it in #11895 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-14134] [core] [test-maven] Change the p...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11941#issuecomment-201369432 However, #11895 is blocked by #11940. It can not pass tests :( --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-14157][SQL] Parse Drop Function DDL com...

2016-03-25 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11959#issuecomment-201370396 Let me submit a PR for parsing `TOK_DESCDATABASE` and `TOK_DROPDATABASE`. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-14134] [core] [test-maven] Change the p...

2016-03-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11941#issuecomment-201370479 Cool, guess I'll wait for those, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14158][SQL] implement buildReader for j...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11960#discussion_r57465015 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -0,0 +1,50 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14116][SQL] Implements buildReader() fo...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11936#issuecomment-201373230 **[Test build #54183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54183/consoleFull)** for PR 11936 at commit [`a26973e`](https://g

[GitHub] spark pull request: [SPARK-14158][SQL] implement buildReader for j...

2016-03-25 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11960#issuecomment-201373343 Overall looks good. Let's fix the serialization issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-14158][SQL] implement buildReader for j...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11960#discussion_r57465176 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -58,7 +58,8 @@ private[sql] object FileSourceStra

[GitHub] spark pull request: [SPARK-14116][SQL] Implements buildReader() fo...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11936#issuecomment-201373903 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14116][SQL] Implements buildReader() fo...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11936#issuecomment-201373865 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [MINOR] Typo fixes

2016-03-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11802#discussion_r57465383 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -246,9 +246,7 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-14114][SQL] implement buildReader for t...

2016-03-25 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11934#issuecomment-201374918 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-14071][PySpark][ML]Change MLWritable.wr...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11945#issuecomment-201374973 **[Test build #54187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54187/consoleFull)** for PR 11945 at commit [`d8f5d6e`](https://gi

[GitHub] spark pull request: [SPARK-14158][SQL] implement buildReader for j...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11960#discussion_r57465871 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -0,0 +1,50 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-14071][PySpark][ML]Change MLWritable.wr...

2016-03-25 Thread wangmiao1981
Github user wangmiao1981 commented on the pull request: https://github.com/apache/spark/pull/11945#issuecomment-201375961 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11961#issuecomment-201375885 **[Test build #54186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54186/consoleFull)** for PR 11961 at commit [`080a506`](https://g

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11961#issuecomment-201376022 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-11730][ML] Add feature importances for ...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11961#issuecomment-201376017 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14089][CORE][MLLIB] Remove methods that...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11910#issuecomment-201376311 **[Test build #54175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54175/consoleFull)** for PR 11910 at commit [`631c44c`](https://g

[GitHub] spark pull request: [SPARK-14089][CORE][MLLIB] Remove methods that...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11910#issuecomment-201376459 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14114][SQL] implement buildReader for t...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11934#issuecomment-201376305 **[Test build #54188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54188/consoleFull)** for PR 11934 at commit [`947fc70`](https://gi

[GitHub] spark pull request: [SPARK-14089][CORE][MLLIB] Remove methods that...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11910#issuecomment-201376467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12555][SQL] Result should not be corrup...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11623#issuecomment-201378532 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12555][SQL] Result should not be corrup...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11623#issuecomment-201378536 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14071][PySpark][ML]Change MLWritable.wr...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11945#issuecomment-201378991 **[Test build #54187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54187/consoleFull)** for PR 11945 at commit [`d8f5d6e`](https://g

[GitHub] spark pull request: [SPARK-14071][PySpark][ML]Change MLWritable.wr...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11945#issuecomment-201379419 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14071][PySpark][ML]Change MLWritable.wr...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11945#issuecomment-201379414 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14116][SQL] Implements buildReader() fo...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11936#discussion_r57467872 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -117,6 +123,66 @@ private[sql] class DefaultSource extends FileFormat

[GitHub] spark pull request: [SPARK-14116][SQL] Implements buildReader() fo...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11936#discussion_r57467951 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -117,6 +123,66 @@ private[sql] class DefaultSource extends FileFormat

[GitHub] spark pull request: [SPARK-13579][build][wip] Stop building the ma...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11796#issuecomment-201381299 **[Test build #54190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54190/consoleFull)** for PR 11796 at commit [`01d9f0d`](https://gi

[GitHub] spark pull request: [SPARK-14144][SQL] Explicitly identify/catch U...

2016-03-25 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/11950#issuecomment-201381524 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-14116][SQL] Implements buildReader() fo...

2016-03-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11936#discussion_r57468288 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala --- @@ -291,10 +308,10 @@ private[orc] case class OrcTableScan( val

[GitHub] spark pull request: [SPARK-14137][SPARK-14150][SQL] Infer IsNotNul...

2016-03-25 Thread sameeragarwal
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/11953#issuecomment-201382352 thanks, done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-14137][SPARK-14150][SQL] Infer IsNotNul...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11953#issuecomment-201382523 **[Test build #54191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54191/consoleFull)** for PR 11953 at commit [`2f43d41`](https://gi

[GitHub] spark pull request: [SPARK-12555][SQL] Result should not be corrup...

2016-03-25 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/11623#issuecomment-201383428 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-14137][SPARK-14150][SQL] Infer IsNotNul...

2016-03-25 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/11953#issuecomment-201383385 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-12555][SQL] Result should not be corrup...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11623#issuecomment-201383515 **[Test build #54192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54192/consoleFull)** for PR 11623 at commit [`c0e73cd`](https://gi

[GitHub] spark pull request: [SPARK-14089][CORE][MLLIB] Remove methods that...

2016-03-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11910#issuecomment-201383708 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14102][CORE] Block `reset` command in S...

2016-03-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/11920#issuecomment-201385003 OK. If it's pretty simple and there is no good reason to call reset (since it kills the `SparkContext`) maybe it is worth blocking after all. --- If your project is set

[GitHub] spark pull request: [SPARK-14089][CORE][MLLIB] Remove methods that...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11910#issuecomment-201385697 **[Test build #54193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54193/consoleFull)** for PR 11910 at commit [`631c44c`](https://gi

[GitHub] spark pull request: [SPARK-13973] [PySpark]: `ipython notebook` is...

2016-03-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11829#discussion_r57470219 --- Diff: bin/pyspark --- @@ -49,7 +49,10 @@ if [[ -n "$IPYTHON_OPTS" || "$IPYTHON" == "1" ]]; then # If IPython options are specified, assume user want

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11940#discussion_r57470403 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -101,6 +102,65 @@ class StreamExecution( priva

[GitHub] spark pull request: SPARK-14091 [core] Consider improving performa...

2016-03-25 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11911#issuecomment-201389883 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13891] [SQL] [TEST] Issue Exceptions wh...

2016-03-25 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/11714 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14102][CORE] Block `reset` command in S...

2016-03-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11920#issuecomment-201392194 Thank you for the positive feedback. Indeed, it was just one-line change to block `reset`. All the others changes in that file is just about ScalaStyle correct

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/11940#discussion_r57471001 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -101,6 +102,65 @@ class StreamExecution( priv

[GitHub] spark pull request: [SPARK-14109][SQL] Fix HDFSMetadataLog to fall...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11925#issuecomment-201393248 **[Test build #54194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54194/consoleFull)** for PR 11925 at commit [`5eb63b6`](https://gi

[GitHub] spark pull request: SPARK-14091 [core] Consider improving performa...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11911#issuecomment-201393204 **[Test build #54195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54195/consoleFull)** for PR 11911 at commit [`f59c85f`](https://gi

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201393496 LGTM; looking at the code after my comment I think the shorter code is safe, but that's a super minor thing anyway. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201397068 **[Test build #54197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54197/consoleFull)** for PR 11940 at commit [`37343ca`](https://gi

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201397074 **[Test build #54196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54196/consoleFull)** for PR 11942 at commit [`61d220b`](https://gi

[GitHub] spark pull request: [SPARK-13241] [Web UI] Added long values for d...

2016-03-25 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/11326#discussion_r57471730 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala --- @@ -157,12 +157,14 @@ class HistoryServerSuite extends SparkFunS

[GitHub] spark pull request: [SPARK-13241] [Web UI] Added long values for d...

2016-03-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/11326#discussion_r57471934 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala --- @@ -157,12 +157,14 @@ class HistoryServerSuite extends SparkFunSuit

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201399466 @vanzin Oh, no... `interrupted = Thread.interrupted()` is wrong. It should not change `interrupted` to `false` if `Thread.interrupted()` returns false. Reverting it.

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201399560 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201399555 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201399544 **[Test build #54196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54196/consoleFull)** for PR 11942 at commit [`61d220b`](https://g

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/11940#discussion_r57472410 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -228,13 +288,17 @@ class StreamExecution( comm

[GitHub] spark pull request: [SPARK-14073][Streaming][test-maven]Move flume...

2016-03-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11895#issuecomment-201398249 LGTM once we fix the maven test issues. Thanks for taking care of this! --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201400855 This logic is a little complex. Could you add some unit tests to make sure this is correct? --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201401202 > It should not change interrupted to false if Thread.interrupted() returns false Is that because there are multiple calls to `runUninterruptiblyInMicroBatchThr

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201401173 **[Test build #54198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54198/consoleFull)** for PR 11940 at commit [`d8dcd04`](https://gi

[GitHub] spark pull request: [SPARK-13241] [Web UI] Added long values for d...

2016-03-25 Thread ajbozarth
Github user ajbozarth commented on a diff in the pull request: https://github.com/apache/spark/pull/11326#discussion_r57473060 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala --- @@ -157,12 +157,14 @@ class HistoryServerSuite extends SparkFunS

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/11940#discussion_r57473579 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -101,6 +102,65 @@ class StreamExecution( private

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201404121 > Is that because there are multiple calls to runUninterruptiblyInMicroBatchThread, and you want to keep the interrupted status across those calls? If that's the case,

[GitHub] spark pull request: [SPARK-14161] [SQL] Native Parsing for DDL Com...

2016-03-25 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/11962 [SPARK-14161] [SQL] Native Parsing for DDL Comand Drop Database What changes were proposed in this pull request? Based on the Hive DDL document https://cwiki.apache.org/confluence/displ

[GitHub] spark pull request: [SPARK-14161] [SQL] Native Parsing for DDL Com...

2016-03-25 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11962#issuecomment-201405785 cc @yhuai @hvanhovell @viirya Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201405645 **[Test build #54200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54200/consoleFull)** for PR 11940 at commit [`9809acf`](https://gi

[GitHub] spark pull request: [SPARK-14161] [SQL] Native Parsing for DDL Com...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11962#issuecomment-201405643 **[Test build #54199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54199/consoleFull)** for PR 11962 at commit [`8f20e5a`](https://gi

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201407257 > Since L157 calls microBatchThread.interrupt(), it's safe I see; so how about simplifying things a bit. Can I suggest that you change the name of the variable t

[GitHub] spark pull request: [SPARK-13241] [Web UI] Added long values for d...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11326#issuecomment-201408109 **[Test build #54202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54202/consoleFull)** for PR 11326 at commit [`c20035c`](https://gi

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201408099 **[Test build #54201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54201/consoleFull)** for PR 11942 at commit [`c3d0f61`](https://gi

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201414609 > I see; so how about simplifying things a bit. Can I suggest that you change the name of the variable to shouldInterruptThread or something that more explicitly says t

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201415819 I'm going to merge this one after tests pass to unblock other PRs. Will address further comments in another PR. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-14131][SQL]Add a workaround for HADOOP-...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11940#issuecomment-201417804 **[Test build #54203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54203/consoleFull)** for PR 11940 at commit [`45f1452`](https://gi

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201418463 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201418364 **[Test build #54201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54201/consoleFull)** for PR 11942 at commit [`c3d0f61`](https://g

[GitHub] spark pull request: [SPARK-14135] Add off-heap storage memory book...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11942#issuecomment-201418469 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14144][SQL] Explicitly identify/catch U...

2016-03-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11950 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12343][YARN] Simplify Yarn client and c...

2016-03-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11603#discussion_r57476945 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -33,6 +34,10 @@ package object config { private[spark] val DRIVER_U

[GitHub] spark pull request: [SPARK-12343][YARN] Simplify Yarn client and c...

2016-03-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11603#discussion_r57477150 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMasterArguments.scala --- @@ -101,12 +91,8 @@ class ApplicationMasterArguments(val args

[GitHub] spark pull request: [SPARK-14098][SQL][WIP] Generate Java code tha...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11956#issuecomment-201425493 **[Test build #54178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54178/consoleFull)** for PR 11956 at commit [`c26cead`](https://g

[GitHub] spark pull request: [SPARK-14098][SQL][WIP] Generate Java code tha...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11956#issuecomment-201426182 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14098][SQL][WIP] Generate Java code tha...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11956#issuecomment-201426171 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12343][YARN] Simplify Yarn client and c...

2016-03-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11603#discussion_r57477923 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -480,25 +495,16 @@ private[spark] class Client( } /**

[GitHub] spark pull request: [SPARK-13579][build][wip] Stop building the ma...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11796#issuecomment-201430662 **[Test build #54185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54185/consoleFull)** for PR 11796 at commit [`976ce2c`](https://g

[GitHub] spark pull request: [SPARK-13579][build][wip] Stop building the ma...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11796#issuecomment-201431670 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SQL][HOTFIX] Fix flakiness in StateStoreRDDSu...

2016-03-25 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11924#issuecomment-201431779 Let's merge it now as it fails several times per day. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-13579][build][wip] Stop building the ma...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11796#issuecomment-201431676 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SQL][HOTFIX] Fix flakiness in StateStoreRDDSu...

2016-03-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11924 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12343][YARN] Simplify Yarn client and c...

2016-03-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11603#discussion_r57478478 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -510,10 +516,12 @@ private[spark] class Client( * (3) whether to add

<    1   2   3   4   5   6   7   >