[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39891704 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1159,4 +1159,104 @@ class JsonSuite extends Query

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8825#issuecomment-141543896 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8825#issuecomment-141543857 [Test build #42680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42680/console) for PR 8825 at commit [`a0e7fc6`](https://github.

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8825#issuecomment-141543900 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10540] Fixes flaky all-data-type test

2015-09-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8768 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39891131 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1159,4 +1159,104 @@ class JsonSuite extends Query

[GitHub] spark pull request: [SPARK-10540] Fixes flaky all-data-type test

2015-09-18 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/8768#issuecomment-141541623 Thanks! Merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39891065 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1159,4 +1159,104 @@ class JsonSuite extends Query

[GitHub] spark pull request: [SPARK-10540] Fixes flaky all-data-type test

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8768#issuecomment-141540489 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39890833 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1159,4 +1159,104 @@ class JsonSuite extends Query

[GitHub] spark pull request: [SPARK-10540] Fixes flaky all-data-type test

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8768#issuecomment-141540485 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10540] Fixes flaky all-data-type test

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8768#issuecomment-141539957 [Test build #42671 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42671/console) for PR 8768 at commit [`5e3361a`](https://github.

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39890261 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1159,4 +1159,104 @@ class JsonSuite extends Query

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8825#issuecomment-141537831 [Test build #42680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42680/consoleFull) for PR 8825 at commit [`a0e7fc6`](https://gith

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39889891 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1159,4 +1159,104 @@ class JsonSuite extends Query

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39889832 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonGenerator.scala --- @@ -73,6 +73,38 @@ private[sql] object JacksonGene

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8825#issuecomment-141536602 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8825#issuecomment-141536569 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10704] Consolidate HashShuffleReader an...

2015-09-18 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/8825 [SPARK-10704] Consolidate HashShuffleReader and ShuffleReader; refactor ShuffleManager.getReader() The current shuffle code has an interface named ShuffleReader with only one implementation, Hash

[GitHub] spark pull request: [SPARK-10626][MLLIB] create java friendly meth...

2015-09-18 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8782#issuecomment-141536508 I don't have a strong opinion on it. It looks like there is this parallel structure for everything except the random and random-vector methods, for some reason. But yeah

[GitHub] spark pull request: [SPARK-10671] [SQL] Throws an analysis excepti...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141536165 [Test build #42679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42679/consoleFull) for PR 8800 at commit [`e0e1a4d`](https://gith

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r39889195 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonParser.scala --- @@ -62,10 +62,23 @@ private[sql] object JacksonPa

[GitHub] spark pull request: [SPARK-10671] [SQL] Throws an analysis excepti...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141535625 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10671] [SQL] Throws an analysis excepti...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141535653 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6028][Core]A new RPC implemetation base...

2015-09-18 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/6457#issuecomment-141535317 Hi @zsxwing, There were a couple of things that were bothering me in the code but I couldn't really put my finger on them. Instead of trying to point them out in

[GitHub] spark pull request: [SPARK-9774][ML][PYSPARK] Add python api for m...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8214#issuecomment-141535347 [Test build #42678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42678/consoleFull) for PR 8214 at commit [`eac380b`](https://gith

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r3981 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonGenerator.scala --- @@ -73,6 +73,38 @@ private[sql] object Jackson

[GitHub] spark pull request: [SPARK-10495] [SQL] Read date values in JSON d...

2015-09-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8806#discussion_r3965 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonGenerator.scala --- @@ -73,6 +73,38 @@ private[sql] object Jackson

[GitHub] spark pull request: [SPARK-10626][MLLIB] create java friendly meth...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8782#issuecomment-141534938 [Test build #42677 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42677/consoleFull) for PR 8782 at commit [`dae7346`](https://gith

[GitHub] spark pull request: [SPARK-10677] [SQL] [WIP] UnsafeExternalSorter...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8805#issuecomment-141534690 Actually @yhuai and I just found the real root cause of the original issue SPARK-10474. I'm closing this PR. --- If your project is set up for it, you can reply to t

[GitHub] spark pull request: [SPARK-10677] [SQL] [WIP] UnsafeExternalSorter...

2015-09-18 Thread andrewor14
Github user andrewor14 closed the pull request at: https://github.com/apache/spark/pull/8805 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-9774][ML][PYSPARK] Add python api for m...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8214#issuecomment-141534546 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9774][ML][PYSPARK] Add python api for m...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8214#issuecomment-141534572 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10447][WIP][PYSPARK] upgrade pyspark to...

2015-09-18 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/8615#issuecomment-141534364 So I left this running on a machine just to see if it would ever complete, and it did although: Finished test(python): pyspark.streaming.tests (762976s) Finished

[GitHub] spark pull request: [SPARK-9642] [ML] [WIP] LinearRegression shoul...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8631#issuecomment-141533658 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9642] [ML] [WIP] LinearRegression shoul...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8631#issuecomment-141533663 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10626][MLLIB] create java friendly meth...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8782#issuecomment-141533488 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9642] [ML] [WIP] LinearRegression shoul...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8631#issuecomment-141533529 [Test build #42670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42670/console) for PR 8631 at commit [`854d0bb`](https://github.

[GitHub] spark pull request: [SPARK-10626][MLLIB] create java friendly meth...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8782#issuecomment-141533463 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10626][MLLIB] create java friendly meth...

2015-09-18 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/8782#issuecomment-141532528 @srowen I figured providing a general method would be easier than playing whack-a-mole with all of the other wrapper methods, but I can either add vector to this one or

[GitHub] spark pull request: Spark-10625: Spark SQL JDBC read/write is unab...

2015-09-18 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/8785#issuecomment-141532159 Looks like good improvements, less duplicated code, although there are still some minor style issues. You might want to merge in the latest master branch so that tests c

[GitHub] spark pull request: Spark-10625: Spark SQL JDBC read/write is unab...

2015-09-18 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/8785#discussion_r39886832 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -450,4 +449,12 @@ class JDBCSuite extends SparkFunSuite with BeforeAndAfte

[GitHub] spark pull request: Spark-10625: Spark SQL JDBC read/write is unab...

2015-09-18 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/8785#discussion_r39886795 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala --- @@ -75,6 +76,19 @@ private[sql] object JDBCRelation

[GitHub] spark pull request: Spark-10625: Spark SQL JDBC read/write is unab...

2015-09-18 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/8785#discussion_r39886861 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -151,4 +150,13 @@ class JDBCWriteSuite extends SharedSQLContext with

[GitHub] spark pull request: [SPARK-10672] [SQL] Do not fail when we cannot...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8824#issuecomment-141525792 [Test build #42676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42676/consoleFull) for PR 8824 at commit [`dc01065`](https://gith

[GitHub] spark pull request: [SPARK-10539][SQL]Project should not be pushed...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8823#issuecomment-141525663 [Test build #42675 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42675/consoleFull) for PR 8823 at commit [`1f56d2e`](https://gith

[GitHub] spark pull request: [SPARK-9808] Remove hash shuffle file consolid...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8812#issuecomment-141525608 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9808] Remove hash shuffle file consolid...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8812#issuecomment-141525611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-9808] Remove hash shuffle file consolid...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8812#issuecomment-141525459 [Test build #42669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42669/console) for PR 8812 at commit [`0ffbbeb`](https://github.

[GitHub] spark pull request: [SPARK-10672] [SQL] Do not fail when we cannot...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8824#issuecomment-141525293 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10671] [SPARK-10672] [SQL] Throws an an...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141525340 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10671] [SPARK-10672] [SQL] Throws an an...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141525341 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10672] [SQL] Do not fail when we cannot...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8824#issuecomment-141525275 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10677] [SQL] [WIP] UnsafeExternalSorter...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8805#discussion_r39885230 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -254,22 +281,25 @@ public int getNumberOfAlloca

[GitHub] spark pull request: [SPARK-10671] [SPARK-10672] [SQL] Throws an an...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141525164 [Test build #42672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42672/console) for PR 8800 at commit [`e133bee`](https://github.

[GitHub] spark pull request: [SPARK-10672] [SQL] Do not fail when we cannot...

2015-09-18 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/8824 [SPARK-10672] [SQL] Do not fail when we cannot save the metadata of a data source table in a hive compatible way https://issues.apache.org/jira/browse/SPARK-10672 You can merge this pull request into

[GitHub] spark pull request: [SPARK-10539][SQL]Project should not be pushed...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8823#issuecomment-141524234 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10539][SQL]Project should not be pushed...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8823#issuecomment-141524170 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10539][SQL]Project should not be pushed...

2015-09-18 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/8823#issuecomment-141524014 The fix is good. I only added comments. Will merge it to both master and branch 1.5 once it passes jenkins. --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-10539][SQL]Project should not be pushed...

2015-09-18 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/8742#issuecomment-141523674 @yjshen I added the comments and create a new PR (https://github.com/apache/spark/pull/8823). Can you close this one? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-10539][SQL]Project should not be pushed...

2015-09-18 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/8823 [SPARK-10539][SQL]Project should not be pushed down through Intersect or Except #8742 Intersect and Except are both set operators and they use the all the columns to compare equality between rows. Wh

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/8749#discussion_r39884511 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -264,6 +264,7 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAllo

[GitHub] spark pull request: [Spark-5068][SQL]Fix bug query data when path ...

2015-09-18 Thread moustaki
Github user moustaki commented on the pull request: https://github.com/apache/spark/pull/5059#issuecomment-141522701 Perfect, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8749#issuecomment-141522803 [Test build #42674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42674/consoleFull) for PR 8749 at commit [`eaf6169`](https://gith

[GitHub] spark pull request: [SPARK-10671] [SPARK-10672] [SQL] Throws an an...

2015-09-18 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141522666 Will split this PR to two PRs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/8749#discussion_r39883770 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -264,6 +264,7 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAll

[GitHub] spark pull request: [Spark-5068][SQL]Fix bug query data when path ...

2015-09-18 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/5059#issuecomment-141521839 We turned this off by default in Spark 1.5 as it was causing problems similar to what you saw. --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8749#issuecomment-141521108 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8749#issuecomment-141521071 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/8822#discussion_r39883150 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -97,7 +97,7 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAlloc

[GitHub] spark pull request: [SPARK-10695] [Documentation][Mesos] Fixing in...

2015-09-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8816#issuecomment-141521035 Docs are not built by Jenkins so there is no need to run Jenkins here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [SPARK-10695] [Documentation][Mesos] Fixing in...

2015-09-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8816#issuecomment-141521056 cc @tnachen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread tedyu
Github user tedyu closed the pull request at: https://github.com/apache/spark/pull/8822 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/8749#discussion_r39883147 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -264,6 +264,7 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAllo

[GitHub] spark pull request: [SPARK-10684] [SQL] StructType.interpretedOrde...

2015-09-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8808#issuecomment-141520379 @navis can you give us the data type that caused this problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: [SPARK-10671] [SPARK-10672] [SQL] Throws an an...

2015-09-18 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/8800#discussion_r39882820 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala --- @@ -59,19 +59,52 @@ private[hive] class HiveFunctionRegistry(underlying: analysi

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8749#issuecomment-141520185 Yes I agree @kmadhugit can you add a java doc to explain the semantics? --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark pull request: [SPARK-10458] [Spark Core] Added isStopped() m...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8749#issuecomment-141520083 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141520065 OK let's close this PR then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10684] [SQL] StructType.interpretedOrde...

2015-09-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8808#issuecomment-141519938 Yes - I cherry-picked it now. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141519890 This is a duplicate. Let's move convo to https://github.com/apache/spark/pull/8749 Yes, definitely not OK to expose the mutable object. --- If your project is set up f

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141519586 @rxin yes that was my comment, but do you think we should add a getter for it? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [WIP] [SPARK-9392] [SQL] Dataframe drop should...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8821#issuecomment-141519458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141519443 [Test build #42673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42673/consoleFull) for PR 8822 at commit [`007c6d6`](https://gith

[GitHub] spark pull request: [WIP] [SPARK-9392] [SQL] Dataframe drop should...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8821#issuecomment-141519457 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [WIP] [SPARK-9392] [SQL] Dataframe drop should...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8821#issuecomment-141519322 [Test build #42668 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42668/console) for PR 8821 at commit [`a4cc671`](https://github.

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8822#discussion_r39882168 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -97,7 +97,7 @@ class SparkContext(config: SparkConf) extends Logging with Executor

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141519199 @srowen @JoshRosen do you guys think this is a good idea? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: SPARK-10701 Expose SparkContext#stopped flag w...

2015-09-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141519233 -1 I don't think we should just expose an atomic variable directly. --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8781#issuecomment-141518821 @tdas this looks good. Once you address the comments I can merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8781#discussion_r39881884 --- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala --- @@ -86,4 +87,62 @@ private[spark] object ThreadUtils { val threadFactory

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8781#discussion_r39881817 --- Diff: core/src/test/scala/org/apache/spark/util/ThreadUtilsSuite.scala --- @@ -66,4 +66,24 @@ class ThreadUtilsSuite extends SparkFunSuite { v

[GitHub] spark pull request: Expose SparkContext#stopped flag with @Develop...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141518002 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Expose SparkContext#stopped flag with @Develop...

2015-09-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8822#issuecomment-141518019 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8781#discussion_r39881618 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -180,6 +180,38 @@ class StreamingContextSuite extends Spa

[GitHub] spark pull request: Expose SparkContext#stopped flag with @Develop...

2015-09-18 Thread tedyu
GitHub user tedyu opened a pull request: https://github.com/apache/spark/pull/8822 Expose SparkContext#stopped flag with @DeveloperApi See this thread: http://search-hadoop.com/m/q3RTtqvncy17sSTx1 We should expose this flag to developers @andrewor14 You can merge

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8781#discussion_r39881407 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -588,12 +588,16 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8781#discussion_r39881201 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -588,12 +588,16 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-10649][STREAMING] Prevent inheriting jo...

2015-09-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8781#discussion_r39881028 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -588,12 +588,16 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-10671] [SPARK-10672] [SQL] Throws an an...

2015-09-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8800#issuecomment-141516265 [Test build #42672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42672/consoleFull) for PR 8800 at commit [`e133bee`](https://gith

<    1   2   3   4   5   6   7   >