[GitHub] spark pull request: [SPARK-8032] [PySpark] Make version checking f...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6579#issuecomment-107992268 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6578#issuecomment-107992316 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6578#issuecomment-107993019 [Test build #33993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33993/consoleFull) for PR 6578 at commit [`720d293`](https://gith

[GitHub] spark pull request: [SPARK-8032] [PySpark] Make version checking f...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6579#issuecomment-107993015 [Test build #33992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33992/consoleFull) for PR 6579 at commit [`22703dd`](https://gith

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/6578#issuecomment-107993570 This failed a totally unrelated streaming test, so I think it's okay. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6578 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-7331][SQL] Re-use HiveConf in HiveQl

2015-06-02 Thread nitin2goyal
Github user nitin2goyal commented on the pull request: https://github.com/apache/spark/pull/5880#issuecomment-107994772 Sorry for not being clear in my first comment but I meant that for queries like "select a,b,c from table where a=10" where table is cached as schemaRDD (spark-1.2),

[GitHub] spark pull request: [SPARK-8015] [flume] Remove Guava dependency f...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6555#issuecomment-107994949 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-5479] [yarn] Handle --py-files correctl...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6360#issuecomment-107994995 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5479] [yarn] Handle --py-files correctl...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6360#issuecomment-107994968 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-8015] [flume] Remove Guava dependency f...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6555#issuecomment-107994988 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8015] [flume] Remove Guava dependency f...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6555#issuecomment-107995146 [Test build #33994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33994/consoleFull) for PR 6555 at commit [`c0ceea8`](https://gith

[GitHub] spark pull request: [SPARK-5479] [yarn] Handle --py-files correctl...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6360#issuecomment-107995533 [Test build #33995 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33995/consoleFull) for PR 6360 at commit [`220358b`](https://gith

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-06-02 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5748#discussion_r31537671 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -426,38 +422,40 @@ class Word2Vec extends Serializable with Logging {

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-06-02 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/5748#discussion_r31537727 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala --- @@ -508,7 +507,7 @@ class Word2VecModel private[mllib] ( */

[GitHub] spark pull request: [SPARK-7432] [MLLIB] fix flaky CrossValidator ...

2015-06-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6572#issuecomment-107996927 With `maxIter=0`, `areaUnderROC` is 0.5. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-7705][Yarn] Cleanup of .sparkStaging di...

2015-06-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/6409#issuecomment-107997181 Ok, last patch LGTM. I wonder if it's worth it to add KILLED to the condition here: https://github.com/apache/spark/blob/master/yarn/src/main/scala/org/apache/spark/deplo

[GitHub] spark pull request: [SPARK-6324] [core] Centralize handling of scr...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5841#issuecomment-107997557 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6324] [core] Centralize handling of scr...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5841#issuecomment-107997533 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7432] [MLLIB] fix flaky CrossValidator ...

2015-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6572 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-8015] [flume] Remove Guava dependency f...

2015-06-02 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/6555#issuecomment-107997452 Lgtm pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-7432] [MLLIB] fix flaky CrossValidator ...

2015-06-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6572#issuecomment-107997600 Merged into master and branch-1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-6324] [core] Centralize handling of scr...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5841#issuecomment-107998131 [Test build #33996 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33996/consoleFull) for PR 5841 at commit [`2821481`](https://gith

[GitHub] spark pull request: [SPARK-7884] Allow Spark shuffle APIs to be mo...

2015-06-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6423#issuecomment-107998999 By the way, you don't need to squash your commits together, since our merge script will now take care of it when we commit things. Leaving them separate makes it easi

[GitHub] spark pull request: [WIP][SPARK-7025] Java-friendly InputSource AP...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5603#issuecomment-107999296 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6324] [core] Centralize handling of scr...

2015-06-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5841#issuecomment-107996878 Ok, last patch LGTM. I wonder if it's worth it to add `KILLED` to the condition here: https://github.com/apache/spark/blob/master/yarn/src/main/scala/org/apache/spark/dep

[GitHub] spark pull request: [WIP][SPARK-7025] Java-friendly InputSource AP...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5603#issuecomment-107999369 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP][SPARK-7025] Java-friendly InputSource AP...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5603#issuecomment-107999889 [Test build #33997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33997/consoleFull) for PR 5603 at commit [`2a2f213`](https://gith

[GitHub] spark pull request: [SPARK-4122][STREAMING] Add a library that can...

2015-06-02 Thread BenFradet
Github user BenFradet commented on the pull request: https://github.com/apache/spark/pull/2994#issuecomment-108002092 Great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-7884] Allow Spark shuffle APIs to be mo...

2015-06-02 Thread massie
Github user massie commented on the pull request: https://github.com/apache/spark/pull/6423#issuecomment-108002390 Thanks for the tip. I'll make sure to leave the commits separate moving forward. I looked into the use of `Try` between the `BlockStoreShuffleFetcher` and `Shuff

[GitHub] spark pull request: [SPARK-4118] [MLlib] [PySpark] Python bindings...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6499#issuecomment-108002278 [Test build #33987 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33987/consoleFull) for PR 6499 at commit [`eaace71`](https://git

[GitHub] spark pull request: [SPARK-4118] [MLlib] [PySpark] Python bindings...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6499#issuecomment-108002307 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31540947 --- Diff: core/src/main/java/org/apache/spark/network/buffer/LargeByteBufferOutputStream.java --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31540997 --- Diff: core/src/main/java/org/apache/spark/network/buffer/LargeByteBufferOutputStream.java --- @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108005849 Jenkins, retest this pleaas.e --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541402 --- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java --- @@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541397 --- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java --- @@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541390 --- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaParamsSuite.java --- @@ -50,6 +50,7 @@ public void testParams() { testParams.setMyIntPara

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541382 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -345,6 +346,12 @@ class DistributedLDAModel private ( }

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541432 --- Diff: mllib/src/test/java/org/apache/spark/mllib/stat/JavaStatisticsSuite.java --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541405 --- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java --- @@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541385 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/stat/Statistics.scala --- @@ -96,6 +101,9 @@ object Statistics { */ def corr(x: RDD[Double

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541392 --- Diff: mllib/src/test/java/org/apache/spark/ml/param/JavaTestParams.java --- @@ -72,15 +72,31 @@ public JavaTestParams setMyStringParam(String value) {

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541375 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -190,6 +186,7 @@ class DoubleParam(parent: String, name: String, doc: String, isVa

[GitHub] spark pull request: [SPARK-7529] [mllib] Added several Java-friend...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6562#discussion_r31541414 --- Diff: mllib/src/test/java/org/apache/spark/mllib/classification/JavaStreamingLogisticRegressionSuite.java --- @@ -15,7 +15,7 @@ * limitations under

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108006519 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108006498 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31541648 --- Diff: core/src/main/scala/org/apache/spark/util/io/ByteArrayChunkOutputStream.scala --- @@ -91,4 +95,45 @@ class ByteArrayChunkOutputStream(chunkSize: Int

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108006503 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [Spark-8043][MLlib][Doc] update NaiveBayes and...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6584#discussion_r31541724 --- Diff: docs/mllib-linear-methods.md --- @@ -282,8 +277,8 @@ public class SVMClassifier { System.out.println("Area under ROC = " + auROC);

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108007032 [Test build #33998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33998/consoleFull) for PR 6580 at commit [`0d52b89`](https://gith

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-06-02 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-108007247 @jkbradley fixed! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-108007496 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-108007522 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7045] [MLlib] Avoid intermediate repres...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5748#issuecomment-108007641 [Test build #33999 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33999/consoleFull) for PR 5748 at commit [`14ee596`](https://gith

[GitHub] spark pull request: [SPARK-7884] Allow Spark shuffle APIs to be mo...

2015-06-02 Thread massie
Github user massie commented on a diff in the pull request: https://github.com/apache/spark/pull/6423#discussion_r31542306 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -290,22 +287,15 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108008248 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108008241 [Test build #33998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33998/consoleFull) for PR 6580 at commit [`0d52b89`](https://git

[GitHub] spark pull request: [SPARK-4122][STREAMING] Add a library that can...

2015-06-02 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2994#issuecomment-108008961 I will get around to this in a few days. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3586][streaming]Support nested director...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6588#issuecomment-108009312 [Test build #33990 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33990/consoleFull) for PR 6588 at commit [`2d85159`](https://git

[GitHub] spark pull request: [SPARK-3586][streaming]Support nested director...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6588#issuecomment-108009327 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31543107 --- Diff: core/src/test/scala/org/apache/spark/network/buffer/LargeByteBufferInputStreamSuite.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apach

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31543142 --- Diff: core/src/test/scala/org/apache/spark/network/buffer/LargeByteBufferInputStreamSuite.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed to the Apach

[GitHub] spark pull request: [SPARK-8032] [PySpark] Make version checking f...

2015-06-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/6579#discussion_r31543689 --- Diff: python/pyspark/mllib/__init__.py --- @@ -23,7 +23,14 @@ # MLlib currently needs NumPy 1.4+, so complain if lower import numpy -if

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31543772 --- Diff: network/common/src/main/java/org/apache/spark/network/buffer/LargeByteBufferHelper.java --- @@ -0,0 +1,90 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [Spark-7983] [MLlib] Add require for one-based...

2015-06-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/6538#issuecomment-108012118 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108012121 [Test build #869 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/869/consoleFull) for PR 6580 at commit [`0d52b89`](https://githu

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31543872 --- Diff: network/common/src/main/java/org/apache/spark/network/buffer/WrappedLargeByteBuffer.java --- @@ -0,0 +1,280 @@ +/* +* Licensed to the Apache

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31544111 --- Diff: network/common/src/main/java/org/apache/spark/network/buffer/WrappedLargeByteBuffer.java --- @@ -0,0 +1,280 @@ +/* +* Licensed to the Apache

[GitHub] spark pull request: [Spark-7983] [MLlib] Add require for one-based...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6538#issuecomment-108013062 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [Spark-7983] [MLlib] Add require for one-based...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6538#issuecomment-108013087 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-8038][SQL][PySpark] Fix when function o...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6580#issuecomment-108013191 [Test build #869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/869/consoleFull) for PR 6580 at commit [`0d52b89`](https://gith

[GitHub] spark pull request: [Spark-7983] [MLlib] Add require for one-based...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6538#issuecomment-108013554 [Test build #34000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/34000/consoleFull) for PR 6538 at commit [`20a2811`](https://gith

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/6589 [HOT-FIX] Add EvaluatedType back to RDG https://github.com/apache/spark/commit/87941ff8c49a6661f22c31aa7b84ac1fce768135 accidentally removed the EvaluatedType. You can merge this pull request into a

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6589#issuecomment-108014087 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6589#issuecomment-108014117 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/6589#issuecomment-108014083 I tested locally. Will merge it to branch 1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-8037] [SQL] Ignores files whose name st...

2015-06-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/6581#issuecomment-108014214 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [SPARK-8037] [SQL] Ignores files whose name st...

2015-06-02 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/6581#issuecomment-108014407 Merging to master and branch-1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6589#issuecomment-108014782 [Test build #34001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/34001/consoleFull) for PR 6589 at commit [`618c2eb`](https://gith

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31545220 --- Diff: core/src/main/scala/org/apache/spark/util/io/ByteArrayChunkOutputStream.scala --- @@ -91,4 +95,45 @@ class ByteArrayChunkOutputStream(chunkSize:

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31545270 --- Diff: core/src/main/scala/org/apache/spark/util/io/ByteArrayChunkOutputStream.scala --- @@ -91,4 +95,45 @@ class ByteArrayChunkOutputStream(chunkSize:

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31545490 --- Diff: network/common/src/test/java/org/apache/spark/network/buffer/LargeByteBufferHelperSuite.java --- @@ -0,0 +1,84 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request: [SPARK-8037] [SQL] Ignores files whose name st...

2015-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6581 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31545767 --- Diff: network/common/src/test/java/org/apache/spark/network/buffer/LargeByteBufferHelperSuite.java --- @@ -0,0 +1,84 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/5400#discussion_r31546071 --- Diff: network/common/src/test/java/org/apache/spark/network/buffer/LargeByteBufferHelperSuite.java --- @@ -0,0 +1,84 @@ +/* + * Licensed to the

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/6589#issuecomment-108020607 I have merged it into branch 1.4. I am closing it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [HOT-FIX] Add EvaluatedType back to RDG

2015-06-02 Thread yhuai
Github user yhuai closed the pull request at: https://github.com/apache/spark/pull/6589 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/6578#issuecomment-108020938 @pwendell but Python hasn't even run yet because it runs after the Scala streaming tests --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-8038] [SQL] [PYSPARK] fix Column.when()...

2015-06-02 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/6590 [SPARK-8038] [SQL] [PYSPARK] fix Column.when() and otherwise() Thanks @ogirardot, closes #6580 cc @rxin @JoshRosen You can merge this pull request into a Git repository by running: $

[GitHub] spark pull request: [SPARK-8010][SQL]Promote numeric types to stri...

2015-06-02 Thread OopsOutOfMemory
Github user OopsOutOfMemory commented on the pull request: https://github.com/apache/spark/pull/6551#issuecomment-108021959 @cloud-fan and @chenghao-intel. Updated, Any more comments? --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6578#issuecomment-108022324 [Test build #870 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/870/consoleFull) for PR 6578 at commit [`720d293`](https://githu

[GitHub] spark pull request: [SPARK-8038] [SQL] [PYSPARK] fix Column.when()...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6590#issuecomment-108022532 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6190][core] create LargeByteBuffer for ...

2015-06-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5400#issuecomment-108022625 Hey, one idea for avoiding tests that write huge files: can we use a data generator + streaming checksum function to be able to detect data corruption without having t

[GitHub] spark pull request: [SPARK-8038] [SQL] [PYSPARK] fix Column.when()...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6590#issuecomment-108022561 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8038] [SQL] [PYSPARK] fix Column.when()...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6590#issuecomment-108022747 [Test build #34002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/34002/consoleFull) for PR 6590 at commit [`c0f2069`](https://gith

[GitHub] spark pull request: [SPARK-8032] [PySpark] Make version checking f...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6579#issuecomment-108023111 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8032] [PySpark] Make version checking f...

2015-06-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6579#issuecomment-108023097 [Test build #33992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33992/consoleFull) for PR 6579 at commit [`22703dd`](https://git

[GitHub] spark pull request: [SPARK-8021] [SQL] [PYSPARK] make Python read/...

2015-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6578#issuecomment-108023314 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8032] [PySpark] Make version checking f...

2015-06-02 Thread MechCoder
Github user MechCoder commented on a diff in the pull request: https://github.com/apache/spark/pull/6579#discussion_r31547675 --- Diff: python/pyspark/mllib/__init__.py --- @@ -23,7 +23,14 @@ # MLlib currently needs NumPy 1.4+, so complain if lower import numpy -

<    1   2   3   4   5   6   7   8   9   10   >