[GitHub] spark pull request #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to regis...

2016-10-07 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9766#discussion_r82450939 --- Diff: python/pyspark/sql/context.py --- @@ -202,6 +202,10 @@ def registerFunction(self, name, f, returnType=StringType()): """

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14963 I think this could be a good change for allowing more developers to onboard with PySpark - is there any interest in the current PySpark/Build focused committers [ @davies @srowen @rxin ] in seeing

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #66506 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66506/consoleFull)** for PR 15377 at commit

[GitHub] spark pull request #15393: [HOTFIX][BUILD] Do not use contains in Option in ...

2016-10-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15393#discussion_r82450501 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala --- @@ -70,7 +70,7 @@ class

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15239#discussion_r82450472 --- Diff: R/pkg/R/SQLContext.R --- @@ -341,11 +342,13 @@ setMethod("toDF", signature(x = "RDD"), #' @name read.json #' @method read.json

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2016-10-07 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/11601 Thanks for the comments @MLnick @jkbradley @sethah I have sent update according to the comments and change `ImputerModel.surrogate` and persistence format into DataFrame. As for the

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66507/ Test PASSed. ---

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/9766 Maybe @marmbrus could take a look if @davies is busy? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66521/consoleFull)** for PR 15393 at commit

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #66507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66507/consoleFull)** for PR 15377 at commit

[GitHub] spark issue #14653: [SPARK-10931][PYSPARK][ML] PySpark ML Models should cont...

2016-10-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14653 Huh I'm not sure why jenkins isn't picking this up - @jkbradley or @davidnavas can you tell jenkins this is ok to test again? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #15393: [HOTFIX][BUILD] Do not use contains in Option in ...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15393#discussion_r82449644 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala --- @@ -70,7 +70,7 @@ class

[GitHub] spark issue #15355: [SPARK-17782][STREAMING][BUILD] Add Kafka 0.10 project t...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15355 Also check-picked this one into branch 2.0 since it's also helpful for 2.0 backport PRs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #15393: [HOTFIX][BUILD] Do not use contains in Option in ...

2016-10-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15393#discussion_r82449290 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala --- @@ -70,7 +70,7 @@ class

[GitHub] spark pull request #15393: [HOTFIX][BUILD] Do not use contains in Option in ...

2016-10-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15393#discussion_r82449361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala --- @@ -70,7 +70,7 @@ class

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15384 **[Test build #66520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66520/consoleFull)** for PR 15384 at commit

[GitHub] spark issue #15384: [SPARK-17346][SQL][Tests]Fix the flaky topic deletion in...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15384 Since deleting a normal topic may be timeout as well:

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15365 It would be great to get some feedback on the name `spark.logit` What do folks think about it? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #15365: [SPARK-17157][SPARKR]: Add multiclass logistic re...

2016-10-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15365#discussion_r82448237 --- Diff: R/pkg/R/mllib.R --- @@ -647,6 +654,195 @@ setMethod("predict", signature(object = "KMeansModel"), predict_internal(object,

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15375 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66517/ Test FAILed. ---

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66517/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15375 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15393 **[Test build #66519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66519/consoleFull)** for PR 15393 at commit

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15263 Oh, thank you for pointing this out. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15393: [HOTFIX][BUILD] Do not use contains in Option in JdbcRel...

2016-10-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15393 @yhuai @zsxwing Do you mind if I ask to take a look please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15148 Related to the docs, some more comments defining terminology would be useful for non-experts: * OR-amplification * probing buckets * false positives/negatives (w.r.t. finding nearest

[GitHub] spark pull request #15393: [HOTFIX][BUILD] Do not use contains in Option in ...

2016-10-07 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15393 [HOTFIX][BUILD] Do not use contains in Option in JdbcRelationProvider ## What changes were proposed in this pull request? This PR proposes the fix the use of `contains` API which only

[GitHub] spark issue #15375: [SPARK-17790] Support for parallelizing R data.frame lar...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15375 **[Test build #66517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66517/consoleFull)** for PR 15375 at commit

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15366 **[Test build #66518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66518/consoleFull)** for PR 15366 at commit

[GitHub] spark pull request #15367: [SPARK-17346][SQL][test-maven]Add Kafka source fo...

2016-10-07 Thread zsxwing
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/15367 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14561: [SPARK-16972][CORE] Move DriverEndpoint out of CoarseGra...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14561 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15375: [SPARK-17790] Support for parallelizing R data.fr...

2016-10-07 Thread falaki
Github user falaki commented on a diff in the pull request: https://github.com/apache/spark/pull/15375#discussion_r82446808 --- Diff: R/pkg/R/context.R --- @@ -123,19 +126,48 @@ parallelize <- function(sc, coll, numSlices = 1) { if (numSlices > length(coll))

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15366 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445466 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445905 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/LSHTest.scala --- @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445623 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445705 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445915 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/LSHTest.scala --- @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445482 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445651 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445670 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445698 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445744 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445909 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/LSHTest.scala --- @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445506 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445715 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445919 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/LSHTest.scala --- @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445897 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445477 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/MinHash.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445726 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445890 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #11601: [SPARK-13568] [ML] Create feature transformer to impute ...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11601 **[Test build #66516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66516/consoleFull)** for PR 11601 at commit

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15239 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/12004 # Packaging: 1. this addresses the problem that it's not always immediately obvious to people what they have to do to get, say s3a working. Do you know precisely which version of

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445385 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15239: [SPARK-17665][SPARKR] Support options/mode all fo...

2016-10-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15239#discussion_r82445435 --- Diff: R/pkg/R/SQLContext.R --- @@ -341,11 +342,13 @@ setMethod("toDF", signature(x = "RDD"), #' @name read.json #' @method read.json

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-07 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82445404 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15366 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66508/ Test FAILed. ---

[GitHub] spark issue #15367: [SPARK-17346][SQL][test-maven]Add Kafka source for Struc...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15367 Thanks! I'm going to merge this one since the concern from @koeninger is addressed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15366 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15366: [SPARK-17793] [Web UI] Sorting on the description on the...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15366 **[Test build #66508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66508/consoleFull)** for PR 15366 at commit

[GitHub] spark pull request #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15389#discussion_r82443930 --- Diff: python/pyspark/rdd.py --- @@ -2029,7 +2030,11 @@ def coalesce(self, numPartitions, shuffle=False): >>> sc.parallelize([1, 2, 3, 4, 5],

[GitHub] spark pull request #15389: [SPARK-17817][PySpark] PySpark RDD Repartitioning...

2016-10-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15389#discussion_r82444378 --- Diff: python/pyspark/rdd.py --- @@ -2029,7 +2030,11 @@ def coalesce(self, numPartitions, shuffle=False): >>> sc.parallelize([1, 2, 3, 4, 5],

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

2016-10-07 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15354#discussion_r82443073 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -343,4 +343,23 @@ class

[GitHub] spark pull request #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Str...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14087 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15263 Option.contains is only in Scala 2.11... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15263 Seems it breaks scala 2.10 compilation. Can you take a look? Thanks! ``` [error]

[GitHub] spark issue #14087: [SPARK-16411][SQL][STREAMING] Add textFile to Structured...

2016-10-07 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/14087 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14426 **[Test build #66515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66515/consoleFull)** for PR 14426 at commit

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-10-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14426 Hi, @gatorsmile . Could you review this PR when you have some time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15377: [SPARK-17802] Improved caller context logging.

2016-10-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/15377#discussion_r82441129 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2474,25 +2474,36 @@ private[spark] class CallerContext( val context =

[GitHub] spark issue #15386: [SPARK-17808][PYSPARK] Upgraded version of Pyrolite to 4...

2016-10-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15386 Yeah I figured you'd need to run `./dev/test-dependencies.sh --replace-manifest` here --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15370 Ah you are right, sorry I totally missed that this is purely a sorting problem. I was thinking the %05d was causing an issue but it doesn't. --- If your project is set up for it, you can

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15370 **[Test build #66514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66514/consoleFull)** for PR 15370 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/12004 @steveloughran can you clarify what this does? It seems like just 5000 lines of examples and test cases? Users can already use these cloud stores by just adding the proper dependencies, can't they?

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15390 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/15370 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15370: [SPARK-17417][Core] Fix # of partitions for Reliable RDD...

2016-10-07 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/15370 All tests passed. Error unrelated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12004 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66513/ Test FAILed. ---

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #66513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66513/consoleFull)** for PR 12004 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12004 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelationProvid...

2016-10-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15263 Thanks! Merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15263: [SPARK-14525][SQL][FOLLOWUP] Clean up JdbcRelatio...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15263 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15392: [SPARK-17830] Annotate spark.sql package with InterfaceS...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15392 **[Test build #66511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66511/consoleFull)** for PR 15392 at commit

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #66513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66513/consoleFull)** for PR 12004 at commit

[GitHub] spark issue #15391: [MINOR][ML]:remove redundant comment in LogisticRegressi...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15391 **[Test build #66512 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66512/consoleFull)** for PR 15391 at commit

[GitHub] spark issue #15392: [SPARK-17830] Annotate spark.sql package with InterfaceS...

2016-10-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15392 cc @marmbrus want to review this? It is pretty important to make sure the APIs are properly annotated. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15392: [SPARK-17830] Annotat spark.sql package with Inte...

2016-10-07 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15392 [SPARK-17830] Annotat spark.sql package with InterfaceStability ## What changes were proposed in this pull request? This patch annotates the InterfaceStability level for top level classes in

[GitHub] spark pull request #15391: [MINOR][ML]:remove redundant comment in LogisticR...

2016-10-07 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/15391 [MINOR][ML]:remove redundant comment in LogisticRegression ## What changes were proposed in this pull request? While adding R wrapper for LogisticRegression, I found one extra comment. It

[GitHub] spark issue #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submit appli...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15381 FYI, I fixed import conflicts in JettyUtils manually for branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15374: [SPARK-17800] Introduce InterfaceStability annota...

2016-10-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15374#discussion_r82433404 --- Diff: common/tags/src/main/java/org/apache/spark/annotation/InterfaceStability.java --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submi...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15381 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15381: [SPARK-17707] [WEBUI] Web UI prevents spark-submit appli...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15381 LGTM. Merging to master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15390 **[Test build #66510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66510/consoleFull)** for PR 15390 at commit

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-07 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/15218 @mridulm Thanks for the comments. Your concern regarding the locality is right. The patch does not change this behavior, which takes priority of locality preference. But if multiple executors

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15390 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15390: [SPARK-17806] [SQL] fix bug in join key rewritten...

2016-10-07 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/15390 [SPARK-17806] [SQL] fix bug in join key rewritten in HashJoin ## What changes were proposed in this pull request? In HashJoin, we try to rewrite the join key as Long to improve the

[GitHub] spark pull request #15374: [SPARK-17800] Introduce InterfaceStability annota...

2016-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15374 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15365: [SPARK-17157][SPARKR]: Add multiclass logistic regressio...

2016-10-07 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15365 cc @sethah @yanboliang --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

<    1   2   3   4   5   6   7   >