[GitHub] spark pull request: [SPARK-6954] [YARN] Dynamic allocation: numExe...

Github user sryza commented on the pull request: https://github.com/apache/spark/pull/5536#issuecomment-96334096 #5704 demonstrates what I outlined in my above comment, and should supersede this PR. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: SPARK-6954. ExecutorAllocationManager can end ...

GitHub user sryza opened a pull request: https://github.com/apache/spark/pull/5704 SPARK-6954. ExecutorAllocationManager can end up requesting a negative n... ...umber of executors You can merge this pull request into a Git repository by running: $ git pull https://github.com/s

[GitHub] spark pull request: [SPARK-6738] [CORE] Improve estimate the size ...

Github user shenh062326 commented on a diff in the pull request: https://github.com/apache/spark/pull/5608#discussion_r29107048 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -204,25 +204,36 @@ private[spark] object SizeEstimator extends Logging {

[GitHub] spark pull request: [SPARK-6505][SQL]Remove the reflection call in...

Github user baishuo commented on the pull request: https://github.com/apache/spark/pull/5660#issuecomment-96318903 can this patch be merged? :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user AiHe commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-9631 @srowen Sure. Pass the style checking and will do that for PRs in the feature. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-7120][SPARK-7121] Closure cleaner nesti...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5685#issuecomment-96315314 [Test build #709 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/709/consoleFull) for PR 5685 at commit [`4aab379`](https://github

[GitHub] spark pull request: [SPARK-7017][Build][Project Infra]: Refactor d...

Github user brennonyork commented on a diff in the pull request: https://github.com/apache/spark/pull/5694#discussion_r29105800 --- Diff: dev/run-tests --- @@ -17,239 +17,394 @@ # limitations under the License. # -# Go to the Spark project root directory -FWD

[GitHub] spark pull request: [SPARK-7145] [CORE] commons-lang (2.x) classes...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5703#issuecomment-96299091 [Test build #30955 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30955/consoleFull) for PR 5703 at commit [`21fbe03`](https://gith

[GitHub] spark pull request: [SPARK-6862][Streaming][WebUI] Add BatchPage t...

Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/5473#discussion_r29105467 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala --- @@ -40,6 +43,8 @@ private[streaming] class Streaming

[GitHub] spark pull request: [SPARK-6014] [core][hotfix] Add try-catch bloc...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5672 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-7120][SPARK-7121] Closure cleaner nesti...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5685#issuecomment-96297682 [Test build #708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/708/consoleFull) for PR 5685 at commit [`8b71cdb`](https://githu

[GitHub] spark pull request: [SPARK-6014] [core][hotfix] Add try-catch bloc...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5672#issuecomment-96297281 This test actually succeeded normally, just wasn't able to post back here. Going ahead. --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-7100][MLLib] Fix persisted RDD leak in ...

Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/5669#discussion_r29105356 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/GradientBoostedTrees.scala --- @@ -177,102 +177,108 @@ object GradientBoostedTrees extends Logging {

[GitHub] spark pull request: [SPARK-7120][SPARK-7121] Closure cleaner nesti...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5685#issuecomment-96295779 [Test build #708 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/708/consoleFull) for PR 5685 at commit [`8b71cdb`](https://github

[GitHub] spark pull request: [SPARK-7100][MLLib] Fix persisted RDD leak in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5669#issuecomment-96295778 [Test build #704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/704/consoleFull) for PR 5669 at commit [`e5be57c`](https://githu

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5697#discussion_r29105224 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -63,20 +63,27 @@ sealed trait Vector extends Serializable {

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5697#discussion_r29105222 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -556,6 +579,28 @@ class SparseVector( i += 1 }

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/5697#discussion_r29105223 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -63,20 +63,27 @@ sealed trait Vector extends Serializable {

[GitHub] spark pull request: [SPARK-6738] [CORE] Improve estimate the size ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5608#issuecomment-96289979 [Test build #702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/702/consoleFull) for PR 5608 at commit [`fe202a2`](https://githu

[GitHub] spark pull request: SPARK-7103: Fix crash with SparkContext.union ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5679#issuecomment-96288688 [Test build #705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/705/consoleFull) for PR 5679 at commit [`5a3d846`](https://githu

[GitHub] spark pull request: [SPARK-6629] cancelJobGroup() may not work for...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5288#issuecomment-96288336 [Test build #707 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/707/consoleFull) for PR 5288 at commit [`9e29654`](https://github

[GitHub] spark pull request: [SPARK-7145] [CORE] commons-lang (2.x) classes...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5703#issuecomment-96286649 [Test build #30955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30955/consoleFull) for PR 5703 at commit [`21fbe03`](https://githu

[GitHub] spark pull request: [SPARK-7092] Update spark scala version to 2.1...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5662 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6629] cancelJobGroup() may not work for...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5288#issuecomment-96286305 [Test build #706 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/706/consoleFull) for PR 5288 at commit [`9e29654`](https://github

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96286267 @AiHe this fails style checks. I think the lines are too long. Can you fix those up and try `./dev/lint-scala` to check it out locally? --- If your project is set up for

[GitHub] spark pull request: SPARK-7103: Fix crash with SparkContext.union ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5679#issuecomment-96286235 [Test build #705 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/705/consoleFull) for PR 5679 at commit [`5a3d846`](https://github

[GitHub] spark pull request: [SPARK-7100][MLLib] Fix persisted RDD leak in ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5669#issuecomment-96286212 [Test build #704 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/704/consoleFull) for PR 5669 at commit [`e5be57c`](https://github

[GitHub] spark pull request: [SPARK-7145] [CORE] commons-lang (2.x) classes...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5703#issuecomment-96286209 [Test build #703 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/703/consoleFull) for PR 5703 at commit [`21fbe03`](https://github

[GitHub] spark pull request: [SPARK-7100][MLLib] Fix persisted RDD leak in ...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5669#issuecomment-96286195 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96286191 [Test build #699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/699/consoleFull) for PR 5687 at commit [`b6380c6`](https://githu

[GitHub] spark pull request: [SPARK-6738] [CORE] Improve estimate the size ...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5608#issuecomment-96285979 [Test build #702 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/702/consoleFull) for PR 5608 at commit [`fe202a2`](https://github

[GitHub] spark pull request: [SPARK-6014] [core][hotfix] Add try-catch bloc...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5672#issuecomment-96285802 [Test build #701 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/701/consoleFull) for PR 5672 at commit [`0f1abd0`](https://github

[GitHub] spark pull request: [SPARK-7120][SPARK-7121] Closure cleaner nesti...

Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5685#issuecomment-96285701 @andrewor14 I manually triggered this using Josh's tool - looks like PRB is being tempermental. --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: [SPARK-7120][SPARK-7121] Closure cleaner nesti...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5685#issuecomment-96285685 [Test build #700 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/700/consoleFull) for PR 5685 at commit [`eb127e5`](https://github

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96282719 [Test build #699 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/699/consoleFull) for PR 5687 at commit [`b6380c6`](https://github

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96282678 LGTM pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-6888][SQL] Make the jdbc driver handlin...

Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/#issuecomment-96282583 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-7120][SPARK-7121] Closure cleaner nesti...

Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5685#issuecomment-96282586 Jenkins, test this please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-6113] [ml] Tree ensembles for Pipelines...

Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5626#issuecomment-96282516 @mengxr Curious: Why does it say there are unmerged commits? (I checked, and the last commit was merged correctly.) --- If your project is set up for it, you can rep

[GitHub] spark pull request: [SPARK-7145] [CORE] commons-lang (2.x) classes...

GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/5703 [SPARK-7145] [CORE] commons-lang (2.x) classes used instead of commons-lang3 (3.x); commons-io used without dependency Remove use of commons-lang in favor of commons-lang3 classes; remove comm

[GitHub] spark pull request: [SQL] Update SQL readme to include instruction...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5702 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SQL] Update SQL readme to include instruction...

Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5702#issuecomment-96275540 Thanks. Merged in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6856] [R] Make RDD information more use...

Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/5667#issuecomment-96275266 I'm out of town for the weekend. I can check Sunday night. On Apr 25, 2015 12:17 PM, "Shivaram Venkataraman" wrote: > @shaneknapp

[GitHub] spark pull request: [SQL] Update SQL readme to include instruction...

GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/5702 [SQL] Update SQL readme to include instructions on generating golden answer files based on Hive 0.13.1. You can merge this pull request into a Git repository by running: $ git pull https://gith

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/5697#discussion_r29103607 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -77,6 +77,11 @@ sealed trait Vector extends Serializable { // re

[GitHub] spark pull request: [SPARK-3376] Add in-memory shuffle option.

Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5403#issuecomment-96269096 Actually I lied - in the codebase we do have some flags we use only for performance analysis. One is "spark.shuffle.sync" which forces writes to sync to disk much more a

[GitHub] spark pull request: [SPARK-3376] Add in-memory shuffle option.

Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5403#issuecomment-96267909 My understanding was this wouldn't be an "experimental" feature in terms of how we've defined that in the past (i.e. it's not on a path to being something we'd expect pe

[GitHub] spark pull request: [SPARK-6113] [ml] Tree ensembles for Pipelines...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5626 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6113] [ml] Tree ensembles for Pipelines...

Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/5626#issuecomment-96267165 Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-6856] [R] Make RDD information more use...

Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5667#issuecomment-96264560 @shaneknapp Jenkins doesn't seem to picking up the re-test request. Is this related to the outages we had yesterday ? --- If your project is set up for it, you can repl

[GitHub] spark pull request: [SPARK-7143][ML] Add BM25 Estimator

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5701#issuecomment-96264124 [Test build #30954 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30954/consoleFull) for PR 5701 at commit [`8b3e724`](https://gith

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5430#issuecomment-96258683 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6856] [R] Make RDD information more use...

Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5667#issuecomment-96253660 Thanks for the update. I tried it out and change looks good. Waiting for Jenkins now --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-6856] [R] Make RDD information more use...

Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/5667#issuecomment-96253477 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96245388 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-7143][ML] Add BM25 Estimator

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5701#issuecomment-96239138 [Test build #30954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30954/consoleFull) for PR 5701 at commit [`8b3e724`](https://githu

[GitHub] spark pull request: [SPARK-6479][Block Manager]Create off-heap blo...

Github user zhzhan commented on the pull request: https://github.com/apache/spark/pull/5430#issuecomment-96237252 Jenkins, test it please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user AiHe commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96234324 Good to go. @jkbradley Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-5891][ML] Add Binarizer ML Transformer

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/5699#issuecomment-96234149 The failure is caused by an unrelated test in streaming. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-5891][ML] Add Binarizer ML Transformer

Github user viirya commented on the pull request: https://github.com/apache/spark/pull/5699#issuecomment-96234157 please retest. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-7143][ML] Add BM25 Estimator

GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/5701 [SPARK-7143][ML] Add BM25 Estimator JIRA: https://issues.apache.org/jira/browse/SPARK-7143 You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya

[GitHub] spark pull request: [SPARK-1442][SQL][WIP] Window Function Support...

Github user hbutani commented on the pull request: https://github.com/apache/spark/pull/5604#issuecomment-96220742 Hi, I spent some time on this patch. This is a good start. But there are several Semantic issues. And I have some comments/suggestions about the execution.

[GitHub] spark pull request: [SPARK-7142][SQL]: Minor enhancement to Boolea...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5700#issuecomment-96220491 [Test build #30953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30953/consoleFull) for PR 5700 at commit [`9e1f8dd`](https://gith

[GitHub] spark pull request: [SPARK-6888][SQL] Make the jdbc driver handlin...

Github user rtreffer commented on the pull request: https://github.com/apache/spark/pull/#issuecomment-96204329 @marmbrus I've fixed the linebreaks, but jenkins does not seem to pick up the changes and rerun the tests :-( What should I do? --- If your project is set up for it, yo

[GitHub] spark pull request: [SPARK-7109] [SQL] Push down left side filter ...

Github user scwf commented on the pull request: https://github.com/apache/spark/pull/5677#issuecomment-96203624 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-7142][SQL]: Minor enhancement to Boolea...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5700#issuecomment-96200802 [Test build #30953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30953/consoleFull) for PR 5700 at commit [`9e1f8dd`](https://githu

[GitHub] spark pull request: [SPARK-7142][SQL]: Minor enhancement to Boolea...

GitHub user saucam opened a pull request: https://github.com/apache/spark/pull/5700 [SPARK-7142][SQL]: Minor enhancement to BooleanSimplification Optimizer rule Use these in the optimizer as well: A and (not(A) or B) => A and B not(A and B) => not

[GitHub] spark pull request: [SPARK-3090] [core] Stop SparkContext if user ...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5696#issuecomment-96195518 LGTM. I dislike the proliferation of shutdown hooks but we have a better centralized system for it now, and this one is fairly important. Let me leave it for comments for

[GitHub] spark pull request: SPARK-7103: Fix crash with SparkContext.union ...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5679#issuecomment-96195182 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-7086][Deploy]Do not retry when public s...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5657#issuecomment-96194910 This is related to https://github.com/apache/spark/pull/5575. I am not sure this is something we must force on users. There could be decent reasons to retry binding, and i

[GitHub] spark pull request: update the deprecated CountMinSketchMonoid fun...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5629 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6738] [CORE] Improve estimate the size ...

Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/5608#discussion_r29100309 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -204,25 +204,36 @@ private[spark] object SizeEstimator extends Logging {

[GitHub] spark pull request: [SPARK-6738] [CORE] Improve estimate the size ...

Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/5608#discussion_r29100295 --- Diff: core/src/main/scala/org/apache/spark/util/SizeEstimator.scala --- @@ -204,25 +204,36 @@ private[spark] object SizeEstimator extends Logging {

[GitHub] spark pull request: [SPARK-6014] [core][hotfix] Add try-catch bloc...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5672#issuecomment-96191936 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-6014] [core][hotfix] Add try-catch bloc...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5672#issuecomment-96191749 @nishkamravi2 this isn't the same method or exception handling as before. The question is what's right to do given the current state of the code. It feels a little arbitra

[GitHub] spark pull request: [Minor][MLLIB] Refactor toString method in MLL...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5687#issuecomment-96191595 Looks OK, I think Joseph just had one more tiny comment on one `toString` method, to add a space. --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: [SPARK-5891][ML] Add Binarizer ML Transformer

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5699#issuecomment-96189797 [Test build #30952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30952/consoleFull) for PR 5699 at commit [`1682f8c`](https://gith

[GitHub] spark pull request: [SPARK-5891][ML] Add Binarizer ML Transformer

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5699#issuecomment-96176516 [Test build #30952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30952/consoleFull) for PR 5699 at commit [`1682f8c`](https://githu

[GitHub] spark pull request: [SPARK-2750][WIP]Add Https support for Web UI

Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5664#issuecomment-96175514 I have tested https/http on Master/Worker/DriverUI/HistoryServer. All are fine but it didn't work on yarn mode, the exception is as: https://issues.apache.org/jir

[GitHub] spark pull request: [SPARK-6113] [ml] Tree ensembles for Pipelines...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5626#issuecomment-96174080 [Test build #698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/698/consoleFull) for PR 5626 at commit [`729167a`](https://githu

[GitHub] spark pull request: [SPARK-5891][ML] Add Binarizer ML Transformer

GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/5699 [SPARK-5891][ML] Add Binarizer ML Transformer JIRA: https://issues.apache.org/jira/browse/SPARK-5891 You can merge this pull request into a Git repository by running: $ git pull https://gi

[GitHub] spark pull request: [SPARK-1406] Mllib pmml model export

Github user selvinsource commented on the pull request: https://github.com/apache/spark/pull/3062#issuecomment-96157430 @mengxr 3) I have done point 3, if it is a multinomial logistic regression the export will cause an IllegalArgumentException. I didn't realize now Logistic s

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user ogirardot commented on the pull request: https://github.com/apache/spark/pull/5698#issuecomment-96157353 is this the job now ? https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/ triggered by https://spark-prs.appspot.com/ ? --- If your project is

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user ogirardot commented on the pull request: https://github.com/apache/spark/pull/5698#issuecomment-96157282 Somehow Jenkins doesn't seem to be build this PR : https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/ --- If your project is set up for it, you can re

[GitHub] spark pull request: [SPARK-6113] [ml] Tree ensembles for Pipelines...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5626#issuecomment-96157201 [Test build #698 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/698/consoleFull) for PR 5626 at commit [`729167a`](https://github

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5697#issuecomment-96150282 [Test build #30951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30951/consoleFull) for PR 5697 at commit [`1ebad60`](https://gith

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5698#issuecomment-96146294 LGTM also. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5698#issuecomment-96146286 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

GitHub user ogirardot opened a pull request: https://github.com/apache/spark/pull/5698 [SPARK-7118] [Python] Add the coalesce Spark SQL function available in PySpark This patch adds a proxy call from PySpark to the Spark SQL coalesce function and this patch comes out of a discussio

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user ogirardot closed the pull request at: https://github.com/apache/spark/pull/5683 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5683#issuecomment-96145908 Jenkins really doesn't like coalesce. Maybe close this one and open a new one? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-7017][Build][Project Infra]: Refactor d...

Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5694#issuecomment-96145897 Just a naming suggestion - would be better to have an explicit .py file, and the shell file just calls that. --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-7118] [Python] Add the coalesce Spark S...

Github user ogirardot commented on the pull request: https://github.com/apache/spark/pull/5683#issuecomment-96145809 Jenkins, pretty please :) ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...

Github user Sephiroth-Lin commented on the pull request: https://github.com/apache/spark/pull/5478#issuecomment-96145643 @andrewor14 @sryza how about your opinions? thanks. @lianhuiwang please help me review this, thanks. --- If your project is set up for it, you can reply to this e

[GitHub] spark pull request: SPARK-6333 [CORE] Added compression option to ...

Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5030#issuecomment-96145516 Thanks for pinging this - yeah this is fine to do, just add a MIMA exception. We allow additions to JavaRDDLike trait, since the expectation is that trait will not be im

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5697#discussion_r29098649 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -77,6 +77,11 @@ sealed trait Vector extends Serializable { // refe

[GitHub] spark pull request: [SPARK-7140][MLLIB] only scan the first 16 non...

Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5697#discussion_r29098644 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala --- @@ -63,13 +63,13 @@ sealed trait Vector extends Serializable { Ve

[GitHub] spark pull request: SPARK-6333 [CORE] Added compression option to ...

Github user deenar commented on the pull request: https://github.com/apache/spark/pull/5030#issuecomment-96145043 Hi Sean Do I need to do something to get @pwendell and/or @mateiz look into this? Deenar

[GitHub] spark pull request: [SPARK-5253] [ML] LinearRegression with L1/L2 ...

Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/4259#discussion_r29098568 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -42,34 +50,122 @@ private[regression] trait LinearRegressionParams