[GitHub] spark pull request #14949: [SPARK-17057] [ML] ProbabilisticClassifierModels'...

2016-09-12 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/14949 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14949: [SPARK-17057] [ML] ProbabilisticClassifierModels' predic...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14949 Oh, I get it now. That makes sense. If this were being applied to decision trees only, that would make sense and we could fix this up and document the meaning. I agree it only makes sense to return

[GitHub] spark issue #14678: [MINOR][SQL] Add missing functions for some options in S...

2016-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14678 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14678: [MINOR][SQL] Add missing functions for some options in S...

2016-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14678 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14949: [SPARK-17057] [ML] ProbabilisticClassifierModels' predic...

2016-09-12 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/14949 The original JIRA [SPARK-8069](https://issues.apache.org/jira/browse/SPARK-8069) refers to https://cran.r-project.org/web/packages/randomForest/randomForest.pdf. That R package calls it

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78362121 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-12 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/1 yes, I thought about it, this is a one-liner modification that canbe copy-pasted from the failure, I'll do it tonight. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78361404 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark issue #14444: [SPARK-16839] [SQL] redundant aliases after cleanupAlias...

2016-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/1 we can update the `spark/sql/hive/src/test/resources/sqlgen/subquery_in_having_2.sql` file directly --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78360879 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark pull request #14444: [SPARK-16839] [SQL] redundant aliases after clean...

2016-09-12 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r78360821 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -218,9 +221,41 @@ class AnalysisSuite extends

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78360606 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark pull request #14444: [SPARK-16839] [SQL] redundant aliases after clean...

2016-09-12 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r78359970 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala --- @@ -218,9 +221,41 @@ class AnalysisSuite extends

[GitHub] spark issue #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector based ...

2016-09-12 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/14597 hi @srowen , thanks. This is my first PR. learn much from you. thanks very much. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15024 @gatorsmile yea that's a good point. I have updated my PR to always set location for file-based data source table --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78358994 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78358827 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark issue #14597: [SPARK-17017][MLLIB][ML] add a chiSquare Selector based ...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14597 @mpjlu I'm still following this one, and I looked again today. I couldn't see anything else to change. For a minute I thought we need to update ChiSqSelectorModelReader / ChiSqSelectorModelWriter

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-09-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14118 I support this PR. But just to make sure, I'd like to bring a reference. It seems at least `na.strings` option in `read.csv` in R does as proposed here, ```r bt <- "A,B,C,D

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78358229 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78357482 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78357353 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78356812 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark issue #14452: [SPARK-16849][SQL][WIP] Improve subquery execution by de...

2016-09-12 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14452 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78356355 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78355993 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/12819 @yanboliang I will make usage of `weightCol` keeping in line with other algorithms, and I will do some performance test. Thanks for reviewing this. --- If your project is set up for it, you

[GitHub] spark issue #14949: [SPARK-17057] [ML] ProbabilisticClassifierModels' predic...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14949 The problem is that it's called 'threshold' and not 'weight', and 'threshold' means something different. Is anyone suggesting that it was always meant as a 'weight', and/or has a reference for this

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r7832 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark issue #14858: [SPARK-17219][ML] Add NaN value handling in Bucketizer

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14858 Great, I have one last request @VinceShieh and that is to update the docs for QuantileDiscretizer in Scala and Python to reflect the additional comment about NaN that you put in the main docs. That

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread mpjlu
Github user mpjlu commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78355383 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark issue #14949: [SPARK-17057] [ML] ProbabilisticClassifierModels' predic...

2016-09-12 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/14949 I think both this change and current design are reasonable. And I personally prefer to current one which treat threshould as a kind of weight. --- If your project is set up for it, you can

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78355162 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SPARK-15144][SQL] Make CSV ca...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14118 @lw-lin just checking that you think this is still good to go? @HyukjinKwon do you have an opinion on the current state? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15045: [Spark Core][MINOR] fix "default partitioner cann...

2016-09-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15045 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14863: [SPARK-16992][PYSPARK] use map comprehension in d...

2016-09-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14863 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15045: [Spark Core][MINOR] fix "default partitioner cannot part...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15045 OK, can't hurt to be clear and specific about this. Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14863: [SPARK-16992][PYSPARK] use map comprehension in doc

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14863 OK, this one's trivial in any event so I'm OK to merge this much. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15042: [SPARK-17449] [Documentation] [Relation between h...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15042#discussion_r78354202 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -32,6 +32,8 @@ import org.apache.spark.util._ * A heartbeat from executors

[GitHub] spark pull request #15005: [SPARK-17421] [DOCS] Documenting the current trea...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15005#discussion_r78353885 --- Diff: docs/building-spark.md --- @@ -16,24 +16,27 @@ Building Spark using Maven requires Maven 3.3.9 or newer and Java 7+. ### Setting up

[GitHub] spark pull request #15005: [SPARK-17421] [DOCS] Documenting the current trea...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15005#discussion_r78353932 --- Diff: docs/building-spark.md --- @@ -16,24 +16,31 @@ Building Spark using Maven requires Maven 3.3.9 or newer and Java 7+. ### Setting up

[GitHub] spark pull request #15005: [SPARK-17421] [DOCS] Documenting the current trea...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15005#discussion_r78353444 --- Diff: docs/building-spark.md --- @@ -16,24 +16,31 @@ Building Spark using Maven requires Maven 3.3.9 or newer and Java 7+. ### Setting up

[GitHub] spark pull request #15005: [SPARK-17421] [DOCS] Documenting the current trea...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15005#discussion_r78353504 --- Diff: docs/building-spark.md --- @@ -16,24 +16,31 @@ Building Spark using Maven requires Maven 3.3.9 or newer and Java 7+. ### Setting up

[GitHub] spark pull request #15039: [SPARK-17447] Performance improvement in Partitio...

2016-09-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15039 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15039: [SPARK-17447] Performance improvement in Partitioner.def...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15039 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassif...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15058#discussion_r78352961 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala --- @@ -43,10 +43,13 @@ import

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78352402 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78352242 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13762 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14969: [SPARK-17406][WEB UI] limit timeline executor events

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14969 Looking quite good. The code is significantly simpler after this change too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15058: [MLLIB]Add setBins for BinaryClassificationMetric...

2016-09-12 Thread mpjlu
GitHub user mpjlu opened a pull request: https://github.com/apache/spark/pull/15058 [MLLIB]Add setBins for BinaryClassificationMetrics ## What changes were proposed in this pull request? Add a setBins method for BinaryClassificationMetrics. BinaryClassificationMetrics

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78352044 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark issue #14640: [SPARK-17055] [MLLIB] add groupKFold to CrossValidator

2016-09-12 Thread finleyb
Github user finleyb commented on the issue: https://github.com/apache/spark/pull/14640 @VinceShieh I was wondering if require in the groupKFold method of MLUtils should be a greater than or equal rather than less than or equal? I was testing this branch because I need this

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78351884 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark issue #14737: [SPARK-17171][WEB UI] DAG will list all partitions in th...

2016-09-12 Thread cenyuhai
Github user cenyuhai commented on the issue: https://github.com/apache/spark/pull/14737 thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-12 Thread josh-howes
Github user josh-howes commented on the issue: https://github.com/apache/spark/pull/13762 I recently pushed a change to this but @SparkQA didn't kick off an automated build. Is that because I squashed my commits? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78351627 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark pull request #14737: [SPARK-17171][WEB UI] DAG will list all partition...

2016-09-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14737 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14969: [SPARK-17406][WEB UI] limit timeline executor eve...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14969#discussion_r78351548 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala --- @@ -38,47 +37,68 @@ private[ui] class ExecutorsTab(parent: SparkUI) extends

[GitHub] spark issue #14737: [SPARK-17171][WEB UI] DAG will list all partitions in th...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14737 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14737: [SPARK-17171][WEB UI] DAG will list all partitions in th...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14737 I looked one more time and the only thing that crossed my mind is whether spark.ui.dagGraph.retainedRootRDDs is the rightest naming convention, but, it's a hidden internal property at the moment,

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14949: [SPARK-17057] [ML] ProbabilisticClassifierModels' predic...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14949 Trying @holdenk or @mengxr maybe. I think this behavior should be changed because it doesn't match the common meaning of 'threshold', but I feel like I'm missing context about why it was done this

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65249/ Test PASSed. ---

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14426 **[Test build #65249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65249/consoleFull)** for PR 14426 at commit

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65250/ Test PASSed. ---

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #65250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65250/consoleFull)** for PR 14116 at commit

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65252/ Test PASSed. ---

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14623 **[Test build #65252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65252/consoleFull)** for PR 14623 at commit

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14995 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65247/ Test PASSed. ---

[GitHub] spark issue #14995: [Test Only][SPARK-6235][CORE]Address various 2G limits

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14995 **[Test build #65247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65247/consoleFull)** for PR 14995 at commit

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65248/ Test PASSed. ---

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #65248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65248/consoleFull)** for PR 14527 at commit

[GitHub] spark pull request #14644: [MESOS] Enable GPU support with Mesos

2016-09-12 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14644#discussion_r78348094 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -103,6 +103,7 @@ private[spark]

[GitHub] spark issue #15056: [SPARK-17503][Core] Fix memory leak in Memory store when...

2016-09-12 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15056 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14638 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65251/ Test PASSed. ---

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #65251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65251/consoleFull)** for PR 14638 at commit

[GitHub] spark issue #15056: [SPARK-17503][Core] Fix memory leak in Memory store when...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15056 Oh I get it now. That looks good, to my understanding. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15056: [SPARK-17503][Core] Fix memory leak in Memory store when...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15056 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15056: [SPARK-17503][Core] Fix memory leak in Memory store when...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65243/ Test PASSed. ---

[GitHub] spark issue #15056: [SPARK-17503][Core] Fix memory leak in Memory store when...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15056 **[Test build #65243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65243/consoleFull)** for PR 15056 at commit

[GitHub] spark issue #13513: [SPARK-15698][SQL][Streaming] Add the ability to remove ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13513 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65246/ Test PASSed. ---

[GitHub] spark issue #13513: [SPARK-15698][SQL][Streaming] Add the ability to remove ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13513 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13513: [SPARK-15698][SQL][Streaming] Add the ability to remove ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13513 **[Test build #65246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65246/consoleFull)** for PR 13513 at commit

[GitHub] spark issue #13513: [SPARK-15698][SQL][Streaming] Add the ability to remove ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13513 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65245/ Test PASSed. ---

[GitHub] spark issue #13513: [SPARK-15698][SQL][Streaming] Add the ability to remove ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13513 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13513: [SPARK-15698][SQL][Streaming] Add the ability to remove ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13513 **[Test build #65245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65245/consoleFull)** for PR 13513 at commit

[GitHub] spark issue #15057: [BUILD] Closing some stale PRs and ones suggested to be ...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15057 I looked at the list earlier and added mine to it, so I'm OK with these. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15057: [BUILD] Closing some stale PRs and ones suggested to be ...

2016-09-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15057 @srowen and @JoshRosen - Could you take a look please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15057: [BUILD] Closing some stale PRs and ones suggested...

2016-09-12 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15057 [BUILD] Closing some stale PRs and ones suggested to be closed by committer(s) ## What changes were proposed in this pull request? This PR proposes to close some stale PRs and ones

[GitHub] spark issue #15052: [SPARK-17500][PySpark]Make DiskBytesSpilled metric in Py...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15052 OK well if you see evidence later that the disk spilled bytes are unreasonably high, it's worth reinvestigating to see if there's a problem like this. If you aren't seeing bad metrics though, then

[GitHub] spark pull request #15052: [SPARK-17500][PySpark]Make DiskBytesSpilled metri...

2016-09-12 Thread djvulee
Github user djvulee closed the pull request at: https://github.com/apache/spark/pull/15052 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15052: [SPARK-17500][PySpark]Make DiskBytesSpilled metric in Py...

2016-09-12 Thread djvulee
Github user djvulee commented on the issue: https://github.com/apache/spark/pull/15052 @srowen Yes, the file seems always empty before write, so the origin way is OK. Sorry for this PR is not thoughtful enough, I just get a mislead by the other method in the shuffle.py, which used

[GitHub] spark issue #15053: [Doc] improve python API docstrings

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15053 Looks like the expected output doesn't match now. Maybe you all can teach me something -- how did this docstring work as a test at all before? or if it doesn't run correctly is the output not

[GitHub] spark pull request #14916: [SPARK-17340][YARN] cleanup .sparkStaging when ap...

2016-09-12 Thread Devian-ua
Github user Devian-ua closed the pull request at: https://github.com/apache/spark/pull/14916 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

<    1   2   3   4   5   6   >