[GitHub] spark issue #19424: [SPARK-22197][SQL] push down operators to data source be...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19424 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19424: [SPARK-22197][SQL] push down operators to data source be...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19424 **[Test build #82547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82547/testReport)** for PR 19424 at commit

[GitHub] spark issue #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-09 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19437 @susanxhuynh this a mesos 1.4 feature, shouldn't we document this for users? https://issues.apache.org/jira/browse/MESOS-7418 ---

[GitHub] spark issue #19106: [SPARK-21770][ML] ProbabilisticClassificationModel fix c...

2017-10-09 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19106 @srowen Any other comments? Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19419: [SPARK-22188] [CORE] Adding security headers for ...

2017-10-09 Thread krishna-pandey
Github user krishna-pandey commented on a diff in the pull request: https://github.com/apache/spark/pull/19419#discussion_r143427428 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -89,6 +92,9 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark pull request #15770: [SPARK-15784][ML]:Add Power Iteration Clustering ...

2017-10-09 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/15770#discussion_r143426157 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/PowerIterationClustering.scala --- @@ -0,0 +1,216 @@ +/* + * Licensed to the

[GitHub] spark issue #19433: [SPARK-3162] [MLlib][WIP] Add local tree training for de...

2017-10-09 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19433 @smurching Does it still WIP ? If done remove "[WIP]", I will begin review, thanks! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19266: [SPARK-22033][CORE] BufferHolder, other size chec...

2017-10-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19266#discussion_r143421894 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -35,6 +35,11 @@ * if the fields of

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-09 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r143415419 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -170,9 +174,122 @@

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-09 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r143413006 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/config.scala --- @@ -21,6 +21,39 @@ import java.util.concurrent.TimeUnit

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-09 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r143411366 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -170,9 +174,122 @@

[GitHub] spark issue #19363: [Minor]Override toString of KeyValueGroupedDataset

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19363 **[Test build #82548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82548/testReport)** for PR 19363 at commit

[GitHub] spark issue #19363: [Minor]Override toString of KeyValueGroupedDataset

2017-10-09 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19363 Ok,i will add a jira tgt and fix RelationalGroupedDataset --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #18748: [SPARK-20679][ML] Support recommending for a subs...

2017-10-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18748 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19082 Yes no need for a back-and-forth. @gatorsmile I think it's reasonable to ask for a little more detail on your comment. --- - To

[GitHub] spark pull request #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, whil...

2017-10-09 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/17357#discussion_r143407147 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/DriverWrapper.scala --- @@ -23,14 +23,15 @@ import org.apache.commons.lang3.StringUtils

[GitHub] spark pull request #19438: [SPARK-22208] [SQL] Improve percentile_approx by ...

2017-10-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19438#discussion_r143406940 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameStatSuite.scala --- @@ -157,21 +157,21 @@ class DataFrameStatSuite extends QueryTest with

[GitHub] spark pull request #19438: [SPARK-22208] [SQL] Improve percentile_approx by ...

2017-10-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19438#discussion_r143406835 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/QuantileSummariesSuite.scala --- @@ -58,7 +58,7 @@ class QuantileSummariesSuite

[GitHub] spark issue #19424: [SPARK-22197][SQL] push down operators to data source be...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19424 **[Test build #82547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82547/testReport)** for PR 19424 at commit

[GitHub] spark issue #19424: [SPARK-22197][SQL] push down operators to data source be...

2017-10-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19424 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 @gatorsmile If my words make you upset, I'm sorry. It's you right to raise suspicion against any PRs. I do respect this right. Maybe I'm wrong and there actually is a possible regression.

[GitHub] spark issue #19363: [Minor]Override toString of KeyValueGroupedDataset

2017-10-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19363 shall we fix RelationalGroupedDataset too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19433: [SPARK-3162] [MLlib][WIP] Add local tree training...

2017-10-09 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19433#discussion_r143398990 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/LocalDecisionTreeUtils.scala --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 OK. Based on your attitudes, I do not care it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19420: [SPARK-22191] [SQL] Add hive serde example with s...

2017-10-09 Thread crlalam
Github user crlalam commented on a diff in the pull request: https://github.com/apache/spark/pull/19420#discussion_r143118902 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/hive/JavaSparkHiveExample.java --- @@ -124,6 +124,13 @@ public static void main(String[]

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 If you really can't or don't willing to explain, we can wait for @kiszk or @rednaxelafx to explain it. --- - To unsubscribe,

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 @gatorsmile You can't just drop few words without explanation and simply say that others works don't work. I do respect your work/comments and you should respect my works too. ---

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 Just try to help you understand it. If you don't want it, I can keep quiet. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 No. I think you should explain it because this is your concern. You just need to point out, where is the single blocking loop for the two operators? ---

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 Sorry, it is really hard to explain it to you. Maybe you can first think about why the whole-stage codegen works better the previous solution? ---

[GitHub] spark issue #19145: [spark-21933][yarn] Spark Streaming request more executo...

2017-10-09 Thread klion26
Github user klion26 commented on the issue: https://github.com/apache/spark/pull/19145 sorry for the late response, IIUC, in MR, this case handled by below 1. AM received the container failed message 2. AM will check whether there are any attempts of the same task is RUNNING

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 > After horizontal/ring cutting, each operator will be in a single blocking loop. That means, whole-stage codegen is off. And do we have anything to cut the operators to individual single

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 Yeah, once we exceed `hugeMethodLimit`, wholestage codegen is disabled. Can you tell me how `hugeMethodLimit` makes each operator to a single blocking loop? We didn't run codegen now. ---

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-10-09 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/19287 Thanks all reviewers! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 The relation is very clear. Let me copy what I said above > Just imagine we have two nodes, we want to do a horizontal/ring cut. Basically, in this scenario, horizontal/ring cutting

[GitHub] spark issue #19399: [SPARK-22175][WEB-UI] Add status column to history page

2017-10-09 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/19399 Nice, I think it should be merged. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19444: [SPARK-22214][SQL] Refactor the list hive partiti...

2017-10-09 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19444#discussion_r143390226 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -405,6 +405,11 @@ object CatalogTypes { *

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-10-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19287 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-10-09 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19287 LGTM, merging to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 Ok. Any comments from @kiszk and @rednaxelafx on this topic? :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82546/ Test PASSed. ---

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19287 **[Test build #82546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82546/testReport)** for PR 19287 at commit

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19082 Nope. You can wait for the answers from the JVM experts. : ) --- - To unsubscribe, e-mail:

[GitHub] spark issue #19082: [SPARK-21870][SQL] Split aggregation code into small fun...

2017-10-09 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19082 @gatorsmile Are you talking about this: https://en.wikipedia.org/wiki/Loop_nest_optimization? --- - To unsubscribe,

<    1   2   3   4