[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-05-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r62410489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLMetrics.scala --- @@ -19,200 +19,106 @@ package org.apache.spark.sql.execution.

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-05-06 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r62369241 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLMetrics.scala --- @@ -19,200 +19,106 @@ package org.apache.spark.sql.execution.met

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61658910 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-29 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61655071 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-29 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215851284 Created https://issues.apache.org/jira/browse/SPARK-15010 for the reported issue. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-29 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215839576 Shall we revert this commit? Got some similar errors: ~~~ 16/04/29 11:25:45 ERROR Utils: Uncaught exception in thread heartbeat-receiver-event-loop-thre

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61419435 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61414486 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12612 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215334099 Merging in master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215332700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215332701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215332517 **[Test build #57220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57220/consoleFull)** for PR 12612 at commit [`124568b`](https://g

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61374828 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -175,124 +172,143 @@ class TaskMetrics private[spark] () extends Serializ

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61374697 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,391 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215312526 **[Test build #57220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57220/consoleFull)** for PR 12612 at commit [`124568b`](https://gi

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371630 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregateExec.scala --- @@ -46,7 +46,7 @@ case class SortBasedAggregateExec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371318 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregateExec.scala --- @@ -46,7 +46,7 @@ case class SortBasedAggregat

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215304306 This looks pretty good to me. We should get it to pass tests and then merge it asap. Some of the comments can be addressed later. --- If your project is set up for it, y

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371102 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/SortBasedAggregateExec.scala --- @@ -46,7 +46,7 @@ case class SortBasedAggregateExec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61371062 --- Diff: project/MimaExcludes.scala --- @@ -674,6 +674,19 @@ object MimaExcludes { ) ++ Seq( // [SPARK-4452][Core]Shuffle data structure

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61370631 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61370129 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -175,124 +172,143 @@ class TaskMetrics private[spark] () extends Serializable

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61369538 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61366808 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61366766 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61366618 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12612#discussion_r61366592 --- Diff: core/src/main/scala/org/apache/spark/NewAccumulator.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215168119 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215168116 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215167898 **[Test build #57135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57135/consoleFull)** for PR 12612 at commit [`c74320d`](https://g

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215125236 **[Test build #57135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57135/consoleFull)** for PR 12612 at commit [`c74320d`](https://gi

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215114675 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215114644 **[Test build #57132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57132/consoleFull)** for PR 12612 at commit [`9c0`](https://g

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215114672 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215112246 **[Test build #57132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57132/consoleFull)** for PR 12612 at commit [`9c0`](https://gi

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215102784 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215102790 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215102766 **[Test build #57130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57130/consoleFull)** for PR 12612 at commit [`be8ff0e`](https://g

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215101163 **[Test build #57130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57130/consoleFull)** for PR 12612 at commit [`be8ff0e`](https://gi