[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2016-01-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-168317081 **[Test build #48567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48567/consoleFull)** for PR 9571 at commit [`06e5289`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-12-09 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-163358148 latest branch cut out health checks. They'd be nice, but as they are useful across all of spark, it's probably best to wait for something central to go in and pick

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-163357569 **[Test build #47440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47440/consoleFull)** for PR 9571 at commit [`2e04803`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160007597 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160007599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160007559 **[Test build #46781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46781/consoleFull)** for PR 9571 at commit [`1dcbb5f`](https://git

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160005832 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160005835 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160005802 **[Test build #46780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46780/consoleFull)** for PR 9571 at commit [`1dcbb5f`](https://git

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160005780 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160005781 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-160005758 **[Test build #46782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46782/consoleFull)** for PR 9571 at commit [`25e77bd`](https://git

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-16264 (Note that the POMs changed to pull in some more of the codahale servlets, though only the health checks & thread dump are being registered. Hooking up those servl

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159998497 **[Test build #46782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46782/consoleFull)** for PR 9571 at commit [`25e77bd`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159997461 **[Test build #46781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46781/consoleFull)** for PR 9571 at commit [`1dcbb5f`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159997153 **[Test build #46780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46780/consoleFull)** for PR 9571 at commit [`1dcbb5f`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159996648 jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159996634 Added reworked design * all metrics go through the `MetricsSystem`; the providers return an optional `Source` from the `start()` call. * `FsHistoryProv

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159996343 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159996339 **[Test build #46779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46779/consoleFull)** for PR 9571 at commit [`f6bf558`](https://git

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159996342 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-159996173 **[Test build #46779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46779/consoleFull)** for PR 9571 at commit [`f6bf558`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-25 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/9571#discussion_r45874984 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -678,6 +709,69 @@ private[history] class FsHistoryProvider(

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-25 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/9571#discussion_r45874777 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala --- @@ -121,20 +132,36 @@ class HistoryServer( /** * In

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-11 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155900033 ...I thought it did. If I'm wrong, then yes, all you need is a context. Once that's in there, the HistorySource should be what's needed --- If your project is set

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-11 Thread CharlesYeh
Github user CharlesYeh commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155863566 I'm assuming you didn't make a HistorySource in HistoryServer (what MasterSource in Master does) because it requires the MetricsSystem. Where does MetricsSyst

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-10 Thread CharlesYeh
Github user CharlesYeh commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155695451 Thanks! I'll look into this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155169298 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155165489 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155165525 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155153349 **[Test build #45386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45386/consoleFull)** for PR 9571 at commit [`cb7cddb`](https://git

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155153352 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155150767 **[Test build #45386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45386/consoleFull)** for PR 9571 at commit [`cb7cddb`](https://gith

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155148739 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9571#issuecomment-155148705 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11373] [CORE] WiP Add metrics to the Hi...

2015-11-09 Thread steveloughran
GitHub user steveloughran opened a pull request: https://github.com/apache/spark/pull/9571 [SPARK-11373] [CORE] WiP Add metrics to the History Server and providers This adds metrics to the history server, with the `FsHistoryProvider` metering its load, performance and reliabili