[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68524/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68524/consoleFull)** for PR 15410 at commit [`46357ee`](https://github.com/apache/spark/commit/46357eedfbd36b152f5088627754cc3fa4b40570). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68524/consoleFull)** for PR 15410 at commit [`46357ee`](https://github.com/apache/spark/commit/46357eedfbd36b152f5088627754cc3fa4b40570). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 @vijoshi please look at test failures in more detail to see if these changes could have affected. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68471/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68471/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68471/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 I've seen other pr tests passing now, let try again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68409/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68409 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68409/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68409/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68357/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68357/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). * This patch **fails from timeout after a configured wait of \`250m\`**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68357/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68341/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68341/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #68341 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68341/consoleFull)** for PR 15410 at commit [`596c7ae`](https://github.com/apache/spark/commit/596c7aec2c282a50702829c5de97dafd318f2fd8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @tgravescs thanks, have updated the fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 couple minor spacing nits, otherwise looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @tgravescs @andrewor14 - any outstanding concerns with this change, please let me know? Can this merge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @ajbozarth @tgravescs @andrewor14 - updated the implementation to always display "Last Updated Date-Time" when we have atleast one completed scan of the event log directory. I have changed the impl slightly - the javascript for "last updated" goes into a "historypage-common.js" since it's now always needed for inclusion on the history page (earlier it was inside the bigger "historypage.js" file that gets included only if there are some apps to list) - if there are pending event logs, do not show the "Did you specify the correct logging directory?" message Have updated description with screenshots for each of the above scenarios. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15410 I would say just not displaying it would be fine, so making the `else if` an `if` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @ajbozarth yeah sounds useful. but note that for the very first load, we would have no 'Last Updated' value to display since that gets set only after the log scan+replay cycle completes at least once. any suggestions for an alternate text when we're in this situation? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15410 Should we display the Last Updated even if the currently processing message is displayed? Then the user will know the last update even when a new update is processing, rare case, but still useful. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @tgravescs Yes, the description for the pull request now has screen shots for the "Last Updated: XXX" case as well. I added description for the "Last Updated" message on the app list page just now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 were the description and screen shots updated? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 This needs to remain open - clicked the wrong button!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @tgravescs @ajbozarth @andrewor14 can this be merged now? can we get it into the 2.0 branch - should i simply open a pull request on 2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67070/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #67070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67070/consoleFull)** for PR 15410 at commit [`b43e241`](https://github.com/apache/spark/commit/b43e2412444f8da29f04ecab16a4955db1f8b35f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15410 **[Test build #67070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67070/consoleFull)** for PR 15410 at commit [`b43e241`](https://github.com/apache/spark/commit/b43e2412444f8da29f04ecab16a4955db1f8b35f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 can you update the description with latest implementation and screen shot. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @ajbozarth @tgravescs @andrewor14 - updated pull request to show a Last Updated message on the app list page when there are no pending logs being processed. I would have liked to have the ApplicationListResource use a json schema where the application list and associated metadata - updated date, pending logs etc could be included together. but given that the json schema returned today is just a collection of application info json - change to this would be too much code change for this smallish fix - probably something for the /api/v2/ to consider. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15410 @andrewor14 I plan to address that in another issue soon (see by end of the month) as soon as I have time. Fixing how the History Server loads has been coming up in a lot of PRs lately. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/15410 We shouldn't display file names but we should display application names and IDs, something the user understands. We don't have to do that as part of this issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 personally I would rather not display directory or filenames unless we are doing it right. User shouldn't know the internal implementation or storage option. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 *Display applications that are pending, instead of a count* Though agree that this would be better UX-wise, but we would only have the filenames (`logInfos`) to go by to build this list until a clean way to extract app metadata exists without parsing event log file contents. If the UI just displayed the pending ones is a different table as "names" of pending logs without asserting that they are necessarily the app-id (though presently the names may match the app-ids) - would that be good enough ? The message on the top could link to a new page listing pending logs to avoid cluttering the current page? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue:
https://github.com/apache/spark/pull/15410
1. display "Last updated " with a generic message, possibly
omitting the count of logs
Ok, while looking at this I notice that the scan date-time the
FsHistoryProvider currently internally captures `lastScanTime` is based on the
filesystem date-time:
`
private def getNewLastScanTime(): Long = {
val fileName = "." + UUID.randomUUID().toString
val path = new Path(logDir, fileName)
val fos = fs.create(path)
try {
fos.close()
fs.getFileStatus(path).getModificationTime
`
This probably made sense when earlier versions of FsHistoryProvider used
this value to determine new/updated logs it needed to replay. This is no longer
the case. In `checkForLogs()` the check is based on file size now. So my
question is that the fact that `getNewLastScanTime` is still present - is there
a reason to rely on or use the filesystem time value? We don't get a timezone
out of the `FileStatus` api. If I were to display an "Updated as of: " value
on the app listing page, I would rather read the value from the JVM and ensure
the Web UI converted it to user's local timezone for display.
2. Display applications that are pending, instead of a count
Though agree that this would be better UX-wise, but we would only have the
filenames (`logInfos`) to go by to build this list until a clean way to
extract app metadata exists without parsing event log file contents. If the UI
just displayed the pending ones is a different table as "names" of pending logs
without asserting that they are necessarily the app-id (though presently the
names may match the app-ids) - would that be good enough ? The message on the
top could link to a new page listing pending logs to avoid cluttering the
current page?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 Note the reason I mention the above things is I don't want us to start relying on the filename if its just going to change again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 If we are going to list the actual applications being loaded in the table then you have to rely on the filename to know the application id. This may be ok, but its then the interface and we can't change it or whatever we change it to has to be compatible and backwards compatible. Personally I would rather see us do something much smarter with file names/paths to include enough metadata to do simple table list or have metadata file that would allow loading of the history table much faster and load the actual contents only when actually clicked on. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/15410 ok to test I think the idea is good, but it would be a better UX if we display the pending applications as rows in the existing table (or a new one) and indicate there that it's still being processed. It might be more code change but I think it's worth it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15410 Just to raise an idea that would possibly mean less code change, would simply having a flag that causing a "currently processing applications" type message to display without an actual count with it? Overall I think this is a good addition though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 ah, yeah startup would definitely be a good case for this and like I mentioned its better then nothing so I'm ok with concept. I wonder for the other use case where it hasn't looked in ~ 10 seconds if it would be more clear to users if we put a little string at the top that is like "last updated time XX:XX:XX" or app list current as of XX:XX:XX. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15410 @tgravescs - you're right - for newer logs that are generated, there could be a window of time (10 secs or whatever the user configures) where the new logs are not picked up for replay and the UI doesn't say anything about them. however the issue we see is more with old completed apps. a little after history server startup, user browses to the app list and has no idea why the older completed apps are missing (perhaps those that were visible just before the history server was restarted). since the polling for logs/replay is scheduled immediately as part of history server startup (zero delay for the first round), this fix could help these cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15410 while I understand the problem, the issue here is that what you are printing might not be accurate either. We only pull for new logs every X number of seconds (default is 10s, user could change to longer) so if I come in one of those periods then it could say 0 when there are actually some to be read yet.Maybe this is ok as its better then it was but its also not guaranteed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15410: [SPARK-17843][Web UI] Indicate event logs pending for pr...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15410 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
