[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178285540 Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10860 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178273616 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50503/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178273615 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178273402 **[Test build #50503 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50503/consoleFull)** for PR 10860 at commit [`f829ac7`](https://github.com/apache/spark/commit/f829ac7f7cfb0b9caffef64242826ed314cd7da6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178269135 LGTM, I can't wait to merge this!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178220256 **[Test build #50503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50503/consoleFull)** for PR 10860 at commit [`f829ac7`](https://github.com/apache/spark/commit/f829ac7f7cfb0b9caffef64242826ed314cd7da6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178169396 I see, that seems unnecessary. We deleted all the code path that would parse the format so it's pretty much impossible for it to be picked up right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178166829 sure - that was just to make sure it doesn't get picked up, as a negative test. I can purge all of that since it seems both you and @vanzin have the same feedback? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178166468 @felixcheung this looks great! I'm excited about how many lines of code we're deleting in this patch. I had one question about the tests but this is good to go otherwise. In particular I don't think we should keep around event logs in the old format; if there is still a consumer then we should just remove that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10860#discussion_r51472679 --- Diff: .rat-excludes --- @@ -63,14 +63,15 @@ logs .*dependency-reduced-pom.xml known_translations json_expectation -local-1422981759269/* -local-1422981780767/* -local-1425081759269/* -local-1426533911241/* -local-1426633911242/* -local-1430917381534/* +local-1422981759269 +local-1422981780767 +local-1425081759269 +local-1426533911241 +local-1426633911242 +local-1430917381534 local-1430917381535_1 local-1430917381535_2 +local-1425081759268/* --- End diff -- what does this mean? Shouldn't we just remove this line and delete the old logs that we're not even using anymore? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r51472337
--- Diff:
core/src/test/resources/spark-events/local-1425081759268/EVENT_LOG_1 ---
@@ -0,0 +1,88 @@
+{"Event":"SparkListenerBlockManagerAdded","Block Manager ID":{"Executor
ID":"","Host":"localhost","Port":57967},"Maximum
Memory":278302556,"Timestamp":1425081759407}
--- End diff --
I might be missing something, why do we still have test files using the old
log format? Shouldn't we just replace all old test logs using the new format?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-178159771 Could we merge this please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-177062676 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50427/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-177062675 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-177062624 **[Test build #50427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50427/consoleFull)** for PR 10860 at commit [`b2d514f`](https://github.com/apache/spark/commit/b2d514fde0a638b22b3e89b452a54e5c74702702). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-177046250 **[Test build #50427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50427/consoleFull)** for PR 10860 at commit [`b2d514f`](https://github.com/apache/spark/commit/b2d514fde0a638b22b3e89b452a54e5c74702702). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176969146 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176969148 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50398/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176968992 **[Test build #50398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50398/consoleFull)** for PR 10860 at commit [`772f045`](https://github.com/apache/spark/commit/772f045865462cb419bfc402c9d50e559934fc89). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176930477 **[Test build #50398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50398/consoleFull)** for PR 10860 at commit [`772f045`](https://github.com/apache/spark/commit/772f045865462cb419bfc402c9d50e559934fc89). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176927504 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r51304370
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -261,8 +259,8 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
}
.flatMap { entry => Some(entry) }
.sortWith { case (entry1, entry2) =>
- val mod1 = getModificationTime(entry1).getOrElse(-1L)
- val mod2 = getModificationTime(entry2).getOrElse(-1L)
+ val mod1 = entry1.getModificationTime()
+ val mod2 = entry2.getModificationTime()
mod1 >= mod2
--- End diff --
super nit, but L262-264 can be a single line now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176925381 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176619085 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176618998 **[Test build #50349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50349/consoleFull)** for PR 10860 at commit [`772f045`](https://github.com/apache/spark/commit/772f045865462cb419bfc402c9d50e559934fc89). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176619087 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50349/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r51229062
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -248,9 +248,7 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
val logInfos: Seq[FileStatus] = statusList
.filter { entry =>
try {
-getModificationTime(entry).map { time =>
- time >= lastScanTime
-}.getOrElse(false)
+entry.getModificationTime() >= lastScanTime
--- End diff --
That's a great point, thank you for bringing that up.
I wasn't sure if filtering out directory here was too early, and instead
should pass it on and let `EventLoggingListener.openEventLog` decide if it was
valid (directory would not be).
For now I add the check.
Also perhaps we should pass a Filter to `fs.listStatus` if the file naming
convention is known (again assuming we want `checkForLogs` as the gatekeeper as
above).
And I'm pretty sure it no longer throws AcessControlException on [line
253](https://github.com/apache/spark/pull/10860/files#diff-a7befb99e7bd7e3ab5c46c2568aa5b3eR253)
since `listStatus` is not getting called in that path.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176591953 **[Test build #50349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50349/consoleFull)** for PR 10860 at commit [`772f045`](https://github.com/apache/spark/commit/772f045865462cb419bfc402c9d50e559934fc89). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176545233 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176545237 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50324/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176545154 **[Test build #50324 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50324/consoleFull)** for PR 10860 at commit [`48d2e06`](https://github.com/apache/spark/commit/48d2e0681a7541fe8b8c0a58f6668d2d82a63593). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r51215482
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -248,9 +248,7 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
val logInfos: Seq[FileStatus] = statusList
.filter { entry =>
try {
-getModificationTime(entry).map { time =>
- time >= lastScanTime
-}.getOrElse(false)
+entry.getModificationTime() >= lastScanTime
--- End diff --
Shouldn't you also filter out directories here? I don't see that anywhere
else, sorry if I'm missing something.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-176511358 **[Test build #50324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50324/consoleFull)** for PR 10860 at commit [`48d2e06`](https://github.com/apache/spark/commit/48d2e0681a7541fe8b8c0a58f6668d2d82a63593). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r51214677
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -406,10 +380,9 @@ class FsHistoryProviderSuite extends SparkFunSuite
with BeforeAndAfter with Matc
SparkListenerApplicationStart("v10Log", None, 2L, "test", None),
SparkListenerApplicationEnd(4L))
+// Old-style logs are ignored.
--- End diff --
Sure. Updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/10860#discussion_r50772556 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -57,6 +57,15 @@ class FsHistoryProviderSuite extends SparkFunSuite with BeforeAndAfter with Matc Utils.deleteRecursively(testDir) } + val SPARK_VERSION_KEY = "SPARK_VERSION" + val COMPRESSION_CODEC_KEY = "COMPRESSION_CODEC" + + // Constants used to parse Spark 1.0.0 log directories. + val LOG_PREFIX = "EVENT_LOG_" --- End diff -- Test is writing them in old format to test, that's why it is moved here to *Suite.scala --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/10860#discussion_r50772500 --- Diff: .rat-excludes --- @@ -63,14 +63,15 @@ logs .*dependency-reduced-pom.xml known_translations json_expectation -local-1422981759269/* -local-1422981780767/* -local-1425081759269/* -local-1426533911241/* -local-1426633911242/* -local-1430917381534/* +local-1422981759269 +local-1422981780767 +local-1425081759269 +local-1426533911241 +local-1426633911242 +local-1430917381534 local-1430917381535_1 local-1430917381535_2 +local-1425081759268/* --- End diff -- This is used as a negative test that it should not be picked up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r50739205
--- Diff:
core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala
---
@@ -406,10 +380,9 @@ class FsHistoryProviderSuite extends SparkFunSuite
with BeforeAndAfter with Matc
SparkListenerApplicationStart("v10Log", None, 2L, "test", None),
SparkListenerApplicationEnd(4L))
+// Old-style logs are ignored.
--- End diff --
You should remove `writeOldLog` and code related to it. If you really want
to make sure old logs are ignored, just create a directory with some empty
files in it, not need to make it look like a proper old log.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/10860#discussion_r50738942 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -57,6 +57,15 @@ class FsHistoryProviderSuite extends SparkFunSuite with BeforeAndAfter with Matc Utils.deleteRecursively(testDir) } + val SPARK_VERSION_KEY = "SPARK_VERSION" + val COMPRESSION_CODEC_KEY = "COMPRESSION_CODEC" + + // Constants used to parse Spark 1.0.0 log directories. + val LOG_PREFIX = "EVENT_LOG_" --- End diff -- Are these constants needed? They're for the old log format. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/10860#discussion_r50738693 --- Diff: .rat-excludes --- @@ -63,14 +63,15 @@ logs .*dependency-reduced-pom.xml known_translations json_expectation -local-1422981759269/* -local-1422981780767/* -local-1425081759269/* -local-1426533911241/* -local-1426633911242/* -local-1430917381534/* +local-1422981759269 +local-1422981780767 +local-1425081759269 +local-1426533911241 +local-1426633911242 +local-1430917381534 local-1430917381535_1 local-1430917381535_2 +local-1425081759268/* --- End diff -- Is this log used anywhere? It's in the old format, which cannot be parsed anymore after your changes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10860#discussion_r50738479
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -562,90 +544,18 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
}
/**
- * Loads a legacy log directory. This assumes that the log directory
contains a single event
- * log file (along with other metadata files), which is the case for
directories generated by
- * the code in previous releases.
- *
- * @return input stream that holds one JSON record per line.
- */
- private[history] def openLegacyEventLog(dir: Path): InputStream = {
-val children = fs.listStatus(dir)
-var eventLogPath: Path = null
-var codecName: Option[String] = None
-
-children.foreach { child =>
- child.getPath().getName() match {
-case name if name.startsWith(LOG_PREFIX) =>
- eventLogPath = child.getPath()
-case codec if codec.startsWith(COMPRESSION_CODEC_PREFIX) =>
- codecName =
Some(codec.substring(COMPRESSION_CODEC_PREFIX.length()))
-case _ =>
- }
-}
-
-if (eventLogPath == null) {
- throw new IllegalArgumentException(s"$dir is not a Spark application
log directory.")
-}
-
-val codec = try {
-codecName.map { c => CompressionCodec.createCodec(conf, c) }
- } catch {
-case e: Exception =>
- throw new IllegalArgumentException(s"Unknown compression codec
$codecName.")
- }
-
-val in = new BufferedInputStream(fs.open(eventLogPath))
-codec.map(_.compressedInputStream(in)).getOrElse(in)
- }
-
- /**
- * Return whether the specified event log path contains a old
directory-based event log.
- * Previously, the event log of an application comprises of multiple
files in a directory.
- * As of Spark 1.3, these files are consolidated into a single one that
replaces the directory.
- * See SPARK-2261 for more detail.
- */
- private def isLegacyLogDirectory(entry: FileStatus): Boolean =
entry.isDirectory
-
- /**
* Returns the modification time of the given event log. If the status
points at an empty
--- End diff --
Can this method just return `Long` now? The comment mentions that it would
return `None` in case the directory was empty, but there are no more
directories involved now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174386559 I'd also ping @vanzin for this, since I believe that he was involved in a lot of the original refactoring of this HistoryServer code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174386459 @andrewor14 @JoshRosen ready for review. thx --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174288131 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49950/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174288129 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174288089 **[Test build #49950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49950/consoleFull)** for PR 10860 at commit [`11fbe11`](https://github.com/apache/spark/commit/11fbe11f77af34d5f29fe39c1663033d869e92d7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174273829 **[Test build #49950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49950/consoleFull)** for PR 10860 at commit [`11fbe11`](https://github.com/apache/spark/commit/11fbe11f77af34d5f29fe39c1663033d869e92d7). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174271821 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174271820 **[Test build #49949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49949/consoleFull)** for PR 10860 at commit [`e2b785a`](https://github.com/apache/spark/commit/e2b785a44924632c7dc53fe323ce9f5e6f3edce4). * This patch **fails RAT tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `` * `` * `` * `` * `` * `` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174271823 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49949/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174271800 **[Test build #49949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49949/consoleFull)** for PR 10860 at commit [`e2b785a`](https://github.com/apache/spark/commit/e2b785a44924632c7dc53fe323ce9f5e6f3edce4). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174266111 Thanks for checking - looked into it and it was because of the removal of the legacy log format. It looks like HistoryServerSuite has more tests on the legacy format than the current format, as simply updating test expectations would no-op a bunch of tests, it would seem like the best course of action is to convert legacy log into new format. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-174244196 These look like legitimate test failures. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-173467237 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-173467240 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49852/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-173466952 **[Test build #49852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49852/consoleFull)** for PR 10860 at commit [`4c302ec`](https://github.com/apache/spark/commit/4c302ec0a9212158c5b1cd444670ab4bad81d218). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10860#issuecomment-173450200 **[Test build #49852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49852/consoleFull)** for PR 10860 at commit [`4c302ec`](https://github.com/apache/spark/commit/4c302ec0a9212158c5b1cd444670ab4bad81d218). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12790][CORE] Remove HistoryServer old m...
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10860 [SPARK-12790][CORE] Remove HistoryServer old multiple files format Removed isLegacyLogDirectory code path and updated tests @andrewor14 You can merge this pull request into a Git repository by running: $ git pull https://github.com/felixcheung/spark historyserverformat Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/10860.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #10860 commit 4c302ec0a9212158c5b1cd444670ab4bad81d218 Author: felixcheung Date: 2016-01-19T04:06:39Z remove isLegacyLogDirectory code path and update tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
