[
https://issues.apache.org/jira/browse/FLINK-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924279#comment-15924279
]
ASF GitHub Bot commented on FLINK-1579:
---------------------------------------
Github user uce commented on a diff in the pull request:
https://github.com/apache/flink/pull/3460#discussion_r105907455
--- Diff:
flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/MemoryArchivist.scala
---
@@ -183,6 +198,45 @@ class MemoryArchivist(private val max_entries: Int)
}
}
+ private def archiveJsonFiles(graph: ArchivedExecutionGraph) {
+ future {
+ try {
+ val rootPath = new Path(flinkConfiguration.getString(
+ JobManagerOptions.ARCHIVE_DIR))
+ val fs = rootPath.getFileSystem
+ val path = new Path(rootPath, s"${graph.getJobID.toString}")
+ val out = fs.create(path, WriteMode.NO_OVERWRITE)
+
+ try {
+ val gen = jacksonFactory.createGenerator(out, JsonEncoding.UTF8)
+ try {
+ gen.writeStartObject()
+ gen.writeArrayFieldStart(ArchivedJson.ARCHIVE)
+ for (archiver <- WebMonitorUtils.getJsonArchivists) {
+ for (archive <- archiver.archiveJsonWithPath(graph).asScala)
{
+ gen.writeStartObject()
+ gen.writeStringField(ArchivedJson.PATH, archive.getPath)
+ gen.writeStringField(ArchivedJson.JSON, archive.getJson)
+ gen.writeEndObject()
+ }
+ }
+ gen.writeEndArray()
+ gen.writeEndObject()
--- End diff --
Should we add a log message that the job has been archived in the path XYZ?
> Create a Flink History Server
> -----------------------------
>
> Key: FLINK-1579
> URL: https://issues.apache.org/jira/browse/FLINK-1579
> Project: Flink
> Issue Type: New Feature
> Components: Distributed Coordination
> Affects Versions: 0.9
> Reporter: Robert Metzger
> Assignee: Chesnay Schepler
>
> Right now its not possible to analyze the job results for jobs that ran on
> YARN, because we'll loose the information once the JobManager has stopped.
> Therefore, I propose to implement a "Flink History Server" which serves the
> results from these jobs.
> I haven't started thinking about the implementation, but I suspect it
> involves some JSON files stored in HDFS :)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
