Hey,
YARN-321 is targeted for the Hadoop 2.4. The minimum feature set doesn't
include application-specific data, so that probably won't be part of 2.4
unless other things delay the release for a while. There are no APIs for
it yet and pluggable UIs have been discussed but not agreed upon. I thi
Purav, look up the Singleton pattern which is what you seem to be
describing.
The strategy you describe does not sound like a good idea, however. It
couples the "lookup" service rather strongly (and serially) to its data
processing clients. This is usually, though not always, less robust and
effic
Hey Sandy,
Do you know what the status is for YARN-321 and what version of YARN
it's targeted for? Also, is there any kind of documentation or API for
this? Does it control the presentation of the data itself (e.g. it
actually has its own UI)?
@Tom - having an optional history server sounds like
As a sidenote, it would be nice to make sure that whatever done here will
work with the YARN Application History Server (YARN-321), a generic history
server that functions similarly to MapReduce's JobHistoryServer. It will
eventually have the ability to store application-specific data.
-Sandy
O
I don't think you want to save the html/xml files. I would rather see the info
saved into a history file in like a json format that could then be re-read and
the web ui display the info, hopefully without much change to the UI parts.
For instance perhaps the history server could read the file a
Thanks.
Broadcasting such huge entities does not seem like a feasible solution.
Serialization-Deserialization and network seem to have a huge overhead for
large files.
Before I consider moving into an external lookup service (as Christopher
rightly suggested) I was wondering if I could make each s
Hi all
Spark job web ui will not be available when job is over, but it is
convenient for developer to debug with persisting job web ui. I just come up
with draft for this issue.
1. We could simply save the web page with html/xml
format(stages/executors/storages/environment) to ce
