[
https://issues.apache.org/jira/browse/MAPREDUCE-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243181#comment-13243181
]
Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-4059:
-----------------------------------------------------------
Hi Robert,
Again I have some comments/doubts on the attached patch
There is an API
{noformat}JobsInfo getPartialJobs(Long offset, Long count, String user, String
queue, Long sBegin, Long sEnd, Long fBegin, Long fEnd);{noformat}
which supports returning only a limited number of jobs (no of jobs = offset +
count - 1). But
[MAPREDUCE-3971|https://issues.apache.org/jira/browse/MAPREDUCE-3971] is also
trying to solve the same issue. So are we solving
[MAPREDUCE-3971|https://issues.apache.org/jira/browse/MAPREDUCE-3971] also as
part of this issue or am I missing anything here?
Also I *think* it is better to add query param like
{noformat}@QueryParam("offset") String offset{noformat}
in HsWebServices#getJobs method to support offset. In the patch offset
is hardcoded as 0.Or any reason for doing so?
Also I *think* it is better to use LinkedHashMap in
CachedHistoryStorage#getAllPartialJobs and LinkedList in
HistoryFileManager.getAllMetaInfo for reasons I metioned at
[MAPREDUCE-3971|https://issues.apache.org/jira/browse/MAPREDUCE-3971?focusedCommentId=13226923&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13226923].
Do you see any problems with these?
> The history server should have a separate pluggable storage/query interface
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-4059
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4059
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.24.0, 0.23.3
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Attachments: MR-4059.txt, MR-4059.txt
>
>
> The history server currently caches all parsed jobs in RAM. These jobs can
> be very large because of counters. It would be nice to have a pluggable
> interface for the cacheing and querying of the cached data so that we can
> play around with different implementations. Also just for cleanness of the
> code it would be nice to split the very large JobHistoryServer.java into a
> few smaller ones that are more understandable and readable.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
