[
https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754830#action_12754830
]
Philip Zeyliger commented on MAPREDUCE-157:
-------------------------------------------
Some comments:
* EventWriter class needs javadoc.
* You can use Java 1.5 enum fanciness
(http://java.sun.com/j2se/1.5.0/docs/guide/language/enums.html) to associate
the event types with their enums.
So, instead of storing this map in a separate file, where it will get out of
date if someone not intimately familiar with the code wishes to modify it, you
store the mapping inside the Enum object itself. So, instead of:
{code}
classMap =
new HashMap<HistoryEvent.EventType, Class<? extends HistoryEvent>>();
classMap.put(HistoryEvent.EventType.JOB_SUBMITTED,
JobSubmittedEvent.class);
classMap.put(HistoryEvent.EventType.JOB_INITED,
JobInitedEvent.class);
...
{code}
you might do something like this example here:
{code}
enum Collection {
MAP(java.util.Map.class),
LIST(java.util.List.class),
QUEUE(java.util.Queue.class);
Class klass;
Collection(Class klass) {
this.klass = klass;
}
Class getKlass() {
return this.klass;
}
private static Map<Class, Collection> klasses = new HashMap();
static {
for (Collection f : Collection.values()) {
klasses.put(f.getKlass(), f);
}
}
public static Collection getEnumForClass(Class klass) {
return klasses.get(klass);
}
}
{code}
This puts the 1-1 mapping in the same place. It's impossible to add an enum
value without specifying its associated value.
* Honestly, the dozen or so {{*Event.java}} classes are begging to have a
framework. Each of those classes merely specifies a schema (a set of fields,
each with a type), and implements
a simple struct. Code generation is a typical solution for this sort of
problem. You could also specify very little in each class, and implement
readFields and writeFields generically in each class.
-- Philip
> Job History log file format is not friendly for external tools.
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-157
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-157
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Affects Versions: 0.20.1
> Reporter: Owen O'Malley
> Assignee: Jothi Padmanabhan
> Fix For: 0.21.0
>
> Attachments: mapred-157-10Sep.patch, mapred-157-4Sep.patch,
> mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch,
> MAPREDUCE-157-avro.patch
>
>
> Currently, parsing the job history logs with external tools is very difficult
> because of the format. The most critical problem is that newlines aren't
> escaped in the strings. That makes using tools like grep, sed, and awk very
> tricky.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
