[
https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated MAPREDUCE-157:
-----------------------------------
Attachment: MAPREDUCE-157-avro.patch
Here's a patch that illustrates how Avro can be used to generate the classes
equivalent to those in your patch.
To write them to an output stream, you'd use something like:
{code}
Encoder encoder = new BinaryEncoder(outputStream);
DatumWriter writer = new SpecificDatumWriter(Events.Event._SCHEMA);
...
Events.JobSubmitted submitted = new Events.JobSubmitted();
submitted.kind = Events.Kind.SUBMITTED;
submitted.jobId = ... ;
submitted.jobName = ...;
...
writer.write(submitted, encoder);
...
writer.close();
{code}
To read them, use something like:
{code}
Decoder decoder = new BinaryDecoder(inputStream);
DatumReader reader = new SpecificDatumReader(Events.Event._SCHEMA);
...
Events.Event event = reader.read(decoder);
switch (event.kind) {
JOB_SUBMITTED :
Events.JobSubmitted submission = (Events.JobSubmitted)event.event;
...
break;
...
}
{code}
> Job History log file format is not friendly for external tools.
> ---------------------------------------------------------------
>
> Key: MAPREDUCE-157
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-157
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Owen O'Malley
> Assignee: Jothi Padmanabhan
> Attachments: mapred-157-prelim.patch, MAPREDUCE-157-avro.patch
>
>
> Currently, parsing the job history logs with external tools is very difficult
> because of the format. The most critical problem is that newlines aren't
> escaped in the strings. That makes using tools like grep, sed, and awk very
> tricky.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
