[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Release Note: Changes the Job History file format to use JSON. Simplifies the Job History Parsing logic Removes duplication of code between HistoryViewer and the JSP files History Files are now named as JobID_user Introduces a new cluster level configuration "mapreduce.cluster.jobhistory.maxage" for configuring the amount of time history files are kept before getting cleaned up The configuration "hadoop.job.history.user.location" is no longer supported. was: Changes the Job History file format to use JSON. Simplifies the Job History Parsing logic Removes duplication of code between HistoryViewer and the JSP files History Files are now named as JobID_user Introduces a new cluster level configuration "mapreduce.cluster.jobhistory.maxage" for configuring the amount of time history files are kept before getting cleaned up > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Release Note: Changes the Job History file format to use JSON. Simplifies the Job History Parsing logic Removes duplication of code between HistoryViewer and the JSP files History Files are now named as JobID_user Introduces a new cluster level configuration "mapreduce.cluster.jobhistory.maxage" for configuring the amount of time history files are kept before getting cleaned up > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sharad Agarwal updated MAPREDUCE-157: - Resolution: Fixed Hadoop Flags: [Incompatible change, Reviewed] (was: [Incompatible change]) Status: Resolved (was: Patch Available) I just committed this. Thanks Jothi. > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-16Sep-v1.patch Missed out ivy.xml changes to include jackson jars in the newly added contrib/gridmix > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) retrying hudson > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-16Sep.patch > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: (was: mapred-157-16Sep.patch) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-16Sep.patch Patch updated to the trunk. Removed the version event; version information is logged in EventWriter constructor and read in EventReader ctor Cleaned up the history cleaner time-to-run logic > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-15Sep-v1.patch Incorporated some offline comments from Sharad > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, > mapred-157-15Sep.patch, mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sharad Agarwal updated MAPREDUCE-157: - Hadoop Flags: [Incompatible change] > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: (was: mapred-157-14Sep-v1.patch) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-15Sep.patch Attaching the correct patch > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-15Sep.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-14Sep-v1.patch Patch that incorporates most of the review comments, including the use of enum fanciness to associate types with their enums. bq. Honestly, the dozen or so *Event.java classes are begging to have a framework. Yes, I agree, I realized it too. But given that we were moving to Avro which would do the automatic code generation, just left it simple for now, without trying to develop any other framework. bq. I am working to convert this patch to use Avro Doug, thanks. If possible, could you use this patch as your base patch? > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-14Sep-v1.patch, > mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting updated MAPREDUCE-157: --- Attachment: MAPREDUCE-157-avro.patch > Honestly, the dozen or so *Event.java classes are begging to have a framework. I am working to convert this patch to use Avro. I have a schema that generates code for all of the event classes. I am in the process of converting each of the event classes to be a wrapper around the generated class, providing constructors and accessor methods to the generated class. The serialization will switch to using Avro binary. If Avro 1.1 is released this week as expected, then this could be trivially changed to generate JSON instead. > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-10Sep.patch Iyappan identified a few bugs. This patch fixes them. > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-10Sep.patch, mapred-157-4Sep.patch, > mapred-157-7Sep-v1.patch, mapred-157-7Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-7Sep-v1.patch Now, sqoop's ivy.xml needs to be updated too! > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch, > mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) Resubmitting the patch so that Hudson picks it up again -- unable to explain why Hudson failed the patch > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-7Sep.patch Patch that includes ivy.xml changes to the contrib modules so that they have access to the Jackson library Also modified TestCopyFiles so that it checks for the map count correctly > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-7Sep.patch, > mapred-157-prelim.patch, MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Status: Open (was: Patch Available) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-4Sep.patch Patch for changing History to use JSON format. Some notes about the patch: All history information are logged using events. A version event is prepended to all history files. History viewer and History Parser have been cleaned up and duplication of code in the jsp files and HistoryViewer has been removed. History files are named JobID_username. Filters on the UI page will now be based only on JobID and User name History Viewer now takes a history file as an argument instead of output directory All events are made up of new API objects, including counters. As a result I had to open up a couple of constructors in Counters to public. Hadoop-Vaidya has been changed to use the new History Viewer, but has not been tested with it. A temporary fix has been put for Rumen to get it compiled, it still works only with the old history format and not the new one. > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Fix Version/s: 0.21.0 Affects Version/s: 0.20.1 Status: Patch Available (was: Open) > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Affects Versions: 0.20.1 >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Fix For: 0.21.0 > > Attachments: mapred-157-4Sep.patch, mapred-157-prelim.patch, > MAPREDUCE-157-avro.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated MAPREDUCE-157:
---
Attachment: MAPREDUCE-157-avro.patch
Here's a patch that illustrates how Avro can be used to generate the classes
equivalent to those in your patch.
To write them to an output stream, you'd use something like:
{code}
Encoder encoder = new BinaryEncoder(outputStream);
DatumWriter writer = new SpecificDatumWriter(Events.Event._SCHEMA);
...
Events.JobSubmitted submitted = new Events.JobSubmitted();
submitted.kind = Events.Kind.SUBMITTED;
submitted.jobId = ... ;
submitted.jobName = ...;
...
writer.write(submitted, encoder);
...
writer.close();
{code}
To read them, use something like:
{code}
Decoder decoder = new BinaryDecoder(inputStream);
DatumReader reader = new SpecificDatumReader(Events.Event._SCHEMA);
...
Events.Event event = reader.read(decoder);
switch (event.kind) {
JOB_SUBMITTED :
Events.JobSubmitted submission = (Events.JobSubmitted)event.event;
...
break;
...
}
{code}
> Job History log file format is not friendly for external tools.
> ---
>
> Key: MAPREDUCE-157
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-157
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
>Reporter: Owen O'Malley
>Assignee: Jothi Padmanabhan
> Attachments: mapred-157-prelim.patch, MAPREDUCE-157-avro.patch
>
>
> Currently, parsing the job history logs with external tools is very difficult
> because of the format. The most critical problem is that newlines aren't
> escaped in the strings. That makes using tools like grep, sed, and awk very
> tricky.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jothi Padmanabhan updated MAPREDUCE-157: Attachment: mapred-157-prelim.patch Preliminary patch as requested by Doug. I got this to produce a history file in Json format (but only after commenting lots of code), so more useful for illustration than otherwise. My guess is that this should be fairly straight forward to port this to use Avro. > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > Attachments: mapred-157-prelim.patch > > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
[ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated MAPREDUCE-157: -- Issue Type: Sub-task (was: Bug) Parent: MAPREDUCE-863 > Job History log file format is not friendly for external tools. > --- > > Key: MAPREDUCE-157 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-157 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Owen O'Malley >Assignee: Jothi Padmanabhan > > Currently, parsing the job history logs with external tools is very difficult > because of the format. The most critical problem is that newlines aren't > escaped in the strings. That makes using tools like grep, sed, and awk very > tricky. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
