[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859523#action_12859523 ]
Chao Wang commented on PIG-1342: -------------------------------- Rebase the patch against the latest trunk. > [Zebra] Avoid making unnecessary name node calls for writes in Zebra > -------------------------------------------------------------------- > > Key: PIG-1342 > URL: https://issues.apache.org/jira/browse/PIG-1342 > Project: Pig > Issue Type: Improvement > Affects Versions: 0.6.0, 0.7.0 > Reporter: Chao Wang > Assignee: Chao Wang > Fix For: 0.8.0 > > Attachments: PIG-1342.patch, PIG-1342.patch > > > Currently, table and column group level meta data is extracted from job > configuration object and written onto HDFS disk within checkOutputSpec(). > Later on, writers at back end will open these files to access the meta data > for doing writes. This puts extra load to name node since all writers need to > make name node calls to open files. > We propose the following approach to this problem: > For writers at back end, they extract meta information from job configuration > object directly, rather than making name node calls and going to HDFS disk to > fetch the information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.