[jira] Commented: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12860035#action_12860035 ] Hadoop QA commented on PIG-1342: +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442480/PIG-1342.patch against trunk revision 936395. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/299/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/299/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/299/console This message is automatically generated. [Zebra] Avoid making unnecessary name node calls for writes in Zebra Key: PIG-1342 URL: https://issues.apache.org/jira/browse/PIG-1342 Project: Pig Issue Type: Improvement Affects Versions: 0.6.0, 0.7.0 Reporter: Chao Wang Assignee: Chao Wang Fix For: 0.8.0 Attachments: PIG-1342.patch, PIG-1342.patch Currently, table and column group level meta data is extracted from job configuration object and written onto HDFS disk within checkOutputSpec(). Later on, writers at back end will open these files to access the meta data for doing writes. This puts extra load to name node since all writers need to make name node calls to open files. We propose the following approach to this problem: For writers at back end, they extract meta information from job configuration object directly, rather than making name node calls and going to HDFS disk to fetch the information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12860042#action_12860042 ] Yan Zhou commented on PIG-1342: --- +1 [Zebra] Avoid making unnecessary name node calls for writes in Zebra Key: PIG-1342 URL: https://issues.apache.org/jira/browse/PIG-1342 Project: Pig Issue Type: Improvement Affects Versions: 0.6.0, 0.7.0 Reporter: Chao Wang Assignee: Chao Wang Fix For: 0.8.0 Attachments: PIG-1342.patch, PIG-1342.patch Currently, table and column group level meta data is extracted from job configuration object and written onto HDFS disk within checkOutputSpec(). Later on, writers at back end will open these files to access the meta data for doing writes. This puts extra load to name node since all writers need to make name node calls to open files. We propose the following approach to this problem: For writers at back end, they extract meta information from job configuration object directly, rather than making name node calls and going to HDFS disk to fetch the information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12859523#action_12859523 ] Chao Wang commented on PIG-1342: Rebase the patch against the latest trunk. [Zebra] Avoid making unnecessary name node calls for writes in Zebra Key: PIG-1342 URL: https://issues.apache.org/jira/browse/PIG-1342 Project: Pig Issue Type: Improvement Affects Versions: 0.6.0, 0.7.0 Reporter: Chao Wang Assignee: Chao Wang Fix For: 0.8.0 Attachments: PIG-1342.patch, PIG-1342.patch Currently, table and column group level meta data is extracted from job configuration object and written onto HDFS disk within checkOutputSpec(). Later on, writers at back end will open these files to access the meta data for doing writes. This puts extra load to name node since all writers need to make name node calls to open files. We propose the following approach to this problem: For writers at back end, they extract meta information from job configuration object directly, rather than making name node calls and going to HDFS disk to fetch the information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12859625#action_12859625 ] Hadoop QA commented on PIG-1342: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12442480/PIG-1342.patch against trunk revision 936395. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/298/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/298/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/298/console This message is automatically generated. [Zebra] Avoid making unnecessary name node calls for writes in Zebra Key: PIG-1342 URL: https://issues.apache.org/jira/browse/PIG-1342 Project: Pig Issue Type: Improvement Affects Versions: 0.6.0, 0.7.0 Reporter: Chao Wang Assignee: Chao Wang Fix For: 0.8.0 Attachments: PIG-1342.patch, PIG-1342.patch Currently, table and column group level meta data is extracted from job configuration object and written onto HDFS disk within checkOutputSpec(). Later on, writers at back end will open these files to access the meta data for doing writes. This puts extra load to name node since all writers need to make name node calls to open files. We propose the following approach to this problem: For writers at back end, they extract meta information from job configuration object directly, rather than making name node calls and going to HDFS disk to fetch the information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
