[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899225#action_12899225 ] Namit Jain commented on HIVE-1543: -- +1 for 0.6 also Will commit in both once the tests pass > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1543.1.patch, HIVE-1543.2_branch0.6.patch, > HIVE-1543.patch, HIVE-1543_branch0.6.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899209#action_12899209 ] Namit Jain commented on HIVE-1543: -- +1 for trunk > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1543.1.patch, HIVE-1543.patch, > HIVE-1543_branch0.6.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899207#action_12899207 ] Namit Jain commented on HIVE-1543: -- can you reload the patch for 0.6 - i got some merge conflicts > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.6.0 > > Attachments: HIVE-1543.1.patch, HIVE-1543.patch, > HIVE-1543_branch0.6.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899169#action_12899169 ] He Yongqiang commented on HIVE-1543: we can do two different patches for trunk and 0.6. I think BucketizedHiveRecordReader also extends HiveContextAwareRecordReader. > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.7.0 > > Attachments: HIVE-1543.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899165#action_12899165 ] Namit Jain commented on HIVE-1543: -- What about BucketizedHiveRecordReader ? > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.7.0 > > Attachments: HIVE-1543.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899158#action_12899158 ] Ning Zhang commented on HIVE-1543: -- @yongqiang, HiveContextAwareRecordReader is only available in 0.7 (trunk) but not 0.6. I think this should be back ported to 0.6 as well (as we did for HIVE-1492)? > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.7.0 > > Attachments: HIVE-1543.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1543) set abort in ExecMapper when Hive's record reader got an IOException
[ https://issues.apache.org/jira/browse/HIVE-1543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899098#action_12899098 ] He Yongqiang commented on HIVE-1543: let's do it in HiveContextAwareRecordReader. And maybe store the var in IOContext? > set abort in ExecMapper when Hive's record reader got an IOException > > > Key: HIVE-1543 > URL: https://issues.apache.org/jira/browse/HIVE-1543 > Project: Hadoop Hive > Issue Type: Improvement >Reporter: Ning Zhang >Assignee: Ning Zhang > Fix For: 0.7.0 > > Attachments: HIVE-1543.patch > > > When RecordReader got an IOException, ExecMapper does not know and will close > the operators as if there is not error. We should catch this exception and > avoid writing partial results to HDFS which will be removed later anyways. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
