[jira] [Commented] (HIVE-5639) Allow caching of Orc footers in Tez AM
[ https://issues.apache.org/jira/browse/HIVE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13805751#comment-13805751 ] Brock Noland commented on HIVE-5639: I cannot comment on the correctness of the notify stuff but +1 on logging the non-IO errors. Thanks!! > Allow caching of Orc footers in Tez AM > -- > > Key: HIVE-5639 > URL: https://issues.apache.org/jira/browse/HIVE-5639 > Project: Hive > Issue Type: Improvement > Components: Tez >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: tez-branch > > Attachments: HIVE-5639.1.txt, HIVE-5639.2.patch, > HIVE-5639-addendum.txt > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5639) Allow caching of Orc footers in Tez AM
[ https://issues.apache.org/jira/browse/HIVE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13805564#comment-13805564 ] Siddharth Seth commented on HIVE-5639: -- You're right, the current catch Throwable and delay till later shouldn't be done for all Exceptions. I think Throwable is being caught to prevent threads from dying without the system knowing. Will upload an ammended patch to log specific exceptions. > Allow caching of Orc footers in Tez AM > -- > > Key: HIVE-5639 > URL: https://issues.apache.org/jira/browse/HIVE-5639 > Project: Hive > Issue Type: Improvement > Components: Tez >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: tez-branch > > Attachments: HIVE-5639.1.txt, HIVE-5639.2.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5639) Allow caching of Orc footers in Tez AM
[
https://issues.apache.org/jira/browse/HIVE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804136#comment-13804136
]
Brock Noland commented on HIVE-5639:
bq. ORC collects them all together at the end, and throws a single
InvalidInputException.
This is a lot like Hadoop converting everything to IOException. What do you
think about converting to the following paradigm:
{noformat}
+ } catch (Throwable th) {
if(th instanceof Error) {
LOG.error("Unexpected Error", th);
}
+synchronized (context.errors) {
+ context.errors.add(th);
+}
+ }
{noformat}
The reason is that after an Error is thrown, especially sub-classes of
VirtualMachineError, I have found the JVM to be dramatically more likely to be
unstable and die of some other error. It's best to inform the operator of an
Error right away as it's possible that the JVM will segfault or sigbus before
it ever gets the chance log said Error and therefore mask the issue.
VirtualMachineError
> Allow caching of Orc footers in Tez AM
> --
>
> Key: HIVE-5639
> URL: https://issues.apache.org/jira/browse/HIVE-5639
> Project: Hive
> Issue Type: Improvement
> Components: Tez
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Fix For: tez-branch
>
> Attachments: HIVE-5639.1.txt, HIVE-5639.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5639) Allow caching of Orc footers in Tez AM
[ https://issues.apache.org/jira/browse/HIVE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13803758#comment-13803758 ] Siddharth Seth commented on HIVE-5639: -- bq. I see we are catching Throwable here without logging it straightway. That is generally frowned upon. What is the rational? ORC collects them all together at the end, and throws a single InvalidInputException. > Allow caching of Orc footers in Tez AM > -- > > Key: HIVE-5639 > URL: https://issues.apache.org/jira/browse/HIVE-5639 > Project: Hive > Issue Type: Improvement > Components: Tez >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: tez-branch > > Attachments: HIVE-5639.1.txt, HIVE-5639.2.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5639) Allow caching of Orc footers in Tez AM
[ https://issues.apache.org/jira/browse/HIVE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13803721#comment-13803721 ] Brock Noland commented on HIVE-5639: I see we are catching Throwable here without logging it straightway. That is generally frowned upon. What is the rational? > Allow caching of Orc footers in Tez AM > -- > > Key: HIVE-5639 > URL: https://issues.apache.org/jira/browse/HIVE-5639 > Project: Hive > Issue Type: Improvement > Components: Tez >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: tez-branch > > Attachments: HIVE-5639.1.txt > > -- This message was sent by Atlassian JIRA (v6.1#6144)
