[
https://issues.apache.org/jira/browse/HIVE-18831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418971#comment-16418971
]
Rui Li commented on HIVE-18831:
-------------------------------
Besides, I think the {{JobResultSerializer}} has some problems:
# It shouldn't clear the Output when java serializer fails. The Output is
maintained by kryo and may already contain some data in it. Instead, I think we
can try (with the fallback) to serialize JobResult into a byte array, and then
write the byte array into the Output. Deserializer can read the byte array and
use an ObjectInputStream to deserialize the data.
# How about we just send the stack trace if the Throwable can't be serialized?
We can set the Throwable to null in JobResult and keep the stack trace string
as we do now. I think this is simpler and doesn't need the assumption that
SparkException is always serializable.
> Differentiate errors that are thrown by Spark tasks
> ---------------------------------------------------
>
> Key: HIVE-18831
> URL: https://issues.apache.org/jira/browse/HIVE-18831
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Attachments: HIVE-18831.1.patch, HIVE-18831.2.patch,
> HIVE-18831.3.patch, HIVE-18831.4.patch, HIVE-18831.6.patch,
> HIVE-18831.7.patch, HIVE-18831.8.WIP.patch
>
>
> We propagate exceptions from Spark task failures to the client well, but we
> don't differentiate between errors from HS2 / RSC vs. errors thrown by
> individual tasks.
> Main motivation is that when the client sees a propagated Spark exception its
> difficult to know what part of the excution threw the exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
