Chao Sun created HIVE-10434:
-------------------------------
Summary: Cancel connection to HS2 when remote Spark driver process
has failed [Spark Branch]
Key: HIVE-10434
URL: https://issues.apache.org/jira/browse/HIVE-10434
Project: Hive
Issue Type: Improvement
Components: Spark
Affects Versions: 1.2.0
Reporter: Chao Sun
Assignee: Chao Sun
Currently in HoS, in SparkClientImpl it first launch a remote Driver process,
and then wait for it to connect back to the HS2. However, in certain situations
(for instance, permission issue), the remote process may fail and exit with
error code. In this situation, the HS2 process will still wait for the process
to connect, and wait for a full timeout period before it throws the exception.
What makes it worth, user may need to wait for two timeout periods: one for the
SparkSetReducerParallelism, and another for the actual Spark job. This could be
very annoying.
We should cancel the timeout task once we found out that the process has
failed, and set the promise as failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
