[
https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16324349#comment-16324349
]
Xuefu Zhang commented on HIVE-14162:
------------------------------------
[~belugabehr], Spark on Yarn is powered by a feature call dynamic allocation,
which is strongly recommended in a multi-tenancy or cost sensitive env. This is
something that you might have missed. With it, unused executors are returned to
the cluster so others can use.
SparkContext is stateful, and can be considered as an extension of HIve
session. If you allow a SparkSession to time out, then part of the state is
gone. In that case, you might just as well let the whole session expire.
What you requested isn't completely off, but I'd like to see if existing
features are enough to reach what you want achieve.
> Allow disabling of long running job on Hive On Spark On YARN
> ------------------------------------------------------------
>
> Key: HIVE-14162
> URL: https://issues.apache.org/jira/browse/HIVE-14162
> Project: Hive
> Issue Type: New Feature
> Components: Spark
> Reporter: Thomas Scott
> Assignee: Aihua Xu
> Priority: Minor
> Attachments: HIVE-14162.1.patch
>
>
> Hive On Spark launches a long running process on the first query to handle
> all queries for that user session. In some use cases this is not desired, for
> instance when using Hue with large intervals between query executions.
> Could we have a property that would cause long running spark jobs to be
> terminated after each query execution and started again for the next one?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
