[
https://issues.apache.org/jira/browse/HIVE-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18153:
------------------------------------
Attachment: HIVE-18153.patch
This patch basically cleans up and rewrites file management for TezSessionState
to be less random.
It is not equivalent to the old code because the logic of the old code seems to
have evolved over time into something weird.
reopen changes themselves are relatively small.
> refactor reopen in TezTask or WM
> --------------------------------
>
> Key: HIVE-18153
> URL: https://issues.apache.org/jira/browse/HIVE-18153
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-18153.patch
>
>
> TezTask reopen relies on getting the same session object in terms of setup;
> WM reopen returns a new session from the pool.
> The former has the advantage of not having to reupload files and stuff... but
> the object reuse results in a lot of ugly code, and also reopen might be
> slower on average with the session pool than just getting a session from the
> pool. Either WM needs to do the object-preserving reopen, or TezTask needs to
> be refactored. It looks like DAG would have to be rebuilt to do the latter
> because of some paths tied to a directory of the old session. Let me see if I
> can get around that; if not we can do the former; and then if the former
> results in too much ugly code in WM to account for object reuse for different
> Tez client I'd do the latter anyway since it's a failure path :)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
