[
https://issues.apache.org/jira/browse/HIVE-23596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125151#comment-17125151
]
Gopal Vijayaraghavan commented on HIVE-23596:
---------------------------------------------
LGTM - +1
Will add extra comments in the LlapTaskScheduler, since this takes advantage of
arbitrary container-ids which have a full 64 bit range & do not get used
outside of the runtime of the query (unlike say, task-ids, which show up in
filenames).
The LLAP daemon side doesn't care about a container-id, since we use a 1:1
correspondence between containers & tasks in LLAP.
So a "tagged pointer" or "colored pointer" equivalent implementation is safe.
> Encode guaranteed task information in containerId
> -------------------------------------------------
>
> Key: HIVE-23596
> URL: https://issues.apache.org/jira/browse/HIVE-23596
> Project: Hive
> Issue Type: Improvement
> Reporter: Mustafa Iman
> Assignee: Mustafa Iman
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We should avoid calling LlapTaskScheduler to get initial isguaranteed flag
> for all the tasks. It causes arbitrary delays in sending tasks out. Since
> communicator is a single thread, any blocking there delays all the tasks.
> There are [https://jira.apache.org/jira/browse/TEZ-4192] and
> [https://jira.apache.org/jira/browse/HIVE-23589] for a proper solution to
> this. However, that requires a Tez release which seems far right now. We can
> replace the current hack with another hack that does not require locking.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
