[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261838#comment-16261838
]
Andrew Sherman commented on HIVE-18054:
---------------------------------------
To fix this, move LineageState from SessionState to QueryState.
In MoveTask/MoveWork use the LineageState from the MoveTask's QueryState
rather than trying to use it from MoveWork.
Add a test which runs multiple jdbc queries in a thread pool
against the same connection and show that Vertices are not lost from Lineage.
As part of this test, add ReadableHook, an ExecuteWithHookContext that stores
HookContexts in memory and makes them available for reading.
Make LineageLogger methods static so they can be used elsewhere.
Sometimes a running query (originating in a Driver) will instantiate
another Driver to run or compile another query. Because these Drivers
shared a Session, the child Driver would accumulate Lineage information
along with that of the parent Driver. For consistency a LineageState is
passed to these child Drivers and stored in the new Driver's QueryState.
> Make Lineage work with concurrent queries on a Session
> -------------------------------------------------------
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
> Reporter: Andrew Sherman
> Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch,
> HIVE-18054.6.patch, HIVE-18054.7.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
