[
https://issues.apache.org/jira/browse/HIVE-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-10404:
-----------------------------------
Attachment: HIVE-10404.01.patch
After discussing with [~ashutoshc], we would like to estimate the efforts
needed if we would like to set hive.exec.parallel=true as default.
> hive.exec.parallel=true causes "out of sequence response" and
> SocketTimeoutException: Read timed out
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-10404
> URL: https://issues.apache.org/jira/browse/HIVE-10404
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Eugene Koifman
> Attachments: HIVE-10404.01.patch
>
>
> With hive.exec.parallel=true, Driver.lauchTask() calls Task.initialize() from
> 1 thread on several Tasks. It then starts new threads to run those tasks.
> Taks.initiazlie() gets an instance of Hive and holds on to it. Hive.java
> internally uses ThreadLocal to hand out instances, but since
> Task.initialize() is called by a single thread from the Driver multiple tasks
> share an instance of Hive.
> Each Hive instances has a single instance of MetaStoreClient; the later is
> not thread safe.
> With hive.exec.parallel=true, different threads actually execute the tasks,
> different threads end up sharing the same MetaStoreClient.
> If you make 2 concurrent calls, for example Hive.getTable(String), the Thrift
> responses may return to the wrong caller.
> Thus the first caller gets "out of sequence response", drops this message and
> reconnects. If the timing is right, it will consume the other's response,
> but the the other caller will block for hive.metastore.client.socket.timeout
> since its response message has now been lost.
> This is just one concrete example.
> One possible fix is to make Task.db use ThreadLocal.
> This could be related to HIVE-6893
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
