increase concurrency
--------------------
Key: HIVE-1557
URL: https://issues.apache.org/jira/browse/HIVE-1557
Project: Hadoop Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Namit Jain
Copying Joy's comment from https://issues.apache.org/jira/browse/HIVE-1293
a little bummed that locks need to be held for entire query execution. that
could mean a writer blocking readers for hours.
hive's query plans seem to be of two distinct stages:
1. read a bunch of stuff, compute intermediate/final data
2. move final data into output locations
ie. - a single query never reads what it writes (into a final output location).
even if #1 and #2 are mingled today - they can easily be put in order.
in that sense - we only need to get shared locks for all read entities involved
in #1 to begin with. once phase #1 is done, we can drop all the read locks and
get the exclusive locks for all the write entities in #2, perform #2 and quit.
that way exclusive locks are held for a very short duration. i think this
scheme is similarly deadlock free (now there are two independent lock
acquire/release phases - and each of them can lock stuff in lex. order).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
