Hello, I'm using HDP 2.5.0.0 with included hive 1.2.1. And I have problem
with locking mechanism.
Most of my queries to hive looks like this.
*(1) insert into table results_table partition(task_id=${task_id})*
* select * from data_table where ....;*
results_table partitioned by task_id field and i expect to get exclusive
lock on corresponding partition.Which is true:
Lock ID Database Table Partition State Blocked By Type
Transaction ID
136639682.4 default results_table task_id=5556 ACQUIRED
EXCLUSIVE NULL
Another type of query is fetching data from results_table:
*(2) select * from results_table where task_id = ${task_id}*
This select doesn't require any map reduce and executes fast. This is
exactly what I want.
But if I execute this two queries at the same time I can't perform read
from result_table partition while inserting data into another.
Locks looks like this:
Lock ID Database Table Partition State Blocked By Type
Transaction ID
136639682.4 default results_table task_id=5556 ACQUIRED
EXCLUSIVE NULL
136639700.1 default results_table NULL WAITING 136639682.4
SHARED_READ NULL
136639700.2 default results_table task_id=5535 WAITING
SHARED_READ NULL
Reading data from specified partition requires shared lock on whole table.
This prevents me to get data untill first query completes.
As I can see on this page
<https://cwiki.apache.org/confluence/display/Hive/Locking#Locking-UseCases>
this
is expected behaivor. But I don't understand why we need lock on table.
Can I get rid of shared lock on whole table, while still having shared lock
on specific partition?
