[
https://issues.apache.org/jira/browse/HIVE-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-3509:
------------------------------
Attachment: HIVE-3509.D10065.2.patch
MattMartin updated the revision "HIVE-3509 [jira] Exclusive locks are not
acquired when using dynamic partitions".
- Updated Driver.java to make sure that dynamic partitions are handled
properly when 1 or more parts of the dynamic partition is specified (e.g.
"insert overwrite table tstsrcpart partition (ds ='2008-04-08', hr) select key,
value, hr where ds = '2008-04-08';"). The earlier check-in on this
branch/issue only addressed queries that used dynamic partitions without
specifying any of the partitions (e.g. "insert overwrite table tstsrcpart
partition (ds, hr) select key, value, ds, hr where ds <= '2008-04-08';")
- Fixing LINT error.
Reviewers: JIRA
REVISION DETAIL
https://reviews.facebook.net/D10065
CHANGE SINCE LAST DIFF
https://reviews.facebook.net/D10065?vs=31467&id=31563#toc
AFFECTED FILES
data/conf/hive-site.xml
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
ql/src/java/org/apache/hadoop/hive/ql/hooks/HookContext.java
ql/src/test/org/apache/hadoop/hive/ql/hooks/PreExecutePrinter.java
ql/src/test/queries/clientnegative/lockneg1.q
ql/src/test/queries/clientnegative/lockneg2.q
ql/src/test/queries/clientnegative/lockneg3.q
ql/src/test/queries/clientnegative/lockneg4.q
ql/src/test/queries/clientnegative/lockneg5.q
ql/src/test/queries/clientpositive/lock1.q
ql/src/test/queries/clientpositive/lock2.q
ql/src/test/queries/clientpositive/lock3.q
ql/src/test/queries/clientpositive/lock4.q
ql/src/test/results/clientnegative/lockneg1.q.out
ql/src/test/results/clientnegative/lockneg2.q.out
ql/src/test/results/clientnegative/lockneg3.q.out
ql/src/test/results/clientnegative/lockneg4.q.out
ql/src/test/results/clientnegative/lockneg5.q.out
ql/src/test/results/clientpositive/lock1.q.out
ql/src/test/results/clientpositive/lock2.q.out
ql/src/test/results/clientpositive/lock3.q.out
ql/src/test/results/clientpositive/lock4.q.out
To: JIRA, MattMartin
Cc: njain
> Exclusive locks are not acquired when using dynamic partitions
> --------------------------------------------------------------
>
> Key: HIVE-3509
> URL: https://issues.apache.org/jira/browse/HIVE-3509
> Project: Hive
> Issue Type: Bug
> Components: Locking
> Affects Versions: 0.9.0
> Reporter: Matt Martin
> Assignee: Matt Martin
> Attachments: HIVE-3509.1.patch.txt, HIVE-3509.D10065.1.patch,
> HIVE-3509.D10065.2.patch
>
>
> If locking is enabled, the acquireReadWriteLocks() method in
> org.apache.hadoop.hive.ql.Driver iterates through all of the input and output
> entities of the query plan and attempts to acquire the appropriate locks. In
> general, it should acquire SHARED locks for all of the input entities and
> exclusive locks for all of the output entities (see the Hive wiki page on
> [locking|https://cwiki.apache.org/confluence/display/Hive/Locking] for more
> detailed information).
> When the query involves dynamic partitions, the situation is a little more
> subtle. As the Hive wiki notes (see previous link):
> {quote}
> in some cases, the list of objects may not be known - for eg. in case of
> dynamic partitions, the list of partitions being modified is not known at
> compile time - so, the list is generated conservatively. Since the number of
> partitions may not be known, an exclusive lock is taken on the table, or the
> prefix that is known.
> {quote}
> After [HIVE-1781|https://issues.apache.org/jira/browse/HIVE-1781], the
> observed behavior is no longer consistent with the behavior described above.
> [HIVE-1781|https://issues.apache.org/jira/browse/HIVE-1781] appears to have
> altered the logic so that SHARED locks are acquired instead of EXCLUSIVE
> locks whenever the query involves dynamic partitions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
