Not simultaneously. In Hive 2 the first delete started will obtain a lock, and the second will have to wait. In Hive 3, the first one to commit will win and the second will fail (at commit time).
Alan. On Mon, Sep 9, 2019 at 10:55 AM David Morin <morin.david....@gmail.com> wrote: > Thanks Alan, > > When you say "you just can't have two simultaneous deletes in the same > partition", simultaneous means for the same transaction ? > If a create 2 "transactions" for 2 deletes on the same table/partition it > works. Am I right ? > > > Le lun. 9 sept. 2019 à 19:04, Alan Gates <alanfga...@gmail.com> a écrit : > >> In Hive 2 update and delete take what are called semi-shared locks >> (meaning they allow shared locks through, while not allowing other >> semi-shared locks), and insert and select take shared locks. So you can >> insert or select while deleting, you just can't have two simultaneous >> deletes in the same partition. >> >> The reason insert can take a shared lock is because Hive does not enforce >> uniqueness constraints, so there's no concept of overwriting an existing >> row. Multiple inserts can also proceed simultaneously. >> >> This changes in Hive 3, where update and delete also take shared locks >> and a first committer wins strategy is employed instead. >> >> Alan. >> >> On Mon, Sep 9, 2019 at 8:29 AM David Morin <morin.david....@gmail.com> >> wrote: >> >>> Hello, >>> >>> I use in production HDP 2.6.5 with Hive 2.1.0 >>> We use transactional tables and we try to ingest data in a streaming way >>> (despite the fact we still use Hive 2) >>> I've read some docs but I would like some clarifications concerning the >>> use of Locks with transactional tables. >>> Do we have to use locks during insert or delete ? >>> Let's consider this pipeline: >>> 1. create a transaction for delete and related lock (shared) >>> 2. create delta directory + file with this new transaction (original >>> transaction != current with original is the transaction used for the last >>> insert) >>> 3 Same steps 1 and 2 for Insert (except original transaction = current) >>> 4. commit transactions >>> >>> Can we use Shared lock here ? Thus select queries can still be used >>> >>> Thanks >>> David >>> >>> >>> >>> >>> >>> >>>
