> On Feb. 13, 2020, 3:08 p.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Lines 4546 (patched) > > <https://reviews.apache.org/r/72129/diff/2/?file=2211214#file2211214line4594> > > > > Hi Rajesh, could you please explain, what is the reason of doing > > partition filtering on HMS side, not backend db?
By adding all the partition details, the query can become large and has the issue of overflowing in the case of oracle (i,e have to batch with 1000 entries). Also, its incurs parsing in sql server side, as it is executed as Statement. Given that we have additional filter now, it would make it a lot simpler to do this in client side. This was pointed out in the JIRA by Gopal. - Rajesh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72129/#review219575 ----------------------------------------------------------- On Feb. 13, 2020, 1:22 p.m., Rajesh Balamohan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72129/ > ----------------------------------------------------------- > > (Updated Feb. 13, 2020, 1:22 p.m.) > > > Review request for hive, Gopal V, Peter Vary, and Zoltan Chovan. > > > Repository: hive-git > > > Description > ------- > > - Main change is in TxnHandler::checkLock. > - When all incoming requests are SHARED_READ, we can add a condition in > the query to retrieve only relevant rows. This avoids significant number of > rows fetched in the form of "SHARED_READ + ACQUIRED". There is a corner > condition of "SHARED_WRITE --> SHARED_READ::ACQUIRED", which is misleading in > the jumpttable. This condition can be optimised later. > - Also, removed the "HL_PARTITION IN" clause which could potentially > overflow for oracle. Partition details can be filtered out, if the earlier > query actually returned any rows. > - Rest of the changes, are related to refactoring > "TxnHandler::enqueueLockWithRetry" to reduce lock scope. > > > Diffs > ----- > > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > f53aebe4ad > > > Diff: https://reviews.apache.org/r/72129/diff/2/ > > > Testing > ------- > > > File Attachments > ---------------- > > HIVE-22850.5.patch > > https://reviews.apache.org/media/uploaded/files/2020/02/13/74ec6cbd-c552-4d46-b5a6-e2fa6da41bdc__HIVE-22850.5.patch > > > Thanks, > > Rajesh Balamohan > >
