deniskuzZ opened a new pull request, #6911: URL: https://github.com/apache/hive/pull/6911
### What The HS2-level metastore cache keys every entry on a `TableWatermark` of `(validWriteIdList, tableId)`. `isValid()` requires a non-null write-id list, so a non-transactional table can never be cached. Six call sites built that watermark by fetching the table id first and testing validity afterwards. The id comes from `getTable(catName, dbName, tableName)`, which carries no write-id list of its own, so the watermark guarding `getTable` is itself invalid and the lookup is never served from the cache: it is always a metastore round-trip, made only to discover that nothing can be cached. This tests the write-id list before fetching the id. ### Safety No response that was previously uncached becomes cached. At each site the new test is on the same null that `isValid()` would have rejected one line later, so the outcome is the same uncached delegate call, minus one `get_table`. Non-transactional and external tables remain outside the cache: neither the request-supplied list nor `getValidWriteIdList()` — which returns null for a non-transactional table, and otherwise only finds an entry for a table the txn manager tracks — can produce one for them. ### Scope of the saving | sites | write-id list from | effect | |---|---|---| | `listPartitionsByExpr`, `getPartitionsSpecByExpr`, `getAggrColStatsFor`, `getPartitionsByNames` | the request | the `get_table` is removed | | `listPartitionNames`, `getTableColumnStatistics` | local `getValidWriteIdList()` | one of two removed; the helper still fetches the table to test `isTransactionalTable` | ### Tests `TestHiveMetaStoreClientWithLocalCache` asserts the delegate sees no `getTable` for a non-transactional table, with a positive control that fails if the fixture leaves the cache disabled. ### Measured HMS log of a TPC-DS 1 TB session on external Parquet tables (about 795 compilations: the 99 queries three times per codec for two codecs, plus EXPLAINs): 16,111 `get_table` calls, 9,356 of which come immediately before one of these sites. At 19.8 ms each on the HMS side, that is about 0.23 s of compile time per query. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
