deniskuzZ opened a new pull request, #6921: URL: https://github.com/apache/hive/pull/6921
### What changes were proposed in this pull request? - CachingDeleteLoader loads a data file's position deletes through the LLAP metadata cache: every entry is a deletion vector in its serialized form, keyed by the delete file and the data file it applies to - V2 position delete files are decoded once per query on a daemon (PositionDeletesReader, Parquet straight from the path and pos columns) and cached in the same form, split per data file - DeletionVectors.deserialize: Iceberg has no public DV deserializer that does not validate against a DV DeleteFile - `hive.llap.io.cache.deletedeltas` is renamed `hive.llap.io.cache.deletes` and covers Iceberg too: `metadata` caches DVs, `all` also V2 position deletes; the old name is still accepted - DELETE_CACHE_* task counters: hits, misses, puts and failed puts - Stacked on HIVE-30102 ### Why are the changes needed? Every task re-read and re-decoded the same delete files. 1TRC part2 V2, 40 x r5d: 711.9 s without the cache, 394.3 s with it, hit rate 81-84%; local miss-path decode 235 -> 20 ns per record ### Does this PR introduce _any_ user-facing change? Yes: `hive.llap.io.cache.deletedeltas` is deprecated in favour of `hive.llap.io.cache.deletes`, which also applies to Iceberg deletes ### How was this patch tested? - TestCachingDeleteLoader, TestPositionDeletesReader, TestDeletionVectors, TestHiveVectorizedReader, TestHiveConf (old name), TestVectorizedOrcAcidRowBatchReaderTailCache - llap_iceberg_delete_cache.q -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
