swamirishi commented on PR #10876: URL: https://github.com/apache/ozone/pull/10876#issuecomment-5127244469
> Then all the deleteRange patch to OM need to be prohibited until snapshot feature handles it. > > believe we should change the Compaction type to kForceOptimized. > > @swamirishi please provide the full reasoning why kForceOptimized is better. > > > Seek should be a relatively safe > > Also please provide reasoning how this is "relatively safe", relative to what? Unless seek is a point query and will not need to iterate over tombstones, seek is unsafe. When you do seek it doesn't iterate through all the tombstones it would try to search in the LSM using indexes in the memtable and ssts to efficiently figure out the keys. Only iterator.next() would have troubles when there are lot of tombstones. With deleteRange we are actually solving the tombstone problem and thus solving the slow iterator problem. With deleteRange the iterator would skip an entire range instead of having to go through each tombstone on by one. > > > But I just realized we cannot use deleteRange because of snapshots since snapshot diff depends on the individual tombstone entry for deletes. > > Then all the deleteRange patch to OM need to be prohibited until snapshot feature handles it. > We cannot support DeleteRange on the user path on keyTable/DirTable/FileTable(background services are ok) till the point non defragmented snapshot diff doesn't understand deleteRange tombstones. If we add a simple loop to identify the keys with deleteRange on the snapshot diff flows we can find the diff then. > Regarding the multi-get idea, I don't think it's possible since OM still uses the table cache and RocksDB cannot update both the table cache (which is Ozone logic) and the RocksDB itself. In that case, we can consider whether this OM table cache (double buffer, etc) is actually worth the effort or just another premature optimization (we can check how other Raft + RocksDB system like TiKV actually does it). We can then consider removing OM table cache entirely. Table cache is also a source of inconsistency in OM since linearizability seem to be violated (see #10748) and I don't think anyone (including me) actually knows the consistency guarantee to the table cache since the design document never specifies it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
