TheR1sing3un commented on PR #12531: URL: https://github.com/apache/hudi/pull/12531#issuecomment-2560642746
> > @danny0405 For rocksdb in our benchmark, there will be a performance regression when the filtering ratio is higher than 50%, which is related to rocksdb's own optimization of iterator, and for bitcask, there will be no performance regresstion. > > If the RocksDB has uncertainty for performance, let's keep it as it is specifically. Does that mean we'll only make predicate push-down on bitcask's implementation, but will the rocksdb implementation still have the same logic? > > @danny0405 For rocksdb in our benchmark, there will be a performance regression when the filtering ratio is higher than 50%, which is related to rocksdb's own optimization of iterator, and for bitcask, there will be no performance regresstion. > > If the RocksDB has uncertainty for performance, let's keep it as it is specifically. Nice suggestion! I still keep `RocksDbDiskMap::iterator(filter)` method but not push down predicate before iterating all key-values, just iterate all key-values and then filter by key. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
