JingsongLi commented on PR #4341: URL: https://github.com/apache/paimon/pull/4341#issuecomment-2448943178
> > But I think there's a very big risk here, can we change it a little bit one by one to make everything in there thread-safe? > > Hi, @JingsongLi What do you mean change it a little bit one by one? > > In the current design, `FileStoreLookupFunction` is non thread safe. This class is the entry point for lookup, so it is necessary to ensure the thread safety of `FileStoreLookupFunction`. The current design mainly focuses on the following points: > > 1. Ensure thread safety when initializing `FileStoreLookupFunction`; > 2. The implementation of two lookup caches, namely in `LookupLevels` and `RocksDBState`, ensures thread safety. > 3. Since both lookup serialization and deserialization use the same object, this PR also ensures thread safety when serializing and deserializing lookup keys and values. Thanks for your reply, can you split the PR to multiple PRs? Ensure that every change is thoroughly discussed, otherwise there are too many things to discuss in a large PR, making it difficult to review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@paimon.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
