anuengineer commented on issue #108: HDDS-1987. Fix listStatus API URL: https://github.com/apache/hadoop-ozone/pull/108#issuecomment-550034610 > As I discussed with @bharatviswa504 last week, since DB iterator is grabbed after TableCache iteration, there shouldn't really be a missing key from the result. I am not sure why @bharatviswa504 thinks that way. Let me show you a case where you can miss an existing key with this proposed patch. In the above patch, this is what we are trying to do; 1. We read the current list of in-memory keys from the cache. 2. Then we go and read the list of keys from the RocksDB table. 3. Then we merge both the results (and sort?) and return However, between these actions, we take no locks and let our world flow around us. So imagine this; 1. We read the current list of in-memory keys from the cache. **Your thread is now suspended.** Since we hold no locks, it is possible for the double buffer thread to flush changes to the underlying DB. Now imagine, that a user **recreates** a key that is already in your deleted key list. At this point, your thread wakes up and gets a pointer to the underlying DB. 2. You already have a record of the deleted key. Since you took a snapshot of the cache before you got suspended. 3. The current code will walk the deleted key list and assume that DB is making a mistake. 4. Ergo, The list key can miss a fully functional key. Unless I am completely mistaken with this patch, it is possible to miss a fully valid key while listing the keys. Please let me know if I am making a flawed argument.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
