gianm commented on a change in pull request #8116: remove unnecessary lock in ForegroundCachePopulator leading to a lot of contention URL: https://github.com/apache/incubator-druid/pull/8116#discussion_r306044913
########## File path: server/src/main/java/org/apache/druid/client/cache/BackgroundCachePopulator.java ########## @@ -38,6 +38,14 @@ import java.util.concurrent.ExecutorService; import java.util.function.Function; +/** + * {@link CachePopulator} implementation that uses a {@link ExecutorService} thread pool to populate a cache in the + * background. Used if config "druid.*.cache.numBackgroundThreads" is greater than 0. If maximum cache entry size, + * specified by config "druid.*.cache.maxEntrySize", is exceeded, this {@link CachePopulator} implementation will + * be a bit less efficient than {@link ForegroundCachePopulator} which can stop retaining values early, in exchange Review comment: This comment has some details - https://github.com/apache/incubator-druid/pull/8116#discussion_r306041975 I do think it's too inside-baseball for user-facing docs, and maybe even for javadocs. It's referring to the fact that the foreground populator avoids some work, relative to the background populator, in the case that maxEntrySize is exceeded. In particular, when building a cache entry from a `Sequence<T>`, the background populator will run `cacheFn` on all `T` first, and then when they're all done it will start serializing them one by one. At that point, if maxEntrySize is exceeded, it will throw the rest away and exit early without populating the cache. But it still went through the effort of computing `cacheFn` on the `T`s that it threw away. The foreground populator, on the other hand, does not do that. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org