deniskuzZ commented on PR #4946: URL: https://github.com/apache/hive/pull/4946#issuecomment-1867916262
> > > > Could we replace this map-based cache with caffeine cache? > > > > > > > > > I think we can not do that. Because if we just change it to caffeine, it will still trigger properties.equals(). If we don't use static-cache, there is no point in using caffeine at all. > > > > > > Cache is currently reused between multiple tasks. Caffeine was proposed as a way to define an eviction policy. I am checking the code cause existing structure `Map<Map<Path, PartitionDesc>, Map<Path, PartitionDesc>>` looks very weird. > > yea. i think so... very strange from [HIVE-9976](https://issues.apache.org/jira/browse/HIVE-9976) > There's a static cache in there which seems a little strange. Will create a follow up jira to investigate this. For now I've changed that to a ConcurrentMap since split generation can run in parallel. is there an existing test that makes use of `cacheMap` in `getFromPathRecursively`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
