[
https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281677#comment-17281677
]
Ben Manes commented on SPARK-34309:
-----------------------------------
FYI, your usage is with Guava’s default concurrency level (4) whereas the
benchmarks are generous / fair by setting to 64. This means you would have a
lower maximum in existing code, but that might be acceptable. An increase can
cause issues with weighted caches by lowering the threshold (max / level).
You might have better timing by using a same thread executor
(Caffeine.executor(Runnable::run)) rather than deferring to FJP. This would
better match Guava.
The improved eviction policy will likely have the most impact on your
performance.
> Use Caffeine instead of Guava Cache
> -----------------------------------
>
> Key: SPARK-34309
> URL: https://issues.apache.org/jira/browse/SPARK-34309
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Affects Versions: 3.2.0
> Reporter: Yang Jie
> Priority: Minor
> Attachments: image-2021-02-05-18-08-48-852.png, screenshot-1.png
>
>
> Caffeine is a high performance, near optimal caching library based on Java 8,
> it is used in a similar way to guava cache, but with better performance. The
> comparison results are as follow are on the [caffeine benchmarks
> |https://github.com/ben-manes/caffeine/wiki/Benchmarks]
> At the same time, caffeine has been used in some open source projects like
> Cassandra, Hbase, Neo4j, Druid, Spring and so on.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
