[
https://issues.apache.org/jira/browse/METRON-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389724#comment-16389724
]
ASF GitHub Bot commented on METRON-1467:
----------------------------------------
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/947#discussion_r172888378
--- Diff:
metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/JoinBolt.java
---
@@ -89,29 +91,25 @@ public void prepare(Map map, TopologyContext
topologyContext, OutputCollector ou
if (this.maxTimeRetain == null) {
throw new IllegalStateException("maxTimeRetain must be specified");
}
- loader = new CacheLoader<String, Map<String, Tuple>>() {
- @Override
- public Map<String, Tuple> load(String key) throws Exception {
- return new HashMap<>();
- }
- };
- cache = CacheBuilder.newBuilder().maximumSize(maxCacheSize)
- .expireAfterWrite(maxTimeRetain,
TimeUnit.MINUTES).removalListener(new JoinRemoveListener())
- .build(loader);
+ loader = s -> new HashMap<>();
+ cache = Caffeine.newBuilder().maximumSize(maxCacheSize)
+ .expireAfterWrite(maxTimeRetain, TimeUnit.MINUTES)
+ .removalListener(new JoinRemoveListener())
--- End diff --
Yes, it is pre-existing. We can address at a later time.
I remember now, maxing out this cache causes the Split/Join to fail, which
is a major problem for the Split/Join topology. And this cache here is only
for the Split/Join, not the Unified topology.
We should probably look at adding similar logging (only when ERROR enabled)
for the other places where we use the cache. Or just some mechanism to
periodically log cache stats. Anywho, down the road.
> Replace guava caches in places where the keyspace might be large
> ----------------------------------------------------------------
>
> Key: METRON-1467
> URL: https://issues.apache.org/jira/browse/METRON-1467
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
> Priority: Major
>
> Based on the performance tuning exercise as part of METRON-1460, guava has
> difficulties with cache sizes over 10k. We, unfortunately, are quite
> demanding of guava in this regard so we should transition a few uses of guava
> to Caffeine:
> * Stellar processor cache
> * The JoinBolt cache
> * The Enrichment Bolt Cache
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
