jnioche commented on code in PR #1700:
URL: https://github.com/apache/stormcrawler/pull/1700#discussion_r2452025527
##########
external/opensearch/src/main/java/org/apache/stormcrawler/opensearch/persistence/StatusUpdaterBolt.java:
##########
@@ -160,11 +159,16 @@ public void prepare(
fieldNameForRoutingKey = fieldNameForRoutingKey.replaceAll("\\.",
"%2E");
}
- waitAck =
- Caffeine.newBuilder()
- .expireAfterWrite(60, TimeUnit.SECONDS)
- .removalListener(this)
- .build();
+ String defaultSpec =
String.format("maximumSize=10000,expireAfterWrite=%ds",
Review Comment:
Maybe we should not limit on size if we rely on the`
topology.message.timeout.secs ` - users can override it if needed
Entries are removed from the cache so size should never be an issue +
limiting on size can lead to the issues highlighted in #1698
what do you think?
##########
external/opensearch/opensearch-conf.yaml:
##########
@@ -114,6 +114,10 @@ config:
opensearch.status.recentDate.increase: -1
opensearch.status.recentDate.min.gap: -1
+ # Caffeine cache specification for the waitAck cache used in
StatusUpdaterBolt.
+ # Set expireAfterWrite to match topology.message.timeout.secs (default:
300s),
+ opensearch.status.waitack.cache.spec:
"maximumSize=10000,expireAfterWrite=300s"
Review Comment:
comment out the config otherwise the default will never be read
Change ` # Set expireAfterWrite to match topology.message.timeout.secs
(default: 300s),`
to something like `"if not set, the value of topology.message.timeout.secs
is used for expireAfterWrite "`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
