geniusjoe commented on issue #25021: URL: https://github.com/apache/pulsar/issues/25021#issuecomment-3605549241
> I‘m trying to reproduce the issue, so I'd like to ask what this do you mean, `managedLedgerCacheSizeMB =0` ? > > > Make sure broker ledger cache is always missed @Technoboy- Thank you very much for your reply. I am currently using the default `managedLedgerCache` settings from broker.conf without any modifications. To achieve "broker ledger cache is always missed" I think we need to ensure that subscriptions are not all started at once. For example, starting a new subscription every 10 seconds. As long as there is a message backlog in the current subscription, the ledger cache is generally not hit. The relevant configurations used for managedLedgerCacheare as follows: ``` # Amount of memory to use for caching data payload in managed ledger. This memory # is allocated from JVM direct memory and it's shared across all the topics # running in the same broker. By default, uses 1/5th of available direct memory managedLedgerCacheSizeMB= # Whether we should make a copy of the entry payloads when inserting in cache managedLedgerCacheCopyEntries=false # Threshold to which bring down the cache level when eviction is triggered managedLedgerCacheEvictionWatermark=0.9 # Configure the cache eviction frequency for the managed ledger cache (evictions/sec) managedLedgerCacheEvictionFrequency=100.0 # All entries that have stayed in cache for more than the configured time, will be evicted managedLedgerCacheEvictionTimeThresholdMillis=1000 # Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged' # and thus should be set as inactive. managedLedgerCursorBackloggedThreshold=1000 # Rate limit the amount of writes per second generated by consumer acking the messages managedLedgerDefaultMarkDeleteRateLimit=1.0 ``` I'm not entirely sure whether the current issue of frequent memory allocation and deallocation leading to inconsistent physical memory usage, is related to glibc or the Linux distribution version. However, based on my observations across different clusters, this problem appears more likely to occur with older JDK versions. If you want to reproduce this issue, I would recommend first trying with Broker v2.10 and JDK 8 in your Linux environment. If the problem can be reproduced, you could then upgrade the Broker version to the currently maintained v4.x LTS release for further observation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
