andrekramer1 commented on issue #5896: Out of Memory with many consumers URL: https://github.com/apache/pulsar/issues/5896#issuecomment-576363300 The OOM appear to be in reading from Bookkeeper and possibly the broker is issuing too many read requests at a time. We've had some limited success with the dispatcher throttling introduced in 2.5. Using both: ./bin/pulsar-admin namespaces set-dispatch-rate public/default \ --msg-dispatch-rate 500 \ --dispatch-rate-period 1 ./bin/pulsar-admin namespaces set-subscription-dispatch-rate public/default \ --msg-dispatch-rate 50 \ --dispatch-rate-period 1 allows runs to not OOM most but not all of the time. A rate limit is difficult to use as the number of topics and subscriptions will not be known and we would have to plan for the worst case scenario. One or more new limits on bookie parallel reads seem to be required?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
