[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-03 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-789959546 > I have a WIP solution already. Need to look it a bit more to see if I'm missing anything and I'll create a PR. Ok great. I'm looking forward to that. I'll close this

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-03 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-789956789 > To conclude, in my opinion the easiest/lower-risk/lower-overhead change here would be to switch the forEach() to do the stamped+read-lock combination for each entry, instead of

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-03 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-789608339 > how can we fall into a deadlock ? Not an easy question. I don't have an answer to that. Perhaps @Vanlightly could help us? -

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-03 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-789562722 > This map is used in many places throughout the code base and, depending on the use cases, there can be instances with a lot of entries or many instances with few entries. Also,

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-02 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-788764981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-01 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-788693090 /pulsarbot run-failure-checks This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-01 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-788689691 > The primary goal of having these custom concurrent maps implementation was actually to avoid memory allocations to the max possible extent :) > > Even if these allocation

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-01 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-787887121 Here's an example of a suspicious stacktrace from one of the ReplicatorTest timeouts ([full threaddump](https://jstack.review/?https://gist.github.com/lhotari/3d2aa04041f86645622

[GitHub] [pulsar] lhotari commented on pull request #9764: [Common] Reduce duration of locks when processing items held in ConcurrentOpenHashMaps

2021-03-01 Thread GitBox
lhotari commented on pull request #9764: URL: https://github.com/apache/pulsar/pull/9764#issuecomment-787866127 > But I am not sure we can touch so many sensible parts of the codebase in one single patch. > We could break things without knowing and it will be hard to track down to the c