Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-12-15 Thread via GitHub
merlimat closed issue #21442: [Bug] RateLimiter lock contention when use precise publish rate limiter URL: https://github.com/apache/pulsar/issues/21442 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-12-10 Thread via GitHub
lhotari commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1849380390 @Shawyeok There's now "PIP-322: Pulsar Rate Limiting Refactoring" (#21680) to address this issue. The changes are in the draft PR #21681 . Please review! Thanks -- This is an aut

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-22 Thread via GitHub
lhotari commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1823569304 > @lhotari Cool! I will study and verify your solution as soon as possible after my vacation, which may be a few days. Sounds good. Please check the blog post https://cod

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-22 Thread via GitHub
Shawyeok commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1822778672 > > Of course, we can also optimize the implementation of `org.apache.pulsar.common.util.RateLimiter`. Perhaps an implementation based on Compare-And-Swap (CAS) would perform bette

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-22 Thread via GitHub
lhotari commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1822660666 > Hello, I am also learning pulsar recently, have you found a solution to this problem? @1625567290 Just curious to learn about your motivation to start learning Pulsar here.

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-21 Thread via GitHub
lhotari commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1820668270 > Of course, we can also optimize the implementation of `org.apache.pulsar.common.util.RateLimiter`. Perhaps an implementation based on Compare-And-Swap (CAS) would perform bett

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-16 Thread via GitHub
Shawyeok commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1814781576 > > Sorry for late reply. > > The current publish rate limiting is based on setting the `autoRead` flag of the Netty channel to false. However, `autoRead` is just a hint in Nett

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-16 Thread via GitHub
Shawyeok commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1814780242 In Pulsar internal, there are multiple topics that depend on the same thread for message writing (such as Netty's EventLoop and OrderedExecutor). If, for some reason, these threads

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-15 Thread via GitHub
1625567290 commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1812704939 > Sorry for late reply. > > The current publish rate limiting is based on setting the `autoRead` flag of the Netty channel to false. However, `autoRead` is just a hint in N

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-06 Thread via GitHub
Shawyeok commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1797947739 There is a related discuss in mail list: https://lists.apache.org/thread/13ncst2nc311vxok1s75thl2gtnk7w1t -- This is an automated message from the Apache Git Service. To respond

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-11-03 Thread via GitHub
Shawyeok commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1792001008 Sorry for late reply. The current publish rate limiting is based on setting the `autoRead` flag of the Netty channel to false. However, `autoRead` is just a hint in Netty, no

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-10-26 Thread via GitHub
1625567290 commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1782248694 ![image](https://github.com/apache/pulsar/assets/79852400/472dac4b-f770-41a8-acca-a4bbc0d6438d) ![image](https://github.com/apache/pulsar/assets/79852400/320a27eb-310b-45c

Re: [I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-10-26 Thread via GitHub
1625567290 commented on issue #21442: URL: https://github.com/apache/pulsar/issues/21442#issuecomment-1782200083 Hello, I am also learning pulsar recently, have you found a solution to this problem? -- This is an automated message from the Apache Git Service. To respond to the message, pl

[I] [Bug] RateLimiter lock contention when use precise publish rate limiter [pulsar]

2023-10-25 Thread via GitHub
Shawyeok opened a new issue, #21442: URL: https://github.com/apache/pulsar/issues/21442 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version After enabled the `preciseTopicPublishRat