Thanks for your explanation. Reusing the corrupted recycle object is dangerous, and it's hard to investigate issues from the heap dump.
Thanks, Ran Gao On 2025/07/17 13:38:38 Lari Hotari wrote: > Recycling can cause bugs that are very hard to detect. The main reason to > avoid recycling in exceptional cases is to prevent such bugs. One possible > scenario is when another thread continues to hold a reference to an object > instance. After the object instance has been recycled, it could refer to a > completely different object instance. We have encountered such bugs in the > past with Pulsar, which is why it's better to avoid the risk of these bugs > altogether. > > Netty doesn't recycle all object instances when recycling is enabled. There's > currently a per-thread limit of 4,096 object instances by default. Omitting > recycling in exceptional paths won't add noticeable extra GC pressure because > many instances will be discarded in any case, due to recycling cache overflow > when using default Netty settings. > > -Lari > > On 2025/07/17 12:47:22 Yunze Xu wrote: > > Hi all, > > > > I noticed there is a difference of opinion for whether to recycle > > failed recyclable objects. [1][2][3] So I'd like to open the > > discussion in the mail list. > > > > Netty recycler is an object pool to reduce GC pressure for frequently > > created and discarded objects. If a recyclable object fails, e.g. when > > an `OpAddEntry` encounters an unexpected exception, IMO, it's > > reasonable not to recycle it. > > > > When a recyclable object is not recycled, it just won't be reused from > > the object pool. The worst case is that the memory of the whole object > > pool is exhausted. After that > > > > It's true that if such objects are not recycled, there will be a > > "memory leak". However, the worst result is that the pool memory is > > exhausted. In this case, it will fall back to allocating objects via > > the `newObject` method with a dummy handle [4]. The implementation of > > this method usually allocates memory from JVM. In such cases, the > > recyclable object will eventually be garbage collected. > > > > It's actually not a memory leak. Recyclable objects usually have short > > life timing, for example, an `OpAddEntry` object is created when > > starting an asynchronous send and recycled after the send is done. The > > object is never referenced by any other object, so even if it's > > allocated from JVM, it will eventually be garbage collected. > > > > The benefit to skip recycling objects for failures is that for rare > > cases, retaining the object could help diagnose issues that are hard > > to reproduce. Still take `OpAddEntry` for example, if it encountered > > an unexpected exception, recycling it could set all fields with null, > > so it's hard to know which ledger it belongs to (and other useful > > fields) from the heap dump. > > > > [1] https://github.com/apache/pulsar/pull/24515#discussion_r2212602435 > > [2] https://github.com/apache/pulsar/pull/24522#discussion_r2210814071 > > [3] https://github.com/apache/pulsar/pull/24522#discussion_r2213100905 > > [4] > > https://github.com/netty/netty/blob/ab2d2cf3ff3a6f055368405af7f6e9dd5b8d144e/common/src/main/java/io/netty/util/Recycler.java#L189 > > > > Thanks, > > Yunze > > >
