[GitHub] [pulsar] merlimat commented on issue #6791: Use consistent hashing in KeyShared distribution

2020-04-23 Thread GitBox
merlimat commented on issue #6791: URL: https://github.com/apache/pulsar/pull/6791#issuecomment-618531886 > I see there is a problem in the current algorithm when scaling down the number of consumers. But I don't think its broken. The Key Shared subscriptions has 2 stated goals:

[GitHub] [pulsar] merlimat commented on issue #6791: Use consistent hashing in KeyShared distribution

2020-04-23 Thread GitBox
merlimat commented on issue #6791: URL: https://github.com/apache/pulsar/pull/6791#issuecomment-618223059 > Since the current algorithm has been used in many production environments already, we should be concerned about removing an existing implementation. Providing a way to opt-in a diffe

[GitHub] [pulsar] merlimat commented on issue #6791: Use consistent hashing in KeyShared distribution

2020-04-22 Thread GitBox
merlimat commented on issue #6791: URL: https://github.com/apache/pulsar/pull/6791#issuecomment-618176136 > If users want to more evenly distributed, they will increase the points. I'm worried about too many consumers(e.g. 5000 consumers) may result in too many nodes in the TreeMap.

[GitHub] [pulsar] merlimat commented on issue #6791: Use consistent hashing in KeyShared distribution

2020-04-22 Thread GitBox
merlimat commented on issue #6791: URL: https://github.com/apache/pulsar/pull/6791#issuecomment-618109262 @codelipenghui AUTO_SPLIT and CONSISTENT_HASHING are essentially 2 different implementations for the same stated behavior. In my view, we should just give the user one good default opt

[GitHub] [pulsar] merlimat commented on issue #6791: Use consistent hashing in KeyShared distribution

2020-04-22 Thread GitBox
merlimat commented on issue #6791: URL: https://github.com/apache/pulsar/pull/6791#issuecomment-617958166 > I need to rethink how to leverage this idea on the consistent hashing approach. If you have any idea, please ping me, thanks. @codelipenghui Thanks for pointing to this. I have