> IMO, the main reason for storing broker ip is that messages are bound to broker, not topic.
I am not sure if this the main reason or not... binding messages to node not topic is obviously a design flaw. > We must find out which broker the message is stored on before we fetch the message. And it is no way to migrate messages from their original broker to another. It is indeed something we need to improve. But it deserves a dedicated RIP. > The new storage layer design should consider this problem, at least not to add new obstacles. Definitely... The design of this RIP is to grant maximum freedom to the business layer and make the storage flexible and adaptable enough. > Eg. Where to store the mapping of topic name and topic identity? Is this mapping consistent across brokers? Ideally, the mapping should be accessible to the whole cluster. The mapping, at least, needs to be consistent among brokers of the same group. In the perspective of the storage layer, it expects the topic ID on topic creation from broker module. On Sun, Oct 16, 2022 at 11:57 AM SSpirits <ad...@lv5.moe> wrote: > Hi Zhanhui, > > IMO, the main reason for storing broker ip is that messages are bound to > broker, not topic. We must find out which broker the message is stored on > before we fetch the message. And it is no way to migrate messages from > their original broker to another. > > The new storage layer design should consider this problem, at least not to > add new obstacles. Eg. Where to store the mapping of topic name and topic > identity? Is this mapping consistent across brokers? > > > On Oct 9, 2022, 21:52 +0800, Zhanhui Li <lizhan...@gmail.com>, wrote: > Hi, > > RocketMQ has stuck to its data format for many years and we are > experiencing many blocking problems originating from this design. > > RIP-47 https://github.com/apache/rocketmq/wiki/RIP-47-Data-Layout-V2 is to > designed to solve the mentioned issues. > > Please review the design. Constructive feedback and comments are > anticipated. > > It is a major RIP to RocketMQ, multiple critical data paths are involved. > It's welcome to join this RIP if you are interested. > > Zhanhui Li >
