Hi all,
In the exclusive consumption mode of exclusive, failover, reader, etc. if a
large number of messages accumulate due to limited consumption capacity or
consumption logic problems, expanding partitions or modifying consumption logic
cannot reduce the consumption of new messages in time. Influence, a certain
proportion of newly added messages will still be produced in the accumulated
paritition, and new messages cannot be quickly consumed by restarted consumers
or new consumers. In this way, the time affected by the accumulation factor may
very long.
In order to reduce the impact on the business, through the control of read and
write permissions, configure some partitions as read-only, so that newly
produced messages can be sent to the partitions with less accumulation or newly
expanded, reducing the impact on the business caused by the accumulation of
messages. Reduce the overall time-consuming of production--consumption.
This feature needs to be changed as follows：
1、PB protocl
message CommandPartitionedTopicMetadataResponse {
enum LookupType {
Success = 0;
Failed = 1;
}
optional uint32 partitions = 1; // Optional in case of error
required uint64 request_id = 2;
optional LookupType response = 3;
optional ServerError error = 4;
optional string message = 5;
repeated KeyValue metadata = 6；// key parition index，value：r、w、rw
}
2、Modify choosePartition method in all implement class of MessageRouter, and it
will not choose read-only partition to send msg.
3、ProducerImpl adds permission attributes, and if permission is read-only will
not be used sending msg.
4、Modify admin interface updatePartitionedTopic to update metadata of topic
Thanks,
Mingyu.
