bump this discuss thread. best, hudeqi
"hudeqi" <16120...@bjtu.edu.cn>写道： > Sorry for not getting email reminders and ignoring your reply for getting > back so late, Yash Mayya, Greg Harris, Sagar. > > Thank you for your thoughts and suggestions, I learned a lot, I will give my > thoughts and answers in a comprehensive way: > 1. The default configuration of 50MB is the online configuration I actually > used to solve this problem, and the effect is better (see the description of > jira:https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-15086?filter=allopenissues. > In fact, I think it may be better to set this value smaller, so I abandoned > the default value like __consumer_offsets, but I don't know how much the > default value is the best.). Secondly, I also set the default value of 50MB > online through ConfigDef#defineInternal, and if the value configured by the > user is greater than the default value, the warning log will be displayed, > but the only difference from your said is that I will overwrite the value > configured by the user with the default value (emmm, this point was denied by > Chris Egerton: https://github.com/apache/kafka/pull/13852, in fact, you all > agree that should not directly override the user-configured value, and now I > agree with this). > 2. I think the potential bug that Greg mentioned may lead to inconsistent > state between workers is a great point. It is true that we cannot directly > change the configuration for an existing internal topics. Perhaps a more > tricky and disgusting approach is that we manually find that the active > segment sizes of all current partitions are relatively small, first stop all > connect instances, then change the topic configuration, and finally start the > instances. > > To sum up, I think whether the scope of the KIP could be reduced to: only set > the default value of the 'segment.bytes' of the internal topics and make a > warning for the bigger value configured by the user. What do you think? If > there's a better way I'm all ears. > > best, > hudeqi
