Hey Hudeqi,
I took some time to read through the PR link as well where you and Chris
had an informative discussion.
I think even over there and in this discussion thread, it seems to me that
the consensus is to reduce the scope of the KIP to reduce the default value
of segment.bytes config for
bump this discuss thread.
best,
hudeqi
hudeqi 16120...@bjtu.edu.cn写道：
> Sorry for not getting email reminders and ignoring your reply for getting
> back so late, Yash Mayya, Greg Harris, Sagar.
>
> Thank you for your thoughts and suggestions, I learned a lot, I will give my
> thoughts and
Sorry for not getting email reminders and ignoring your reply for getting back
so late, Yash Mayya, Greg Harris, Sagar.
Thank you for your thoughts and suggestions, I learned a lot, I will give my
thoughts and answers in a comprehensive way:
1. The default configuration of 50MB is the online
Hey Hudeqi,
Thanks for the KIP! After reading the KIP and the comments by Yash and Greg
I agree with these aspects:
1) While I agree that having a high value for segment.btes config can lead
to higher startup times, we don't necessarily need to expose a separate
config for it(as Yash suggested).
Hey hudeqi,
Thanks for the KIP! I did not know about the existing segment.bytes
default value, and it does seem rather high in the context of the
Connect internal topics.
If I think about the segment.size as a "minimum per-partition data
transfer on startup", 1GB is certainly not appropriate for
Also, just adding to the above point - we don't necessarily need to
explicitly add a new worker configuration right? Instead, we could
potentially just use the new proposed default value internally which can be
overridden by users through setting a value for
"offset.storage.segment.bytes" (via the
Hi hudeqi,
Thanks for the KIP! Just to clarify - since KIP-605 (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-605%3A+Expand+Connect+Worker+Internal+Topic+Settings)
already allows configuring "segment.bytes" for the Connect cluster's
offsets topic via a worker configuration
Is anyone following this KIP? Bump this thread.
