[VOTE] KIP-31 - Move to relative offsets in compressed message sets.
Hi, Thanks a lot for the reviews and feedback on KIP-31. It looks all the concerns of the KIP has been addressed. I would like to start the voting process. The short summary for the KIP: We are going to use the relative offset in the message format to avoid server side recompression. In case you haven't got a chance to check, here is the KIP link. https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets Thanks, Jiangjie (Becket) Qin
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1000 ! -Todd On Wednesday, September 23, 2015, Jiangjie Qin wrote: > Hi, > > Thanks a lot for the reviews and feedback on KIP-31. It looks all the > concerns of the KIP has been addressed. I would like to start the voting > process. > > The short summary for the KIP: > We are going to use the relative offset in the message format to avoid > server side recompression. > > In case you haven't got a chance to check, here is the KIP link. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > Thanks, > > Jiangjie (Becket) Qin >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino wrote: > +1000 > > ! > > -Todd > > On Wednesday, September 23, 2015, Jiangjie Qin > wrote: > > > Hi, > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks all the > > concerns of the KIP has been addressed. I would like to start the voting > > process. > > > > The short summary for the KIP: > > We are going to use the relative offset in the message format to avoid > > server side recompression. > > > > In case you haven't got a chance to check, here is the KIP link. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > -- Thanks, Neha
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede wrote: > +1 > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino wrote: > > > +1000 > > > > ! > > > > -Todd > > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > wrote: > > > > > Hi, > > > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks all the > > > concerns of the KIP has been addressed. I would like to start the > voting > > > process. > > > > > > The short summary for the KIP: > > > We are going to use the relative offset in the message format to avoid > > > server side recompression. > > > > > > In case you haven't got a chance to check, here is the KIP link. > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > > > > > -- > Thanks, > Neha >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
I'm +1 though that is dependent on having a graceful migration path for people. -Jay On Wed, Sep 23, 2015 at 5:53 PM, Jiangjie Qin wrote: > Hi, > > Thanks a lot for the reviews and feedback on KIP-31. It looks all the > concerns of the KIP has been addressed. I would like to start the voting > process. > > The short summary for the KIP: > We are going to use the relative offset in the message format to avoid > server side recompression. > > In case you haven't got a chance to check, here is the KIP link. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > Thanks, > > Jiangjie (Becket) Qin >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < aaurad...@linkedin.com.invalid> wrote: > +1 > > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede wrote: > > > +1 > > > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino wrote: > > > > > +1000 > > > > > > ! > > > > > > -Todd > > > > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > > > > wrote: > > > > > > > Hi, > > > > > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks all the > > > > concerns of the KIP has been addressed. I would like to start the > > voting > > > > process. > > > > > > > > The short summary for the KIP: > > > > We are going to use the relative offset in the message format to > avoid > > > > server side recompression. > > > > > > > > In case you haven't got a chance to check, here is the KIP link. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > > > > > Thanks, > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > -- > > Thanks, > > Neha > > > -- -- Guozhang
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang wrote: > +1 > > On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > aaurad...@linkedin.com.invalid> wrote: > > > +1 > > > > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede > wrote: > > > > > +1 > > > > > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino > wrote: > > > > > > > +1000 > > > > > > > > ! > > > > > > > > -Todd > > > > > > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks all > the > > > > > concerns of the KIP has been addressed. I would like to start the > > > voting > > > > > process. > > > > > > > > > > The short summary for the KIP: > > > > > We are going to use the relative offset in the message format to > > avoid > > > > > server side recompression. > > > > > > > > > > In case you haven't got a chance to check, here is the KIP link. > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > > > > > > > Thanks, > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Neha > > > > > > > > > -- > -- Guozhang > -- -Regards, Mayuresh R. Gharat (862) 250-7125
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 on everything but the upgrade plan, which is a bit scary - will comment on the discuss thread. On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat wrote: > +1 > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang wrote: > >> +1 >> >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < >> aaurad...@linkedin.com.invalid> wrote: >> >> > +1 >> > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede >> wrote: >> > >> > > +1 >> > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino >> wrote: >> > > >> > > > +1000 >> > > > >> > > > ! >> > > > >> > > > -Todd >> > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin >> > > > > > >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks all >> the >> > > > > concerns of the KIP has been addressed. I would like to start the >> > > voting >> > > > > process. >> > > > > >> > > > > The short summary for the KIP: >> > > > > We are going to use the relative offset in the message format to >> > avoid >> > > > > server side recompression. >> > > > > >> > > > > In case you haven't got a chance to check, here is the KIP link. >> > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets >> > > > > >> > > > > Thanks, >> > > > > >> > > > > Jiangjie (Becket) Qin >> > > > > >> > > > >> > > >> > > >> > > >> > > -- >> > > Thanks, >> > > Neha >> > > >> > >> >> >> >> -- >> -- Guozhang >> > > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1. I agree that it's worth thinking through the migration plan a bit more. Thanks, Jun On Thu, Sep 24, 2015 at 6:14 PM, Joel Koshy wrote: > +1 on everything but the upgrade plan, which is a bit scary - will > comment on the discuss thread. > > On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat > wrote: > > +1 > > > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang > wrote: > > > >> +1 > >> > >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > >> aaurad...@linkedin.com.invalid> wrote: > >> > >> > +1 > >> > > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede > >> wrote: > >> > > >> > > +1 > >> > > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino > >> wrote: > >> > > > >> > > > +1000 > >> > > > > >> > > > ! > >> > > > > >> > > > -Todd > >> > > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin > >> > >> > > > > >> > > > wrote: > >> > > > > >> > > > > Hi, > >> > > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks > all > >> the > >> > > > > concerns of the KIP has been addressed. I would like to start > the > >> > > voting > >> > > > > process. > >> > > > > > >> > > > > The short summary for the KIP: > >> > > > > We are going to use the relative offset in the message format to > >> > avoid > >> > > > > server side recompression. > >> > > > > > >> > > > > In case you haven't got a chance to check, here is the KIP link. > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > >> > > > > > >> > > > > Thanks, > >> > > > > > >> > > > > Jiangjie (Becket) Qin > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > Thanks, > >> > > Neha > >> > > > >> > > >> > >> > >> > >> -- > >> -- Guozhang > >> > > > > > > > > -- > > -Regards, > > Mayuresh R. Gharat > > (862) 250-7125 >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 -Ewen On Fri, Sep 25, 2015 at 11:15 AM, Jun Rao wrote: > +1. I agree that it's worth thinking through the migration plan a bit more. > > Thanks, > > Jun > > On Thu, Sep 24, 2015 at 6:14 PM, Joel Koshy wrote: > > > +1 on everything but the upgrade plan, which is a bit scary - will > > comment on the discuss thread. > > > > On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat > > wrote: > > > +1 > > > > > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang > > wrote: > > > > > >> +1 > > >> > > >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > > >> aaurad...@linkedin.com.invalid> wrote: > > >> > > >> > +1 > > >> > > > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede > > >> wrote: > > >> > > > >> > > +1 > > >> > > > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino > > >> wrote: > > >> > > > > >> > > > +1000 > > >> > > > > > >> > > > ! > > >> > > > > > >> > > > -Todd > > >> > > > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin > > >> > > >> > > > > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It looks > > all > > >> the > > >> > > > > concerns of the KIP has been addressed. I would like to start > > the > > >> > > voting > > >> > > > > process. > > >> > > > > > > >> > > > > The short summary for the KIP: > > >> > > > > We are going to use the relative offset in the message format > to > > >> > avoid > > >> > > > > server side recompression. > > >> > > > > > > >> > > > > In case you haven't got a chance to check, here is the KIP > link. > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > >> > > > > > > >> > > > > Thanks, > > >> > > > > > > >> > > > > Jiangjie (Becket) Qin > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > -- > > >> > > Thanks, > > >> > > Neha > > >> > > > > >> > > > >> > > >> > > >> > > >> -- > > >> -- Guozhang > > >> > > > > > > > > > > > > -- > > > -Regards, > > > Mayuresh R. Gharat > > > (862) 250-7125 > > > -- Thanks, Ewen
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
Hi folks, Sorry for this prolonged voting session and thanks for the votes. There is an additional broker configuration change added to the KIP after the vote. We propose to add a message.format.version configuration to the broker to indicate which version it should use to store the message on disk. It is mainly trying to minimize the format conversion for consumption during rolling out. Because the client upgrade could take some time and it can be expensive to give up zero-copy for the majority of the consumers, we want to avoid doing that. I would like to see if people have concerns over this change or not. If there is no concerns, I will close the vote as passed. Otherwise I will initiate another vote. Thanks, Jiangjie (Becket) QIn On Fri, Sep 25, 2015 at 4:41 PM, Ewen Cheslack-Postava wrote: > +1 > > -Ewen > > On Fri, Sep 25, 2015 at 11:15 AM, Jun Rao wrote: > > > +1. I agree that it's worth thinking through the migration plan a bit > more. > > > > Thanks, > > > > Jun > > > > On Thu, Sep 24, 2015 at 6:14 PM, Joel Koshy wrote: > > > > > +1 on everything but the upgrade plan, which is a bit scary - will > > > comment on the discuss thread. > > > > > > On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat > > > wrote: > > > > +1 > > > > > > > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang > > > wrote: > > > > > > > >> +1 > > > >> > > > >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > > > >> aaurad...@linkedin.com.invalid> wrote: > > > >> > > > >> > +1 > > > >> > > > > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede > > > > >> wrote: > > > >> > > > > >> > > +1 > > > >> > > > > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino > > > > >> wrote: > > > >> > > > > > >> > > > +1000 > > > >> > > > > > > >> > > > ! > > > >> > > > > > > >> > > > -Todd > > > >> > > > > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > >> > > > >> > > > > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Hi, > > > >> > > > > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It > looks > > > all > > > >> the > > > >> > > > > concerns of the KIP has been addressed. I would like to > start > > > the > > > >> > > voting > > > >> > > > > process. > > > >> > > > > > > > >> > > > > The short summary for the KIP: > > > >> > > > > We are going to use the relative offset in the message > format > > to > > > >> > avoid > > > >> > > > > server side recompression. > > > >> > > > > > > > >> > > > > In case you haven't got a chance to check, here is the KIP > > link. > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > >> > > > > > > > >> > > > > Thanks, > > > >> > > > > > > > >> > > > > Jiangjie (Becket) Qin > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > -- > > > >> > > Thanks, > > > >> > > Neha > > > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> -- Guozhang > > > >> > > > > > > > > > > > > > > > > -- > > > > -Regards, > > > > Mayuresh R. Gharat > > > > (862) 250-7125 > > > > > > > > > -- > Thanks, > Ewen >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
+1 Dong On Tue, Oct 6, 2015 at 2:58 PM, Jiangjie Qin wrote: > Hi folks, > > Sorry for this prolonged voting session and thanks for the votes. > > There is an additional broker configuration change added to the KIP after > the vote. We propose to add a message.format.version configuration to the > broker to indicate which version it should use to store the message on > disk. > > It is mainly trying to minimize the format conversion for consumption > during rolling out. Because the client upgrade could take some time and it > can be expensive to give up zero-copy for the majority of the consumers, we > want to avoid doing that. > > I would like to see if people have concerns over this change or not. If > there is no concerns, I will close the vote as passed. Otherwise I will > initiate another vote. > > Thanks, > > Jiangjie (Becket) QIn > > > On Fri, Sep 25, 2015 at 4:41 PM, Ewen Cheslack-Postava > wrote: > > > +1 > > > > -Ewen > > > > On Fri, Sep 25, 2015 at 11:15 AM, Jun Rao wrote: > > > > > +1. I agree that it's worth thinking through the migration plan a bit > > more. > > > > > > Thanks, > > > > > > Jun > > > > > > On Thu, Sep 24, 2015 at 6:14 PM, Joel Koshy > wrote: > > > > > > > +1 on everything but the upgrade plan, which is a bit scary - will > > > > comment on the discuss thread. > > > > > > > > On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat > > > > wrote: > > > > > +1 > > > > > > > > > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang < > wangg...@gmail.com> > > > > wrote: > > > > > > > > > >> +1 > > > > >> > > > > >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > > > > >> aaurad...@linkedin.com.invalid> wrote: > > > > >> > > > > >> > +1 > > > > >> > > > > > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede < > n...@confluent.io > > > > > > > >> wrote: > > > > >> > > > > > >> > > +1 > > > > >> > > > > > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino < > tpal...@gmail.com > > > > > > > >> wrote: > > > > >> > > > > > > >> > > > +1000 > > > > >> > > > > > > > >> > > > ! > > > > >> > > > > > > > >> > > > -Todd > > > > >> > > > > > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > > >> > > > > >> > > > > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi, > > > > >> > > > > > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It > > looks > > > > all > > > > >> the > > > > >> > > > > concerns of the KIP has been addressed. I would like to > > start > > > > the > > > > >> > > voting > > > > >> > > > > process. > > > > >> > > > > > > > > >> > > > > The short summary for the KIP: > > > > >> > > > > We are going to use the relative offset in the message > > format > > > to > > > > >> > avoid > > > > >> > > > > server side recompression. > > > > >> > > > > > > > > >> > > > > In case you haven't got a chance to check, here is the KIP > > > link. > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > >> > > > > > > > > >> > > > > Thanks, > > > > >> > > > > > > > > >> > > > > Jiangjie (Becket) Qin > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > -- > > > > >> > > Thanks, > > > > >> > > Neha > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> -- > > > > >> -- Guozhang > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > -Regards, > > > > > Mayuresh R. Gharat > > > > > (862) 250-7125 > > > > > > > > > > > > > > > -- > > Thanks, > > Ewen > > >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
The updated upgrade path looks reasonable to me. Not all combinations of the configs are valid though. For example, we probably should disallow message.format.version=1 and intra.cluster.protocol = 0.9.0. Thanks, Jun On Tue, Oct 6, 2015 at 2:58 PM, Jiangjie Qin wrote: > Hi folks, > > Sorry for this prolonged voting session and thanks for the votes. > > There is an additional broker configuration change added to the KIP after > the vote. We propose to add a message.format.version configuration to the > broker to indicate which version it should use to store the message on > disk. > > It is mainly trying to minimize the format conversion for consumption > during rolling out. Because the client upgrade could take some time and it > can be expensive to give up zero-copy for the majority of the consumers, we > want to avoid doing that. > > I would like to see if people have concerns over this change or not. If > there is no concerns, I will close the vote as passed. Otherwise I will > initiate another vote. > > Thanks, > > Jiangjie (Becket) QIn > > > On Fri, Sep 25, 2015 at 4:41 PM, Ewen Cheslack-Postava > wrote: > > > +1 > > > > -Ewen > > > > On Fri, Sep 25, 2015 at 11:15 AM, Jun Rao wrote: > > > > > +1. I agree that it's worth thinking through the migration plan a bit > > more. > > > > > > Thanks, > > > > > > Jun > > > > > > On Thu, Sep 24, 2015 at 6:14 PM, Joel Koshy > wrote: > > > > > > > +1 on everything but the upgrade plan, which is a bit scary - will > > > > comment on the discuss thread. > > > > > > > > On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat > > > > wrote: > > > > > +1 > > > > > > > > > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang < > wangg...@gmail.com> > > > > wrote: > > > > > > > > > >> +1 > > > > >> > > > > >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > > > > >> aaurad...@linkedin.com.invalid> wrote: > > > > >> > > > > >> > +1 > > > > >> > > > > > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede < > n...@confluent.io > > > > > > > >> wrote: > > > > >> > > > > > >> > > +1 > > > > >> > > > > > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino < > tpal...@gmail.com > > > > > > > >> wrote: > > > > >> > > > > > > >> > > > +1000 > > > > >> > > > > > > > >> > > > ! > > > > >> > > > > > > > >> > > > -Todd > > > > >> > > > > > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > > >> > > > > >> > > > > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi, > > > > >> > > > > > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It > > looks > > > > all > > > > >> the > > > > >> > > > > concerns of the KIP has been addressed. I would like to > > start > > > > the > > > > >> > > voting > > > > >> > > > > process. > > > > >> > > > > > > > > >> > > > > The short summary for the KIP: > > > > >> > > > > We are going to use the relative offset in the message > > format > > > to > > > > >> > avoid > > > > >> > > > > server side recompression. > > > > >> > > > > > > > > >> > > > > In case you haven't got a chance to check, here is the KIP > > > link. > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > >> > > > > > > > > >> > > > > Thanks, > > > > >> > > > > > > > > >> > > > > Jiangjie (Becket) Qin > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > -- > > > > >> > > Thanks, > > > > >> > > Neha > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> -- > > > > >> -- Guozhang > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > -Regards, > > > > > Mayuresh R. Gharat > > > > > (862) 250-7125 > > > > > > > > > > > > > > > -- > > Thanks, > > Ewen > > >
Re: [VOTE] KIP-31 - Move to relative offsets in compressed message sets.
Thanks a lot for the reply, Jun. Yes, during implementation we can validate the version configurations to make sure they make sense. Since there haven't been objections in a couple of days. I am just closing this KIP with a pass. KIP-31 is passed with four +1(binding), four +1(non binding) and no -1. Thanks a lot for all the reviews and votes. Jiangjie (Becket) Qin On Thu, Oct 8, 2015 at 6:08 PM, Jun Rao wrote: > The updated upgrade path looks reasonable to me. Not all combinations of > the configs are valid though. For example, we probably should disallow > message.format.version=1 > and intra.cluster.protocol = 0.9.0. > > Thanks, > > Jun > > On Tue, Oct 6, 2015 at 2:58 PM, Jiangjie Qin > wrote: > > > Hi folks, > > > > Sorry for this prolonged voting session and thanks for the votes. > > > > There is an additional broker configuration change added to the KIP after > > the vote. We propose to add a message.format.version configuration to the > > broker to indicate which version it should use to store the message on > > disk. > > > > It is mainly trying to minimize the format conversion for consumption > > during rolling out. Because the client upgrade could take some time and > it > > can be expensive to give up zero-copy for the majority of the consumers, > we > > want to avoid doing that. > > > > I would like to see if people have concerns over this change or not. If > > there is no concerns, I will close the vote as passed. Otherwise I will > > initiate another vote. > > > > Thanks, > > > > Jiangjie (Becket) QIn > > > > > > On Fri, Sep 25, 2015 at 4:41 PM, Ewen Cheslack-Postava < > e...@confluent.io> > > wrote: > > > > > +1 > > > > > > -Ewen > > > > > > On Fri, Sep 25, 2015 at 11:15 AM, Jun Rao wrote: > > > > > > > +1. I agree that it's worth thinking through the migration plan a bit > > > more. > > > > > > > > Thanks, > > > > > > > > Jun > > > > > > > > On Thu, Sep 24, 2015 at 6:14 PM, Joel Koshy > > wrote: > > > > > > > > > +1 on everything but the upgrade plan, which is a bit scary - will > > > > > comment on the discuss thread. > > > > > > > > > > On Thu, Sep 24, 2015 at 9:51 AM, Mayuresh Gharat > > > > > wrote: > > > > > > +1 > > > > > > > > > > > > On Wed, Sep 23, 2015 at 10:16 PM, Guozhang Wang < > > wangg...@gmail.com> > > > > > wrote: > > > > > > > > > > > >> +1 > > > > > >> > > > > > >> On Wed, Sep 23, 2015 at 9:32 PM, Aditya Auradkar < > > > > > >> aaurad...@linkedin.com.invalid> wrote: > > > > > >> > > > > > >> > +1 > > > > > >> > > > > > > >> > On Wed, Sep 23, 2015 at 8:03 PM, Neha Narkhede < > > n...@confluent.io > > > > > > > > > >> wrote: > > > > > >> > > > > > > >> > > +1 > > > > > >> > > > > > > > >> > > On Wed, Sep 23, 2015 at 6:21 PM, Todd Palino < > > tpal...@gmail.com > > > > > > > > > >> wrote: > > > > > >> > > > > > > > >> > > > +1000 > > > > > >> > > > > > > > > >> > > > ! > > > > > >> > > > > > > > > >> > > > -Todd > > > > > >> > > > > > > > > >> > > > On Wednesday, September 23, 2015, Jiangjie Qin > > > > > >> > > > > > >> > > > > > > > > >> > > > wrote: > > > > > >> > > > > > > > > >> > > > > Hi, > > > > > >> > > > > > > > > > >> > > > > Thanks a lot for the reviews and feedback on KIP-31. It > > > looks > > > > > all > > > > > >> the > > > > > >> > > > > concerns of the KIP has been addressed. I would like to > > > start > > > > > the > > > > > >> > > voting > > > > > >> > > > > process. > > > > > >> > > > > > > > > > >> > > > > The short summary for the KIP: > > > > > >> > > > > We are going to use the relative offset in the message > > > format > > > > to > > > > > >> > avoid > > > > > >> > > > > server side recompression. > > > > > >> > > > > > > > > > >> > > > > In case you haven't got a chance to check, here is the > KIP > > > > link. > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Move+to+relative+offsets+in+compressed+message+sets > > > > > >> > > > > > > > > > >> > > > > Thanks, > > > > > >> > > > > > > > > > >> > > > > Jiangjie (Becket) Qin > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > -- > > > > > >> > > Thanks, > > > > > >> > > Neha > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> -- > > > > > >> -- Guozhang > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -Regards, > > > > > > Mayuresh R. Gharat > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Ewen > > > > > >