If we can be sure as per Magnus mentions then I'd actually prefer not to be bumping the magic bit, as we take the default as false as such no behaviour change.
I had included the magic bit and api changes only because of uncertainty how true that by default the flags should be false. The less change the less intrusive we are the better imo (this was my intent on kip-82 also), here we can simply update the protocol to state the attribute bit is used for tombstone flag and make the server side change and update the Java client. After all the attributes as mentioned in kip-82 meetings are for essentially core Kafka header flags as such we should as per our arguement for headers be able to allocate and use without a wire protocol change. It be damn ugly if we had to do this every time we need to support a new header value for a feature. If this isn't true then I think that's an argument further for headers (alas I think though it is and as such we should be able to) Like wise I think typically the upgrade path so far has been to upgrade the broker then the clients. ________________________________________ From: Nacho Solis <nso...@linkedin.com.INVALID> Sent: Tuesday, October 25, 2016 8:36:24 PM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone Flag I think you probably require a MagicByte bump if you expect correct behavior of the system as a whole. From a client perspective you want to make sure that when you deliver a message that the broker supports the feature you're expecting (compaction). So, depending on the behavior of the broker on encountering a previously undefined bit flag I would suggest making some change to make certain that flag-based compaction is supported. I'm going to guess that the MagicByte would do this. While we're at it, we might as well see if anything else needs to be changed. BTW, this could potentially be handled differently in other circumstances, like when you're expecting the flag to be interpreted end-to-end. Nacho (I haven't looked at the specific code so maybe my assumptions are wrong) On Tue, Oct 25, 2016 at 10:17 AM, Magnus Edenhill <mag...@edenhill.se> wrote: > It is safe to assume that a previously undefined attributes bit will be > unset in protocol requests from existing clients, if not, such a client is > already violating the protocol and needs to be fixed. > > So I dont see a need for a MagicByte bump, both broker and client has the > information it needs to construct or parse the message according to request > version. > > > 2016-10-25 18:48 GMT+02:00 Michael Pearce <michael.pea...@ig.com>: > > > Hi Magnus, > > > > I was wondering if I even needed to change those also, as technically > > we’re just making use of a non used attribute bit, but im not 100% that > it > > be always false currently. > > > > If someone can say 100% it will already be set false with current and > > historic bit wise masking techniques used over the time, we could do away > > with both, and simply just start to use it. Unfortunately I don’t have > that > > historic knowledge so was hoping it would be flagged up in this > discussion > > thread ☺ > > > > Cheers > > Mike > > > > On 10/25/16, 5:36 PM, "Magnus Edenhill" <mag...@edenhill.se> wrote: > > > > Hi Michael, > > > > With the version bumps for Produce and Fetch requests, do you really > > need > > to bump MagicByte too? > > > > Regards, > > Magnus > > > > > > 2016-10-25 18:09 GMT+02:00 Michael Pearce <michael.pea...@ig.com>: > > > > > Hi All, > > > > > > I would like to discuss the following KIP proposal: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > 87+-+Add+Compaction+Tombstone+Flag > > > > > > This is off the back of the discussion on KIP-82 / KIP meeting > > where it > > > was agreed to separate this issue and feature. See: > > > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610. > > > mbox/%3cCAJS3ho8OcR==EcxsJ8OP99pD2hz=iiGecWsv- > > > EZsBsNyDcKr=g...@mail.gmail.com%3e > > > > > > Thanks > > > Mike > > > > > > The information contained in this email is strictly confidential > and > > for > > > the use of the addressee only, unless otherwise indicated. If you > > are not > > > the intended recipient, please do not read, copy, use or disclose > to > > others > > > this message or any attachment. Please also notify the sender by > > replying > > > to this email or by telephone (+44(020 7896 0011) and then delete > > the email > > > and any copies of it. Opinions, conclusion (etc) that do not relate > > to the > > > official business of this company shall be understood as neither > > given nor > > > endorsed by it. IG is a trading name of IG Markets Limited (a > company > > > registered in England and Wales, company number 04008957) and IG > > Index > > > Limited (a company registered in England and Wales, company number > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate > > Hill, > > > London EC4R 2YA. Both IG Markets Limited (register number 195355) > > and IG > > > Index Limited (register number 114059) are authorised and regulated > > by the > > > Financial Conduct Authority. > > > > > > > > > The information contained in this email is strictly confidential and for > > the use of the addressee only, unless otherwise indicated. If you are not > > the intended recipient, please do not read, copy, use or disclose to > others > > this message or any attachment. Please also notify the sender by replying > > to this email or by telephone (+44(020 7896 0011) and then delete the > email > > and any copies of it. Opinions, conclusion (etc) that do not relate to > the > > official business of this company shall be understood as neither given > nor > > endorsed by it. IG is a trading name of IG Markets Limited (a company > > registered in England and Wales, company number 04008957) and IG Index > > Limited (a company registered in England and Wales, company number > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > > Index Limited (register number 114059) are authorised and regulated by > the > > Financial Conduct Authority. > > > -- Nacho (Ignacio) Solis Kafka nso...@linkedin.com The information contained in this email is strictly confidential and for the use of the addressee only, unless otherwise indicated. If you are not the intended recipient, please do not read, copy, use or disclose to others this message or any attachment. Please also notify the sender by replying to this email or by telephone (+44(020 7896 0011) and then delete the email and any copies of it. Opinions, conclusion (etc) that do not relate to the official business of this company shall be understood as neither given nor endorsed by it. IG is a trading name of IG Markets Limited (a company registered in England and Wales, company number 04008957) and IG Index Limited (a company registered in England and Wales, company number 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG Index Limited (register number 114059) are authorised and regulated by the Financial Conduct Authority.
