Couple of things I think we still need to work out: 1. I think we agree about the key, but I think we haven't talked about the value yet. I think if our goal is an open ecosystem of these header spread across many plugins from many systems we should consider making this a string as well so it can be printed, set via a UI, set in config, etc. Basically encouraging pluggable serialization formats here will lead to a bit of a tower of babel. 2. This proposal still includes a pretty big change to our serialization and protocol definition layer. Essentially it is introducing an optional type, where the format is data dependent. I think this is actually a big change though it doesn't seem like it. It means you can no longer specify this type with our type definition DSL, and likewise it requires custom handling in client libs. This isn't a huge thing, since the Record definition is custom anyway, but I think this kind of protocol inconsistency is very non-desirable and ties you to hand-coding things. I think the type should instead by [Key Value] in our BNF, where key and value are both short strings as used elsewhere. This brings it in line with the rest of the protocol. 3. Could we get more specific about the exact Java API change to ProducerRecord, ConsumerRecord, Record, etc?
-Jay On Tue, Feb 14, 2017 at 9:42 AM, Michael Pearce <michael.pea...@ig.com> wrote: > Hi all, > > We would like to start the voting process for KIP-82 – Add record headers. > The KIP can be found > at > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 82+-+Add+Record+Headers > > Discussion thread(s) can be found here: > > http://search-hadoop.com/m/Kafka/uyzND1nSTOHTvj81?subj= > Re+DISCUSS+KIP+82+Add+Record+Headers > http://search-hadoop.com/m/Kafka/uyzND1Arxt22Tvj81?subj= > Re+DISCUSS+KIP+82+Add+Record+Headers > http://search-hadoop.com/?project=Kafka&q=KIP-82 > > > > Thanks, > Mike > > The information contained in this email is strictly confidential and for > the use of the addressee only, unless otherwise indicated. If you are not > the intended recipient, please do not read, copy, use or disclose to others > this message or any attachment. Please also notify the sender by replying > to this email or by telephone (+44(020 7896 0011) and then delete the email > and any copies of it. Opinions, conclusion (etc) that do not relate to the > official business of this company shall be understood as neither given nor > endorsed by it. IG is a trading name of IG Markets Limited (a company > registered in England and Wales, company number 04008957) and IG Index > Limited (a company registered in England and Wales, company number > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > Index Limited (register number 114059) are authorised and regulated by the > Financial Conduct Authority. >
