> > And you can see the difference between ProtoBuf and ProtoBufNative: > > https://pulsar.apache.org/docs/next/schema-get-started/#protobufnative > > https://pulsar.apache.org/docs/next/schema-get-started/#protobuf > I read it and they look identical. What's the difference between them?
Current avro,json, and protobuf schemas are all implemented based on AVRO. What do you mean, they are all implemented based on Avro? You mean the protobuf schema is converted into an Avro Schema, and then you use Avro compatibility validation? > *Here are the basic compatibility rules we've defined:* I think you should structure the validation rules differently: * Backward checks ** List down rules, where use newSchema (the schema used by producer or consumer) and existingSchema (last schema used) * Forward ** List down rules, where use newSchema (the schema used by producer or consumer) and existingSchema (last schema used) So each strategy should have its own section. I'm saying this since you used "writttenSchema" word but it represents something completely different if it's backward or forward check. Once you'll have that structure like that, I personally will be able to read and understand it. The motivation and problem statement are now good - thanks for improving it. On Mon, Feb 27, 2023 at 8:20 AM SiNan Liu <liusinan1...@gmail.com> wrote: > Hi! I updated the PIP issue again. This time I've added some background and > some explanations. > > The compatibility check rules are already written in the Implementation. > ProtoBufNative implements the same canRead method as Apache Avro. > It does this by checking whether the schema for writing and reading is > compatible. I also indicate whether the writtenSchema and readSchema of the > Backward, Forward, and Full strategies are the old or the new version of > the schema. > > Thanks, > sinan > > Asaf Mesika <asaf.mes...@gmail.com> 于2023年2月26日周日 23:24写道： > > > I'm sorry, but this PIP lacks a lot of background knowledge, so you need > to > > add IMO for people to understand it. You don't need to explain the entire > > pulsar in this PIP, but at the very least a few paragraphs detailing all > > you need to know, to put you in context: > > > > > > - Start by saying Pulsar as a built-in schema registry inside Pulsar > > broker. > > - Every time the client updates the schema, it uploads it to the > > broker. When that happens, it has a feature which validates if the > > new > > schema version is compatible with the previous versions. There > > are 4 types > > of compatibility: Full, ... (complete and explain each one briefly) > > - Also explain Pulsar Schema registry supports various schema > > protocols: Avro, protobuf native, ... (complete the rest), each > > protocol > > has a schema which dictates how to serialize and deserialize the > message > > content into typed object. > > - Explain in short what is protobuf native (compare protobuf > non-native) > > - Please don't paste code instead of explaining. > > - Explain that protobuf native current validation check is only > > composed of checking the root message name is the same between > > the current > > schema version and the new version. > > - Explain briefly what is a root message and its name. > > - Explain the problem (list scenarios) that we have because > protobuf > > native schema only supports FULL compatibility validation. > > > > > > Regarding high level design - as in what you plan to do. > > I suggest you add "High Level Design" and in it detail how you plan to > > validate, per protobuf version, per compatibility check (backward, > forward, > > full,...). > > I tried reading the implementation - for me , it's all over the place. > Can > > you please list in order what I wrote above, and list the validation > rules > > with a good explanation why you validate it like that? > > > > Lastly, one you have all the validation rules clearly stated, you can use > > it to document it properly so users can know what validation to expect. > > > > Thanks, > > > > Asaf > > > > > > On Wed, Feb 22, 2023 at 5:10 PM SiNan Liu <liusinan1...@gmail.com> > wrote: > > > > > Sorry, my mistake. I removed the code and described the design to > improve > > > the PROTOBUF_NATIVE schema compatibility checks. You can have a look. > 😊 > > > > > > Asaf Mesika <asaf.mes...@gmail.com> 于2023年2月22日周三 21:16写道： > > > > > > > I read it but you're almost directly diving into the code - it will > > take > > > me > > > > hours just to reverse engineer your design. > > > > > > > > Can you please include a "High Level Design" section in which you > > explain > > > > how you plan to tackle any issue? > > > > If I can read that section and explain to someone else how this will > > > work, > > > > it means the section is complete. > > > > > > > > Let's leave the code to the PRs. > > > > > > > > > > > > On Sun, Feb 19, 2023 at 2:59 PM SiNan Liu <liusinan1...@gmail.com> > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I made a PIP to discuss: > > https://github.com/apache/pulsar/issues/19565 > > > . > > > > > > > > > > We can talk about the current design here. Especially for the field > > > type > > > > > change check rules, please give your valuable advice. > > > > > > > > > > Thanks, > > > > > Sinan > > > > > > > > > > > > > > >
