[
https://issues.apache.org/jira/browse/NIFI-16427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alaksiej Ščarbaty updated NIFI-16427:
-------------------------------------
Description:
h2. Summary
_ConfluentProtobufMessageNameResolver_ fails to parse Protobuf schemas
retrieved from Confluent Schema Registry when a schema contains the
{{{}(confluent.field_meta){}}}. [Antlr
grammar|https://github.com/antlr/grammars-v4/blob/master/protobuf/protobuf3/Protobuf3.g4]
doesn't support square brackets in field options. [Protobuf
spec|https://protobuf.dev/reference/protobuf/proto3-spec/#normal_field] doesn't
mention they are supported either.
The failure occurs during schema parsing, while name is being resolved.
h2. Example schema
{code:java}
syntax = "proto3";
package example;
message Value {
optional int32 enabled = 31 [(confluent.field_meta) = {
params: [
{
key: "connect.type",
value: "int16"
}
]
}];
}{code}
h1. Proposed options
_ConfluentProtobufMessageNameResolver_ doesn't use Wire library for parsing.
Instead it uses ANTLR.
As a part of the fix, we won't replace ANTLR with Wire library in
_nifi-confluent-protobuf-message-name-resolver_ module.
Instead, schema cleanup will happen in [{_}StandardProtobufReader{_}, before
compilation|https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-protobuf-bundle/nifi-protobuf-services/src/main/java/org/apache/nifi/services/protobuf/StandardProtobufReader.java#L257].
*Cleaned up schemas will be stored into SchemaDefinitions,*
_messageNameResolver_ will receive _SchemaDefinition_ with already cleaned up
schema.
_Long term we might re-visit the decision to use ANTLR in name resolver._
was:
h2. Summary
_ConfluentProtobufMessageNameResolver_ fails to parse Protobuf schemas
retrieved from Confluent Schema Registry when a schema contains the
{{{}(confluent.field_meta){}}}. [Antlr
grammar|https://github.com/antlr/grammars-v4/blob/master/protobuf/protobuf3/Protobuf3.g4]
doesn't support square brackets in field options. [Protobuf
spec|https://protobuf.dev/reference/protobuf/proto3-spec/#normal_field] doesn't
mention they are supported either.
The failure occurs during schema parsing, while name is being resolved.
h2. Example schema
{code}
syntax = "proto3";
package example;
message Value {
optional int32 enabled = 31 [(confluent.field_meta) = {
params: [
{
key: "connect.type",
value: "int16"
}
]
}];
}{code}
h1. Proposed options
_ConfluentProtobufMessageNameResolver_ doesn't use Wire library for parsing.
Instead it uses ANTLR.
As a part of the fix, we won't replace ANTLR with Wire library in
_nifi-confluent-protobuf-message-name-resolver_ module.
Instead, schema cleanup will happen in [{_}StandardProtobufReader{_}, before
compilation|https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-protobuf-bundle/nifi-protobuf-services/src/main/java/org/apache/nifi/services/protobuf/StandardProtobufReader.java#L257].
*Cleaned up schemas will be stored into SchemaDefinitions,*
_messageNameResolver_ will receive _SchemaDefinition_ with already cleaned up
schema.
> ConfluentProtobufMessageNameResolver fails to parse Confluent Protobuf
> schemas with implicit field_meta options
> ---------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-16427
> URL: https://issues.apache.org/jira/browse/NIFI-16427
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Alaksiej Ščarbaty
> Assignee: Alaksiej Ščarbaty
> Priority: Major
>
> h2. Summary
> _ConfluentProtobufMessageNameResolver_ fails to parse Protobuf schemas
> retrieved from Confluent Schema Registry when a schema contains the
> {{{}(confluent.field_meta){}}}. [Antlr
> grammar|https://github.com/antlr/grammars-v4/blob/master/protobuf/protobuf3/Protobuf3.g4]
> doesn't support square brackets in field options. [Protobuf
> spec|https://protobuf.dev/reference/protobuf/proto3-spec/#normal_field]
> doesn't mention they are supported either.
> The failure occurs during schema parsing, while name is being resolved.
> h2. Example schema
> {code:java}
> syntax = "proto3";
> package example;
> message Value {
> optional int32 enabled = 31 [(confluent.field_meta) = {
> params: [
> {
> key: "connect.type",
> value: "int16"
> }
> ]
> }];
> }{code}
> h1. Proposed options
> _ConfluentProtobufMessageNameResolver_ doesn't use Wire library for parsing.
> Instead it uses ANTLR.
> As a part of the fix, we won't replace ANTLR with Wire library in
> _nifi-confluent-protobuf-message-name-resolver_ module.
> Instead, schema cleanup will happen in [{_}StandardProtobufReader{_}, before
> compilation|https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-protobuf-bundle/nifi-protobuf-services/src/main/java/org/apache/nifi/services/protobuf/StandardProtobufReader.java#L257].
> *Cleaned up schemas will be stored into SchemaDefinitions,*
> _messageNameResolver_ will receive _SchemaDefinition_ with already cleaned up
> schema.
> _Long term we might re-visit the decision to use ANTLR in name resolver._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
