[
https://issues.apache.org/jira/browse/FLINK-36609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986477#comment-17986477
]
Sai Sharath Dandi commented on FLINK-36609:
-------------------------------------------
Hi Hongshun,
Thanks for the response.
I believe the confusion may stem from the class being passed to
{{{}Class.forName(){}}}.
When using {{{}ProtoToRowConverter{}}}, we expect
{{formatConfig.getMessageClassName()}} to point to the {*}fully-qualified
message class{*}, such as {{{}com.example.MyMessage{}}}, not the outer class
that represents the proto file.
In that case, calling:
{{MyMessage.getDescriptor()}}
...returns a {{{}Descriptors.Descriptor{}}}, not a {{FileDescriptor}} — even
for PROTO3. This is the behavior generated by {{{}protoc{}}}, and it holds
across both PROTO2 and PROTO3. That's why this line works correctly:
{{return (Descriptors.Descriptor)
pbClass.getMethod("getDescriptor").invoke(null);}}
Our PROTO3 tests pass because we compile the message and correctly point to the
message class in {{{}formatConfig{}}}.
Could you share the message class or config you're using? It's possible the
class you're passing is the {*}enclosing proto file class{*}, which would
indeed return a {{{}FileDescriptor{}}}. If so, switching to the actual message
class should fix the issue.
> Protobuf Format not support proto3
> ----------------------------------
>
> Key: FLINK-36609
> URL: https://issues.apache.org/jira/browse/FLINK-36609
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.20.0
> Reporter: Hongshun Wang
> Assignee: Hongshun Wang
> Priority: Major
> Fix For: 1.20.3
>
> Attachments: image-2024-10-28-14-11-17-931.png
>
>
> Currently, though ProtoToRowConverter try to handle PROTO3 as follows:
> {code:java}
> public ProtoToRowConverter(RowType rowType, PbFormatConfig formatConfig)
> throws PbCodegenException {
> try {
> Descriptors.Descriptor descriptor =
>
> PbFormatUtils.getDescriptor(formatConfig.getMessageClassName());
> Class<?> messageClass =
> Class.forName(
> formatConfig.getMessageClassName(),
> true,
> Thread.currentThread().getContextClassLoader());
> String fullMessageClassName =
> PbFormatUtils.getFullJavaName(descriptor);
> boolean readDefaultValuesForPrimitiveTypes =
> formatConfig.isReadDefaultValues();
> if (descriptor.getFile().getSyntax() == Syntax.PROTO3) {
> // pb3 always read default values for primitive types
> readDefaultValuesForPrimitiveTypes = true;
> }
> //...
> } {code}
> However, protoc will compile message of PROTO3 as
> FileDescriptor rather than Descriptor, thus the following code will throw
> exception:
> !https://alidocs.dingtalk.com/core/api/resources/img/5eecdaf48460cde59040feb26895f4d9b9874851c547dab758e70b814913bc360a414d3de9277d871abf3af1cbd75249a9fcba3598ff5ae2936e5b530f5e23a6779e2fa1453b2c8d24c0547dfa0c758ff70226a219283534fc653b69905bac42?tmpCode=ada2b509-fde1-48f9-81f7-3a57fefa6367!
> {code:java}
> public static Descriptors.Descriptor getDescriptor(String className) {
> try {
> Class<?> pbClass =
> Class.forName(className, true,
> Thread.currentThread().getContextClassLoader());
> return (Descriptors.Descriptor)
>
> pbClass.getMethod(PbConstant.PB_METHOD_GET_DESCRIPTOR).invoke(null);
> } catch (Exception e) {
> throw new IllegalArgumentException(
> String.format("get %s descriptors error!", className), e);
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
