arunkumarucet opened a new pull request, #18839: URL: https://github.com/apache/pinot/pull/18839
## Summary Fixes a regression introduced in #17593 where `ProtoBufRecordExtractor` throws `IllegalArgumentException: FieldDescriptor does not match message type` after a Confluent Schema Registry Protobuf schema evolves (e.g. a new column is added to the schema). **Root cause:** The field-descriptor cache added in #17593 used `descriptor.getFullName()` equality to detect schema changes. When a new schema version is registered, `KafkaProtobufDeserializer` returns `DynamicMessage` instances built from a *new* `Descriptors.Descriptor` object — same full name, different instance. The stale cache was never invalidated, so `FieldDescriptor` objects from the old `Descriptor` were passed to `DynamicMessage.hasField()` / `getField()` on the new message. Protobuf enforces strict object identity (`fd.getContainingType() != descriptor`), causing the crash. **Fix:** Replace the full-name string comparison with object identity (`!=`). This is cheaper (no string allocation) and correctly detects any descriptor change, including same-name schema evolutions. ```java // Before (insufficient — same full name across schema versions) if (_descriptorFullName == null || !_descriptorFullName.equals(descriptor.getFullName())) // After (correct — new registry version → new Descriptor instance) if (_cachedDescriptor != descriptor) ``` ## Test plan - Added `testSchemaEvolutionSameFullNameDifferentDescriptorInstance` to `ProtoBufRecordExtractorCachingTest` — builds two `Descriptor` instances programmatically with the same full name (`Order`) but different fields (V1: `id`, `amount`; V2: `id`, `amount`, `status`), exactly replicating what Confluent Schema Registry does on schema evolution. Without the fix this test throws `IllegalArgumentException`; with the fix all 13 caching tests pass. - `./mvnw -pl pinot-plugins/pinot-input-format/pinot-protobuf -Dtest=ProtoBufRecordExtractorCachingTest test` → 13/13 pass - `./mvnw spotless:apply checkstyle:check license:check -pl pinot-plugins/pinot-input-format/pinot-protobuf` → clean -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
