exceptionfactory commented on code in PR #10829:
URL: https://github.com/apache/nifi/pull/10829#discussion_r2743710369
##########
nifi-extension-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/avro/AvroReader.java:
##########
@@ -60,12 +60,27 @@ public class AvroReader extends SchemaRegistryService
implements RecordReaderFac
.required(true)
.build();
+ static final PropertyDescriptor FAST_READER_ENABLED = new
PropertyDescriptor.Builder()
+ .name("Fast Reader Enabled")
+ .description("""
+ When enabled, the Avro library uses an optimized reader
implementation that improves read performance
+ by creating a detailed execution plan at initialization.
However, this optimization can lead to
+ significantly higher memory consumption, especially when
using schema inference. If OutOfMemory errors
+ occur during Avro processing, consider disabling this
option.""")
+ .addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+ .allowableValues("true", "false")
+ .defaultValue("true")
Review Comment:
Thanks, it would be helpful to get the exact error message for confirmation.
Inferring the schema seems more likely to result in the memory consumption
issue, based on the way the Avro Fast Reader builds and retains definitions.
For the moment, I'm inclined to proceed with merging the change, as it at
least provides the configuration option. If you can share more details on the
Jira issue, glad to revisit the default setting as needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
