Josef Zahner created NIFI-15526:
-----------------------------------
Summary: OOM triggered by AVRO org.apache.avro.fastread
Key: NIFI-15526
URL: https://issues.apache.org/jira/browse/NIFI-15526
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 2.7.2, 2.7.1
Reporter: Josef Zahner
Attachments: image-2026-01-29-16-14-08-609.png,
image-2026-01-29-16-14-29-698.png, image-2026-01-29-16-16-01-232.png
We found an issue causes OOM (used java heap full) for NiFi > v2.5.0 if we use
AVRO Readers/Writers. It seems that the AVRO Library used in the newest NiFi
versions has AVRO {{{}org.apache.avro.fastread{}}}=true per default. This
causes massive OOM problems on our side. We suspect it is due to the
inheritance of the reader/writer schemas.
If we disable fastread in {{{}/conf/bootstrap.conf{}}}, the memory leak is gone:
{{}}
{code:java}
java.arg.28=-Dorg.apache.avro.fastread=false{code}
{{}}
Here real world outputs based on prometheus metrics from used heap for NiFi.
The screenshot 1 (blue line) shows a max 32GB heap, which never goes down
again, it just grows (until it would reach 32GB and crashes). Thats taken with
NiFI 2.7.2 with nothing changed regarding AVRO fast read (default, seems to be
enabled).
!image-2026-01-29-16-14-08-609.png!
Screenshot 2 (orange line) shows another cluster with more or less the same use
case, but with fastread=false. No memory leak anymore. So it's clearly related
to the AVRO fastread feature.
!image-2026-01-29-16-14-29-698.png!
Yourkit Profiler example of the FastReaderBuilder object and memory usage. 1
Object consumes about 10GB memory (fastread is on default, so enabled) .
!image-2026-01-29-16-16-01-232.png!
Would be great of someone of the devs is more familiar with all this topics
could investigate it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
