Hi Chris, As you have mentioned, I am trying to split the large csv file in multiple stages. But this error is thrown at the first stage even without creating a single flow file. It seems like the issue is not with the processor, but with the CSV record reader. This error is thrown while reading the csv file. I tried to write the data in the large csv file into a kudu table using a putKudu processor with the same CSV reader. Then also I got the same error message.
Hi Otto, Only following information is available in log file related to the exception *2021-04-14 17:48:28,628 ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.standard.SplitRecord SplitRecord[id=c9a981db-0178-1000-363d-c767653a6f34] SplitRecord[id=c9a981db-0178-1000-363d-c767653a6f34] failed to process session due to java.lang.OutOfMemoryError: Requested array size exceeds VM limit; Processor Administratively Yielded for 1 sec: java.lang.OutOfMemoryError: Requested array size exceeds VM limit* *java.lang.OutOfMemoryError: Requested array size exceeds VM limit* *2021-04-14 17:48:28,628 WARN [Timer-Driven Process Thread-1] o.a.n.controller.tasks.ConnectableTask Administratively Yielding SplitRecord[id=c9a981db-0178-1000-363d-c767653a6f34] due to uncaught Exception: java.lang.OutOfMemoryError: Requested array size exceeds VM limit* *java.lang.OutOfMemoryError: Requested array size exceeds VM limit* Thanks & Regards *Vibhath Ileperuma* On Wed, Apr 14, 2021 at 7:47 PM Otto Fowler <[email protected]> wrote: > What is the complete stack trace of that exception? > > On Apr 14, 2021, at 02:36, Vibhath Ileperuma <[email protected]> > wrote: > > Requested array size exceeds VM limit > > >
