RE: [EXT] Re: FlowFile Repository can't checkpoint, out of heap space.

2019-08-15 Thread Peter Wicks (pwicks)
aInputStream.close(); System.out.println(file + " - " + saved + " / " + total); } From: Joe Witt Sent: Thursday, August 15, 2019 10:58 AM To: users@nifi.apache.org Subject: Re: [EXT] Re: FlowFile Repository can't checkpoint, out of heap space. Peter All the deta

Re: [EXT] Re: FlowFile Repository can't checkpoint, out of heap space.

2019-08-15 Thread Joe Witt
Peter All the details you can share on this would be good. First, we should be resilient to any sort of repo corruption in the event of heap issues. While obviously the flow isn't in a good state at that point the saved state should be reliable/recoverable. Second, how the repo/journals got that

RE: [EXT] Re: FlowFile Repository can't checkpoint, out of heap space.

2019-08-15 Thread Peter Wicks (pwicks)
We were able to recover this morning, in the end we deleted the queues that were causing trouble from the Flow, and when the problem node came online it deleted the FlowFile’s all on its own, since the queue did not exist. Since this is done during the FlowFile Repository load into memory, it di