My nifi.properties uses the filesystem repository implementations for everything except the component status repository. So it's sticking to the defaults. Could the VolatileComponentStatusRepository be what's eating up my memory. It would fit well with the use pattern of the test framework creating many processors and connections but deleting a processor also makes its component status unavailable, so I would assume that also releases associated memory.
On Wed, Aug 16, 2023 at 2:35 PM Eric Secules <[email protected]> wrote: > Hi, > > I was wondering what the recommendation is for how much memory to start > the nifi jvm with and then how much to set the docker container memory > limit as a function of the jvm memory setting. > > I have nifi started with -xms and -xmx set to 4gb and docker memory > resource limits of 8GB on the container for a small test environment. And > it gets killed by k8s for consuming too much memory after the tests run for > a while. My workload is automated tests where each test puts a flow (avg 70 > processors) on the canvas, runs files through, and removes the flow when > the test is done. About 4 of these tests run in parallel and we are only > testing flow functionality so the flows only deal with 10s of flowfiles > each. > > I am still on version 1.14.0 and I would like to know what other things in > nifi contribute to memory use besides flowfiles in flight. Also is it > normal that I see my containers using much more memory than I allotted to > the jvm? > > Thanks, > Eric >
