Eric Try using 2GB for the heap and seeing that helps. I also believe there are specific pod settings youll want to use to avoid it getting nuked by k8s.
This blog may give you great things to consider https://home.robusta.dev/blog/kubernetes-memory-limit Thanks On Wed, Aug 16, 2023 at 3:29 PM Eric Secules <[email protected]> wrote: > My nifi.properties uses the filesystem repository implementations for > everything except the component status repository. So it's sticking to the > defaults. Could the VolatileComponentStatusRepository be what's eating up > my memory. It would fit well with the use pattern of the test framework > creating many processors and connections but deleting a processor also > makes its component status unavailable, so I would assume that also > releases associated memory. > > On Wed, Aug 16, 2023 at 2:35 PM Eric Secules <[email protected]> wrote: > >> Hi, >> >> I was wondering what the recommendation is for how much memory to start >> the nifi jvm with and then how much to set the docker container memory >> limit as a function of the jvm memory setting. >> >> I have nifi started with -xms and -xmx set to 4gb and docker memory >> resource limits of 8GB on the container for a small test environment. And >> it gets killed by k8s for consuming too much memory after the tests run for >> a while. My workload is automated tests where each test puts a flow (avg 70 >> processors) on the canvas, runs files through, and removes the flow when >> the test is done. About 4 of these tests run in parallel and we are only >> testing flow functionality so the flows only deal with 10s of flowfiles >> each. >> >> I am still on version 1.14.0 and I would like to know what other things >> in nifi contribute to memory use besides flowfiles in flight. Also is it >> normal that I see my containers using much more memory than I allotted to >> the jvm? >> >> Thanks, >> Eric >> >
