Hi Matthieu, Thanks for raising this question for discussion. Other maintainers may be able to provide additional background, but part of the reason for removing the VolatileContentRepository implementation was that there were some more fundamental problems with the implementation. Although various framework updates included patching the implementation along the way, the repository was not maintained on a regular basis, which resulted in it being broken for several releases.
As Mike said, it would be helpful to share more details about your use case, and also to hear more about whether you still experience memory issues with the file system repository in current releases. On a related note, NiFi Stateless includes a new in-memory content repository named ByteArrayContentRepository [1]. It is currently packaged in the NiFi Stateless bundle, but it might be possible to consider promoting it to the framework level, if there is value in a non-persistent content repository going forward. Regards, David Handermann [1] https://github.com/apache/nifi/blob/main/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/repository/ByteArrayContentRepository.java On Wed, Mar 30, 2022 at 7:45 AM Mike Thomsen <mikerthom...@gmail.com> wrote: > We've been moving away from supporting it for a while, and I think it > comes down to a lot of both factors when you consider the time > involved in getting good patches and reviewing them. That said, until > 1.17 is released, I think there's room for community members like you > and your team to work with us on fixing the gaps that made a strong > case for removing it. > > I think I saw in your ticket that you provided patches through Jira. > My recommendation would be to do a feature branch that reverts the > removal, applies your patches and submit it as a PR on GitHub. Then > request a review. Obviously, there's no guarantees there because it's > based on folks' time and energy to do a review, but that would be the > right process at least to move your request forward. > > In the long run, I think it would be a lot better for you to share > your use case with us and to see if there's a better route ahead for > your team and NiFi. Sounds like an interesting use case, so it would > be good to get those requirements on the table since most users aren't > operating with those constraints. > > Thanks, > > Mike > > On Tue, Mar 29, 2022 at 12:20 PM Matthieu Ré <re.matth...@gmail.com> > wrote: > > > > Hi everyone, > > > > We wanted to talk about this ticket > https://issues.apache.org/jira/browse/NIFI-8760 and the > VolatileContentRepository... I understood that we weren't many to still use > this Repository, but in our use case with a very limited cloud environment > with strict IOps regulations, it fitted perfectly and we managed several To > of data per day efficiently. > > > > We tested other repositories, even a FileSystemContentRepo with RAM > based disk that did not match the case since we experimented numerous OOMs > with the same amount of RAM mounted. > > > > I provided a patch to fix it, that should be applied after 1.13.0 and a > refactor of Claims handling, waiting for a discussion about it. Now I read > that it should disappear in 1.17.0 :( > > > > Is it due to a technical limitation for further features ? Or is it too > costly to maintain it ? > > > > Thanks! Regards, > > Matthieu >
