Dear Apache NiFi Community,
I hope you are doing well.
I am working as a Business Analyst and have recently been exploring Apache
NiFi. I really appreciate the way data flows can be modeled and visualized-it's
been very intuitive and powerful for our use cases.
We are currently evaluating whether we can integrate NiFi into one of our
Quarkus-based applications. We also looked into MiNiFi as a lightweight
alternative. However, based on our initial proof of concept, it seems that
embedding or tightly integrating NiFi (or MiNiFi) directly into an application
might not be the intended usage pattern, as we were not able to make this
approach work successfully.
In addition, one of our goals was to deploy each flow independently (e.g., one
flow per deployment unit) in order to achieve a high degree of scalability and
isolation. This also proved challenging in our experiments.
This leads to a few questions we were hoping the community could help clarify:
1. Is embedding NiFi (or MiNiFi) within an application such as a Quarkus
service a supported or recommended approach?
2. Is deploying individual flows as separate, independently scalable units
aligned with NiFi's design, or does this conflict with its intended usage model?
3. If these approaches are not recommended, could you share the reasoning
behind these architectural decisions?
4. Our concern is that NiFi might become too heavyweight over time in our
scenario:
* We expect a growing number of flows
* Many flows would run at scheduled times each day
* Data volumes vary significantly-from small messages to files in the GB
range
* Flows would include polling from external systems, importing, and
exporting data
Given these characteristics, would NiFi still be an appropriate choice, or
should it rather be operated as a standalone service instead of being
integrated into an application?
We would greatly appreciate any guidance, best practices, or architectural
recommendations you can share based on your experience.
Thank you very much for your time and support.
Best regards,
Arthur
P.S. Resending this as I recently subscribed to the mailing list and my
previous message may not have gone through.
[Logo]<https://www.l-p-a.com>
Arthur Derewjankin
Business Analyst
Lucht Probst Associates GmbH
Gro?e Gallusstra?e 9
Frankfurt am Main 60311, Hessen, DE
e: [email protected] | w: www.l-p-a.com<http://www.l-p-a.com>
m: | p: +4969971485245
LinkedIn<https://www.linkedin.com/company/lpa-lucht-probst-associates-gmbh>
[Logo]<https://www.l-p-a.com/wp-content/uploads/payoff-magazine_0326_RZ.pdf>
________________________________
The information contained in this e-mail is privileged and confidential,
intended only for the use of the recipient named above. If the reader of this
message is not the above mentioned recipient or is not acting on behalf of the
respective recipient, please note that any saving, distribution or copying of
this e-mail is strictly prohibited. Please notify us immediately by telephone
at + 49-69-97 14 85-0 or e-mail if you have received such misdirected messages,
and kindly destroy this e-mail. Thank you.
Lucht Probst Associates GmbH, Gro?e Gallusstra?e 9, 60311 Frankfurt, Germany
Managing Directors: Stefan Lucht
Vat id number : DE203779866 I commercial register entry: Amtsgericht
Frankfurt/Main I commercial register no.: HRB 48809
All necessary information for processing your data is available here:
https://www.l-p-a.com/privacy-policy
