Hi Ethan, While you can definitely come up with a way to achieve your deployment architecture for NiFi, it will introduce few issues. 1. Consider the time of execution to be increased significantly, considering the time NiFi takes to startup. Which may vary significantly on each start-up. This proves to be inefficient for REST APIs hosted on NiFi. 2. NiFi must be deployed in a clustered architecture for best performance. The nodes may also be on different physical machine. It adds to the complexity of managing the resources in your proposed architecture (Cloud).
To tackle this issue NiFi now has a stateless component. "nifi-stateless". This can be run on a demand basis or even continuously with minimum required dependencies. I have recently implemented running nifi-stateless as AWS Lambda function. Thanks, Deepak K. On Mon, 13 Sep 2021, 8:05 pm Ethan Siegel, <ethan.siegel@1up.health> wrote: > Hi, > > I had a question about setting up a “start” trigger. The company I’m > working for would like to find a way to deploy NiFi using a managed cloud > service to save on costs. What we’d like to do is to have NiFi be deployed > in a container where it is almost entirely turned off when not in use. > However, ideally it would be able to receive a message via POST request or > some similar message, after which it would start itself up, process a file > using the content of the message for configuration, and then turn itself > off after completing the flow. > > The issue we’ve run into is that NiFi seems designed to be setup more as a > service which is continually running with listener processors, meaning that > in order to setup the architecture described above, we would need to setup > a secondary service to start/stop a NiFi host, which would then turn on a > listener processor. While this is feasible, it does introduce more > complexity and I wanted to ask if the NiFi dev team has any recommendations > for a simpler architecture for this use case. > > Thanks, > Ethan > > > Ethan Siegel > Team Lead, Data Application Engineering > Senior Data Engineer > <https://calendly.com/ethansiegel>1upHealth <https://1up.health/> | > Let’s Chat <https://calendly.com/ethansiegel> > > // <https://jobs.lever.co/1up-health/>We're Hiring < > https://angel.co/company/1uphealth/jobs> > > > > ---- > Secured by Paubox - HITRUST CSF certified > https://www.paubox.com > ---- > >
