Nifi is used for ETL and focus on transformations and data movement. Airflow is good for orchestration. Both tools have scheduling in common.
Airflow is good at where the data is already available and you want to control jobs. If your focus is more like the data integertion, manipulation and transformation and movement then NiFi is a great choice. No doubt there is some overlap but ultimately it comes down to your use case. Thanks, Muazma On Thu, Aug 15, 2019, 1:15 PM Joe Witt <joe.w...@gmail.com> wrote: > i blame spell check for ‘complimentary’ > > On Thu, Aug 15, 2019 at 4:11 PM Joe Witt <joe.w...@gmail.com> wrote: > > > Love it. > > > > Airflow I see as an orchestrator of making other systems do things. Nifi > > i see as very focused on acquiring, manipulating, and routing data > between > > systems. > > > > In short, I believe theyre complimentary in a given architecture. > Someone > > always tries to use one to do both of the things but each is better at > > their sweet spot. > > > > Thanks > > > > On Thu, Aug 15, 2019 at 4:07 PM Mike Thomsen <mikerthom...@gmail.com> > > wrote: > > > >> Yeah, pretty much. > >> > >> On Thu, Aug 15, 2019 at 4:02 PM Peter Wicks (pwicks) <pwi...@micron.com > > > >> wrote: > >> > >> > User: "I want to change my production flow while it's running, you > know, > >> > mid stream just route my content to a completely different flow, > re-run > >> > data mid run through a new set of processors, fork it on the fly, you > >> know, > >> > whatever I want anytime I want " > >> > > >> > NiFi: "Yeah, I can do that" > >> > > >> > Everyone Else: ".... I'm sorry... what?" > >> > > >> > -----Original Message----- > >> > From: Mike Thomsen <mikerthom...@gmail.com> > >> > Sent: Thursday, August 15, 2019 2:00 PM > >> > To: dev@nifi.apache.org > >> > Subject: [EXT] Airflow and NiFi > >> > > >> > Does anyone see any areas where the two can complement each other and > >> > where we might want to give users the ability to offload processing to > >> > Airflow? > >> > Curious since our poking around the docs lead us to conclude it was > >> > probably more "Airflow vs Spark+Oozie" than really competing with > NiFi. > >> > > >> > > >
