Rai, NiFi can certainly be used for some data replication scenarios and quite often is. If you can treat the source like a continuous data source then there is some way to keep state about what has been pulled already, what has changed or needs yet to be pulled, and it can just keep running then generally speaking it will work out well. Depending on how the flow is setup, error conditions that can occur in remote delivery, and cluster topology NiFi won't be able to ensure the order that data is received is the order in which data is delivered. So, if you need to ensure data is copied in precisely the same order (like log replication) and each object/message/event is on the order of KBs in size then I'd recommend looking at Apache Kafka and Kafka Connect's support for keeping things ordered within the same partition of the same topic.
Thanks Joe On Thu, Oct 13, 2016 at 11:05 AM, Gop Krr <[email protected]> wrote: > Hi All, > I am learning NiFi as well as trying to deploy it in production for few use > cases. One of the use case is ETL and another use case is, using NiFi as a > backup solution, where it takes the data from one source and moves to > another database|file. Is anyone using NiFi for this purpose? Does NiFi > support incremental data move? > It would be awesome if someone can point me to right documentation. > Thanks > Rai
