Re: Re: Re: QueryDatabaseTable - Deleted Records

2017-09-18 Thread Matt Burgess
ion. The blogs of > Nifi and Kafka have really good samples of extracting data but none of them > touches the topic of deletes. > > Rgds, > > Uwe > > > > Gesendet: Samstag, 16. September 2017 um 13:52 Uhr > Von: "Andrew Grande" > An: users@nifi.ap

Aw: Re: Re: QueryDatabaseTable - Deleted Records

2017-09-18 Thread Uwe Geercken
.   Rgds,   Uwe       Gesendet: Samstag, 16. September 2017 um 13:52 Uhr Von: "Andrew Grande" An: users@nifi.apache.org Betreff: Re: Re: QueryDatabaseTable - Deleted Records As an interesting architectural approach we took eons ago, before NiFi, was to take daily snapshots of a

Re: Re: QueryDatabaseTable - Deleted Records

2017-09-16 Thread Andrew Grande
ted to make sure that I have not missed > anything. I will try to build my use case around something else then. > > Thanks for your response(s). > > Rgds, > > Uwe > > *Gesendet:* Freitag, 15. September 2017 um 16:15 Uhr > *Von:* "Bryan Bende" > *An:* use

Aw: Re: QueryDatabaseTable - Deleted Records

2017-09-15 Thread Uwe Geercken
something else then.   Thanks for your response(s).   Rgds,   Uwe   Gesendet: Freitag, 15. September 2017 um 16:15 Uhr Von: "Bryan Bende" An: users@nifi.apache.org Betreff: Re: QueryDatabaseTable - Deleted Records Uwe, Typically you need to process the change log of the database in

Re: QueryDatabaseTable - Deleted Records

2017-09-15 Thread Bryan Bende
Uwe, Typically you need to process the change log of the database in this case, which unfortunately usually becomes database specific. I believe we have a processor CaptureChangeMySQL that can process the MySQL change log. -Bryan On Tue, Sep 12, 2017 at 1:39 PM, Uwe Geercken wrote: > Hello, >

QueryDatabaseTable - Deleted Records

2017-09-12 Thread Uwe Geercken
Hello,   apparently the QueryDatabaseTable processor catches changes made to the data of the source database - updates and inserts.   Has anybody a good idea or strategy how to handle deletes in the source database? Of course one could flag a record as deleted instead of phisically deleting it.