Good evening, I am a beginner with Apache NiFi, and I have a question about automating my data flow.
Here is what I have done so far: • I created my first “ExecuteSQL” processor, which successfully connects to the database. • The query executes without any issues, and I retrieve the data correctly. • The data is then sent directly to “PutElasticsearchRecord” processor to be stored in Elasticsearch. When I check “View Data Provenance”, I see several files (Drop, Fork, Send). My question is the following: I would like to: 1. Schedule the “ExecuteSQL” processor to run once a day (at a specific interval). 2. Generate a single file containing all the extracted data and send this file to the “PutElasticsearchRecord” processor. 3. Automatically detect and retrieve modifications in the database and inject them into the respective processors. Thank you very much for your assistance and guidance in configuring this flow. Best regards, -- Abel Bellati Université de Limoges DSI : SERVICE SUPPORT ET APPLICATION DIRECTION GENERALE DES SERVICES 123 Avenue Albert THOMAS 87060 LIMOGES CEDEX Tél. : 0587080801 Portable : 06 32 64 94 59 Num. de poste : 3801 Web : http://www.unilim.fr Mail : [email protected]
