Loading set-of files to tables

Hi all, We have a need where we get a list containing a set of source files and target table names. Would like to load the files in parallel (as much as possible). The list need to be picked up from the database and it can change for each of the runs. Could you pls suggest an approach for this. F

R: Triggered fetching of multiple files from FTP - how?

Thank you, Harald. I ended up solving my problem with an ExecuteScript in Python that does the listing on demand and can be triggered by a flowfile. Luca Luca Giovannini Information Systems Analyst Dedagroup Public Services www.linkedin.com/in/lucagio/ T +3

Error with FetchFTP when filename has non-ASCII charachters

When FetchFTP tries to fetch a file with name containing non-ASCII characters (e.g. "Xml_APE001_unitàimmobiliareguarda.xml"), the process succeeds but the corresponding flowfile is empty. Is there a way to make the FetchFTP processor succeed for real and really fetch the file? Thank you ve

Re: Loading set-of files to tables

Hello, Can you describe the flow and processors that you used to make it work with parameter context? It will help people better understand what you are trying to do and how to make it more dynamic. Thanks, Bryan On Tue, Jul 14, 2020 at 9:09 AM Samarendra Sahoo wrote: > Hi all, > We have a n

Re: Error with FetchFTP when filename has non-ASCII charachters

Luca, I'm guessing the issue is the same as the one in [1] but it just wasn't fixed for FetchFTP. Please feel free to write an improvement Jira [2] to add this to FetchFTP as well. Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-4137 [2] https://issues.apache.org/jira/browse/NIFI

Split a field in a CSV record to an array of strings

Hi, I have a csv which looks like this: field,another field,"aaa,bbb,ccc",yet another field I want to ingest "aaa,bbb,ccc" as an array of strings ["aaa","bbb","ccc"], I've tried to use convertRecord with a csv reader and an avro writer but it seems that csv reader doesn't have the concept of array

Re: Split a field in a CSV record to an array of strings

I would recommend taking a look at JoltTransformRecord. Jolt has some capabilities for splitting strings into arrays. You can see [1] as an example. Thanks -Mark [1] https://stackoverflow.com/questions/58726817/conversion-of-string-to-array-in-jolt-transformation On Jul 14, 2020, at 11:38 AM,

Re: Loading set-of files to tables

Thanks a lot Bryan. Below is what I ve done so far. 1. Set the source files and target tables mapping. 2.Create separate flow files. Extract data and load it to target table. On Tue, Jul 14, 2020 at 7:23 PM Bryan Bende wrote: > Hello, > > Can you describe the flow and processors tha

NiFi 1.11.4 Upgrade

Hi Users, I'm here with another question regarding the upgrade from NiFi-1.9.2 to NiFi-1.11.4. After testing a different instance of NiFi-1.11.4 and getting comfortable with the features, we have decided to upgrade our existing NiFi instance to 1.11.4. All of the process was smooth and I was