Extract the min and max date of the input file
Hi ALL, I want to extract the min and max date of the input csv file. I have a date column in that file. Also please let me know how to add some custom attribute depending on the date column value. Any help would be appreciated. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Extract-the-min-and-max-date-of-the-input-file-tp16012.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Not able to see the uploaded file in S3
Hi All, I have a simple flow consists of following processors. GetFile-->PutS3Object The flow is successful but not able to see the file in S3. When I checked NiFi DataProvenance of PutS3Object in details section i can see the below message. Details Auto-Terminated by failure Relationship in PutS3Object i have given bucket name and used AWSCredentialsProviderController Service and the user is having proper rights for the S3 Bucket. Please let me know if i am doing anything wrong. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-see-the-uploaded-file-in-S3-tp15978.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: How to use ConvertRecord Processor
Hi, Thanks for your help . It worked. I used the following processors GetFile-->PutDatabaseRecord-->PutFile Only doubt i have is how to specify particular dataType for a column. For Eg: I have file like below and want to insert into the table with City as Varchar and Count as Integer in Postgres. City,Count Mumbai,10 Mumbai,10 Pune,10 Pune,10 -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-use-ConvertRecord-Processor-tp15873p15901.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
How to use ConvertRecord Processor
Hi All, I am using nifi version 1.2.0. We have CSV files given by Vendor with random number of Columns in the file. We want to save the contents of the File in Postgres Database. I have created a Flow with the following processors. GetFile-->ConvertRecord-->ConvertJSONToSql--->PutSql In ConvertRecord i have used CSVReader to read the record and JSONRecordSetWriter. In JSONRecordSetWriter Controller Service is INVALID telling SCHEMA Registry is invalid . How to fix it. I just want to take the CSV file column headers as name. Please Help -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-use-ConvertRecord-Processor-tp15873.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Save CSV file contents to database tables
Hi , We receive csv files from client. Each csv files contains different headers . We want to save the csv content in different tables . each table per csv and column name as per csv header. Please advice. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Save-CSV-file-contents-to-database-tables-tp15644.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Convert CSV File to JSON
Hi , I have a csv file which is provided by client having different column name. I wanted to covert the csv contents to JSON. Please advice -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Convert-CSV-File-to-JSON-tp15643.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Not able to ingest the csv data to RDBS Database
Hi , I tried with Header Line count 0 in SplitText Processor still getting the same result. I am trying with the header row. City,Count Mumbai,10 Mumbai,10 Pune,10 Pune,10 Do i need to add any processor -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-ingest-the-csv-data-to-RDBS-Database-tp15610p15641.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Not able to ingest the csv data to RDBS Database
I am just testing currently with a sample file containing only 4 rows. -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-ingest-the-csv-data-to-RDBS-Database-tp15610p15637.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Not able to ingest the csv data to RDBS Database
SplitText by line 1 and header count 1
Extract Text : City : (.*),.*
Count : .*,(.*)
ReplaceText : INSERT INTO CITY(City,Count) values ('${City}',${Count})
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-ingest-the-csv-data-to-RDBS-Database-tp15610p15635.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Not able to ingest the csv data to RDBS Database
Still not able to insert the data in table.
I tested with PutFile instead of PutSql and file contains
INSERT INTO CITY(city,count) VALUES('city',count)
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-ingest-the-csv-data-to-RDBS-Database-tp15610p15634.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Not able to ingest the csv data to RDBS Database
Hi, Thanks for helping . I have modified the flow according to your suggestion. My flow consists of below processors. GetFile-->SplitText-->ExtractText-->ReplaceText-->PutSql. I have only 4 rows in my csv then why the SplitText queue contains huge MB of data. Also in PutSql how to specify the table name where the data needs to be inserted. The table needs to be present before? My file City,Count Mumbai,10 Mumbai,10 Pune,10 Pune,10 -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-ingest-the-csv-data-to-RDBS-Database-tp15610p15630.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Not able to ingest the csv data to RDBS Database
Hi ,
I have a csv file and i want to ingest the csv file data into the database.
I have used getfile,RelaceText,PutSQL Processors.
It's failing in replaceText Processor.
File Sample :
City,Count
Mumbai,10
Mumbai,10
Pune,10
Pune,10
ReplaceText :
INSERT INTO CITY(city,count) values ('${city}',${count})
Please help.
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Not-able-to-ingest-the-csv-data-to-RDBS-Database-tp15610.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
