Hi All, I'm using NIFI to extract a postgreSQL table into a csv file. I'm using an 'executeSQLRecord' processor with a 'CSVRecordSetWriter' to extract data in csv format and a PutFile processor to write the csv file into the machine. >From a single sql query, around 60 million rows can be fetched.
When I started the 'executeSQLRecord', after several minutes, NIFI web UI became not responsive. When I checked the memory, only about 250-300MB RAM was free. I have given 50GB heap space from the boostracp configs. It seems like NIFI loads all the records into the RAM. Is there a way to write row by row without loading all the records into the memory? Best Regards, Vibhath.
