Re: Most memory-efficient way in NiFi to fetch an entire RDBMS table?

2022-06-22 Thread Mike Thomsen
Thanks, Matt. On Wed, Jun 22, 2022 at 10:46 AM Matt Burgess wrote: > > Mike, > > I recommend QueryDatabaseTableRecord with judicious choices for the > following properties: > > Fetch Size: This should be tuned to return the most number of rows > without causing network issues such as timeouts.

Re: Most memory-efficient way in NiFi to fetch an entire RDBMS table?

2022-06-22 Thread Matt Burgess
Mike, I recommend QueryDatabaseTableRecord with judicious choices for the following properties: Fetch Size: This should be tuned to return the most number of rows without causing network issues such as timeouts. Can be set to the same value as Max Rows Per Flow File ensuring one fetch per

Most memory-efficient way in NiFi to fetch an entire RDBMS table?

2022-06-22 Thread Mike Thomsen
We have a table with 68M records that will blow up to over 250M soon, and need to do a full table fetch on it. What's the best practice for efficiently doing a partial or full select on it? Thanks, Mike