Re: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

[Matt Burgess]

Oops sorry, had replied before I saw this :)


> On Aug 15, 2016, at 11:15 PM, Peter Wicks (pwicks)  wrote:
> 
> Oh, disregard J. I misread GenerateTableFetch as being an actual data fetch 
> vs a query builder.
>  
> From: Peter Wicks (pwicks) 
> Sent: Monday, August 15, 2016 9:11 PM
> To: 'users@nifi.apache.org' 
> Subject: v0.* QueryDatabaseTable vs v1 GenerateTableFetch
>  
> What is the future of QueryDatabaseTable? Unless I’m misunderstanding how it 
> works it looks like GenerateTableFetch can do everything QueryDatabaseTable 
> can do and then some.  Is there a plan to phase out QueryDatabaseTable?  Is 
> there a reason for a new processor instead of an update to 
> QueryDatabaseTable? It looked like the only user facing change was result 
> paging, which could have had a default of 0 (no paging).  Just curious.
>  
> Regards,
>   Peter

Re: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

[Matt Burgess]

Peter,

Another difference between the two (besides the paging) is that 
QueryDatabaseTable executes SQL and GenerateTableFetch generates SQL. With the 
paging capability (which with Remote Process Groups enables distributed fetch a 
la Sqoop), you're likely correct that GTF will replace / deprecate QDB.

Adding paging to QDB would not have given real distributed fetch, as multiple 
tasks of QDB would not be able to get the correct pages since they each execute 
the same code in parallel. GTF is designed to be run on the primary node only, 
but would be connected (likely via RPG) to ExecuteSQL which can be run in 
parallel.

Regards,
Matt

> On Aug 15, 2016, at 11:10 PM, Peter Wicks (pwicks)  wrote:
> 
> What is the future of QueryDatabaseTable? Unless I’m misunderstanding how it 
> works it looks like GenerateTableFetch can do everything QueryDatabaseTable 
> can do and then some.  Is there a plan to phase out QueryDatabaseTable?  Is 
> there a reason for a new processor instead of an update to 
> QueryDatabaseTable? It looked like the only user facing change was result 
> paging, which could have had a default of 0 (no paging).  Just curious.
>  
> Regards,
>   Peter

RE: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

[Peter Wicks (pwicks)]

Oh, disregard :). I misread GenerateTableFetch as being an actual data fetch vs 
a query builder.

From: Peter Wicks (pwicks)
Sent: Monday, August 15, 2016 9:11 PM
To: 'users@nifi.apache.org' 
Subject: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

What is the future of QueryDatabaseTable? Unless I'm misunderstanding how it 
works it looks like GenerateTableFetch can do everything QueryDatabaseTable can 
do and then some.  Is there a plan to phase out QueryDatabaseTable?  Is there a 
reason for a new processor instead of an update to QueryDatabaseTable? It 
looked like the only user facing change was result paging, which could have had 
a default of 0 (no paging).  Just curious.

Regards,
  Peter