Re: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

2016-08-15 Thread Matt Burgess
Oops sorry, had replied before I saw this :) > On Aug 15, 2016, at 11:15 PM, Peter Wicks (pwicks) wrote: > > Oh, disregard J. I misread GenerateTableFetch as being an actual data fetch > vs a query builder. > > From: Peter Wicks (pwicks) > Sent: Monday, August 15, 2016

Re: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

2016-08-15 Thread Matt Burgess
Peter, Another difference between the two (besides the paging) is that QueryDatabaseTable executes SQL and GenerateTableFetch generates SQL. With the paging capability (which with Remote Process Groups enables distributed fetch a la Sqoop), you're likely correct that GTF will replace /

RE: v0.* QueryDatabaseTable vs v1 GenerateTableFetch

2016-08-15 Thread Peter Wicks (pwicks)
Oh, disregard :). I misread GenerateTableFetch as being an actual data fetch vs a query builder. From: Peter Wicks (pwicks) Sent: Monday, August 15, 2016 9:11 PM To: 'users@nifi.apache.org' Subject: v0.* QueryDatabaseTable vs v1 GenerateTableFetch What is the future of

v0.* QueryDatabaseTable vs v1 GenerateTableFetch

2016-08-15 Thread Peter Wicks (pwicks)
What is the future of QueryDatabaseTable? Unless I'm misunderstanding how it works it looks like GenerateTableFetch can do everything QueryDatabaseTable can do and then some. Is there a plan to phase out QueryDatabaseTable? Is there a reason for a new processor instead of an update to

Re: Exception on Processor ConvertJSONToSQL

2016-08-15 Thread Bryan Bende
Carlos/Peter, Thanks for reporting this issue. It seems IS_AUTOINCREMENT is causing problems in a couple of situations, I know there was another issue with Hive where they return IS_AUTO_INCREMENT rather than IS_AUTOINCREMENT. We should definitely address this issue... would either of you be

RE: PutSQL ERROR bulletin

2016-08-15 Thread Peter Wicks (pwicks)
Sven, I've run into this a couple times. In my case some records would insert and some would not. To find my issue: - I routed all failures back to PutSQL - Reduced the batch size down to about 10 - Changed the prioritization on the failure relationship so that hopefully failures will

RE: Exception on Processor ConvertJSONToSQL

2016-08-15 Thread Peter Wicks (pwicks)
Carlos, I ran into this same error when querying Teradata. It looks like a lot of databases don't include this. I submitted a bug a couple weeks ago: https://issues.apache.org/jira/browse/NIFI-2356 I did something similar to your suggestion locally in a modified version of the code. Regards,