Re: InvokeScriptedProcessor

2016-10-11 Thread Andy LoPresto
Jason, Sorry to hear you had a frustrating experience with InvokeScriptedProcessor. While it is marked with the “Experimental” annotation for just this reason (it’s performed well with standard behaviors but we’re sure there are still some edge cases remaining), no one should accept that as an

RE: InvokeScriptedProcessor

2016-10-11 Thread Jason Hamilton
Well, I solved it myself - the script was fine, but apparently after some earlier failed attempts that threw exceptions, the DBCPService was hosed. SOLUTION: I had to disable and then re-enable the DBCPService In summary, there are 2 buggy behaviors that have been huge gotchas (aka. hours

RE: InvokeScriptedProcessor

2016-10-11 Thread Jason Hamilton
Sorry, it looks like the mailing list hates attachments. The script is: import org.apache.nifi.dbcp.DBCPService import groovy.sql.Sql import groovy.json.JsonBuilder class MySQLToJSON implements Processor { def REL_SUCCESS = new Relationship.Builder()

InvokeScriptedProcessor

2016-10-11 Thread Jason Hamilton
Hello everyone, I have been struggling all day trying to make a simple MySQL to JSON processor (avoids the nasty issues with type casting MySQL numerics to Avro just to go to JSON anyway) in Groovy (see attached file). There's more interesting processors I need to make but this is my starting

Provenance repo corruption

2016-10-11 Thread Brandon DeVries
Devs, I just opened a ticket to address an issue we've encountered with Provenance repo corruption[1]. This would address (as is currently partially being done) how to recover from a corrupt provenance repo. However, the question is whether we can avoid this sort of corruption in the first

Re: [DISCUSS] Closing in on a 0.x release

2016-10-11 Thread Tony Kurc
Awesome. I propose we start building a release candidate off of 40618364e70a966f9c1e425674b53b22b1fb0fb0 soon. I believe I was the sole volunteer to RM, and unless I hear otherwise, I presume I will be doing so. I'd like to give the commit at least a good 24 hours for some people to bang on it

Re: Which are the underlying files for NiFi?

2016-10-11 Thread Jeff
Hello Russ! I'm glad you brought this up on the list. I've been thinking about reworking/enhancing the NiFi testing framework to enable testing of flows and flow templates from a programmatic perspective, without needing to stand up an entire NiFi instance. A lot of this is in the idea stage

Re: Which are the underlying files for NiFi?

2016-10-11 Thread Russell Bateman
Andy, First, many thanks for your answer which was very complete. It's exactly what I expected and I'm sure I had seen /flow.xml.gz/ in some thread before, but I could not remember it nor did I pay attention to that file while looking through the NiFi subdirectories (/*_repository/, /run/,

Re: [DISCUSS] Closing in on a 0.x release

2016-10-11 Thread Michael Moser
NIFI-2774 is now complete and merged to both master and 0.x branches. +1 on a release from the 0.x branch now. -- Mike On Mon, Oct 10, 2016 at 10:24 AM, Michael Moser wrote: > I feel that Oleg was really close, and it would be nice for this to be in > 0.7.1 but it isn't