Re: Jython in 1.3 ExecuteScript config?

2017-07-05 Thread Matt Burgess
Jim, The Jython script engine returns its name as "python" for some reason, so that's what's displayed as the option, but it is definitely Jython not Python. If you want to run your script as-is and use stdout for the results, you can use ExecuteStreamCommand for that, otherwise Joe's

Re: Jython in 1.3 ExecuteScript config?

2017-07-05 Thread Joe Witt
To understand jython/python relationship [1] To understand example usage of the processor a bit more [2, 3, 4]. [1] http://www.jython.org/archive/21/docs/whatis.html [2] https://community.hortonworks.com/articles/75032/executescript-cookbook-part-1.html [3]

Jython in 1.3 ExecuteScript config?

2017-07-05 Thread James McMahon
Hello. I have a script I run at the command line via jython /a/b/c/parseIncoming.py At the moment, all it does is print a message to the shell. I expected to be able to execute this script from an ExecuteScript processor. May not do anything right now, but I still anticipated that the flowfiles

Re: Processor classpath

2017-07-05 Thread Bryan Bende
James, Just to clarify... the conf directory of NiFi is not on the classpath of processors. The behavior with core-site.xml is because the Hadoop client used by NiFi provides this behavior, and not NiFi itself. Basically if you create a new Hadoop Configuration object and you don't provide it

Re: Processor classpath

2017-07-05 Thread James Srinivasan
Thanks, I ended up stracing the NiFi process to see where it was actually trying to load my config file from. Helpfully, the list of locations included the NiFi conf directory, which is just perfect for me. I don't want to keep the config file (core-site.xml for Hadoop) in my NAR itself since it

Multicolumn lookups with LookupRecord

2017-07-05 Thread Mika Borner
I've been playing with the LookupRecord processor. So far so good. I have use-cases, where I want to enrich multiple columns at the same time (e.g. enrich with customer information for a certain customer_id), or where there are multiple keys that should match. I think this would be a useful

Re: QueryRecord and AvroReader: What is the expect syntax ?

2017-07-05 Thread Mark Payne
What Matt laid out here is accurate. QueryFlowFile currently operates only on the 'root level' fields. I definitely want to have a mechanism for querying embedded fields as well, but right now it's not really possible to do so directly. As Matt pointed out, you can certainly use UpdateRecord to