Advance usage documentation not displayed

2016-03-04 Thread Russell Bateman
Just getting back to this... I have so far been unable to get the Advanced "Usage" documentation feature to work in any of my processors. Whether I right-click on the processor in the workspace and choose Usage or click Help in the workspace, I get nothing that resembles or contains what I've

Re: ExecuteSQL Extract database tables multiple times.

2016-03-04 Thread Matt Burgess
Currently ExecuteSql will put all available rows into a single flow file. There is a Jira case (https://issues.apache.org/jira/browse/NIFI-1251) to allow the user to break up the result set into flow files containing a specified number of records. I'm not sure why you get 26 flow files, although

Re: Re: List Files

2016-03-04 Thread Thad Guidry
It should be currently possible to use the ExecuteScript processor (with Groovy language) and set those needed user variables like Paths,etc using whatever System environment variables you wish to use and retrieve later. In Groovy scripts you can do this:

Re: Re: List Files

2016-03-04 Thread Charlie Frasure
Thanks Matthew and Joe, I must have missed the automatic environment variable mapping in expression language. I will plan on utilizing this method until the Variable Registry is available. On Fri, Mar 4, 2016 at 1:57 PM, Joe Witt wrote: > Charlie, > > Yeah you're right

Re: Re: List Files

2016-03-04 Thread Joe Witt
Charlie, Yeah you're right that is a viable option now too. And we support environment variables from the system too. So for things that support expression language that helps. The variable registry obviously will give us a nice step forward. Thanks Joe On Fri, Mar 4, 2016 at 1:55 PM,

Re: Re: List Files

2016-03-04 Thread Charlie Frasure
Thanks Joe, That option is certainly more comprehensive than what I was looking to do. An UpdateAttribute at the beginning of a flow (or even before a process group) could establish the variables I need. One case that either UpdateAttribute or the Variable Registry should consider is being able

Re: ExecuteSQL and NiFi 0.5.1 - Error org.apache.avro.SchemaParseException: Empty name

2016-03-04 Thread Juan Sequeiros
I wonder if on the controller service DBCPConnectionPool associated to your ExecuteSQL processor you have something that can't be found since it's stored on your older release. On Fri, Mar 4, 2016 at 11:12 AM, Marcelo Valle Ávila wrote: > Hello community, > > I'm starting my

Re: Re: List Files

2016-03-04 Thread Joe Witt
No. But we need to and plan to https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry On Fri, Mar 4, 2016 at 1:22 PM, Uwe Geercken wrote: > Matthew, > > does NiFi itself allow to define such things as constants? The idea would be > to re-use these constants

Aw: Re: List Files

2016-03-04 Thread Uwe Geercken
Matthew,   does NiFi itself allow to define such things as constants? The idea would be to re-use these constants (e.g. a path) in different processors.   Rgds,   Uwe   Gesendet: Freitag, 04. März 2016 um 19:04 Uhr Von: "Matthew Clarke" An: 

Re: List Files

2016-03-04 Thread Matthew Clarke
The expression language can also be used to return values from both JVM properties set in the NiFi bootstrap.conf and system environment variables for the user running NiFi. On Mar 4, 2016 11:00 AM, "Joe Percivall" wrote: > Hello, > > ListFile is a source processor so

Re: Nifi JSON event storage in HDFS

2016-03-04 Thread Andrew Grande
Sumo, True, MapR FS implementation may have compatibility issues. Additionally, things are complicated by a need to bundle some of their proprietary jars which can't be redistributed with NiFi. We at Hortonworks, have enabled some of our customers to have NiFi and MapR working together

ExecuteSQL and NiFi 0.5.1 - Error org.apache.avro.SchemaParseException: Empty name

2016-03-04 Thread Marcelo Valle Ávila
Hello community, I'm starting my first steps with NiFi, and enjoining how it works! I started with version 0.4.1 and a simple flow: ExecuteSQL -> ConvertAvroToJSON -> PutEventHub Reading from an Oracle database, and everything works like a charm! Few days ago NiFi 0.5.1 has been released, and

Re: List Files

2016-03-04 Thread Joe Percivall
Hello, ListFile is a source processor so this behavior is expected. It supports expression language so that it can be configured to utilize certains methods. For example, some people may want to get a list of files from a rotating directory that gets created every hour. To do that they would

List Files

2016-03-04 Thread Charlie Frasure
I'm using the 0.5.1 build and having what I think is odd behavior for ListFile. The processor supports expression language in the Input Directory property, however I can't figure out how to configure an attribute as input. I tried using UpdateAttribute prior to ListFile, but ListFile doesn't