Re: ListSFTP incoming relationship

2018-03-29 Thread Andy LoPresto
You would still need to store the state per-directory-scanned, and this would scale with the number of directories used — this raises resolution questions as well, like does “~” == “/usr/xyz/home”, “/Users/xyz”, etc.? Is “~” the user NiFi is running as? So eventually you will end up with a map

[CANCEL][VOTE] Release Apache NiFi 1.6.0 (RC2)

2018-03-29 Thread Joe Witt
The Apache NiFi 1.6.0 RC2 is cancelled due to NIFI-5033. Thanks all - really appreciate your efforts to review here! Joe On Thu, Mar 29, 2018 at 6:38 PM, Joe Witt wrote: > Wellbummer. Good find though Pierre. Lets get that fixed. I'll > send the RC2 cancel email. > > I'll turn around RC3

Re: [VOTE] Release Apache NiFi 1.6.0 (RC2)

2018-03-29 Thread Joe Witt
Wellbummer. Good find though Pierre. Lets get that fixed. I'll send the RC2 cancel email. I'll turn around RC3 as soon as it is fixed. thanks On Thu, Mar 29, 2018 at 5:36 PM, Pierre Villard wrote: > Guys... I'll have to change my vote to -1. > > While playing with NiFi, I think I found a

Re: [VOTE] Release Apache NiFi 1.6.0 (RC2)

2018-03-29 Thread Pierre Villard
Guys... I'll have to change my vote to -1. While playing with NiFi, I think I found an issue: when updating a variable at pg level that references a restricted component it will fail. It seems the code is the same for secured and unsecured instance and it fails when NiFi is unsecured since the use

Re: [VOTE] Release Apache NiFi 1.6.0 (RC2)

2018-03-29 Thread Matt Gilman
+1 (binding) Release this package as nifi-1.6.0 - Ran through release help - Verified issued discovered by Andrew Lim in previous RC Thanks for RMing Joe! Matt On Thu, Mar 29, 2018 at 8:36 AM, Aldrin Piri wrote: > +1, binding > > > comments: > > * commit looks correct > * signature and hashes

Re: ListSFTP incoming relationship

2018-03-29 Thread Charlie Meyer
Just a thought, Could a processor that did the scan and stored state be implemented similar to GenerateTableFetch, where there is a minimum value attribute that is specified that could be read from the source (such as created date, updated date, etc)? That way the state could potentially be manag

Re: ListSFTP incoming relationship

2018-03-29 Thread Andy LoPresto
Bryan, No, that was exactly what I was referencing regarding the attribute output. It would have been clearer if I had said it like you did. Thanks. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Mar 29, 201

Re: ListSFTP incoming relationship

2018-03-29 Thread Bryan Bende
Scott, You are correct that the overall discussion is about allowing incoming flow files to ListSFTP. However, the previous discussion on this thread highlighted that the main reason ListSFTP currently doesn't allow incoming flow files is because of challenges when storing state. This led to the

Re: ListSFTP incoming relationship

2018-03-29 Thread Andy LoPresto
Scott, I think there are two conversations going on here. You are finding the requirements for your specific use case, and that’s great. But I echo Bryan’s point that a community processor for this scenario should not store state at all. Sivaprasanna’s point that given dynamic directory input,

Re: ListSFTP incoming relationship

2018-03-29 Thread scott
Sorry Bryan, but I disagree with you. Not storing state is NOT the main point of this new processor. The main point is to allow an incoming relationship flowfile to trigger the action, and allow variables to be used from the attributes therein. I agree that if the NiFi community deems it too r

Re: Read processor property in init()

2018-03-29 Thread Sivaprasanna
Yep. That’s correct. On Thu, 29 Mar 2018 at 6:45 PM, Jeff Zemerick wrote: > Thanks! Just to confirm, each time the processor is started the > @OnScheduled annotated method is executed, right? > > Jeff > > > On Wed, Mar 28, 2018 at 9:07 AM, Sivaprasanna > wrote: > > > Just to add on top of what

Re: Read processor property in init()

2018-03-29 Thread Jeff Zemerick
Thanks! Just to confirm, each time the processor is started the @OnScheduled annotated method is executed, right? Jeff On Wed, Mar 28, 2018 at 9:07 AM, Sivaprasanna wrote: > Just to add on top of what Mike said. The @OnScheduled annotation indicates > that the method that is marked with this a

Re: [VOTE] Release Apache NiFi 1.6.0 (RC2)

2018-03-29 Thread Aldrin Piri
+1, binding comments: * commit looks correct * signature and hashes check out * build and tests verified on CentOS 7 and OS X 10.12 * ran some simple flows on each and things looked fine On Thu, Mar 29, 2018 at 1:06 AM, Sivaprasanna wrote: > Thanks, Scott. That helped. > > On Thu, 29 Mar 2018

Re: ListSFTP incoming relationship

2018-03-29 Thread Bryan Bende
The main point of the new processor is to NOT store state so that it becomes more reasonable to allow incoming flow files. You could probably implement your own custom processor that does both because you can make assumptions about how you are going to use it, but if the NiFi community provides on

Re: NiFi 1.6 build failure nifi-web-ui

2018-03-29 Thread Sivaprasanna
The latest master should be able to build without this error. I recently faced this problem. Please make sure you’re using the latest master. Take a look at this PR : https://github.com/apache/nifi/pull/2571 Sivaprasanna On Thu, 29 Mar 2018 at 1:07 PM, luis_size wrote: > Hi, > I am trying to b

NiFi 1.6 build failure nifi-web-ui

2018-03-29 Thread luis_size
Hi, I am trying to build NiFi from source but the build fails with the below error. Can anyone help me with this? Sorry if it's a newbie question. I am new to the NiFi development world. Thanks [INFO] [INFO] BUILD FAILURE