Re: [VOTE] Release Apache NiFi 1.2.0

2017-05-04 Thread Bryan Rosander
+1 non-binding Validated checksums, signature Contrib-check build with -P docker - NOTE: Ran into a problem with docker build that nifi.version is a separate property that is still set to 1.2.0-SNAPSHOT, easily worked around by adding -Dnifi.version=1.2.0 Clustered docker-compose configuration w

Re: [VOTE] Release Apache NiFi 1.2.0

2017-05-04 Thread Andrew Psaltis
+1- non-binding -- Release this package as nifi-1.2.0 Did the following: - downloaded - verified sig - verified keys - built (mvn clean install -Pcontrib-check) - imported some templates of various flows - ran various flows *Toolchain:* Apache Maven 3.3.9 (bb52d8502b132ec0a5a3

Apache NiFi 1.2.0 RC1 Release Helper Guide

2017-05-04 Thread Bryan Bende
Hello Apache NiFi community, Please find the associated guidance to help those interested in validating/verifying the release so they can vote. # Download latest KEYS file: https://dist.apache.org/repos/dist/dev/nifi/KEYS # Import keys file: gpg --import KEYS # [optional] Clear out local maven

[VOTE] Release Apache NiFi 1.2.0

2017-05-04 Thread Bryan Bende
Hello, I am pleased to be calling this vote for the source release of Apache NiFi nifi-1.2.0. The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1103 The Git tag is nifi-1.2.0-RC1 The Git commit ID is 243ed742badd

Re: [DISCUSS] NiFi MiNiFi C++ 0.2.0 Release

2017-05-04 Thread Aldrin Piri
Looks like we have resolved the remaining items. I will start the RC process and have something out tonight/tomorrow morning. Thanks! Aldrin On Wed, May 3, 2017 at 6:30 PM, Kevin Doran wrote: > Thanks, Aldrin. > > PR now open for MINIFI-294 > https://github.com/apache/nifi-minifi-cpp/pull/90 >

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
Although I still think the ANTLR4 upgrade would be a considerably-sized effort, I did make some headway at lunch and pushed up an "antlr4" branch to my fork [1]. For now I kept the antlr3 stuff side-by-side with the antlr4 stuff, mostly so I can refer to the original as reference. It still builds

Re: Issues with creating and connecting RPG programmatically

2017-05-04 Thread Pushkara R
Thanks a lot Matt -Pushkar On Thu, May 4, 2017 at 11:00 PM, Matt Gilman wrote: > I've responded to the question on StackOverflow [1]. > > Thanks > > Matt > > [1] > http://stackoverflow.com/questions/43788780/nifi-issues-with-creating-and- > connecting-rpg-programmatically/43789310#43789310 > >

Re: Issues with creating and connecting RPG programmatically

2017-05-04 Thread Matt Gilman
I've responded to the question on StackOverflow [1]. Thanks Matt [1] http://stackoverflow.com/questions/43788780/nifi-issues-with-creating-and-connecting-rpg-programmatically/43789310#43789310 On Thu, May 4, 2017 at 12:49 PM, Pushkara R wrote: > Hi, > > I am trying to write a function that ta

Re: Closing in on a NiFi 1.2.0 release?

2017-05-04 Thread Bryan Bende
The issues from yesterday have been resolved so I'll start kicking off another attempt at the RC process. On Wed, May 3, 2017 at 6:10 PM, Bryan Bende wrote: > Quick update... I ran into two issues that will need to be addressed to > create the RC. > > I've created JIRAs for them and tagged them

Issues with creating and connecting RPG programmatically

2017-05-04 Thread Pushkara R
Hi, I am trying to write a function that takes two parameters, a processor ID in machine A and an input port in machine B and creates an RPG in machine B which connects the above to. Now, how I am doing that is 1. A POST to the */nifi-api/process-groups//remote-process-groups* endpoint to create

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
Good point, we should select a parser generator that produces tight code. Having said that, the MVP grammar is so simple that it could easily be put into a parser generator DSL later, and the MVP implementation could be hand-written for the first iteration (I'm picturing a couple dozen lines of

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Marc
Andy, Depending on the type of parser chosen you may run into memory limitations on devices that run MiNiFi-C++. Since that's a deployment concern -- and one that few will face, I would add that we can make this a conditional build and only including the necessary components within the binary i

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
The island grammar sounds very appealing for an MVP. Simple to implement, yet it covers a very common EL use case (dynamically inserting attr vals into property values). If we have general consensus I would love to see a MINIFI JIRA ticket added for this. -Andy _

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
What you have sounds good to me. IMO minimum viable product would be an island grammar (meaning you can have any characters outside a ${} expression) and inside would support an attribute name. Next steps could be nested expressions and/or support for functions, added piecemeal as the contributo

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
My bad, what does the sketch of the plan *look like*? -Andy From: Andrew Christianson Sent: Thursday, May 4, 2017 10:59:07 AM To: dev@nifi.apache.org Subject: Re: MiNiFi C++ Expression Language What does the sketch of the plan to do the separate implementa

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
What does the sketch of the plan to do the separate implementation? Write a flex/bison grammar, hook it into the cmake build, and start using it? Any constraints on features or syntax that this separate implementation must support? -Andy From: Matt Burge

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
No plans that I know of. In the meantime, EL support for MiNiFi is kind of held hostage, so maybe the separate implementation is more viable in the nearer term. If/When the ANTLR4 upgrade happens, we could replace whatever exists by then with the cross-platform ANTLR target generation, and test

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
Got it. So the crux of the problem is porting from v3 to v4, plus the added uncertainty of the C++ v4 target. I'm assuming that NiFi wants to eventually get onto v4 anyway. If that's the case, then porting to v4 is probably the ticket. Are there any concrete plans to do so in the NiFi mother pr

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
Correct, the current NiFi EL grammar is ANTLR3. > On May 4, 2017, at 9:12 AM, Andrew Christianson > wrote: > > Do I understand correctly that NiFi is currently using ANTLRv3? > > From: Matt Burgess > Sent: Thursday, May 4, 2017 9:05:35 AM > To: dev@nif

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
Do I understand correctly that NiFi is currently using ANTLRv3? From: Matt Burgess Sent: Thursday, May 4, 2017 9:05:35 AM To: dev@nifi.apache.org Subject: Re: MiNiFi C++ Expression Language I haven't used Flex/Bison since a trivial example in college, so I

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
I haven't used Flex/Bison since a trivial example in college, so I'm not sure about the LOE for getting that set up, maybe there's a Maven-built project out there that we could look at for inspiration, but that seems unlikely :) An ANTLR4 refactor (assuming the C++ target is in good shape) woul

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
> I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the > existing grammar. Since the source and target for MiNiFi C++ is, well, C++, > an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel > [3], etc. The downside is maintaining two grammars, but we a

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
> I am not aware of it being actively worked [1]. This would require using > ANTLR4, but I don't believe C++ support is well tested [2]. Someone can > correct me if I'm wrong, but there would have to be changes to both sides. Both sides as in ANTLR4 and MiNiFi, or as in MiNiFi and NiFi? > It gen

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Matt Burgess
I tried a quick ANTLR4 upgrade myself, it's indeed a big job to refactor the existing grammar. Since the source and target for MiNiFi C++ is, well, C++, an alternative could be to use lex/yacc (Flex/Bison) [1], Lemon [2], Ragel [3], etc. The downside is maintaining two grammars, but we are doing

Re: MiNiFi C++ Expression Language

2017-05-04 Thread Marc P.
Andrew, I am not aware of it being actively worked [1]. This would require using ANTLR4, but I don't believe C++ support is well tested [2]. Someone can correct me if I'm wrong, but there would have to be changes to both sides. I attempted a quick straw man with grammars, but didn't take it very

MiNiFi C++ Expression Language

2017-05-04 Thread Andrew Christianson
All, I see that we do not have support for the expression language yet in MiNiFi C++. Is anyone actively working on this, and if so, is there an ETA? If no one is working on it, is there a general plan for how it should be implemented? I think I recall seeing references to ANTLR

Need help with NiFi use-case on Partition and merge

2017-05-04 Thread Anshuman Ghosh
Hello everyone, Thank you for the continued support for NiFi. As we are planning to implement couple of our DataFlow to production, need some help on best practices. This is how the Flow looks like 1. NiFi-Kafka consumer to get data from Kafka cluster 2. Convert data from JSON to Avro 3

Re: whats new in nifi 1.2.0 / careful use of marks and being clear about non released features

2017-05-04 Thread Anshuman Ghosh
Hello all, Thank you for the response! We have built the NiFi from the master branch on git and able to use the Google Connectors properly. Should there be any challenges or observations, will update here. ​Cheers!​ ​ __ *Kind Regards,* *Anshuman Ghosh* *Contact - +49 179 90