Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.9.0 (RC2)
+1, binding comments: * verified hashes and signature * verified build and tests on Debian 10 and macOS 10.15 * No strong issues with L&N noticed nit: Copyright in NOTICE is dated to 2019 * ran a simple flow doing site to site without issue * verified docker build/image - looks good Nice work, folks! The amount of ground covered is impressive! On Tue, Feb 23, 2021 at 2:26 AM Marton Szasz wrote: > Hello, > > I am pleased to be calling this vote for the source release of Apache > NiFi MiNiFi C++ 0.9.0 > > The source tarball, including signatures, digests, etc. can be found at: > https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/0.9.0/ > > The release helper guide can be found at: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173087303 > > The Git tag is minifi-cpp-0.9.0-RC2 > The Git commit ID is ae746065319c89b6df23ef6a1bd6902306087cb8 > > https://git-wip-us.apache.org/repos/asf?p=nifi-minifi-cpp.git;a=commit;h=ae746065319c89b6df23ef6a1bd6902306087cb8 > > Checksums of nifi-minifi-cpp-0.9.0-source.tar.gz: > SHA256: 519bbf273cc70874a8542d7e422302731c99f44e0aa6439d0e285fbcf4a6be74 > SHA512: > 7ebc413f0bf85d25de4e50a2ca4d68925a362f2caa1f30c53a279591dca25d4866923c28418085b8485f73c36af737cf3549ed8990977ee55e4266f8b81b08bd > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/szaszm.asc > > KEYS file available here: > https://dist.apache.org/repos/dist/release/nifi/KEYS > > 200 issues were closed/resolved for this release: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12321520&version=12345444 > > Release note highlights can be found here: > > https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Versioncpp-0.9.0 > > The vote will be open for 72 hours. > Please download the release candidate and evaluate the necessary items > including checking hashes, signatures, build from source, and test. > Then please vote: > > [ ] +1 Release this package as nifi-minifi-cpp-0.9.0 > [ ] +0 no opinion > [ ] -1 Do not release this package because... >
Re: [EXT] Re: NiFi 2.0 Roadmap
Ryan No specific developments for a 2.0 push right now. There is still so much progress being made without breaking any major eggs that we've just kept moving ahead. However, it is likely that once we have a clear path to split out nars from the core framework that we should really dive into this. We have no choice but to not grow our convenience binary at this point. Thanks On Tue, Feb 23, 2021 at 2:53 PM Ryan Hendrickson < ryan.andrew.hendrick...@gmail.com> wrote: > Resurrecting an old thread here... Any updates on a 2.0 or new major > release of NiFi? > > Thanks, > Ryan > > On Fri, Aug 2, 2019 at 10:14 AM ski n wrote: > > > "a Flow Registry that can hold extensions and we can then make nifi 2.0 > > fundamentally about distributing nifi as a kernel (small as possible) and > > all extensions come from a flow registry on demand. " > > > > Nice! Great to hear NiFi as an extensible platform. I would see that the > > NiFi Registry is linked to a repository. Where one is the official > > repository and another for community. At this repo/library people can > > public put Processors/Services/Templates/Extensions. Browsing/Installing > > should be similar to an app store (Jenkins does this nicely and also > > Node-Red Library https://flows.nodered.org). > > > > Another thing that I would like to see is that for developing/testing you > > don't need to leave NiFi. For example directly code/debug > > (codemirror/ace/monaco) and unit test it with input/output of a flowfile > > for a specific processor. > > > > Also a thing is making the GUI more responsive (rendering) and > comfortable > > (like undo). Most big features are already on Confluence: > > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > > > > > > Raymond > > > > > > > > > > Op vr 14 jun. 2019 om 22:59 schreef Peter Wicks (pwicks) < > > pwi...@micron.com > > >: > > > > > I've also heard something about a big change to the way relationships > > > work. Maybe grouping relationships into larger groupings "failure" as a > > > parent, with multiple optional/fine grained, children. Something like > > > that. If that rings a bell, maybe add it to your list 😊. > > > > > > -Original Message- > > > From: Joe Witt > > > Sent: Friday, June 14, 2019 11:31 AM > > > To: dev@nifi.apache.org > > > Subject: [EXT] Re: NiFi 2.0 Roadmap > > > > > > It makes sense there would be an mvp set of registry capability and > thus > > a > > > dependency for nifi 2.0 on registry readiness/version. Otherwise I > would > > > largely hope not. > > > > > > > > > On Fri, Jun 14, 2019 at 1:29 PM Otto Fowler > > > wrote: > > > > > > > Will that effort or planning be across all the nifi projects? minifi > > > > / cpp / registry etc? > > > > > > > > > > > > On June 14, 2019 at 13:01:36, Joe Witt (joe.w...@gmail.com) wrote: > > > > > > > > Peter, > > > > > > > > Yeah I think we're all circling around similar thoughts on things > > > > which are 'best for a major release' and we need to start codifying > > > > that. At the same time we need this to be focused on items which can > > > > only reasonably happen in a major release and not become a new > kitchen > > > > sink for JIRAs/ideas. We should frame up a wiki page for this effort. > > > > I'm happy to kick that off soon (time permitting). In my mind the key > > > > domino here is having a Flow Registry that can hold extensions and we > > > > can then make nifi > > > > 2.0 fundamentally about distributing nifi as a kernel (small as > > > > possible) and all extensions come from a flow registry on demand. > > > > Other obvious things like Java 11 as the base requirement and killing > > > > off deprecated things come to mind. > > > > > > > > Thanks > > > > > > > > On Fri, Jun 14, 2019 at 11:45 AM Peter Wicks (pwicks) > > > > > > > > wrote: > > > > > > > > > I've seen a lot of comments along the line of, "I don't think this > > > > > will happen before NiFi 2.0". Do we have a roadmap/list somewhere > of > > > > > the big general changes planned for NiFi 2.0 or some kind of 2.0 > > > roadmap? > > > > > > > > > > --Peter > > > > > > > > > > > > > > >
Re: Release diff checker
Adam, This is a good idea. For now I recommend placing it or linking to it from the release helper wiki page. Thanks On Tue, Feb 23, 2021 at 6:10 AM Ádám Markovics wrote: > > Hi everyone, > I have just created a release diff checker that works for NiFi and > MiNiFi. Its purpose is to tell whether there is any difference between > a release source package and a given tag in a given repo, by running a > simple command in the command line. > Motivation: there was a difference in the last MiNiFi RC, it was only > spotted after many people have already approved. And this problem is > highly automatizable. > Code: https://github.com/adam-markovics/nifi-release-tools > Please try it, I'm open to fixes and improvements. > Could we even include it in official NiFi/MiNiFi repos? > Regards, > > Ádám
NIFI Processor Question
Good Evening, At work I'm using your ListenBeat processor to collect data from a metricbeat service logstash output.The ListenBeat processor is on server and the metricbeat service is on another server. After running for about two or three minutes I get the following error in the metricbeat log file. failed to publish events caused by: read tcp x.xxx.xxx.x:x -> y.yyy.yyy.y:y: i/o timeout When I do a "netstat -a" on the listenbeat server I see a socket in a CLOSE_WAIT state which doesn't go away until I stop the litenbeat processor. If I let everything run over night, the listenbeat server will run out of socket connections. There will benultiple sockets in a CLOSE_WAIT state. I've tried changing the listenbeat properties and the logstash settings. Is there anyone that can help me with this problem? Thanks,Cornelius Barkley
Re: Preconfiguring dynamic properties
Thanks, that probably gets me all or most of what I wanted. I'll try it out. On 2/24/21 11:19 AM, Bryan Bende wrote: I don't think it was the intent to pre-add a dynamic property. You should be able to set a default value though, the user still has to click the + icon to add the property though. On Wed, Feb 24, 2021 at 12:02 PM Russell Bateman wrote: I have a dynamic property in a custom processor that my down-streamers struggle a little bit to configure (requires newlines and a peculiar format). I would like to "preconfigure" a dynamic property as an example that they can either modify or erase to add their own. Most of them would probably just use what I preconfigure. The point is that I don't wish it to be a full-raging, static property. I want to see a trash can to the right of it. The trash can is not a feature of static properties. Is this possible, wrong-headed, what? Thanks, Russ
Re: Preconfiguring dynamic properties
I don't think it was the intent to pre-add a dynamic property. You should be able to set a default value though, the user still has to click the + icon to add the property though. On Wed, Feb 24, 2021 at 12:02 PM Russell Bateman wrote: > > I have a dynamic property in a custom processor that my down-streamers > struggle a little bit to configure (requires newlines and a peculiar > format). I would like to "preconfigure" a dynamic property as an example > that they can either modify or erase to add their own. Most of them > would probably just use what I preconfigure. > > The point is that I don't wish it to be a full-raging, static property. > I want to see a trash can to the right of it. The trash can is not a > feature of static properties. > > Is this possible, wrong-headed, what? > > Thanks, > Russ
Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.9.0 (RC2)
+1, binding source build on Ubuntu 18.04 Verified signatures Tested simple flow, would have liked to have tested more This process has became much more pleasant with the evolving CMake changes. Good work! Jeremy Dyer On Wed, Feb 24, 2021 at 11:15 AM Ádám Markovics wrote: > +1, non-binding > > Built and tested on Ubuntu 20.04. > Verified checksums and signatures. > Connected to EFM and updated with simple workflow: > GetFile >> PutFile > Regards, > > Ádám > > On Wed, Feb 24, 2021 at 10:48 AM Adam Debreceni > wrote: > > > > +1 (non-binding) > > > > Verified hashes, signatures, commit. > > Built on Windows 10, c2-update and > > a CWEL -> PublishKafka flow working as expected. > > > > Thanks, > > Adam > > > > On Tue, Feb 23, 2021 at 3:59 PM Ferenc Gerlits > wrote: > > > > > +1 (non-binding) > > > > > > Verified hashes and signatures. > > > Built on (a Docker container running) Ubuntu 20.04, enabling everything > > > except TensorFlow and JNI. > > > Ran the binary using a simple InvokeHTTP -> LogAttribute flow yaml > config; > > > it worked as expected. > > > > > > Thanks, > > > Ferenc > > > >
Preconfiguring dynamic properties
I have a dynamic property in a custom processor that my down-streamers struggle a little bit to configure (requires newlines and a peculiar format). I would like to "preconfigure" a dynamic property as an example that they can either modify or erase to add their own. Most of them would probably just use what I preconfigure. The point is that I don't wish it to be a full-raging, static property. I want to see a trash can to the right of it. The trash can is not a feature of static properties. Is this possible, wrong-headed, what? Thanks, Russ
Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.9.0 (RC2)
+1, non-binding Built and tested on Ubuntu 20.04. Verified checksums and signatures. Connected to EFM and updated with simple workflow: GetFile >> PutFile Regards, Ádám On Wed, Feb 24, 2021 at 10:48 AM Adam Debreceni wrote: > > +1 (non-binding) > > Verified hashes, signatures, commit. > Built on Windows 10, c2-update and > a CWEL -> PublishKafka flow working as expected. > > Thanks, > Adam > > On Tue, Feb 23, 2021 at 3:59 PM Ferenc Gerlits wrote: > > > +1 (non-binding) > > > > Verified hashes and signatures. > > Built on (a Docker container running) Ubuntu 20.04, enabling everything > > except TensorFlow and JNI. > > Ran the binary using a simple InvokeHTTP -> LogAttribute flow yaml config; > > it worked as expected. > > > > Thanks, > > Ferenc > >
Re: NIFI-8154
Glenn We have a really large backlog of PRs to work through and while we don't tend to work them in order there are things which make them harder/easier. 1. Please subscribe to the mailing list so your emails to the list do not require moderation and you receive the notes. 2. The JIRA has a Github PR on it but that was closed in favor of your patch. Patches alone can work but they add far more time consumption to the reviewer as the responsibility goes to them to verify the full clean build which the PR process takes care of. So it just adds risk/likelihood that someone wont want to jump on it. 3. This touches Parquet versions and historically we've been burned by various independent version changes in the Hadoop family around hdfs clients, avro versions, parquet versions, zookeeper versions, etc. It requires more time/confirmation. Anyway, I provide this for perspective and how to help increase the odds of this being worked in a timely fashion. Doesn't mean it cant/wont as is. Thanks On Wed, Feb 24, 2021 at 6:13 AM Glenn Jones wrote: > > Hello, > > Just wondering if there is anything I can do to help progress NIFI-8154? > I've submitted a PR that I think addresses the issue, but I guess it's a > low priority so it hasn't received any attention. It would be great to get > this fixed in a release of NiFi so we don't have to keep building our own > patched builds. > > Regards, > > Glenn
NIFI-8154
Hello, Just wondering if there is anything I can do to help progress NIFI-8154? I've submitted a PR that I think addresses the issue, but I guess it's a low priority so it hasn't received any attention. It would be great to get this fixed in a release of NiFi so we don't have to keep building our own patched builds. Regards, Glenn
Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.9.0 (RC2)
+1 (non-binding) Verified hashes, signatures, commit. Built on Windows 10, c2-update and a CWEL -> PublishKafka flow working as expected. Thanks, Adam On Tue, Feb 23, 2021 at 3:59 PM Ferenc Gerlits wrote: > +1 (non-binding) > > Verified hashes and signatures. > Built on (a Docker container running) Ubuntu 20.04, enabling everything > except TensorFlow and JNI. > Ran the binary using a simple InvokeHTTP -> LogAttribute flow yaml config; > it worked as expected. > > Thanks, > Ferenc >
