Re: [DISCUSS] First Release of NiFi Registry
Is there a good description/detail page somewhere going over the registry? On Thu, Dec 7, 2017 at 2:06 PM, Pierre Villard wrote: > Strong +1!! > > Really impressed by all the work you guys did on the registry stuff. Very > impatient to use it in official releases! > > Le 7 déc. 2017 18:52, "Jeff" a écrit : > > Bryan, > > +1 to getting an initial release of NiFi Registry out to the community. > Definitely a huge step in the evolution of NiFi! > > On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman > wrote: > > > Our down-stream users are excited at the prospect of using this registry > > capability for their flows. So, we're eager to see it integrated into > > the earliest NiFi version you can choose (1.5.0?). > > > > Russ > > > > On 12/07/2017 08:49 AM, Kevin Doran wrote: > > > Thanks for kicking off this discussion thread, Bryan. > > > > > > I support prepping a release of NiFi Registry and making it version > > 0.1.0 as you propose. > > > > > > Thanks! > > > Kevin > > > > > > On 12/7/17, 10:45, "Joe Witt" wrote: > > > > > > Bryan - very exciting and awesome. Having experimented with the > > > registry on the JIRAs/PRs you mention I must say this is going to > > be a > > > huge step forward for NiFi! > > > > > > Since we'll also be doing a NiFi release soon (1.5.0?) I am happy > to > > > volunteer to RM that as well if needed. > > > > > > Thanks > > > > > > On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende > > wrote: > > > > Hey folks, > > > > > > > > There has been a lot of great work done on the NiFi Registry [1] > > and I > > > > think we are probably very close to an initial release focused > on > > > > storing "versioned flows". > > > > > > > > Since NiFi will have a dependency on client code provided by the > > > > registry, the first release of the registry would need to occur > > before > > > > the first release of NiFi that integrates with it. The work on > the > > > > NiFi side is being done as part of NIFI-4436, which can be > > followed > > > > along on PR 2219 [2]. > > > > > > > > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I > > would > > > > propose the first release should be 0.1.0. > > > > > > > > Let me know if anyone has any thoughts or comments. I'm happy > to > > act > > > > as RM if no one else is interested in doing so, and we can start > > the > > > > process of going through JIRA to see what is left. > > > > > > > > Thanks, > > > > > > > > Bryan > > > > > > > > [1] https://nifi.apache.org/registry.html > > > > [2] https://github.com/apache/nifi/pull/2219 > > > > > > > > > > > > > >
Re: [DISCUSS] First Release of NiFi Registry
Strong +1!! Really impressed by all the work you guys did on the registry stuff. Very impatient to use it in official releases! Le 7 déc. 2017 18:52, "Jeff" a écrit : Bryan, +1 to getting an initial release of NiFi Registry out to the community. Definitely a huge step in the evolution of NiFi! On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman wrote: > Our down-stream users are excited at the prospect of using this registry > capability for their flows. So, we're eager to see it integrated into > the earliest NiFi version you can choose (1.5.0?). > > Russ > > On 12/07/2017 08:49 AM, Kevin Doran wrote: > > Thanks for kicking off this discussion thread, Bryan. > > > > I support prepping a release of NiFi Registry and making it version > 0.1.0 as you propose. > > > > Thanks! > > Kevin > > > > On 12/7/17, 10:45, "Joe Witt" wrote: > > > > Bryan - very exciting and awesome. Having experimented with the > > registry on the JIRAs/PRs you mention I must say this is going to > be a > > huge step forward for NiFi! > > > > Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to > > volunteer to RM that as well if needed. > > > > Thanks > > > > On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende > wrote: > > > Hey folks, > > > > > > There has been a lot of great work done on the NiFi Registry [1] > and I > > > think we are probably very close to an initial release focused on > > > storing "versioned flows". > > > > > > Since NiFi will have a dependency on client code provided by the > > > registry, the first release of the registry would need to occur > before > > > the first release of NiFi that integrates with it. The work on the > > > NiFi side is being done as part of NIFI-4436, which can be > followed > > > along on PR 2219 [2]. > > > > > > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I > would > > > propose the first release should be 0.1.0. > > > > > > Let me know if anyone has any thoughts or comments. I'm happy to > act > > > as RM if no one else is interested in doing so, and we can start > the > > > process of going through JIRA to see what is left. > > > > > > Thanks, > > > > > > Bryan > > > > > > [1] https://nifi.apache.org/registry.html > > > [2] https://github.com/apache/nifi/pull/2219 > > > > > > > >
RE: [EXT] Re: [DISCUSS] CI / Travis / Jenkins
That's great! Thanks Joe for chasing this down! Let's make sure we keep it stable to help reviewing PRs. Le 7 déc. 2017 13:14, "Peter Wicks (pwicks)" a écrit : Best news I've heard in a long time :) Thanks for all the hard work! -Original Message- From: af...@fucs.org [mailto:af...@fucs.org] On Behalf Of Andre Sent: Thursday, December 07, 2017 12:15 To: dev@nifi.apache.org Subject: [EXT] Re: [DISCUSS] CI / Travis / Jenkins Joe, Thanks for that! yes, you are correct. The builds occurred twice as the parallel build was broken. :-) Since we can now build and run tests in parallel with contrib-check (YEAY) the dual build is no longer necessary. THANK YOU for chasing this down. While travis is one of the drivers all the developers will benefit from being able to run contrib-check in parallel moving forward. Cheers On Thu, Dec 7, 2017 at 1:38 PM, Joe Witt wrote: > Team, > > Ok so finally some really solid news to share on the Travis-CI front. > First, huge thanks to Aldrin for getting this started and folks like > Andre and Pierre who have tweaked it to make it more usable as well. > After a long run of it helping us out as we all know it went poorly > with every build failing for what seemed like months. > > After some improvements and updates to our usage of maven which now > means parallel builds with contrib check seem to be working and after > going ruthless mode on hunting down unstable tests and either fixing > them or making them integration-tests the build is far more stable. > We all need to try and stay on top of that. Today though i realized > that our builds were happening twice and that appeared to be why it > took roughly 50 minutes to finish, at best, and we'd timeout and fail. > So after adjusting our travis.yml we now only build once and the > process takes about 25 mins so we're well within. > > Latest build on travis-ci: https://travis-ci.org/apache/ > nifi/builds/312629807 > Appveyor builds: > https://ci.appveyor.com/project/ApacheSoftwareFoundation/nifi/ > build/1.0.0-SNAPSHOT-6649 > > So we're heading in the right direction. If it stays stable perhaps > we could add openjdk builds as well. > > THanks > Joe > > On Tue, Dec 5, 2017 at 4:11 PM, Joe Witt wrote: > > OK well things are looking pretty good. The only obvious problem > > now is that our builds take about 45-50 mins on travis-ci.org and > > the build time limit is 50 mins [1] so some jobs get killed. > > > > Will look at areas we can avoid spending build time on at least in > > travis-ci land. Probably no great option but let's see. > > > > [1] > > https://docs.travis-ci.com/user/customizing-the-build#Build-Timeouts > > > > On Tue, Dec 5, 2017 at 2:56 PM, Joe Witt wrote: > >> Will try it out for PR https://github.com/apache/nifi/pull/2319 > >> which is being built under > >> https://travis-ci.org/apache/nifi/builds/312043710 > >> > >> On Tue, Dec 5, 2017 at 2:51 PM, Joe Witt wrote: > >>> Andre > >>> > >>> Thanks - read through > >>> https://issues.apache.org/jira/browse/NIFI-1657 > >>> where this was discussed and where the relevant multi-env commit > >>> came in. > >>> > >>> Seems like five environments may be too taxing based on the build > >>> failures I'm observing. I'll cut it down to three FR JP US For > >>> now. We can evaluate if that helps at all and add more back if > >>> things become stable. > >>> > >>> Thanks > >>> Joe > >>> > >>> On Tue, Dec 5, 2017 at 12:20 AM, Andre wrote: > Joe, > > Glad to help! Few notes: > > If I recall correctly there was a reason we chose to add default > and > BR but > to be honest I can't really remember what it was. I think it has > to > do with > Time Zones + Locale issues and has helped detecting bizarre > issues on > time > based junits (Matt B and Pierre may remember this). > > Regarding the rat check. The idea behind that was a fast failure > in > case of > basic style violations, rather than wait until the end of the > compilation. > To be honest I don't know if this has worked as desired but > should > allow us > to quickly identify validation errors which if I recall correctly > were only > detected at the end of contrib-check. > > And apologies for the anecdotal comments. I am away from my dev > environment > atm so I can't truly validate them. > > > Kind regards > > > On Tue, Dec 5, 2017 at 3:31 PM, Joe Witt wrote: > > > Great news! So for the first time in a long time we now have > > travis-ci builds passing! > > > > I incorporated Dustin's PR which changed to the -Ddir-only > > instead of -P, added Andre's idea of dropping the -quiet flag, > > and dropped the number of builds in the config to a single > > parallel build with > contrib > > check now that we're seeing those pass with rat/checkstyle. > > > > https://travis-ci.org/apache/nifi/builds/311660398 > >
Re: [DISCUSS] First Release of NiFi Registry
Bryan, +1 to getting an initial release of NiFi Registry out to the community. Definitely a huge step in the evolution of NiFi! On Thu, Dec 7, 2017 at 11:29 AM Russell Bateman wrote: > Our down-stream users are excited at the prospect of using this registry > capability for their flows. So, we're eager to see it integrated into > the earliest NiFi version you can choose (1.5.0?). > > Russ > > On 12/07/2017 08:49 AM, Kevin Doran wrote: > > Thanks for kicking off this discussion thread, Bryan. > > > > I support prepping a release of NiFi Registry and making it version > 0.1.0 as you propose. > > > > Thanks! > > Kevin > > > > On 12/7/17, 10:45, "Joe Witt" wrote: > > > > Bryan - very exciting and awesome. Having experimented with the > > registry on the JIRAs/PRs you mention I must say this is going to > be a > > huge step forward for NiFi! > > > > Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to > > volunteer to RM that as well if needed. > > > > Thanks > > > > On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende > wrote: > > > Hey folks, > > > > > > There has been a lot of great work done on the NiFi Registry [1] > and I > > > think we are probably very close to an initial release focused on > > > storing "versioned flows". > > > > > > Since NiFi will have a dependency on client code provided by the > > > registry, the first release of the registry would need to occur > before > > > the first release of NiFi that integrates with it. The work on the > > > NiFi side is being done as part of NIFI-4436, which can be > followed > > > along on PR 2219 [2]. > > > > > > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I > would > > > propose the first release should be 0.1.0. > > > > > > Let me know if anyone has any thoughts or comments. I'm happy to > act > > > as RM if no one else is interested in doing so, and we can start > the > > > process of going through JIRA to see what is left. > > > > > > Thanks, > > > > > > Bryan > > > > > > [1] https://nifi.apache.org/registry.html > > > [2] https://github.com/apache/nifi/pull/2219 > > > > > > > >
Re: [DISCUSS] First Release of NiFi Registry
Our down-stream users are excited at the prospect of using this registry capability for their flows. So, we're eager to see it integrated into the earliest NiFi version you can choose (1.5.0?). Russ On 12/07/2017 08:49 AM, Kevin Doran wrote: Thanks for kicking off this discussion thread, Bryan. I support prepping a release of NiFi Registry and making it version 0.1.0 as you propose. Thanks! Kevin On 12/7/17, 10:45, "Joe Witt" wrote: Bryan - very exciting and awesome. Having experimented with the registry on the JIRAs/PRs you mention I must say this is going to be a huge step forward for NiFi! Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to volunteer to RM that as well if needed. Thanks On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende wrote: > Hey folks, > > There has been a lot of great work done on the NiFi Registry [1] and I > think we are probably very close to an initial release focused on > storing "versioned flows". > > Since NiFi will have a dependency on client code provided by the > registry, the first release of the registry would need to occur before > the first release of NiFi that integrates with it. The work on the > NiFi side is being done as part of NIFI-4436, which can be followed > along on PR 2219 [2]. > > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would > propose the first release should be 0.1.0. > > Let me know if anyone has any thoughts or comments. I'm happy to act > as RM if no one else is interested in doing so, and we can start the > process of going through JIRA to see what is left. > > Thanks, > > Bryan > > [1] https://nifi.apache.org/registry.html > [2] https://github.com/apache/nifi/pull/2219
Re: [DISCUSS] First Release of NiFi Registry
Thanks for kicking off this discussion thread, Bryan. I support prepping a release of NiFi Registry and making it version 0.1.0 as you propose. Thanks! Kevin On 12/7/17, 10:45, "Joe Witt" wrote: Bryan - very exciting and awesome. Having experimented with the registry on the JIRAs/PRs you mention I must say this is going to be a huge step forward for NiFi! Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to volunteer to RM that as well if needed. Thanks On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende wrote: > Hey folks, > > There has been a lot of great work done on the NiFi Registry [1] and I > think we are probably very close to an initial release focused on > storing "versioned flows". > > Since NiFi will have a dependency on client code provided by the > registry, the first release of the registry would need to occur before > the first release of NiFi that integrates with it. The work on the > NiFi side is being done as part of NIFI-4436, which can be followed > along on PR 2219 [2]. > > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would > propose the first release should be 0.1.0. > > Let me know if anyone has any thoughts or comments. I'm happy to act > as RM if no one else is interested in doing so, and we can start the > process of going through JIRA to see what is left. > > Thanks, > > Bryan > > [1] https://nifi.apache.org/registry.html > [2] https://github.com/apache/nifi/pull/2219
Re: [DISCUSS] First Release of NiFi Registry
Bryan - very exciting and awesome. Having experimented with the registry on the JIRAs/PRs you mention I must say this is going to be a huge step forward for NiFi! Since we'll also be doing a NiFi release soon (1.5.0?) I am happy to volunteer to RM that as well if needed. Thanks On Thu, Dec 7, 2017 at 10:39 AM, Bryan Bende wrote: > Hey folks, > > There has been a lot of great work done on the NiFi Registry [1] and I > think we are probably very close to an initial release focused on > storing "versioned flows". > > Since NiFi will have a dependency on client code provided by the > registry, the first release of the registry would need to occur before > the first release of NiFi that integrates with it. The work on the > NiFi side is being done as part of NIFI-4436, which can be followed > along on PR 2219 [2]. > > Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would > propose the first release should be 0.1.0. > > Let me know if anyone has any thoughts or comments. I'm happy to act > as RM if no one else is interested in doing so, and we can start the > process of going through JIRA to see what is left. > > Thanks, > > Bryan > > [1] https://nifi.apache.org/registry.html > [2] https://github.com/apache/nifi/pull/2219
[DISCUSS] First Release of NiFi Registry
Hey folks, There has been a lot of great work done on the NiFi Registry [1] and I think we are probably very close to an initial release focused on storing "versioned flows". Since NiFi will have a dependency on client code provided by the registry, the first release of the registry would need to occur before the first release of NiFi that integrates with it. The work on the NiFi side is being done as part of NIFI-4436, which can be followed along on PR 2219 [2]. Currently nifi-registry master is set to 0.0.1-SNAPSHOT, but I would propose the first release should be 0.1.0. Let me know if anyone has any thoughts or comments. I'm happy to act as RM if no one else is interested in doing so, and we can start the process of going through JIRA to see what is left. Thanks, Bryan [1] https://nifi.apache.org/registry.html [2] https://github.com/apache/nifi/pull/2219
RE: [EXT] Re: [DISCUSS] CI / Travis / Jenkins
Best news I've heard in a long time :) Thanks for all the hard work! -Original Message- From: af...@fucs.org [mailto:af...@fucs.org] On Behalf Of Andre Sent: Thursday, December 07, 2017 12:15 To: dev@nifi.apache.org Subject: [EXT] Re: [DISCUSS] CI / Travis / Jenkins Joe, Thanks for that! yes, you are correct. The builds occurred twice as the parallel build was broken. :-) Since we can now build and run tests in parallel with contrib-check (YEAY) the dual build is no longer necessary. THANK YOU for chasing this down. While travis is one of the drivers all the developers will benefit from being able to run contrib-check in parallel moving forward. Cheers On Thu, Dec 7, 2017 at 1:38 PM, Joe Witt wrote: > Team, > > Ok so finally some really solid news to share on the Travis-CI front. > First, huge thanks to Aldrin for getting this started and folks like > Andre and Pierre who have tweaked it to make it more usable as well. > After a long run of it helping us out as we all know it went poorly > with every build failing for what seemed like months. > > After some improvements and updates to our usage of maven which now > means parallel builds with contrib check seem to be working and after > going ruthless mode on hunting down unstable tests and either fixing > them or making them integration-tests the build is far more stable. > We all need to try and stay on top of that. Today though i realized > that our builds were happening twice and that appeared to be why it > took roughly 50 minutes to finish, at best, and we'd timeout and fail. > So after adjusting our travis.yml we now only build once and the > process takes about 25 mins so we're well within. > > Latest build on travis-ci: https://travis-ci.org/apache/ > nifi/builds/312629807 > Appveyor builds: > https://ci.appveyor.com/project/ApacheSoftwareFoundation/nifi/ > build/1.0.0-SNAPSHOT-6649 > > So we're heading in the right direction. If it stays stable perhaps > we could add openjdk builds as well. > > THanks > Joe > > On Tue, Dec 5, 2017 at 4:11 PM, Joe Witt wrote: > > OK well things are looking pretty good. The only obvious problem > > now is that our builds take about 45-50 mins on travis-ci.org and > > the build time limit is 50 mins [1] so some jobs get killed. > > > > Will look at areas we can avoid spending build time on at least in > > travis-ci land. Probably no great option but let's see. > > > > [1] > > https://docs.travis-ci.com/user/customizing-the-build#Build-Timeouts > > > > On Tue, Dec 5, 2017 at 2:56 PM, Joe Witt wrote: > >> Will try it out for PR https://github.com/apache/nifi/pull/2319 > >> which is being built under > >> https://travis-ci.org/apache/nifi/builds/312043710 > >> > >> On Tue, Dec 5, 2017 at 2:51 PM, Joe Witt wrote: > >>> Andre > >>> > >>> Thanks - read through > >>> https://issues.apache.org/jira/browse/NIFI-1657 > >>> where this was discussed and where the relevant multi-env commit > >>> came in. > >>> > >>> Seems like five environments may be too taxing based on the build > >>> failures I'm observing. I'll cut it down to three FR JP US For > >>> now. We can evaluate if that helps at all and add more back if > >>> things become stable. > >>> > >>> Thanks > >>> Joe > >>> > >>> On Tue, Dec 5, 2017 at 12:20 AM, Andre wrote: > Joe, > > Glad to help! Few notes: > > If I recall correctly there was a reason we chose to add default > and > BR but > to be honest I can't really remember what it was. I think it has > to > do with > Time Zones + Locale issues and has helped detecting bizarre > issues on > time > based junits (Matt B and Pierre may remember this). > > Regarding the rat check. The idea behind that was a fast failure > in > case of > basic style violations, rather than wait until the end of the > compilation. > To be honest I don't know if this has worked as desired but > should > allow us > to quickly identify validation errors which if I recall correctly > were only > detected at the end of contrib-check. > > And apologies for the anecdotal comments. I am away from my dev > environment > atm so I can't truly validate them. > > > Kind regards > > > On Tue, Dec 5, 2017 at 3:31 PM, Joe Witt wrote: > > > Great news! So for the first time in a long time we now have > > travis-ci builds passing! > > > > I incorporated Dustin's PR which changed to the -Ddir-only > > instead of -P, added Andre's idea of dropping the -quiet flag, > > and dropped the number of builds in the config to a single > > parallel build with > contrib > > check now that we're seeing those pass with rat/checkstyle. > > > > https://travis-ci.org/apache/nifi/builds/311660398 > > > > A couple failed due to test failures and I filed JIRAs to > > convert these into integration tests or resolve. > >>
