Team As you commit to the main line now please start using whatever the latest relevant feature release is on the 2.x line which right now would be 2.0.0. If we find we need to do a 1.21 and so on we'll pull those commits down as we would for any other mx release in the past but the 'main' line becomes 2.0.0 now.
Thanks Joe On Mon, Feb 6, 2023 at 10:44 AM Joe Witt <joe.w...@gmail.com> wrote: > Team, > > I think we are there! Going to kick out RC1 now. > > Thanks > > On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <joe.w...@gmail.com> wrote: > >> Team, >> >> As you can see I've not kicked off the RC yet. Many bug fixes/dependency >> updates are happening. Ideally we'll wait until nar Maven plugin goes and >> we're trying to fix some nifi registry/nifi interaction issues as well. >> Still will get the RC out as soon as we can. >> >> Thanks >> >> On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <joe.w...@gmail.com> wrote: >> >>> Hello >>> >>> Here is a good picture of what the 1.20 RC looks like. I've >>> tagged several JIRAs today to ensure we get them in. A theme is really >>> around stabilizing nifi/nifi-registry integration as we're seeing >>> substantial uptick in usage and thus various community reported findings. >>> We'll get that quite smooth with these included. >>> >>> https://issues.apache.org/jira/projects/NIFI/versions/12352581 >>> >>> Thanks >>> >>> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <joe.w...@gmail.com> wrote: >>> >>>> Team, >>>> >>>> I'm going through the outstanding JIRAs/PRs and flagging which look >>>> like they should be 'must have' for 1.20 and then will work the RC as soon >>>> as those land. >>>> >>>> Hopefully have the RC up within a day or two but we'll see how these >>>> land as some have review comments pending action. >>>> >>>> Thanks >>>> >>>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo < >>>> isha.lam...@virtualsciences.nl> wrote: >>>> >>>>> Hi all, >>>>> >>>>> I would like to contribute to the migration tooling (mostly testing I >>>>> suppose) when that comes up. >>>>> >>>>> My team's largest client has a completely template-based pipeline with >>>>> external scripts replacing variable values before deploying to target >>>>> clusters, so we've already started looking at this when the goals for 2.0 >>>>> were discussed and approved. The migration to flowdefinitions and >>>>> parameters is quite complex and we've hit several blockers when we tried >>>>> to >>>>> implement a direct translation. >>>>> >>>>> I expect that any time I spend helping to improve the tooling will pay >>>>> off handsomely for our clients. >>>>> >>>>> Regards, >>>>> >>>>> Isha >>>>> >>>>> -----Oorspronkelijk bericht----- >>>>> Van: Adam Taft <a...@adamtaft.com> >>>>> Verzonden: woensdag 11 januari 2023 23:42 >>>>> Aan: dev@nifi.apache.org >>>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0 >>>>> >>>>> This is really insightful and spot on ... >>>>> >>>>> Kevin wrote: >>>>> > Good migration tooling will take a while to develop and test, and >>>>> the >>>>> > core contributors to that effort may not have sufficient variety of >>>>> > flows to evaluate when the migration tools are "done" for the >>>>> majority >>>>> > of the community to have success upgrading to 2.x. A milestone >>>>> release >>>>> > would >>>>> allow >>>>> > us get more feedback on migration over a longer period than the vote >>>>> window >>>>> > of an RC candidate. >>>>> >>>>> It's exactly this case, that an early 2.0 release might not have had >>>>> time to fully work its way through existing production deployments, that's >>>>> concerning. The pace and voting of an "RC" is much too short to get any >>>>> quality feedback from users in the field. >>>>> >>>>> I think it's really smart to consider the "Milestone" release approach >>>>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of >>>>> time >>>>> for feedback. We can put these milestones on a calendar, as needed, so >>>>> that >>>>> feedback is required some 'x' number of weeks/months after each milestone. >>>>> >>>>> And to this end, I'd personally rather see us keep the 'main' branch >>>>> current with the 1.x line _until_ we're ready and are satisfied with the >>>>> end goals of the 2.0 release objectives. When the milestone releases have >>>>> been completed and there's a comfort level with the 2.x line, it's at the >>>>> point we'd isolate the 1.x line into its own branch and switch main over >>>>> to >>>>> the 2.x line. >>>>> >>>>> This is an attractive way of: >>>>> a) continuing business-as-usual with the 1.x line >>>>> b) making headway on the 2.x release milestones >>>>> c) giving adequate time for feedback against the 2.0 milestones coming >>>>> from the field >>>>> >>>>> I don't mind the known-unknowns. But it's really the unknown-unknowns >>>>> that are going to drive a delay in the 2.0 release. I think it's smart to >>>>> be able to get some of the unknowns ironed out before we finalize the 2.0 >>>>> release ceremony. The milestone approach really helps with that. >>>>> >>>>> /Adam >>>>> >>>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kdo...@apache.org> >>>>> wrote: >>>>> >>>>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are >>>>> > somewhat unrelated in my mind too :) >>>>> > >>>>> > I agree that good migration tooling is key. Otherwise, we risk users >>>>> > staying on 1.x or creating a schism of 1.x and 2.x users. >>>>> > >>>>> > Good migration tooling will take a while to develop and test, and >>>>> the >>>>> > core contributors to that effort may not have sufficient variety of >>>>> > flows to evaluate when the migration tools are "done" for the >>>>> majority >>>>> > of the community to have success upgrading to 2.x. A milestone >>>>> release >>>>> > would allow us get more feedback on migration over a longer period >>>>> > than the vote window of an RC candidate. >>>>> > >>>>> > Perhaps we could continue to release from the 1.x line (including >>>>> > minor releases with some features) until we are ready to drop the >>>>> "milestone" >>>>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only >>>>> status. >>>>> > It would be the same proposal to move main to target 2.0.0-M1, but >>>>> > relax restrictions for what can land on the 1.x branch and be open >>>>> to >>>>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For >>>>> > example, maybe we would be open to landing new/backported processors >>>>> > on the 1.x branch, but not core framework features or API changes. >>>>> > >>>>> > This might not be necessary, but I think it is fair that saying "no >>>>> > new features on 1.x" and also "no new features in 2.0.0" puts the >>>>> > project in a rough place if 2.0.0 takes longer than a few months, so >>>>> > if we go that route, we need to commit to a quick release of 2.0.0 >>>>> > that most users can move to easily. >>>>> > >>>>> > Thanks, >>>>> > Kevin >>>>> > >>>>> > On Jan 11, 2023 at 12:32:46, Joe Witt <joe.w...@gmail.com> wrote: >>>>> > >>>>> > > Kevin, >>>>> > > >>>>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that >>>>> are >>>>> > prior >>>>> > > to us officially considering it 2.0/stable. >>>>> > > >>>>> > > That said - the migration tooling will be key to provide as we >>>>> need >>>>> > > to >>>>> > make >>>>> > > the bridge as solid and stable as possible to help someone move >>>>> from >>>>> > > 1.x >>>>> > to >>>>> > > 2.x. I dont know how related these two concepts (milestone >>>>> releases >>>>> > > and 1.x to 2.x ease really are). >>>>> > > >>>>> > > Thanks >>>>> > > >>>>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kdo...@apache.org> >>>>> wrote: >>>>> > > >>>>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts] >>>>> > > >>>>> > > >>>>> > > On this point from David: >>>>> > > >>>>> > > >>>>> > > We may need to have a longer release candidate period, or more >>>>> > incremental >>>>> > > >>>>> > > > fix releases >>>>> > > >>>>> > > > for the initial 2.0.0 release train, but I do not expect >>>>> delaying >>>>> > > > a >>>>> > 2.0.0 >>>>> > > >>>>> > > > release for new features, as that is not part of the release >>>>> goals. >>>>> > > >>>>> > > > >>>>> > > >>>>> > > >>>>> > > Would the community benefit from one or more milestone releases of >>>>> > > 2.0, >>>>> > to >>>>> > > >>>>> > > allow for a wider group to run / live on the proposed 2.0 prior to >>>>> > > >>>>> > > releasing it as "stable"? I know we've never done a milestone >>>>> > > release in >>>>> > > >>>>> > > the past, and I'm not sure what ASF guidance is on the topic, but >>>>> if >>>>> > > it >>>>> > > >>>>> > > could be beneficial we could look into that. >>>>> > > >>>>> > > >>>>> > > Cheers, >>>>> > > >>>>> > > Kevin >>>>> > > >>>>> > > >>>>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kdo...@apache.org> >>>>> wrote: >>>>> > > >>>>> > > >>>>> > > > I think this is a good, practical discussion. >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > On the one hand, we can't put off 2.x any longer as we really >>>>> need >>>>> > > > to >>>>> > > >>>>> > > > updated the minimum required Java to 11. Updating main >>>>> development >>>>> > > > to >>>>> > > >>>>> > > > target 2.x feels like a good way drive progress on that along >>>>> with >>>>> > > > the >>>>> > > >>>>> > > > other 2.0 goals. >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > On the other hand, the concerns are valid: moving all >>>>> development >>>>> > > > to >>>>> > > >>>>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 >>>>> on >>>>> > > > a >>>>> > > >>>>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this >>>>> > > > stated >>>>> > > >>>>> > > > release goal feels risky to me: >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > Implement Migration Tools for Upgrading Flows >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > - Implement automated migration where possible to remap >>>>> > > > properties >>>>> > and >>>>> > > >>>>> > > > features >>>>> > > >>>>> > > > - Implement migration tools for manual conversion of XML >>>>> > Templates >>>>> > > >>>>> > > > to JSON Flow Definitions >>>>> > > >>>>> > > > - Create documentation for manual steps necessary where >>>>> > > >>>>> > > > programmatic migration cannot be implemented >>>>> > > >>>>> > > > - NiFi 2.0 should be capable of starting with ghosted >>>>> > > > components >>>>> > > >>>>> > > > for removed Processors or Controller Services >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > Removing deprecated components should be fairly straightforward >>>>> > > > and >>>>> > > >>>>> > > quick, >>>>> > > >>>>> > > > but automating and documenting migration is a large effort. >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > On this po >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > >>>>> > > >>>>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bbe...@gmail.com> >>>>> wrote: >>>>> > > >>>>> > > > >>>>> > > >>>>> > > >> The plan as I understand it is not to diverge and create >>>>> separate >>>>> > > >>>>> > > feature >>>>> > > >>>>> > > >> development on the 1.x line, so I would expect all PRs to >>>>> > > >> continue to >>>>> > be >>>>> > > >>>>> > > >> submitted only to main. We would release 1.x as needed with >>>>> major >>>>> > > >> bug >>>>> > > >>>>> > > >> fixes >>>>> > > >>>>> > > >> or critical security updates, and these would be cherry-picked >>>>> > > >> and/or >>>>> > > >>>>> > > >> backported as necessary, mostly without the need for PRs, the >>>>> > > >> same as >>>>> > we >>>>> > > >>>>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) >>>>> > > >> back >>>>> > to a >>>>> > > >>>>> > > >> maintenance line like (1.19.x). For precedent, we followed this >>>>> > > >> same >>>>> > > >>>>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any >>>>> > > >> major >>>>> > > >>>>> > > >> issue. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler >>>>> > > >> <ottobackwa...@gmail.com> >>>>> > > >>>>> > > >> wrote: >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> It was also mentioned in another thread that we need to have >>>>> > agreement >>>>> > > >>>>> > > on >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> our explicit strategy and support for 1.x going forward, did >>>>> that >>>>> > > >>>>> > > happen? >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> From: Otto Fowler <ottobackwa...@gmail.com> >>>>> > > >> <ottobackwa...@gmail.com> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Reply: Otto Fowler <ottobackwa...@gmail.com> >>>>> > > >> <ottobackwa...@gmail.com >>>>> > > >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Date: January 10, 2023 at 07:02:34 >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >>>>> > > >> <dev@nifi.apache.org> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0 >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> There needs to be an update to the contributing guide as to how >>>>> > > >> to >>>>> > > >>>>> > > submit >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> PRs to 1.x or 2.x etc. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> From: Joe Witt <joew...@apache.org> <joew...@apache.org> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Reply: dev@nifi.apache.org <dev@nifi.apache.org> >>>>> > > >> <dev@nifi.apache.org >>>>> > > >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Date: January 9, 2023 at 15:53:16 >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >>>>> > > >> <dev@nifi.apache.org> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0 >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Team, >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> As David mentioned in [1] following a successful NiFi 2.0 >>>>> release >>>>> > > >> goal >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> planning - we are now going to start moving the 'main' line to >>>>> be >>>>> > > >> the >>>>> > > >>>>> > > NiFi >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> 2.0 line which will allow for the key work to take place. We >>>>> will >>>>> > > >> also >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> move niFi 1.x to its appropriate support line. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 >>>>> > > >> and we >>>>> > > >>>>> > > have >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> work in there including security items so it is time to make a >>>>> > release. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> The intent then is to initiate 1.20 and immediate after that >>>>> > > >> change >>>>> > > >>>>> > > 'main' >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> to 2.0. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Going forward then all work on the 1.x line should be focused on >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> maintaining existing features, dependencies, and helping 1.x >>>>> > > >> users >>>>> > > >>>>> > > migrate >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> to the 2.x line. Otherwise, new feature work will happen on >>>>> > > >> 'main' as >>>>> > it >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> normally does and will come out in the NiFi 2.x release line. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Please flag key outstanding items as we narrow down the release >>>>> > > >>>>> > > candidate >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> for NiFi 1.20. >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Thanks >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> Joe >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> [1] >>>>> > > >> >>>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2 >>>>> > > >> Flists.apache.org >>>>> %2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat >>>>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl >>>>> %7Ccbea974a2c1f479d48 >>>>> > > >> >>>>> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809 >>>>> > > >> >>>>> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo >>>>> > > >> >>>>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0 >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >> >>>>> > > >>>>> > > >>>>> > > >>>>> > >>>>> >>>>
