Team, I think we are there! Going to kick out RC1 now.
Thanks On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <joe.w...@gmail.com> wrote: > Team, > > As you can see I've not kicked off the RC yet. Many bug fixes/dependency > updates are happening. Ideally we'll wait until nar Maven plugin goes and > we're trying to fix some nifi registry/nifi interaction issues as well. > Still will get the RC out as soon as we can. > > Thanks > > On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <joe.w...@gmail.com> wrote: > >> Hello >> >> Here is a good picture of what the 1.20 RC looks like. I've >> tagged several JIRAs today to ensure we get them in. A theme is really >> around stabilizing nifi/nifi-registry integration as we're seeing >> substantial uptick in usage and thus various community reported findings. >> We'll get that quite smooth with these included. >> >> https://issues.apache.org/jira/projects/NIFI/versions/12352581 >> >> Thanks >> >> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <joe.w...@gmail.com> wrote: >> >>> Team, >>> >>> I'm going through the outstanding JIRAs/PRs and flagging which look like >>> they should be 'must have' for 1.20 and then will work the RC as soon as >>> those land. >>> >>> Hopefully have the RC up within a day or two but we'll see how these >>> land as some have review comments pending action. >>> >>> Thanks >>> >>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo < >>> isha.lam...@virtualsciences.nl> wrote: >>> >>>> Hi all, >>>> >>>> I would like to contribute to the migration tooling (mostly testing I >>>> suppose) when that comes up. >>>> >>>> My team's largest client has a completely template-based pipeline with >>>> external scripts replacing variable values before deploying to target >>>> clusters, so we've already started looking at this when the goals for 2.0 >>>> were discussed and approved. The migration to flowdefinitions and >>>> parameters is quite complex and we've hit several blockers when we tried to >>>> implement a direct translation. >>>> >>>> I expect that any time I spend helping to improve the tooling will pay >>>> off handsomely for our clients. >>>> >>>> Regards, >>>> >>>> Isha >>>> >>>> -----Oorspronkelijk bericht----- >>>> Van: Adam Taft <a...@adamtaft.com> >>>> Verzonden: woensdag 11 januari 2023 23:42 >>>> Aan: dev@nifi.apache.org >>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0 >>>> >>>> This is really insightful and spot on ... >>>> >>>> Kevin wrote: >>>> > Good migration tooling will take a while to develop and test, and the >>>> > core contributors to that effort may not have sufficient variety of >>>> > flows to evaluate when the migration tools are "done" for the >>>> majority >>>> > of the community to have success upgrading to 2.x. A milestone >>>> release >>>> > would >>>> allow >>>> > us get more feedback on migration over a longer period than the vote >>>> window >>>> > of an RC candidate. >>>> >>>> It's exactly this case, that an early 2.0 release might not have had >>>> time to fully work its way through existing production deployments, that's >>>> concerning. The pace and voting of an "RC" is much too short to get any >>>> quality feedback from users in the field. >>>> >>>> I think it's really smart to consider the "Milestone" release approach >>>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time >>>> for feedback. We can put these milestones on a calendar, as needed, so that >>>> feedback is required some 'x' number of weeks/months after each milestone. >>>> >>>> And to this end, I'd personally rather see us keep the 'main' branch >>>> current with the 1.x line _until_ we're ready and are satisfied with the >>>> end goals of the 2.0 release objectives. When the milestone releases have >>>> been completed and there's a comfort level with the 2.x line, it's at the >>>> point we'd isolate the 1.x line into its own branch and switch main over to >>>> the 2.x line. >>>> >>>> This is an attractive way of: >>>> a) continuing business-as-usual with the 1.x line >>>> b) making headway on the 2.x release milestones >>>> c) giving adequate time for feedback against the 2.0 milestones coming >>>> from the field >>>> >>>> I don't mind the known-unknowns. But it's really the unknown-unknowns >>>> that are going to drive a delay in the 2.0 release. I think it's smart to >>>> be able to get some of the unknowns ironed out before we finalize the 2.0 >>>> release ceremony. The milestone approach really helps with that. >>>> >>>> /Adam >>>> >>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kdo...@apache.org> wrote: >>>> >>>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are >>>> > somewhat unrelated in my mind too :) >>>> > >>>> > I agree that good migration tooling is key. Otherwise, we risk users >>>> > staying on 1.x or creating a schism of 1.x and 2.x users. >>>> > >>>> > Good migration tooling will take a while to develop and test, and the >>>> > core contributors to that effort may not have sufficient variety of >>>> > flows to evaluate when the migration tools are "done" for the >>>> majority >>>> > of the community to have success upgrading to 2.x. A milestone >>>> release >>>> > would allow us get more feedback on migration over a longer period >>>> > than the vote window of an RC candidate. >>>> > >>>> > Perhaps we could continue to release from the 1.x line (including >>>> > minor releases with some features) until we are ready to drop the >>>> "milestone" >>>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only >>>> status. >>>> > It would be the same proposal to move main to target 2.0.0-M1, but >>>> > relax restrictions for what can land on the 1.x branch and be open to >>>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For >>>> > example, maybe we would be open to landing new/backported processors >>>> > on the 1.x branch, but not core framework features or API changes. >>>> > >>>> > This might not be necessary, but I think it is fair that saying "no >>>> > new features on 1.x" and also "no new features in 2.0.0" puts the >>>> > project in a rough place if 2.0.0 takes longer than a few months, so >>>> > if we go that route, we need to commit to a quick release of 2.0.0 >>>> > that most users can move to easily. >>>> > >>>> > Thanks, >>>> > Kevin >>>> > >>>> > On Jan 11, 2023 at 12:32:46, Joe Witt <joe.w...@gmail.com> wrote: >>>> > >>>> > > Kevin, >>>> > > >>>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are >>>> > prior >>>> > > to us officially considering it 2.0/stable. >>>> > > >>>> > > That said - the migration tooling will be key to provide as we need >>>> > > to >>>> > make >>>> > > the bridge as solid and stable as possible to help someone move >>>> from >>>> > > 1.x >>>> > to >>>> > > 2.x. I dont know how related these two concepts (milestone >>>> releases >>>> > > and 1.x to 2.x ease really are). >>>> > > >>>> > > Thanks >>>> > > >>>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kdo...@apache.org> >>>> wrote: >>>> > > >>>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts] >>>> > > >>>> > > >>>> > > On this point from David: >>>> > > >>>> > > >>>> > > We may need to have a longer release candidate period, or more >>>> > incremental >>>> > > >>>> > > > fix releases >>>> > > >>>> > > > for the initial 2.0.0 release train, but I do not expect delaying >>>> > > > a >>>> > 2.0.0 >>>> > > >>>> > > > release for new features, as that is not part of the release >>>> goals. >>>> > > >>>> > > > >>>> > > >>>> > > >>>> > > Would the community benefit from one or more milestone releases of >>>> > > 2.0, >>>> > to >>>> > > >>>> > > allow for a wider group to run / live on the proposed 2.0 prior to >>>> > > >>>> > > releasing it as "stable"? I know we've never done a milestone >>>> > > release in >>>> > > >>>> > > the past, and I'm not sure what ASF guidance is on the topic, but >>>> if >>>> > > it >>>> > > >>>> > > could be beneficial we could look into that. >>>> > > >>>> > > >>>> > > Cheers, >>>> > > >>>> > > Kevin >>>> > > >>>> > > >>>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kdo...@apache.org> wrote: >>>> > > >>>> > > >>>> > > > I think this is a good, practical discussion. >>>> > > >>>> > > > >>>> > > >>>> > > > On the one hand, we can't put off 2.x any longer as we really >>>> need >>>> > > > to >>>> > > >>>> > > > updated the minimum required Java to 11. Updating main >>>> development >>>> > > > to >>>> > > >>>> > > > target 2.x feels like a good way drive progress on that along >>>> with >>>> > > > the >>>> > > >>>> > > > other 2.0 goals. >>>> > > >>>> > > > >>>> > > >>>> > > > On the other hand, the concerns are valid: moving all development >>>> > > > to >>>> > > >>>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on >>>> > > > a >>>> > > >>>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this >>>> > > > stated >>>> > > >>>> > > > release goal feels risky to me: >>>> > > >>>> > > > >>>> > > >>>> > > > Implement Migration Tools for Upgrading Flows >>>> > > >>>> > > > >>>> > > >>>> > > > >>>> > > >>>> > > > - Implement automated migration where possible to remap >>>> > > > properties >>>> > and >>>> > > >>>> > > > features >>>> > > >>>> > > > - Implement migration tools for manual conversion of XML >>>> > Templates >>>> > > >>>> > > > to JSON Flow Definitions >>>> > > >>>> > > > - Create documentation for manual steps necessary where >>>> > > >>>> > > > programmatic migration cannot be implemented >>>> > > >>>> > > > - NiFi 2.0 should be capable of starting with ghosted >>>> > > > components >>>> > > >>>> > > > for removed Processors or Controller Services >>>> > > >>>> > > > >>>> > > >>>> > > > >>>> > > >>>> > > > Removing deprecated components should be fairly straightforward >>>> > > > and >>>> > > >>>> > > quick, >>>> > > >>>> > > > but automating and documenting migration is a large effort. >>>> > > >>>> > > > >>>> > > >>>> > > > On this po >>>> > > >>>> > > > >>>> > > >>>> > > > >>>> > > >>>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bbe...@gmail.com> >>>> wrote: >>>> > > >>>> > > > >>>> > > >>>> > > >> The plan as I understand it is not to diverge and create separate >>>> > > >>>> > > feature >>>> > > >>>> > > >> development on the 1.x line, so I would expect all PRs to >>>> > > >> continue to >>>> > be >>>> > > >>>> > > >> submitted only to main. We would release 1.x as needed with >>>> major >>>> > > >> bug >>>> > > >>>> > > >> fixes >>>> > > >>>> > > >> or critical security updates, and these would be cherry-picked >>>> > > >> and/or >>>> > > >>>> > > >> backported as necessary, mostly without the need for PRs, the >>>> > > >> same as >>>> > we >>>> > > >>>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) >>>> > > >> back >>>> > to a >>>> > > >>>> > > >> maintenance line like (1.19.x). For precedent, we followed this >>>> > > >> same >>>> > > >>>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any >>>> > > >> major >>>> > > >>>> > > >> issue. >>>> > > >>>> > > >> >>>> > > >>>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler >>>> > > >> <ottobackwa...@gmail.com> >>>> > > >>>> > > >> wrote: >>>> > > >>>> > > >> >>>> > > >>>> > > >> It was also mentioned in another thread that we need to have >>>> > agreement >>>> > > >>>> > > on >>>> > > >>>> > > >> >>>> > > >>>> > > >> our explicit strategy and support for 1.x going forward, did that >>>> > > >>>> > > happen? >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> From: Otto Fowler <ottobackwa...@gmail.com> >>>> > > >> <ottobackwa...@gmail.com> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Reply: Otto Fowler <ottobackwa...@gmail.com> >>>> > > >> <ottobackwa...@gmail.com >>>> > > >>>> > > >>>> > > >> >>>> > > >>>> > > >> Date: January 10, 2023 at 07:02:34 >>>> > > >>>> > > >> >>>> > > >>>> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >>>> > > >> <dev@nifi.apache.org> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0 >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> There needs to be an update to the contributing guide as to how >>>> > > >> to >>>> > > >>>> > > submit >>>> > > >>>> > > >> >>>> > > >>>> > > >> PRs to 1.x or 2.x etc. >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> From: Joe Witt <joew...@apache.org> <joew...@apache.org> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Reply: dev@nifi.apache.org <dev@nifi.apache.org> >>>> > > >> <dev@nifi.apache.org >>>> > > >>>> > > >>>> > > >> >>>> > > >>>> > > >> Date: January 9, 2023 at 15:53:16 >>>> > > >>>> > > >> >>>> > > >>>> > > >> To: dev@nifi.apache.org <dev@nifi.apache.org> >>>> > > >> <dev@nifi.apache.org> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0 >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Team, >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> As David mentioned in [1] following a successful NiFi 2.0 >>>> release >>>> > > >> goal >>>> > > >>>> > > >> >>>> > > >>>> > > >> planning - we are now going to start moving the 'main' line to >>>> be >>>> > > >> the >>>> > > >>>> > > NiFi >>>> > > >>>> > > >> >>>> > > >>>> > > >> 2.0 line which will allow for the key work to take place. We >>>> will >>>> > > >> also >>>> > > >>>> > > >> >>>> > > >>>> > > >> move niFi 1.x to its appropriate support line. >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 >>>> > > >> and we >>>> > > >>>> > > have >>>> > > >>>> > > >> >>>> > > >>>> > > >> work in there including security items so it is time to make a >>>> > release. >>>> > > >>>> > > >> >>>> > > >>>> > > >> The intent then is to initiate 1.20 and immediate after that >>>> > > >> change >>>> > > >>>> > > 'main' >>>> > > >>>> > > >> >>>> > > >>>> > > >> to 2.0. >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Going forward then all work on the 1.x line should be focused on >>>> > > >>>> > > >> >>>> > > >>>> > > >> maintaining existing features, dependencies, and helping 1.x >>>> > > >> users >>>> > > >>>> > > migrate >>>> > > >>>> > > >> >>>> > > >>>> > > >> to the 2.x line. Otherwise, new feature work will happen on >>>> > > >> 'main' as >>>> > it >>>> > > >>>> > > >> >>>> > > >>>> > > >> normally does and will come out in the NiFi 2.x release line. >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Please flag key outstanding items as we narrow down the release >>>> > > >>>> > > candidate >>>> > > >>>> > > >> >>>> > > >>>> > > >> for NiFi 1.20. >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> Thanks >>>> > > >>>> > > >> >>>> > > >>>> > > >> Joe >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> [1] >>>> > > >> >>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2 >>>> > > >> Flists.apache.org >>>> %2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat >>>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl >>>> %7Ccbea974a2c1f479d48 >>>> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809 >>>> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo >>>> > > >> >>>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0 >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >> >>>> > > >>>> > > >>>> > > >>>> > >>>> >>>
