Timeline - we remain in full blitz mode to get things ready for 2.0. No clear ETA but we need to be getting it out soon. At least a milestone release of it for people to work with. There is a big change needed to get rid of the flow.xml.gz in favor of the json form and that is in progress. I am not sure offhand whether templates got the boot yet.
Latest fun is wrestling our rather messy situation with Groovy in the build as that seems not ready for Java 21 generally. On Fri, Sep 15, 2023 at 10:19 AM Ryan Hendrickson < [email protected]> wrote: > I think NiFi 2.x going to Java 21 for all the reasons outlined makes a lot > of sense. > > Is there a timeline for 2.x? > > On Mon, Sep 11, 2023 at 5:00 AM Pierre Villard < > [email protected]> > wrote: > > > Thanks Joe, it makes total sense and I agree that the ones that would > > likely be slow at adopting Java 21 would not go to NiFi 2.0 super quickly > > anyway. Being able to bring the latest and greatest in NiFi is great and > > given all of the features announced in Java 21, I imagine a lot of > projects > > we depend on will be doing the same. > > > > Le jeu. 7 sept. 2023 à 19:36, Joe Witt <[email protected]> a écrit : > > > > > Pierre > > > > > > A few concerns you raised so want to address my view on each: > > > > > > Will users be able to adopt Java 21 fast enough? > > > I share Brandon's view on that in terms of their adoption timeline. It > > > will likely align well with NiFi 2.0 itself in my estimation. > > > > > > Will this delay NiFi 2.0? > > > If it would then I'd not be supportive. I don't think we need to > bother > > > with adopting any of the features now. What I would like us to have is > > the > > > option to adopt them as we progress. We should get 2.0 done asap and > if > > > this added delay then I'd be way less interested in this idea. > > > > > > Feature benefits of 21 and what will that bring? > > > Mark spoke well to the key one that stood out to me which was the new > > > threading model available. It would be awfully nice to leverage that > for > > > the efficiency it represents and especially if it can reduce some of > our > > > heap usage which is valuable in cloud/shared compute contexts. > > > > > > Performance benefits of Java 21? > > > It appears from some analysis found with googling that Java 21 brings > out > > > of the box 4-5% performance increases generally. Not amazing but > useful. > > > > > > User experience otherwise with Java 21? > > > I believe it would be consistent with Java 17 for their point of view > in > > > terms of install/config/etc.. > > > > > > My motivation for this is fairly pure honestly. Since we're setting > our > > > new minimum bar that lives for as long as the 2.x release line lives > I'd > > > like to set it at the current LTS available when we ship that line as > > well. > > > > > > Thanks > > > > > > > > > > > > On Thu, Sep 7, 2023 at 4:22 AM Brandon DeVries < > > [email protected]> > > > wrote: > > > > > > > +1 to requiring java 21. Starting off as "up to date" as possibly > > makes a > > > > lot of sense, and some of the new features seem especially relevant > to > > > NiFi. > > > > > > > > I definitely understand the concerns about organizations being > willing > > / > > > > able to approve Java 21... But those same organizations might also be > > > > hesitant to move to NiFi 2.0. We will continue to support java 17 & > > NiFi > > > > 1.x for some time, so hopefully those groups will have the time they > > need > > > > to get approvals, do evaluations, and upgrade. > > > > > > > > Brandon > > > > ________________________________ > > > > From: Pierre Villard <[email protected]> > > > > Sent: Thursday, September 7, 2023 6:15:58 AM > > > > To: [email protected] <[email protected]> > > > > Subject: Re: [discuss] nifi 2.0 and Java 21… > > > > > > > > Hi all, > > > > > > > > I share the concerns raised by Chris regarding how quickly users of > > NiFi > > > > will be able to adopt Java 21. > > > > > > > > While I'm definitely in favor of using the latest and greatest, > > > especially > > > > when it brings to the table such significant features, we need to be > > > > careful as it may significantly delay the adoption of NiFi 2.0 in big > > > > companies where deploying Java 21 will take time. I acknowledge that > > > going > > > > from Java 8 to Java 17 is certainly the same effort as going from > Java > > 8 > > > to > > > > Java 21 but how quickly security-sensitive environments will adopt a > > new > > > > release of Java that is completely new? > > > > > > > > In addition to that, it sounds like we would add a significant rework > > of > > > > the framework in NiFi 2.0 assuming we adopt Java 21 as the minimum > > > version. > > > > Do we think this is going to significantly delay the first release of > > > NiFi > > > > 2.0? We still have work to do but adding this on top could delay the > > > > release, no? > > > > > > > > Finally, the features that Java 21 are bringing sound super > interesting > > > in > > > > the context of NiFi but do we already have an idea of what it will > > > improve? > > > > is it the user experience, and if so, how will it change? is it the > > > > performance, and if so, do we have an idea of how things will > improve? > > > > > > > > Thanks, > > > > Pierre > > > > > > > > Le mer. 6 sept. 2023 à 23:07, Chris Sampson > > > > <[email protected]> a écrit : > > > > > > > > > Yeah, I understand the need to move to 21 as a minimum to take > > > advantage > > > > of > > > > > its features. Hopefully the wider java ecosystem won't be an issue > in > > > the > > > > > short term. > > > > > > > > > > I just wanted the discussion to be clear about this being a change > to > > > the > > > > > Java baseline/minimum for NiFi 2.0. > > > > > > > > > > It's a +1 from me. > > > > > > > > > > On Wed, 6 Sept 2023, 19:01 Joe Witt, <[email protected]> wrote: > > > > > > > > > > > Chris > > > > > > > > > > > > My suggestion is rooted in making Java 21 the minimum of the NiFi > > 2.0 > > > > > > line. It would not work on Java 17. The reason for this is so > > that > > > we > > > > > can > > > > > > leverage the longest duration of a given LTS line while also > > > benefiting > > > > > > from the language improvements that affords. Maintaining > > > compatibility > > > > > > with future versions we generally have to do. But whatever the > > > minimum > > > > > > version we accept dictates which language features we can > leverage. > > > So > > > > > if > > > > > > it is 17 then we can't leverage anything from the 21 line for > > > example. > > > > > > > > > > > > GIven the nature and timelines of LTS I don't really think there > is > > > the > > > > > > same burn in logic that we'd have all known in the past before > the > > > > > > LTS/STS/etc.. release constructs existed. > > > > > > > > > > > > Thanks > > > > > > > > > > > > On Wed, Sep 6, 2023 at 10:53 AM Chris Sampson > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > To be clear, is the discussion one of making Java 21 the > minimum > > > > > > > requirement for NiFi 2.0.0, or rather NiFi 2.x be compatible > with > > > > Java > > > > > > 21, > > > > > > > while retaining Java 17 as a minimum? > > > > > > > > > > > > > > If we moved straight to a Java 21 requirement, will we run into > > > > > > > compatibility issues that delay initial NiFi 2 release? Will > the > > > move > > > > > to > > > > > > > Java 21 mean some organisations delay their migration to NiFi 2 > > > > through > > > > > > not > > > > > > > wanting to move to the latest Java LTS version until it's had a > > > time > > > > > for > > > > > > > "settling" through security/bug patches, etc.? And are either > of > > > > these > > > > > > > sufficient concern to pause Java 21 becoming the requirement, > as > > we > > > > may > > > > > > > then need to extend NiFi 1.x maintenance for longer into the > > > future? > > > > > > > > > > > > > > Generally, I'm in favour of moving to "latest and greatest", > > > > > particularly > > > > > > > for LTS versions of technologies, but the rate of Java version > > > > adoption > > > > > > > across the community gives me pause. > > > > > > > > > > > > > > I certainly see the advantage of new Java features for NiFi in > > Java > > > > 21, > > > > > > > such as the already mentioned virtual threads. > > > > > > > > > > > > > > On Wed, 6 Sept 2023, 18:34 Mike Thomsen, < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > +1 100% > > > > > > > > > > > > > > > > On Wed, Sep 6, 2023 at 11:48 AM Adam Taft <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > Yes, please. +1 Exactly what Mark said. Virtual threads > have > > > > > > potential > > > > > > > to > > > > > > > > > be extremely impactful to applications like NiFi. > > > > > > > > > > > > > > > > > > /Adam > > > > > > > > > > > > > > > > > > On Wed, Sep 6, 2023 at 7:26 AM Mark Payne < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Thanks for bringing his up, Joe. > > > > > > > > > > > > > > > > > > > > I would definitely be a +1. I think the new virtual > thread > > > > > concept > > > > > > > > would > > > > > > > > > > have great impact on us. > > > > > > > > > > It would allow us to significantly simplify our > scheduling > > > > logic, > > > > > > > which > > > > > > > > > > would help with code maintainability > > > > > > > > > > but would also make configuration simpler. This is one of > > the > > > > > most > > > > > > > > > > difficult things for users to configure, > > > > > > > > > > and I would very much welcome the ability to simplify > this. > > > It > > > > > > would > > > > > > > > > > likely also yield better off-heap memory > > > > > > > > > > utilization by reducing the number of native threads > > > necessary. > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > -Mark > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sep 6, 2023, at 10:20 AM, Joe Witt < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Team > > > > > > > > > > > > > > > > > > > > > > Thought it might be worth relighting this thread with > > Java > > > 21 > > > > > GA > > > > > > > > > > imminent. > > > > > > > > > > > Given the timing we should give consideration to having > > > Java > > > > 21 > > > > > > as > > > > > > > > the > > > > > > > > > > > basis for nifi 2.x to buy maximum time with LTS > > alignment. > > > > > There > > > > > > > are > > > > > > > > > > also > > > > > > > > > > > a couple interesting language features we can likely > take > > > > > > advantage > > > > > > > > of. > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > Joe > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 21, 2023 at 6:21 AM David Handermann < > > > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > >> Hi Dirk, > > > > > > > > > > >> > > > > > > > > > > >> Thanks for summarizing your findings in the referenced > > > Jira > > > > > > > issues. > > > > > > > > It > > > > > > > > > > >> sounds like subsequent discussion of Nashorn support > may > > > be > > > > > > better > > > > > > > > on > > > > > > > > > a > > > > > > > > > > new > > > > > > > > > > >> thread. > > > > > > > > > > >> > > > > > > > > > > >> The Spring 6 and Jetty 11 upgrades are going to > require > > > > > > > significant > > > > > > > > > > work. > > > > > > > > > > >> One incremental step in that direction was making Java > > 17 > > > > the > > > > > > > > minimum > > > > > > > > > > >> version, and upgrading to Jetty 10 should also help > move > > > > > things > > > > > > > > > forward. > > > > > > > > > > >> > > > > > > > > > > >> Although compiling NiFi modules with a reference to > the > > > > > > standalone > > > > > > > > > > Nashorn > > > > > > > > > > >> library may introduce issues, there should be other > > > options > > > > > for > > > > > > > > > > referencing > > > > > > > > > > >> the library at runtime. That requires custom class > > > loading, > > > > > > which > > > > > > > > some > > > > > > > > > > >> Processors support, so that seems like the general > > > direction > > > > > to > > > > > > > go. > > > > > > > > > > >> > > > > > > > > > > >> If you have additional findings, feel free to start a > > new > > > > > > > developer > > > > > > > > > list > > > > > > > > > > >> thread and that may gather additional feedback. > > > > > > > > > > >> > > > > > > > > > > >> Regards, > > > > > > > > > > >> David Handermann > > > > > > > > > > >> > > > > > > > > > > >> On Wed, Jun 21, 2023 at 12:17 AM Dirk Arends < > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > >> > > > > > > > > > > >>> Since initially raising concerns about the move to > Java > > > 17 > > > > > > losing > > > > > > > > > > >> Nashorn, > > > > > > > > > > >>> I have been investigating the suggestion to use > Nashorn > > > as > > > > a > > > > > > > > > standalone > > > > > > > > > > >>> package as potential easier alternative to GraalVM. > [1] > > > > > > > > > > >>> > > > > > > > > > > >>> While making some progress, a number of issues have > > been > > > > > > > > encountered > > > > > > > > > > >> which > > > > > > > > > > >>> I haven't been able to resolve as yet. More details > are > > > > > > included > > > > > > > in > > > > > > > > > > >>> relevant JIRA tickets, but summarising: > > > > > > > > > > >>> > > > > > > > > > > >>> - Building NiFi with a recent Nashorn dependency > leads > > to > > > > > > errors > > > > > > > > > > >>> "Unsupported class file major version 61" [2] > > > > > > > > > > >>> - Building NiFi using Java 17 highlights issues with > > the > > > > > > current > > > > > > > > > Jetty > > > > > > > > > > >>> version, which I believe would require an upgrade > from > > > > 9.4.51 > > > > > > to > > > > > > > > > > 11.0.15 > > > > > > > > > > >>> [3] > > > > > > > > > > >>> - Jetty 11 then requires an upgrade of the Spring > > > Framework > > > > > > > > version 5 > > > > > > > > > > to > > > > > > > > > > >> 6. > > > > > > > > > > >>> [4] > > > > > > > > > > >>> > > > > > > > > > > >>> The current steps to remove references to > "Javascript" > > > as a > > > > > > > > > > preinstalled > > > > > > > > > > >>> scripting language [5] are understandable, but it > does > > > seem > > > > > > there > > > > > > > > is > > > > > > > > > > >> still > > > > > > > > > > >>> quite a bit to do before Nashorn or another external > > > > > scripting > > > > > > > > engine > > > > > > > > > > >> could > > > > > > > > > > >>> be used. > > > > > > > > > > >>> > > > > > > > > > > >>> [1] https://issues.apache.org/jira/browse/NIFI-11700 > : > > > Java > > > > > 17 > > > > > > > > > Nashorn > > > > > > > > > > >>> standalone support > > > > > > > > > > >>> [2] https://issues.apache.org/jira/browse/NIFI-11701 > : > > > > > Support > > > > > > > > > building > > > > > > > > > > >>> with > > > > > > > > > > >>> version 61 class files > > > > > > > > > > >>> [3] https://issues.apache.org/jira/browse/NIFI-11702 > : > > > > > Upgrade > > > > > > > > Jetty > > > > > > > > > to > > > > > > > > > > >>> version 11 > > > > > > > > > > >>> [4] https://issues.apache.org/jira/browse/NIFI-11703 > : > > > > > Upgrade > > > > > > > > Spring > > > > > > > > > > >>> Framework to version 6 > > > > > > > > > > >>> [5] https://issues.apache.org/jira/browse/NIFI-11713 > : > > > > Remove > > > > > > > > > > Deprecated > > > > > > > > > > >>> ECMAScript Support > > > > > > > > > > >>> > > > > > > > > > > >>> Regards, > > > > > > > > > > >>> Dirk Arends > > > > > > > > > > >>> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
