Re: hello. I would like to inquire about an error when upgrading the version.

[Pierre Villard]

Hi,

I downloaded NiFi 1.23 and I confirm that the problem is not happening
there.

However I can also confirm that this is not because of the Jackson version.
If I use NiFi 2 and an old version of Jackson (the one used in NiFi 1.23),
the problem will happen.

I will keep looking into it to understand what change could explain this
difference.

Thanks,
Pierre

Le mer. 6 nov. 2024 à 05:28, 박정화  a écrit :

> hello.
>
> The data provided appears to be sufficient for testing as we have tested it
> several times and sent the problematic data.
>
> I think it's almost certain because we've tested it multiple times on Linux
> and Mac and only the problematic data has been clearly extracted.
>
> I'm almost 100% sure it doesn't happen in version 1.23.2.
>
> If you test with the data provided in versions 1.27.0 and 1.23.2, you will
> probably be able to see that it works differently.
>
> Since the data is wrong, it is correct to change the data itself, but in
> reality it may be difficult, so we are looking for a way to solve this
> problem without losing performance.
>
> nifi 2.0.0 version has been released, but I am worried because if this
> problem is not resolved, I may not be able to upgrade to version 2.x. (1.x
> version of nar does not run in 2.x)
>

Re: hello. I would like to inquire about an error when upgrading the version.

[Pierre Villard]

Hi,

I looked into it and this seems to be a rather interesting situation :)

First of all, this does not relate to Jackson and I don't think there is a
regression in NiFi versions. I think you may have received specific data
for the first time that is causing this issue. Are you absolutely sure that
the exact same data is working in NiFi 1.23?

It is very easy to simulate the problem in a unit test on the NiFi side and
the problem appears as soon as you're using a JSON Reader and JSON Writer
with the data that you shared.

Specifically the unicode character \uD83D is problematic because, according
to this RFC [1]:
"The definition of UTF-8 prohibits encoding character numbers between
U+D800 and U+DFFF, which are reserved for use with the UTF-16 encoding form
(as surrogate pairs) and do not directly represent characters."

So it looks like you need UTF-16 when writing data - currently we use UTF-8
as the default. Do you have more details on how this data is generated?
Is {\"name\":\"\uD83D*\"} the exact data you're receiving? or do you have
additional characters in that field? If yes, can you give the full content
of the name field?

[1] https://datatracker.ietf.org/doc/html/rfc3629#page-4

Thanks,
Pierre

Le lun. 4 nov. 2024 à 14:50, 박정화  a écrit :

> hello
>
> I waited because I thought you didn't receive a reply, but I saw that you
> had responded through the following link.
>
> https://lists.apache.org/thread/qgc2ffgwv00htpykvmvxcq4fx7df2qkx
>
>
> 
>
> First of all, I would like to thank you.  I downgraded the library version
> as you said, and the error did not occur.
>
> I am attaching the error data in case it may be of help.
>
> I know the data is in the wrong format, but I was worried because it
> affected the overall data loading, and the advice you sent was very
> helpful.
>
>
> Thank you again.  have a good day.
>
>
> Data Sample
>
>
> {"timestamp":"2024-10-20T16:10:00.322+09:00","data":"[{\"name\":\"\uD83D*\"}]"}
>
>
> PartitionRecord property
> mmddhh : format(toDate( /timestamp, "-MM-dd'T'HH:mm:ss.SSSXXX",
> "Asia/Seoul"),"MMddHH")
>
> 2024년 10월 22일 (화) 오후 10:14, 박정화 (Bank) 님이 작성:
>
> > hello.
> >
> > I am using nifi well.
> >
> > I recently upgraded the version from 1.23.2 to 1.27.0 and encountered the
> > following error in the PartitionRecord process.
> >
> >
> >
> > com.fasterxml.jackson.core.JsonGenerationException: Incomplete surrogate
> > pair: first char 0xD83D, second 0x002A
> >
> >
> >
> > It seems to be a serrogate pair issue, but it seems to have been caused
> by
> > masking some characters on our side.
> >
> > Nevertheless, I wish it were possible to process json like in previous
> > versions or like python.  Is there any way?
> >
> > Could it be that the error occurring differently from the previous
> version
> > is related to the update of the jackson library?
> >
> > The jackson version of nifi 1.23.2 is 2.15.2, and the jackson version of
> > nifi 1.27.0 is 2.17.1, so I thought it might be a problem.
> >
> > If so, we can change the jackson version and compile it, but we will need
> > to continuously change the nifi version in the future.  Could you please
> > give me some advice on this?
> >
> > thank you  have a good day
> >
>
>
> --
> *박정화*
> Data Engineer, Data Platform Team (뱅크)
> 010-8000-6713 | vesuv...@toss.im
> 서울시 강남구 테헤란로 131, 한국지식재산센터 13층 (06133)
> [image: Toss BI]
>

Re: [DISCUSS] End-of-life timing for NiFi 1

[Pierre Villard]

Hi all,

While I think we could set an EOL date a bit further in the future, it is
important to keep in mind what EOL means. It only means we won't be
providing security fixes / bug fixes through new releases. It does not
mean that NiFi 1.x is gone. If that is a big concern for some users when
running EOL software then we should remind those users that they've been
doing it for 2+ years already when using NiFi 1.x (taking Jetty as an
example here). And Joe is definitely right when saying that we have a
smaller and smaller group of people willing to spend an extensive amount of
time taking care of PR/reviews, of release candidates, testing/validating
RCs, etc, for the 1.x line.

I also agree that, even if many users are already using NiFi 2 in
production, many places have strict policies to not adopt a new major
release. I don't want to start a debate whether this is making sense or not
but we know those rules exist in many places :) And the fact that we had
milestone releases for one year is not going to be enough of an argument.

Given what we've seen in the past, we usually make a new release every 3
months or so. It's probably fair to assume a 2.1.0 release will happen
early next year. With that in mind, I tend to agree with Michael suggesting
an EOL date at the end of January (3 months from now). We could also say
that 1.28.1 will happen at this time and will be the last one in the
community.

Vendors have already announced support for NiFi 1.x for multiple additional
years so this approach follows what we see in other projects where extended
support is only provided through paid options with specific companies.

It is awesome to finally see 2.0 out and this decision will help drive
users to that new release, which is much better in so many ways...

Thanks,
Pierre


Le mar. 5 nov. 2024 à 10:36, Isha Lamboo  a
écrit :

> Hi all,
>
> I understand the reasons to declare an EOL quickly, given the external
> dependencies, but like Russell said before the short notice is going to
> cause trouble with our bigger corporate customers. It would have been nice
> to have the EOL date announced about a year ago, even if it had been a
> provisional one. The more you can delay it now, the less credibility I (and
> NiFi itself) lose :-\
>
> I've been pushing since the first announcement of NiFi 2.0 for our
> customers to prepare. The smaller NiFi instances are all prepared. But
> there are also big customers with hundreds of flows that depend on
> variables and XML templates, and as you can imagine this was never a
> priority for them without either a NiFi 2.0 GA to move to or an actual EOL
> date to get security officers upping the priority.
>
> Now we have a GA release finally, but corporate Q4 plans are set in stone
> and Q1 2025 plans are already filling up. Telling the customers'
> development teams to upend their plans and tell their business customers to
> forget deliveries because NiFi needs to be fixed ASAP is probably not going
> to fly and instead going to seriously dent NiFi's reputation and position.
> Unless we can automate the flow migration process it's going to be a
> year-long migration at least.
>
> That said, are there any tools or scripts to make the migration smoother?
> Configuring multiple levels of parameter contexts with inheritance is a
> labor-intensive process if we are to mirror the current setup with
> variables being inherited from main canvas, team PG, subject PG and flow
> PG, etc. Anything that could go through the process groups and configure
> this automatically would be greatly appreciated. I will look into that
> myself too, but anything helps really.
>
> Regards,
>
> Isha
>
> -Oorspronkelijk bericht-
> Van: Joe Witt 
> Verzonden: maandag 4 november 2024 23:44
> Aan: dev@nifi.apache.org
> Onderwerp: Re: [DISCUSS] End-of-life timing for NiFi 1
>
> The EOL discussion is not here because we have a new problem.  It is here
> because we finally have an answer.
>
> The inability to address reported vulnerabilities or fundamental end of
> life status for key underlying components in the 1.x line is a problem that
> was fully recognized three years ago.
>
> In that time we created a plan for what NiFi 2.0 would be and how we'd
> manage both maintaining the 1.x line while building to the 2.x GA.  In the
> past year we've conducted four milestone releases of NiFi 2.x and we've
> continued putting out feature, bug fix, and security improvement releases
> of 1.x.
>
> Feature bearing releases of 1.x are no longer appropriate as 2.x is here
> and GA and that was the plan all along.
>
> Bug fixes are still reasonable in spirit but you need people to submit the
> JIRAs, fix the JIRAs, peer review the changes, and to conduct releases and
> make votes.  That is in increasingly short supply as it has been quite the
> task splitting attention across two major lines and naturally developers
> and users will gravitate toward the go forward path.
>
> Vulnerability/Security related consideration

Re: [VOTE] Release Apache NiFi 2.0.0 (RC2)

[Pierre Villard]

+1 binding

Went through the usual steps.
Deployed multiple flows in a secured cluster.
Confirmed fixes for some of the JIRAs raised since RC1.
LGTM

Thanks David for taking care of the release!

Pierre

Le ven. 1 nov. 2024 à 21:36, Michael Moser  a écrit :

> +1 (binding)
>
> Stepped through release verification for signature, hashes, branch and
> commit ID.
>
> Built with an empty local repo using:
> Apache Maven 3.9.6 Java version: 21.0.5, vendor: Red Hat, Inc.
> OS name: Red Hat Enterprise Linux 8.9
>
> Verified a few simple flows.
> Verified connection to NiFi Registry.
> Verified a few other issues in the notes for this release.
>
> Thanks for putting together this release, David.  It's really great to see
> the number of improvements made in 2.0.0 over the past year!
>
> -- Mike
>
>
> On Fri, Nov 1, 2024 at 2:51 PM Márk Báthori 
> wrote:
>
> > +1 (non-binding)
> >
> > - went through the helper guide
> > - did a full build with contrib check
> > - verified signatures and hashes
> > - created and ran some simple flows
> >
> > Environment:
> > - macOS Version 14.3.1
> > - OpenJDK Runtime Environment Zulu21.28+85-CA (build 21+35)
> > - Apache Maven 3.9.8
> >
> > Thanks for RMing David!
> >
> > Regards,
> > Mark
> >
> > Ferenc Kis  ezt írta (időpont: 2024. nov. 1.,
> P,
> > 19:21):
> >
> > > +1 (non-binding)
> > >
> > > Went through the helper guide, local maven repo cleaned up, full clean
> > > build with contrib check, verified signatures and hashes
> > >
> > > Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)
> > > Maven home: /Users/fkis/.m2/wrapper/dists/apache-maven-3.9.9/3477a4f1
> > > Java version: 21, vendor: Azul Systems, Inc., runtime:
> > > /Users/fkis/.sdkman/candidates/java/21-zulu/zulu-21.jdk/Contents/Home
> > > Default locale: en_HU, platform encoding: UTF-8
> > > OS name: "mac os x", version: "14.6.1", arch: "aarch64", family: "mac"
> > >
> > > Validations performed:
> > > - Started NiFi, created a simple flow with ListenHTTP and Input Port.
> > > Validated ListenHTTP processor is able to receive data
> > > - Started MiNiFi Java:
> > >   * created a simple GenerateFlowFile -> InvokeHttp flow and
> > > GenerateFlowFile -> RemoteProcessGroup flow and pushed to MiNiFi via
> > > C2 protocol
> > >   * Validated that connectivity between NiFi and MiNiFi works via both
> > > InvokeHTTP and S2S
> > >
> > > Thanks for RMing David!
> > >
> > > Regards
> > > Ferenc
> > >
> > > On Fri, Nov 1, 2024 at 7:05 PM Matt Burgess 
> > wrote:
> > > >
> > > > +1 (binding)
> > > >
> > > > Ran through release helper, tested various flows and versioning with
> > the
> > > > Github Registry Client, ran into [1] but I'll try to knock that out
> > > shortly
> > > > (not a blocker as it's documentation). We should also improve the
> > > > documentation for the Registry clients in general, such as the
> > > requirement
> > > > for a repo, a personal access token (PAT) and at least one branch to
> be
> > > > created in Github before the Github Registry Client can be used.
> > > >
> > > > Thanks for RM'ing David, and to the whole community on all the hard
> > work
> > > > and awesome new features!
> > > >
> > > > [1] https://issues.apache.org/jira/browse/NIFI-13230
> > > >
> > > > On Tue, Oct 29, 2024 at 11:26 PM David Handermann <
> > > > exceptionfact...@apache.org> wrote:
> > > >
> > > > > Team,
> > > > >
> > > > > I am pleased to be calling this vote for the source release of
> Apache
> > > > > NiFi 2.0.0.
> > > > >
> > > > > Please review the following guide for how to verify a release
> > candidate
> > > > > build:
> > > > >
> > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> > > > >
> > > > > The source being voted on the and the convenience binaries are
> > > > > available on the Apache Distribution Repository:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-2.0.0
> > > > >
> > > > > The build artifacts are available on the Apache Nexus Repository:
> > > > >
> > > > >
> > https://repository.apache.org/content/repositories/orgapachenifi-1294/
> > > > >
> > > > > Git Tag: nifi-2.0.0-RC2
> > > > > Git Commit ID: 2f13b609bdb77cb985aa35e8b66b4f04274d7c59
> > > > > GitHub Commit Link:
> > > > >
> > > > >
> > >
> >
> https://github.com/apache/nifi/commit/2f13b609bdb77cb985aa35e8b66b4f04274d7c59
> > > > >
> > > > > Checksums of nifi-2.0.0-source-release.zip
> > > > >
> > > > > SHA512:
> > > > >
> > >
> >
> deff4c81842759ef14651874a1b3204f75d0287a578174505b409ab2869704ed77527a5e262dd030bb7a5d1531a8d55105b015444c5802f55caf80439fe11220
> > > > >
> > > > > Release artifacts are signed with the following key:
> > > > >
> > > > > https://people.apache.org/keys/committer/exceptionfactory.asc
> > > > >
> > > > > KEYS file is available on the Apache Distribution Repository:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > > >
> > > > > Issues resolved for this version: 358
> > > > >
> > > > >
> > > > >
> >

Re: NiFi Primary Node

[Pierre Villard]

Hi,

I believe you can do
cli.sh nifi get-nodes -ot json

HTH,
Pierre

Le ven. 1 nov. 2024 à 17:52, Ravi Majmudar  a
écrit :

> Hello
>
> Is there a way to find out primary node of the cluster via command line
> (cli)?
>
>
>
> *Ravi Majmudar*
>
> *Messaging & Data Management Platform Engineering*
>
>
>
>
>
>
> Nothing in this message is intended to constitute an electronic signature
> unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or
> entity to which it is addressed. It may contain confidential and/or
> privileged material. Any review, transmission, dissemination or other use,
> or taking of any action in reliance upon this message by persons or
> entities other than the intended recipient is prohibited and may be
> unlawful. If you received this message in error, please contact the sender
> and delete it from your computer.
>

Re: [VOTE] Release Apache NiFi 2.0.0 (RC1)

[Pierre Villard]

+1 binding

Went through the usual steps of deploying a secured cluster with NiFi
Registry and testing some flows.

Noticed a couple of things:
- The delete-param CLI command is no longer working [1]
- There are some lucene related warnings at provenance repo initialization
that might require some attention at some point

But nothing that could impact this release candidate.

There is also one outstanding PR [2] that might introduce a breaking change
depending on the approach we will take but I think we still have a bit of
margin around such changes as we receive feedback from the larger community
post 2.0 release for a potential 2.0.1 release.

Thanks for taking care of the release David, great to see 2.0 finally going
out!

[1] https://issues.apache.org/jira/browse/NIFI-13924
[2] https://github.com/apache/nifi/pull/9347

Thanks,
Pierre

Le jeu. 24 oct. 2024 à 13:35, Ferenc Kis  a écrit :

> +1 (non-binding)
>
> Went through the helper guide, local maven repo cleaned up, full clean
> build with contrib check, verified signatures and hashes
>
> Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)
> Maven home: /Users/fkis/.m2/wrapper/dists/apache-maven-3.9.9/3477a4f1
> Java version: 21, vendor: Azul Systems, Inc., runtime:
> /Users/fkis/.sdkman/candidates/java/21-zulu/zulu-21.jdk/Contents/Home
> Default locale: en_HU, platform encoding: UTF-8
> OS name: "mac os x", version: "14.6.1", arch: "aarch64", family: "mac"
>
> Validations performed:
> - Started NiFi, created a simple flow with ListenHTTP and Input Port.
> Validated ListenHTTP processor is able to receive data
> - Started MiNiFi Java:
>   * created a simple GenerateFlowFile -> InvokeHttp flow and
> GenerateFlowFile -> RemoteProcessGroup flow and pushed to MiNiFi via C2
> protocol
>   * Validated that connectivity between NiFi and MiNiFi works via both
> InvokeHTTP and S2S
> - NIFI-13883 [MiNiFI][C2] Fix MiNiFI recovery if exception happens on
> config update
> - NIFI-13846 [MiNiFi] Create default parameter context to enable running
> empty flows received via C2
> - NIFI-13812 [MiNiFi][C2] Add component type into Agent Manifest Hash
> calculation
> - NIFI-13746 [MiNiFi] Override controller services SSL Context Service with
> parent SSL Context Service. Parent SSL Context Service is added to root
> process group instead of management controller services
> - NIFI-13762 Expose processor metrics as a part of FlowInfo
> - NIFI-13614: populate failure cause in operation state
> - NIFI-13635 Expose processor bulletins as a part of FlowInfo
> - NIFI-13541 [MiNiFi] Validate flow during startup
> - NIFI-13568 MiNiFi Fix concurrent SAXParser issue in manifest parsing
> - NIFI-13436 Retain queue for non-modified connections during MiNiFi flow
> update
>
> Thanks for RMing David!
>
> Regards
> Ferenc
>
> On Wed, Oct 23, 2024 at 9:46 PM Nissim Shiman 
> wrote:
>
> >  +1 (non-binding)
> >
> > checksums are good
> >
> > Compiled with
> > Apache Maven 3.9.9  1.8.0_422 (openjdk)  RHEL 8
> >
> > Ran some basic flows  Upgraded nifi registry (with pre-existing versioned
> > flows) from 1.27 to 1.28 successfully.  Created new PGs from versioned
> > flows.  Pushed new versions of flows to registry.
> >
> > Thank you Ferenc and team for the upcoming release.
> >
> > Thanks,
> > Nissim Shiman
> >
> >
> > On Tuesday, October 22, 2024 at 10:45:12 AM UTC, Ferenc Kis <
> > briansolo1...@gmail.com> wrote:
> >
> >  Team,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.28.0.
> >
> > Please review the following guide for how to verify a release candidate
> > build:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> >
> > The source being voted on the and the convenience binaries are available
> on
> > the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.28.0
> >
> > The build artifacts are available on the Apache Nexus Repository:
> >
> > https://repository.apache.org/content/repositories/orgapachenifi-1289
> >
> > Git Tag: nifi-1.28.0-RC1
> > Git Commit ID: 8ecf23e77c8ca828a77f3b84554ed3347d8f7fa2
> > GitHub Commit Link:
> >
> >
> https://github.com/apache/nifi/commit/8ecf23e77c8ca828a77f3b84554ed3347d8f7fa2
> >
> > Checksums of nifi-1.28.0-source-release.zip
> >
> > SHA256: 96ddd83ee11f6dd0889ff2f4b4112487f021b2b3f0573d7c0eeff40672620e93
> > SHA512:
> >
> >
> 22f0051b5e4a41b913e36f1fa2fabe6871471b0a4a51f3673b2fcc453382cd5d6eb132f7a6686ecdc065c295a2cfaf8199a653b822837f73a23016b6ae4bd143
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/briansolo1985.asc
> >
> > KEYS file is available on the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > Issues resolved for this version: 63
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354883
> >
> > Release note highlights can be found on the pr

Re: hello. I would like to inquire about an error when upgrading the version.

[Pierre Villard]

Hi,

Is there any chance you can provide some crafted dummy data to reproduce
the issue on our side? If this is due to an upgrade in some dependencies in
NiFi, we might want to look into it.

Note that you can always download the NARs from previous versions of NiFi
and use those with a more recent version of NiFi. However, keep in mind
that you would need multiple NARs in your case (standard processors NAR +
NAR(s) for the readers/writers). In the end, you'd use NiFi 1.27.0 but
you'd have the PartitionRecord processor and readers/writers from NiFi
1.23.2.

Hope this helps,
Pierre


Le mar. 22 oct. 2024 à 15:36, 박정화 (Bank)  a écrit :

> hello.
>
> I am using nifi well.
>
> I recently upgraded the version from 1.23.2 to 1.27.0 and encountered the
> following error in the PartitionRecord process.
>
>
>
> com.fasterxml.jackson.core.JsonGenerationException: Incomplete surrogate
> pair: first char 0xD83D, second 0x002A
>
>
>
> It seems to be a serrogate pair issue, but it seems to have been caused by
> masking some characters on our side.
>
> Nevertheless, I wish it were possible to process json like in previous
> versions or like python.  Is there any way?
>
> Could it be that the error occurring differently from the previous version
> is related to the update of the jackson library?
>
> The jackson version of nifi 1.23.2 is 2.15.2, and the jackson version of
> nifi 1.27.0 is 2.17.1, so I thought it might be a problem.
>
> If so, we can change the jackson version and compile it, but we will need
> to continuously change the nifi version in the future.  Could you please
> give me some advice on this?
>
> thank you  have a good day
>

Re: [DISCUSS] Preparing Release of Apache NiFi 2.0.0

[Pierre Villard]

+1

Still a few things that would be "nice to have" for parity with 1.x but
definitely time to get the first release out!

Thanks for taking care of this David!

Pierre

Le mer. 16 oct. 2024 à 19:36, Matt Gilman  a
écrit :

> +1
>
> Thanks for RMing David!!
>
> On Wed, Oct 16, 2024 at 12:46 PM Lucas Ottersbach <
> lucas.ottersb...@gmail.com> wrote:
>
> > +1
> >
> > Thanks David. And to everyone who contributed. Looking forward to the
> > release.
> >
> > David Handermann  schrieb am Mi., 16. Okt.
> > 2024, 17:59:
> >
> > > Team,
> > >
> > > It has been almost one year since we released Milestone 1 of NiFi
> > > 2.0.0. Over the four milestone releases, we have resolved almost 2000
> > > Jira issues, added numerous capabilities, and removed large amounts of
> > > technical debt. Although there is always more that could be done, we
> > > have reached the point where we should release a stable 2.0.0 version.
> > >
> > > The current release version [1] has almost 300 Jira issues resolved.
> > > With the release of NiFi API 2.0.0 and the NiFi NAR Maven Plugin
> > > 2.1.0, we are in a solid position to go forward with a 2.0.0 version.
> > > Although there are bound to be additional areas we want to improve, we
> > > should consider those after a general availability release.
> > >
> > > With that background, I would be glad to handle Release Manager
> > > responsibilities for this version. Unless there are any critical
> > > blockers, I plan to start the release candidate process before the end
> > > of this week. That should allow time for a few more items to be
> > > completed.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > [1]
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354889
> > >
> >
>

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 2.1.0 (RC1)

[Pierre Villard]

+1 binding

Confirmed the proper behavior when building NiFi while referencing nifi-api
2.0.0

Thanks,
Pierre

Le mar. 24 sept. 2024 à 12:06, Ferenc Kis  a
écrit :

> +1 (non-binding)
>
> * ran through the release verification guide
> * verified signatures and hashes
> * successfully built built nifi-nar-mavan-plugin and installed the jar into
> local maven repository
> * built NiFi from the latest main using the nifi-nar-mavan-plugin 2.1.0
> * created simple flow, all looked good
>
> Apache Maven 3.9.6 (bc0240f3c744dd6b6ec2920b3cd08dcc295161ae)
> Maven home: /Users/fkis/.sdkman/candidates/maven/current
> Java version: 21.0.3, vendor: Azul Systems, Inc., runtime:
> /Users/fkis/.sdkman/candidates/java/21.0.3-zulu/zulu-21.jdk/Contents/Home
> Default locale: en_HU, platform encoding: UTF-8
> OS name: "mac os x", version: "14.6.1", arch: "aarch64", family: "mac"
>
> Thanks for RMing David!
>
> Regards
> Ferenc Kis
>
>
> On Mon, Sep 23, 2024 at 11:10 PM Matt Burgess 
> wrote:
>
> > +1 (binding) Verified successful build of the plugin and NiFi using main
> > (with the new version).
> >
> > Thanks for RM'ing David!
> >
> > On Mon, Sep 23, 2024 at 3:03 PM David Handermann <
> > exceptionfact...@apache.org> wrote:
> >
> > > Team,
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > > NiFi NAR Maven Plugin 2.1.0.
> > >
> > > Please review the following guide for how to verify a release candidate
> > > build:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification+for+NiFi+NAR+Maven+Plugin
> > >
> > > The source being voted on the and the convenience binaries are
> > > available on the Apache Distribution Repository:
> > >
> > >
> https://dist.apache.org/repos/dist/dev/nifi/nifi-nar-maven-plugin-2.1.0
> > >
> > > The build artifacts are available on the Apache Nexus Repository:
> > >
> > > https://repository.apache.org/content/repositories/orgapachenifi-1283/
> > >
> > > Git Tag: nifi-maven-2.1.0-RC1
> > > Git Commit ID: 33591a2b3cc2134218f6a56ed3fe4a5eb2e3fdad
> > > GitHub Commit Link:
> > >
> > >
> >
> https://github.com/apache/nifi-maven/commit/33591a2b3cc2134218f6a56ed3fe4a5eb2e3fdad
> > >
> > > Checksums of nifi-nar-maven-plugin-2.1.0-source-release.zip
> > >
> > > SHA512:
> > >
> >
> 098eeecc7e9217d5676320688c405eae26db502eb86098b2aafb7a02f5c9651da31558affec9804efbb387835d625d899a588b6f38ca059c1a82769b05e0e785
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/exceptionfactory.asc
> > >
> > > KEYS file is available on the Apache Distribution Repository:
> > >
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > Issues resolved for this version: 1
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12355125
> > >
> > > Release note highlights can be found on the project wiki:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiNARMavenPluginVersion2.1.0
> > >
> > > The vote will be open for 48 hours.
> > >
> > > Please download the release candidate and evaluate the necessary items
> > > including checking hashes, signatures, build from source, and test.
> > > Then please vote:
> > >
> > > [] +1 Release this package as nifi-nar-maven-plugin-2.1.0
> > > [] +0 no opinion
> > > [] -1 Do not release this package because...
> > >
> >
>

Re: Unable to run podium nifi registry locally

[Pierre Villard]

Hi Shalini,

The link you're referencing is internal to Goldman Sachs and it sounds like
this is about a custom build of the NiFi Registry component. I don't think
the Apache NiFi community can help you with this.

Thanks,
Pierre

Le lun. 23 sept. 2024 à 14:05, shalini.khand...@ln.email.gs.com
 a écrit :

> Hi,
>
> I need to run podium nifi registry locally on my system.
> I am following this document
> https://gitlab.aws.site.gs.com/secdiv-engg-management/ce/podium-nifi/-/tree/feature-ce31597-2/podium-nifi-registry/podium-nifi-registry-bin.
> But unable to find files mentioned below.
>
> •  [linux/osx] execute bin/podium-nifi-registry.sh start
> •  [windows] execute bin/run-podium-nifi-registry.bat
>
> Please let me know where I can find these files to run it.
>
> Thanks,
> Shalini
>
>
>
> 
>
> © Copyright 2024 Goldman Sachs. All rights reserved. See
> http://www.gs.com/disclaimer/email-salesandtrading.html for risk
> disclosure, order handling practices, conflicts of interest and other terms
> and conditions relating to this e-mail and your reliance on it, and
> http://www.gs.com/disclaimer/ipo/ for recent prospectuses for initial
> public offerings to which this message may relate. See
> http://www.gs.com/swaps-related-disclosures for important disclosures
> relating to swap transactions, and
> http://www.goldmansachs.com/terms-of-dealing for general terms of
> dealing. For information on the nature and risks of investments for MiFID
> Professional Clients, see
> http://www.goldmansachs.com/disclosures/mifid/info-on-nature-risks-of-investments.pdf
> See http://www.goldmansachs.com/disclosures/mifid<
> http://www.goldmansachs.com/disclosures/mifid/> for important disclosures
> and GS policies in relation to MiFID II and MiFIR. This e-mail may contain
> confidential or privileged information. If you are not the intended
> recipient, please advise us immediately and delete it. See
> http://www.gs.com/disclaimer/email/ on confidentiality and the risks of
> electronic communication. If you cannot access these links, please notify
> us by reply message and we will send the contents to you. This material is
> a solicitation of derivatives business generally, only for the purposes of,
> and to the extent it would otherwise be subject to, CFTC Regulations 1.71
> and 23.605. Goldman Sachs International ("GSI") is authorised by the
> Prudential Regulation Authority and regulated by the Financial Conduct
> Authority and Prudential Regulation Authority.
>
> 
>
> Your Personal Data: We may collect and process information about you that
> may be subject to data protection laws. For more information about how we
> use and disclose your personal data, how we protect your information, our
> legal basis to use your information, your rights and who you can contact,
> please refer to: www.gs.com/privacy-notices<
> http://www.gs.com/privacy-notices>
>

Re: Unable to Install Apache NiFi

[Pierre Villard]

Hi Srinivas,

Your image didn't go through. Can you explain what the error is and
copy/paste the relevant parts of the error / stacktrace?

Thanks,
Pierre

Le lun. 23 sept. 2024 à 12:56, srinivasareddy 48 <
srinivasareddy...@gmail.com> a écrit :

> Hi Team,
>
> Greetings of the day!
>
> I am trying to install apache NiFi on my machine, but I'm facing the below
> error.
> [image: image.png]
> I request you to please help on this issue.
>
> point of contact: Srinivas (8297683841)
>
>
> Thanks,
> Srinivas
>

Re: [VOTE] Release Apache NiFi API 2.0.0 (RC1)

[Pierre Villard]

+1 binding

Everything looks good. Exciting to see the progress and NiFi 2 getting
close!

Le ven. 20 sept. 2024, 22:21, Dan S  a écrit :

> +1 (non-binding)
> I ran through all the steps in the Release Candidate Verification for NiFi
> API document.
> In addition I checked the README, NOTICE, and LICENSE files. I did note a
> slight inconsistency though. In the README for the Maven minimum version it
> has '3.9' and not the value in the pom.xml which is 3.9.9 (In the Apache
> NIFI README it has the minor version also 3.9.6).
>
> I ran with
> Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937)
> Maven home: ~/.m2/wrapper/dists/apache-maven-3.9.9/3477a4f1
> Java version: 21.0.3, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-21-openjdk-21.0.3.0.9-1.el8.x86_64
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.18.0-513.24.1.el8_9.x86_64", arch: "amd64",
> family: "unix"
>
> On Fri, Sep 20, 2024 at 3:46 PM Matt Burgess  wrote:
>
> > +1 (binding)
> >
> > Ran through release helper, verified successful build
> >
> > Apache Maven 3.9.9
> > ]Java version: 21.0.4, vendor: Eclipse Adoptium, runtime:
> > /Users/mburgess/.sdkman/candidates/java/21.0.4-tem
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "mac os x", version: "14.6.1", arch: "aarch64", family: "mac"
> >
> > Thanks for RM'ing David!
> >
> > On Thu, Sep 19, 2024 at 2:43 PM David Handermann <
> > exceptionfact...@apache.org> wrote:
> >
> > > Team,
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > > NiFi API 2.0.0.
> > >
> > > This is the first release of nifi-api as a separate module, following
> > > the recent vote in favor:
> > >
> > > https://lists.apache.org/thread/p6d7qsyto8x8xqzr6scp3fp33lrbb82c
> > >
> > > Please review the following guide for how to verify a release candidate
> > > build:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification+for+NiFi+API
> > >
> > > The source being voted on the and the convenience binaries are
> > > available on the Apache Distribution Repository:
> > >
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-api-2.0.0
> > >
> > > The build artifacts are available on the Apache Nexus Repository:
> > >
> > > https://repository.apache.org/content/repositories/orgapachenifi-1282/
> > >
> > > Git Tag: nifi-api-2.0.0-RC1
> > > Git Commit ID: 0a171b92813959cc4d84727b7b5c74c763fd4292
> > > GitHub Commit Link:
> > >
> > >
> >
> https://github.com/apache/nifi-api/commit/0a171b92813959cc4d84727b7b5c74c763fd4292
> > >
> > > Checksums of nifi-api-2.0.0-source-release.zip
> > >
> > > SHA512:
> > >
> >
> fe9790322797f80283c22b803ff8e8dbe17385e2e81349299a6b8d05be820a746d06dd75ea1ae26c9144f5e5f7c5ff349cc9681e5f4ab283ea91001411d74cbf
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/exceptionfactory.asc
> > >
> > > KEYS file is available on the Apache Distribution Repository:
> > >
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > Issues resolved for this version: 1
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12355111
> > >
> > > Release note highlights can be found on the project wiki:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiAPIVersion2.0.0
> > >
> > > The vote will be open for 72 hours.
> > >
> > > Please download the release candidate and evaluate the necessary items
> > > including checking hashes, signatures, build from source, and test.
> > > Then please vote:
> > >
> > > [ ] +1 Release this package as nifi-api-2.0.0
> > > [ ] +0 no opinion
> > > [ ] -1 Do not release this package because...
> > >
> >
>

Re: [DISCUSS] Removal of MissingComponentsCheck from Flow Synchronization

[Pierre Villard]

Hi Bryan,

I'm a +1 on the proposal. Seems to be a straightforward change and a step
in the right direction to better support rolling upgrades.

Thanks for putting this together,
Pierre

Le mar. 27 août 2024, 17:41, Bryan Bende  a écrit :

> All,
>
> I'd like to propose a change to the flow synchronization behavior and
> since we are still defining the new NIP process, I created a proposal
> under the existing Feature Proposal page:
>
>
> https://cwiki.apache.org/confluence/display/NIFI/Remove+MissingComponentsCheck+from+Flow+Synchronization
>
> This is more of a change in behavior, rather than a new feature.
> Please provide any feedback or concerns. If there are no objections, I
> can proceed with creating a JIRA and PR, the code changes should be
> very narrow.
>
> Thanks,
>
> Bryan
>

Re: [DISCUSS] 1.x Maintenance

[Pierre Villard]

I'm not discussing the motivations behind the removal of what has been
removed this week. It makes sense. But we can't expect to have users
quickly move to 2.x when the amount of things users will have to take care
of when moving to 2.x keeps growing and is very significant.

For the extensions, I'm a bit surprised by the removal of some extensions
such as Kudu, that we know is used a lot by NiFi users, and has been
actively maintained over the last few years. The one thing that concerns me
the most is the removal of the Kafka 2.6 components knowing that we don't
have feature parity, yet, with the new approach. I understand that we have
to do the cut at some point and there is no easy solution but again it
means that we should likely actively maintain 1.x for a longer period of
time.

On the framework side, I'm a bit concerned by the removal of repository
encryption which is also something many users are using. Not saying this is
perfect and should not be removed but we can't just say "switch to OS level
encryption / cloud provider encryption". Again, this will take a lot of
time for users using all of those things to adjust.

We are putting more and more expectations on users to move to 2.x when they
have been using NiFi for many many years and have systems running smoothly.
I'm not saying we should not clean things up, but by taking this approach,
I think we owe our community an active maintenance of the 1.x branch as
many users will need time to upgrade.

Le dim. 7 juil. 2024 à 15:35, Joe Witt  a écrit :

> Pierre
>
> Which things that have been removed are problematic and which bits got
> removed too fast?  There is no intent to make migration harder but there is
> to make it more sustainable.
>
> The focus is getting things with less known usage but highest tech debt
> removed. These are things with problematic socket code, limited security
> features, capabilities we made new versions for and ultimately which have
> minimal maintenance.
>
> If there is something controversial getting removed lets discuss it.  I
> dont think keeping any of this in the main codebase is workable but it can
> prompt having an extensions repository for lesser maintained components.
> This would help communicate these components have limited maintenance but
> may still be useful.
>
> If the bar to clean things up is higher than the bar to add new we have a
> known quality problem.  We either have a similar bar or we need other
> solutions.
>
> If there is a desire to see some (all?) decomissioned components kept and
> placed in an extensions repo to ease migration or just end user needs is
> there anybody available and interested to do that work?
>
> Thanks
>
> On Sun, Jul 7, 2024 at 5:42 AM Pierre Villard  >
> wrote:
>
> > Given all the things that are being removed in 2.x without even having a
> > single discussion about it (PR opened and merged within 48 hours), we
> > should not expect users (especially very large ones) to move from 1.x to
> > 2.x any time soon. Things that have been removed over the last few days
> are
> > going to be extremely impactful and will require users to do a
> significant
> > amount of work / re-architecturing to move to 2.x. It's a bit sad to see
> so
> > many things being done without any discussion in the community.
> >
> > Le ven. 5 juil. 2024 à 20:11, Michael Moser  a
> écrit :
> >
> > > My opinion is that the NiFi 1.x line is already in maintenance and
> > doesn't
> > > need new features or APIs, but bug fixes are still important to get in,
> > and
> > > security fixes are still critical.
> > >
> > > That said, this is an open source community-driven project, so if you
> can
> > > find contributors and committers that want to put in extra work to
> > support
> > > 1.x, then why not?  I'm sure many of us have seen something new in NiFi
> > and
> > > thought "I wouldn't have done that".  But others in the community
> thought
> > > it was a good idea, so why not?
> > >
> > > I don't really agree with the argument that putting too much work into
> > 1.x
> > > will delay people's transition to 2.0.  People will transition to 2.0
> on
> > > their own schedule, with myriad reasons for that schedule.  As a
> > community,
> > > we can simply make recommendations and provide solutions that help
> with a
> > > preferred direction.
> > >
> > > Kind regards,
> > > -- Mike
> > >
> > >
> > > On Wed, Jul 3, 2024 at 1:27 PM Joe Witt  wrote:
> > >
> > > > Judith,
> > > >
> > > > 

[RESULT][VOTE] Release Apache NiFi 1.27.0 (RC2)

[Pierre Villard]

Apache NiFi Community,

I am pleased to announce that the 1.27.0 release of Apache NiFi passes:

7 +1 (binding) votes
4 +1 (non-binding) votes
0 0 votes
0 -1 votes

Thanks to all who helped make this release possible!

The release artifacts will be published in the next 24 hours.

Here is the vote thread:
https://lists.apache.org/thread/f2vs42o32h8065xxbloc6fooy2j3s8mj

Thanks,
Pierre

Re: [VOTE] Release Apache NiFi 1.27.0 (RC2)

[Pierre Villard]

+1 binding - closing the vote, thanks all!

Le sam. 6 juil. 2024 à 13:58, Peter Turcsanyi  a
écrit :

> +1 (binding)
>
> Verified signatures and hashes.
>
> Built NiFi on Ubuntu 22.04 with Java 8 (Zulu 1.8.0_412-b08) and Java 11
> (Zulu 11.0.23+9-LTS), Maven 3.9.8 with empty local repo.
>
> Started up NiFi in secure / unsecure mode on Java 8 / 11.
>
> Ran flows for validating:
>  - NIFI-13181 Azure Blob and ADLS processors throw NoSuchMethodError when
> Service Principal is used
>  - NIFI-12231 Add completion strategy to FetchSmb
>
> Thanks for RMing Pierre!
>
> Regards,
> Peter Turcsanyi
>
> On Fri, Jul 5, 2024 at 10:00 PM Csaba Bejan  wrote:
>
> > +1 (binding)
> >
> >- Went through the helper guide and did a clean build
> >- Verified signatures and hashes
> >- Built on OSX 14.2.1
> >- Java version: 11.0.20, OpenJDK 64-Bit Server VM Zulu11.66+15-CA
> (build
> >11.0.20+8-LTS, mixed mode)
> >- Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
> >- Started NiFi and created a simple flow
> >- Started MiNiFi and verified integration with C2 Server. Played
> around
> >with the C2 protocol (Operations - Update, Transfer, Describe...)
> >
> > Thanks for RMing, Pierre!
> >
> > Regards,
> >
> > Csaba
> >
> > On Fri, Jul 5, 2024 at 2:30 PM Matt Burgess 
> wrote:
> >
> > > +1 (binding) ran through the release helper and tested various flows
> > > including NIFI-13422.
> > >
> > > Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256)
> > > Java version: 1.8.0_372, vendor: Azul Systems, Inc., runtime:
> > >
> > >
> >
> /Users/mburgess/.sdkman/candidates/java/8.0.372-zulu/zulu-8.jdk/Contents/Home/jre
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "mac os x", version: "14.5", arch: "aarch64", family: "mac"
> > >
> > > Thanks for RM'ing Pierre!
> > >
> > > On Wed, Jul 3, 2024 at 9:01 AM Pierre Villard <
> > pierre.villard...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Team,
> > > >
> > > > I am pleased to be calling this vote for the source release of Apache
> > > NiFi
> > > > 1.27.0.
> > > >
> > > > Please review the following guide for how to verify a release
> candidate
> > > > build:
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> > > >
> > > > The source being voted on the and the convenience binaries are
> > available
> > > on
> > > > the Apache Distribution Repository:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.27.0
> > > >
> > > > The build artifacts are available on the Apache Nexus Repository:
> > > >
> > > >
> https://repository.apache.org/content/repositories/orgapachenifi-1276
> > > >
> > > > Git Tag: nifi-1.27.0-RC2
> > > > Git Commit ID: e0c4461d90bd4f6e5f2b81765bcff5cd97ed3e18
> > > > GitHub Commit Link:
> > > >
> > > >
> > >
> >
> https://github.com/apache/nifi/commit/e0c4461d90bd4f6e5f2b81765bcff5cd97ed3e18
> > > >
> > > > Checksums of nifi-1.27.0-source-release.zip
> > > >
> > > > SHA256:
> > 0ac01d54eeceb4a4f4863880bf67dfa71e6a585036c5caf0546c592bf55ced48
> > > > SHA512:
> > > >
> > > >
> > >
> >
> 675c7750752bf3092061c9eaac39a975955e9bf881e6bee3124c5842738d8c8626b6a551f33ef7a678018bd83e0323c1f0f0d79101255494d8ca91be7fc750f5
> > > >
> > > > Release artifacts are signed with the following key:
> > > >
> > > > https://people.apache.org/keys/committer/pvillard.asc
> > > >
> > > > KEYS file is available on the Apache Distribution Repository:
> > > >
> > > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > >
> > > > Issues resolved for this version: 37
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354832
> > > >
> > > > Release note highlights can be found on the project wiki:
> > > >
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.27.0
> > > >
> > > > The vote will be open for 72 hours.
> > > >
> > > > Please download the release candidate and evaluate the necessary
> items
> > > > including checking hashes, signatures, build from source, and test.
> > Then
> > > > please vote:
> > > >
> > > > [] +1 Release this package as nifi-1.27.0
> > > > [] +0 no opinion
> > > > [] -1 Do not release this package because...
> > > >
> > >
> >
>

Re: [DISCUSS] 1.x Maintenance

[Pierre Villard]

Given all the things that are being removed in 2.x without even having a
single discussion about it (PR opened and merged within 48 hours), we
should not expect users (especially very large ones) to move from 1.x to
2.x any time soon. Things that have been removed over the last few days are
going to be extremely impactful and will require users to do a significant
amount of work / re-architecturing to move to 2.x. It's a bit sad to see so
many things being done without any discussion in the community.

Le ven. 5 juil. 2024 à 20:11, Michael Moser  a écrit :

> My opinion is that the NiFi 1.x line is already in maintenance and doesn't
> need new features or APIs, but bug fixes are still important to get in, and
> security fixes are still critical.
>
> That said, this is an open source community-driven project, so if you can
> find contributors and committers that want to put in extra work to support
> 1.x, then why not?  I'm sure many of us have seen something new in NiFi and
> thought "I wouldn't have done that".  But others in the community thought
> it was a good idea, so why not?
>
> I don't really agree with the argument that putting too much work into 1.x
> will delay people's transition to 2.0.  People will transition to 2.0 on
> their own schedule, with myriad reasons for that schedule.  As a community,
> we can simply make recommendations and provide solutions that help with a
> preferred direction.
>
> Kind regards,
> -- Mike
>
>
> On Wed, Jul 3, 2024 at 1:27 PM Joe Witt  wrote:
>
> > Judith,
> >
> > The Apache NiFi 2.x line is built on and requires Java 21 and will not
> > support any versions prior.
> >
> > NiFi 1.x line is built on Java 8 and requires either Java 8, 11, or 17.
> >
> > The dates you mention which extend to 2030/etc.. for a particular
> > distributor of JVMs is a function of what sort of support that commercial
> > vendor is making available to customers and it is different from what
> that
> > means for the public generally.
> >
> > Notably though it is also a very different concern from critical
> > dependencies we have and how they evolve. For example we rely on things
> > like Spring and Jetty and so on.  And these all choose to adopt language
> > features or discontinue key lines which tie to certain Java versions,
> > etc..  We can't control whether they choose to move to new major lines
> when
> > they fix vulnerabilities or whether they backport, etc..  We held the
> line
> > on Java 8 support far longer than I imagined we could but the current
> > confluence of events/scenarios means we made the jump all the way to Java
> > 21 (an LTS) release with the hopes that we can set a good clock for
> people
> > to work with for some time.
> >
> > The Java 1.x line will continue to be available for users to download and
> > will still see general maintenance to the extent there are contributors
> > available to do that.  You also have access to vendor supported paths to
> > consider there which is similar to the stated 'Extended Support
> > Availability' of Java.  But they too will have to navigate the complexity
> > of the reported vulnerabilities.
> >
> > My experience in the enterprise customer space is that tolerance for
> > components which contain reported vulnerabilities is far far lower than
> > ever before.  This in turn means the focus shifts to giving people strong
> > upgrade paths to consider.
> >
> > Thanks
> > Joe
> >
> > On Wed, Jul 3, 2024 at 10:10 AM Maucieri, Judith 
> wrote:
> >
> > > If I may:  One large obstacle to "our" shift to v2.x is the absence of
> > > Java 8 support (unless I overlooked updates to the plan stated in
> > November
> > > 2022 during release of version 1.19.0, Java 8 only remains in NiFi 1.x
> > > releases, which you hinted at remaining accurate).
> > >
> > > I make this statement in support of multiple scenarios where we employ
> > > Apache NiFi.  CVEs that are not addressed in v1.x would be a major
> > > concern.  In our case, per requirements levied upon us, we must
> maintain
> > > compatibility with all active Long Term Support (LTS) versions of Java.
> > I
> > > feel this is reinforced by LTS of RHEL 7, which just began and extends
> > into
> > > 2028 and employs (by default) Java 8.
> > >
> > > Note the LTS dates currently indicated (reference
> > > https://en.wikipedia.org/wiki/Java_version_history):
> > > - Java 8 December 2030
> > > - Java 11 January 2032
> > > - Java 17 September 2029
> > > - Java 21 September 2031
> > >
> > > Thank you,
> > > JM
> > >
> > > -Original Message-
> > > From: Matt Burgess 
> > > Sent: Tuesday, July 2, 2024 4:05 PM
> > > To: dev@nifi.apache.org
> > > Subject: [DISCUSS] 1.x Maintenance
> > >
> > > !---|
> > >   This Message Is From an External Sender
> > >   Please use caution when clicking links or opening
> > >   attachments.
> > > |---!
> > >
> > > There have been 

[VOTE] Release Apache NiFi 1.27.0 (RC2)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.27.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.27.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1276

Git Tag: nifi-1.27.0-RC2
Git Commit ID: e0c4461d90bd4f6e5f2b81765bcff5cd97ed3e18
GitHub Commit Link:
https://github.com/apache/nifi/commit/e0c4461d90bd4f6e5f2b81765bcff5cd97ed3e18

Checksums of nifi-1.27.0-source-release.zip

SHA256: 0ac01d54eeceb4a4f4863880bf67dfa71e6a585036c5caf0546c592bf55ced48
SHA512:
675c7750752bf3092061c9eaac39a975955e9bf881e6bee3124c5842738d8c8626b6a551f33ef7a678018bd83e0323c1f0f0d79101255494d8ca91be7fc750f5

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 37

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354832

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.27.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.27.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [VOTE] Release Apache NiFi 2.0.0-M4 (RC1)

[Pierre Villard]

+1, binding

Went through the usual steps, deployed a 3-nodes secured cluster in GCP
with NiFi Registry and confirmed everything is working as expected.

Thanks for taking care of this release David!

Pierre

Le ven. 28 juin 2024 à 22:22, Dan S  a écrit :

> +1 (non-binding)
> Ran through Release Candidate Verification
> Verified signatures and hashes
> Built with:
>
> Maven home: /opt/apache-maven-3.9.6
> Java version: 21.0.3, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-21-openjdk-21.0.3.0.9-1.el8.x86_64
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "4.18.0-513.24.1.el8_9.x86_64", arch: "amd64",
> family: "unix"
>
> Exercised new feature NIFI-13304 SplitExcel
> Discovered bug NIFI-13466
>  and
> created task to research NIFI-13467
> 
>
> Thanks for RM'ing David and all your hard work to get to this point!
>
> On Fri, Jun 28, 2024 at 1:16 PM Matt Burgess  wrote:
>
> > +1 (binding) Ran through release helper and tried various flows, verified
> > versioning against a NiFi Registry (not the Git-backed one)
> >
> > Apache Maven 3.9.7 (8b094c9513efc1b9ce2d952b3b9c8eaedaf8cbf0)
> > Java version: 21, vendor: Oracle Corporation, runtime:
> > /Users/mburgess/.sdkman/candidates/java/21-oracle
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "mac os x", version: "14.5", arch: "aarch64", family: "mac"
> >
> > Thanks for RM'ing David!
> >
> > On Fri, Jun 28, 2024 at 9:40 AM David Handermann <
> > exceptionfact...@apache.org> wrote:
> >
> > > Team,
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > > NiFi 2.0.0-M4.
> > >
> > > Please review the following guide for how to verify a release candidate
> > > build:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> > >
> > > The source being voted on the and the convenience binaries are
> > > available on the Apache Distribution Repository:
> > >
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-2.0.0-M4
> > >
> > > The build artifacts are available on the Apache Nexus Repository:
> > >
> > > https://repository.apache.org/content/repositories/orgapachenifi-1275
> > >
> > > Git Tag: nifi-2.0.0-M4-RC1
> > > Git Commit ID: 19c5be01d463bc38ec0f5008549a2a42e589436d
> > > GitHub Commit Link:
> > >
> > >
> >
> https://github.com/apache/nifi/commit/19c5be01d463bc38ec0f5008549a2a42e589436d
> > >
> > > Checksums of nifi-2.0.0-M4-source-release.zip
> > >
> > > SHA256:
> d882f05cec09ee1bfafaa3d4cde8f8660512d09765b5c400471f3a6e014029a6
> > > SHA512:
> > >
> >
> d429cd67fb0b7d9737c59cb834106d7b6e25cbdb91e3ecc5290be865a1313cbebbc314c2e0e54228226f021c44f0a86c745a18c148247c632a739c871c5fa013
> > >
> > > Release artifacts are signed with the following key:
> > >
> > > https://people.apache.org/keys/committer/exceptionfactory.asc
> > >
> > > KEYS file is available on the Apache Distribution Repository:
> > >
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > Issues resolved for this version: 153
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354678
> > >
> > > Release note highlights can be found on the project wiki:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.0.0-M4
> > >
> > > The vote will be open for 72 hours.
> > >
> > > Please download the release candidate and evaluate the necessary items
> > > including checking hashes, signatures, build from source, and test.
> > > Then please vote:
> > >
> > > [] +1 Release this package as nifi-2.0.0-M4
> > > [] +0 no opinion
> > > [] -1 Do not release this package because...
> > >
> >
>

Re: [VOTE] Release Apache NiFi 1.27.0 (RC1)

[Pierre Villard]

-1, binding

I have the same error, thanks for flagging Nassim.
I'm cancelling the vote for the RC1 and will start a RC2 once we fix the
issue.

Le ven. 28 juin 2024 à 17:10, Nissim Shiman  a
écrit :

>  Team,
>
> I am running into difficulty starting nifi registry.
> It was compiled (and is being run) with java 1.8 (openjdk-1.8.0.412)
> on RHEL 8
>
> nifi itself compiles and runs fine.
>
>
> I was wondering is other people have run into anything similar.
>
>
> The error is (from nifi-registry-app.log):
> 2024-06-28 14:44:33,150 WARN [main] d.o.a.n.r.bootstrap.RunNiFiRegistry
> Support for Java 8 is deprecated. Java 11 is the minimum recommended
> versionorg.apache.nifi.deprecation.log.DeprecationException: Reference
> Class [org.apache.nifi.registry.bootstrap.RunNiFiRegistry] ClassLoader
> [sun.misc.Launcher$AppClassLoader@232204a1] at
> org.apache.nifi.deprecation.log.StandardDeprecationLogger.getExtendedArguments(StandardDeprecationLogger.java:63)
> at
> org.apache.nifi.deprecation.log.StandardDeprecationLogger.warn(StandardDeprecationLogger.java:54)
> at
> org.apache.nifi.registry.bootstrap.RunNiFiRegistry.start(RunNiFiRegistry.java:941)
> at
> org.apache.nifi.registry.bootstrap.RunNiFiRegistry.main(RunNiFiRegistry.java:206)2024-06-28
> 14:44:33,506 INFO [main] org.apache.nifi.registry.NiFiRegistry Launching
> NiFi Registry...2024-06-28 14:44:33,508 INFO [main]
> org.apache.nifi.registry.NiFiRegistry Read property protection key from
> /registry_1_27_0_RC1/nifi-registry-1.27.0/conf/bootstrap.conf2024-06-28
> 14:44:33,525 WARN [main] o.a.n.p.AbstractBootstrapPropertiesLoader
> No encryption key present in the bootstrap.conf file at
> /registry_1_27_0_RC1/nifi-registry-1.27.0/conf/bootstrap.conf
>
>
> and (nifi-registry-bootstrap.log)
>
> 2024-06-28 14:44:34,477 ERROR [NiFi logging handler]
> org.apache.nifi.registry.StdErr Exception in thread "main"
> java.lang.NoClassDefFoundError:
> org/apache/nifi/deprecation/log/DeprecationLoggerFactory2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> at
> org.apache.nifi.registry.properties.NiFiRegistryPropertiesLoader.(NiFiRegistryPropertiesLoader.java:38)2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> at java.lang.Class.forName0(Native Method)2024-06-28 14:44:34,477 ERROR
> [NiFi logging handler] org.apache.nifi.registry.StdErr  at
> java.lang.Class.forName(Class.java:348)2024-06-28 14:44:34,477 ERROR [NiFi
> logging handler] org.apache.nifi.registry.StdErr  at
> org.apache.nifi.registry.NiFiRegistry.initializeProperties(NiFiRegistry.java:189)2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> at
> org.apache.nifi.registry.NiFiRegistry.main(NiFiRegistry.java:153)2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> Caused by: java.lang.ClassNotFoundException:
> org.apache.nifi.deprecation.log.DeprecationLoggerFactory2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)2024-06-28
> 14:44:34,477 ERROR [NiFi logging handler] org.apache.nifi.registry.StdErr
> ... 5 more2024-06-28 14:44:35,174 INFO [main]
> o.a.n.registry.bootstrap.RunNiFiRegistry NiFi Registry never started. Will
> not restart NiFi Registry
>
>
> Thanks,
> Nissim Shiman
> On Thursday, June 27, 2024 at 09:01:30 PM UTC, Pierre Villard <
> pierre.villard...@gmail.com> wrote:
>
>  Team,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> 1.27.0.
>
> Please review the following guide for how to verify a release candidate
> build:
>
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
>
> The source being voted on the and the convenience binaries are available on
> the Apache Distribution Repository:
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.27.0
>
> The build artifacts are available on the Apache Nexus Repository:
>
> https://repository.apache.org/content/repositories/orgapachenifi-1272
>
> Git Tag: nifi-1.27.0-RC1
> Git Commit ID: bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143
> GitHub Commit Link:
>
> https://github.com/apache/nifi/commit/bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143
>
> Checksums of nifi-1.27.0-source-release.zip
>
> SHA256: 7df2933b366aff8e1dc3abe9bafb3e8891186ca

Re: [VOTE] Release Apache NiFi 1.27.0 (RC1)

[Pierre Villard]

Hi Matt,

I extended my key weeks ago but for some reason the ASF has not synced the
extended key. I'll try to get this fixed asap.
Using my fingerprint, you can find my extended key on well known key
servers.

I checked the checksums again and it seems to be ok on my side.

Sorry for the inconvenience,
Pierre

Le ven. 28 juin 2024 à 00:15, Matt Burgess  a écrit :

> I'm seeing SHA256 and SHA512 mismatches, also my automated RC script
> noticed your GPG key is expired.
>
> On Thu, Jun 27, 2024 at 5:11 PM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > Team,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.27.0.
> >
> > Please review the following guide for how to verify a release candidate
> > build:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> >
> > The source being voted on the and the convenience binaries are available
> on
> > the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.27.0
> >
> > The build artifacts are available on the Apache Nexus Repository:
> >
> > https://repository.apache.org/content/repositories/orgapachenifi-1272
> >
> > Git Tag: nifi-1.27.0-RC1
> > Git Commit ID: bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143
> > GitHub Commit Link:
> >
> >
> https://github.com/apache/nifi/commit/bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143
> >
> > Checksums of nifi-1.27.0-source-release.zip
> >
> > SHA256: 7df2933b366aff8e1dc3abe9bafb3e8891186ca937d34d646952d43db624d4e2
> > SHA512:
> >
> >
> 36c0f107b5a98388c2fb2c48d24f4df9980d9685a9b31d08c0a7a6a47b54de556ff368a8c622a5011dab4448274fc16f6e18c5d29368a81d003baa7c7382bd4d
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/pvillard.asc
> >
> > KEYS file is available on the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > Issues resolved for this version: 35
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354832
> >
> > Release note highlights can be found on the project wiki:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.27.0
> >
> > The vote will be open for 72 hours.
> >
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [] +1 Release this package as nifi-1.27.0
> > [] +0 no opinion
> > [] -1 Do not release this package because...
> >
>

[VOTE] Release Apache NiFi 1.27.0 (RC1)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.27.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.27.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1272

Git Tag: nifi-1.27.0-RC1
Git Commit ID: bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143
GitHub Commit Link:
https://github.com/apache/nifi/commit/bd86d1f07fe8d0f45b72e91e925ee85ffa7ed143

Checksums of nifi-1.27.0-source-release.zip

SHA256: 7df2933b366aff8e1dc3abe9bafb3e8891186ca937d34d646952d43db624d4e2
SHA512:
36c0f107b5a98388c2fb2c48d24f4df9980d9685a9b31d08c0a7a6a47b54de556ff368a8c622a5011dab4448274fc16f6e18c5d29368a81d003baa7c7382bd4d

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 35

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354832

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.27.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.27.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [DISCUSS] Release Planning for NiFi 2.0.0-M4 and 1.26.1

[Pierre Villard]

Team,

I'll get started with the release process. In light of the changes, I'll go
with a 1.27 release as it's not only containing bug fixes.

Thanks,
Pierre

Le mer. 26 juin 2024 à 00:01, David Handermann 
a écrit :

> Team,
>
> Thanks for the replies thus far. In a conversation with Shane, and
> reviewing the current status of the PR [1], it looks like there is a
> bit more work to do, plus review, to get that completed.
>
> Other than that, I am planning to review the Python API PR for
> NIFI-13427 [2] and then I think we are in a good position for an
> initial release candidate build.
>
> Regards,
> David Handermann
>
> [1] https://github.com/apache/nifi/pull/8974
> [2] https://github.com/apache/nifi/pull/9000
>
> On Mon, Jun 17, 2024 at 11:12 AM Shane Ardell 
> wrote:
> >
> > Hi team,
> >
> > Pierre is correct that I'm aiming to have a PR open today to expose the
> > flow analysis rules engine in the new UI. I'll be sure to address any
> > feedback left on my PR asap.
> >
> > Best,
> > Shane
> >
> > On Mon, Jun 17, 2024 at 12:03 PM Pierre Villard <
> pierre.villard...@gmail.com>
> > wrote:
> >
> > > Hi team,
> > >
> > > Happy to take care of the 1.26.1.
> > >
> > > I believe Shane will open a PR today to have the UI bits of the rules
> > > engine available in the new UI. We may want to wait for this, assuming
> this
> > > would be merged quickly, to get as much feedback as possible on the
> new UI.
> > >
> > > Very exciting to see all of this coming together! Thanks David for
> starting
> > > this discussion.
> > >
> > > Pierre
> > >
> > > Le lun. 17 juin 2024 à 17:39, David Handermann <
> > > exceptionfact...@apache.org>
> > > a écrit :
> > >
> > > > Team,
> > > >
> > > > With recent changes on the main branch to make the nifi-web-frontend
> > > > module the default version of the user interface, we are in a good
> > > > position for another milestone release of NiFi 2.0.0.
> > > >
> > > > As always, we also have a number of bug fixes and dependency upgrades
> > > > that are ready for release. It seems helpful to have a Milestone 4
> > > > release version, with a focus on the new user interface, as it will
> > > > provide an opportunity for additional feedback before a full general
> > > > availability version of NiFi 2.0.0. Thanks again to Matt Gilman, Rob
> > > > Fellows, and Scott Aslan for their incredible effort to rebuild the
> > > > user interface on modern frameworks and libraries!
> > > >
> > > > Although NiFi 2.0.0 remains the primary focus, there have been
> several
> > > > important bug fixes applied to the support branch [1]. I renamed the
> > > > 1.27.0 version to 1.26.1 since the number of changes and the types of
> > > > changes are narrowly focused. Pierre Villard mentioned that he is
> > > > ready to handle the Release Manager responsibilities for the 1.26.1
> > > > version, and I would be glad to handle the 2.0.0-M4 version.
> > > >
> > > > I am in the process of reviewing open pull requests, and I believe we
> > > > are close to ready for release candidates this week.
> > > >
> > > > Regards,
> > > > David Handermann
> > > >
> > > > [1] https://issues.apache.org/jira/projects/NIFI/versions/12354651
> > > >
> > >
>

Re: Migration

[Pierre Villard]

Hi Karl,

If using recent versions (I think 1.20+) of NiFi 1.x, you'd have
flow.xml.gz and flow.json.gz side by side. The JSON representation is what
remains in NiFi 2.x (any XML representation of the flows is going away).
You could also right click on a process group in the NiFi and download a
flow definition as a JSON file.

HTH,
Pierre

Le mar. 25 juin 2024 à 17:07, Schumacher, Karl (CDPQ)
 a écrit :

> Hi,
>
> My organization use Apache Nifi and we have many fows that are saved
> (Registry) in the file flow.xml.gz
> They are huge (millions of rows).
>
> We have installed the new Apache Nifi version on EC2 instances and another
> one for Registry, and we try to figure how to convert the flow.xml.gz to
> json format.
> The goal is to replace the file flow.xml.gz of the version 2 with the
> converted flow.xml.gz of the version 1.
>
> In the migration process, do a tool exist, or some instructions (steps),
> to achieve this migration.
>
> Thanks,
>
> __
>
> Karl Schumacher
> Developer
> Data
> CDPQ
> T   +1 514 847-4182
>
>
>
>
> 
>
> Avis de confidentialit? : Ce courriel et les pi?ces qui y sont jointes
> contiennent de l'information confidentielle et peuvent ?tre prot?g?s par le
> secret professionnel ou constituer de l'information privil?gi?e. Ils sont
> destin?s ? l'usage exclusif de la (des) personne(s) ? qui ils sont
> adress?s. Si vous n'?tes pas le destinataire vis? ou la personne charg?e de
> transmettre ce document ? son destinataire, vous ?tes avis? par la pr?sente
> que toute divulgation, reproduction, copie, distribution ou autre
> utilisation de cette information est strictement interdite. Si vous avez
> re?u ce courriel par erreur, veuillez en aviser imm?diatement l'exp?diteur
> par t?l?phone ainsi que d?truire et effacer l'information que vous avez
> re?ue de tout disque dur ou autre m?dia sur lequel elle peut ?tre
> enregistr?e et ne pas en conserver de copie. Merci de votre collaboration.
>
> 
>
> Notice of Confidentiality: This electronic mail message, including any
> attachments, is confidential and may be privileged and protected by
> professional secrecy. They are intended for the exclusive use of the
> addressee. If you are not the intended addressee or the person responsible
> for delivering this document to the intended addressee, you are hereby
> advised that any disclosure, reproduction, copy, distribution or other use
> of this information is strictly forbidden. If you have received this
> document by mistake, please immediately inform the sender by telephone,
> destroy and delete the information received from any hard disk or any media
> on which it may have been registered and do not keep any copy. Thank you
> for your cooperation.
>

Re: Nifi-2.0.M3 Install

[Pierre Villard]

Hi Venu,

The image didn't go through but I suspect that you're running into the SNI
issue. If that's the case, please share with us the steps you've taken to
configure nifi.properties in your deployment. This blog post [1] may also
be helpful.

[1]
https://medium.com/@chnzhoujun/how-to-resolve-sni-issue-when-upgrading-to-nifi-2-0-907e07d465c5

HTH,
Pierre

Le mar. 25 juin 2024 à 15:04, Gorantla, Venugopal EX1
 a écrit :

> Hi Team,
>
> I have installed NIfi-2.0.M3 in the ubuntu server.
> Nifi service started successfully and Nifi url also generated.
>
> But when try accessing URL getting error as below :
> URL : https://ec2-3-233-74-147.compute-1.amazonaws.com:9443/nifi
>
>
>
> Could you please help regarding this.
>
> Regards,
> Venu G
>
>
> 
> *IMPORTANT* - PLEASE READ: This electronic message, including its
> attachments, is CONFIDENTIAL and may contain PROPRIETARY or LEGALLY
> PRIVILEGED or PROTECTED information and is intended for the authorized
> recipient of the sender. If you are not the intended recipient, you are
> hereby notified that any use, disclosure, copying, or distribution of this
> message or any of the information included in it is unauthorized and
> strictly prohibited. If you have received this message in error, please
> immediately notify the sender by reply e-mail and permanently delete this
> message and its attachments, along with any copies thereof, from all
> locations received (e.g., computer, mobile device, etc.). To the extent
> permitted by law, we may monitor electronic communications for the purposes
> of ensuring compliance with our legal and regulatory obligations and
> internal policies. We may also collect email traffic headers for analyzing
> patterns of network traffic and managing client relationships. For further
> information see our privacy-policy
> . Thank you.
>
>

Re: Apache Nifi Compatibility with Oracle Linux 9

[Pierre Villard]

Hi,

There should not be any particular issue running NiFi with Oracle Linux as
NiFi is OS-agnostic and really just depends on the installed JDK. NiFi
1.9.2 is an extremely old version of NiFi and we would highly recommend
upgrading to a more recent version of NiFi.

HTH,
Pierre

Le mar. 25 juin 2024 à 15:03,  a écrit :

> Good Afternoon
>
>
>
> I am having trouble finding the information I am after online and was
> hoping you could assist.
>
>
>
> Could you please let me know what version(s) of Oracle Linux Nifi 1.9.2
> supports and whether it can support a mixture of different versions of OL?
>
>
>
> Thanks in advance!
>
>
>
>
>
> *Fatima Jarjue*
> SRE Associate
> Secure Development, Networks
>
>
> E: fatima.jar...@bt.com
>
> [image: BT Group]
>
> This email contains information from BT Group that might be privileged or
> confidential. And it's only meant for the person above. If that's not you,
> we're sorry - we must have sent it to you by mistake. Please email us to
> let us know, and don't copy or forward it to anyone else. Thanks.
>
> We monitor our email systems and may record all our emails.
>
> British Telecommunications plc
> R/O : 1 Braham Street, London, E1 8EE
> Registered in England: No 180
>
> British Telecommunications plc is authorised and regulated by Financial
> Conduct Authority for the provision of consumer credit
>
>
>
>
>

Re: Removal of pub/sub lite processors - GCP pub/sub lite end of life

[Pierre Villard]

Hey Zoltan,

The shutdown of Pub/Sub Lite is in a long time but I guess we can take the
opportunity of NiFi 2.0 to deprecate those processors in 1.x and remove the
processors in 2.x and if someone really needs the processors, they could
use the NARs that have been already released as part of the milestone
releases?

Or we just keep them, given this is not adding any significant overhead,
and we remove the processors as part of a minor 2.x release in two years.

I don't have a strong opinion. But whatever we decide we can definitely add
a deprecation notice anyway.

Thanks,
Pierre


Le mar. 18 juin 2024 à 13:52, Zoltán Kornél Török  a
écrit :

> Hi team,
> I recently read that GCP pub/sub lite will be removed in 2026 -
> https://cloud.google.com/pubsub/lite/docs/migrate-pubsub-lite-to-pubsub
>
>
> In the codebase there is two processors written for gcp lite
>
> https://github.com/apache/nifi/tree/main/nifi-extension-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/pubsub/lite
>
> Since pub/sub lite is deprecated, do we want to remove it from main and
> from nifi 2.0?
>

Re: [DISCUSS] Release Planning for NiFi 2.0.0-M4 and 1.26.1

[Pierre Villard]

Hi team,

Happy to take care of the 1.26.1.

I believe Shane will open a PR today to have the UI bits of the rules
engine available in the new UI. We may want to wait for this, assuming this
would be merged quickly, to get as much feedback as possible on the new UI.

Very exciting to see all of this coming together! Thanks David for starting
this discussion.

Pierre

Le lun. 17 juin 2024 à 17:39, David Handermann 
a écrit :

> Team,
>
> With recent changes on the main branch to make the nifi-web-frontend
> module the default version of the user interface, we are in a good
> position for another milestone release of NiFi 2.0.0.
>
> As always, we also have a number of bug fixes and dependency upgrades
> that are ready for release. It seems helpful to have a Milestone 4
> release version, with a focus on the new user interface, as it will
> provide an opportunity for additional feedback before a full general
> availability version of NiFi 2.0.0. Thanks again to Matt Gilman, Rob
> Fellows, and Scott Aslan for their incredible effort to rebuild the
> user interface on modern frameworks and libraries!
>
> Although NiFi 2.0.0 remains the primary focus, there have been several
> important bug fixes applied to the support branch [1]. I renamed the
> 1.27.0 version to 1.26.1 since the number of changes and the types of
> changes are narrowly focused. Pierre Villard mentioned that he is
> ready to handle the Release Manager responsibilities for the 1.26.1
> version, and I would be glad to handle the 2.0.0-M4 version.
>
> I am in the process of reviewing open pull requests, and I believe we
> are close to ready for release candidates this week.
>
> Regards,
> David Handermann
>
> [1] https://issues.apache.org/jira/projects/NIFI/versions/12354651
>

[RESULT][VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0

[Pierre Villard]

Apache NiFi Community,

I am pleased to announce that the 2.0.0 release of Apache NiFi NAR Maven
Plugin passes:

4 +1 (binding) votes
1 +1 (non-binding) votes
0 0 votes
0 -1 votes

Thanks to all who helped make this release possible!

The release artifacts will be published in the next 24 hours.

Here is the vote thread:
https://lists.apache.org/thread/kbrhjf72p3brbc17z3h16kh06zhnzx5r

Thanks,
Pierre

Re: [VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0

[Pierre Villard]

+1 binding

Le jeu. 23 mai 2024 à 19:40, Joe Witt  a écrit :

> +1 binding
>
> On Thu, May 23, 2024 at 10:33 AM Matt Burgess  wrote:
>
> > +1 binding, verified the commit hash, applied the patch to NiFi main and
> > verified the NAR Maven plugin works as expected.
> >
> > Thanks for RM'ing Pierre!
> >
> > On Wed, May 22, 2024 at 7:31 AM Pierre Villard <
> > pierre.villard...@gmail.com>
> > wrote:
> >
> > > Hello Apache NiFi Community,
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> > NiFi
> > > NAR Maven Plugin 2.0.0. This new release is intended to be used with
> NiFi
> > > 2+.
> > >
> > > The source being voted upon, including signatures, digests, etc. can be
> > > found at:
> > >
> https://dist.apache.org/repos/dist/dev/nifi/nifi-nar-maven-plugin-2.0.0/
> > >
> > > A helpful reminder on how the release candidate verification process
> > works:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+NAR+Maven+Plugin+release+candidate
> > >
> > > Please note that when trying this release, some changes are expected on
> > the
> > > NiFi side as proposed in the PR:
> > https://github.com/apache/nifi/pull/8860
> > >
> > > The Git tag is nifi-maven-2.0.0-RC1
> > > The Git commit ID is d0de49a34a1197988eaa007d28e6eb20ab6d701f
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi-maven.git;a=commit;h=d0de49a34a1197988eaa007d28e6eb20ab6d701f
> > >
> > > Checksums of nifi-nar-maven-plugin-2.0.0-source-release.zip:
> > > SHA256:
> ae889c843a82c4823fca0bf5fedf275dbb812c2b11e25ae26b0bd4ad7d26ccae
> > > SHA512:
> > >
> > >
> >
> c9afefb093d8b9568012d6697f302eb4a138f0a65a7045eaeb1bff551fb4bff23c805294700ca20984a3493eb61fe80a39d8b5b393afc85cf3f6ab1a28df831c
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/pvillard.asc
> > >
> > > KEYS file available here:
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > 5 issues were closed/resolved for this release:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354745
> > >
> > > Release note highlights can be found here:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiNARMavenPluginVersion2.0.0
> > >
> > > The vote will be open for 72 hours. Please download the release
> > > candidate and evaluate the necessary items including checking hashes,
> > > signatures, build from source, and test. Then please vote:
> > >
> > > [ ] +1 Release this package as nifi-nar-maven-plugin-2.0.0
> > > [ ] +0 no opinion
> > > [ ] -1 Do not release this package because …
> > >
> >
>

[VOTE] Release Apache NiFi NAR Maven Plugin 2.0.0

[Pierre Villard]

Hello Apache NiFi Community,

I am pleased to be calling this vote for the source release of Apache NiFi
NAR Maven Plugin 2.0.0. This new release is intended to be used with NiFi
2+.

The source being voted upon, including signatures, digests, etc. can be
found at:
https://dist.apache.org/repos/dist/dev/nifi/nifi-nar-maven-plugin-2.0.0/

A helpful reminder on how the release candidate verification process works:
https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+NAR+Maven+Plugin+release+candidate

Please note that when trying this release, some changes are expected on the
NiFi side as proposed in the PR: https://github.com/apache/nifi/pull/8860

The Git tag is nifi-maven-2.0.0-RC1
The Git commit ID is d0de49a34a1197988eaa007d28e6eb20ab6d701f
https://gitbox.apache.org/repos/asf?p=nifi-maven.git;a=commit;h=d0de49a34a1197988eaa007d28e6eb20ab6d701f

Checksums of nifi-nar-maven-plugin-2.0.0-source-release.zip:
SHA256: ae889c843a82c4823fca0bf5fedf275dbb812c2b11e25ae26b0bd4ad7d26ccae
SHA512:
c9afefb093d8b9568012d6697f302eb4a138f0a65a7045eaeb1bff551fb4bff23c805294700ca20984a3493eb61fe80a39d8b5b393afc85cf3f6ab1a28df831c

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pvillard.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

5 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354745

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiNARMavenPluginVersion2.0.0

The vote will be open for 72 hours. Please download the release
candidate and evaluate the necessary items including checking hashes,
signatures, build from source, and test. Then please vote:

[ ] +1 Release this package as nifi-nar-maven-plugin-2.0.0
[ ] +0 no opinion
[ ] -1 Do not release this package because …

Re: [DISCUSS] Release NiFi NAR Maven Plugin

[Pierre Villard]

My plan was to go with the latest for everything (including Java 21).
This way, NiFi NAR Maven plugin 2.0.0 would be used only for NiFi 2.x.
I actually think that the addition of the extension type "flow analysis
rules" would break things if used in the 1.x line (unless I also update the
enum in NiFi 1.x to include flow analysis rules as an extension type but
not sure that would be a great approach).

Le mar. 21 mai 2024 à 17:24, David Handermann 
a écrit :

> Pierre,
>
> Thanks for highlighting the differences for Flow Analysis Rules. As
> far as making this a 2.0.0 release version, do you anticipate
> including any breaking changes?
>
> The current version of the plugin is built with Java 8, so this could
> be updated to Java 21, which would be a breaking change, however, that
> should be handled before the release process.
>
> If there are no breaking changes, I recommend a version 1.6.0 release
> before a 2.0.0 release of the plugin.
>
> Regards,
> David Handermann
>
> On Tue, May 21, 2024 at 10:19 AM Pierre Villard
>  wrote:
> >
> > Given that the flow analysis rules are something specific to NiFi 2, and
> > given that this is an opportunity to do some house cleaning, I'll go
> > directly to a 2.0.0 release directly and have it used only on the main
> > branch. The things that have been added as improvements are not really
> > required for 1.x anyway.
> >
> > Le mar. 21 mai 2024 à 15:40, David Handermann <
> exceptionfact...@apache.org>
> > a écrit :
> >
> > > Thanks Pierre, sounds good!
> > >
> > > Regards,
> > > David Handermann
> > >
> > > On Tue, May 21, 2024 at 8:36 AM Pierre Villard
> > >  wrote:
> > > >
> > > > Thanks will go ahead with 1.6.0 release process
> > > >
> > > > Le lun. 20 mai 2024 à 19:30, Matt Burgess  a
> écrit
> > > :
> > > >
> > > > > +1 for a 1.6.0 release
> > > > >
> > > > > On Mon, May 20, 2024 at 1:25 PM Bryan Bende 
> wrote:
> > > > >
> > > > > > Thanks Pierre. I agree it would be good to kick out a release. I
> > > would
> > > > > > lean towards 1.6.0 since the commits seem to be improvements
> rather
> > > > > > than bug fixes for a patch.
> > > > > >
> > > > > > On Mon, May 20, 2024 at 1:08 PM Pierre Villard
> > > > > >  wrote:
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > We just merged a couple of improvements for the NiFi NAR Maven
> > > Plugin
> > > > > and
> > > > > > > another improvement has been merged some time ago for which the
> > > > > community
> > > > > > > expressed interest [1]. I think it could be a good time to
> release
> > > a
> > > > > new
> > > > > > > version, either as 1.5.2 or as 1.6.0. Once we have a new
> release,
> > > there
> > > > > > > will be a need for updating the NiFi code base and I have a PR
> > > ready
> > > > > for
> > > > > > > that [2].
> > > > > > >
> > > > > > > I'm happy to help with the release process.
> > > > > > >
> > > > > > > Do you agree it is time for a new release or do you see
> additional
> > > > > > changes
> > > > > > > we should make?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Pierre
> > > > > > >
> > > > > > > [1] https://github.com/apache/nifi-maven/pull/35
> > > > > > > [2] https://issues.apache.org/jira/browse/NIFI-13267
> > > > > >
> > > > >
> > >
>

Re: [DISCUSS] Release NiFi NAR Maven Plugin

[Pierre Villard]

Given that the flow analysis rules are something specific to NiFi 2, and
given that this is an opportunity to do some house cleaning, I'll go
directly to a 2.0.0 release directly and have it used only on the main
branch. The things that have been added as improvements are not really
required for 1.x anyway.

Le mar. 21 mai 2024 à 15:40, David Handermann 
a écrit :

> Thanks Pierre, sounds good!
>
> Regards,
> David Handermann
>
> On Tue, May 21, 2024 at 8:36 AM Pierre Villard
>  wrote:
> >
> > Thanks will go ahead with 1.6.0 release process
> >
> > Le lun. 20 mai 2024 à 19:30, Matt Burgess  a écrit
> :
> >
> > > +1 for a 1.6.0 release
> > >
> > > On Mon, May 20, 2024 at 1:25 PM Bryan Bende  wrote:
> > >
> > > > Thanks Pierre. I agree it would be good to kick out a release. I
> would
> > > > lean towards 1.6.0 since the commits seem to be improvements rather
> > > > than bug fixes for a patch.
> > > >
> > > > On Mon, May 20, 2024 at 1:08 PM Pierre Villard
> > > >  wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > We just merged a couple of improvements for the NiFi NAR Maven
> Plugin
> > > and
> > > > > another improvement has been merged some time ago for which the
> > > community
> > > > > expressed interest [1]. I think it could be a good time to release
> a
> > > new
> > > > > version, either as 1.5.2 or as 1.6.0. Once we have a new release,
> there
> > > > > will be a need for updating the NiFi code base and I have a PR
> ready
> > > for
> > > > > that [2].
> > > > >
> > > > > I'm happy to help with the release process.
> > > > >
> > > > > Do you agree it is time for a new release or do you see additional
> > > > changes
> > > > > we should make?
> > > > >
> > > > > Thanks,
> > > > > Pierre
> > > > >
> > > > > [1] https://github.com/apache/nifi-maven/pull/35
> > > > > [2] https://issues.apache.org/jira/browse/NIFI-13267
> > > >
> > >
>

Re: [DISCUSS] Release NiFi NAR Maven Plugin

[Pierre Villard]

Thanks will go ahead with 1.6.0 release process

Le lun. 20 mai 2024 à 19:30, Matt Burgess  a écrit :

> +1 for a 1.6.0 release
>
> On Mon, May 20, 2024 at 1:25 PM Bryan Bende  wrote:
>
> > Thanks Pierre. I agree it would be good to kick out a release. I would
> > lean towards 1.6.0 since the commits seem to be improvements rather
> > than bug fixes for a patch.
> >
> > On Mon, May 20, 2024 at 1:08 PM Pierre Villard
> >  wrote:
> > >
> > > Hi all,
> > >
> > > We just merged a couple of improvements for the NiFi NAR Maven Plugin
> and
> > > another improvement has been merged some time ago for which the
> community
> > > expressed interest [1]. I think it could be a good time to release a
> new
> > > version, either as 1.5.2 or as 1.6.0. Once we have a new release, there
> > > will be a need for updating the NiFi code base and I have a PR ready
> for
> > > that [2].
> > >
> > > I'm happy to help with the release process.
> > >
> > > Do you agree it is time for a new release or do you see additional
> > changes
> > > we should make?
> > >
> > > Thanks,
> > > Pierre
> > >
> > > [1] https://github.com/apache/nifi-maven/pull/35
> > > [2] https://issues.apache.org/jira/browse/NIFI-13267
> >
>

[DISCUSS] Release NiFi NAR Maven Plugin

[Pierre Villard]

Hi all,

We just merged a couple of improvements for the NiFi NAR Maven Plugin and
another improvement has been merged some time ago for which the community
expressed interest [1]. I think it could be a good time to release a new
version, either as 1.5.2 or as 1.6.0. Once we have a new release, there
will be a need for updating the NiFi code base and I have a PR ready for
that [2].

I'm happy to help with the release process.

Do you agree it is time for a new release or do you see additional changes
we should make?

Thanks,
Pierre

[1] https://github.com/apache/nifi-maven/pull/35
[2] https://issues.apache.org/jira/browse/NIFI-13267

Re: [discuss] What to do with the Cassandra components

[Pierre Villard]

Hey guys, what's the latest on this?

Le sam. 23 mars 2024 à 01:49, Mike Thomsen  a
écrit :

> Fair enough, Joe.
>
> Matt,
>
> I poked around your branch a little this evening. I agree with you and
> David 100% now on the need for some sort of abstraction. I think the Graph
> Bundle's model could serve as a good starting point for how to approach the
> problem. The client drivers in that bundle do the heavy lifting of doing
> the querying and passing the results back via a callback system to the
> processors that call them to ensure that the processors don't know anything
> other than they're getting back a Map of result data each iteration.
> Thoughts?
>
> On Fri, Mar 22, 2024 at 2:48 PM Joe Witt  wrote:
>
> > Mike,
> >
> > The bundles we include cannot have libs with know vulns and that last a
> > very long time.  That is a more pressing blocker.
> >
> > As noted top of thread we all recognize the importance of being able to
> > integrate with Cassandra but including that must come with active mx
> > especially as it relates to vulns.
> >
> > Thanks
> > Joe
> >
> > On Fri, Mar 22, 2024 at 11:42 AM Mike Thomsen 
> > wrote:
> >
> > > The scope tag was probably copy pasta. You raise a valid point about
> the
> > > processor dependencies that completely slipped my mind. That said, I
> > think
> > > it's premature to remove the cassandra bundle until we have a working
> > > replacement. I would consider that support a blocker for 2.X.
> > >
> > > On Fri, Mar 22, 2024 at 12:00 PM Matt Burgess 
> > wrote:
> > >
> > > > David beat me to it :) IMO the only NAR that should have any
> > dependencies
> > > > on Cassandra is the services NAR, not the processors or services API.
> > > >
> > > > On Fri, Mar 22, 2024 at 11:10 AM David Handermann <
> > > > exceptionfact...@apache.org> wrote:
> > > >
> > > > > Mike,
> > > > >
> > > > > Thanks for sharing the branch, it is helpful to have that as a
> > > > > reference example. Have you been able to exercise any of that
> > approach
> > > > > at runtime?
> > > > >
> > > > > Based on what is there right now, attempting to mark the DataStax
> > > > > java-driver-core as provided does not look like it will work. It
> may
> > > > > pass unit tests, but runtime NAR class loading requires that
> classes
> > > > > be available in the same NAR, or in a parent NAR. That means when
> > NiFi
> > > > > tries to load the Controller Service interface, it must have access
> > to
> > > > > a version of the relevant Cassandra driver classes. By marking the
> > > > > dependency as provided, it will not be available in the API NAR,
> and
> > > > > thus not available when loading the service interface. Including it
> > in
> > > > > the API NAR won't work either, because it conflicts with the
> ScyllaDB
> > > > > java-driver-core in the implementation NAR.
> > > > >
> > > > > This is the reason Matt and I highlighted for providing a layer of
> > > > > abstraction at the Controller Service API level.
> > > > >
> > > > > Regards,
> > > > > David Handermann
> > > > >
> > > > > On Fri, Mar 22, 2024 at 8:13 AM Mike Thomsen <
> mikerthom...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > Work so far:
> https://github.com/MikeThomsen/nifi/tree/cql-changes
> > > > > >
> > > > > > On Thu, Mar 21, 2024 at 9:52 AM Mike Thomsen <
> > mikerthom...@gmail.com
> > > >
> > > > > wrote:
> > > > > >
> > > > > > > Matt/David,
> > > > > > >
> > > > > > > By this evening, I should be at a point where I can share my
> > > branch.
> > > > It
> > > > > > > should be far enough along that y'all can see what I mean about
> > how
> > > > > most of
> > > > > > > the changes really weren't that complicated. My sense is that
> if
> > we
> > > > > > > collaborate on it, we can probably get it ready for a PR
> within a
> > > > week
> > > > > or
> > > > > > > two.
> > > > > > >
> > > > > > > It would probably be a good idea to plan to revisit the
> Cassandra
> > > > DMC's
> > > > > > > design and make it more flexible.
> > > > > > >
> > > > > > > One nice thing about the new DataStax driver is that it
> supports
> > > > > > > configuration by a very detailed configuration file format, so
> we
> > > can
> > > > > give
> > > > > > > users that option + combine it with EL/parameters (I envision
> an
> > > > option
> > > > > > > where the user puts EL in the file, we load the file,
> preprocess
> > > the
> > > > > EL and
> > > > > > > load that into the driver)
> > > > > > >
> > > > > > > On Wed, Mar 20, 2024 at 4:01 PM Mike Thomsen <
> > > mikerthom...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> If it were that simple, they would probably have just gone
> with
> > > that
> > > > > > >> solution. That said, the API is functionally vendor agnostic
> at
> > > this
> > > > > point
> > > > > > >> at the Java API level. So I see no need to add abstraction
> above
> > > > > that. I've
> > > > > > >> got probably 2/3 of nifi-cassandra-bundle converted. Hitting a
> > few
> > > > > pain
> > > > > > >> poin

Re: [VOTE] Release Apache NiFi 2.0.0-M3 (RC1)

[Pierre Villard]

+1 (binding)

Went through the usual steps, deployed a secured cluster with a secured
NiFi Registry instance on GCP and tested a bunch of flows that I have
around.

Thanks for RM'ing David!


Le jeu. 16 mai 2024 à 15:26, Matt Burgess  a écrit :

> +1 (binding)
>
> Ran through release helper, verified various flows. Thanks for RM'ing
> David!
>
> On Mon, May 13, 2024 at 11:48 PM David Handermann <
> exceptionfact...@apache.org> wrote:
>
> > Team,
> >
> > I am pleased to be calling this vote for the source release of Apache
> > NiFi 2.0.0-M3.
> >
> > Please review the following guide for how to verify a release candidate
> > build:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> >
> > The source being voted on and the convenience binaries are available
> > on the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-2.0.0-M3
> >
> > The build artifacts are available on the Apache Nexus Repository:
> >
> > https://repository.apache.org/content/repositories/orgapachenifi-1269
> >
> > Git Tag: nifi-2.0.0-M3-RC1
> > Git Commit ID: f2215c6522a5571189290760c55f0317f8562cbd
> > GitHub Commit Link:
> >
> >
> https://github.com/apache/nifi/commit/f2215c6522a5571189290760c55f0317f8562cbd
> >
> > Checksums of nifi-2.0.0-M3-source-release.zip
> >
> > SHA256: d471a0a9e4e04faf13bbe13c7d83be6f8002fba598bf0968a3c1b339802a185a
> > SHA512:
> >
> cd0b8bbd3fe4ea7ebe8cdac6ac8a7afa97fd7b6a521c2cbcb2c0cdc94899b652bf3726c8fe432e16f44a9dc81907414bbb42e03113f0cd9fb51aa1de9cd727a7
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/exceptionfactory.asc
> >
> > KEYS file is available on the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > Issues resolved for this version: 411
> >
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354155
> >
> > Release note highlights can be found on the project wiki:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.0.0-M3
> >
> > The vote will be open for 72 hours.
> >
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test.
> > Then please vote:
> >
> > [] +1 Release this package as nifi-2.0.0-M3
> > [] +0 no opinion
> > [] -1 Do not release this package because...
> >
>

Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.99.0 (RC4)

[Pierre Villard]

+1 (binding)

Went through the usual steps
Built on MacOS X.
Tested a basic flow tailing a local file, doing some transformations and
sending data with InvokeHTTP to a NiFi instance with FlowFile format.

Thanks team for taking care of this release and the exciting upcoming 1.0.

Le mer. 15 mai 2024 à 17:30, Ferenc Gerlits  a écrit :

> +1 (non-binding)
>
> verified signature and hashes
> installed the Windows MSI and ran the service on Windows
> compiled on Fedora 40 and ran ExecuteScript with lua
>
> It mostly worked as expected. I found two minor issues:
> https://issues.apache.org/jira/browse/MINIFICPP-2374,
> https://issues.apache.org/jira/browse/MINIFICPP-2375, but neither of
> these is a showstopper, we'll fix them in the next release.
>
> Thank you for RMing!
> Ferenc
>

Re: Azure lib issue in 1.26

[Pierre Villard]

This will likely motivate a 1.26.1 release. I should be able to do that
next week as I'm traveling for the rest of this week.

Thanks,
Pierre

Le jeu. 9 mai 2024 à 05:01, Joe Witt  a écrit :

> Eduardo
>
> Yes this was found/fixed today[1]
>
> Sorry for the trouble
>
> [1] https://issues.apache.org/jira/browse/NIFI-13181
>
> Thanks
> Joe
>
> On Wed, May 8, 2024 at 6:59 PM Eduardo Fontes 
> wrote:
>
> > Hi!
> >
> > Someone with errors like that em Azure components in Nifi 1.26?
> >
> >  java.lang.NoSuchMethodError:
> > 'com.microsoft.aad.msal4j.AbstractApplicationBase$Builder
> >
> >
> com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.logPii(boolean)'
> >
> > I see thar msal lib was updated since 1.25.
> >
> > Best regards.
> > Eduardo Fontes
> >
>

[RESULT][VOTE] Release Apache NiFi 1.26.0 (RC1)

[Pierre Villard]

Apache NiFi Community,

I am pleased to announce that the 1.26.0 release of Apache NiFi passes:

10 +1 (binding) votes
7 +1 (non-binding) votes
0 0 votes
0 -1 votes

Thanks to all who helped make this release possible!

The release artifacts will be published in the next 24 hours.

Here is the vote thread:
https://lists.apache.org/thread/8tqrzlfq289nf4hlm98ocsnqk88fkqo6

Thanks,
Pierre

Re: [VOTE] Release Apache NiFi 1.26.0 (RC1)

[Pierre Villard]

+1 (binding)

Went through the usual steps and verified the RC with a 3-nodes secured
NiFi cluster + secured NiFi Registry and some flows.

Closing the vote and proceeding with the release.

Thanks,
Pierre

Le lun. 6 mai 2024 à 19:05, Dan S  a écrit :

> +1 (non-binding)
>
> Went through the helper guide and did a clean build
> Verified signatures and hashes
> Built on Linux 4.18.0-513.24.1.el8_9.x86_64
> Java version: Java openjdk version 17.0.11
> Apache Maven 3.9.6
>
> Verified tickets
> [NIFI-12785] InvokeHTTP handler should not urlencode HTTP URL
> [NIFI-12842] InvokeHTTP version wrong encoding of % in URL
> [NIFI-12677] ExcelReader documentation gives an example of a non-existent
> strategy
>
> Thanks for RMing Pierre!
>
> On Mon, May 6, 2024 at 3:52 PM Joe Witt  wrote:
>
> > +1 binding.
> >
> > Thanks Pierre!
> >
> > On Mon, May 6, 2024 at 8:08 AM Krisztina Zsihovszki 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > - Verified signatures and hashes
> > > - Ran build using Maven 3.8.5 on macOS 13.5 with 8.0.392-zulu
> > > - Contrib check, unit tests passed
> > >
> > > Verified these tickets:
> > >
> > > [NIFI-12677] - ExcelReader documentation gives an example of a
> > non-existent
> > > strategy
> > > [NIFI-12960] - Support reading password-protected files in ExcelReader
> > >
> > >
> > > Thanks for RM'ing, Pierre!
> > >
> > > Krisztina
> > >
> > > On Mon, May 6, 2024 at 4:55 PM Michael Moser 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > verified signature, hashes, commit ID
> > > > had a glance at the README, LICENSE, NOTICE files
> > > >
> > > > built on:
> > > > Apache Maven 3.9.6 Maven home:
> > > > /home/mwmoser/.m2/wrapper/dists/apache-maven-3.9.6/a53741d1
> > > > Java version: 1.8.0_412, vendor: Red Hat, Inc., runtime:
> > > > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-2.el8.x86_64/jre
> > > > Default locale: en_US, platform encoding: UTF-8
> > > > OS name: "linux", version: "4.18.0-513.24.1.el8_9.x86_64", arch:
> > "amd64",
> > > > family: "unix"
> > > >
> > > > Ran some simple flows and performed some operations against NiFi
> > > Registry.
> > > >
> > > > Many thanks to Pierre for being RM.
> > > >
> > > > -- Mike
> > > >
> > > >
> > > > On Mon, May 6, 2024 at 10:25 AM Peter Turcsanyi <
> turcsa...@apache.org>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Verified signatures and hashes.
> > > > > Built NiFi on Ubuntu 22.04 with Java 8 (Temurin 1.8.0_412+8) and
> Java
> > > 11
> > > > > (Temurin 11.0.23+9), Maven 3.9.6 with empty local repo.
> > > > >
> > > > > Ran flows for validating:
> > > > >  - NIFI-12837 Add DFS setting to smb processors
> > > > >  - NIFI-12732 ListS3 should reset its tracking state after
> > > configuration
> > > > > change
> > > > >  - NIFI-12936 ListGCSBucket should reset its tracking state after
> > > > > configuration change
> > > > >  - NIFI-12928 Add Simple Write strategy in PutAzureDataLakeStorage
> > > > >
> > > > > Thanks for RMing Pierre!
> > > > >
> > > > > Regards,
> > > > > Peter Turcsanyi
> > > > >
> > > > > On Mon, May 6, 2024 at 4:09 PM Chris Sampson
> > > > >  wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Ran through the release helper. Verified signatures and hashes.
> > > > > >
> > > > > > Full clean build of RC with Java openjdk 11.0.23 2024-04-16 on
> > macOS
> > > > > > Sonoma 14.4.1 using the Maven Wrapper 3.9.6.
> > > > > >
> > > > > > Ran a selection of processors/flows, primarily focussed on
> > > > Elasticsearch
> > > > > > integration and HTTP Request/Response.
> > > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > ---
> > > > > > Chris Sampson
> > > > > > IT Consultant
> > > > > > chris.s

[VOTE] Release Apache NiFi 1.26.0 (RC1)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.26.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.26.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1265

Git Tag: nifi-1.26.0-RC1
Git Commit ID: 338083f6850b61cd2651e41bbd73b3cb5330d98c
GitHub Commit Link:
https://github.com/apache/nifi/commit/338083f6850b61cd2651e41bbd73b3cb5330d98c

Checksums of nifi-1.26.0-source-release.zip

SHA256: 75ea201c12bb99672b1f075c9520b5f7df8b24e033ec47121848b351a0d47790
SHA512:
5b754d899ce8cff900d5871d44c2fda9224e69fe8a0fe9a7121f3359ed8881300e4d4d0b2fe5befea276e0495c7ebbed04cc307c18527aa7a1cea25a923a

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 128

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12354185

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.26.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.26.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [DISCUSS] Release Timeline for NiFi 2.0.0-M3

[Pierre Villard]

I'm starting the process for the 1.26 RC as I don't believe we're waiting
for any additional PRs there.

Le jeu. 2 mai 2024 à 07:30, Joe Witt  a écrit :

> Matt
>
> You are referring to NIFI-12998 and it causing rebasing to be necessary for
> outstanding PRs?
>
> The rebase should be quite straight forward for most cases.
>
> If you have any questions Im happy to help.
>
>  I rebased a tremendous number of times to keep up with main changing while
> the PR took several weeks to create and test and validate against previous
> builds.  Also rebased repeatedly during the review and feedback stage.  As
> a result I got to practice my rebase skills about 20 times across 50 or
> more commits to main during that time.
>
> On your pr branch
>
> git fetch —all
> git rebase upstream/main
> git status
> fix any conflicts
> git add .
> git rebase —continue
>
> If any conflicts it would likely be for new modules created in the pr which
> would just be moving from a path in nifi-nar-bundles to
> nifi-extension-bundles.
>
> If you have a pr in mind you have a question on for the rebase let me know.
>
> Thanks
> Joe
>
> On Wed, May 1, 2024 at 9:51 PM Matt Burgess  wrote:
>
> > One thing that was mentioned was an included Jira for  "providing a
> clearer
> > distinction between framework and
> > extensions," This involved moving extensions to a new module and for
> > community folks this is causing problems with any
> > new improvements in-flight before that. Seems like it needed more
> > announcement than an upcoming RC due to the impact?
> >
> > On Mon, Apr 29, 2024 at 10:22 AM David Handermann <
> > exceptionfact...@apache.org> wrote:
> >
> > > Team,
> > >
> > > We are getting closer to ready for a release candidate build. Based on
> > > some conversations with those working on the user interface, there are
> > > still a couple more items to complete this week. Getting a solid build
> > > of the new UI into this next version will be very helpful, so getting
> > > a few more of those issues resolved should put things in a good
> > > position.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > On Wed, Apr 24, 2024 at 8:32 AM Joe Witt  wrote:
> > > >
> > > > Also agreed there.  Profile should be there to exclude perhaps but
> > > include
> > > > should be default.
> > > >
> > > > On Wed, Apr 24, 2024 at 6:30 AM David Handermann <
> > > > exceptionfact...@apache.org> wrote:
> > > >
> > > > > Thanks for the replies thus far!
> > > > >
> > > > > With the goal of including the new UI, it seems like a good time to
> > > > > move the module from an optional profile to part of the standard
> > > > > build. That would make the release candidate build and review
> process
> > > > > simpler, not requiring the optional build profile.
> > > > >
> > > > > Regards,
> > > > > David Handermann
> > > > >
> > > > > On Tue, Apr 23, 2024 at 2:42 PM Matt Gilman <
> matt.c.gil...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > I agree. Including the updated UI and getting some feedback would
> > be
> > > > > great.
> > > > > >
> > > > > > On Mon, Apr 22, 2024 at 8:50 PM Joe Witt 
> > wrote:
> > > > > >
> > > > > > > Id be a big fan of including the new UI.
> > > > > > >
> > > > > > > On Mon, Apr 22, 2024 at 5:45 PM Pierre Villard <
> > > > > > > pierre.villard...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks David, I'm happy to take care of a 1.26 release at the
> > > same
> > > > > time.
> > > > > > > >
> > > > > > > > Regarding the M3 release, should the new UI be included and
> > > > > instructions
> > > > > > > > given on how to access it to start collecting feedback? I'm
> > > under the
> > > > > > > > impression that almost all of the work has been done, no?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Pierre
> > > > > > > >
> > > > > > &

Re: [DISCUSS] Release Timeline for NiFi 2.0.0-M3

[Pierre Villard]

Thanks David, I'm happy to take care of a 1.26 release at the same time.

Regarding the M3 release, should the new UI be included and instructions
given on how to access it to start collecting feedback? I'm under the
impression that almost all of the work has been done, no?

Thanks,
Pierre

Le mar. 23 avr. 2024, 02:03, David Handermann 
a écrit :

> Team,
>
> We are approaching 300 Jira issues resolved for version 2.0.0-M3 [1],
> including a number of important dependency upgrades and Python
> framework improvements.
>
> There are several open pull requests that are getting close to
> completion, which should be possible to include in an upcoming release
> version. There is also a significant pull request [2] for NIFI-12998
> [3] that includes some important changes to module directory
> hierarchy, providing a clearer distinction between framework and
> extensions, and implementing significant cleanup of dependency
> declarations across the repository.
>
> With these changes in view, we should be ready for another milestone
> release soon after merging the above pull request.
>
> Another milestone release seems to be the best course of action for
> now, providing the opportunity for additional review and testing
> before a full 2.0.0 release version.
>
> I would be glad to handle Release Manager responsibilities for
> 2.0.0-M3 when things are ready.
>
> Regards,
> David Handermann
>
> [1]
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%202.0.0-M3
> [2] https://github.com/apache/nifi/pull/8677
> [3] https://issues.apache.org/jira/browse/NIFI-12998
>

Re: Request for nifi custom processor development guidance in python

[Pierre Villard]

Hi,

If you're using NiFi 2.0, then you can use Python. I recommend the below
resources:
https://www.youtube.com/watch?v=9Oi_6nFmbPg
https://nifi.apache.org/documentation/nifi-2.0.0-M2/html/python-developer-guide.html

HTH,
Pierre

Le mer. 17 avr. 2024 à 15:18, Usha Kiran Mahato 
a écrit :

> Hi Team,
>
> I am Usha, a python developer. Recently I got a project to work on nifi. To
> implement Client specific requirement, I have to build a custom processor
> in nifi. Currently I use java to build that.
>
> As you know python is reach in library, I know jython is there but it
> supports till python 2.7. So there are some limitations.
>
> Is there any way, so that I can build my custom processor in python instead
> of java with the LTS python version.
>
> It will be very helpful to me.
>
> Thanking you
>
> Regards
> Usha Kiran Mahato
> email-ushakiranmaha...@gmail.com
> Ph No.- +91 858 285 2477/
> +91 890 637 6213
>

Re: [DISCUSS] MiNiFi C++ 1.0.0-M1 release

[Pierre Villard]

I'm a +1 with this approach. Being able to use the Python extensions in
MiNiFi cpp is great!

Thanks,
Pierre

Le mar. 16 avr. 2024 à 18:39, Gábor Gyimesi  a écrit :

> Hi community,
>
> I'd like to initiate a discussion about the next release of MiNiFi C++. The
> last release was more than seven months ago, and there have been many new
> features, bug fixes and stability improvements committed to the development
> branch since then: 107 tickets closed, over 122 commits as of today.
>
> I would be happy to take on RM duties for this release.
>
> Notable features and improvements since the 0.15.0 release:
>
> New notable features:
> - Added support for using NiFi 2.0 Python processors in MiNiFi C++
>   - This also includes several improvements to the previous MiNiFi style
> python processors, like additional property options, custom relationships
> and virtualenv support
> - Added new python based multiplatform bootstrap script
> - Added encryption support for sensitive properties in flow configuration
> - Releasing Windows installer now can be done (and will be done) under the
> Apache license
> - Added support for service installation on MacOS
> - Added C2 debug command to MiNiFi Controller
> - Added support for setting MiNiFi properties from command line
> - Added system load average field to C2 and Prometheus metrics
> - Added support for manually configuring RocksDB options
> - Added custom delimiter property for ListenTCP processor
> - Added bandwidth limit properties to InvokeHTTP processor
> - Added JSON flow configuration examples
>
> New processors:
> - Added PutSmb, FetchSmb and ListSmb processor for SMB networking protocol
> support
> - Added PushGrafanaLokiGrpc and PushGrafanaLokiREST processors for pushing
> logs to Grafana Loki
> - Added JoltTransform to use Jolt JSON transformations
> - Added SplitText processor
> - Added AttributeRollingWindow processor
>
> Changes and improvements:
> - Dropped support for disabling peer verification in InvokeHTTP
> - Corrupt flow files are now filtered to avoid errors in the flow
> - Using administrative yield duration instead of onschedule retry interval
> in scheduling adjusting to NiFi's functionality
> - Fixed high disk IO usage issue with MergeContent
> - Fixed the site-to-site transfer or large files
> - Fixed memory leak caused by unused loggers
> - Fixed yielding processors to still respect scheduling period
>
> Upgraded dependencies:
> - Upgraded OpenSSL to version 3.3.0
> - Upgraded AWS SDK to version 1.11.219 with support for new AWS regions
> - Upgraded libuvc to version 0.0.7
> - Upgraded docker base image to alpine:3.18
> - Upgraded Sol2 to version 3.3.0
>
> With the current maturity level of the project and with the support for
> NiFi 2.0 style python processors and json flow configuration, I suggest
> releasing it as version 1.0.0-M1 milestone release.
>
> Do you agree it is time for a new release? Do you agree with the suggested
> version? Are there any blockers that we should definitely include in this
> release?
>
> Thanks,
> Gabor
>

Re: Is there a roadmap for Nifi V2 ?

[Pierre Villard]

Things on top of my head that are still being worked on:
- improvements, stability and performance of the Python API for
writing components in Python
- the rewrite of the NiFi UI using the latest version of some libraries but
it has no impact on NiFi adoption, you can use the current UI (same as in
1.x)
- the addition of the rules engine feature in the UI, but it has also no
impact on adoption as this is really something very specific

Le mer. 13 mars 2024 à 17:19, Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> a écrit :

> If we wanted to evaluate if the NiFi 2.0-M2 release is sufficiently
> production worthy for our individual use cases, is there a list of the
> remaining items/known key gaps/etc to get in so we can make a decision if
> those remaining items are noteworthy to our cases?
>
> Thanks,
> Ryan
>
> On Mon, Mar 11, 2024 at 11:50 AM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > Hi Robert,
> >
> > I'd expect an M3 release in the next couple of weeks and the first
> official
> > 2.0 release should happen in the next couple of months I think. There is
> no
> > official timeline as it depends on the community efforts around the
> > remaining things we want to get it as well as the feedback from the
> > community on the milestone releases.
> >
> > Having said that, if you're new with NiFi, I'd say it is safe to start
> with
> > 2.0-M2 right now as long as you're ok with doing upgrades when the next
> > releases are going out. Some companies are already using 2.0-M2 in
> > production use cases.
> >
> > Hope this helps,
> > Pierre
> >
> > Le lun. 11 mars 2024 à 15:14, Provencher, Robert <
> > provencher.rob...@uqam.ca>
> > a écrit :
> >
> > >
> > > Hi!
> > >
> > >
> > > I'm trying to find if there is a roadmap for Nifi V2.
> > >
> > > Are there dates, a calendar, to show the steps and to aim when the
> first
> > > production-safe Nifi V2 version would be published ?
> > >
> > > Are there approximations of this date ?
> > >
> > > The reason I'm asking is that we are seriously considering using Nifi.
> > > We are asking ourselves if it would be better to start using Nifi by
> > > directly begin with V2, configuring our flows with V2 ?
> > >
> > > So a roadmap would help in taking our decision 🙂
> > >
> > > Thanks for your help and your good work !!!
> > >
> > >
> > > Robert
> > >
> >
>

Re: Is there a roadmap for Nifi V2 ?

[Pierre Villard]

Hi Robert,

I'd expect an M3 release in the next couple of weeks and the first official
2.0 release should happen in the next couple of months I think. There is no
official timeline as it depends on the community efforts around the
remaining things we want to get it as well as the feedback from the
community on the milestone releases.

Having said that, if you're new with NiFi, I'd say it is safe to start with
2.0-M2 right now as long as you're ok with doing upgrades when the next
releases are going out. Some companies are already using 2.0-M2 in
production use cases.

Hope this helps,
Pierre

Le lun. 11 mars 2024 à 15:14, Provencher, Robert 
a écrit :

>
> Hi!
>
>
> I'm trying to find if there is a roadmap for Nifi V2.
>
> Are there dates, a calendar, to show the steps and to aim when the first
> production-safe Nifi V2 version would be published ?
>
> Are there approximations of this date ?
>
> The reason I'm asking is that we are seriously considering using Nifi.
> We are asking ourselves if it would be better to start using Nifi by
> directly begin with V2, configuring our flows with V2 ?
>
> So a roadmap would help in taking our decision 🙂
>
> Thanks for your help and your good work !!!
>
>
> Robert
>

Re: QueryRecord processor fails with use of CONTAINS_SUBSTR

[Pierre Villard]

Hi Dan,

As you can see on the doc there is a 'b' next to CONTAINS_SUBSTR and the
documentation explains its meaning:

The ‘C’ (compatibility) column contains value:
> ‘*’ for all libraries,
> ‘b’ for Google BigQuery (‘fun=bigquery’ in the connect string),
> ‘c’ for Apache Calcite (‘fun=calcite’ in the connect string),
> ‘h’ for Apache Hive (‘fun=hive’ in the connect string),
> ‘m’ for MySQL (‘fun=mysql’ in the connect string),
> ‘q’ for Microsoft SQL Server (‘fun=mssql’ in the connect string),
> ‘o’ for Oracle (‘fun=oracle’ in the connect string),
> ‘p’ for PostgreSQL (‘fun=postgresql’ in the connect string),
> ’s’ for Apache Spark (‘fun=spark’ in the connect string).


I believe we only support the ones for Apache Calcite. This is probably
something we could/should improve.

HTH,
Pierre


Le mer. 6 mars 2024 à 18:14, Dan S  a écrit :

> I am trying to use the QueryRecord processor with a SQL statement similar
> to:
>
> SELECT mi FROM FLOWFILE WHERE CONTAIN_SUBSTR(somedetails, 'Fred')
>
> which fails with the following error message in the logs:
> Caused by org.apache.calcite.runtime.CalciteContextException: From column
> 31 to line 1, column 66: No match found for function signature
> CONTAINS_SUBSTR(,  )
>
> In the Apache Calcite documentation
> I see CONTAINS_SUBSTR
> defined. Why cannot I use it in NIFI? Are there limitations of what
> functions that can be used from Apache Calcite?
>

Re: NiFi 2.0.0-M2

[Pierre Villard]

Hi,

NiFi is OS-agnostic for linux based distributions. You can download the
tar.gz from the download page of the website and follow the instructions to
get started on CentOS 7.

HTH,
Pierre

Le mar. 5 mars 2024 à 14:07, Gleb Efimov  a écrit :

> Good afternoon I'm trying to deploy NiFi 2.0.0-M2 on Centos 7, but,
> unfortunately, I can't find a tar archive with the distribution I need.
> Could you tell me where I can get it?
> And is it even possible to deploy NiFi 2.0.0-M2 on Centos 7?
> Thank you very much.
>
> Sincerely, Efimov Gleb.
>

[ANNOUNCE] Apache NiFi 1.25.0 Released

[Pierre Villard]

The Apache NiFi Team is pleased to announce the release of Apache NiFi
1.25.0.

Apache NiFi is an easy to use, powerful, and reliable system to
process and distribute data.

https://nifi.apache.org

The release artifacts can be downloaded from the project website.

https://nifi.apache.org/download/

Maven artifacts have been released and mirrored according to ASF
artifact distribution processes.

Issues resolved in Apache NiFi 1.25.0 are listed in Jira Release Notes.

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353860

Highlights of the release are available on the project wiki page:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.25.0

Thank you,
The Apache NiFi Team

[RESULT][VOTE] Release Apache NiFi 1.25.0 (RC1)

[Pierre Villard]

Apache NiFi Community,

I am pleased to announce that the 1.25.0 release of Apache NiFi passes:

6 +1 (binding) votes
8 +1 (non-binding) votes
0 0 votes
0 -1 votes

Thanks to all who helped make this release possible!

The release artifacts will be published in the next 24 hours.

Here is the vote thread:

https://lists.apache.org/thread/hn0tcvpl8d70t7zcm3j1bydql2dx91hj

Re: [VOTE] Release Apache NiFi 1.25.0 (RC1)

[Pierre Villard]

> >
> > > > > Went through the helper guide and did a clean build
> > > > > Verified signatures and hashes
> > > > > Built on OSX 14.2.1
> > > > > Java version: 11.0.20, OpenJDK 64-Bit Server VM Zulu11.66+15-CA
> > (build
> > > > > 11.0.20+8-LTS, mixed mode)
> > > > > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
> > > > > Started NiFi and created a simple flow
> > > > > Started MiNiFi and verified integration with C2 Server. Played
> around
> > > > with
> > > > > the C2 protocol (Operations - Update, Transfer, Describe...)
> > > > > Thanks for RMing, Pierre!
> > > > >
> > > > > Regards,
> > > > >
> > > > > Csaba
> > > > >
> > > > >
> > > > > > On 28 Jan 2024, at 05:28, Ferenc Kis 
> > > wrote:
> > > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Went through the helper guide, local maven repo cleaned up, full
> > > clean
> > > > > > build with contrib check, verified signatures and hashes
> > > > > >
> > > > > > Apache Maven 3.9.0 (9b58d2bad23a66be161c4664ef21ce219c2c8584)
> > > > > > Maven home: /Users/fkis/.sdkman/candidates/maven/current
> > > > > > Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
> > > > > >
> > > > >
> > > >
> > >
> >
> /Users/fkis/.sdkman/candidates/java/8.0.362-zulu/zulu-8.jdk/Contents/Home/jre
> > > > > > Default locale: en_HU, platform encoding: UTF-8
> > > > > > OS name: "mac os x", version: "14.2.1", arch: "aarch64", family:
> > > "mac"
> > > > > >
> > > > > > Validations performed:
> > > > > > - Started NiFi, created a simple flow with ListenHTTP and Input
> > Port.
> > > > > > Validated ListenHTTP processor is able to receive data
> > > > > > - Started MiNiFi Java:
> > > > > >  * created a simple GenerateFlowFile -> InvokeHttp flow and
> > > > > > GenerateFlowFile -> RemoteProcessGroup flow and pushed to MiNiFi
> > via
> > > C2
> > > > > > protocol. Validated that connectivity between NiFi and MiNiFi
> works
> > > via
> > > > > > both InvokeHTTP and S2S
> > > > > >
> > > > > > Thanks for RMing Pierre!
> > > > > >
> > > > > > Regards
> > > > > > Ferenc
> > > > > >
> > > > > > On Sat, Jan 27, 2024 at 9:05 PM Krisztina Zsihovszki <
> > > > zsikr...@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> +1 (non-binding)
> > > > > >>
> > > > > >> - Verified signatures and hashes
> > > > > >> - Ran build using Maven 3.8.7 on macOS 13.5 with openjdk 11.0.18
> > > > > 2023-01-17
> > > > > >> and Zulu17.42+19-CA (build 17.0.7+7-LTS)
> > > > > >> - Contrib check, unit tests passed
> > > > > >>
> > > > > >> Checked basic flows and verified the following tickets:
> > > > > >>
> > > > > >> [NIFI-12331] - Introduce a PublishSlack processor
> > > > > >> [NIFI-12386] - Add a FilterAttribute processor
> > > > > >>
> > > > > >>
> > > > > >> Thanks for RM'ing, Pierre!
> > > > > >> Regards,
> > > > > >>
> > > > > >> Krisztina
> > > > > >>
> > > > > >> On Sat, Jan 27, 2024 at 4:43 PM Márk Báthori <
> > > bathori.m...@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> +1 non-binding
> > > > > >>>
> > > > > >>> Went through the verification guide, verified hashes and
> > > signatures,
> > > > > ran
> > > > > >> a
> > > > > >>> full build on NiFi with contrib-check successfully.
> > > > > >>>
> > > > > >>> Environment:
> > > > > >>> - MacOs 13.4 (22F66)
> > > > >

Re: INTEGRATION OF APACHE NIFI AS THIRD-PARTY SOFTWARE

[Pierre Villard]

Hi,

There is no attached image in your email.

Thanks,
Pierre

Le mar. 30 janv. 2024 à 17:12, Durante Lorenzo  a
écrit :

> Good morning,
>
>
>
> I would like to engage in a discussion with you regarding some
> functionalities of Apache NiFi, to assess its potential application in a
> professional setting.
>
>
>
> I need to understand if Apache NiFi is capable of performing tasks as
> described in the attached images (when used as a SUPERVISOR - Third-Party
> Control Software). Specifically, I am interested in knowing whether Apache
> NiFi can receive and process Modbus signals, convert them into XML format
> to communicate these messages to another software, and, if necessary, read
> and process the responses sent back to Apache NiFi from this software.
>
>
>
> I look forward to your response and wish you a pleasant day and successful
> work.
>
>
>
> Kind regards,
>
>
>
> *Lorenzo Durante*
>
> System Engineer
> Via Carlo Veneziani, 58 – 00148 Rome (Italy)
>
> General Dynamics Mission Systems-Italy
>
>
> --
> This message and/or attachments may include information subject to GD
> Corporate Policies 07-103 and 07-105 and is intended to be accessed only by
> authorized recipients. Use, storage and transmission are governed by
> General Dynamics and its policies. Contractual restrictions apply to third
> parties. Recipients should refer to the policies or contract to determine
> proper handling. Unauthorized review, use, disclosure or distribution is
> prohibited. If you are not an intended recipient, please contact the sender
> and destroy all copies of the original message.
>

Re: Apache NIFI: Query on logging.

[Pierre Villard]

LogAttribute can log the flowfile's content. Set Log Payload to true.

HTH,
Pierre

Le sam. 27 janv. 2024 à 18:45, Kumar Kapil  a
écrit :

> Hi to NIFI dev team,
>
>
>
> I have below requirements where documentation or your assistance is
> required in the logging. Currently LogMessage and LogAttribute process are
> supported. But is it enough to log json object or request body and response
> body. We need a processor which can log file-flow content, JSON documents
> and other useful information.
>
>
>
> *Requirement 1: *
>
>- Consume events from the MQTT.
>- Log the events in the log file.
>- Transform the event by using JOLT Transformation.
>- Log the transformed JSON document in the log file.
>- Invoke HTTP request and post it.
>- Log request body and response body with http status code in log
>file.
>
>
>
> *Requirement 2: *
>
>- Exposed Rest API from NIFI
>- Log request body in the log file
>- Send it to Kafka.
>- Log the Kafka event in the log file.
>
>
>
> Regards,
>
>
>
> Kumar Kapil
>
> Software Architect
>
>
>
> checkpoint
> systems.com
> 
>
> +(91) 8050050113
>
> kumar.ka...@checkpt.com
>
>
>
>
>

Re: [DISCUSS] New Project Repository for Python Extensions?

[Pierre Villard]

Hi David,

While I agree with your summary, I have a concern here which is about user
awareness of this feature. We've seen in the past: as soon as we don't
include NARs in the convenience binary, we see that users have no clue
about those NARs (and some are super powerful/useful). I agree that python
is a bit different because it requires a user action to enable it in the
first place but I still think that including the components in the
convenience binary of Apache NiFi would drive user awareness, adoption, etc.

If we have a separated repo with its own release cycle can we imagine a
process where, when releasing Apache NiFi, it'd include whatever is the
latest version of the Python repo? Or something along those lines?

Pierre

Le ven. 26 janv. 2024 à 08:01, David Handermann 
a écrit :

> Team,
>
> As we get closer to a full release of Apache NiFi 2.0.0, we have an
> important opportunity to set the direction for future development of
> Python-based Processors.
>
> The introduction of native Python support presents a number of new
> integration opportunities, and it also raises questions about
> maintenance and versioning. As the journey to NiFi 2.0.0 has shown, it
> requires significant effort to coordinate maintenance and
> modernization across hundreds of project modules. Although the
> internal project structure has maintained helpful separation of API
> and implementation, the current release strategy highlights the
> challenges of verifying multiple layers of changes. Introducing a new
> programming language provides greater possibilities, but also makes it
> more difficult to maintain a single repository with a single
> versioning strategy.
>
> I propose creating a new Git repository named nifi-python-extensions,
> which would have its own versioning and release process. This would
> contain the extensions now under the module of the same name in the
> NiFi repository. Having a separate repository and release process for
> Python-based extensions has the following advantages:
>
> 1. Clean separation between NiFi APIs for Python and Python-based
> Processors
> 2. Independent release cycles for Python-based Processors
> 3. Focused release verification and testing on Python-based modules
>
> These advantages can also enable more rapid iteration on Python-based
> Processors, without impacting the NiFi Framework or requiring new
> releases at that level. Although this would require a separate
> installation process for Python-based components, this could follow an
> approach similar to what is already required for optional Java-based
> components.
>
> Thanks in advance for your consideration.
>
> Regards,
> David Handermann
> Apache NiFi PMC Member
>

[VOTE] Release Apache NiFi 1.25.0 (RC1)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.25.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.25.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1263

Git Tag: nifi-1.25.0-RC1
Git Commit ID: 6ecc398d3f92425447e43242af4992757e25b3c5
GitHub Commit Link:
https://github.com/apache/nifi/commit/6ecc398d3f92425447e43242af4992757e25b3c5

Checksums of nifi-1.25.0-source-release.zip

SHA256: 6a14b35bf5beb4ae3fcf76df8d09676e61c6bc309a97dc6785eae84b7cbaef78
SHA512:
1ffb2586ce9591ce2c5aa39fdec43a89ffd29081a31d51dc95dd953cb7f94584d0a0171bb1ba7c9495f431aec3770d324dbabb319b01bb6fdce5a0a00426fffa

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 103

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353860

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.25.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.25.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [DISCUSS] Validator hook for Registry events

[Pierre Villard]

Good question Bence. I'm not sure it's worth the effort to have this
setting per registry. The reason is that, as of today, a flow can only be
versioned into a single registry. And a process group cannot be "attached"
to a versioned flow in the registry. So I'm not sure how frequent the
scenario you're describing is. One would commit new versions in dev
registry, and then when pushing to prod registry they'd have to stop
versioning and commit as a new version in "prod" registry, that seems very
unlikely and a very complicated process. A more common scenario is people
committing in the dev registry and then they have some scripts in place
(using the scripted event hook that you mentioned) to sync flows into a
prod registry. IMO we should just have something working for all configured
registries regardless of the implementations of the endpoint.

Le jeu. 18 janv. 2024 à 10:07, Simon Bence  a
écrit :

> Thank you for the feedback! I look into this and check how it would be
> best to apply the rules engine here. Most probably the changes should be
> somewhere around StandardFlowRegistryClientNode so this can be used
> regardless of the actual plugged in client. However this comes with a
> further question: if it is part of the framework and there are more
> registry clients added to the NiFi instance, do we want to provide the
> option to differentiate between registries? As far as I know the rules
> engine, differentiation on the level of applied rules per registry would be
> challenging with the current setup but we can provide a toggle for the
> feature on a “per registry” basis. The motivation for this: in multiple
> occasions I saw cases where people used “in dev” registry for development
> before committing their flow into the final one. So from flow development
> lifecycle it might be too restricting to apply this for every registry (if
> the feature is turned on in the first place)
>
> Regards,
> Bence
>
> >
> > On 2024. Jan 17., at 17:59, Bryan Bende  wrote:
> >
> > Good point, since registry client is an extension point, using the
> > rules on NiFi side allows it to work for any registry client.
> >
> > On Wed, Jan 17, 2024 at 11:49 AM Pierre Villard
> >  wrote:
> >>
> >> Yeah so the more I think about it the more we should probably couple
> this
> >> with the new rules engine feature of NiFi 2.0. Users could define their
> >> rules there and then it could just be a property in NiFi saying whether
> or
> >> not it should prevent someone from committing a new version if one of
> the
> >> required rules is raising a violation in the process group being
> committed.
> >> This way this would work with any registry implementation, not just the
> >> NiFi Registry (I'm saying this in case we ever get to a pure Git-based
> >> implementation of the Registry endpoint).
> >>
> >> Le mer. 17 janv. 2024 à 20:33, Bryan Bende  a écrit :
> >>
> >>> Michael,
> >>>
> >>> I think this validation idea came from the absence of a pull request
> >>> review model. Meaning, the rules being discussed here would check for
> >>> the things someone would most likely look for in a review of the flow
> >>> changes to decide if the flow is good. Obviously it wouldn't be able
> >>> to do everything a person would, but it could be better than nothing.
> >>> As far as I know, there isn't any active work around building a review
> >>> model.
> >>>
> >>> -Bryan
> >>>
> >>> On Wed, Jan 17, 2024 at 11:18 AM Michael Hogue
> >>>  wrote:
> >>>>
> >>>> Could this be accomplished through git webhooks once changes are
> >>> committed
> >>>> through Registry and CI pipelines to perform whatever validation you
> >>> wish?
> >>>>
> >>>> The glaring weakness in this approach is that the changes have already
> >>> been
> >>>> committed. This makes me wonder if we can improve the "flow
> development
> >>>> lifecycle" to enable teams to review flow changes before they're
> >>>> committed, much like a pull request. Is this on the roadmap for
> Registry?
> >>>>
> >>>> Mike
> >>>>
> >>>> On Wed, Jan 17, 2024 at 4:14 PM Bryan Bende  wrote:
> >>>>
> >>>>> Thanks guys, that makes sense in the context of the review model.
> >>>>>
> >>>>> Obviously some kind of general pre-event hook is the most flexible,
> >&g

Re: Toolkit Errors

[Pierre Villard]

Hi Christhian,

This is a known issue and will be fixed in the upcoming 2.0-M2 release that
should go out very soon.

Thanks,
Pierre

Le jeu. 18 janv. 2024 à 06:40, Cristhian Alvarez Cifuentes <
kristhian_...@hotmail.com> a écrit :

> Hi Nifi Team, how are you?
>
>  i got a problem with toolkit...
>
> I have installed Nifi 2.0 + registry + toolkit
>
> i designed flows, used procesors, and everything its working fine
>
> but mi problem its with toolkit, why?
>
> because i want to know how to use Nifi from command line thinking on
> administration team, but i have an error:
>
> #> registry list-buckets -u -baseline http://localhost:18080/
>
> ERROR: Error executing command 'list-buckets' :
> java.lang.ClassNotFoundException: Provider for
> jakarta.ws.rs.client.ClientBuilder cannot be found
>
> the registry its working fine with buckets versioned from Nifi
>
> you can help me?
>
> thanks
>

Re: [DISCUSS] Validator hook for Registry events

[Pierre Villard]

Yeah so the more I think about it the more we should probably couple this
with the new rules engine feature of NiFi 2.0. Users could define their
rules there and then it could just be a property in NiFi saying whether or
not it should prevent someone from committing a new version if one of the
required rules is raising a violation in the process group being committed.
This way this would work with any registry implementation, not just the
NiFi Registry (I'm saying this in case we ever get to a pure Git-based
implementation of the Registry endpoint).

Le mer. 17 janv. 2024 à 20:33, Bryan Bende  a écrit :

> Michael,
>
> I think this validation idea came from the absence of a pull request
> review model. Meaning, the rules being discussed here would check for
> the things someone would most likely look for in a review of the flow
> changes to decide if the flow is good. Obviously it wouldn't be able
> to do everything a person would, but it could be better than nothing.
> As far as I know, there isn't any active work around building a review
> model.
>
> -Bryan
>
> On Wed, Jan 17, 2024 at 11:18 AM Michael Hogue
>  wrote:
> >
> > Could this be accomplished through git webhooks once changes are
> committed
> > through Registry and CI pipelines to perform whatever validation you
> wish?
> >
> > The glaring weakness in this approach is that the changes have already
> been
> > committed. This makes me wonder if we can improve the "flow development
> > lifecycle" to enable teams to review flow changes before they're
> > committed, much like a pull request. Is this on the roadmap for Registry?
> >
> > Mike
> >
> > On Wed, Jan 17, 2024 at 4:14 PM Bryan Bende  wrote:
> >
> > > Thanks guys, that makes sense in the context of the review model.
> > >
> > > Obviously some kind of general pre-event hook is the most flexible,
> > > but I would say we should also consider whether we really want to be
> > > calling out to arbitrary scripts during the main API requests, as well
> > > as consider what someone would have to do to implement a scripted
> > > rule. The current events are all metadata related, they don't have the
> > > content of the snapshots, so in this case we'd have to pass the entire
> > > json string of a snapshot as an arg to the script and then the script
> > > has to parse through it looking for fields to validate/check. I wonder
> > > if we should consider defining the types of rules ourselves and
> > > allowing some kind of rules config file to be provided. This way the
> > > rules are loaded during start up and applied in Java code by our
> > > framework code. Downside is it is less flexible in that we probably
> > > won't be able to know every rule someone wants ahead of time. I guess
> > > the other options are to consider whether tagging or nifi side rules
> > > are better suited to solve this problem.
> > >
> > > On Wed, Jan 17, 2024 at 10:48 AM Simon Bence  >
> > > wrote:
> > > >
> > > > HI,
> > > >
> > > > I think, both of you are correct, my initial example of the possible
> > > usage was not the best. Having a control over committing versions for
> flows
> > > is maybe the most important benefit we can gain. Contrary to naming
> > > conventions this can be a more complex endeavour. So I suggest moving
> on
> > > with this example instead of the original one.
> > > >
> > > > Regards,
> > > > Bence
> > > >
> > > > > On 2024. Jan 17., at 16:02, Pierre Villard <
> > > pierre.villard...@gmail.com> wrote:
> > > > >
> > > > > I guess the issue relates to some recurrent discussions around the
> > > lack of
> > > > > mechanism for a Pull Request kind of model for "approving" a
> versioned
> > > > > flow. An alternative we've been discussing for a long time is the
> > > ability
> > > > > to set tags to versions so that a versioned flow could be reviewed
> > > (diff
> > > > > with previous version(s)) and if approved a tag would be applied
> and
> > > this
> > > > > tag would be used to potentially trigger an automatic deployment.
> > > > >
> > > > > The proposal is not helping with the "PR model" but would allow
> > > someone to
> > > > > control what is versioned. A user may have the permissions to
> commit a
> > > new
> > > > > ver

Re: [DISCUSS] Validator hook for Registry events

[Pierre Villard]

I guess the issue relates to some recurrent discussions around the lack of
mechanism for a Pull Request kind of model for "approving" a versioned
flow. An alternative we've been discussing for a long time is the ability
to set tags to versions so that a versioned flow could be reviewed (diff
with previous version(s)) and if approved a tag would be applied and this
tag would be used to potentially trigger an automatic deployment.

The proposal is not helping with the "PR model" but would allow someone to
control what is versioned. A user may have the permissions to commit a new
version but one may not want to accept a flow where a processor is
configured with 50 concurrent tasks (for example). That would get deployed
in prod and potentially impact existing deployed flows. I guess one could
see this approach as a "checkstyle" plugin to allow or not the commit of a
new version.

Le mer. 17 janv. 2024 à 18:56, Bryan Bende  a écrit :

> I think before we embed something into synchronous API requests we
> should try to define a few more real uses besides just a bucket naming
> pattern to see if we really need to add something like this. For
> bucket naming pattern, it could just be a simple property in
> nifi-registry.properties with a regex to check names against. If we
> did add some kind of pluggable validator, I don't think these
> validators should be making authorization decisions, it should be
> strictly for rules about the content being version controlled. If we
> are worried about a user version controlling something that interferes
> with a CI/CD pipeline, that seems to imply our current authorization
> policy model needs improvement, or it is not being used correctly
> (i.e. why does a user have write to a bucket they shouldn't be writing
> to?).
>
> On Wed, Jan 17, 2024 at 8:53 AM Pierre Villard
>  wrote:
> >
> > I do think this would be a good idea. That would provide users of the
> NiFi
> > Registry a way to implement custom conditions on what can be versioned or
> > not based on their requirements. Right now if one has write permissions
> for
> > a bucket/flow, you could commit anything as a new version which could
> > potentially be an issue when using CI/CD pipelines to automate
> deployments.
> >
> > As a side-note, another improvement could be to prevent a flow from being
> > versioned in the registry if a required rule is violated (I'm talking
> about
> > the new feature coming with NiFi 2.0). Just a thought for an additional
> > improvement.
> >
> > Overall I do think that giving options to users for better controlling
> what
> > is being versioned is good.
> >
> > Le mer. 17 janv. 2024 à 17:13, Simon Bence  a
> > écrit :
> >
> > > Hi,
> > >
> > > I recently found the need for custom validation to maintain NiFi
> Registry
> > > content. This includes checks such as enforcing naming conventions when
> > > creating a Bucket and similar usage specific cases. While exploring the
> > > Registry's codebase, I came across the EventHookProvider, which aligns
> with
> > > a similar concept. However, it does not cover the case here due to its
> > > asynchronous nature and being a "post-event" activity.
> > >
> > > Although the EventHookProvider is not suitable for this specific need,
> I
> > > find the Event construct and the "whitelist" concept pretty overlapping
> > > with my objectives. Consequently, I propose the addition of a new type
> of
> > > Provider covering for "pre-event" validation, operating in a manner
> similar
> > > to the EventHookProvider: a call from the request methods to the set of
> > > providers but filtered using a whitelist. Similarly to the mentioned
> > > provider, I believe an implementation capable of executing scripts
> (akin to
> > > ScriptEventHookProvider) would be a good starting point, to cover a
> most
> > > use cases.
> > >
> > > I am keen to hear your opinion on this proposal and welcome any further
> > > ideas. Thank you for your consideration!
> > >
> > > PS.: using the "event" term comes from the already existing
> > > EventHookProvider. In practice these are the methods of the Registry
> web
> > > API.
> > >
> > > Regards,
> > > Bence Simon
>

Re: [DISCUSS] Validator hook for Registry events

[Pierre Villard]

I do think this would be a good idea. That would provide users of the NiFi
Registry a way to implement custom conditions on what can be versioned or
not based on their requirements. Right now if one has write permissions for
a bucket/flow, you could commit anything as a new version which could
potentially be an issue when using CI/CD pipelines to automate deployments.

As a side-note, another improvement could be to prevent a flow from being
versioned in the registry if a required rule is violated (I'm talking about
the new feature coming with NiFi 2.0). Just a thought for an additional
improvement.

Overall I do think that giving options to users for better controlling what
is being versioned is good.

Le mer. 17 janv. 2024 à 17:13, Simon Bence  a
écrit :

> Hi,
>
> I recently found the need for custom validation to maintain NiFi Registry
> content. This includes checks such as enforcing naming conventions when
> creating a Bucket and similar usage specific cases. While exploring the
> Registry's codebase, I came across the EventHookProvider, which aligns with
> a similar concept. However, it does not cover the case here due to its
> asynchronous nature and being a "post-event" activity.
>
> Although the EventHookProvider is not suitable for this specific need, I
> find the Event construct and the "whitelist" concept pretty overlapping
> with my objectives. Consequently, I propose the addition of a new type of
> Provider covering for "pre-event" validation, operating in a manner similar
> to the EventHookProvider: a call from the request methods to the set of
> providers but filtered using a whitelist. Similarly to the mentioned
> provider, I believe an implementation capable of executing scripts (akin to
> ScriptEventHookProvider) would be a good starting point, to cover a most
> use cases.
>
> I am keen to hear your opinion on this proposal and welcome any further
> ideas. Thank you for your consideration!
>
> PS.: using the "event" term comes from the already existing
> EventHookProvider. In practice these are the methods of the Registry web
> API.
>
> Regards,
> Bence Simon

Re: [DISCUSS] Preparing for NiFi 2.0.0-M2

[Pierre Villard]

Sounds good to me David. As discussed on another thread, I am happy to take
care of a 1.25 release as well to have some of the fixes we added there for
migrating to 2.0.

Thanks,
Pierre

Le mar. 16 janv. 2024 à 00:44, Joe Witt  a écrit :

> Wonderful progress. Definitely time and an M2 seems like a good idea for
> the reasons noted.
>
> Happy to help take RM if you get squeezed on time.
>
> Thanks
>
> On Mon, Jan 15, 2024 at 12:46 PM David Handermann <
> exceptionfact...@apache.org> wrote:
>
> > Team,
> >
> > We have had some great feedback and many improvements since the
> > release of NiFi 2.0.0-M1 at the end of November. With over 160 [1]
> > Jira issues already resolved for the next iteration, we are in a good
> > position to prepare for another milestone release version.
> >
> > The main branch now includes significant framework dependency upgrades
> > such as Spring 6 and Jetty 12, along with several new components and
> > other bug fixes. Considering the significant number of changes to
> > framework libraries already in place, it would be useful to release a
> > second milestone version before a full general availability release.
> >
> > There is a pull request for NIFI-9458 [2] that includes impactful
> > lower-level changes to date and time parsing and formatting, which is
> > part of the NiFi 2.0 Release Goals [3]. Following the review and
> > incorporation of these changes, I would be glad to handle Release
> > Manager responsibilities for a NiFi 2.0.0-M2 version, ideally at some
> > point this week.
> >
> > Regards,
> > David Handermann
> >
> > [1] https://issues.apache.org/jira/projects/NIFI/versions/12353861
> > [2] https://github.com/apache/nifi/pull/8248
> > [3]
> > https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals
> >
>

Re: Subject: Issue with ExecuteScript Processor in NiFi - Unable to Select Python Script Engine

[Pierre Villard]

Hi,

The jython engine has been completely removed in NiFi 2.0 in favor of the
new Python API allowing you to build NiFi components in Python. Only
Clojure and Groovy are supported in NiFi 2.x.

Thanks,
Pierre

Le sam. 13 janv. 2024 à 18:47, Michael Byron 
a écrit :

> Dear Apache NiFi Developers,
>
> I hope this email finds you well. I am writing to bring to your attention
> an issue I am encountering while working with the ExecuteScript processor
> in Apache NiFi. I have outlined the problem and the steps I have taken to
> address it below:
>
> Problem Description:
>
> I am trying to use the ExecuteScript processor.
> I commented out the Python extension lines in the nifi.properties source
> file as I do not want to use Python as an extension language.
> When I ran the run-nifi batch file, I encountered an error.
> Attempted Solutions:
>
> I noticed that I do not have a "bin" folder in my work repository.
> To address this, I created a "bin" folder and placed my Python executable
> in the "scripts" folder. I updated the link to the Python executable in the
> nifi.properties file.
> Despite these changes, the issue persisted.
> I also changed "python3" to "python" in the nifi.properties file.
> Subsequently, I set up the "bin" folder, and now NiFi launches
> successfully.
> Current Challenge:
>
> However, even though NiFi launches without errors, I am unable to select
> Python as the script engine in the ExecuteScript processor. The available
> options are limited to Clojure and Groovy.
> Request for Assistance:
>
> I would appreciate any guidance or assistance you can provide in resolving
> this issue. I have reviewed the documentation and community forums, but I
> have not found a solution to this specific problem.
> System Information:
>
> Apache NiFi Version: 2.0.0 M1 standard
> Operating System: Windows
> I appreciate your time and support in addressing this matter. If you
> require any additional information or logs, please let me know. Thank you
> in advance for your assistance.
>
> Best regards,
> Michael
>
>
>

Re: NiFi 1.24.1 or 1.25.0 release date? - NIFI-12513 bug

[Pierre Villard]

Hi Martin,

I do think that having a NiFi 1.25 release soon would make sense. It would
be nice to have a release that contains
https://issues.apache.org/jira/browse/NIFI-12524 which can also be seen as
an annoyance to upgrade to 2.0-M1.

I'm happy to take care of the RM duties if there is a consensus for a
release. So I guess this could happen in the next few weeks. I think we're
also getting close to the point where we would have 2.0-M2 so we'll
probably do that around the same time like we did with 1.24 and 2.0-M1.

Thanks,
Pierre

Le jeu. 11 janv. 2024 à 22:18, Martin Fong  a
écrit :

> Howdy,
>
> We are trying to migrate 1.19.1 to 1.24.0 as a pre-req to jump to 2.0.0-M1
> later on.
>
> It seems there is a roadblock of "URL properties specified in components
> should be URL-encoded."  We are trying to migrate as smoothly as possible
> without making many changes.
>
> NIFI-12513 seems to be fixed with 1.25.0/2.0.0
>
> Is there a timeline for a fix as 1.24.1 or 1.25.0?
>
> Please advise,
> Martin Fong
> Enterprise Technical Support Specialist, Infrastructure & Platform (IAG)
> Technology Services Division, Technology Infrastructure Services
> City of Toronto
> 703 Don Mills Road, 2nd Floor
> Toronto, ON
> M3C 3N3
> Tel:   416-397-7565
> e-mail: martin.f...@toronto.ca
>
> This e-mail message is confidential and subject to copyright. Any
> unauthorized use or disclosure is prohibited. If you have received this
> email and are not the intended recipient, please advise and delete it.
> Thank you.
>
>

Re: New Project Website Design Staged for Deployment

[Pierre Villard]

This is great, thanks to all the people contributing to this, I know it's a
significant amount of work and it's amazing to see this getting real!



Le jeu. 4 janv. 2024 à 09:57, Lucas Ottersbach 
a écrit :

> The new site looks refreshing.
>
> Thank you James, David and all others contributing to this.
>
> While browsing the site on mobile I noticed a small area of future
> improvement. I added more details to the GitHub issue linked by David.
> Maybe that's something we want to improve on in the future. However, that
> shouldn't keep us from releasing the updated site.
>
> Joe Witt  schrieb am Do., 4. Jan. 2024, 05:00:
>
> > Super appreciative David and James.  A good step forward to refresh and
> > retool and grow from.
> >
> > Thanks!
> >
> > On Wed, Jan 3, 2024 at 8:39 PM David Handermann <
> > exceptionfact...@apache.org>
> > wrote:
> >
> > > Team,
> > >
> > > Thanks to background work from several designers, and significant
> > > recent effort from James Mingardi-Elliott [1], we have a refreshed
> > > project website design deployed to staging and ready for production.
> > >
> > > The combined set of changes can be reviewed in the following pull
> > request:
> > >
> > > https://github.com/apache/nifi-site/pull/82
> > >
> > > The staging site provides a functional way to view the new design,
> > > based on the main-staging branch [2] of the nifi-site repository.
> > >
> > > https://nifi.staged.apache.org
> > >
> > > The new design includes a modernized home page and streamlined
> > > navigation, with a focus on making the most popular pages quickly
> > > accessible.
> > >
> > > There is room for improvement in areas like generated project
> > > documentation, but the primary goal of this new design is to provide a
> > > fresh foundation for future improvements.
> > >
> > > The pull request provides an opportunity for correcting any notable
> > > problems, with the goal of pushing the new version to production early
> > > next week.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > [1] https://github.com/james-elliott
> > > [2] https://github.com/apache/nifi-site/tree/main-staging
> > >
> >
>

Re: [DISCUSS] State Management improvements for DR scenarios

[Pierre Villard]

>
> Even with an option for deterministic state identifiers, I am still
> concerned about a composite state manager provider. It seems prone to
> getting providers out of sync, as either backing state store could fail,
> resulting in inconsistent state across regions. This is the primary reason
> why it seems better to evaluate a resilient storage solution instead of a
> composite solution.
>

I 100% agree with this statement. And my preference would be to have
implementations that leverage solutions where cross-region replication is
already available.

Supporting customization of the state identifier at the component seems
> like it gets complicated at the user level. Bryan's suggestion about a
> Process Group setting would be helpful along these lines. Taking that one
> step further, if this were a configurable mode, deterministic UUID version
> 5 could be used, which could base the state identifier on the Process Group
> Name and Component Name. That would yield an identifier that would be the
> same across flows without component level configuration. It still raises
> questions about uniqueness, but as a Process Group setting or even
> application property, it is much less complex for a user.
>

Having a deterministic UUID based on PG ID and name would be awesome but I
share the same concern around the unicity. I would not be surprised if a
single process group contains multiple instances of the same component type
that requires a state and where the user is not changing the default name.

I'm not sure I completely follow the suggestion by Bryan. Is the below what
we're suggesting?
- We would have a custom ID at the process group level that would have to
be set by the user when starting version control OR when checking out a
versioned flow? (this would not be part of what is versioned and could
default to the UUID of the process group and only exposed in an "advanced
section" to not confuse normal users)
- If this is set, then the state ID would be something like a deterministic
UUID made of this custom ID + versioned ID of the component

Still trying to wrap my head around this in case the same versioned flow is
instantiated multiple times in both regions.

Le lun. 11 déc. 2023 à 22:17, David Handermann 
a écrit :

> Pierre,
>
> Thanks for the helpful reply.
>
> I am not concerned about complexity where necessary, so the main issue is
> about the complexity to support custom state identifiers at the framework
> level, versus a resilient state management solution.
>
> Supporting customization of the state identifier at the component seems
> like it gets complicated at the user level. Bryan's suggestion about a
> Process Group setting would be helpful along these lines. Taking that one
> step further, if this were a configurable mode, deterministic UUID version
> 5 could be used, which could base the state identifier on the Process Group
> Name and Component Name. That would yield an identifier that would be the
> same across flows without component level configuration. It still raises
> questions about uniqueness, but as a Process Group setting or even
> application property, it is much less complex for a user.
>
> Even with an option for deterministic state identifiers, I am still
> concerned about a composite state manager provider. It seems prone to
> getting providers out of sync, as either backing state store could fail,
> resulting in inconsistent state across regions. This is the primary reason
> why it seems better to evaluate a resilient storage solution instead of a
> composite solution.
>
> Regards,
> David Handermann
>
> On Mon, Dec 11, 2023, 6:23 AM Pierre Villard 
> wrote:
>
> > Hi David and Bryan, thanks for the feedback.
> >
> > David,
> >
> > Regarding the source and destination. It was just an example. There are
> > plenty of different use cases you can think of. Of course, this is
> assuming
> > that the user of NiFi has a source and destination which are highly
> > available but this is the user's responsibility. You can take the example
> > of ListSFTP -> FetchSFTP -> doSomething -> PutS3.
> >
> > I definitely agree that having a state manager that supports cross region
> > replication would be ideal (and IIRC Redis does support this). The
> approach
> > of leader/follower makes it easier from a user point of view but I'm
> > definitely fine considering state manager implementations - for example,
> in
> > the past, I suggested a database based implementation (Postgres would be
> a
> > good candidate for example).
> >
> > When you say it's adding a layer of complexity, it definitely does but
> only
> > for users that are looking for options when it comes to 

Re: [DISCUSS] State Management improvements for DR scenarios

[Pierre Villard]

topic worth considering, but I am not
> > confident that the implementation steps outlined in the initial two
> > Jira issues will provide a robust or maintainable solution. Supporting
> > component-level configuration of a custom state identifier seems prone
> > to error, and also requires a lot of manual configuration at the
> > individual Processor level. Supporting a composite state management
> > could have other benefits, but it also adds a layer of complexity that
> > may not even achieve the desired outcome, depending on the
> > capabilities of the underlying storage implementations.
> >
> > With that background, I think it would be worth evaluating alternative
> > approaches before moving to any kind of implementation. I'm sure there
> > are aspects I have not considered, so I welcome additional perspective
> > on the positives and negatives of the proposed solution.
> >
> > Regards,
> > David Handermann
> >
> > On Fri, Dec 8, 2023 at 8:32 AM Pierre Villard
> >  wrote:
> > >
> > > Team,
> > >
> > > I just published a feature proposal here:
> > >
> https://cwiki.apache.org/confluence/display/NIFI/State+Management+improvements+for+Disaster+Recovery+scenarios
> > >
> > > This feature proposal is to provide a more detailed explanation around
> the
> > > two below JIRAs:
> > > https://issues.apache.org/jira/browse/NIFI-11776
> > > https://issues.apache.org/jira/browse/NIFI-11777
> > >
> > > I'd love to hear your thoughts before we get started with the actual
> > > implementation.
> > >
> > > Thanks,
> > > Pierre
>

[DISCUSS] State Management improvements for DR scenarios

[Pierre Villard]

Team,

I just published a feature proposal here:
https://cwiki.apache.org/confluence/display/NIFI/State+Management+improvements+for+Disaster+Recovery+scenarios

This feature proposal is to provide a more detailed explanation around the
two below JIRAs:
https://issues.apache.org/jira/browse/NIFI-11776
https://issues.apache.org/jira/browse/NIFI-11777

I'd love to hear your thoughts before we get started with the actual
implementation.

Thanks,
Pierre

[ANNOUNCE] Apache NiFi 1.24.0 Released

[Pierre Villard]

The Apache NiFi Team is pleased to announce the release of Apache NiFi
1.24.0.

Apache NiFi is an easy to use, powerful, and reliable system to process and
distribute data.

https://nifi.apache.org

The release artifacts can be downloaded from the project website.

https://nifi.apache.org/download.html

Maven artifacts have been released and mirrored according to ASF artifact
distribution processes.

Issues resolved in Apache NiFi 1.24.0 are listed in Jira Release Notes.

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443

Highlights of the release are available on the project wiki page:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0

Thank you,
The Apache NiFi Team

[RESULT][VOTE] Release Apache NiFi 1.24.0-RC5

[Pierre Villard]

Apache NiFi Community,

I am pleased to announce that the 1.24.0 release of Apache NiFi passes:

8 +1 (binding) votes
5 +1 (non-binding) votes
0 0 votes
0 -1 votes

Thanks to all who helped make this release possible!

Here is the vote thread:
https://lists.apache.org/thread/7cjmddzglpoz1yp40nm84761bb20z23q

Re: [VOTE] Release Apache NiFi 1.24.0 (RC5)

[Pierre Villard]

+1 (binding)

Thanks everyone, I will now proceed with the release.

Pierre

Le dim. 26 nov. 2023 à 14:08, Peter Turcsanyi  a
écrit :

> +1 (binding)
>
> Verified signatures and hashes.
> Built NiFi on Ubuntu 20.04 with Java 8 (Adoptium 1.8.0_392-b08), Java 11
> (Adoptium 11.0.21+9) and Java 17 (Adoptium 17.0.9+9), Maven 3.9.5 with
> empty local repo.
>
> Ran flows for validating:
>  - NIFI-12151 StandardPrivateKeyService fails due to missing
> BouncyCastleProvider
>  - NIFI-12364 Upgrade snowflake-ingest-sdk to 2.0.4 and snowflake-jdbc to
> 3.14.3
>  - NIFI-12271 PuAzureBlobStorage with FileResourceService - FileNotFound
> should not rollback
>  - NIFI-11987 PutAzureBlobStorage_v12 using FileResourceService may fail
> with OutOfMemoryError
>  - NIFI-11926 Add proxy handling in Azure Storage Credentials Services
>  - NIFI-6240 Proxy Support for AzureEventHub processor via Websockets
>  - NIFI-11918 ListenGRPC with SSL fails on Java 17
>  - NIFI-12373 Add License and Notice to nifi-standard-shared-nar
>
> Found an issue with NIFI-6240 (Proxy Support for AzureEventHub processor
> via Websockets) but it is not a deal breaker. Filed NIFI-12412
> (ConsumeAzureEventHub should support Blob checkpoint store access via
> proxy) for fixing it.
>
> Thanks for RMing Pierre!
>
> Regards,
> Peter Turcsanyi
>
> On Sun, Nov 26, 2023 at 7:51 AM Eric Secules  wrote:
>
> > +1 (non-binding)
> >
> > Built on linux Mint:
> > Apache Maven 3.6.3
> > Maven home: /usr/share/maven
> > Java version: 17.0.8.1, vendor: Private Build, runtime:
> > /usr/lib/jvm/java-17-openjdk-amd64
> > Default locale: en_CA, platform encoding: UTF-8
> > OS name: "linux", version: "5.15.0-89-generic", arch: "amd64", family:
> > "unix"
> >
> > - verified signatures and hashes
> > - verified git commit
> > - built sources with contrib-check
> > - ran nifi and ran a couple sample flows
> >
> > Thanks Pierre!
> >
> > -Eric
> >
> > On Sat, Nov 25, 2023 at 1:38 PM Márk Báthori 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > Built on:
> > > - OSX 13.4
> > > - Java version: Azul Zulu version 11.0.18
> > > - Apache Maven 3.8.6
> > >
> > > - Went through the helper guide and did a full clean build with contrib
> > > check
> > > - Verified signatures and hashes
> > >
> > > Tested various simple flows successfully.
> > >
> > > Verified the following tickets:
> > > - [NIFI-11177] - PutIceberg cannot write null Timestamp into Iceberg
> > table
> > > - [NIFI-12130] - PutIceberg: Ability to configure snapshot properties
> via
> > > dynamic attributes
> > > - [NIFI-11739] - Add ability to ignore missing fields in PutIceberg
> > > - [NIFI-12054] - PutIceberg should produce a provenance send event
> > >
> > > Thank you for RMing Pierre!
> > >
> > > Regards,
> > > Mark
> > >
> > > David Handermann  ezt írta (időpont:
> 2023.
> > > nov. 25., Szo, 18:32):
> > >
> > > > +1 binding
> > > >
> > > > - Verified signatures and hashes
> > > > - Ran build using Maven Wrapper 3.9.5
> > > > - Ran build on macOS 14.1.1 with Azul Zulu JDK 1.8.0-392 AArch64
> > > > - Ran build on macOS 14.1.1 with Azul Zulu JDK 17.0.9 AArch64
> > > > - Ran build on Ubuntu 22.04 with Azul Zulu JDK 17.0.9 x86_64
> > > >
> > > > - Ran NiFi on macOS 14.1.1 with Azul Zulu JDK 17.0.9 AArch64
> > > > - Verified NiFi OpenID Connect integration with Okta
> > > >
> > > > - NIFI-1874 Verified IdentityMimeType character set detection
> > > > - NIFI-11303 Verified go to component link from Provenance view
> > > > - NIFI-11874 Verified updated layout of Process Group settings
> > > > - NIFI-12033 Verified EncryptContentAge and DecryptContentAge
> > Processors
> > > > - NIFI-12093 Verified EncryptContent Processor marked as deprecated
> > > > - NIFI-12115 Verified ListenOTLP receiving Logs and Traces over
> HTTP/2
> > as
> > > > JSON
> > > > - NIFI-12153 Verified Max String Length configurable in JSON Tree
> > Reader
> > > > - NIFI-12219 Verified Flow Configuration History search and filter
> > > > - NIFI-12239 Verified system information printed to
> nifi-bootstrap.log
> > > > - NIFI-12287 Verified Sources and Java Documentation JAR binaries
> > > > skipped for NAR packages
> > >

Re: [VOTE] Release Apache NiFi 2.0.0-M1 (RC6)

[Pierre Villard]

+1 (binding)

Went through the usual steps. Everything LGTM.
Very excited by this first milestone for NiFi 2.0!

Thanks David for RM'ing!

Le ven. 24 nov. 2023 à 17:37, Csaba Bejan  a écrit :

> +1 (binding)
>
> - Went through the helper guide and did a clean build
> - Verified signatures and hashes
> - Built on OSX 14.1.1
> - Java version: Zulu21.28+85-CA (build 21+35)
> - Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
>
> - Started NiFi and created a simple flow
> - Started MiNiFi and verified integration with C2 Server. Played around
> with the C2 protocol (Operations - Update, Transfer, Describe...)
>
> Verified:
> - NIFI-11514 MiNiFi Flow JSON support and deprecating YAML format.
> Migration tool from YAML to JSON
> - NIFI-12333 MiNiFi property override through env and system variables
> - NIFI-12335 Fix MiNiFi startup issue
>
> Thanks for RMing, David!
>
> Regards,
> Csaba
>
> > On 24 Nov 2023, at 15:12, Márk Báthori  wrote:
> >
> > +1 (non-binding)
> >
> > Built on:
> > - OSX 13.4
> > - Java version: Azul Zulu version 1.8.0_332
> > - Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
> >
> > - Went through the helper guide and did a full clean build with contrib
> > check
> > - Verified signatures and hashes
> >
> > Tested various simple flows successfully.
> >
> > Verified the following tickets:
> > - [NIFI-9206] - Create a processor that is capable of removing fields
> from
> > records
> > - [NIFI-12130] - PutIceberg: Ability to configure snapshot properties via
> > dynamic attributes
> > - [NIFI-11263] - Add case-insensitive and order independent field mapping
> > to PutIceberg record converter
> > - [NIFI-12088] - UI - Dependent Properties are not shown when depending
> on
> > newly created Service
> > - [NIFI-11342] - HDFS processors fail to get ClassloaderIsolationKey at
> > startup
> > - [NIFI-11177] - PutIceberg cannot write null Timestamp into Iceberg
> table
> > - [NIFI-11215] - Add custom validation for KerberosUserService in
> PutIceberg
> >
> > Thank you for RMing David!
> >
> > Regards,
> > Mark
> >
> > Ferenc Kis  ezt írta (időpont: 2023. nov. 24.,
> P,
> > 11:31):
> >
> >> +1 (non-binding)
> >>
> >> Went through the helper guide, local maven repo cleaned up, full clean
> >> build with contrib check, verified signatures and hashes
> >>
> >> Apache Maven 3.9.0 (9b58d2bad23a66be161c4664ef21ce219c2c8584)
> >> Maven home: /Users/fkis/.sdkman/candidates/maven/current
> >> Java version: 21, vendor: Azul Systems, Inc., runtime:
> >> /Users/fkis/.sdkman/candidates/java/21-zulu/zulu-21.jdk/Contents/Home
> >> Default locale: en_HU, platform encoding: UTF-8
> >> OS name: "mac os x", version: "13.4.1", arch: "aarch64", family: "mac"
> >>
> >> Validations performed:
> >> - Started NiFi, created a simple flow with ListenHTTP and Input Port.
> >> Validated ListenHTTP processor is able to receive data
> >> - Started MiNiFi Java:
> >>  * created a simple GenerateFlowFile -> InvokeHttp flow and
> >> GenerateFlowFile -> RemoteProcessGroup flow and pushed to MiNiFi via C2
> >> protocol. Validated that connectivity between NiFi and MiNiFi works via
> >> both InvokeHTTP and S2S
> >>
> >> Thanks for RMing David!
> >>
> >> Regards
> >> Ferenc
> >>
> >> On Fri, Nov 24, 2023 at 1:05 AM Dan S  wrote:
> >>
> >>> +1 (non-binding)
> >>>
> >>> - Verified binary hashes and signatures
> >>> - Successfully built NiFi with contrib-check using:
> >>>  - Apache Maven 3.9.5
> >>>  - Java 21 Oracle Corporation
> >>>  - Centos 7 Linux 3.10.0-1160.90.1.el7.x86_64
> >>>
> >>> Exercised the following bug fixes and features
> >>> NIFI-2964 Confirmed the ability to allow attributes already JSON to
> >> remain
> >>> JSON and not be escaped when the Nested strategy is chosen. Created
> >>> NIFI-12408  to
> improve
> >>> AttributesToJson with an option to pretty print the resulting JSON when
> >> the
> >>> destination is flowfile content.
> >>> NIFI-11156 Ensured the 'validatexml.invalid.error' contains an error
> >>> message whether an XSD is specified or not.
> >>> NIFI-11959 Confirmed Single comment allowed.
> >>> NIFI-12165 Confirmed  "Custom Transformation Class Name" and "Custom
> >> Module
> >>> Directory" no longer vanish when configuring JoltTransformJSON and
> >>> JoltTransformRecord.
> >>> NIFI-11167 Exercised Excel Record Reader by running sample Excel files
> >>> converting them to CSV.
> >>> NIFI-11197 Exercised Yaml Record Reader Ran by running sample Yaml
> files
> >>> converting them to JSON and in the process discovered that Yaml aliases
> >> are
> >>> not resolved. Created NIFI-12404
> >>>  to at least
> document
> >>> that YamlTreeReader does not support resolving Yaml aliases.
> >>> NIFI-11509 Confirmed source JSON can have comments.
> >>> NIFI-12100 Confirmed removal of ConvertExcelToCSVProcessor and that it
> no
> >>> longer is in the documentation.
> >>> NIFI-12392 Confirmed those pro

[VOTE] Release Apache NiFi 1.24.0 (RC5)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.24.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1251

Git Tag: nifi-1.24.0-RC5
Git Commit ID: 5241f434829ca46a26a475600ef7c00e1e271e37
GitHub Commit Link:
https://github.com/apache/nifi/commit/5241f434829ca46a26a475600ef7c00e1e271e37

Checksums of nifi-1.24.0-source-release.zip

SHA256: 715eb61cdc017a5f7ba4d845ae962fdf83d389829db2a8948be14f99f2cc8272
SHA512:
574147002b905ef64447edec0c7308f4fc3a97b3c7f01edca05464b2b418bcd3922f956d093736014443eec88ceba36af81df37398c5fe23a1288235b79d7306

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 285

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.24.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [VOTE] Release Apache NiFi 1.24.0 (RC4)

[Pierre Villard]

Team,

Appreciate everyone's effort to verify this release candidate (and the
previous ones!). Unfortunately, I have to cancel this release candidate to
include NIFI-12403. Everything else will remain the same so I think that
for people who already tested previous RCs, just verifying the hashes,
signatures, etc, of the upcoming release candidate would be enough. Thanks
to everyone helping with this release process, I definitely didn't
anticipate so many release candidates!

Thanks, Pierre


Le jeu. 23 nov. 2023 à 02:40, Mark Bean  a écrit :

> +1 (non-binding)
>
> Verified signature and hashes
> Built source code with Java 8 and 11 on Ubuntu 22.04.3 LTS
>   openjdk version "1.8.0_382"
>   openjdk version "11.0.20.1" 2023-08-24
>
> Installed NiFI Registry as an upgrade from an older Registry more than one
> version away (1.18.0)
> Installed NiFi as an upgrade from an older NiFI more than one version away
> (1.18.0)
>
> Both applications started without issue
> NiFi tested in single-instance, non-cluster mode using managed-authorizer
> and keystore/truststore generated from NiFi Toolkit. No issues discovered.
> Verified basic functionality with several versioned flows from Registry and
> export/import flows from JSON.
>
> [NIFI-11877] Add a comments field for UpdateAttribute Rules
> [NIFI-11914] Allow expression language support in SegmentSize property of
> SegmentContent
> [NIFI-11934] InvokeHTTP does not send filename attribute
> [NIFI-11874] Reorganize UI presentation of Process Group configuration
> [NIFI-12392] Additional Details Documentation Not Linked
>
> Thanks Pierre for RM'ing and handling the multiple RCs. Thanks to all for
> evaluating the several RCs especially as we're busy with many of us heading
> into a (US) holiday. This one is looking good!
>
> -Mark
>
> On Wed, Nov 22, 2023 at 2:53 PM Michael Moser  wrote:
>
> > +1 (binding)
> >
> > Built using maven wrapper on CentOS 7.9 with:
> > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
> > Maven home: /home/mosermw/.m2/wrapper/dists/apache-maven-3.9.5/2021cb71
> > Java version: 1.8.0_382, vendor: Red Hat, Inc., runtime:
> > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.382.b05-1.el7_9.x86_64/jre
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "linux", version: "3.10.0-1160.95.1.el7.x86_64", arch: "amd64",
> > family: "unix"
> >
> > verified signature and hashes
> > verified git commit and tag
> > tested NiFi and NiFi Registry versioning flows
> > tested various flows, including verifying:
> > [NIFI-12038] - PackageFlowFile
> > [NIFI-11874] - UI presentation of Process Group configuration
> > [NIFI-11934] - InvokeHTTP does not send filename attribute
> > [NIFI-10904] - Changing Font Color for Dropdown Menu When Selecting
> > Controller Services
> >
> > Thank you Pierre for doing the RM work, and thanks to the community for
> > supporting both main and 1.x branches.
> >
> > -- Mike
> >
> >
> > On Wed, Nov 22, 2023 at 10:13 AM Márk Báthori 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > Built on:
> > > - OSX 13.4
> > > - Java version: Azul Zulu version 1.8.0_332
> > > - Apache Maven 3.8.6
> > >
> > > - Went through the helper guide and did a clean build
> > > - Verified signatures and hashes
> > >
> > > Tested various simple flows successfully.
> > >
> > > Thanks for RMing, Pierre!
> > >
> > > Regards,
> > > Mark
> > >
> > > Ferenc Erdei  ezt írta (időpont: 2023. nov.
> > 22.,
> > > Sze, 15:35):
> > >
> > > > +1 (non-binding)
> > > >
> > > > - Went through the helper guide and did a clean build
> > > > - Verified signatures and hashes
> > > > - Built on
> > > > Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
> > > > Java version: 1.8.0_392, vendor: Azul Systems, Inc.
> > > > OS name: "mac os x", version: "14.1.1", arch: "aarch64", family:
> "mac"
> > > >
> > > > Verified the following:
> > > > MINiFi C2 protocol features like:
> > > > - Heartbeat / Acknowledge
> > > > - Update Asset
> > > > - Transfer Debug
> > > > - Property Update
> > > > - Flow Update
> > > >
> > > > Ran simple flow in MiNiFi with parameter contexts + in NiFi
> > > > Also verified S2S connection between MiNiFi and NiFi 1.24.0
> > &

[VOTE] Release Apache NiFi 1.24.0 (RC4)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.24.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1247

Git Tag: nifi-1.24.0-RC4
Git Commit ID: 6029cd00c032c0596f0d4211c206762895de120b
GitHub Commit Link:
https://github.com/apache/nifi/commit/6029cd00c032c0596f0d4211c206762895de120b

Checksums of nifi-1.24.0-source-release.zip

SHA256: 1c2d7d94abf7586cc014cea041852796882bd41ced449e950481a0033e8e3abc
SHA512:
121cd42b61100e170cd2f3716077aaf099d1e2759355432d949239e6ccd1ee25aa6b06d306a4bc1318f9717b838687127594d223b448cae994680a23d2a2f04c

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 283

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.24.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [VOTE] Release Apache NiFi 1.24.0 (RC3)

[Pierre Villard]

Thanks all for spending time verifying this RC. Given the issue around
additional details, I'll kill this RC and start a new one. The additional
details feature of our documentation is an important piece for all of our
users and I personally think it's worth starting another RC...

Le lun. 20 nov. 2023 à 05:51, David Handermann 
a écrit :

> Drew,
>
> Thanks for finding this problem, and thanks Chris for the confirmation.
>
> I was able to reproduce the problem and filed NIFI-12392 [1] for
> tracking. I also submitted GitHub PR 8052 [2] to resolve the issue,
> which can be applied to both the main and support branches. The pull
> request introduces additional unit tests for the NarUnpacker, which
> was the source of the problem as a result of recent changes for
> NIFI-12294 [3].
>
> Regards,
> David Handermann
>
> [1] https://issues.apache.org/jira/browse/NIFI-12392
> [2] https://github.com/apache/nifi/pull/8052
> [3] https://issues.apache.org/jira/browse/NIFI-12294
>
> On Sun, Nov 19, 2023 at 2:35 AM Chris Sampson
>  wrote:
> >
> > Drew,
> >
> > Good spot. I’ve removed my local install of 1.24.0 RC now in favour of
> 2.0.0-M1 RC release testing, but can confirm that the same problem exists
> there and it’s not limited to the Elasticsearch components, e.g.
> UpdateRecord is also missing the additionalDetails link. So it looks like
> there’s a general documentation issue that has crept in to both release
> branches.
> >
> > This is probably enough to warrant further investigation and inclusion
> in the releases through another set of RC votes.
> >
> > I wonder whether there’s a set of tests that could be added (maybe to
> the assembly builds) that extract the constructed binaries and check for
> LICE/NOTICE files along with additionalDetails.html files being present
> (probably a separate ticket to fixing the immediate documentation issue)?
> >
> >
> > Cheers,
> >
> > ---
> > Chris Sampson
> >
> >
> > > On 19 Nov 2023, at 04:07, Andrew Lim 
> wrote:
> > >
> > > This might just be user error on my part, but I don’t see the
> “Additional Details” link for processors, controller services, etc. in the
> embedded documentation.
> > >
> > > I successfully ran a full clean install on macOS (Ventura 13.0.1,
> OpenJDK version 1.8.0_372).  I wanted to verify the Documentation changes
> in 1.24.0 and started with
> https://issues.apache.org/jira/browse/NIFI-12322.
> > >
> > > But I don’t see the “Additional Details” link in the 1.24.0 embedded
> doc for this controller service, like I do here in 1.23.0:
> > >
> > >
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-client-service-nar/1.23.2/org.apache.nifi.elasticsearch.ElasticSearchClientServiceImpl/index.html
> > >
> > > I confirmed that in the path
> nifi-1.24.0/work/docs/components/org.apache.nifi/nifi-elasticsearch-client-service-nar/1.24.0
> the additionalDetails.html file doesn’t exist.
> > >
> > > Is anyone else seeing the same thing?
> > >
> > > Thanks,
> > > Drew
> > >
> > >
> > >
> > >> On Nov 16, 2023, at 2:00 PM, Pierre Villard <
> pierre.villard...@gmail.com> wrote:
> > >>
> > >> Team,
> > >>
> > >> I am pleased to be calling this vote for the source release of Apache
> NiFi
> > >> 1.24.0.
> > >>
> > >> Please review the following guide for how to verify a release
> candidate
> > >> build:
> > >>
> > >>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> > >>
> > >> The source being voted on the and the convenience binaries are
> available on
> > >> the Apache Distribution Repository:
> > >>
> > >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0
> > >>
> > >> The build artifacts are available on the Apache Nexus Repository:
> > >>
> > >> https://repository.apache.org/content/repositories/orgapachenifi-1241
> > >>
> > >> Git Tag: nifi-1.24.0-RC3
> > >> Git Commit ID: 586a1a8789e9c39914f4a6088ac26e22d60eeb91
> > >> GitHub Commit Link:
> > >>
> https://github.com/apache/nifi/commit/586a1a8789e9c39914f4a6088ac26e22d60eeb91
> > >>
> > >> Checksums of nifi-1.24.0-source-release.zip
> > >>
> > >> SHA256:
> 590cf9b1219f9fd66c81cc1740b2e245d85e341cdc280c769d354084dc27ee8a
> > >> SHA512:
> > >>
&

[VOTE] Release Apache NiFi 1.24.0 (RC3)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.24.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1241

Git Tag: nifi-1.24.0-RC3
Git Commit ID: 586a1a8789e9c39914f4a6088ac26e22d60eeb91
GitHub Commit Link:
https://github.com/apache/nifi/commit/586a1a8789e9c39914f4a6088ac26e22d60eeb91

Checksums of nifi-1.24.0-source-release.zip

SHA256: 590cf9b1219f9fd66c81cc1740b2e245d85e341cdc280c769d354084dc27ee8a
SHA512:
8d3b9cb9c4686242620ad40ad83fadb972403ee70a101cbb6fa0116b54ad7793702da230871281c0de40322ddfdbfc89dacba7b690465e7b2329850dca5132e8

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 280

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.24.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [VOTE] Release Apache NiFi 1.24.0 (RC2)

[Pierre Villard]

Killing this RC2 in light of the license/notice issue.
Will include a few additional commits in the RC3 that I'll start later
today or tomorrow.

Le mer. 15 nov. 2023 à 16:31, Pierre Villard 
a écrit :

> Team,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> 1.24.0.
>
> Please review the following guide for how to verify a release candidate
> build:
>
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
>
> The source being voted on the and the convenience binaries are available
> on the Apache Distribution Repository:
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0
>
> The build artifacts are available on the Apache Nexus Repository:
>
> https://repository.apache.org/content/repositories/orgapachenifi-1239
>
> Git Tag: nifi-1.24.0-RC2
> Git Commit ID: b67f9add3f9cfe77146b8339c044f72f89684d51
> GitHub Commit Link:
> https://github.com/apache/nifi/commit/b67f9add3f9cfe77146b8339c044f72f89684d51
>
> Checksums of nifi-1.24.0-source-release.zip
>
> SHA256: c9f8e493bf6b5eef8b53c8e96be5139f904e3187c89edb6b5b112ca3a172b6c5
> SHA512:
> 76859c5e8e888dc04604ee355bbe4e9050551bacecc68df8467ad0938cb2f1d7137211c98fb1793ab521556c6b7bf6899d97e754a33f0dafc109624125a5e46b
>
> Release artifacts are signed with the following key:
>
> https://people.apache.org/keys/committer/pvillard.asc
>
> KEYS file is available on the Apache Distribution Repository:
>
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> Issues resolved for this version: 272
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443
>
> Release note highlights can be found on the project wiki:
>
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0
>
> The vote will be open for 72 hours.
>
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build from source, and test. Then
> please vote:
>
> [] +1 Release this package as nifi-1.24.0
> [] +0 no opinion
> [] -1 Do not release this package because...
>

[VOTE] Release Apache NiFi 1.24.0 (RC2)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.24.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1239

Git Tag: nifi-1.24.0-RC2
Git Commit ID: b67f9add3f9cfe77146b8339c044f72f89684d51
GitHub Commit Link:
https://github.com/apache/nifi/commit/b67f9add3f9cfe77146b8339c044f72f89684d51

Checksums of nifi-1.24.0-source-release.zip

SHA256: c9f8e493bf6b5eef8b53c8e96be5139f904e3187c89edb6b5b112ca3a172b6c5
SHA512:
76859c5e8e888dc04604ee355bbe4e9050551bacecc68df8467ad0938cb2f1d7137211c98fb1793ab521556c6b7bf6899d97e754a33f0dafc109624125a5e46b

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 272

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.24.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [VOTE] Release Apache NiFi 1.24.0 (RC1)

[Pierre Villard]

Killing this RC, thanks David for catching it, I thought I did pull
everything correctly! Will start over, thanks again!

Le mar. 14 nov. 2023 à 19:55, David Handermann 
a écrit :

> -1 binding
>
> Thanks for preparing the release build Pierre, unfortunately, it
> appears to be missing a number of recent commits.
>
> The commit history for the NIFI-12362-RC1 branch [1] appears to be
> missing commits from the support branch [2].
>
> [1] https://github.com/apache/nifi/commits/NIFI-12362-RC1
> [2] https://github.com/apache/nifi/commits/support/nifi-1.x
>
> Regards,
> David Handermann
>
> On Tue, Nov 14, 2023 at 11:02 AM Pierre Villard
>  wrote:
> >
> > Team,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.24.0.
> >
> > Please review the following guide for how to verify a release candidate
> > build:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification
> >
> > The source being voted on the and the convenience binaries are available
> on
> > the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0
> >
> > The build artifacts are available on the Apache Nexus Repository:
> >
> > https://repository.apache.org/content/repositories/orgapachenifi-1237
> >
> > Git Tag: nifi-1.24.0-RC1
> > Git Commit ID: 9620b723f663684611a0cfb3468eff19dde7e5d6
> > GitHub Commit Link:
> >
> https://github.com/apache/nifi/commit/9620b723f663684611a0cfb3468eff19dde7e5d6
> >
> > Checksums of nifi-1.24.0-source-release.zip
> >
> > SHA256: 2e5c3946ebe3f9f741233ba385d35284f8a11303f05f5f91680f827af522d2b1
> > SHA512:
> >
> b2cd74b5ae589f5053e6c04967b176d6fdd7574f19e73b545385e6794244330d213d6d2be9c03bf1bb521bf1adcb4f5fba56257c40260a610d54a702eb04239b
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/pvillard.asc
> >
> > KEYS file is available on the Apache Distribution Repository:
> >
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > Issues resolved for this version: 272
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443
> >
> > Release note highlights can be found on the project wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0
> >
> > The vote will be open for 72 hours.
> >
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [] +1 Release this package as nifi-1.24.0
> > [] +0 no opinion
> > [] -1 Do not release this package because...
>

[VOTE] Release Apache NiFi 1.24.0 (RC1)

[Pierre Villard]

Team,

I am pleased to be calling this vote for the source release of Apache NiFi
1.24.0.

Please review the following guide for how to verify a release candidate
build:

https://cwiki.apache.org/confluence/display/NIFI/Release+Candidate+Verification

The source being voted on the and the convenience binaries are available on
the Apache Distribution Repository:

https://dist.apache.org/repos/dist/dev/nifi/nifi-1.24.0

The build artifacts are available on the Apache Nexus Repository:

https://repository.apache.org/content/repositories/orgapachenifi-1237

Git Tag: nifi-1.24.0-RC1
Git Commit ID: 9620b723f663684611a0cfb3468eff19dde7e5d6
GitHub Commit Link:
https://github.com/apache/nifi/commit/9620b723f663684611a0cfb3468eff19dde7e5d6

Checksums of nifi-1.24.0-source-release.zip

SHA256: 2e5c3946ebe3f9f741233ba385d35284f8a11303f05f5f91680f827af522d2b1
SHA512:
b2cd74b5ae589f5053e6c04967b176d6fdd7574f19e73b545385e6794244330d213d6d2be9c03bf1bb521bf1adcb4f5fba56257c40260a610d54a702eb04239b

Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/pvillard.asc

KEYS file is available on the Apache Distribution Repository:

https://dist.apache.org/repos/dist/release/nifi/KEYS

Issues resolved for this version: 272

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12353443

Release note highlights can be found on the project wiki:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.24.0

The vote will be open for 72 hours.

Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build from source, and test. Then
please vote:

[] +1 Release this package as nifi-1.24.0
[] +0 no opinion
[] -1 Do not release this package because...

Re: [DISCUSS] Release Timing for NiFi 2.0.0 M1 and 1.24.0

[Pierre Villard]

Thanks David, sounds good, will get started on the 1.24 release soon.

Le lun. 13 nov. 2023 à 19:37, David Handermann 
a écrit :

> Thanks for replies Pierre and Mark!
>
> With several pull requests merged last week addressing the Python
> concerns Mark raised, I believe we are ready to proceed with the
> release candidate process.
>
> Based on a conversation Pierre, we should be able to collaborate on
> preparing candidate builds for both 2.0.0-M1 and 1.24.0.
>
> We will need to make a few website adjustments for the 2.0.0-M1
> version in terms of documentation and downloads, but it should be
> possible to handle that with some alternative navigation links for
> now.
>
> I plan on starting the release candidate build process for 2.0.0-M1 soon.
>
> Regards,
> David Handermann
>
> On Wed, Nov 8, 2023 at 5:53 PM Mark Payne  wrote:
> >
> > Hey Pierre,
> >
> > I ran into a couple of issues (they appear to be timing issues) around
> the Python stuff on startup. I think we need to hold off on the release
> until these are resolved but I think I should have a PR up some time
> tomorrow hopefully.
> >
> > Thanks
> > -Mark
> >
> >
> >
> > > On Nov 8, 2023, at 10:30 AM, Pierre Villard <
> pierre.villard...@gmail.com> wrote:
> > >
> > > Hey David,
> > >
> > > With all of the recent changes we merged, I believe we are in the right
> > > spot to start the process with 1.24 and 2.0M1 releases.
> > >
> > > Happy to help with this,
> > > Pierre
> > >
> > > Le ven. 3 nov. 2023 à 22:46, David Handermann <
> exceptionfact...@apache.org>
> > > a écrit :
> > >
> > >> Team,
> > >>
> > >> We have made great progress on a large number of features and fixes
> > >> for NiFi 2.0.0. The Jira version page for 2.0.0 [1] lists almost 900
> > >> issues! At the same time, we have 250 issues tagged for version 1.24.0
> > >> [2].
> > >>
> > >> Although there is still more work to do for a full release of 2.0.0, I
> > >> believe we are at a good point to release a milestone version.
> > >> Releasing 1.24.0 at the same time will also provide the latest
> > >> upgrades, plus new features and deprecations that will help prepare
> > >> for upgrading to 2.0.0.
> > >>
> > >> We have informally referred to the initial release as a milestone
> > >> version, so it seems like officially naming it 2.0.0-M1 would be the
> > >> most straightforward option.
> > >>
> > >> There are a couple more open issues that I believe we can wrap up in
> > >> the next few days, but otherwise we should be able to target early
> > >> next week for preparing a release build.
> > >>
> > >> With the goal of coordinating a release of multiple versions, I would
> > >> be glad to handle release manager duties for one of the versions.
> > >>
> > >> Regards,
> > >> David Handermann
> > >>
> > >> [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599
> > >> [2] https://issues.apache.org/jira/projects/NIFI/versions/12353443
> > >>
> >
>

Re: [DISCUSS] Release Timing for NiFi 2.0.0 M1 and 1.24.0

[Pierre Villard]

Hey David,

With all of the recent changes we merged, I believe we are in the right
spot to start the process with 1.24 and 2.0M1 releases.

Happy to help with this,
Pierre

Le ven. 3 nov. 2023 à 22:46, David Handermann 
a écrit :

> Team,
>
> We have made great progress on a large number of features and fixes
> for NiFi 2.0.0. The Jira version page for 2.0.0 [1] lists almost 900
> issues! At the same time, we have 250 issues tagged for version 1.24.0
> [2].
>
> Although there is still more work to do for a full release of 2.0.0, I
> believe we are at a good point to release a milestone version.
> Releasing 1.24.0 at the same time will also provide the latest
> upgrades, plus new features and deprecations that will help prepare
> for upgrading to 2.0.0.
>
> We have informally referred to the initial release as a milestone
> version, so it seems like officially naming it 2.0.0-M1 would be the
> most straightforward option.
>
> There are a couple more open issues that I believe we can wrap up in
> the next few days, but otherwise we should be able to target early
> next week for preparing a release build.
>
> With the goal of coordinating a release of multiple versions, I would
> be glad to handle release manager duties for one of the versions.
>
> Regards,
> David Handermann
>
> [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599
> [2] https://issues.apache.org/jira/projects/NIFI/versions/12353443
>

Re: CaptureChangePostgreSQL not available in latest build?

[Pierre Villard]

Hi,

No, the corresponding pull request has not been merged. You can see the
full discussion here https://github.com/apache/nifi/pull/6053 and the PR
has been closed until further improvements are made. I believe there is a
custom NAR that is made available by some community members in the comments
of that pull request if you want to give it a try.

Hope this helps,
Pierre

Le ven. 3 nov. 2023 à 09:27, Andreas Paredi  a
écrit :

> Hi
>
> I am interested in the CaptureChangePostgreSQL processor.
>
> From Jira / Git I understand the code has been approved and merged (
> https://github.com/apache/nifi/pull/4065 |
> https://issues.apache.org/jira/browse/NIFI-4239)
>
> However, I am not able to find this processor in the list of available
> processors in the UI after startup.
>
> I am running a PoC with various solutions and use docker run --name nifi
> -p 8443:8443 -d apache/nifi:latest
>
> Thanks for any kind of support to better understand your powerful solution
> 😊
>
> Kind regards/Freundliche Grüsse
>
> Andreas Paredi
> Head Rieter TechHub (AMFR)
>
> Büro TechHub: Technopark Winterthur . Technoparkstrasse 2 . 8406 Winterthur
> Hauptsitz: Rieter Maschinenfabrik AG . Klosterstrasse 20 . 8406 Winterthur
> . Switzerland
> M +41 76 399 91 94
> andreas.par...@rieter.com .
> www.rieter.com
>
>
> Informationen für betroffene Personen gemäss Artikel 13 und 14 der
> EU-DSGVO und gemäss Art. 19 DSG finden Sie in unserer Datenschutzerklärung<
> https://www.rieter.com/de/datenschutzerklaerung>.
> Information for data subjects pursuant to Article 13 and 14 of EU-GDPR and
> Article 19 SDPA can be found in our Privacy Statement<
> https://www.rieter.com/privacy-statement>.
>

Re: Provenance events without FlowFiles?

[Pierre Villard]

>
> Rather, I’d say it's an UPLOAD_FILE event. So I’d lean more toward an
> uploadFile() method on ProvenanceReporter that takes as an argument a
> `File` (as well as a FlowFile). The size would come from the File itself,
> and the event would convey the information about the local file that was
> uploaded - probably in the Event Details.
>

Would that mean that for the "bytes transferred" graph in Status History,
we would combine SEND and UPLOAD_FILE events? Because, right now, it's not
showing anything which is confusing.

Also, I'm not sure about the 'File' object. While we have only the local
file system as an option today for the File Resource Service, I'd expect
additional implementations such as implementations for CSPs. So we could
have the case where PutAzureBlobStorage is used with a FileResourceService
for Google Cloud Storage for example (in order to improve efficiency of
data movement between cloud providers) and, in this case, not sure we would
have a 'File' object. Unless you're talking about a more generic File
object here and not the object for local file system.

Le jeu. 26 oct. 2023 à 09:16, Matt Burgess  a écrit :

> AFAIK it is fine and appropriate to issue multiple provenance events
> for a single FlowFile. In the case for PutAzureBlobStorage uploading a
> file to Azure, it is the incoming FlowFile that triggers the upload.
> Before reporting a provenance event, attributes are added to the
> FlowFile, so that "version" of the FlowFile can be the one used to
> report a SEND event. I have done this to said processor as part of a
> large refactor/improvement of the provenance capability:
>
> session.getProvenanceReporter().send(flowFile,
> blob.getSnapshotQualifiedUri().toString(), transferMillis,
> REL_SUCCESS);
>
> Having said that, to Mark's point it's probably better to have a
> separate UPLOAD_FILE event, I can change that in my code.
>
> I added a couple like this to similar processors, such as
> TriggerHiveMetastoreEvent:
>
> session.getProvenanceReporter().invokeRemoteProcess(flowFile,
> hiveMetastoreUrl, REL_SUCCESS);
>
> I am still working on this, I need to write up a Jira with a thorough
> treatment of the material and eventually get a PR up for review.
>
> Regards,
> Matt
>
> On Thu, Oct 26, 2023 at 12:02 PM Mark Payne  wrote:
> >
> > Lehel,
> >
> > I don’t believe we should be trying to create a “Mock FlowFile.” I am ok
> with an update to the ProvenanceReporter interface. But I don’t think it
> should accept a “size” parameter. Rather, I think this is a completely
> different type of event that is occurring. This is not a “send” in that
> it’s not sending the contents of the FlowFile to a remote system. Rather,
> I’d say it's an UPLOAD_FILE event. So I’d lean more toward an uploadFile()
> method on ProvenanceReporter that takes as an argument a `File` (as well as
> a FlowFile). The size would come from the File itself, and the event would
> convey the information about the local file that was uploaded - probably in
> the Event Details.
> >
> > Thanks
> > -Mark
> >
> >
> > > On Oct 26, 2023, at 10:36 AM, Lehel Boér  wrote:
> > >
> > > Hi everyone,
> > >
> > > I would like to address a particular scenario that has recently come
> to my attention regarding the use of the PutAzureBlobStorage processor with
> the FileResourceService.
> > >
> > > When the PutAzureBlobStorage processor is used with the
> FileResourceService, it currently uploads a file from the user's local
> filesystem to Azure, but it does not create a FlowFile. Instead, it
> utilizes the incoming FlowFile solely to send a provenance event. In this
> case the size of the provenance event is the incoming FlowFile's size
> instead of the uploaded one.
> > >
> > > There are potential solutions to address this issue and ensure that
> the provenance events are handled effectively. Two main options have been
> proposed:
> > >
> > >
> > >  *   Create a Mock FlowFile: A mock FlowFile with a size matching that
> of the local file being uploaded could be generated. This mock FlowFile
> would serve as the basis for the provenance event, even though its size
> might not reflect the actual content.
> > >
> > >  *   Modify the ProvenanceReporter Interface: Alternatively, we could
> introduce a new method in the ProvenanceReporter interface that doesn't
> require a FlowFile but instead accepts a "size" parameter as an argument.
> This would eliminate the need for a mock FlowFile.
> > >
> > > The lack of a FlowFile operation in this situation creates a distinct
> challenge because provenance events are typically tied to FlowFiles. Still,
> it's important to indicate data transmission for monitoring and tracking.
> > >
> > > While the idea of a "size" parameter for the provenance event seems
> preferable, we need to carefully consider its feasibility, potential
> complexities, and community acceptance. The FileResourceService already
> deviates from NiFi's concept of using FlowFiles to hold payload data, and
> we must avoid fur

Re: Property management - reducing duplication

[Pierre Villard]

Hey Bence and team,

I'd definitely be in favor of a better approach here. When removing
variables, I found myself with the need to update a lot of copies of
nifi.properties as well as other configuration files across many places of
the codebase. I don't know what is the best option/approach here but having
a single source of truth somewhere and being able to reference this
everywhere with customization definitely sounds nice.

Pierre

Le mar. 26 sept. 2023 à 09:19, Simon Bence  a
écrit :

> Hi Team,
>
> I was touching some test related code in the other day and it brought to
> my attention how much partly duplicated nifi.properties files we do have in
> the project in various places.
>
> While I was searching for the value of a given property in these files, it
> got me thinking that when a property is changing (for example related to
> the 2.x efforts) or added, it indicates changes in multiple places, which
> could lead to oversights and inconsistencies. Additionally, it seems to me
> that duplicating whole configuration files might make one reluctant to
> create specific properties files for specific tests like in case of the
> system tests.
>
> I would like to propose a discussion about this, being curious if the
> community sees any value in improving the configuration management. My
> initial thoughts is to maintain one single “source of truth” properties
> file and providing some kind of utility, which could generate instances as
> needed allowing to override or extend properties when necessary.
>
> Looking forward to your insights and suggestions.
>
> Regards,
> Bence

Re: [discuss] Time for a NiFi 2.0 M1 release?

[Pierre Villard]

Hey Joe,

Definitely a +1 to get a M1 release ASAP. I'd still recommend waiting on
the flow.xml removal work to be merged. The reason being that users may
give useful feedback when they'll try NiFi 2.0 with existing flows coming
from NiFi 1.x and getting rid of all of the XML based stuff. There is also
a PR coming soon for the frontend work of the templates removal. Hopefully
both can be completed this week or next week.

Pierre

Le mar. 26 sept. 2023 à 17:35, Joe Witt  a écrit :

> Team,
>
> The NiFi 2.0 release has more than 700 resolved JIRAs on it [1] and growing
> every day.
>
> The NiFi 2.0 deprecation plan is well underway and largely complete [2].
>
> We still need to remove a lot of now deprecated code, tests which are never
> run and largely don't work, eliminate the flow.xml which has a JIRA/PR
> underway.  And more.  But we're getting close and we need to start getting
> this in the hands of users.
>
> The docker image can now be built in 'nifi-docker/dockermaven' after a full
> build from root with 'mvn install -Pdocker'.  And it comes up with Ubuntu,
> Java 21, Python 3.9, and NiFi 2.0 ready to roll with Python processors
> enabled.
>
> I propose we start closing down soon to make a NiFi 2.0 M1 release happen
> even before we have all the things done.  We need to start getting feedback
> and giving people a chance to work with it.
>
> Lastly, a huge thank you to the folks in the community that have been
> helping push towards 2.x with code changes, removals, reviews, bug reports,
> etc..  Super awesome to see.  NiFi 2.x is shaping up nicely to be useful
> not only for our well established user base which spans the globe and every
> industry but now we are also seeing a lot of opportunity and fit for NiFi
> in these exciting AI use cases particularly involving orchestrating the
> data flows with embeddings, vector stores, and LLMs.  And the Python
> capabilities in NiFi 2.x make NiFi far easier to use for the very important
> data engineer user base.
>
> [1] https://issues.apache.org/jira/projects/NIFI/versions/12339599
> [2]
>
> https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features
>
> Thanks
> Joe
>

Re: flattening record schemas

[Pierre Villard]

Hi,

Do you have an example of input and the output you'd expect? There are a
few options in terms of processors you could use and you may be able to get
things working the way you want without using JOLT.

Pierre

Le lun. 18 sept. 2023 à 22:05, Mark Woodcock  a
écrit :

> Howdy,
>
> What I'm aiming for:
> Something that takes a fairly ordinary record (think a CSV file--so, the
> name of each column is a distinct entry in the schema), which outputs the
> same data, but where the record is now an array of similarly structured
> items (I presume records, where the fields would be stuff like a columnName
> [where the value is CSV column name], value [where the value is the value],
> and perhaps other fields).
>
> When I asked around, the suggestion was that my need was to flatten the
> schema and that the JOLT processor might be the way to do that...but my
> contact had never used JOLT.  The follow-on suggestion was to ask here
> (well, on "users" but that rejected me, since this address worked before,
> I'm trying it) for advice on how one might go about that.
>
> thx,
>
> mew
>

Re: [discuss] nifi 2.0 and Java 21…

[Pierre Villard]

Templates are already out. Flow.xml being removed should be reviewed soon
(PR was rebased today). I'm working on the removal of variables. I hope to
get a PR for this in the next few days.

Le ven. 15 sept. 2023, 19:22, Joe Witt  a écrit :

> Timeline - we remain in full blitz mode to get things ready for 2.0.  No
> clear ETA but we need to be getting it out soon.  At least a milestone
> release of it for people to work with.  There is a big change needed to get
> rid of the flow.xml.gz in favor of the json form and that is in progress.
> I am not sure offhand whether templates got the boot yet.
>
> Latest fun is wrestling our rather messy situation with Groovy in the build
> as that seems not ready for Java 21 generally.
>
>
>
> On Fri, Sep 15, 2023 at 10:19 AM Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
> > I think NiFi 2.x going to Java 21 for all the reasons outlined makes a
> lot
> > of sense.
> >
> > Is there a timeline for 2.x?
> >
> > On Mon, Sep 11, 2023 at 5:00 AM Pierre Villard <
> > pierre.villard...@gmail.com>
> > wrote:
> >
> > > Thanks Joe, it makes total sense and I agree that the ones that would
> > > likely be slow at adopting Java 21 would not go to NiFi 2.0 super
> quickly
> > > anyway. Being able to bring the latest and greatest in NiFi is great
> and
> > > given all of the features announced in Java 21, I imagine a lot of
> > projects
> > > we depend on will be doing the same.
> > >
> > > Le jeu. 7 sept. 2023 à 19:36, Joe Witt  a écrit :
> > >
> > > > Pierre
> > > >
> > > > A few concerns you raised so want to address my view on each:
> > > >
> > > > Will users be able to adopt Java 21 fast enough?
> > > > I share Brandon's view on that in terms of their adoption timeline.
> It
> > > > will likely align well with NiFi 2.0 itself in my estimation.
> > > >
> > > > Will this delay NiFi 2.0?
> > > > If it would then I'd not be supportive.  I don't think we need to
> > bother
> > > > with adopting any of the features now.  What I would like us to have
> is
> > > the
> > > > option to adopt them as we progress.  We should get 2.0 done asap and
> > if
> > > > this added delay then I'd be way less interested in this idea.
> > > >
> > > > Feature benefits of 21 and what will that bring?
> > > > Mark spoke well to the key one that stood out to me which was the new
> > > > threading model available.  It would be awfully nice to leverage that
> > for
> > > > the efficiency it represents and especially if it can reduce some of
> > our
> > > > heap usage which is valuable in cloud/shared compute contexts.
> > > >
> > > > Performance benefits of Java 21?
> > > > It appears from some analysis found with googling that Java 21 brings
> > out
> > > > of the box 4-5% performance increases generally.  Not amazing but
> > useful.
> > > >
> > > > User experience otherwise with Java 21?
> > > > I believe it would be consistent with Java 17 for their point of view
> > in
> > > > terms of install/config/etc..
> > > >
> > > > My motivation for this is fairly pure honestly.  Since we're setting
> > our
> > > > new minimum bar that lives for as long as the 2.x release line lives
> > I'd
> > > > like to set it at the current LTS available when we ship that line as
> > > well.
> > > >
> > > > Thanks
> > > >
> > > >
> > > >
> > > > On Thu, Sep 7, 2023 at 4:22 AM Brandon DeVries <
> > > brandon.devr...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 to requiring java 21. Starting off as "up to date" as possibly
> > > makes a
> > > > > lot of sense, and some of the new features seem especially relevant
> > to
> > > > NiFi.
> > > > >
> > > > > I definitely understand the concerns about organizations being
> > willing
> > > /
> > > > > able to approve Java 21... But those same organizations might also
> be
> > > > > hesitant to move to NiFi 2.0. We will continue to support java 17 &
> > > NiFi
> > > > > 1.x for some time, so hopefully those groups will have the time
> they
> > > need
> > > > > to get approvals, do evalua

Re: JS scripting and 2.0

[Pierre Villard]

Hi Dirk,

You're definitely not the odd one out there using JS with NiFi. The most
common usage is definitely Groovy but I'm aware of NiFi users using JS. I'm
not aware of a specific timeline as I don't know if someone in the
community is actively looking into it right now but I agree that this
should be something to take into account for NiFi 2.0.

Pierre

Le ven. 15 sept. 2023 à 10:03, Dirk Arends  a
écrit :

> I'm still having several issues with JS scripting in Nifi:
>
> 1. Nifi versions past 1.14 take a very long time to validate JS scripted
> processors, to the point where processor validation starts timing out and
> failing. I haven't created an issue specifically about this as we don't
> have simple instructions to replicate, but I can document this if it would
> help.
>
> 2. In the long term, relying on Nashorn means I can't upgrade to a JDK
> version higher than JDK11, and won't be able to use Nifi 2.x when it
> releases.
>
> 3. Nashorn only fully supports ES5.1, with some support for ES6 features.
> This means I have to use several polyfills, making scripts larger and
> (probably) contributing to startup time issues. For example, a 100 line
> source script becomes ~4000 lines to make modern Javascript run
> consistently. I have experimented with enabling and disabling various Babel
> plugins and polyfills but have not been able to achieve a consistent
> result.
>
> These issues above and my reliance on JS scripting means I can't upgrade
> Nifi past 1.14 in production. Not being able to upgrade means not getting
> any of the new Nifi features and improvements, which is becoming
> increasingly difficult. Is anybody else impacted by these issues?
>
> David mentioned that there are plans to entirely redo the scripting
> implementation in Nifi 2.x:
>
> https://lists.apache.org/thread/ssc19kxoz89g0y0wcfvb410f4x1qwkdg
>
> Is there a timeline yet, or is JS scripting going to be unavailable in
> modern Nifi for the foreseeable future?
>
> One potential option is replacing Nashorn with Graal - see comments here:
>
> https://issues.apache.org/jira/browse/NIFI-6229
>
> The impression I have is that using scripted processors is fairly common
> and a popular feature, but maybe most other people use Python or Groovy and
> I'm the odd one out using JS?
>
>
> Dirk Arends
>

Re: [discuss] nifi 2.0 and Java 21…

[Pierre Villard]

Thanks Joe, it makes total sense and I agree that the ones that would
likely be slow at adopting Java 21 would not go to NiFi 2.0 super quickly
anyway. Being able to bring the latest and greatest in NiFi is great and
given all of the features announced in Java 21, I imagine a lot of projects
we depend on will be doing the same.

Le jeu. 7 sept. 2023 à 19:36, Joe Witt  a écrit :

> Pierre
>
> A few concerns you raised so want to address my view on each:
>
> Will users be able to adopt Java 21 fast enough?
> I share Brandon's view on that in terms of their adoption timeline.  It
> will likely align well with NiFi 2.0 itself in my estimation.
>
> Will this delay NiFi 2.0?
> If it would then I'd not be supportive.  I don't think we need to bother
> with adopting any of the features now.  What I would like us to have is the
> option to adopt them as we progress.  We should get 2.0 done asap and if
> this added delay then I'd be way less interested in this idea.
>
> Feature benefits of 21 and what will that bring?
> Mark spoke well to the key one that stood out to me which was the new
> threading model available.  It would be awfully nice to leverage that for
> the efficiency it represents and especially if it can reduce some of our
> heap usage which is valuable in cloud/shared compute contexts.
>
> Performance benefits of Java 21?
> It appears from some analysis found with googling that Java 21 brings out
> of the box 4-5% performance increases generally.  Not amazing but useful.
>
> User experience otherwise with Java 21?
> I believe it would be consistent with Java 17 for their point of view in
> terms of install/config/etc..
>
> My motivation for this is fairly pure honestly.  Since we're setting our
> new minimum bar that lives for as long as the 2.x release line lives I'd
> like to set it at the current LTS available when we ship that line as well.
>
> Thanks
>
>
>
> On Thu, Sep 7, 2023 at 4:22 AM Brandon DeVries 
> wrote:
>
> > +1 to requiring java 21. Starting off as "up to date" as possibly makes a
> > lot of sense, and some of the new features seem especially relevant to
> NiFi.
> >
> > I definitely understand the concerns about organizations being willing /
> > able to approve Java 21... But those same organizations might also be
> > hesitant to move to NiFi 2.0. We will continue to support java 17 & NiFi
> > 1.x for some time, so hopefully those groups will have the time they need
> > to get approvals, do evaluations, and upgrade.
> >
> > Brandon
> > 
> > From: Pierre Villard 
> > Sent: Thursday, September 7, 2023 6:15:58 AM
> > To: dev@nifi.apache.org 
> > Subject: Re: [discuss] nifi 2.0 and Java 21…
> >
> > Hi all,
> >
> > I share the concerns raised by Chris regarding how quickly users of NiFi
> > will be able to adopt Java 21.
> >
> > While I'm definitely in favor of using the latest and greatest,
> especially
> > when it brings to the table such significant features, we need to be
> > careful as it may significantly delay the adoption of NiFi 2.0 in big
> > companies where deploying Java 21 will take time. I acknowledge that
> going
> > from Java 8 to Java 17 is certainly the same effort as going from Java 8
> to
> > Java 21 but how quickly security-sensitive environments will adopt a new
> > release of Java that is completely new?
> >
> > In addition to that, it sounds like we would add a significant rework of
> > the framework in NiFi 2.0 assuming we adopt Java 21 as the minimum
> version.
> > Do we think this is going to significantly delay the first release of
> NiFi
> > 2.0? We still have work to do but adding this on top could delay the
> > release, no?
> >
> > Finally, the features that Java 21 are bringing sound super interesting
> in
> > the context of NiFi but do we already have an idea of what it will
> improve?
> > is it the user experience, and if so, how will it change? is it the
> > performance, and if so, do we have an idea of how things will improve?
> >
> > Thanks,
> > Pierre
> >
> > Le mer. 6 sept. 2023 à 23:07, Chris Sampson
> >  a écrit :
> >
> > > Yeah, I understand the need to move to 21 as a minimum to take
> advantage
> > of
> > > its features. Hopefully the wider java ecosystem won't be an issue in
> the
> > > short term.
> > >
> > > I just wanted the discussion to be clear about this being a change to
> the
> > > Java baseline/minimum for NiFi 2.0.
> > >
> > > It's a +1 from me.
>

Re: [discuss] nifi 2.0 and Java 21…

[Pierre Villard]

Hi all,

I share the concerns raised by Chris regarding how quickly users of NiFi
will be able to adopt Java 21.

While I'm definitely in favor of using the latest and greatest, especially
when it brings to the table such significant features, we need to be
careful as it may significantly delay the adoption of NiFi 2.0 in big
companies where deploying Java 21 will take time. I acknowledge that going
from Java 8 to Java 17 is certainly the same effort as going from Java 8 to
Java 21 but how quickly security-sensitive environments will adopt a new
release of Java that is completely new?

In addition to that, it sounds like we would add a significant rework of
the framework in NiFi 2.0 assuming we adopt Java 21 as the minimum version.
Do we think this is going to significantly delay the first release of NiFi
2.0? We still have work to do but adding this on top could delay the
release, no?

Finally, the features that Java 21 are bringing sound super interesting in
the context of NiFi but do we already have an idea of what it will improve?
is it the user experience, and if so, how will it change? is it the
performance, and if so, do we have an idea of how things will improve?

Thanks,
Pierre

Le mer. 6 sept. 2023 à 23:07, Chris Sampson
 a écrit :

> Yeah, I understand the need to move to 21 as a minimum to take advantage of
> its features. Hopefully the wider java ecosystem won't be an issue in the
> short term.
>
> I just wanted the discussion to be clear about this being a change to the
> Java baseline/minimum for NiFi 2.0.
>
> It's a +1 from me.
>
> On Wed, 6 Sept 2023, 19:01 Joe Witt,  wrote:
>
> > Chris
> >
> > My suggestion is rooted in making Java 21 the minimum of the NiFi 2.0
> > line.  It would not work on Java 17.  The reason for this is so that we
> can
> > leverage the longest duration of a given LTS line while also benefiting
> > from the language improvements that affords.  Maintaining compatibility
> > with future versions we generally have to do.  But whatever the minimum
> > version we accept dictates which language features we can leverage.  So
> if
> > it is 17 then we can't leverage anything from the 21 line for example.
> >
> > GIven the nature and timelines of LTS I don't really think there is the
> > same burn in logic that we'd have all known in the past before the
> > LTS/STS/etc.. release constructs existed.
> >
> > Thanks
> >
> > On Wed, Sep 6, 2023 at 10:53 AM Chris Sampson
> >  wrote:
> >
> > > To be clear, is the discussion one of making Java 21 the minimum
> > > requirement for NiFi 2.0.0, or rather NiFi 2.x be compatible with Java
> > 21,
> > > while retaining Java 17 as a minimum?
> > >
> > > If we moved straight to a Java 21 requirement, will we run into
> > > compatibility issues that delay initial NiFi 2 release? Will the move
> to
> > > Java 21 mean some organisations delay their migration to NiFi 2 through
> > not
> > > wanting to move to the latest Java LTS version until it's had a time
> for
> > > "settling" through security/bug patches, etc.? And are either of these
> > > sufficient concern to pause Java 21 becoming the requirement, as we may
> > > then need to extend NiFi 1.x maintenance for longer into the future?
> > >
> > > Generally, I'm in favour of moving to "latest and greatest",
> particularly
> > > for LTS versions of technologies, but the rate of Java version adoption
> > > across the community gives me pause.
> > >
> > > I certainly see the advantage of new Java features for NiFi in Java 21,
> > > such as the already mentioned virtual threads.
> > >
> > > On Wed, 6 Sept 2023, 18:34 Mike Thomsen, 
> wrote:
> > >
> > > > +1 100%
> > > >
> > > > On Wed, Sep 6, 2023 at 11:48 AM Adam Taft  wrote:
> > > >
> > > > > Yes, please. +1 Exactly what Mark said. Virtual threads have
> > potential
> > > to
> > > > > be extremely impactful to applications like NiFi.
> > > > >
> > > > > /Adam
> > > > >
> > > > > On Wed, Sep 6, 2023 at 7:26 AM Mark Payne 
> > > wrote:
> > > > >
> > > > > > Thanks for bringing his up, Joe.
> > > > > >
> > > > > > I would definitely be a +1. I think the new virtual thread
> concept
> > > > would
> > > > > > have great impact on us.
> > > > > > It would allow us to significantly simplify our scheduling logic,
> > > which
> > > > > > would help with code maintainability
> > > > > > but would also make configuration simpler. This is one of the
> most
> > > > > > difficult things for users to configure,
> > > > > > and I would very much welcome the ability to simplify this. It
> > would
> > > > > > likely also yield better off-heap memory
> > > > > > utilization by reducing the number of native threads necessary.
> > > > > >
> > > > > > Thanks
> > > > > > -Mark
> > > > > >
> > > > > >
> > > > > > > On Sep 6, 2023, at 10:20 AM, Joe Witt 
> > wrote:
> > > > > > >
> > > > > > > Team
> > > > > > >
> > > > > > > Thought it might be worth relighting this thread with Java 21
> GA
> > > > > > imminent.
> > > > > > > Given the timing we should give consider

Re: [VOTE] Release Apache NiFi MiNiFi C++ 0.15.0

[Pierre Villard]

+1 (binding)

Build on u20, tested a couple of simple flows.

Thanks!

Le jeu. 31 août 2023 à 19:35, Arpad Boda  a écrit :

> +1 (binding)
>
> Verified signature, hashes.
> Built on debian and mac.
>
> Executed all tests successfully, verified c2 functionality, designed
> multiple flows, verified those.
>
> Thanks,
> Arpad
>
>
> On Wed, Aug 30, 2023 at 7:11 PM Gábor Gyimesi  wrote:
>
> > +1 (non-binding)
> >
> > Went through the verification process using the helper guide.
> >
> > Compiled all but the JNI extension successfully on Ubuntu 22.04 with
> > GCC 11, ran all unit and integration tests, did not find any issues.
> >
> > Compiled on Windows using MSVC and Ninja using Visual Studio 2019.
> > Used the following command: win_build_vs.bat build /NINJA /P /K /S /A
> > /SFTP /PDH /SPLUNK /GCP /ELASTIC /Z /PR /ENCRYPT_CONFIG /MQTT /OPC
> > /PYTHON_SCRIPTING
> > I had a compilation issue on Windows with the SFTP extension: linking
> > SFTPLoader.cpp.obj failed with unresolved Curl symbols. Seems to be an
> > issue of the static linkage of Curl, which is worth investigating, but
> > I don't think it's a blocking issue. After removing SFTP from the
> > compilation list the project compiled successfully.
> >
> > Ran two flows on both Windows (using the compiled binaries) and Linux
> > (using the provided convenience binaries) successfully:
> > TailFile -> LogAttribute
> > GenerateFlowFile -> UpdateAttribute -> MergeContent -> CompressContent
> > -> PutS3Object
> >
> > Note: Updated the
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=139627733
> > wiki page with the new OpenSSL build requirements on Windows.
> >
> > Thanks,
> > Gábor
> >
> > On Tue, 29 Aug 2023 at 23:05, Marton Szasz  wrote:
> > >
> > > +1 (binding)
> > >
> > > Verified everything according to the release helper guide.
> > >
> > > On linux, bootstrap.sh installs all the required dependencies for
> > > compiling with GCC.
> > > - Ubuntu 22.04 / GCC: works fine
> > > Clang required additional packages: clang libc++-dev libc++abi-dev
> > > - Ubuntu 22.04 / Clang + libc++: didn't compile, but this is not a
> > > showstopper IMO. We can fix it later and prepare the next release a
> > > bit sooner.
> > > - Ubuntu 22.04 / Clang + libstdc++: works fine
> > >
> > > Arch Linux / any compiler: linker issues related to curl. I wouldn't
> > > tank the release for this.
> > >
> > > Windows steps:
> > > 1. Used Visual Studio Community 2019 (VS2022 support is under review,
> > > not yet included)
> > > 2. Installed scoop (in powershell):> irm get.scoop.sh | iex
> > > 3. Installed the latest cmake (for build), python (for scripting
> > > support), sccache (for build caching, like ccache) and wixtoolset (for
> > > installer generation) with scoop:> scoop install cmake python sccache
> > > wixtoolset
> > > 4. Source checked out at C:\a\m (to avoid long path issues)
> > > 5. Built in "x64 Native Tools Command Prompt for VS2019" with the
> > > following command:> win_build_vs.bat ..\bld /64 /P /K /S /A /SFTP /PDH
> > > /SPLUNK /GCP /ELASTIC /Z /PR /ENCRYPT_CONFIG /MQTT /OPC
> > > /PYTHON_SCRIPTING /D /NONFREEUCRT /SCCACHE
> > > 6. Installed the resulting MSI, and copied cwel_config.yml from the
> > > repo, but modified it to send the logs with PutTCP and PutUDP (2
> > > separate tests) to a netcat listening on a linux box. It worked well,
> > > the logs arrived right away on the other box. Also tried the new saved
> > > log file support.
> > >
> > > My reaction to Ferenc's issues:
> > > - I agree that we should make 64bit the default in the future.
> > > - I also ran into the cpack issue in the past, but we have a note
> > > about it in the README, which is good enough for now IMO.
> > > - I prefer not starting the service right after installation, before I
> > > even have the chance to add my flow to config.yml, but C2 users may
> > > have different preferences.
> > >
> > > Thanks,
> > > Márton
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Aug 29, 2023 at 3:20 PM Ferenc Gerlits 
> > wrote:
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Verified hashes and signature on the source tarball, checked git
> > > > commit hash and tag.
> > > > Built on Windows 10 with 64-bit VS 2019, installed the msi package
> and
> > > > ran a simple CWEL -> LogAttribute flow.
> > > >
> > > > I ran into some issues during the build, but none of them are
> > showstoppers:
> > > >  - the release helper guide should make it clear that
> win_build_vs.bat
> > > > defaults to 32-bit and you have to
> > > >  add /64 to the command line if you want a 64-bit build (should
> we
> > > > make 64-bit the default?);
> > > >  - win_build_vs.bat fails if the build directory path contains
> spaces;
> > > >  - the cpack command in win_build_vs.bat found chocolatey on my
> > > > computer instead of CMake's cpack;
> > > >  - the installer does not start the service (I don't know if it used
> > > > to, but I think it should).
> > > >
> > > > Thank you,
> > > > Ferenc
> >
>

Re: [DISCUSS] MiNiFi C++ 0.15.0 release

[Pierre Villard]

+1

Le jeu. 24 août 2023 à 17:04, Joe Witt  a écrit :

> +1 a lot of nice things in there!
>
> On Thu, Aug 24, 2023 at 7:52 AM Martin Zink  wrote:
>
> > Hi community,
> >
> > I'd like to initiate a discussion about the next release of MiNiFi C++.
> The
> > last release was more than four months ago, and there have been many new
> > features, bug fixes and stability improvements committed to the
> development
> > branch
> > since then: 71 tickets closed, over 96 commits as of today.
> >
> > I would be happy to take on RM duties for this release.
> >
> > Notable features and improvements since the 0.14.0 release:
> >
> > - ConsumeWindowsEventLog can work from log files
> > - ConsumeWindowsEventLog resolve Security/UserID attribute
> > - TLS v1.3 support
> > - PutS3Object multipart upload support
> > - Use systemd service management on Linux
> > - Add ProcessContext::getStateManager to Lua/Python
> > - Reworked GetTCP to be more inline with ListenTCP
> > - SSL support for Prometheus reporter
> > - Documentation improvements
> > - Multiarch docker support
> > - RFC3339 parsing with expression language
> > - Reworked Minifi controller
> > - gcc-13 support
> >
> > - Fix for waking up prematurely after processor yields
> > - Fix system certificate store usage in SSLContextService on Linux
> > - Fix inconsistent naming in C2 machineArch
> > - Fix default CA path for S3 on CentOS
> > - Removed CronScheduler locale requirements
> >
> > We've upgraded our third party dependencies (notable mentions)
> > - Replaced LibreSSL with OpenSSL 3.1.1
> > - Upgraded RocksDB to v8.1.1
> > - Upgraded LibCurl to v8.1.0
> > - Upgraded CivetWeb to v1.16
> > - Upgraded OpenCV to v4.7.0
> > - Upgraded GoogleCloud SDK to v2.10.1
> > - Upgraded Azure SDK to v12.7.0
> >
> > The core API is still not mature enough to be able to commit to it, so
> > in line with previous discussions I suggest releasing it as 0.15.0.
> >
> > Do you agree it is time for a new release? Are there any blockers that we
> > should definitely include in this release?
> >
> > Thanks,
> > Martin
> >
>

Re: Not able to use runtime attributes in lists3

[Pierre Villard]

Hi,

Can you clarify your question? I'm not sure I understand what you mean.

Le jeu. 20 juil. 2023 à 17:58, Ravi Kumar Undapalli 
a écrit :

> Hi Team,
>
> I'm not able to use runtime attributes in lists3 processor in Apache NiFi.
>
> Thanks and regards
> Ravi Kumar Undapalli
>

Re: NiFi 2.0 - QuestDB

[Pierre Villard]

I do think this provides great value. The possibility to get access to
status history of the components and at system level across restart is a
great improvement for NiFi troubleshooting. It also gives the ability to
store this information for a longer period of time. I'm definitely in favor
of making this the default starting with NiFi 2.0.

Le mer. 19 juil. 2023 à 13:49, Simon Bence  a
écrit :

> Hi Community,
>
> I was thinking if it would make sense to set the QuestDB as default for
> status history backend in 2.0? It is there for a while and I would consider
> it as a step forward so the new major version might be a good time for the
> wider audience. It comes with less memory usage for bigger flows, the
> possibility of checking status information when the node is not running or
> restarted so I think it worth consideration. Any insight or improvement
> point is appreciated, thanks!
>
> Regards,
> Bence

Re: Jira contributor access

[Pierre Villard]

Hi,

I reopened the pull request. I'm not very familiar with the code of those
processors so I'm not sure I'm well positioned to help with the review.
However I'd recommend trying to add unit tests that exhibit the issue and
demonstrate that this is solved by your code change. I understand this may
not be easy to write as a unit test but would definitely help.

Thanks,
Pierre

Le mer. 12 juil. 2023 à 09:35, 张美丽  a écrit :

> Hi,
> I am ZmlCode.
> I report the problem of https://issues.apache.org/jira/browse/NIFI-10847.
> But no one helped me review it for a long time.
>
>
> Recently, after reading the official documents, I know that I need to
> verify the current account by email first.
> So I came to the certification.
>
>
> My account is ZmlCode and email is zhangmeili_c...@163.com .Please.
> Thanks again to NIFI for this program, I learned a lot from it and helped
> me grow up a lot.
> I hope I can make a small contribution in the future. May be.
>
>
> Have a nice day!
> Best regards.
>

Re: Re: [discuss] nifi 2.0 and Java 17...

[Pierre Villard]

Hi,

There are still many things we want to do before an alpha release of NiFi
2.0. I'd not expect it before September at best.

Pierre

Le jeu. 6 juil. 2023, 21:15, Monteragi  a écrit :

> Hi,
>
> I tried but didn't find any estimates for Nifi 2.0 release. Could someone
> please let me know what's the approximate date of 2.0 release?
>
> Best Regards,
> Monty
>
>
> On 2023/06/19 15:55:51 David Handermann wrote:
> > Team,
> >
> > With the merge of PR 7397 [1] for NIFI-11717 [2], Java 17.0.6 is the
> > minimum required version for building the main branch.
> >
> > There are still several remaining deprecated features that need to be
> > removed for NiFi 2.0, and there are still areas of the system that need
> to
> > be reviewed for additional cleanup. The Deprecated Components and
> Features
> > page [3] lists the progress thus far.
> >
> > There is still opportunity to introduce new features and improvements in
> > parallel with the technical debt reduction focus, and this could include
> > evaluating a better strategy for supporting additional scripting engines.
> >
> > We should review the status of things after addressing some of the larger
> > outstanding deprecation removals.
> >
> > Regards,
> > David Handermann
> >
> > [1] https://github.com/apache/nifi/pull/7397
> > [2] https://issues.apache.org/jira/browse/NIFI-11717
> > [3]
> >
>
> https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features
> >
> > On Wed, Jun 7, 2023 at 10:34 PM Ryan Hendrickson <
> > ryan.andrew.hendrick...@gmail.com> wrote:
> >
> > > The major issue for our deployments would be the removal of Nashorn as
> > > well.
> > >
> > > Would GraalVM or an alternative be considered as a part of an initial
> NiFi
> > > 2.0 release?
> > >
> > > Thanks,
> > > Ryan
> > >
> > > On Mon, Jun 5, 2023 at 12:38 PM Joe Witt  wrote:
> > >
> > > > Team,
> > > >
> > > > Looking like we will update the NiFi 2.0 goals to be based on Java 17
> > > > instead of 11.
> > > >
> > > > The noted concern around Java removing Nashorn in 11/17 we will need
> to
> > > > identify an alternative plan for regardless and seems like David's
> > > proposal
> > > > would do the trick.
> > > >
> > > > Let's give this thread a few more days and if still seems consensus
> is
> > > > present lets just assume lazy consensus and update the NiFi 2.0 goals
> and
> > > > make it happen.
> > > >
> > > > Thanks
> > > >
> > > > On Fri, Jun 2, 2023 at 8:46 AM David Handermann <
> > > > exceptionfact...@apache.org>
> > > > wrote:
> > > >
> > > > > I agree that moving forward with Java 17 as the minimum for NiFi
> 2.0 is
> > > > the
> > > > > best approach given the extended lifecycle of support for Java 17.
> > > > >
> > > > > With the removal of a number of legacy components, the current main
> > > > branch
> > > > > is in a much better position to make Java 17 the minimum.
> > > > >
> > > > > The deprecation and removal of Nashorn from the JDK is worth
> > > > highlighting,
> > > > > but it should not be a blocker for moving to Java 17. In this case,
> > > NiFi
> > > > is
> > > > > reflecting the deprecation of Nashorn that already exists in Java
> 11. I
> > > > > have submitted a PR for NIFI-11630 to mark ECMAScript as deprecated
> for
> > > > the
> > > > > support branch in subsequent version 1 releases.
> > > > >
> > > > > With that background, there is ongoing maintenance of the Nashorn
> > > engine
> > > > as
> > > > > an external library, in addition to the GraalVM solution. However,
> this
> > > > is
> > > > > a good opportunity to take a different approach to scripting engine
> > > > > integration. For maintenance and security purposes, it would be
> much
> > > > better
> > > > > to reduce the number of bundled scripting engines and make it
> easier to
> > > > > bring your own. The current scripting bundle is around 100 MB,
> which
> > > is a
> > > > > lot of weight for languages and solutions that do not apply across
> the
> > > > > board. Providing an alternative that makes it easier to bring in a
> > > script
> > > > > engine library should provide a better solution for the future.
> This
> > > also
> > > > > should not be a blocker for an initial NiFi 2.0, but it is worth
> > > > > highlighting given the general interest in scripted components.
> > > > >
> > > > > Regards,
> > > > > David Handermann
> > > > >
> > > > > On Thu, Jun 1, 2023, 11:38 PM Dirk Arends 
> > > > > wrote:
> > > > >
> > > > > > Hi Joe,
> > > > > >
> > > > > > > Who will be seriously impacted by the removal of Java 11 and
> what
> > > was
> > > > > > your plan for upgrading to Java 17?
> > > > > >
> > > > > > >
> > > > > >
> > > > > > > thoughts?
> > > > > >
> > > > > > I would support moving the minimum Java version to 17 if it
> wasn’t
> > > for
> > > > > the
> > > > > > fact that Nashorn will be removed. Nashorn is already deprecated
> in
> > > > Java
> > > > > > 11, and was then fully removed in Java 15. I understand GraalVM
> is
> > > > > intended
> > > > > > to be its s

Re: [discuss] nifi 2.0 and Java 17...

[Pierre Villard]

Hey Joe,

I'd recommend doing the move right now and say that NiFi 2.0 requires Java
17. And we would focus on Java 21 with NiFi 3.0.
I honestly don't see any value in absolutely keeping support for Java 11
right now.

Pierre

Le mer. 31 mai 2023 à 19:22, Joe Witt  a écrit :

> Team,
>
> We've discussed in the past having NiFi 2.0 move from Java 8 to Java 11 as
> the required minimum while also working on Java 17.
>
> As we move on in time though we are now 4 months (Sept) from. Java 11
> openJDK going end of support.  Meanwhile, the Spring 5.x line goes end of
> support as of next year and Spring 6.x requires Java 17.  Also Java 21
> comes out in Sept as well and is already the next LTS release.
>
> I am increasingly of the view that we should seriously discuss/consider
> moving to Java 17 as our basis for NiFi 2.0 as otherwise it basically means
> we'll be forced to move to NiFi 3.0 quite quickly.
>
> We already know we can build and run on Java 17 so we're good there.  We'll
> soon want to do the same for Java 21 ... and the more 'old stuff' we hold
> on to the harder it is.
>
> Who will be seriously impacted by the removal of Java 11 and what was your
> plan for upgrading to Java 17?
>
> thoughts?
>
> Thanks
>