Re: Problem with NIFI registry using ssl certificates

[Nathan Gough]

7; defined in file
> [/home/netadmin/nifi-registry-1.19.0/work/jetty/nifi-registry-web-api-1.19.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/api/AccessPolicyResource.class]:
> Unsatisfied dependency expressed through constructor parameter 0; nested
> exception is
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name 'standardServiceFacade' defined in file
> [/home/netadmin/nifi-registry-1.19.0/work/jetty/nifi-registry-web-api-1.19.0.war/webapp/WEB-INF/classes/org/apache/nifi/registry/web/service/StandardServiceFacade.class]:
> Unsatisfied dependency expressed through constructor parameter 2; nested
> exception is
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error
> creating bean with name 'authorizationService' defined in URL
> [jar:file:/home/netadmin/nifi-registry-1.19.0/work/jetty/nifi-registry-web-api-1.19.0.war/webapp/WEB-INF/lib/nifi-registry-framework-1.19.0.jar!/org/apache/nifi/registry/service/AuthorizationService.class]:
> Unsatisfied dependency expressed through constructor parameter 1; nested
> exception is org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'getAuthorizer' defined in class path resource
> [org/apache/nifi/registry/security/authorization/AuthorizerFactory.class]:
> Bean instantiation via factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.apache.nifi.registry.security.authorization.Authorizer]: Factory
> method 'getAuthorizer' threw exception; nested exception is
> org.apache.nifi.registry.security.authorization.AuthorizerFactoryException:
> Failed to construct Authorizer.
> 2023-01-16 12:22:01,018 ERROR [NiFi logging handler]
> org.apache.nifi.registry.StdErr Shutting down...
> 2023-01-16 12:22:02,346 INFO [main]
> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is running at
> PID 234707 but is not responding to ping requests
> 2023-01-16 12:22:02,719 INFO [main]
> o.a.n.registry.bootstrap.RunNiFiRegistry NiFi Registry never started. Will
> not restart NiFi Registry
> 2023-01-16 12:22:06,990 INFO [main]
> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is not running
>
> ==> nifi-registry-event.log <==
>
>
> At the moment, nifi-registry its running over http only. With https
> persist this problem.
>
> Do you have another suggestion about this topic ?
>
> Best Regards.
>
> --
> *From: *"Nathan Gough" 
> *To: *"dev" 
> *Cc: *"ANTHONY YOSHIHITO ADACHI CORDERO" , "EDISON
> FABRICIO NARANJO ESPIN" 
> *Sent: *Thursday, January 12, 2023 12:52:58 PM
> *Subject: *Re: Problem with NIFI registry using ssl certificates
>
> It looks like NiFi Registry is already running on 127.0.0.1:19443?
> Attempts to start it again are showing failing to bind in the bootstrap log:
>
> 2023-01-12 11:02:58,113 ERROR [NiFi logging handler] 
> org.apache.nifi.registry.StdErr Failed to start web server: Failed to bind to 
> 127.0.0.1 :19443
>
>
> I suggest trying a different bind port or figuring out why you're unable
> to bind on the interface/address/hostname you've chosen. You might already
> have a registry instance running or some other service. If running linux
> you should be able to check with sudo lsof -i -P -n | grep 19443
>
> If the registry service is running but you're unable to access it, I would
> kill the process and try and start registry up again.
>
> As far as I can tell your authorizers.xml file is fine.
>
> Nathan
>
>
> On Thu, Jan 12, 2023 at 11:23 AM EDISON FABRICIO NARANJO ESPIN <
> efnara...@telconet.ec> wrote:
>
>> Dear Nathan,
>>
>> This is the output when nifi registry starts. The service stays active
>> for around 20 seconds and then stops working.
>>
>>
>> 1:06 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh start
>>
>> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
>> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>>
>> Bootstrap Config File:
>> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>>
>>
>> 11:07 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh status
>>
>> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
>> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>>
>> Bootstrap Config File:
>> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>>
>> 2023-01-12 11:07:08,260 INFO [main]
>> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is currently
>> running, listening to Bootstrap

Re: Problem with NIFI registry using ssl certificates

[Nathan Gough]

It looks like NiFi Registry is already running on 127.0.0.1:19443? Attempts
to start it again are showing failing to bind in the bootstrap log:

2023-01-12 11:02:58,113 ERROR [NiFi logging handler]
org.apache.nifi.registry.StdErr Failed to start web server: Failed to
bind to 127.0.0.1 :19443


I suggest trying a different bind port or figuring out why you're unable to
bind on the interface/address/hostname you've chosen. You might already
have a registry instance running or some other service. If running linux
you should be able to check with sudo lsof -i -P -n | grep 19443

If the registry service is running but you're unable to access it, I would
kill the process and try and start registry up again.

As far as I can tell your authorizers.xml file is fine.

Nathan


On Thu, Jan 12, 2023 at 11:23 AM EDISON FABRICIO NARANJO ESPIN <
efnara...@telconet.ec> wrote:

> Dear Nathan,
>
> This is the output when nifi registry starts. The service stays active for
> around 20 seconds and then stops working.
>
>
> 1:06 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh start
>
> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>
> Bootstrap Config File:
> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>
>
> 11:07 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh status
>
> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>
> Bootstrap Config File:
> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>
> 2023-01-12 11:07:08,260 INFO [main]
> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is currently
> running, listening to Bootstrap on port 42503, PID=15011
>
> 11:07 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh status
>
> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>
> Bootstrap Config File:
> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>
> 2023-01-12 11:07:12,839 INFO [main]
> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is currently
> running, listening to Bootstrap on port 42503, PID=15011
>
> 11:07 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh status
>
> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>
> Bootstrap Config File:
> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>
> 2023-01-12 11:07:17,734 INFO [main]
> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is currently
> running, listening to Bootstrap on port 42503, PID=15011
>
> 11:07 edinaranjoespin@EFNARANJO-LT:bin $./nifi-registry.sh status
>
> Java home: /usr/lib/jvm/java-11-openjdk-amd64/
> NiFi Registry home: /home/edinaranjoespin/NiFi/nifi-registry-1.19.0
>
> Bootstrap Config File:
> /home/edinaranjoespin/NiFi/nifi-registry-1.19.0/conf/bootstrap.conf
>
> 2023-01-12 11:07:24,044 INFO [main]
> o.apache.nifi.registry.bootstrap.Command Apache NiFi Registry is not running
>
>
> Additionally, I send the logs and the files nifi-registry.properties and
> authorizers.xml
>
> Best Regards,
> --
> *From: *"Nathan Gough" 
> *To: *"dev" 
> *Cc: *"ANTHONY YOSHIHITO ADACHI CORDERO" , "EDISON
> FABRICIO NARANJO ESPIN" 
> *Sent: *Thursday, January 12, 2023 10:03:08 AM
> *Subject: *Re: Problem with NIFI registry using ssl certificates
>
> Those parts of the config look fine.
> Can you share more of the failure log message and/or your
> nifi-registry.properties file?
>
>
>
> On Thu, Jan 12, 2023, 9:34 AM EDISON FABRICIO NARANJO ESPIN <
> efnara...@telconet.ec> wrote:
>
>> Dear Nathan
>>
>> This is the configuration
>>
>> nifi.registry.web.http.host=
>>  nifi.registry.web.http.port=
>>  nifi.registry.web.https.host=127.0.0.1
>>  nifi.registry.web.https.port=19443
>>
>>
>> Best Regards,
>> --
>> *From: *"Nathan Gough" 
>> *To: *"dev" 
>> *Cc: *"EDISON FABRICIO NARANJO ESPIN" , "ANTHONY
>> YOSHIHITO ADACHI CORDERO" 
>> *Sent: *Wednesday, January 11, 2023 6:35:23 PM
>> *Subject: *Re: Problem with NIFI registry using ssl certificates
>>
>> Hi Edison,
>>
>> It sounds like your nifi-registry.properties file may have issues. Can
>> you share this section of configuration nifi.registry.web.https.host=?
>> nifi.registry.web.https.port=?
>>
>> This guide should be able to help:
>>
>> h

Re: Problem with NIFI registry using ssl certificates

[Nathan Gough]

Those parts of the config look fine.
Can you share more of the failure log message and/or your
nifi-registry.properties file?



On Thu, Jan 12, 2023, 9:34 AM EDISON FABRICIO NARANJO ESPIN <
efnara...@telconet.ec> wrote:

> Dear Nathan
>
> This is the configuration
>
> nifi.registry.web.http.host=
>  nifi.registry.web.http.port=
>  nifi.registry.web.https.host=127.0.0.1
>  nifi.registry.web.https.port=19443
>
>
> Best Regards,
> ------
> *From: *"Nathan Gough" 
> *To: *"dev" 
> *Cc: *"EDISON FABRICIO NARANJO ESPIN" , "ANTHONY
> YOSHIHITO ADACHI CORDERO" 
> *Sent: *Wednesday, January 11, 2023 6:35:23 PM
> *Subject: *Re: Problem with NIFI registry using ssl certificates
>
> Hi Edison,
>
> It sounds like your nifi-registry.properties file may have issues. Can you
> share this section of configuration nifi.registry.web.https.host=?
> nifi.registry.web.https.port=?
>
> This guide should be able to help:
>
> https://community.cloudera.com/t5/Community-Articles/Setting-Up-a-Secure-Apache-NiFi-Registry/ta-p/247753
> <https://fm.telconet.net/fmlurlsvc/?fewReq=:B:JVs5MjYyOSV1PjEtMyVqZz4zMjkzMiVwamRtYnd2cWY+MTE2NDUzOjdnNjE3OjpiMDU1ZzFhZjY1MjdgZTU2YWdhNjUwZmA7ZyV3PjI1NDA3OzMyNTslcmpnPjAzQU1iNE9iMzE6NTE2LjAzQU1iNE9gMzE6NTE2JXFgc3c+ZmVtYnFibWlsQ3dmb2BsbWZ3LWZgJWA+NjEla2dvPjM=&url=https%3a%2f%2fcommunity.cloudera.com%2ft5%2fCommunity-Articles%2fSetting-Up-a-Secure-Apache-NiFi-Registry%2fta-p%2f247753>
>
> There may be more exception information you can share with us that's
> above/below the message you provided.
>
> Nathan
>
>
> On Wed, Jan 11, 2023, 6:21 PM EDISON FABRICIO NARANJO ESPIN <
> efnara...@telconet.ec> wrote:
>
>> Dear,
>>
>> After configuring the security parameters in the nifi registry, its
>> operation cannot be started since the logs indicate that the jetty web
>> server could not be started. Is there a solution for this issue or you must
>> work with a special version of the product so that it can be deployed with
>> https.
>>
>> Attached log output
>>
>> ==> nifi-registry-app_2023-01-11_12.0.log <==
>> at
>> org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
>> ... 9 common frames omitted
>> Caused by: java.nio.channels.UnresolvedAddressException: null
>> at java.base/sun.nio.ch.Net.checkAddress(Net.java:131)
>> at
>> java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
>> at
>> java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
>> ... 10 common frames omitted
>> 2023-01-11 12:56:44,477 INFO [Thread-0]
>> org.apache.nifi.registry.NiFiRegistry Initiating shutdown of Jetty web
>> server...
>> 2023-01-11 12:56:44,479 INFO [Thread-0]
>> o.eclipse.jetty.server.AbstractConnector Stopped 
>> ServerConnector@19e4653c{SSL,
>> (ssl, http/1.1)}{localhost :18443}
>> 2023-01-11 12:56:44,479 INFO [Thread-0] org.eclipse.jetty.server.session
>> node0 Stopped scavenging
>>
>>
>> Best regards,
>> --
>> Edison F. Naranjo E.
>> Seguridad Lógica
>> TELCONET LATAM
>> Cel: +593998608233
>> Quito-Ecuador
>> efnara...@telconet.ec
>>
>> Toda la información contenida en este correo electrónico es confidencial
>> y podrá ser usada únicamente por los destinatarios. No imprimir a menos que
>> sea imprescindible.
>>
>>
>
>
> --
> Edison F. Naranjo E.
> Seguridad Lógica
> TELCONET LATAM
> Quito-Ecuador
> efnara...@telconet.ec
>
>
> Toda la información contenida en este correo electrónico es confidencial y
> podrá ser usada únicamente por los destinatarios. No imprimir a menos que
> sea imprescindible.
>
>

Re: Problem with NIFI registry using ssl certificates

[Nathan Gough]

Hi Edison,

It sounds like your nifi-registry.properties file may have issues. Can you
share this section of configuration nifi.registry.web.https.host=?
nifi.registry.web.https.port=?

This guide should be able to help:
https://community.cloudera.com/t5/Community-Articles/Setting-Up-a-Secure-Apache-NiFi-Registry/ta-p/247753

There may be more exception information you can share with us that's
above/below the message you provided.

Nathan


On Wed, Jan 11, 2023, 6:21 PM EDISON FABRICIO NARANJO ESPIN <
efnara...@telconet.ec> wrote:

> Dear,
>
> After configuring the security parameters in the nifi registry, its
> operation cannot be started since the logs indicate that the jetty web
> server could not be started. Is there a solution for this issue or you must
> work with a special version of the product so that it can be deployed with
> https.
>
> Attached log output
>
> ==> nifi-registry-app_2023-01-11_12.0.log <==
> at
> org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
> ... 9 common frames omitted
> Caused by: java.nio.channels.UnresolvedAddressException: null
> at java.base/sun.nio.ch.Net.checkAddress(Net.java:131)
> at
> java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
> at
> java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
> ... 10 common frames omitted
> 2023-01-11 12:56:44,477 INFO [Thread-0]
> org.apache.nifi.registry.NiFiRegistry Initiating shutdown of Jetty web
> server...
> 2023-01-11 12:56:44,479 INFO [Thread-0]
> o.eclipse.jetty.server.AbstractConnector Stopped ServerConnector@19e4653c{SSL,
> (ssl, http/1.1)}{localhost :18443}
> 2023-01-11 12:56:44,479 INFO [Thread-0] org.eclipse.jetty.server.session
> node0 Stopped scavenging
>
>
> Best regards,
> --
> Edison F. Naranjo E.
> Seguridad Lógica
> TELCONET LATAM
> Cel: +593998608233
> Quito-Ecuador
> efnara...@telconet.ec
>
> Toda la información contenida en este correo electrónico es confidencial y
> podrá ser usada únicamente por los destinatarios. No imprimir a menos que
> sea imprescindible.
>
>

Re: [VOTE] Adopt NiFi 2.0 Proposed Release Goals

[Nathan Gough]

+1 (binding)

The proposal sounds great to me



On Wed, Dec 14, 2022, 1:01 PM Edward Armes  wrote:

> -1 (non-binding)
>
> Im not sure if this is covered by 8 and 5 , but I would like to suggest
> that as part of 2.0 focus on removing places were concrete implementations
> are used over interfaces, and update the way the website docs are generated
> to ensure that NARs that not included in the standard distribution are
> included.
>
> I think this would allow for work of a NAR registry to happen outside of a
> major release and allow for the potential consideration of individually
> versions components reducing the sizes of future releases.
>
> Edward
>
>
> On Mon, 12 Dec 2022, 20:32 Andrew Lim,  wrote:
>
> > +1 (binding)
> >
> > > On Dec 12, 2022, at 12:02 PM, David Handermann <
> > exceptionfact...@apache.org> wrote:
> > >
> > > Team,
> > >
> > > Following positive feedback on NiFi 2.0 Proposed Release Goals [1] on
> the
> > > recent discussion thread [2], I am calling this vote to adopt the
> > following
> > > as Release Goals for NiFi 2.0:
> > >
> > > 1. Remove Java 8 support and require Java 11
> > > 2. Remove deprecated components
> > > 3. Remove deprecated component properties
> > > 4. Remove components integrating with unmaintained services
> > > 5. Remove compatibility classes and methods
> > > 6. Remove flow.xml.gz in favor of flow.json.gz
> > > 7. Remove duplicative features
> > > 8. Upgrade internal Java API references
> > > 9. Reorganize standard components
> > > 10. Implement migration tools for upgrading flows
> > >
> > > A positive vote indicates agreement on these goals and the initiation
> of
> > > the following actions:
> > >
> > > 1. Rename NiFi 2.0 Proposed Release Goals to NiFi 2.0 Release Goals
> > > 2. Create version 1 branch in Git for subsequent support releases on
> the
> > > version 1 series
> > > 3. Update the current main branch in Git to version 2.0.0-SNAPSHOT
> > >
> > > The vote will be open for 72 hours and follow standard procedures for
> > > release votes.
> > >
> > > Please review the linked goals and discussions for background.
> > >
> > > [ ] +1 Adopt NiFi 2.0 Release Goals
> > > [ ] +0 No opinion
> > > [ ] -1 Do not adopt NiFi 2.0 Release Goals for the following reasons...
> > >
> > > [1]
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Proposed+Release+Goals
> > > [2] https://lists.apache.org/thread/xo77p9t3xg4k70356xrqbdg4m9sg7sf8
> >
> >
>

Re: [VOTE] Release Apache NiFi 1.19.1 (RC2)

[Nathan Gough]

+1 binding

Verified functionality on 3 node NiFi cluster and test flows
Verified X509 login for NIFi
Verified OIDC login/logout in NiFi and NiFi Registry
Verified LDAP login/logout for NiFi and NiFi Registry


On Tue, Dec 6, 2022 at 5:56 PM Nandor Soma Abonyi
 wrote:

> +1 (non-binding)
>
> - Went through the release helper guide.
> - Tested interaction with NiFi registry.
> - Ran a simple flow to verify NIFI-10785.
> - Verified NIFI-10872.
>
> Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
> Java version: 1.8.0_332, vendor: Temurin
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.16", arch: "x86_64", family: "mac"
>
> Thanks for RM'ing Joe!
> Regards,
> Soma
>
> > On Dec 5, 2022, at 8:57 PM, Joe Witt  wrote:
> >
> > Hello,
> >
> > Please note this is intended as a shorter than normal release cycle
> > given the limited scope and the importance of getting some of these
> > fixes to those already trying 1.19.  If we need to extend due to lack
> > of vote participation we will.
> >
> > I am pleased to be calling this vote for the source release of Apache
> > NiFi 1.19.1.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1218
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.19.1/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.19.1-RC2
> > The Git commit ID is a7236ecc9123113ba5b9aaa3baab06354778116f
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=a7236ecc9123113ba5b9aaa3baab06354778116f
> >
> > Checksums of nifi-1.19.1-source-release.zip:
> > SHA256: 1172b133096c9d88b185413afc1eb6220d3f0542df6cdf82ceb7b01ae6fa15ed
> > SHA512:
> 3b571c9151bb755653f5e5ca3c275813134572c0fc28d8730a0dc5c1390df118b2be1f013f9acc0e394a959ff5dc853402f2c03d8f504010d6b835b43d786784
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 36 issues were closed/resolved for this release:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12352626
> >
> > Release note highlights can be found here:
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.19.1
> >
> > The vote will be open for 36 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build
> > from source, and test. Then please vote:
> >
> > [ ] +1 Release this package as nifi-1.19.1
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
>
>

Re: [VOTE] Release Apache NiFi 1.19.1 (RC1)

[Nathan Gough]

I'll take a look at the PR you submitted for it

On Fri, Dec 2, 2022 at 4:50 PM Mark Payne  wrote:

> -1 (binding)
>
> In doing some testing, I encountered a couple of bugs. Most were minor,
> but I would consider NIFI-10937 [1] critical enough to sink the release.
> It results in a failure to startup nifi with a flow.xml.gz unless the
> flow.json.gz is present. The fact that we write out both of them is a
> safety valve,
> just in case there’s an issue with the newer flow.json.gz. I think it’s
> very important to ensure that this works as expected.
>
> Thanks
> -Mark
>
>
> [1] https://issues.apache.org/jira/browse/NIFI-10937
>
> > On Dec 1, 2022, at 4:56 PM, Joe Witt  wrote:
> >
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> > NiFi 1.19.1.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1217
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.19.1/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.19.1-RC1
> > The Git commit ID is 16ba0c2a5a61e98c5a73769976d8589932bfd43d
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=16ba0c2a5a61e98c5a73769976d8589932bfd43d
> >
> > Checksums of nifi-1.19.1-source-release.zip:
> > SHA256: 0d88e02a00b3fca4ee4c453788caf3ea1cc8d8f3be7ee2e2bd395ce25932bd7e
> > SHA512:
> 861121c17a67020d34f3457d047c3c0486e3c6bd6392bf355f8bdf6f3eed01e7ea441f1175e8232c354a9ebf0c2f709ddf846e694bc2480305d180b0a692ec49
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 26 issues were closed/resolved for this release:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12352626
> >
> > Release note highlights can be found here:
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.19.1
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build
> > from source, and test. Then please vote:
> >
> > [ ] +1 Release this package as nifi-1.19.1
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
>
>

Re: [discuss] NiFi 1.19.0

[Nathan Gough]

We might also want https://issues.apache.org/jira/browse/NIFI-10787

On Tue, Nov 15, 2022 at 4:05 PM Mark Bean  wrote:

> I will be on it in about 2 hours, if not addressed sooner.
>
> On Tue, Nov 15, 2022 at 3:46 PM Joe Witt  wrote:
>
> > NIFI-10703 would be great to get in.  Just a matter of gettin' it done
> and
> > merged I think.  You're on it?
> >
> > Thanks
> >
> > On Tue, Nov 15, 2022 at 1:38 PM Mark Bean  wrote:
> >
> > > Sorry.. check that. There's a typo in the latest commit on that PR.
> > >
> > > On Tue, Nov 15, 2022 at 3:35 PM Mark Bean 
> wrote:
> > >
> > > > Could we get NIFIDEVS-10703 (PR #6638) in there too? AFAIK, it's good
> > to
> > > > go.
> > > >
> > > > Thanks,
> > > > Mark
> > > >
> > > >
> > > > On Tue, Nov 15, 2022 at 2:56 PM Joe Witt  wrote:
> > > >
> > > >> Hello All,
> > > >>
> > > >> I plan to kick off the RC as soon as
> > > >> https://issues.apache.org/jira/browse/NIFI-10701 is merged.
> > > >>
> > > >> Thanks
> > > >>
> > > >> On Thu, Nov 10, 2022 at 12:52 PM David Handermann <
> > > >> exceptionfact...@apache.org> wrote:
> > > >>
> > > >> > Joe,
> > > >> >
> > > >> > Thanks for initiating the discussion. I agree with moving toward a
> > > >> 1.19.0
> > > >> > release instead of 1.18.1 given over 150 Jira issues already
> tagged
> > > with
> > > >> > 1.19.0.
> > > >> >
> > > >> > We should start another thread soon regarding 2.0. One of the key
> > > >> things to
> > > >> > do before that, however, would be to finish adding deprecation
> > logging
> > > >> to
> > > >> > remaining features we plan to remove in 2.0. Having a minor
> release
> > in
> > > >> > version 1 series deprecating targets for removal should be a
> > > >> prerequisite
> > > >> > before branching and starting on 2.0.
> > > >> >
> > > >> > In the meantime, if that means we plan on a 1.20.0 release, I am
> in
> > > >> favor
> > > >> > of moving forward with 1.19.0.
> > > >> >
> > > >> > Regards,
> > > >> > David Handermann
> > > >> >
> > > >> > On Thu, Nov 10, 2022 at 1:44 PM Joe Witt 
> > wrote:
> > > >> >
> > > >> > > Team,
> > > >> > >
> > > >> > > We keep getting a slew of emails on various lists, slack, and
> the
> > > >> > security
> > > >> > > list about commons text.  We need a 1.18.1 or a 1.19.0.
> > > >> > >
> > > >> > > Given all that has already happened on 1.19.0 I'm inclined to
> just
> > > 'do
> > > >> > > that' release and get it going.
> > > >> > >
> > > >> > > I know we need to start getting serious about 2.0 but perhaps
> the
> > > most
> > > >> > > realistic path for that is we branch and come back to main after
> > we
> > > >> get
> > > >> > > that figured out.  But we need to keep moving forward in the
> > > meantime.
> > > >> > >
> > > >> > > Please share if there is anything we know about on the main line
> > > which
> > > >> > > would make doing a 1.19.0 problematic.
> > > >> > >
> > > >> > > https://issues.apache.org/jira/projects/NIFI/versions/12352345
> > > >> > >
> > > >> > > Thanks
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: [VOTE] Release Apache NiFi 1.18.0 (RC4)

[Nathan Gough]

Hi,

I reverified the hashes and compiled from source with OpenJDK Runtime
Environment Zulu11.58+15-CA.

Ran a test cluster again and verified my cluster test flow still works as
expected.

+1 binding

Nathan

On Mon, Oct 3, 2022 at 4:45 PM Joe Witt  wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi 1.18.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1214
>
> The source being voted upon and the convenience binaries can be found at:
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.18.0/
>
> A helpful reminder on how the release candidate verification process works:
>
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
>
> The Git tag is nifi-1.18.0-RC4
> The Git commit ID is 109e54cd585902a981d1b370b3dc4d1620be438c
>
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=109e54cd585902a981d1b370b3dc4d1620be438c
>
> Checksums of nifi-1.18.0-source-release.zip:
> SHA256: 925cbb92c107d0fa3194a349d985cff4933a61b2555eff57b1b81433fe37c139
> SHA512:
> f143215b1746342e7584f5ad65b546fcc378cd78aa17628fb605dfdbbaf11e897a0173dd67807fc90cb18c17124a4227d5fe07e7ed609d9ed1904503b757c604
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/joewitt.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 184 issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12352150
>
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.18.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test. Then please vote:
>
> [ ] +1 Release this package as nifi-1.18.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>

Re: [VOTE] Release Apache NiFi 1.18.0 (RC3)

[Nathan Gough]

+1 (binding)

Running with openjdk version "11.0.16" 2022-07-19 LTS:

I tested this RC with a 3 node cluster with an external Zookeeper, and a
test flow and found things to be working fine.
Also tested authentication with LDAP, SAML, OIDC and X509.

Thanks for the release work!
Nathan

On Fri, Sep 30, 2022 at 10:06 PM Mark Bean  wrote:

> Downloaded source and verified keys and signatures
> Performed full build including contrib-check using:
>   OpenJDK Runtime Environment (build
> 1.8.0_342-8u342-b07-0ubuntu1~22.04-b07)
>   OpenJDK Runtime Environment (build 11.0.16+8-post-Ubuntu-0ubuntu122.04)
>   OpenJDK Runtime Environment (build 17.0.4+8-Ubuntu-122.04)
> Installed and ran using Java 11 JRE on Ubuntu 22.04 and all NiFi defaults.
> Executed simple flow. No issues observed.
>
> +1, non-binding
>
>
> On Thu, Sep 29, 2022 at 3:22 PM Joe Witt  wrote:
>
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> > NiFi 1.18.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1213
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.18.0/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.18.0-RC3
> > The Git commit ID is 5bc64c812b2c76ee2879d8081ceadf62d5e3c702
> >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=5bc64c812b2c76ee2879d8081ceadf62d5e3c702
> >
> > Checksums of nifi-1.18.0-source-release.zip:
> > SHA256: bd1b675f17dbf712089a79f7bc043eae2df63bcc2e08b2012a6431641037679f
> > SHA512:
> >
> cea43af57089128ff65bb53e76b2fdfa8dec7397e2bf45d41e35b758b731355075839b9c018ee6284cb15e293b105e248d88748148960ad80ae387824139f52b
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 171 issues were closed/resolved for this release:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12352150
> >
> > Release note highlights can be found here:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.18.0
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build
> > from source, and test. Then please vote:
> >
> > [ ] +1 Release this package as nifi-1.18.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
>

Re: [VOTE] Release Apache NiFi 1.17.0 (RC2)

[Nathan Gough]

- Tested OIDC with G Suite
- Tested SAML with G Suite
- Tested LDAP
- Tested insecure mode (boo)
- Tested using an X509 secure 3 node cluster, sending data across nodes
with RPG and PostHTTP/ListenHTTP, a Jolt transform.
- Verified that controller services and reporting tasks are starting and
stopping as expected
- Tested distributed map cache client and server

+1 (non-binding)

Cheers for the release,
Nathan

On Thu, Jul 28, 2022 at 2:56 PM Csaba Bejan  wrote:

> +1 (non-binding)
>
> - Went through the helper guide
> - Verified signatures and hashes
> - Built on OSX 11.6.6
> - AdoptOpenJDK (build 1.8.0_282-b08)
> - Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
> - Verified that MiNiFi starts up and published a basic flow via the C2
> protocol which was picked up and executed by MiNiFI as expected.
> - Verified no C2 libraries present in root library directory
>
> Thanks,
> Csaba
>
> On Thu, Jul 28, 2022 at 4:28 PM Joe Witt  wrote:
>
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.17.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1210
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.17.0/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.17.0-RC2
> > The Git commit ID is 8d256784d84cc28bf5642e1bf38dec3eba0c5f23
> >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=8d256784d84cc28bf5642e1bf38dec3eba0c5f23
> >
> > Checksums of nifi-1.17.0-source-release.zip:
> > SHA256: 8b9b2088ad966329248cfae7792f576f4f30fea4b4e50f055f84724dba4fe8a3
> > SHA512:
> >
> >
> 2429348ad514ca7ab9df86aaa57207f1434044c6f7e947d0950ca9826b4f1aa51061617a17444c086eed19b1f26a5ebbe3b455cafed9d219727adf26ecb5f8d2
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 310 issues were closed/resolved for this release:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12351438
> >
> > Release note highlights can be found here:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.17.0
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-1.17.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
>

Re: [VOTE] Release Apache NiFi 1.16.3 (RC1)

[Nathan Gough]

+1 (non-binding)

Ran a 3 node cluster with external Zookeeper, ran some test flows.

Thanks!
Nathan

On Tue, Jun 14, 2022 at 12:31 PM David Handermann <
exceptionfact...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures and hashes
> - Ran build using Maven 3.8.5
> - Ran build on Ubuntu 22.04 with Azul Zulu JDK 1.8.0-332
> - Ran build on Ubuntu 22.04 with Azul Zulu JDK 11.0.15
> - Ran build on Ubuntu 22.04 with Azul Zulu JDK 17.0.3
> - Ran system tests on Azul Zulu JDK 17.0.3
> - Ran stateless system tests on Azul Zulu JDK 17.0.3
>
> - Ran NiFi on Azul Zulu JDK 1.8.0-332
> - NIFI-10086 Verified upgrade to Spring Framework 5.3.20
> - NIFI-10096 Verified correct handling of inherited Parameter Contexts
> - NIFI-10098 Verified upgrade to Apache Tika 2.4.0
> - NIFI-10114 Verified authorization configuration ShellUserGroupProvider
>
> - Ran NiFi Registry on Azul Zulu JDK 1.8.0-322
> - Created Buckets
>
> - Ran NiFi Encrypt Config Toolkit with AES-GCM
>
> Thanks Joe!
>
> Regards,
> David Handermann
>
> On Mon, Jun 13, 2022 at 11:39 PM Joe Witt  wrote:
>
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.16.3.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1205
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.16.3/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.16.3-RC1
> > The Git commit ID is e15bdd7e3d09047d5fed70117b7c3dfd26f3a36e
> >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=e15bdd7e3d09047d5fed70117b7c3dfd26f3a36e
> >
> > Checksums of nifi-1.16.3-source-release.zip:
> > SHA256: c18edf739361246fe22bb4c2e5a4b1936b6199512b78638868f99b1d827b4d9e
> > SHA512:
> >
> >
> e3c9942737a0c2cf43fb3030da3cbee7d6be17038f0cf683c9522db25eb6d8664884594d5a7ce2183733568c06b9ccb52c3a6bf6d5ddcb334d2f84477cc68177
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 13 issues were closed/resolved for this release:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12351844
> >
> > Release note highlights can be found here:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.16.3
> >
> > The vote will be open for 24 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-1.16.3
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
>

Re: [VOTE] Release Apache NiFi 1.16.2 (RC3)

[Nathan Gough]

+1 (non-binding)

Tested out 3 node secure cluster and external ZK with a few different
processors, set up and tested Google OIDC authentication.

Nathan

On Tue, May 24, 2022 at 5:54 PM Mark Payne  wrote:

> +1 (binding)
>
> Performed build with Java 8.
> Ran with Java 8 and Java 11.
> Tested insecure standalone installation
> Tested secured (via certificate) clustered installation
>
> Did heavy testing against controller services, restarts, trying to
> introduce corner cases.
> Validated that NIFI-10001 was addressed.
>
> I did encounter a couple of bugs that I created Jiras for but none were
> critical and none were regressions introduced in this release.
>
> Thanks for handling the RM duties again Joe!
>
> -Mark
>
>
> > On May 22, 2022, at 11:44 PM, Joe Witt  wrote:
> >
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.16.2.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1203
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.16.2/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.16.2-RC3
> > The Git commit ID is 06f04958272dafc30ce357c4c4edcaf470050b52
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=06f04958272dafc30ce357c4c4edcaf470050b52
> >
> > Checksums of nifi-1.16.2-source-release.zip:
> > SHA256: 1fecf7d9f6001cc8e58d4a46ece08e141de705bcd227338ba79e9cb574267415
> > SHA512:
> >
> 1f4fd4e5e9f24949830a75949b302a67b8826049406ab8296c4b8c99a5a0aa1d211f84f98699b3af6fb41efa305f35a3f85b21d7958dc09c027cc1ed836c169f
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 34 issues were closed/resolved for this release:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12351721
> >
> > Release note highlights can be found here:
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.16.2
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-1.16.2
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
>
>

Re: [VOTE] Release Apache NiFi 1.16.1

[Nathan Gough]

+1 (non-binding), ran a 3 node cluster using external ZK with some simple
test flows. Ran NiFi with Google OIDC auth. Checked hashes and other files.

On Wed, Apr 27, 2022 at 10:24 AM Mike Thomsen 
wrote:

> +1 (binding) Ran a simple tika extraction flow on PDFs and DOCX files.
> Worked fine. Saved that flow into the NiFi Registry.
>
> On Tue, Apr 26, 2022 at 4:47 PM Matt Burgess  wrote:
> >
> > +1 (binding)
> >
> > Ran through release helper, tested various flows including version
> > control with the Registry, InvokeScriptedProcessor, Relationship
> > retry, etc.  I did create [1] for MiNiFi but since there's a
> > workaround it's not a dealbreaker IMO, I'm already working on it for
> > 1.17.0.
> >
> > Thanks for RM'ing Joe!
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-9967
> >
> > On Mon, Apr 25, 2022 at 2:29 PM Joe Witt  wrote:
> > >
> > > Hello,
> > >
> > > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > > 1.16.1.
> > >
> > > The source zip, including signatures, digests, etc. can be found at:
> > > https://repository.apache.org/content/repositories/orgapachenifi-1200
> > >
> > > The source being voted upon and the convenience binaries can be found
> at:
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.16.1/
> > >
> > > A helpful reminder on how the release candidate verification process
> works:
> > >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> > >
> > > The Git tag is nifi-1.16.1-RC1
> > > The Git commit ID is 81166797e552b9d14b482807632f2f04321b2018
> > >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=81166797e552b9d14b482807632f2f04321b2018
> > >
> > > Checksums of nifi-1.16.1-source-release.zip:
> > > SHA256:
> 2c39b45ba8eec42d601d5c9facad623d14660dd209c011b4a13b2b559b9f3e32
> > > SHA512:
> > >
> cd670ab558937cac709dea0b4be3351559f57c9e9aedf54d5153706eee386a021262ef199e2bf9485763cf955931edfd6a24ca1c5a0748a77e3eeb91c490cbda
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/joewitt.asc
> > >
> > > KEYS file available here:
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > 83 issues were closed/resolved for this release:
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12351504
> > > Release note highlights can be found here:
> > >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.16.1
> > >
> > > The vote will be open for 72 hours.
> > > Please download the release candidate and evaluate the necessary items
> > > including checking hashes, signatures, build from source, and test.
> Then
> > > please vote:
> > >
> > > [ ] +1 Release this package as nifi-1.16.1
> > > [ ] +0 no opinion
> > > [ ] -1 Do not release this package because...
>

CVE-2022-26850: Apache NiFi: Insufficiently protected credentials

[Nathan Gough]

Severity: moderate

Description:

When creating or updating credentials for single-user access, NiFi
wrote a copy of the Login Identity Providers configuration to the
operating system temporary directory. On most platforms, the operating
system temporary directory has global read permissions. NiFi
immediately moved the temporary file to the final configuration
directory, which significantly limited the window of opportunity for
access.

This issue is being tracked as NIFI-9785

Mitigation:

NiFi 1.16.0 includes updates to replace the Login Identity Providers
configuration without writing a file to the operating system temporary
directory.

Credit:

This issue was discovered by Jonathan Leitschuh
(https://twitter.com/jlleitschuh)

References:
https://nifi.apache.org/security.html#CVE-2022-26850

Re: [VOTE] Release Apache NiFi 1.16.0 (rc3)

[Nathan Gough]

+1 (non-binding), looks good to me.

Ran through the release helper, tested a few flows, tested a secure cluster
with external ZK, and tested OIDC auth.

On Tue, Mar 22, 2022 at 5:59 PM Matt Burgess  wrote:

> +1 Release this package as nifi-1.16.0
>
> Ran through release helper, ran a couple flows through NiFi including
> one for PutDatabaseRecord (to validate the changes), verified MiNiFi
> started and autogenerated the sensitive propert(ies), verified H2
> upgrade on NiFi Registry.
>
> Thanks for RM'ing Joe!
>
> On Mon, Mar 21, 2022 at 5:42 PM Joe Witt  wrote:
> >
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.16.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1198
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.16.0/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.16.0-RC3
> > The Git commit ID is b019a9191f1c83bc7f547dc02c1b679b8936acee
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=b019a9191f1c83bc7f547dc02c1b679b8936acee
> >
> > Checksums of nifi-1.16.0-source-release.zip:
> > SHA256: 2f16cb94df404d1bcc9c32835ba98da8940005a5d7ea5504c155ee42021a221e
> > SHA512:
> >
> cbd95f15cec5ffe506fef204526267b18b77d7266f6dc3e1bbc3c7600aac12e119977f7d8cf93dbbbc86fbb0739ba88aaa11a5381d29a463ec9a0c9a18f4e9e6
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 401 issues were closed/resolved for this release:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12350741
> >
> > Release note highlights can be found here:
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.16.0
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-1.16.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
>

Re: [ANNOUNCE] New Apache NiFi Committer Paul Grey

[Nathan Gough]

Congrats, Paul! Thanks for your contributions so far.

On Wed, Mar 16, 2022 at 9:06 PM Marton Szasz  wrote:

> Congratulations, Paul!
>
> On Thu, 17 Mar 2022 at 00:00, Joe Witt  wrote:
> >
> > Congrats and thanks!
> >
> > On Wed, Mar 16, 2022 at 4:55 PM gre...@yahoo.com.INVALID
> >  wrote:
> >
> > > Thanks much!  Next step is to do something about this "yahoo.com"
> email
> > > address...
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wednesday, March 16, 2022, 06:46:02 PM EDT, David Handermann <
> > > exceptionfact...@apache.org> wrote:
> > >
> > >
> > >
> > >
> > >
> > > Apache NiFi community,
> > >
> > > On behalf of the Apache NiFi PMC, I am very pleased to announce that
> Paul
> > > Grey
> > > has accepted the PMC's invitation to become a committer on the Apache
> NiFi
> > > project.
> > >
> > > Paul has contributed a number of pull requests and code reviews over
> the
> > > past year, improving project security and test stability in a number of
> > > areas. We appreciate Paul's work and look forward to continued
> > > contributions!
> > >
> > > Welcome Paul, and congratulations!
> > >
>

Re: 1.16.0 RC1 checkstyle issue

[Nathan Gough]

I also ran into both of these issues. Though it was just a me problem.

The value in my build.properties is:
Build-Timestamp:${timestamp}

On Fri, Mar 11, 2022 at 6:03 PM Bryan Bende  wrote:

> Mike,
>
> What is the value of timestamp in your
> nifi-manifest/nifi-runtime-manifest/target/classes/build.properties ?
>
> On Fri, Mar 11, 2022 at 5:53 PM Mike Thomsen 
> wrote:
>
> > https://issues.apache.org/jira/browse/NIFI-9791
> >
> > On Fri, Mar 11, 2022 at 5:51 PM Mike Thomsen 
> > wrote:
> > >
> > > Will do
> > >
> > > On Fri, Mar 11, 2022 at 3:41 PM Joe Witt  wrote:
> > > >
> > > > just got word of a bug in some intensive testing so will be sinking
> the
> > > > release anyway.  Please file a JIRA for what you found.
> > > >
> > > > Thanks
> > > >
> > > > On Fri, Mar 11, 2022 at 1:27 PM Mike Thomsen  >
> > wrote:
> > > >
> > > > > I'm getting this build error locally in the nifi-runtime-manifest
> > > > > module. Going to abstain from voting for a little while to see if
> > > > > anyone has the bug since this machine is loaded up with stuff that
> > > > > sometimes makes builds break for me when they don't for others.
> Error
> > > > > is:
> > > > >
> > > > > [INFO] --- exec-maven-plugin:1.6.0:java
> (generate-runtime-manifest) @
> > > > > nifi-runtime-manifest ---
> > > > >
> > > > > Mar 11, 2022 3:24:56 PM
> > > > > org.apache.nifi.runtime.manifest.impl.RuntimeManifestGenerator main
> > > > >
> > > > > INFO: Writing runtime manifest to:
> > > > >
> > > > >
> >
> /private/tmp/nifi-1.16.0/nifi-manifest/nifi-runtime-manifest/target/classes/nifi-runtime-manifest.json
> > > > >
> > > > > [WARNING]
> > > > >
> > > > > java.lang.NumberFormatException: For input string: "${timestamp}"
> > > > >
> > > > > at java.lang.NumberFormatException.forInputString
> > > > > (NumberFormatException.java:65)
> > > > >
> > > > > at java.lang.Long.parseLong (Long.java:678)
> > > > >
> > > > > at java.lang.Long.valueOf (Long.java:1144)
> > > > >
> > > > > at
> > > > >
> > org.apache.nifi.runtime.manifest.impl.RuntimeManifestGenerator.execute
> > > > > (RuntimeManifestGenerator.java:81)
> > > > >
> > > > > at
> > org.apache.nifi.runtime.manifest.impl.RuntimeManifestGenerator.main
> > > > > (RuntimeManifestGenerator.java:143)
> > > > >
> > > > > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0
> (Native
> > > > > Method)
> > > > >
> > > > > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
> > > > > (NativeMethodAccessorImpl.java:62)
> > > > >
> > > > > at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
> > > > > (DelegatingMethodAccessorImpl.java:43)
> > > > >
> > > > > at java.lang.reflect.Method.invoke (Method.java:566)
> > > > >
> > > > > at org.codehaus.mojo.exec.ExecJavaMojo$1.run
> > (ExecJavaMojo.java:282)
> > > > >
> > > > > at java.lang.Thread.run (Thread.java:829)
> > > > >
> > > > > On Fri, Mar 11, 2022 at 2:56 PM Joe Witt 
> wrote:
> > > > > >
> > > > > > Not from my point of view it isn't.  Also try it without the IT
> > while
> > > > > doing
> > > > > > contrib-check then run the ITs without contrib-check.
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > On Fri, Mar 11, 2022 at 12:51 PM Mike Thomsen <
> > mikerthom...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > [INFO]
> > > > > > >
> > > > > > > [INFO] --- maven-checkstyle-plugin:3.1.2:check (check-style) @
> > > > > > > nifi-system-test-suite ---
> > > > > > >
> > > > > > > [WARNING]
> > > > > > >
> > > > >
> >
> src/test/java/org/apache/nifi/tests/system/clustering/OffloadIT.java:[34,8]
> > > > > > > (imports) UnusedImports: Unused import - java.util.Collection.
> > > > > > >
> > > > > > > [INFO]
> > > > > > >
> > > > >
> > 
> > > > > > >
> > > > > > > [INFO] BUILD FAILURE
> > > > > > >
> > > > > > > [INFO]
> > > > > > >
> > > > >
> > 
> > > > > > >
> > > > > > > [INFO] Total time:  15.602 s (Wall Clock)
> > > > > > >
> > > > > > > [INFO] Finished at: 2022-03-11T14:49:43-05:00
> > > > > > >
> > > > > > > [INFO]
> > > > > > >
> > > > >
> > 
> > > > > > >
> > > > > > > [WARNING] The requested profile "include-grpc" could not be
> > activated
> > > > > > > because it does not exist.
> > > > > > >
> > > > > > > [ERROR] Failed to execute goal
> > > > > > > org.apache.maven.plugins:maven-checkstyle-plugin:3.1.2:check
> > > > > > > (check-style) on project nifi-system-test-suite: You have 1
> > Checkstyle
> > > > > > > violation. -> [Help 1]
> > > > > > >
> > > > > > > [ERROR]
> > > > > > >
> > > > > > >
> > > > > > > Is this worth a -1 vote?
> > > > > > >
> > > > >
> >
>

Re: Release plans?

[Nathan Gough]

Hi Enrico, great job on getting the 3.8.0 ZK release across the line. Just
wanted to follow up on the curator release and see if we can get it rolling.

Thanks for your work!
Nathan

On Sat, Feb 19, 2022 at 8:09 PM Enrico Olivelli  wrote:

> Still working on zk 3.8.0...
>
>
> Enrico
>
> Il Ven 11 Feb 2022, 12:18 Enrico Olivelli  ha
> scritto:
>
> > I am doing the ZooKeeper 3.8.0 release now.
> > I will switch to Curator when 3.8.0 is out.
> >
> > Sorry for the delay
> >
> > Enrico
> >
> > Il giorno ven 11 feb 2022 alle ore 12:17 Jordan Zimmerman
> >  ha scritto:
> > >
> > > I don't have the bandwidth currently unfortunately.
> > >
> > > -Jordan
> > >
> > > > On Feb 7, 2022, at 7:31 PM, Enrico Olivelli 
> > wrote:
> > > >
> > > > Nathan,
> > > >
> > > > Il Lun 7 Feb 2022, 18:29 Nathan Gough  > thena...@apache.org>> ha scritto:
> > > >
> > > >> Hi there,
> > > >>
> > > >> I was curious to know when the next planned release (5.2.1 or
> higher)
> > might
> > > >> be made? We at the Apache NiFi project are seeing problems related
> to
> > > >> https://issues.apache.org/jira/browse/CURATOR-561 and are keen to
> > upgrade
> > > >> to 5.2.1 to incorporate this fix. We have a NiFi ticket here:
> > > >> https://issues.apache.org/jira/browse/NIFI-9559.
> > > >>
> > > >
> > > > Thanks
> > > >
> > > > We should cut a release soon then
> > > >
> > > > I nobody volunteers I can prepare a RC next week?
> > > >
> > > > Is there any volunteer among Curator committers?
> > > >
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > >
> > > >> Thanks for your help and we appreciate your work,
> > > >> Nathan
> > >
> >
>

Re: Doco, PutSolrContentStream

[Nathan Gough]

Hi Dwane,

If possible would you be able to test out the PR here and see if it fixes
your issue? https://github.com/apache/nifi/pull/5727

You can checkout the PR and build the code yourself in the
nifi/nifi-nar-bundles/nifi-solr-bundle/nifi-solr-nar directory or you can
backup your current nifi-solr-nar-1.16.0-SNAPSHOT.nar in the ./nifi/lib
directory and download and replace with the one I built here:
https://easyupload.io/74ujf4.

Thanks,
Nathan


On Fri, Jan 28, 2022 at 12:42 PM Andrew Lim 
wrote:

> Hi Dwane,
>
> Thanks for finding and reporting those documentation errors. I filed a
> Jira [1] to fix those.
>
> It looks like the change to the default value of
> nifi.provenance.repository.rollover.time was made in 1.12.0 [2]. I will see
> if we can improve the docs to give more context to why this was done as
> part of [1].
>
> -Drew
>
>
> [1] https://issues.apache.org/jira/browse/NIFI-9642 <
> https://issues.apache.org/jira/browse/NIFI-9642>
> [2] https://issues.apache.org/jira/browse/NIFI-7339 <
> https://issues.apache.org/jira/browse/NIFI-7339>
>
>
> > On Jan 28, 2022, at 7:56 AM, Nathan Gough  wrote:
> >
> > Hi Dwane,
> >
> > I've created a Jira issue to test and rectify the Solr + ZooKeeper
> issue: https://issues.apache.org/jira/browse/NIFI-9641 <
> https://issues.apache.org/jira/browse/NIFI-9641>
> >
> > Thanks for the report!
> > Nathan
> >
> > On Fri, Jan 28, 2022 at 7:22 AM Dwane Hall  <mailto:dwaneh...@hotmail.com>> wrote:
> > Hey NiFi community I hope all is well with everyone wherever they may
> be.  I recently updated our NiFi instances from 1.11.4 to 1.15.3 and have
> made a few observations from this process worth mentioning.
> >
> > Some minor documentation inconsistencies
> > A couple of the default values appear to have changed in nifi.properties
> through versions (listed below are the old and new values along with links
> to the documentation).
> >
> >
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#write-ahead-flowfile-repository
> <
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#write-ahead-flowfile-repository
> >
> > “The FlowFile Repository checkpoint interval. The default value is 2
> mins.” [new default value is 20 secs]
> > 1.11.4 nifi.flowfile.repository.checkpoint.interval=2 mins
> > 11.15.3 nifi.flowfile.repository.checkpoint.interval=20 secs
> >
> >
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#persistent-provenance-repository-properties
> <
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#persistent-provenance-repository-properties
> >
> > “The amount of time to wait before rolling over the latest data
> provenance information so that it is available in the User Interface. The
> default value is 30 secs.”
> >
> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#system-properties
> <
> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#system-properties
> >
> > “If processing a high volume of events, change
> nifi.provenance.repository.rollover.time from a default of 30 secs to 1 min
> and ...” [The new default value is 10 min].
> > 1.11.4 nifi.provenance.repository.rollover.time=30 sec
> > 1.15.3 nifi.provenance.repository.rollover.time=10 min
> > This seems to be a significant change was there any reason for this new
> default setting I was unable to find documentation referencing the increase?
> >
> > PutSolrContentStream processor issues
> >
> > Secondly after a successful upgrade I noticed our use of the
> PutSolrContentStream processor had broken.  Looking through the processor
> code there was an upgrade to the SolrJ client and a commit in March 2020
> (and referenced below) that appears to prevent nested zk chroot paths for
> SolrCloud connections (i.e. the zookeeper connection string is truncated).
> >
> > SolrUtils.java (nifi/SolrUtils.java at master · apache/nifi · GitHub <
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrUtils.java
> >)
> > The commit of intrest regarding the new process for initiating a
> CloudSolrClient in SolrJ
> >
> https://github.com/apache/nifi/commit/9b4292024be6fae188cb1efa3a07dc9489e9a5b4#diff-13320e5b198f236cea296fb01cb7376755d65c444678e781fa0940c2a28db88b
> <
> https://github.com/apache/nifi/commit/9b4292024be6fae188cb1efa3a07dc9489e9a5b4#diff-13320e5b198f236cea296fb01cb7376755d65c444678e781fa0940c2a28db88b
> >
> >
> > For a nested Solr path "/solr/PROD", "/

CVE-2021-44145: Apache NiFi information disclosure by XXE

[Nathan Gough]

Severity: Low

Description:

In the TransformXML processor an authenticated user could configure an
XSLT file which, if it included malicious external entity calls, may
reveal sensitive information.

This issue is being tracked as NIFI-9399

Credit:

This issue was discovered by DangKhai at Viettel Cyber Security.

References:
https://nifi.apache.org/security.html#1.15.1-vulnerabilities

Re: [ANNOUNCE] New Apache NiFi Committer Margot Tien

[Nathan Gough]

Congrats Margot, thanks for all your contributions!

On Wed, Dec 15, 2021 at 3:02 PM Chris Sampson
 wrote:

> Congrat Margot!
>
> ---
> *Chris Sampson*
> IT Consultant
> chris.samp...@naimuri.com
>
>
> On Wed, 15 Dec 2021 at 19:04, Pierre Villard 
> wrote:
>
> > Congrats Margot!
> >
> > Le mer. 15 déc. 2021 à 20:00, Kevin Doran  a écrit :
> >
> > > Congratulations Margot! Well deserved.
> > >
> > > > On Dec 15, 2021, at 13:47, Joe Witt  wrote:
> > > >
> > > > Congrats Margot!   And thanks
> > > >
> > > > On Wed, Dec 15, 2021 at 11:46 AM Matt Gilman 
> > > wrote:
> > > >
> > > >> Apache NiFi community,
> > > >>
> > > >> On behalf of the Apache NiFi PMC, I am very pleased to announce that
> > > Margot
> > > >> has accepted the PMC's invitation to become a committer on the
> Apache
> > > NiFi
> > > >> project. We greatly appreciate all of Margot's hard work and
> generous
> > > >> contributions to the project. We look forward to continued
> involvement
> > > in
> > > >> the project.
> > > >>
> > > >> Margot has been contributing to NiFi and NiFi Registry for years.
> Her
> > > >> contributions have covered both back-end and front-end improvements
> in
> > > both
> > > >> projects in addition to release verification and thoughtful PR
> > reviews.
> > > >>
> > > >> Welcome and congratulations!
> > > >>
> > >
> > >
> >
>

Re: [RESULT][VOTE] Release Apache NiFi 1.15.1 (rc1)

[Nathan Gough]

Little bit late but +1 non binding, verified the hashes and tested a secure
cluster + secure external ZK and some data flows.

On Wed, Dec 15, 2021 at 2:31 PM Joe Witt  wrote:

> Apache NiFi Community,
>
> I am pleased to announce that the 1.15.1 release of Apache NiFi passes with
> 6 +1 (binding) votes
> 3 +1 (non-binding) votes
> 0 0 votes
> 0 -1 votes (non-binding) votes
>
> Thank you all for quickly making this release and vote work out so
> quickly.  It was a shortened 24 hour vote which I'm closing a bit
> early given the language found in policy
> https://www.apache.org/foundation/voting.html#ReleaseVotes.  Doing
> this promptly to get a clear/precise answer to the growing emails,
> slack messages, and JIRAs on this topic.
>
> Here is the PMC vote thread:
> https://lists.apache.org/thread/4ypxoxv2fnlh6wm0njjhxvxnfo846330
>
>
>
> On Wed, Dec 15, 2021 at 12:26 PM Joe Witt  wrote:
> >
> > +1 binding
> >
> > On Wed, Dec 15, 2021 at 12:25 PM Mark Payne 
> wrote:
> > >
> > > +1 (binding)
> > >
> > > Was able to verify hash & signature.
> > > Completed full build w/ all unit tests
> > > Ran system tests with all completing successfully
> > >
> > > Started a standalone instance with OOTB config and verified all was ok
> > >
> > > Started a secure cluster and ran some dummy flows to ensure that data
> was processing as expected. Encountered no issues.
> > >
> > > Built a dataflow that unpacks the entire archive and recursively
> unpacks all nars, jars, tars, gzip, zip, etc. and looks for any
> JndiLookup.class files. This way, even if a log4j dependency were shaded,
> it would still be flagged. Was able to find that older builds have several
> NARs packaged that had a JndiLookup.class but can confirm that this build
> contains no instances of it.
> > >
> > > Thanks for turning around the RC and the vote and performing the RM
> duties so quickly Joe!
> > >
> > > -Mark
> > >
> > >
> > > > On Dec 15, 2021, at 2:02 PM, Joe Gresock  wrote:
> > > >
> > > > +1 (non-binding) -- ran through the release guide and ran a basic
> flow with
> > > > no problems
> > > >
> > > > On Tue, Dec 14, 2021 at 10:35 PM Joe Witt 
> wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> I am pleased to be calling this vote for the source release of
> Apache
> > > >> NiFi 1.15.1.
> > > >>
> > > >> This vote, unlike most, is purely stability and security focused.
> > > >> This vote is rooted
> > > >> in a prompt response to the 'log4shell' vulnerability and related
> > > >> logging announcements.
> > > >> It also includes other easy to incorporate bugs and improvements.
> It
> > > >> should be easy to
> > > >> upgrade from any 1.15 install to this and just as easy as it was to
> go
> > > >> from pre 1.15 to
> > > >> this 1.15.1.
> > > >>
> > > >> The source zip, including signatures, digests, etc. can be found at:
> > > >>
> https://repository.apache.org/content/repositories/orgapachenifi-1192
> > > >>
> > > >> The source being voted upon and the convenience binaries can be
> found at:
> > > >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.15.1/
> > > >>
> > > >> A helpful reminder on how the release candidate verification
> process works:
> > > >>
> > > >>
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> > > >>
> > > >> The Git tag is nifi-1.15.1-RC1
> > > >> The Git commit ID is 2a756372fc7097ece6258c2af47b9a5f26384b02
> > > >>
> > > >>
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=2a756372fc7097ece6258c2af47b9a5f26384b02
> > > >>
> > > >> Checksums of nifi-1.15.1-source-release.zip:
> > > >> SHA256:
> 83d06011f0d2608d2d9cf951deae04d7b0921f2a7c8b1052ca9d058cf46b7d52
> > > >> SHA512:
> > > >>
> 009161e81e207a16060d9efd37e9b9abd1c1d5b5d57024a2b4c0d0ea17050f65b3a025632718161cba41948fe51d93aed65a4daba2542fce4da51d0184872039
> > > >>
> > > >> Release artifacts are signed with the following key:
> > > >> https://people.apache.org/keys/committer/joewitt.asc
> > > >>
> > > >> KEYS file available here:
> > > >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > >>
> > > >> 45 issues were closed/resolved for this release:
> > > >>
> > > >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12351055
> > > >>
> > > >> Release note highlights can be found here:
> > > >>
> > > >>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.15.1
> > > >>
> > > >> Given the nature of the vote being about a prompt release to remove
> > > >> vulnerable
> > > >> logging related libraries the vote will be open for 24 hours
> (instead
> > > >> of the normal 72 hours).
> > > >>
> > > >> Please download the release candidate and evaluate the necessary
> items
> > > >> including checking hashes, signatures, build from source, and test.
> > > >> Then please vote:
> > > >>
> > > >> [ ] +1 Release this package as nifi-1.15.1
> > > >> [ ] +0 no opinion
> > > >> [ ] -1 Do not release this package because...
> > > 

Re: Secure Nifi cluster setup issues

[Nathan Gough]

Hi Firdous,

Can you give us more details about what issue you're facing? Error
logs/config files? Are you trying to use client certificate authentication
as well or some other authentication?

Thanks,
Nathan

On Thu, Nov 11, 2021, 7:38 PM Firdous Fatima 
wrote:

> Good evening,
>
> Hope you’re doing well!
> I have been working on Nifi testing environment. And I have tried multiple
> resources to secure my Nifi cluster using SSL authentication without any
> luck. I am hoping to setup the cluster using company signed certificates
> and was wondering if I could get any help or steps on how I can secure the
> cluster. I have tried all the resources available online and would really
> appreciate any help relevant to my use case.
>
> Thank you,
> Firdous
>

Re: [DISCUSS] NiFi 2.0 Release Goals

[Nathan Gough]

I'm a +1 for removing pretty much all of this stuff. There are security
implications to keeping old dependencies around, so the more old code we
can remove the better. I agree that eventually we need to move to
supporting only Java 11+, and as our next release will probably be about 4
- 6 months from now that doesn't seem too soon. We could potentially break
this in two and remove the deprecated processors and leave 1.x on Java 8,
and finally start on 2.x which would support only Java 11. I'm unsure of
what implications changing the date and time handling would have - for
running systems that use long term historical logs, unexpected impacts to
time logging could be a problem.

As Joe says I think feature work will have to be dedicated to 2.x and we
could support 1.x for security fixes for some period of time. 2.x seems
like a gargantuan task but it's probably time to get started. Not sure how
we handle all open PRs and the transition between 1.x and 2.x.

On Fri, Jul 23, 2021 at 10:57 AM Joe Witt  wrote:

> Jon
>
> You're right we have to be careful and you're right there are still
> significant Java 8 users out there.  But we also have to be careful
> about security and sustainability of the codebase.  If we had talked
> about this last year when that article came out I'd have agreed it is
> too early.  Interestingly that link seems to get updated and I tried
> [1] and found more recent data (not sure how recent).  Anyway it
> suggests Java 8 is still the top dog but we see good growth on 11.  In
> my $dayjob this aligns to what I'm seeing too.  Customers didn't seem
> to care about Java 11 until later half last year and now suddenly it
> is all over the place.
>
> I think once we put out a NiFi 2.0 release we'd see rapid decrease in
> work on the 1.x line just being blunt.  We did this many years ago
> with 0.x to 1.x and we stood behind 0.x for a while (maybe a year or
> so) but it was purely bug fix/security related bits.  We would need to
> do something similar.  But feature work would almost certainly go to
> the 2.x line.  Maybe there are other workable models but my instinct
> suggests this is likely to follow a similar path.
>
> ...anyway I agree it isn't that easy of a call to dump Java 8.  We
> need to make the call in both the interests of the user base and the
> contributor base of the community.
>
> [1] https://www.jetbrains.com/lp/devecosystem-2021/java/
>
>
> Thanks
> Joe
>
> On Fri, Jul 23, 2021 at 7:46 AM Joe Witt  wrote:
> >
> > Russ
> >
> > Yeah the flow registry is a key part of it.  But also now you can
> > download the flow definition in JSON (upload i think is there now
> > too).  Templates offered a series of challenges such as we store them
> > in the flow definition which has made flows massive in an unintended
> > way which isn't fun for cluster behavior.
> >
> > We have a couple cases where we headed down a particular concept and
> > came up with better approaches later.  We need to reconcile these with
> > the benefit of hindsight, and while being careful to be not overly
> > disruptive to existing users, to reduce the codebase/maintenance
> > burden and allow continued evolution of the project.
> >
> > Thanks
> >
> > On Fri, Jul 23, 2021 at 7:43 AM Russell Bateman 
> wrote:
> > >
> > > Joe,
> > >
> > > I apologize for the off-topic intrusion, but what replaces templates?
> > > The Registry? Templates rocked and we have used them since 0.5.x.
> > >
> > > Russ
> > >
> > > On 7/23/21 8:31 AM, Joe Witt wrote:
> > > > David,
> > > >
> > > > I think this is a highly reasonable approach and such a focus will
> > > > greatly help make a 2.0 release far more approachable to knock out.
> > > > Not only that but tech debt reduction would help make work towards
> > > > major features we'd think about in a 'major release' sense more
> > > > approachable.
> > > >
> > > > We should remove all deprecated things (as well as verify we have the
> > > > right list).  We should remove/consider removal of deprecated
> concepts
> > > > like templates.  We should consider whether we can resolve the
> various
> > > > ways we've handled what are now parameters down to one clean
> approach.
> > > > We should remove options in the nifi.properties which turn out to
> > > > never be used quite right (if there are).  There is quite a bit we
> can
> > > > do purely in the name of tech debt reduction.
> > > >
> > > > Lots to consider here but I think this is the right discussion.
> > > >
> > > > Than ks
> > > >
> > > > On Fri, Jul 23, 2021 at 7:26 AM Bryan Bende 
> wrote:
> > > >> I'm a +1 for this... Not sure if this falls under "Removing
> Deprecated
> > > >> Components", but I think we should also look at anything that has
> been
> > > >> marked as deprecated throughout the code base as a candidate for
> > > >> removal. There are quite a few classes, methods, properties, etc
> that
> > > >> have been waiting for a chance to be removed.
> > > >>
> > > >> On Fri, Jul 23, 2021 at 10:13 AM David Handermann
>

Re: [VOTE] Release Apache NiFi 1.14.0 (rc2)

[Nathan Gough]

+1 (non-binding)

- Built from source + test, checked hashes
- Ran with a secure three node cluster and X509 authentication, tested S2S
and some other processors
- Ran single secure node with OIDC authentication (G Suite)
- Ran single secure node with SAML authentication (G Suite)

java -version

openjdk version "1.8.0_292"

OpenJDK Runtime Environment (Zulu 8.54.0.21-CA-macosx) (build 1.8.0_292-b10)

OpenJDK 64-Bit Server VM (Zulu 8.54.0.21-CA-macosx) (build 25.292-b10,
mixed mode)

mvn -version
Apache Maven 3.6.3


Thanks for running the RC process again Joe!


On Tue, Jul 13, 2021 at 9:33 AM Otto Fowler  wrote:

>  +1 ( non binding )
> Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
> Maven home: /usr/local/Cellar/maven/3.8.1/libexec
> Java version: 1.8.0_292, vendor: AdoptOpenJDK, runtime:
> /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.16", arch: "x86_64", family: “mac"
>
>
>
> From: Joe Witt  
> Reply: dev@nifi.apache.org  
> Date: July 10, 2021 at 18:40:26
> To: dev@nifi.apache.org  
> Subject:  [VOTE] Release Apache NiFi 1.14.0 (rc2)
>
> Hello,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi 1.14.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1183
>
> The source being voted upon and the convenience binaries can be found at:
> https://dist.apache.org/repos/dist/dev/nifi/1.14.0/
>
> Please note that this release now includes the convenience binaries for
> Apache NiFi, NiFi toolkit, MiNiFi, MiNiFi Toolkit, Registry, Registry
> Toolkit,
> and Stateless NiFi.
>
> A helpful reminder on how the release candidate verification process works:
>
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
>
> The Git tag is nifi-1.14.0-RC2
> The Git commit ID is fcbf1d5f975dd984e34f3a543b9480c779b0dc2f
>
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=fcbf1d5f975dd984e34f3a543b9480c779b0dc2f
>
> Checksums of nifi-1.14.0-source-release.zip:
> SHA256: a96cf75c4f82d01e1c8e0b678d5ff23dec8c26824611c8d37f2ec245e9932b1c
> SHA512:
>
> 2d23b1a2fae9f545f665c4ee5d9723cdf9c68a62a26d80287b96a55773594e1e80f689ec0f00ba74af92df164c6f4df73ac9b91db7678aaefd69ee8f1eed3f42
>
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/joewitt.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 330+ issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12349644
>
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.14.0
>
> The vote will be open for at least 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking
> hashes, signatures, build from source, and test. Then please vote:
>
> [ ] +1 Release this package as nifi-1.14.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>

Re: [discuss] nifi 1.14.0

[Nathan Gough]

Joe Gresock just pinged me about an issue that may have been introduced by
a dependency upgrade I did for lucene:
https://issues.apache.org/jira/browse/NIFI-8699 which appears to cause an
issue for existing provenance repositories. I tested the upgrade on a fresh
install so I didn't notice the issue. There appears to be a way to add a
backwards codec which should allow the new lucene to keep working with the
existing provenance repo. Looking into reproducing and a fix for it now.

On Wed, Jun 23, 2021 at 9:06 AM Mark Bean  wrote:

> Putting out one more request for the following open PR's before 1.14
>
> https://github.com/apache/nifi/pull/5094
> https://github.com/apache/nifi/pull/5061
>
> Both have been reviewed, but still need attention from a comitter.
>
> Thanks!
> -Mark
>
>
> On Mon, Jun 21, 2021 at 12:17 PM Joe Witt  wrote:
>
> > Team,
> >
> > I'll start pulling 1.14 together more this week as time permits.  As
> > far as specific commits/etc.. please work with reviewers/etc.. to help
> > nail that down.  If anything doesn't make it in when I initiate the RC
> > line then we'll get it on the next one.  There is a shocking amount of
> > goodness in here already.
> >
> > Thanks
> >
> > On Sun, Jun 13, 2021 at 2:27 PM Chris Sampson
> >  wrote:
> > >
> > > Joe,
> > >
> > > Yeah I thought it would be something like that (but didn't spend time
> > > looking at the moment, just thought I'd highlight the thread). Don't
> know
> > > whether there's anything to consider adding to/clarifying in the
> > > documentation in order to highlight that to (first time) users?
> > >
> > > Again, I figured this would probably be "as designed" and I've not
> spent
> > > the time reading the docs for this new default behaviour - so long as
> it
> > > should be clear to first time users (provided they read the docs), then
> > all
> > > good.
> > >
> > >
> > > Cheers,
> > >
> > > Chris Sampson
> > >
> > > On Sun, 13 Jun 2021, 21:42 Joe Witt,  wrote:
> > >
> > > > Chris
> > > >
> > > > I responded to the slack thread. Pretty sure it is doing exactly what
> > > > is expected.  We are not offering a user management and policy
> > > > authoring experience for that.  It is quite literally 'single user
> > > > auth' and in that mode this single user we generate has all the
> > > > authorizations.  This is functionally equivalent to how it was with
> an
> > > > unsecured instance with what is basically 'anonymous' user except in
> > > > this case it is TLS and requires the known single user credentials.
> > > > For real usage, just as before, users need to take advantage of one
> of
> > > > the other existing authentication and authorization plugin options.
> > > >
> > > > Thanks
> > > >
> > > > On Sun, Jun 13, 2021 at 11:26 AM Chris Sampson
> > > >  wrote:
> > > > >
> > > > > FYI, there's a new thread in slack about the new
> > single-user-authoriser
> > > > > setup - user has https but no users/policy screen for setting up
> > AuthZ.
> > > > >
> > > > > Might be worth someone taking a look before an RC to see whether
> > there's
> > > > > documentation (or functionality) that needs clarifying.
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Chris Sampson
> > > > >
> > > > > On Sun, 13 Jun 2021, 13:57 Mark Bean, 
> wrote:
> > > > >
> > > > > > There are three open PR's I would appreciate some eyes on before
> > the RC
> > > > > > process is kicked off. Two of the three have been reviewed, but
> not
> > > > yet by
> > > > > > a committer.
> > > > > >
> > > > > > https://github.com/apache/nifi/pull/5094
> > > > > > https://github.com/apache/nifi/pull/5061
> > > > > > https://github.com/apache/nifi/pull/5064
> > > > > >
> > > > > > Thanks in advance!
> > > > > > -Mark
> > > > > >
> > > > > > On Fri, Jun 11, 2021 at 4:05 PM Joe Witt 
> > wrote:
> > > > > >
> > > > > > > So. Dang. Cool.  I just built from latest main and poof - I'm
> on
> > > > https
> > > > > > > with username/password.
> > > > > > >
> > > > > > > Will start whipping up the process for an RC.  Probably will
> be a
> > > > > > > little slow going with dayjob factors but will get on it.
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > On Fri, Jun 11, 2021 at 12:14 PM David Handermann
> > > > > > >  wrote:
> > > > > > > >
> > > > > > > > Thanks to Mark Payne, NIFI-8516 is now merged, so that covers
> > > > current
> > > > > > > open
> > > > > > > > issues around securing the default configuration.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > David Handermann
> > > > > > > >
> > > > > > > > On Fri, Jun 11, 2021 at 11:55 AM David Handermann <
> > > > > > > > exceptionfact...@apache.org> wrote:
> > > > > > > >
> > > > > > > > > Joe,
> > > > > > > > >
> > > > > > > > > Thanks for following up.  The PR for NIFI-8516 has gone
> > through
> > > > > > several
> > > > > > > > > rounds of feedback, I believe it is about ready to go,
> > pending
> > > > > > > confirmation
> > > > > > > > > that the ability to set custom credentials addresses the

Re: [ANNOUNCE] New NiFi PMC Member Joey Frazee

[Nathan Gough]

Congratulations Joey! Thanks for your past and future contributions.

On Thu, Mar 25, 2021 at 3:15 PM Joey Frazee 
wrote:

> I want to make sure to say thanks for the recognition! I’m thrilled to be
> a part of this. The help and collaboration from the project members and
> contributors over the past years has made this a ton of fun and I’m very
> happy to be continuing with more of it.
>
> -joey
>
> > On Mar 25, 2021, at 11:57 AM, Otto Fowler 
> wrote:
> >
> > Congratulations!
> >
> >> On Thu, Mar 25, 2021 at 2:54 PM Joe Witt  wrote:
> >>
> >> NiFi Community,
> >>
> >> On behalf of the Apache NiFi PMC, I am pleased to announce that Joey
> Frazee
> >> has accepted the PMC's invitation to join the Apache NiFi PMC.
> >>
> >> Joey has been a contributor and committer of Apache NiFi for several
> >> years and has been involved across the whole spectrum of code reviews,
> >> JIRAs, code contributions, maintained a great 'awesome nifi' github
> >> site, and more.
> >>
> >> Please join us in congratulating and welcoming Joey to the Apache NiFi
> PMC.
> >>
> >> Congratulations Joey!
> >>
>

Re: [VOTE] Release Apache NiFi 1.13.1

[Nathan Gough]

+1 (non-binding)

Verified release signatures, built and ran a secure cluster with secure
embedded ZK. Tested some network based processors on the canvas.

Cheers,
Nathan

On Sat, Mar 13, 2021 at 10:50 AM Pierre Villard 
wrote:

> +1 (binding)
>
> Went through the usual steps, checked a set of flows with a secured cluster
> and a secured NiFi Registry.
>
> Thanks Joe for taking care of this!
>
> Pierre
>
> Le sam. 13 mars 2021 à 06:34, M Tien  a écrit :
>
> > +1 (non-binding)
> >
> > Went through the release helper
> > Verified a successful build on -
> > Zulu OpenJDK 1.8.0_282
> > Zulu OpenJDK 11.0.10
> > Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec
> > OS name: "mac os x", version: "10.15.4", arch: "x86_64", family: "mac"
> > Set up a secure standalone instance
> > Set up a secure cluster
> >
> > Thanks,
> > Margot Tien
> >
> >
> > > On Mar 12, 2021, at 12:26 PM, David Handermann <
> > exceptionfact...@gmail.com> wrote:
> > >
> > > +1 non-binding
> > >
> > > Verified release signatures and expected files.
> > > Verified build on Ubuntu 20.10 using Apache Maven 3.6.3 with Azul Zulu
> > JDK
> > > 11.0.10.
> > > Configured and tested InvokeHTTP with multiple configurations including
> > > disabling HTTP/2.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > On Fri, Mar 12, 2021 at 2:19 PM Joey Frazee  > .invalid>
> > > wrote:
> > >
> > >> +1 (non-binding)
> > >>
> > >> - Verified checksums and signatures
> > >> - Ran full build on Java 1.8 and 11 on Linux
> > >> - Verified NIFI-3383, NIFI-8231, and NIFI-8200
> > >>
> > >> -joey
> > >>
> > >>> On Mar 12, 2021, at 10:28 AM, Bryan Bende  wrote:
> > >>>
> > >>> +1 (binding)
> > >>>
> > >>> - Verified everything in the standard release helper
> > >>> - Setup secure standalone instance with SAML authentication
> > >>>
> >  On Fri, Mar 12, 2021 at 10:17 AM Marton Szasz 
> > >> wrote:
> > 
> >  +1 (non-binding)
> > 
> >  Followed the release helper guide, tested the binary with a simple
> > flow.
> > 
> >  Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> >  Maven home: /usr/share/maven-bin-3.6
> >  Java version: 1.8.0_252, vendor: IcedTea, runtime:
> > >> /opt/icedtea-bin-3.16.0/jre
> >  Default locale: en_US, platform encoding: UTF-8
> >  OS name: "linux", version: "5.10.12-gentoo-x86_64", arch: "amd64",
> >  family: "unix"
> > 
> > 
> > > On Fri, 12 Mar 2021 at 13:35, Otto Fowler  >
> > >> wrote:
> > >
> > > +1
> > >
> > > Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> > > Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec
> > > Java version: 1.8.0_282, vendor: AdoptOpenJDK, runtime:
> > >> /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
> > > Default locale: en_US, platform encoding: UTF-8
> > > OS name: "mac os x", version: "10.16", arch: "x86_64", family:
> “mac"
> > >
> > >
> > >> On Mar 11, 2021, at 11:29, Joe Witt  wrote:
> > >>
> > >> Hello,
> > >>
> > >> I am pleased to be calling this vote for the source release of
> > Apache
> > >> NiFi
> > >> 1.13.1.
> > >>
> > >> The source zip, including signatures, digests, etc. can be found
> at:
> > >>
> > https://repository.apache.org/content/repositories/orgapachenifi-1179
> > >>
> > >> The source being voted upon and the convenience binaries can be
> > found
> > >> at:
> > >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.13.1/
> > >>
> > >> A helpful reminder on how the release candidate verification
> process
> > >> works:
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> > >>
> > >> The Git tag is nifi-1.13.1-RC1
> > >> The Git commit ID is acbc217cb7002d7489421f4d346b995a43b6ea01
> > >>
> > >>
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=acbc217cb7002d7489421f4d346b995a43b6ea01
> > >>
> > >> Checksums of nifi-1.13.1-source-release.zip:
> > >> SHA256:
> > >> 0a397df640e579720e26699e38a3738c5be05af01aad8aaeefc04eb58591faac
> > >> SHA512:
> > >>
> > >>
> >
> 7f8df759d4345ccd6e75c169bd0aab1b7f4f64bf5a8b11b45bc1d7c8163acd0035922dcdbef232392279f4ea0710d4a97c55d480281bfe1d50b6401295633d48
> > >>
> > >> Release artifacts are signed with the following key:
> > >> https://people.apache.org/keys/committer/joewitt.asc
> > >>
> > >> KEYS file available here:
> > >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >>
> > >> 48 issues were closed/resolved for this release:
> > >>
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12348700
> > >>
> > >> Release note highlights can be found here:
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.13.1
> > >>
> > >> The vote will be open for 72 hours.
> > >> 

Re: [discuss] we need to enable secure by default...

[Nathan Gough]

I 100% agree that something needs to be done. We cannot allow NiFi to build
a reputation that it is 'insecure' by allowing its default installation to
start up without any security. Especially considering how much work we put
in to make sure it IS a secure product that integrates with many
applications in a secure way. Security reputation is very important for
software. If some major exploitation of NiFi were to happen in the future,
we should be able to confidently say that we did our absolute best to
create a secure application. We shouldn't point at new users and say 'they
didn't configure it right'.

Personally, I am in favor of providing automatically generated certificates
and requiring the user to insert the client certificate in their browser,
and providing instructions and perhaps a YouTube video on how to do that.
Yes, X509 certificate errors are confusing and can be difficult for
beginners to troubleshoot. But those beginner users will also be the most
likely to use NiFi insecurely, connect it to the internet, and become part
of a user base who got burnt by NiFi being 'insecure'. I acknowledge this
is increasing the barrier for entry. If we intend to use a
username/password + server cert for HTTPs but no client cert, as stated
above we could automatically generate the password and provide this to the
user in a log or file.


On Wed, Feb 10, 2021 at 12:21 PM David Handermann <
exceptionfact...@gmail.com> wrote:

> Integrating an option to use Let's Encrypt would be a great improvement
> over self-signed certificates, but it is difficult to support that for
> instances of NiFi running on servers without Internet access.  Even when
> running NiFi for local development purposes, the development system is not
> likely to have a publicly addressable DNS name, so Let's Encrypt is not an
> option in those cases.
>
> Regards,
> David Handermann
>
> On Wed, Feb 10, 2021 at 11:09 AM Joe Witt  wrote:
>
> > Otto
> >
> > Installers like you mention are inherently brutal for portability so very
> > difficult for us in the community.  From the vendor world we of course do
> > things like that.  I think in apache nifi land we need a default 'even
> for
> > devs' which is not wide open.
> >
> > James
> >
> > I do think adding such a warning is good.  But it doesn't help avoid
> these
> > wildly abusive cases.  We need a secure by default path.  We can probably
> > do better than the self signed path if we add a 'before running' step in
> > the CLI for the user.  Not sure.
> >
> > Thanks
> >
> > On Wed, Feb 10, 2021 at 10:05 AM James Srinivasan <
> > james.sriniva...@gmail.com> wrote:
> >
> > > Would a suitably large warning on the http ui be a good starting point?
> > >
> > > Browsers are getting increasingly wary of self signed certs and we
> > probably
> > > don't want to be encouraging people to ignore them.
> > >
> > > What about easier acme+certbot support? (notwithstanding all the non
> > public
> > > deployments)
> > >
> > > On Wed, 10 Feb 2021, 15:25 Otto Fowler, 
> wrote:
> > >
> > > > Aren’t most of these applications installed by an installer?
> > > > Maybe we can have one or more installers that setup a secure
> instance,
> > > and
> > > > those installers
> > > > could be the *production* nifi, and keep the zip distribution open
> for
> > > > developers?
> > > >
> > > >
> > > > > On Feb 10, 2021, at 10:04, David Handermann <
> > > exceptionfact...@gmail.com>
> > > > wrote:
> > > > >
> > > > > I agree that a generated password is the way to go, and the
> challenge
> > > is
> > > > > making it available to the user.  Depending on how it is stored for
> > the
> > > > > identity provider, having a command line tool to read and print it
> > > could
> > > > be
> > > > > a reasonable option.
> > > > >
> > > > > Although this particular thread referenced a specific Twitter post,
> > > this
> > > > > general discussion is more of a reflection of the need to make
> things
> > > > more
> > > > > secure by default as other products have followed similar
> approaches.
> > > > >
> > > > > Regards,
> > > > > David Handermann
> > > > >
> > > > > On Wed, Feb 10, 2021 at 8:53 AM Kevin Doran 
> > wrote:
> > > > >
> > > > >> I am in favor of requiring some authentication by default.
> > > > >>
> > > > >> I’m in favor of an admin username and generated password. (It
> sounds
> > > > li9ke
> > > > >> most of us are on the same page, but I don’t think a static,
> default
> > > > >> password buys us much against the types of abuse shown on the
> > twitter
> > > > >> thread Joe shared.)
> > > > >>
> > > > >> We need some way of making the generated password discoverable…
> > Print
> > > to
> > > > >> the logs on first boot? Not great but are there other mechanisms?
> A
> > > > setup
> > > > >> CLI utility?
> > > > >>
> > > > >> To help not impede automated deployments, maybe we should change
> the
> > > > >> startup flow such that there is a way to provide this password.
> That
> > > > would
> > > > >> be better than people 

Re: [VOTE] Release Apache NiFi 1.13.0 (rc2)

[Nathan Gough]

+1 (non-binding)

- Verified signature and checksums again
- Ran build with
java -version

openjdk version "1.8.0_282"

OpenJDK Runtime Environment (Zulu 8.52.0.23-CA-macosx) (build 1.8.0_282-b08)

OpenJDK 64-Bit Server VM (Zulu 8.52.0.23-CA-macosx) (build 25.282-b08,
mixed mode)

- Tested secure clusters with TLS ZK, secure Site-to-Site, InvokeHTTP,
ListenHTTP

On Tue, Feb 2, 2021 at 7:20 PM Joey Frazee 
wrote:

> +1 (non-binding)
>
> - Verified checksums, signatures, and commit id
> - Ran builds with Java 1.8 and 11 on Linux and macOS, and validated RPM
> build profile
> - Tested cluster coordination and state management with both embedded and
> external ZooKeepers with TLS enabled and disabled
> - Verified fix for PutAzureBlobStorage OOME and tested Blob and Queue
> storage with Azurite emulator
>
> -joey
>
> > On Feb 2, 2021, at 3:16 PM, Sushil Kumar  wrote:
> >
> > +1 (non-binding) Release this package as nifi-1.13.0
> >
> > Deployed this via helm chart(https://github.com/sushilkm/nifi-chart) on
> > kubernetes.
> > Thank you to all the contributors and reviewers.
> >
> >
> >> On Tue, Feb 2, 2021 at 11:02 AM Matt Burgess 
> wrote:
> >>
> >> +1 (binding) Release this package as nifi-1.13.0
> >>
> >> Ran through release helper and tried various flows using some of the
> >> new features and capabilities added in 1.13.0.  Thanks for RM'ing Joe!
> >>
> >>> On Mon, Feb 1, 2021 at 8:10 PM Joe Witt  wrote:
> >>>
> >>> Hello,
> >>>
> >>> I am pleased to be calling this vote for the source release of Apache
> >> NiFi
> >>> 1.13.0.
> >>>
> >>> The source zip, including signatures, digests, etc. can be found at:
> >>> https://repository.apache.org/content/repositories/orgapachenifi-1176
> >>>
> >>> The source being voted upon and the convenience binaries can be found
> at:
> >>> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.13.0/
> >>>
> >>> A helpful reminder on how the release candidate verification process
> >> works:
> >>>
> >>
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >>>
> >>> The Git tag is nifi-1.13.0-RC2
> >>> The Git commit ID is c27e59fc679a2e982102a75b8b8df2b0f062af23
> >>>
> >>
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=c27e59fc679a2e982102a75b8b8df2b0f062af23
> >>>
> >>> Checksums of nifi-1.13.0-source-release.zip:
> >>> SHA256:
> 4913dd3d943afac710d1a277bf31beebf7a6207a20e1148849d69511f44fc97b
> >>> SHA512:
> >>>
> >>
> dc9935f0eb8692cd8493f5863bc8ae2ef0b52653fa69ff8b9a7e8db7dbd9ec6561f6ffdca4a1b55e43b289d04f5671f5ab4de30999838c5fca5c282c00a7c7f8
> >>>
> >>> Release artifacts are signed with the following key:
> >>> https://people.apache.org/keys/committer/joewitt.asc
> >>>
> >>> KEYS file available here:
> >>> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>>
> >>> 238 issues were closed/resolved for this release:
> >>>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12348700
> >>>
> >>> Release note highlights can be found here:
> >>>
> >>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.13.0
> >>>
> >>> The vote will be open for 72 hours.
> >>> Please download the release candidate and evaluate the necessary items
> >>> including checking hashes, signatures, build from source, and test.
> Then
> >>> please vote:
> >>>
> >>> [ ] +1 Release this package as nifi-1.13.0
> >>> [ ] +0 no opinion
> >>> [ ] -1 Do not release this package because...
> >>
>

Re: [VOTE] Release Apache NiFi 1.13.0

[Nathan Gough]

+1 non-binding from me.

Ran through the release helper and set up a secure cluster with a secure
embedded ZooKeeper, a simpler secure cluster without secured ZK, and a
secure cluster with secure external ZK.

On Fri, Jan 29, 2021 at 12:46 PM Matt Burgess  wrote:

> +1 Release this package as nifi-1.13.0
>
> Ran through release helper, tested various flows including features
> new to 1.13, all worked well. Thanks for RM'ing Joe!
>
> On Wed, Jan 27, 2021 at 11:15 PM Joe Witt  wrote:
> >
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > 1.13.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1175
> >
> > The source being voted upon and the convenience binaries can be found at:
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.13.0/
> >
> > A helpful reminder on how the release candidate verification process
> works:
> >
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >
> > The Git tag is nifi-1.13.0-RC1
> > The Git commit ID is be1ac1c49726f366423b28fe75a08cbe9885ada3
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=be1ac1c49726f366423b28fe75a08cbe9885ada3
> >
> > Checksums of nifi-1.13.0-source-release.zip:
> > SHA256: 02f5cfff3a3c2a82f82270b03e3533449330d9fbc102da8e920ab0cb39361218
> > SHA512:
> >
> 84769ec5791b6af1e9bca3088d535a611457ac13d046191021faf7c40ea0f1fc31238de43b9349e8034c2ec7d0cfeb570880fd9d4e6575d88ec52042b9fd997a
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/joewitt.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 228 issues were closed/resolved for this release:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12348700
> >
> > Release note highlights can be found here:
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.13.0
> >
> > Please note the convenience binary tar.gz and zip provided for NiFi
> include
> > one less nar (the 'nifi-ignite-nar') to ensure the total file size
> remains
> > under the ASF infrastructure limit. The migration guide and nifi-assembly
> > have been updated so explicit removal won't be necessary in the future.
> > But we will need to remain careful about any growth of our binaries and
> > will need to continue to prune out old nars until we move to another
> model
> > for users to acquire the nars they need.
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build from source, and test. Then
> > please vote:
> >
> > [ ] +1 Release this package as nifi-1.13.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
>

Re: [DISCUSS] Release of Apache NiFi 1.13.0

[Nathan Gough]

If possible I would like to get
https://issues.apache.org/jira/browse/NIFI-7356 in for this release as
well. This enables TLS + embedded Zookeeper. Right now, the PR for TLS +
external Zookeeper is merged, and there is an open PR for TLS + embedded
Zookeeper here: https://github.com/apache/nifi/pull/4216 which I have been
refactoring and testing. I will be submitting a PR for the embedded option
within a day or two (currently testing).

On Tue, Jan 5, 2021 at 10:08 AM Joe Witt  wrote:

> Thanks Pierre - definitely a good time to pull together a release.  Were
> you looking to do the RM work or do we need someone?  I'm happy to do it if
> needed.
>
> Thanks
>
> On Tue, Jan 5, 2021 at 8:04 AM Pierre Villard  >
> wrote:
>
> > Hi all,
> >
> > Happy new year to everyone, hope you all had a great time. I'm bumping
> this
> > thread up as I think it'd make a lot of sense to release NiFi 1.13.0. We
> > have a ton of bug fixes and improvements that would be very useful for
> our
> > community.
> >
> > Let's try to see if there are any blockers for a release and start the
> > process:
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20NIFI%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%201.13.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
> >
> > Thanks!
> >
> >
> > Le mar. 1 déc. 2020 à 14:49, Bryan Bende  a écrit :
> >
> > > Chris,
> > >
> > > The illegal reflective access warnings are expected when running on
> Java
> > > 11.
> > >
> > > The second warning about the provenance repo seems like a possible
> > > issue due to some recent refactorings made for stateless, we should
> > > take a look at that.
> > >
> > > Thanks,
> > >
> > > Bryan
> > >
> > > On Tue, Dec 1, 2020 at 3:58 AM Chris Sampson
> > >  wrote:
> > > >
> > > > Just spotted the following during startup of NiFi 1.13.0-SNAPSHOT on
> my
> > > > local machine, which might be worth looking at (although I've not
> > deleted
> > > > and re-created the binaries in a little while, so I could have
> > something
> > > > wrong in my /lib folder or such):
> > > >
> > > > 2020-12-01 08:52:48,382 ERROR [NiFi logging handler]
> > > org.apache.nifi.StdErr
> > > > WARNING: An illegal reflective access operation has occurred
> > > > 2020-12-01 08:52:48,383 ERROR [NiFi logging handler]
> > > org.apache.nifi.StdErr
> > > > WARNING: Illegal reflective access by
> > > > com.sun.xml.bind.v2.runtime.reflect.opt.Injector
> > > > (file:.../nifi-1.13.0-SNAPSHOT/lib/java11/jaxb-impl-2.3.0.jar) to
> > method
> > > > java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int)
> > > > 2020-12-01 08:52:48,383 ERROR [NiFi logging handler]
> > > org.apache.nifi.StdErr
> > > > WARNING: Please consider reporting this to the maintainers of
> > > > com.sun.xml.bind.v2.runtime.reflect.opt.Injector
> > > > 2020-12-01 08:52:48,383 ERROR [NiFi logging handler]
> > > org.apache.nifi.StdErr
> > > > WARNING: Use --illegal-access=warn to enable warnings of further
> > illegal
> > > > reflective access operations
> > > > 2020-12-01 08:52:48,383 ERROR [NiFi logging handler]
> > > org.apache.nifi.StdErr
> > > > WARNING: All illegal access operations will be denied in a future
> > release
> > > >
> > > > Also:
> > > > 2020-12-01 08:52:52,486 WARN [main]
> > > > o.a.n.n.StandardExtensionDiscoveringManager Failed to register
> > extension
> > > > org.apache.nifi.provenance.VolatileProvenanceRepository due to:
> Attempt
> > > was
> > > > made to load org.apache.nifi.provenance.VolatileProvenanceRepository
> > from
> > > > org.apache.nifi:nifi-provenance-repository-nar:1.13.0-SNAPSHOT but
> that
> > > > class name is already loaded/registered from
> > > > org.apache.nifi:nifi-stateless-nar:1.13.0-SNAPSHOT and multiple
> > versions
> > > > are not supported for this type
> > > >
> > > > These log messages appear just before the NAR extraction logs during
> > > > instance startup.
> > > >
> > > > Don't know whether these are things that should be fixed before
> 1.13.0
> > > > and/or have already been looked into, but figured I'd mention them so
> > > > they're at least known about.
> > > >
> > > > ---
> > > > *Chris Sampson*
> > > > IT Consultant
> > > > chris.samp...@naimuri.com
> > > > 
> > > >
> > > >
> > > > On Mon, 30 Nov 2020 at 18:50, Nissim Shiman
>  > >
> > > > wrote:
> > > >
> > > > >  Hello Nifi Team,
> > > > >
> > > > > NIFI-7738 [1]  https://github.com/apache/nifi/pull/4563
> > > > > has already been reviewed/tested by a fellow contributer
> > > > > I would greatly appreciate a committer moving this to main for
> 1.13.0
> > > > > Thanks,
> > > > > Nissim Shiman
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/NIFI-7738
> > > > > On Sunday, November 29, 2020, 07:12:12 PM EST, Matt Burgess <
> > > > > mattyb...@apache.org> wrote:
> > > > >
> > > > >  I just merged Mike's Cassandra DMC PR, reviewing the graph stuff
> > now.
> > > > >
> > > > > I have a bunch of open PRs [1], I don't think any are blockers f

Re: Setting up Secure Cluster

[Nathan Gough]

Hi Midhun,

We probably need more logs than that. I think there can be a few reasons
why you're seeing that error. Please provide logs immediately following the
failed replicate request.

Thanks,
Nathan

On Sun, Oct 4, 2020 at 7:16 AM Midhun Mohan  wrote:

> Hi Team,
>
> When I tried setting up a secure cluster, I am getting following error
>
> *[Replicate Request Thread-3] o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> Failed to replicate request GET /nifi-api/flow/current-user*
>
> *Please help me to figure this out.*
> --
>
>
> Regards,
> Midhun Mohan
>

Re: [VOTE] Release Apache NiFi Registry 0.6.0

[Nathan Gough]

+1 (non-binding)

Built with tests, connected a secured nifi to a secured registry. Connected
NiFi LDAP, logged in and out with LDAP and tested API to confirm the token
is invalidated as expected. Version controlled a simple flow.

On Thu, Apr 2, 2020 at 4:09 PM Aldrin Piri  wrote:

> I am of the opinion it is only a minor nuisance.  As it is an arg, it is
> easily overridden on build (this issue is also present on the Docker Hub
> item).  The script I use to publish images  explicitly provides the version
> arg and would not preclude us from having an appropriately versioned image.
>
> On Thu, Apr 2, 2020 at 4:01 PM Chris Sampson
>  wrote:
>
> > Is it a problem that you're building 0.6.0 but the docker image is
> > configured for 1.0.0 (
> >
> >
> https://github.com/apache/nifi-registry/blob/master/nifi-registry-docker-maven/dockermaven/Dockerfile
> > )
> > as per
> > https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFIREG-338?
> >
> >
> >
> > Cheers,
> >
> > Chris Sampson
> >
> > On Thu, 2 Apr 2020, 19:00 Marton Szasz,  wrote:
> >
> > > +1 (non-binding)
> > >
> > > Went through the helper guide with the correct commit ID and verified
> the
> > > release against a minifi c++ instance.
> > >
> > > Thanks,
> > > Marton
> > >
> > > On Thu, 2 Apr 2020 at 18:51, Arpad Boda 
> > > wrote:
> > >
> > > > Thanks for pointing that out, I copied wrong ID.
> > > >
> > > > The correct commit ID is: 4ddfde57f0163baa08eb782fc8be9c51d8f58af7
> > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi-registry.git;a=commit;h=4ddfde57f0163baa08eb782fc8be9c51d8f58af7
> > > > That's the commit associated with the RC tag.
> > > >
> > > > Dev,
> > > > please use this commit ID for the release verification.
> > > >
> > > > Thanks,
> > > > Arpad
> > > >
> > > >
> > > >
> > > > On Thu, Apr 2, 2020 at 6:41 PM Joe Witt  wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Did the usual checks and ran a live nifi against it which worked
> > > > > perfectly.  Did not do secured instances though.
> > > > >
> > > > > Arpdad: You want to clarify the commit that the source release is
> > based
> > > > on
> > > > > is
> > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi-registry.git;a=commit;h=4ddfde57f0163baa08eb782fc8be9c51d8f58af7
> > > > >
> > > > > That is the parent of the one you sent which has 0.6.1-SNAPSHOT.
> > > > >
> > > > > Thanks
> > > > > Joe
> > > > >
> > > > > On Thu, Apr 2, 2020 at 11:54 AM Arpad Boda 
> wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I am pleased to be calling this vote for the source release of
> > Apache
> > > > > > NiFi Registry nifi-registry-0.6.0.
> > > > > >
> > > > > > The source zip, including signatures, digests, etc. can be found
> > at:
> > > > > >
> > > https://repository.apache.org/content/repositories/orgapachenifi-1160
> > > > > >
> > > > > > The Git tag is nifi-registry-0.6.0-RC1
> > > > > > The Git commit ID is ed5c9b3f2faa8b1cab18f157b7d3263dae289aae
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi-registry.git;a=commit;h=ed5c9b3f2faa8b1cab18f157b7d3263dae289aae
> > > > > >
> > > > > > Checksums of nifi-registry-0.6.0-source-release.zip:
> > > > > > SHA256:
> > > > 0ed06ef762588be0154207932d2d1ebd1266c45aed299d0fb170cbeabd4c6e2b
> > > > > > SHA512:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> aa1f03c20902bf9cd94574e45cbd57526763ab135421e9d1df489cac7c487cb9c6d20ba3f6a74d7ae37b97a7de71d12edbc1c9fabf6d6b29f29397164011e097
> > > > > >
> > > > > > Release artifacts are signed with the following key:
> > > > > > https://people.apache.org/keys/committer/aboda.asc
> > > > > >
> > > > > > KEYS file available here:
> > > > > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > > > >
> > > > > > 40 issues were closed/resolved for this release:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320920&version=12347009
> > > > > >
> > > > > > Release note highlights can be found here:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFIREG/Release+Notes#ReleaseNotes-NiFiRegistry0.6.0
> > > > > >
> > > > > > The vote will be open for 72 hours.
> > > > > > Please download the release candidate and evaluate the necessary
> > > items
> > > > > > including checking hashes, signatures, build from source, and
> test.
> > > > > >
> > > > > > Then please vote:
> > > > > > [ ] +1 Release this package as nifi-registry-0.6.0
> > > > > > [ ] +0 no opinion
> > > > > > [ ] -1 Do not release this package because...
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: 1.11.3 trust store error

[Nathan Gough]

I've opened https://issues.apache.org/jira/browse/NIFI-7223 to track and
I'm working on a fix for this.

Nathan

On Tue, Mar 3, 2020 at 6:17 PM Nathan Gough  wrote:

> Hi Joe,
>
> Just to confirm here - was the nifi.security.keyPasswd not defined at all
> in your nifi.properties? Did you have to add the property and give it the
> correct value? Or was it in the nifi.properties file but blank? Or were the
> keyPasswd and keystorePasswd different values?
>
> Thanks,
> Nathan
>
> On Tue, Mar 3, 2020 at 3:38 PM Joe Gresock  wrote:
>
>> Yep, setting the nifi.security.keyPasswd to the same as
>> nifi.security.keystorePasswd fixed it.  Thanks for the insight, Endre!
>>
>> On Tue, Mar 3, 2020 at 2:01 PM Joe Witt  wrote:
>>
>> > relevant change I believe is here:
>> >
>> >
>> https://github.com/apache/nifi/commit/46d3b6b0dc28f04da124be7685f82bec52e88775
>> > and
>> > is from https://issues.apache.org/jira/browse/NIFI-6927
>> >
>> > It *looks* to me like this was fixing an improper naming/usage issue
>> that
>> > has been present but if so we probably should have addressed not in this
>> > bug fix line.  Will defer to Troy/Andy for more context and next steps
>> >
>> > On Tue, Mar 3, 2020 at 5:53 AM Joe Witt  wrote:
>> >
>> > > If accurateWe need to look into whether this was a mistake and
>> fix it
>> > > if so.  And we need to reflect this in the migration guide
>> > >
>> > > On Tue, Mar 3, 2020 at 4:40 AM Ryan Ward 
>> wrote:
>> > >
>> > >> Endre  - thanks that was it
>> > >>
>> > >> On Tue, Mar 3, 2020 at 6:50 AM Endre Kovacs
>> > >>  wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > One additional thing:
>> > >> >
>> > >> > we encountered something strange as well:
>> > >> >
>> > >> > on 1.11.2 clustered, kerberized: request replication worked well.
>> > >> >
>> > >> > on 1.11.3 clustered, kerberized: request replication did not work,
>> > >> unless
>> > >> > you specify, and set
>> > >> > nifi.security.keyPasswd
>> > >> >
>> > >> > to the very same password as the
>> > >> >
>> > >> > nifi.security.keystorePasswd
>> > >> >
>> > >> > For us this resolved the issue.
>> > >> >
>> > >> > Best regards,
>> > >> > Endre
>> > >> >
>> > >> > Sent with [ProtonMail](https://protonmail.com) Secure Email.
>> > >> >
>> > >> > ‐‐‐ Original Message ‐‐‐
>> > >> > On Tuesday, March 3, 2020 12:40 PM, Ryan Ward <
>> ryan.wa...@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> > > Hi Joe - Did you resolve your issue? If so I am wondering what
>> the
>> > fix
>> > >> > was as I'm seeing the same error on my cluster.
>> > >> > >
>> > >> > > On Thu, Feb 27, 2020 at 3:13 AM Endre Kovacs <
>> > >> > andrewsmit...@protonmail.com.invalid> wrote:
>> > >> > >
>> > >> > >> Hi Joe,
>> > >> > >>
>> > >> > >> 1.  Have you tried connecting/debugging with openssl? From one
>> pod
>> > to
>> > >> > the other:
>> > >> > >> (openssl s_client -debug -CAfile
>> > >> > ca-bundle-signing-node-certificates.crt -cert my-client-cert.crt
>> > >> -connect
>> > >> > nifi-3.nifi-headless.lizardspock.svc.cluster.local:6007)
>> > >> > >>
>> > >> > >> 2. certs can also be verified by:
>> > >> > >>  openssl verify -verbose -CAfile ca-bundle.crt
>> my-client-cert.crt
>> > >> > >>
>> > >> > >> 3.  Can you check if no intermediary CAs are missing from the
>> nodes
>> > >> > truststore?
>> > >> > >>
>> > >> > >> 4.  This exception is coming from inter-node communication
>> > >> (replication
>> > >> > of request from one node to the other). This means that it is
>> > unrelated
>> > >> to
>> > >> > external user's a

Re: 1.11.3 trust store error

[Nathan Gough]

Hi Joe,

Just to confirm here - was the nifi.security.keyPasswd not defined at all
in your nifi.properties? Did you have to add the property and give it the
correct value? Or was it in the nifi.properties file but blank? Or were the
keyPasswd and keystorePasswd different values?

Thanks,
Nathan

On Tue, Mar 3, 2020 at 3:38 PM Joe Gresock  wrote:

> Yep, setting the nifi.security.keyPasswd to the same as
> nifi.security.keystorePasswd fixed it.  Thanks for the insight, Endre!
>
> On Tue, Mar 3, 2020 at 2:01 PM Joe Witt  wrote:
>
> > relevant change I believe is here:
> >
> >
> https://github.com/apache/nifi/commit/46d3b6b0dc28f04da124be7685f82bec52e88775
> > and
> > is from https://issues.apache.org/jira/browse/NIFI-6927
> >
> > It *looks* to me like this was fixing an improper naming/usage issue that
> > has been present but if so we probably should have addressed not in this
> > bug fix line.  Will defer to Troy/Andy for more context and next steps
> >
> > On Tue, Mar 3, 2020 at 5:53 AM Joe Witt  wrote:
> >
> > > If accurateWe need to look into whether this was a mistake and fix
> it
> > > if so.  And we need to reflect this in the migration guide
> > >
> > > On Tue, Mar 3, 2020 at 4:40 AM Ryan Ward  wrote:
> > >
> > >> Endre  - thanks that was it
> > >>
> > >> On Tue, Mar 3, 2020 at 6:50 AM Endre Kovacs
> > >>  wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > One additional thing:
> > >> >
> > >> > we encountered something strange as well:
> > >> >
> > >> > on 1.11.2 clustered, kerberized: request replication worked well.
> > >> >
> > >> > on 1.11.3 clustered, kerberized: request replication did not work,
> > >> unless
> > >> > you specify, and set
> > >> > nifi.security.keyPasswd
> > >> >
> > >> > to the very same password as the
> > >> >
> > >> > nifi.security.keystorePasswd
> > >> >
> > >> > For us this resolved the issue.
> > >> >
> > >> > Best regards,
> > >> > Endre
> > >> >
> > >> > Sent with [ProtonMail](https://protonmail.com) Secure Email.
> > >> >
> > >> > ‐‐‐ Original Message ‐‐‐
> > >> > On Tuesday, March 3, 2020 12:40 PM, Ryan Ward  >
> > >> > wrote:
> > >> >
> > >> > > Hi Joe - Did you resolve your issue? If so I am wondering what the
> > fix
> > >> > was as I'm seeing the same error on my cluster.
> > >> > >
> > >> > > On Thu, Feb 27, 2020 at 3:13 AM Endre Kovacs <
> > >> > andrewsmit...@protonmail.com.invalid> wrote:
> > >> > >
> > >> > >> Hi Joe,
> > >> > >>
> > >> > >> 1.  Have you tried connecting/debugging with openssl? From one
> pod
> > to
> > >> > the other:
> > >> > >> (openssl s_client -debug -CAfile
> > >> > ca-bundle-signing-node-certificates.crt -cert my-client-cert.crt
> > >> -connect
> > >> > nifi-3.nifi-headless.lizardspock.svc.cluster.local:6007)
> > >> > >>
> > >> > >> 2. certs can also be verified by:
> > >> > >>  openssl verify -verbose -CAfile ca-bundle.crt my-client-cert.crt
> > >> > >>
> > >> > >> 3.  Can you check if no intermediary CAs are missing from the
> nodes
> > >> > truststore?
> > >> > >>
> > >> > >> 4.  This exception is coming from inter-node communication
> > >> (replication
> > >> > of request from one node to the other). This means that it is
> > unrelated
> > >> to
> > >> > external user's authentication by client certificate. The question
> is:
> > >> is
> > >> > your inter node communication secured by the trusted root CA (that
> you
> > >> are
> > >> > sure that the CA cert is present in the trust store) or is it
> secured
> > by
> > >> > selfsigned CA (which's CA may be lacking from your truststore)?
> > >> > >>
> > >> > >> 5.  `nifi.security.needClientAuth` is not part of NiFi properties
> > any
> > >> > more. If SSL is turned on, and no
> > >> > `nifi.security.user.login.identity.provider` is set, then client
> cert
> > >> based
> > >> > auth is the default. But supplying this property have no detrimental
> > >> effect
> > >> > anyhow.
> > >> > >>
> > >> > >> Best regards,
> > >> > >> Endre
> > >> > >>
> > >> > >> Sent with ProtonMail Secure Email.
> > >> > >>
> > >> > >> ‐‐‐ Original Message ‐‐‐
> > >> > >> On Wednesday, February 26, 2020 6:22 PM, Joe Gresock
> > >> > jgres...@gmail.com wrote:
> > >> > >>
> > >> > >>> Were there any changes with how the trust store is used in
> > 1.11.3? I
> > >> > had a
> > >> > >>> 1.11.0 deployment working with the following settings, but when
> I
> > >> > deployed
> > >> > >>> 1.11.3, the cluster can't seem to replicate requests to itself:
> > >> > >>> nifi.remote.input.host=
> > >> > >>> nifi.remote.input.secure=true
> > >> > >>> nifi.remote.input.socket.port=32440
> > >> > >>> nifi.remote.input.http.enabled=true
> > >> > >>> nifi.cluster.protocol.is.secure=true
> > >> > >>> nifi.cluster.is.node=true
> > >> > >>>
> > >> >
> > >>
> >
> nifi.cluster.node.address=nifi-3.nifi-headless.lizardspock.svc.cluster.local
> > >> > >>> nifi.cluster.node.protocol.port=6007
> > >> > >>>
> > >> nifi.web.https.host=nifi-3.nifi-headless.lizardspock.svc.cluster.local
> > >> > >>> nifi.web.htt

Re: 1.11.3 trust store error

[Nathan Gough]

Hi Joe,

I just set up a secure cluster with NiFi 1.11.3 and am not seeing any
issues like you describe.

Are you running Java 8 or Java 11?

Nathan

On Wed, Feb 26, 2020 at 12:22 PM Joe Gresock  wrote:

> Were there any changes with how the trust store is used in 1.11.3?  I had a
> 1.11.0 deployment working with the following settings, but when I deployed
> 1.11.3, the cluster can't seem to replicate requests to itself:
>
> nifi.remote.input.host=
> nifi.remote.input.secure=true
> nifi.remote.input.socket.port=32440
> nifi.remote.input.http.enabled=true
>
> nifi.cluster.protocol.is.secure=true
> nifi.cluster.is.node=true
>
> nifi.cluster.node.address=nifi-3.nifi-headless.lizardspock.svc.cluster.local
> nifi.cluster.node.protocol.port=6007
>
> nifi.web.https.host=nifi-3.nifi-headless.lizardspock.svc.cluster.local
> nifi.web.https.port=8443
>
> nifi.security.keystore=./conf/keystore.jks
> nifi.security.keystoreType=jks
> nifi.security.keystorePasswd=
> nifi.security.keyPasswd=
> nifi.security.truststore=./conf/truststore.jks
> nifi.security.truststoreType=jks
> nifi.security.truststorePasswd=
> nifi.security.needClientAuth=true
>
> A trusted client cert that worked against the old cluster is getting the
> same trust error (PKIX path building failed).  I've verified that the
> client cert was issued by an issuer that is definitely in the
> ./conf/truststore.jks as a trustedCertEntry.
>
> 2020-02-26 17:11:09,573 WARN [Replicate Request Thread-7]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to r
> equested target
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
> at
>
> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
> at
> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
> at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
> at
> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
> at
>
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
> at
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
> at
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
> at
>
> okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:302)
> at
>
> okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270)
> at
> okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162)
> at
>
> okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)
> at
>
> okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
> at
>
> okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
> at
>
> okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
> okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
>
> okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
>
> okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
> at
>
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
> at
> okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
> at okhttp3.RealCall.execute(RealCall.java:77)
> at
>
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:143)
> at
>
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:137)
> at
>
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.

[ANNOUNCE] Apache NiFi 1.11.0 vulnerability announcements

[Nathan Gough]

Apache NiFi Community,

The https://nifi.apache.org/security.html page has been updated with 2
vulnerabilities discovered in previous NiFi versions which have been
resolved in release 1.11.0. The severity of these were determined to be one
'high' and one 'moderate'. Dependency vulnerabilities that were patched
have also been published. Questions about these vulnerabilities can be
directed to secur...@nifi.apache.org.

If you identify new security issues within the NiFi 1.11.0 release, please
forward your report to secur...@nifi.apache.org and do not disclose the
issue publicly. The security vulnerability reporting and disclosure process
can be found here: https://www.apache.org/security/committers.html.

Regards,
Nathan

Re: [VOTE] Release Apache NiFi 1.11.0 (rc3)

[Nathan Gough]

+1 non-binding

Verified hashes, signing key, did a 'mvn clean install
-Pcontrib-check,include-grpc' and ran a secured NiFi cluster with the
resulting bin.

Nathan

On Wed, Jan 22, 2020 at 11:51 AM Shawn Weeks 
wrote:

> +1 non-binding
>
> On 1/22/20, 10:48 AM, "Shawn Weeks"  wrote:
>
> Noticed that the docker compose file
> nifi-docker/docker-compose/docker-compose.yml still references
> 1.10.0-SNAPSHOT-dockermaven in RC3. Not worth holding up for though.
>
> Thanks
> Shawn
>
> On 1/22/20, 9:55 AM, "Kevin Doran"  wrote:
>
> +1 (binding)
>
> I followed the steps in the release helper guide and verified the
> signs, hashes, readme, license, etc.
> I was able to performa fun build with tests without any issues on
> macOS 10.15 (Catalina).
> The resulting binary runs well for me running a standard suite of
> test flows.
>
> One issue I ran into was building the Docker image (via
> dockermaven) from the release source code using the -Pdocker profile. I was
> unable to build a usable docker image from the source code due to
> assumptions the Dockerfile makes about the permissions of some shell
> scripts. It appears at some point the file permissions for the shell
> scripts included in this release were changed and are no longer executable.
> Not a huge deal so long as we can still build a Dockerhub image for this
> release once finalized, but just something to note as perhaps we could
> improve our automated image creation process to be more robust.
>
> Thanks for everyone who has worked on the improvements landing in
> this release and thanks to Joe for RM’ing again!
>
> Kevin
>
> > On Jan 21, 2020, at 16:43, Andrew Lim <
> andrewlim.apa...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > -Ran full clean install on OS X (Catalina 10.15.2)
> > -Tested secure NiFi with secure NiFi Registry
> > -Ran basic flows successfully
> > -Reviewed core UI and documentation fixes/updates
> >
> > In setting up my secure NiFi and secure NiFi registry, I used
> the NiFi TLS Toolkit [1] to create my config files and certs. I was able to
> access the UIs of both apps using Safari but not able to with Chrome due to
> a NET::ERR_CERT_REVOKED error which I had never seen before.  Turns out
> this is a known issue on Catalina [2]. MacOSX 10.15 requires certs to be
> valid for 825 days or less and a minimum 2048 bit key.  By default, the TLS
> Toolkit sets the validity to 1095 days and the number of bits for generated
> keys to 2048. Creating new certs with the required 825 validity solved the
> issue. I will add a note to the Toolkit Guide for this new requirement [3].
> >
> > Drew
> >
> > [1]
> https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_toolkit
> <
> https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_toolkit
> >
> > [2] https://support.apple.com/en-us/HT210176 <
> https://support.apple.com/en-us/HT210176>
> > [3] https://issues.apache.org/jira/browse/NIFI-7053 <
> https://issues.apache.org/jira/browse/NIFI-7053>
> >
> >
> >> On Jan 19, 2020, at 3:21 PM, Joe Witt 
> wrote:
> >>
> >> Hello,
> >>
> >> I am pleased to be calling this vote for the source release of
> Apache NiFi
> >> nifi-1.11.0.
> >>
> >> The source zip, including signatures, digests, etc. can be
> found at:
> >>
> https://repository.apache.org/content/repositories/orgapachenifi-1155
> >>
> >> The source being voted upon and the convenience binaries can be
> found at:
> >> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.11.0/
> >>
> >> A helpful reminder on how the release candidate verification
> process works:
> >>
> https://cwiki.apache.org/confluence/display/NIFI/How+to+help+verify+an+Apache+NiFi+release+candidate
> >>
> >> The Git tag is nifi-1.11.0-RC3
> >> The Git commit ID is 633408bce7ad34dad727ed9c4edfd36a224f3f12
> >>
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=633408bce7ad34dad727ed9c4edfd36a224f3f12
> >>
> >> Checksums of nifi-1.11.0-source-release.zip:
> >> SHA256:
> 0e2d77265fc7cedfbdb9588df1dd7f456fd18b6288d65eb5e21befe23af7c567
> >> SHA512:
> >>
> 4880fa3482b3e8d8eed439848fe0a6596826d7ad46425a91b0dd4a4bcd178259327380b24045b7991dbdf8449abdfdda145786b6863eb603f6cef3b9e0ae8ec1
> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/joewitt.asc
> >>
> >> KEYS file available here:
> >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>
> >> 129 issues were closed/resolved for this release:
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=123160

Re: Not able to add SSL Certificates to Nifi Cluster

[Nathan Gough]

Hi Uma,

Attached images do not come through to mailing lists. You will need to
post the error as text or send a URL to the image.

Nathan

On Mon, Nov 25, 2019 at 10:39 AM Umasri Vullanki
 wrote:

> Hi Team,
>
> For secure cluster setup,
>
> -> Initially created a 2-node cluster with all the configurations and it
> worked fine. So, I tried to add SSL certificates to it for which I have
> downloaded nifi-toolkit and extracted it. Then for generating
> certificates ran the following command on one of my servers.
>
>
> Here nifi-1 and nifi-2 are hostnames of my servers.
>
> ./bin/tls-toolkit.sh standalone -n nifi-1,nifi-2 -K password -P password
>
> -> After running the above command, these folders and files got generated
>
> Files generated: truststore.jks, keystore.jks, nifi.properties,
> nifi-cert.pem, nifi-key.key
>
> [image: image.png]
>
> ->So, moved nifi-2 folder, nifi-cert.pem and  nifi-key.key to other
> servers (cluster node)
>
> -> Modified nifi-toolkit/nifi.properties and conf/nifi.properties i.e.
> made both the contents of the file as same configurations added
> certifications path, given encrypted passwords and cluster configurations
> in both the nodes
>
> ->  Modified the authorizers.xml as follows:
>
> -->
>
>  
>
> file-provider
>
> org.apache.nifi.authorization.FileAuthorizer
>
> ./conf/authorizations.xml
>
> ./conf/users.xml
>
> CN=admin,
> OU=NIFI
>
> 
>
> CN=nifi-1, OU=NIFI
>
> CN=nifi-2, OU=NIFI
>
> 
>
> -> Now started the cluster in the both nodes
>
> We are getting the below error:
>
> [image: image.png]
>
> -> For this, we tried to delete the existing keys and generated the new
> key pairs and followed the same procedure and started the cluster, but even
> then we are facing the same issue.
>
>
>
> Thanks,
> Uma Sri Vullanki
>

Re: Minimum zookeeper version

[Nathan Gough]

Hi Mark,

The minimum version for an external zookeeper should be 3.5.5. Are you
experiencing any issues?

Nathan

On Fri, Nov 22, 2019 at 3:30 PM Mark Bean  wrote:

> For NiFi 1.10.0, what is the minimum version of zookeeper when using an
> external zookeeper instance?
>
> Thanks,
> Mark
>

Apache NiFi Vulnerability Announcement (NiFi 1.10.0 release)

[Nathan Gough]

Apache NiFi Community,

The https://nifi.apache.org/security.html page has been updated with 3
vulnerabilities discovered in previous NiFi versions which have been
resolved in release 1.10.0. The severity of these were determined to be two
'low' and one 'medium'. Dependency vulnerabilities that were patched have
also been published. Questions about these vulnerabilities can be directed
to secur...@nifi.apache.org.

If you identify new security issues within the current NiFi 1.10.0 release,
please forward your report to secur...@nifi.apache.org and do not disclose
the issue publicly. The security vulnerability reporting and disclosure
process can be found here: https://www.apache.org/security/committers.html.

Regards,
Nathan

Re: [VOTE] Release Apache NiFi 1.9.1 (rc1)

[Nathan Gough]

+1 (non-binding)

- Verified signature
- Verified checksums
- mvn contrib executed successfully
- Created simple test flow in both standalone and clustered modes, secure
and insecure
- Checked license and readme files

Nathan



On Wed, Mar 13, 2019 at 6:33 PM Arpad Boda  wrote:

> +1
>
> -Verified signature
> -Verified checksum
> -Built, executed tests
> -Created a simple flow
> -Sent flow files both using raw and HTTP S2S, verified them.
>
> On 13/03/2019, 22:24, "Rob Fellows" 
> wrote:
>
> +1
>
> Went through the Release Helper Guide.
> Started the app and created a simple data flow. It seemed good to me.
>
> Thanks,
> Rob
>
> On Wed, Mar 13, 2019 at 3:16 PM Mark Payne 
> wrote:
>
> > +1 (binding)
> >
> > Verified signature, hashes. Build succeeded with contrib-check and
> grpc
> > profiles.
> > Started app and performed some basic functionality testing and all
> seemed
> > correct.
> >
> > Thanks
> > -Mark
> >
> > > On Mar 13, 2019, at 1:49 AM, Joe Witt  wrote:
> > >
> > > Hello,
> > >
> > > I am pleased to be calling this vote for the source release of
> Apache
> > NiFi
> > > 1.9.1.
> > >
> > > The source zip, including signatures, digests, etc. can be found
> at:
> > >
> https://repository.apache.org/content/repositories/orgapachenifi-1138
> > > https://dist.apache.org/repos/dist/dev/nifi/nifi-1.9.1-rc1/
> > >
> > > The Git tag is nifi-1.9.1-RC1
> > > The Git commit ID is a5cedc4ad39b17bee97303b63b620f9ac3dddc79
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=a5cedc4ad39b17bee97303b63b620f9ac3dddc79
> > >
> > > Checksums of nifi-1.9.1-source-release.zip:
> > > SHA256:
> 7099abb33e26445788630b69f38b8788117cdd787b7001752b4893d8b6c16f38
> > > SHA512:
> > >
> >
> 678c2ee32f7db8c73393178f329c574315b1b892084b822f9b7a6dc5bc159d5e7e1169812d9676a72f738d03fd2f4366f2b67ddee152b56c8a77751fd5cbb218
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/joewitt.asc
> > >
> > > KEYS file available here:
> > > https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >
> > > 19 issues were closed/resolved for this release:
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12345163
> > >
> > > Release note highlights can be found here:
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.9.1
> > >
> > > The vote will be open for 72 hours.
> > > Please download the release candidate and evaluate the necessary
> items
> > > including checking hashes, signatures, build
> > > from source, and test. Then please vote:
> > >
> > > [ ] +1 Release this package as nifi-1.9.1
> > > [ ] +0 no opinion
> > > [ ] -1 Do not release this package because...
> >
> >
>
> --
> Thanks,
> Rob Fellows
>
>
>

Re: [VOTE] Release Apache NiFi 1.8.0

[Nathan Gough]

-1 (revote)

On further testing I have found that the SSLContextService does not work as 
expected due to this ticket https://jira.apache.org/jira/browse/NIFI-4558 and 
the related PR. This makes it difficult or impossible to use the 
SSLContextService as I believe the customValidate() method takes in user input 
and not the default settings. I would say this is a blocker for nifi-1.8.0-RC1. 
I have submitted a PR to revert this change: 
https://github.com/apache/nifi/pull/3097.

My apologies,
Nathan

On 10/19/18, 3:29 PM, "Andrew Lim"  wrote:

+1 (non-binding)

-Ran full clean install on OS X (10.11.6)
-Tested secure cluster; used UI to: disconnect/connect nodes; offload 
nodes; load balance connections
-Reviewed documentation

Drew

> On Oct 17, 2018, at 11:59 PM, Jeff  wrote:
> 
> Hello,
> 
> I am pleased to be calling this vote for the source release of Apache NiFi
> nifi-1.8.0.
> 
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1133
> 
> The Git tag is nifi-1.8.0-RC1
> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> 
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> 
> Checksums of nifi-1.8.0-source-release.zip:
> SHA256: 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> SHA512:
> 
8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jstorck.asc
> 
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
> 
> 204 issues were closed/resolved for this release:
> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12343482
> 
> Release note highlights can be found here:
> 
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
> 
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test. Then please vote:
> 
> [ ] +1 Release this package as nifi-1.8.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...

Re: [VOTE] Release Apache NiFi 1.8.0

[Nathan Gough]

+1

Verified signatures and hashes.
Built from source and tested my changes for the release.
Checked the README/NOTICE/LICENSE files


On 10/19/18, 7:23 AM, "Marc Parisi"  wrote:

+1 binding
  Did the usual release helper validation, tested with typical use cases
and test flows, with and without minifi in a secure env.

On Fri, Oct 19, 2018 at 1:09 AM Andrew Psaltis 
wrote:

> +1 non-binding
>
> Verified README.md, NOTICE, checksum, keys, Documentation for Load
> Balancing
> Ran various flows -- new features looking geat -- load balancing, JNDI JMS
>
> Great looking release.
>
> On Thu, Oct 18, 2018 at 12:59 PM Jeff  wrote:
>
> > Hello,
> >
> > I am pleased to be calling this vote for the source release of Apache
> NiFi
> > nifi-1.8.0.
> >
> > The source zip, including signatures, digests, etc. can be found at:
> > https://repository.apache.org/content/repositories/orgapachenifi-1133
> >
> > The Git tag is nifi-1.8.0-RC1
> > The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> >
> >
> 
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> >
> > Checksums of nifi-1.8.0-source-release.zip:
> > SHA256: 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> > SHA512:
> >
> >
> 
8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/jstorck.asc
> >
> > KEYS file available here:
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> > 204 issues were closed/resolved for this release:
> >
> >
> 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12343482
> >
> > Release note highlights can be found here:
> >
> >
> 
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
> >
> > The vote will be open for 72 hours.
> > Please download the release candidate and evaluate the necessary items
> > including checking hashes, signatures, build
> > from source, and test. Then please vote:
> >
> > [ ] +1 Release this package as nifi-1.8.0
> > [ ] +0 no opinion
> > [ ] -1 Do not release this package because...
> >
>

Re: [DISCUSS] Closing in on a release of NiFi 1.8.0?

[Nathan Gough]

So, taking a closer look at this PR and doing some further testing I've found 
that upgrading Guava is going to require some more careful assessment, as the 
changes will affect core NiFi functionality around clustering. I will hold off 
on this change until after the release to have more time to upgrade and test.

Nathan

On 10/16/18, 10:58 AM, "Nathan Gough"  wrote:

Hi Mike,

Sure I can look at fixing up PR-2977 today. 

Nathan

On 10/16/18, 6:13 AM, "Mike Thomsen"  wrote:

Does 5562 need to be addressed in 1.8?

https://github.com/apache/nifi/pull/2977

On Mon, Oct 15, 2018 at 6:33 PM Jeff  wrote:

> NiFi Devs,
>
> The Release page [1] for 1.8.0 now reports that all issues are done!  
I'd
> like to start the release candidate preparation tomorrow, around 1200 
EST.
>
> Thanks to everyone for all the great work that's been done!  196 
issues
> resolved in this version with some great new features!
>
> [1] https://issues.apache.org/jira/projects/NIFI/versions/12343482
>
> On Mon, Oct 15, 2018 at 7:30 AM Sivaprasanna 

> wrote:
>
> > Great. Thanks. :)
> >
> > -
> > Sivaprasanna
> >
> > On Mon, Oct 15, 2018 at 7:09 AM Koji Kawamura 

> > wrote:
> >
> > > Jeff, Sivasprasanna,
> > >
> > > NIFI-5698 (PR3073) Fixing DeleteAzureBlob bug is merged.
> > >
> > > Thanks,
> > > Koji
> > > On Mon, Oct 15, 2018 at 10:18 AM Koji Kawamura 
 >
> > > wrote:
> > > >
> > > > Thank you for the fix Sivaprasanna,
> > > > I have Azure account. Reviewing it now.
> > > >
> > > > Koji
> > > > On Sun, Oct 14, 2018 at 11:21 PM Jeff  wrote:
> > > > >
> > > > > Sivaprasanna,
> > > > >
> > > > > Thanks for submitting a pull request for that issue!  Later 
today
> or
> > > > > tomorrow I'll have to check to see if I've already used up my
> > free-tier
> > > > > access to Azure.  If I still have access, I can review your 
PR and
> > > we'll
> > > > > get it into 1.8.0.
> > > > >
> > > > > On Sun, Oct 14, 2018 at 4:30 AM Sivaprasanna <
> > > sivaprasanna...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > All - Just found one bug with DeleteAzureBlobStorage 
processor.
> It
> > > was
> > > > > > shared by one user on StackOverflow [1] and I later 
confirmed it.
> > It
> > > looks
> > > > > > to be introduced by NIFI-4199. I have created a Jira [2] 
and made
> > the
> > > > > > necessary changes (not huge, just few lines) and raised a PR
> [3]. I
> > > think,
> > > > > > if we can spend a little time in getting it reviewed, we 
can mark
> > it
> > > for
> > > > > > 1.8.0. Thoughts?
> > > > > >
> > > > > > [1] -
> > > > > >
> > > > > >
> > >
> >
> 
https://stackoverflow.com/questions/52766991/apache-nifi-deleteazureblobstorage-processor-is-throwing-an-error
> > > > > > [2] - https://issues.apache.org/jira/browse/NIFI-5698
> > > > > > [3] - https://github.com/apache/nifi/pull/3073
> > > > > >
> > > > > > -
> > > > > > Sivaprasanna
> > > > > >
> > > > > > On Fri, Oct 12, 2018 at 9:05 PM Mike Thomsen <
> > mikerthom...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > 4811 should be ready for review now. Rebased and cleaned 
it up
> > > with a
> > > > > > full
> > > > > > > listing of the Spring dependencies.
> > > > > > >
> > > > > >

Re: [DISCUSS] Closing in on a release of NiFi 1.8.0?

[Nathan Gough]

Hi Mike,

Sure I can look at fixing up PR-2977 today. 

Nathan

On 10/16/18, 6:13 AM, "Mike Thomsen"  wrote:

Does 5562 need to be addressed in 1.8?

https://github.com/apache/nifi/pull/2977

On Mon, Oct 15, 2018 at 6:33 PM Jeff  wrote:

> NiFi Devs,
>
> The Release page [1] for 1.8.0 now reports that all issues are done!  I'd
> like to start the release candidate preparation tomorrow, around 1200 EST.
>
> Thanks to everyone for all the great work that's been done!  196 issues
> resolved in this version with some great new features!
>
> [1] https://issues.apache.org/jira/projects/NIFI/versions/12343482
>
> On Mon, Oct 15, 2018 at 7:30 AM Sivaprasanna 
> wrote:
>
> > Great. Thanks. :)
> >
> > -
> > Sivaprasanna
> >
> > On Mon, Oct 15, 2018 at 7:09 AM Koji Kawamura 
> > wrote:
> >
> > > Jeff, Sivasprasanna,
> > >
> > > NIFI-5698 (PR3073) Fixing DeleteAzureBlob bug is merged.
> > >
> > > Thanks,
> > > Koji
> > > On Mon, Oct 15, 2018 at 10:18 AM Koji Kawamura  >
> > > wrote:
> > > >
> > > > Thank you for the fix Sivaprasanna,
> > > > I have Azure account. Reviewing it now.
> > > >
> > > > Koji
> > > > On Sun, Oct 14, 2018 at 11:21 PM Jeff  wrote:
> > > > >
> > > > > Sivaprasanna,
> > > > >
> > > > > Thanks for submitting a pull request for that issue!  Later today
> or
> > > > > tomorrow I'll have to check to see if I've already used up my
> > free-tier
> > > > > access to Azure.  If I still have access, I can review your PR and
> > > we'll
> > > > > get it into 1.8.0.
> > > > >
> > > > > On Sun, Oct 14, 2018 at 4:30 AM Sivaprasanna <
> > > sivaprasanna...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > All - Just found one bug with DeleteAzureBlobStorage processor.
> It
> > > was
> > > > > > shared by one user on StackOverflow [1] and I later confirmed 
it.
> > It
> > > looks
> > > > > > to be introduced by NIFI-4199. I have created a Jira [2] and 
made
> > the
> > > > > > necessary changes (not huge, just few lines) and raised a PR
> [3]. I
> > > think,
> > > > > > if we can spend a little time in getting it reviewed, we can 
mark
> > it
> > > for
> > > > > > 1.8.0. Thoughts?
> > > > > >
> > > > > > [1] -
> > > > > >
> > > > > >
> > >
> >
> 
https://stackoverflow.com/questions/52766991/apache-nifi-deleteazureblobstorage-processor-is-throwing-an-error
> > > > > > [2] - https://issues.apache.org/jira/browse/NIFI-5698
> > > > > > [3] - https://github.com/apache/nifi/pull/3073
> > > > > >
> > > > > > -
> > > > > > Sivaprasanna
> > > > > >
> > > > > > On Fri, Oct 12, 2018 at 9:05 PM Mike Thomsen <
> > mikerthom...@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > 4811 should be ready for review now. Rebased and cleaned it up
> > > with a
> > > > > > full
> > > > > > > listing of the Spring dependencies.
> > > > > > >
> > > > > > > On Fri, Oct 12, 2018 at 11:23 AM Joe Witt 
> > > wrote:
> > > > > > >
> > > > > > > > Jeff,
> > > > > > > >
> > > > > > > > I think for anything not tagged to 1.8.0 we just keep
> rolling.
> > > For
> > > > > > > > anything tagged 1.8.0 that should not be we should remove it
> > > until
> > > > > > > > ready.  For things tagged to 1.8.0 that cannot be moved we
> > should
> > > > > > > > resolve.  For the tagged 1.8.0 section you had.
> > > > > > > >
> > > > > > > >- NIFI-4811 <
> > https://issues.apache.org/jira/browse/NIFI-4811>
> > > -
> > > > > > Use a
> > > > > > > >newer version of spring-data-redis
> > > > > > > >- PR 2856 
> > > > > > > > *This needs to be resolved by either reverting the commit or
> > > ensuring
> > > > > > > > L&N accurately reflects all.  We have to do this always and
> for
> > > every
> > > > > > > > nar.  The process isnt easy or fun but it is necessary to
> > produce
> > > > > > > > valid ASF releases.  Landing commits which change
> dependencies
> > > > > > > > requires this due diligence.  Now, we've put a lot of energy
> > into
> > > > > > > > updating Spring dependencies because some older Spring libs
> had
> > > > > > > > vulnerabilities which while we likely aren't exposed to them
> we
> > > want
> > > > > > > > to fix in due course.  So reverting may require more 
analysis
> > > than if
> > > > > > > > we were just get L&N fixed with this new change.  I 
commented
> > on
> > > the
> > > > > > > > JIRA.  But this needs to be resolved.
> > > > > > > >
> > > > > > > >
> > > > > > > >- NIFI-5426 <
> > https://issues.apache.org/jira/

Re: Zookeeper - help!

[Nathan Gough]

I think you are correct on that, I assumed it was a range of some kind but it 
looks like it's not: 
http://zookeeper.apache.org/doc/r3.4.3/zookeeperStarted.html#sc_RunningReplicatedZooKeeper


On 10/2/18, 5:17 PM, "Phil H"  wrote:

The second port in the zookeeper server config has been a mystery to me.  I 
thought it was a second port used for elections, not the upper bound in a 
range.  Why is the range so large?

Sent from Mail for Windows 10

    From: Nathan Gough
Sent: Wednesday, 3 October 2018 1:26 AM
To: dev@nifi.apache.org
Subject: Re: Zookeeper - help!

Check your configs on nifi2. I don't believe that NiFi is starting two 
instances of Zookeeper but the ports configured are unintentionally configured 
to overlap ie. Ports used twice in different configs where they should be 
different.

It may be that your zookeeper.properties has:

clientPort=2180
...
server.1=nifi1.com:2180:3888
server.2=nifi2.com:2180:3888

where it should be:

clientPort=2180
...
server.1=nifi1.com:2888:3888
server.2=nifi2.com:2888:3888

noticing that the server.1 and server.2 ranges don't overlap with the 
client port.


Not sure if this helps, but the following is the relevant config that I 
have for my NiFi cluster nodes that run on the SAME machine where nifi1.com and 
nifi2.com are configured in /etc/hosts:

nifi1/conf
zookeeper.properties
- clientPort=2180
- server.1=nifi1.com:2888:3888
- server.2=nifi2.com:2888:3888

nifi.properties
- nifi.remote.input.host=nifi1.com
- nifi.remote.input.socket.port=10440
- nifi.web.http.host=nifi1.com
- nifi.web.http.port=9550
- nifi.cluster.node.address=nifi1.com
- nifi.cluster.node.protocol.port=11440

nifi1/state/zookeeper
/myid (file contents = "1")
/state-management.xml (no changes required)
/version-2/


nifi2/conf
zookeeper.properties
- clientPort=2181
- server.1=nifi1.com:2888:3888
- server.2=nifi2.com:2888:3888

nifi.properties
- nifi.remote.input.host=nifi2.com
- nifi.remote.input.socket.port=10441
- nifi.web.http.host=nifi2.com
- nifi.web.http.port=9551
- nifi.cluster.node.address=nifi2.com
- nifi.cluster.node.protocol.port=11441

nifi2/state/zookeeper
/myid (file contents = "2")
/state-management.xml (no changes required)
/version-2/


Nathan



On 10/2/18, 2:07 AM, "Phil H"  wrote:

Hi Andy,

Thanks for the additional info.  I think I saw a link to that while 
searching but was wary since it was such an old version.

I have two VMs (nifi1, and nifi2) both running NiFi with identical 
configs, and trying to use the inbuilt ZK to cluster them.

If I only mention a single machine within the config (eg: if nifi1 
doesn’t refer to nifi2, or visa versa) I don’t get any start up errors.

Phil

From: Andy LoPresto
Sent: Tuesday, 2 October 2018 1:00 PM
To: dev@nifi.apache.org
Subject: Re: Zookeeper - help!

Hi Phil, 

Nathan’s advice is correct but I think he was assuming all other 
configurations are correct as well. Are you trying to run both NiFi nodes and 
ZK instances on the same machine? In that case you will have to ensure that the 
ports in use are different for each service so they don’t conflict. Setting 
them all to the same value only works if each service is running on an 
independent physical machine, virtual machine, or container. 

I find Pierre’s guide [1] to be a helpful step-by-step instruction list 
as well as a good explanation of how the clustering concepts work in practice. 
When you get that working, and you’re ready to set up a secure cluster, he has 
a follow-on guide for that as well [2]. Even as someone who has set up many 
clustered instances of NiFi, I use his guides regularly to ensure I haven’t 
forgotten a step. 

They were originally written for versions 1.0.0 and 1.1.0, but the only 
thing that has changed is the authorizer configuration for the secure instances 
(you’ll need to put the Initial Admin Identity and Node Identities in two 
locations in the authorizers.xml file instead of just once). 

Hopefully this helps you get a working cluster up and running so you 
can experiment. Good luck. 

[1] 
https://pierrevillard.com/2016/08/13/apache-nifi-1-0-0-cluster-setup/
[2] 
https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secured-cluster-setup/


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Oct 1, 2018, at 2:45 PM, Phil H  wr

Re: Zookeeper - help!

[Nathan Gough]

And I forgot the connect.string config:

nifi1/conf/nifi.properties:nifi.zookeeper.connect.string=nifi1.com:2180,nifi1.com:2181
nifi2/conf/nifi.properties:nifi.zookeeper.connect.string=nifi2.com:2180,nifi2.com:2181



Note that the configuration I've given is used for dev purposes. In a 
production environment Zookeeper needs to run on an odd-number of nodes: 
http://www.corejavaguru.com/blog/bigdata/why-zookeeper-on-odd-number-nodes.php. 
If you're still having issues, you could run a single Zookeeper node on nifi1 
only:

nifi1/conf/nifi.properties:nifi.zookeeper.connect.string=nifi1.com:2180
nifi1/conf/nifi.properties:nifi.state.management.embedded.zookeeper.start=true
nifi1/conf/zookeeper.properties
- clientPort=2180
...
- server.1=nifi1.com:2888:3888

nifi2/conf/nifi.properties:nifi.zookeeper.connect.string=nifi1.com:2180  // 
Connect to the node1 zookeeper
nifi2/conf/nifi.properties:nifi.state.management.embedded.zookeeper.start=false
nifi2/conf/zookeeper.properties not required


Nathan



On 10/2/18, 11:25 AM, "Nathan Gough"  wrote:

Check your configs on nifi2. I don't believe that NiFi is starting two 
instances of Zookeeper but the ports configured are unintentionally configured 
to overlap ie. Ports used twice in different configs where they should be 
different.

It may be that your zookeeper.properties has:

clientPort=2180
...
server.1=nifi1.com:2180:3888
server.2=nifi2.com:2180:3888

where it should be:

clientPort=2180
...
server.1=nifi1.com:2888:3888
server.2=nifi2.com:2888:3888

noticing that the server.1 and server.2 ranges don't overlap with the 
client port.


Not sure if this helps, but the following is the relevant config that I 
have for my NiFi cluster nodes that run on the SAME machine where nifi1.com and 
nifi2.com are configured in /etc/hosts:

nifi1/conf
zookeeper.properties
- clientPort=2180
- server.1=nifi1.com:2888:3888
- server.2=nifi2.com:2888:3888

nifi.properties
- nifi.remote.input.host=nifi1.com
- nifi.remote.input.socket.port=10440
- nifi.web.http.host=nifi1.com
- nifi.web.http.port=9550
- nifi.cluster.node.address=nifi1.com
- nifi.cluster.node.protocol.port=11440

nifi1/state/zookeeper
/myid (file contents = "1")
/state-management.xml (no changes required)
/version-2/


nifi2/conf
zookeeper.properties
- clientPort=2181
- server.1=nifi1.com:2888:3888
- server.2=nifi2.com:2888:3888

nifi.properties
- nifi.remote.input.host=nifi2.com
- nifi.remote.input.socket.port=10441
- nifi.web.http.host=nifi2.com
- nifi.web.http.port=9551
- nifi.cluster.node.address=nifi2.com
- nifi.cluster.node.protocol.port=11441

nifi2/state/zookeeper
/myid (file contents = "2")
/state-management.xml (no changes required)
/version-2/


Nathan



On 10/2/18, 2:07 AM, "Phil H"  wrote:

Hi Andy,

Thanks for the additional info.  I think I saw a link to that while 
searching but was wary since it was such an old version.

I have two VMs (nifi1, and nifi2) both running NiFi with identical 
configs, and trying to use the inbuilt ZK to cluster them.

If I only mention a single machine within the config (eg: if nifi1 
doesn’t refer to nifi2, or visa versa) I don’t get any start up errors.

Phil

From: Andy LoPresto
Sent: Tuesday, 2 October 2018 1:00 PM
To: dev@nifi.apache.org
Subject: Re: Zookeeper - help!

Hi Phil, 

Nathan’s advice is correct but I think he was assuming all other 
configurations are correct as well. Are you trying to run both NiFi nodes and 
ZK instances on the same machine? In that case you will have to ensure that the 
ports in use are different for each service so they don’t conflict. Setting 
them all to the same value only works if each service is running on an 
independent physical machine, virtual machine, or container. 

I find Pierre’s guide [1] to be a helpful step-by-step instruction list 
as well as a good explanation of how the clustering concepts work in practice. 
When you get that working, and you’re ready to set up a secure cluster, he has 
a follow-on guide for that as well [2]. Even as someone who has set up many 
clustered instances of NiFi, I use his guides regularly to ensure I haven’t 
forgotten a step. 

They were originally written for versions 1.0.0 and 1.1.0, but the only 
thing that has changed is the authorizer configuration for the secure instances 
(you’ll need to put the Initial Admin Identity and Node Identities in two 
locations in the authorizers.xml file instead of just once). 

Hopefully this helps you get a working cluster up and run

Re: Zookeeper - help!

[Nathan Gough]

r.

2018-10-02 17:36:31,610 INFO [QuorumPeer[myid=2]/0.0.0.0:10500] 
o.a.zookeeper.server.ZooKeeperServer Created server with tickTime 2000 
minSessionTimeout 4000 maxSessionTimeout 4 datadir 
./state/zookeeper/version-2 snapdir ./state/zookeeper/version-2
2018-10-02 17:36:31,612 ERROR [QuorumPeer[myid=2]/0.0.0.0:10500] 
o.apache.zookeeper.server.quorum.Leader Couldn't bind to 
nifi2.domain/192.168.10.102:10500
java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at 
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at java.net.ServerSocket.bind(ServerSocket.java:329)
at org.apache.zookeeper.server.quorum.Leader.(Leader.java:193)
at 
org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:605)
at 
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:798)




From: Nathan Gough
Sent: Tuesday, 2 October 2018 2:22 AM
To: dev@nifi.apache.org
Subject: Re: Zookeeper - help!

Hi Phil,

One thing I notice with your config is that the cluster.node.protol.port 
and the zookeeper ports are the same - these should not be the same. 
Node.protocol.port is used by NiFi cluster to communicate between nodes, the 
zookeeper.connect.string port should be the port that zookeeper service is 
listening on. The zookeeper port is configured by the clientPort property in 
the zookeeper.properties file. This would make your connect string: 
'nifi.zookeeper.connect.string=nifi1.domain:2180,nifi2.domain:2180', where 2180 
is whatever clientPort is configured.

You can read more about how NiFi uses Zookeeper and how to configure it 
here: 
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#state_management.

Let us know what happens once these properties are configured correctly.

Nathan


On 9/30/18, 11:07 PM, "Phil H"  wrote:

   Hi guys,

   Pulling my hair out trying to solve my Zookeeper problems.  I have two 
1.6.0 servers that I am trying to cluster.

   Here is the except from the properties files – all other properties are 
default so omitted for clarity.   The servers are set up to run HTTPS, and the 
interface works via the browser, so I believe the certificates are correctly 
installed.

   Server nifi1.domain:
   nifi.cluster.is.node=true
   nifi.cluster.node.address=nifi1.domain
   nifi.cluster.node.protocol.port=1

   nifi.zookeeper.connect.string=nifi2.domain:1,nifi1.domain:1
   nifi.zookeeper.root.node=/nifi

   Server nifi2.domain:
   nifi.cluster.is.node=true
   nifi.cluster.node.address=nifi2.domain
   nifi.cluster.node.protocol.port=1

   nifi.zookeeper.connect.string=nifi1.domain:1,nifi2.domain:1
   nifi.zookeeper.root.node=/nifi

   I am getting these errors (this is from server 2, but seeing the same on 
server 1 apart from a different address, of course):

   2018-10-01 20:54:16,332 INFO [main] 
org.apache.nifi.io.socket.SocketListener Now listening for connections from 
nodes on port 1
   2018-10-01 20:54:16,381 INFO [main] 
o.apache.nifi.controller.FlowController Successfully synchronized controller 
with proposed flow
   2018-10-01 20:54:16,435 INFO [main] 
o.a.nifi.controller.StandardFlowService Connecting Node: nifi2.domain:443
   2018-10-01 20:54:16,769 ERROR [Process Cluster Protocol Request-1] 
o.a.nifi.security.util.CertificateUtils The incoming request did not contain 
client certificates and thus the DN cannot be extracted. Check that the other 
endpoint is providing a complete client certificate chain
   2018-10-01 20:54:16,771 WARN [Process Cluster Protocol Request-1] 
o.a.n.c.p.impl.SocketProtocolListener Failed processing protocol message from 
nifi2 due to org.apache.nifi.cluster.protocol.ProtocolException: 
java.security.cert.CertificateException: 
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
   org.apache.nifi.cluster.protocol.ProtocolException: 
java.security.cert.CertificateException: 
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
   at 
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.getRequestorDN(SocketProtocolListener.java:225)
   at 
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:131)
   at 
org.apache.nifi.io.socket.SocketListener$2$1.run(SocketListener.java:136)
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.ja

Re: Zookeeper - help!

[Nathan Gough]

Hi Phil,

One thing I notice with your config is that the cluster.node.protol.port and 
the zookeeper ports are the same - these should not be the same. 
Node.protocol.port is used by NiFi cluster to communicate between nodes, the 
zookeeper.connect.string port should be the port that zookeeper service is 
listening on. The zookeeper port is configured by the clientPort property in 
the zookeeper.properties file. This would make your connect string: 
'nifi.zookeeper.connect.string=nifi1.domain:2180,nifi2.domain:2180', where 2180 
is whatever clientPort is configured.

You can read more about how NiFi uses Zookeeper and how to configure it here: 
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#state_management.

Let us know what happens once these properties are configured correctly.

Nathan


On 9/30/18, 11:07 PM, "Phil H"  wrote:

Hi guys,

Pulling my hair out trying to solve my Zookeeper problems.  I have two 
1.6.0 servers that I am trying to cluster.

Here is the except from the properties files – all other properties are 
default so omitted for clarity.   The servers are set up to run HTTPS, and the 
interface works via the browser, so I believe the certificates are correctly 
installed.

Server nifi1.domain:
nifi.cluster.is.node=true
nifi.cluster.node.address=nifi1.domain
nifi.cluster.node.protocol.port=1

nifi.zookeeper.connect.string=nifi2.domain:1,nifi1.domain:1
nifi.zookeeper.root.node=/nifi

Server nifi2.domain:
nifi.cluster.is.node=true
nifi.cluster.node.address=nifi2.domain
nifi.cluster.node.protocol.port=1

nifi.zookeeper.connect.string=nifi1.domain:1,nifi2.domain:1
nifi.zookeeper.root.node=/nifi

I am getting these errors (this is from server 2, but seeing the same on 
server 1 apart from a different address, of course):

2018-10-01 20:54:16,332 INFO [main] 
org.apache.nifi.io.socket.SocketListener Now listening for connections from 
nodes on port 1
2018-10-01 20:54:16,381 INFO [main] o.apache.nifi.controller.FlowController 
Successfully synchronized controller with proposed flow
2018-10-01 20:54:16,435 INFO [main] o.a.nifi.controller.StandardFlowService 
Connecting Node: nifi2.domain:443
2018-10-01 20:54:16,769 ERROR [Process Cluster Protocol Request-1] 
o.a.nifi.security.util.CertificateUtils The incoming request did not contain 
client certificates and thus the DN cannot be extracted. Check that the other 
endpoint is providing a complete client certificate chain
2018-10-01 20:54:16,771 WARN [Process Cluster Protocol Request-1] 
o.a.n.c.p.impl.SocketProtocolListener Failed processing protocol message from 
nifi2 due to org.apache.nifi.cluster.protocol.ProtocolException: 
java.security.cert.CertificateException: 
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
org.apache.nifi.cluster.protocol.ProtocolException: 
java.security.cert.CertificateException: 
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at 
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.getRequestorDN(SocketProtocolListener.java:225)
at 
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:131)
at 
org.apache.nifi.io.socket.SocketListener$2$1.run(SocketListener.java:136)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.security.cert.CertificateException: 
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at 
org.apache.nifi.security.util.CertificateUtils.extractPeerDNFromClientSSLSocket(CertificateUtils.java:314)
at 
org.apache.nifi.security.util.CertificateUtils.extractPeerDNFromSSLSocket(CertificateUtils.java:269)
at 
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.getRequestorDN(SocketProtocolListener.java:223)
... 5 common frames omitted
Caused by: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at 
sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:440)
at 
org.apache.nifi.security.util.CertificateUtils.extractPeerDNFromClientSSLSocket(CertificateUtils.java:299)
... 7 common frames omitted



2018-10-01 20:54:32,249 INFO [Curator-Framework-0] 
o.a.c.f.state.ConnectionStateManager State change: SUSPENDED
2018-10-01 20:54:32,250 ERROR [Curator-Framework-0] 
o.a.c.f.imps.CuratorFrameworkImpl Background operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException: 
KeeperErrorCode = ConnectionLoss
at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at 
org.apache.curato