RE: [External] Re: Issue with PutSQL

[N, Vyshali]

Hi,

There were some jars of earlier version in the lib folder. After removing it, 
everything worked
Thanks for the help

Regards
Vyshali


-Original Message-
From: Koji Kawamura [mailto:ijokaruma...@gmail.com] 
Sent: Monday, October 22, 2018 1:46 PM
To: users@nifi.apache.org
Subject: [External] Re: Issue with PutSQL

Hi Vyshali,

I was able to connect MySQL Server 8.0.12 with mysql-connector-java-8.0.12.jar 
successfully, using username/password.

Here are the SQLs I used to create a MySQL user:
CREATE USER 'nifi'@'%' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON * . * 
TO 'nifi'@'%';

Does your connection url have any configuration? Which NiFi version are you 
using?

Thanks,
Koji


On Mon, Oct 22, 2018 at 1:11 PM N, Vyshali  wrote:
>
> Hi,
>
>
>
> I'm using MySQL server(8.0.12) for my development and PutSQL processor 
> in Nifi for inserting records in DB. I have downloaded 
> mysql-connector-java-8.0.12.jar and pointed it in the 
> DBCPConnectionPool setting but I'm getting an error,
>
>
>
>  "Cannot create PoolableConnectionFactory( client doesn't support 
> authentication protocol supported by user, consider upgrading MySQL client) "
>
>
>
> I guess there is no need to upgrade the client as I have downloaded the 
> suitable version of jar file. Also, I have given full permission for the 
> folder containing the jar file. Please let me know where I have gone wrong.
>
>
>
> Thanks
>
> Vyshali
>
>

Reverting position only versioned changes

[Peter Wicks (pwicks)]

In NiFi 1.7.1, I tried to revert some changes to a versioned processor group 
where the only changes were Processor position. The processor that was moved 
was running for a very long time, and the revert would not complete until it 
stopped the affected processor, even though a stop is not actually necessary to 
revert the change.

I eventually cancelled the revert request. An unintended side effect is that 
the processor is still in a Stopping state, and I have to make sure and go back 
and start it back up once it completes.

Is this an issue that's already been fixed in 1.8? "Do not stop processors 
during a version revert if only position has changed", or something like that.

--Peter

Re: Processor not found on loading template. But exists in the AddProcessor menu.

[Colin Williams]

After looking at the xml more, it appears some of  tags
which contain the com.comp.cp.nifi.processor.JsonValidateProcessor
have the unversioned bundle and some do not. Then maybe this is the
cause of the issue.

In regards to versioning, the NAR was built for 1.4 and it looks like
the pom is using 1.4 dependencies. I didn't see other deployed nars.
We pulled the NAR off the server, and only saw one custom NAR
deployed. Then I'm not sure regarding the source.

However we removed the processor creating a new template. Then I've
been able to move forward with my own work without this functioning.
Thanks for your help!
On Mon, Oct 22, 2018 at 6:09 AM Bryan Bende  wrote:
>
> What was the parent element of that "unversioned" bundle?
>
> I think that entry may be the issue, but hard to say how it got there
> since you have other references to the bundle with the correct
> information.
>
> Usually that would occur if you had a NAR that was built before NiFi
> introduced component versioning (I think before 1.2.x) and then you
> deployed that old NAR into 1.2.x or later, it would look for the
> version info in the NAR and not find it and then default the group and
> version to "default" and "unversioned".
>
> On Fri, Oct 19, 2018 at 4:48 PM Colin Williams
>  wrote:
> >
> > Hi Bryan,
> >
> > From the template .xml I see the processor listed in a  tag.
> >
> >
> > something like
> >
> > com.comp.cp.nifi.processors.common.JsonValidateProcessor
> >
> > I also see
> >
> >
> >  nifi-cp-bundle-nar
> >  com.comp.cp.nifi
> >  1.0.3
> >
> >
> >
> > Then in the UI I see something like
> >
> >
> > Id
> > 8dfb7fb5-0166-1000-65e7-b0080cc28ac1
> > Type
> > JsonValidateProcessor 1.0.3
> > Bundle
> > com.comp.cp.nifi - nifi-cp-bundle-nar
> >
> > It looks pretty similar to me.
> >
> >
> >
> > In the nifi-app.log I see something like:
> >
> > 2018-10-19 13:12:40,705 INFO [main] org.apache.nifi.NiFi Controller
> > initialization took 19191053655 nanoseconds (19 seconds).
> > 2018-10-19 13:12:46,549 INFO [NiFi Web Server-59]
> > c.s.j.s.i.application.WebApplicationImpl Initiating Jersey
> > application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
> > 2018-10-19 13:14:13,740 WARN [NiFi Web Server-18]
> > o.a.nifi.web.StandardNiFiServiceFacade Unable to create Processor of
> > type com.comp.cp.nifi.processor.JsonValidateProcessor to populate
> > default values.
> > 2018-10-19 13:14:13,741 WARN [NiFi Web Server-18]
> > o.a.nifi.web.StandardNiFiServiceFacade Unable to create Processor of
> > type com.comp.cp.nifi.processor.JsonValidateProcessor to populate
> > default values.
> > 2018-10-19 13:14:13,766 INFO [NiFi Web Server-18]
> > o.a.nifi.groups.StandardProcessGroup
> > Template[id=a9075360-984e-4788-922d-1072acbc77c1] added to
> > StandardProcessGroup[identifier=8df558fc-0166-1000-6368-faec63e764af]
> >
> >
> >
> >
> >
> > Also possibly worth mentioning in the template I do see
> >
> >
> > nifi-cp
> > default
> > unversioned
> >   
> >
> > But I didn't see it referenced from the processor node.
> > On Fri, Oct 19, 2018 at 12:36 PM Colin Williams
> >  wrote:
> > >
> > > Thanks Bryan. I will take a look now. Regarding nifi-app.log, I will
> > > look for it. Wasn't sure if I had to configure something for the
> > > binary version to write the log, or set it's verbosity.
> > > On Fri, Oct 19, 2018 at 12:27 PM Bryan Bende  wrote:
> > > >
> > > > Can you look in the template using a text editor and find the element
> > > > for the processor you are referring to, and look for the "bundle"
> > > > elements and see if the bundle info (group, artifact, version) in the
> > > > template matches the info for the processor in the UI?
> > > >
> > > > Also would be helpful to see the stacktrace of the error from 
> > > > nifi-app.log
> > > > On Fri, Oct 19, 2018 at 1:36 PM Colin Williams
> > > >  wrote:
> > > > >
> > > > > We've exported a template from a instance running 1.4.0 and also
> > > > > copied a custom processor nar bundle from this instance. I'd like to
> > > > > load the template to understand the flow without access to the running
> > > > > instance we sourced it from.
> > > > >
> > > > > I get an error that the custom processor is not found, but I see it
> > > > > from Add Processor menu. Is there a bug in 1.4.0 which might cause
> > > > > this? Is there another issue regarding exporting and importing
> > > > > templates in 1.4.0? Can someone suggest a fix for the issue? Also is
> > > > > there an easy way to enable debugging logs for the binary version?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Colin Williams

Re: NiFi fails on cluster nodes

[Bryan Bende]

Yes, to further clarify what I meant...

If you are trying to change the Initial Admin or Node Identities in
authorizers.xml, these will only be used when there are no other
users/group/policies present. People frequently make a mistake during
initial config and then try to edit authorizers.xml and try again, but
it won't actually do anything unless you remove the users.xml and
authorizations.xml to start over.

In your case it sounds like you are trying to convert and existing
standalone node to a cluster, given that I would do the following...

- In standalone mode, use the UI to add users for the DN's of the
server certificates (CN=nifi-node-1, OU=NIFI, CN=nifi-node-2, OU=NIFI)
- In the UI, grant those users Write access to "Proxy"
- Convert to a cluster and keep your same authorizers.xml, users.xml,
and authorizations.xml when you setup your cluster, this way all your
users and policies are already setup and the Initial Admin and Node
Identities are not needed


On Mon, Oct 22, 2018 at 1:06 PM Saip, Alexander (NIH/CC/BTRIS) [C]
 wrote:
>
> Thanks again, Bryan. Just a quick follow-up question: does removing users.xml 
> and authorizations.xml mean that we will need to re-create all users and 
> groups that we had in the original standalone NiFi instance?
>
> -Original Message-
> From: Bryan Bende 
> Sent: Monday, October 22, 2018 12:48 PM
> To: users@nifi.apache.org
> Subject: Re: NiFi fails on cluster nodes
>
> Sorry I was confused when you said two 1 node clusters and I assumed they 
> each had their own ZooKeeper.
>
> You don't need to run ZK on both nodes, you can create a 2 node cluster using 
> the embedded ZK on the first node.
>
> This blog post shows how to setup a secure 2 node cluster:
>
> https://bryanbende.com/development/2016/08/17/apache-nifi-1-0-0-authorization-and-multi-tenancy
>
> The only difference is that the authorizers.xml has changed slightly, so 
> instead of:
>
> 
> file-provider
> org.apache.nifi.authorization.FileAuthorizer
> ./conf/authorizations.xml
> ./conf/users.xml
> CN=bbende, 
> OU=ApacheNiFi
> 
> CN=localhost, OU=NIFI 
> 
>
> You need to add the the users to the user-group-provider and then to the 
> access-policy-provider...
>
> 
> file-user-group-provider
> org.apache.nifi.authorization.FileUserGroupProvider
> ./conf/users.xml
> 
> CN=bbende, OU=Apache 
> NiFI
> CN=nifi-host-1, 
> OU=NIFI
> CN=nifi-host-2, 
> OU=NIFI
> 
>
> 
> file-access-policy-provider
> org.apache.nifi.authorization.FileAccessPolicyProvider
> composite-configurable-user-group-provider
> ./conf/authorizations.xml
> CN=bbende, OU=Apache 
> NiFI
> 
> CN=nifi-host-1, OU=NIFI
> CN=nifi-host-2, OU=NIFI
> 
>
>
> Also, whenever you change any config in the authorizers.xml related to the 
> file-based providers, then you will need to remove users.xml and 
> authorizations.xml On Mon, Oct 22, 2018 at 12:20 PM Saip, Alexander 
> (NIH/CC/BTRIS) [C]  wrote:
> >
> > Hi Bryan,
> >
> >
> >
> > At this point, we don't want to run ZooKeeper on both nodes (as far as I 
> > understand, it prefers an odd number of members in the ensemble). Actually, 
> > the ZooKeeper running on one of them, sees both NiFi instances, but they 
> > don't talk to each other. When we try to make them do so by using a 
> > different authorizers.xml file, which is very much just a customized 
> > version of the “composite” example from the NiFi Admin Guide, then none of 
> > the nodes is able to start at all, throwing the error I mentioned in my 
> > previous post.
> >
> >
> >
> > Are you saying that we have to run ZooKeeper on both nodes? BTW, do we
> > still need
> >
> >
> >
> > nifi.login.identity.provider.configuration.file=./conf/login-identity-
> > providers.xml
> >
> >
> >
> > in the nifi.properties file when we use that new authorizers.xml? I’m 
> > asking since we have the same LDAP authentication/authorization settings in 
> > the latter.
> >
> >
> >
> > Thank you,
> >
> >
> >
> > Alexander
> >
> >
> >
> > -Original Message-
> > From: Bryan Bende 
> > Sent: Monday, October 22, 2018 11:55 AM
> > To: users@nifi.apache.org
> > Subject: Re: NiFi fails on cluster nodes
> >
> >
> >
> > If you are getting separate clusters then each node is likely only using 
> > it's own ZooKeeper and therefore doesn't know about the other node.
> >
> >
> >
> > In nifi.properties the ZK connect string would need to be something like 
> > nifi-node1-hostname:2181,nifi-node2-hostname:2181 and in zoo.properties you 
> > would need entries for both ZooKeepers:
> >
> >
> >
> > server.1=nifi-node1-hostname:2888:3888
> >
> > server.2=nifi-node2-hostname:2888:3888
> >
> > On Mon, Oct 22, 2018 at 11:28 AM Saip, Alexander (NIH/CC/BTRIS) [C] 
> >  wrote:
> >
> > >
> >
> > > I wonder if anyone has run into the same problem when trying to
> >
> > > configure composite 

RE: NiFi fails on cluster nodes

[Saip, Alexander (NIH/CC/BTRIS) [C]]

Thanks again, Bryan. Just a quick follow-up question: does removing users.xml 
and authorizations.xml mean that we will need to re-create all users and groups 
that we had in the original standalone NiFi instance?

-Original Message-
From: Bryan Bende  
Sent: Monday, October 22, 2018 12:48 PM
To: users@nifi.apache.org
Subject: Re: NiFi fails on cluster nodes

Sorry I was confused when you said two 1 node clusters and I assumed they each 
had their own ZooKeeper.

You don't need to run ZK on both nodes, you can create a 2 node cluster using 
the embedded ZK on the first node.

This blog post shows how to setup a secure 2 node cluster:

https://bryanbende.com/development/2016/08/17/apache-nifi-1-0-0-authorization-and-multi-tenancy

The only difference is that the authorizers.xml has changed slightly, so 
instead of:


file-provider
org.apache.nifi.authorization.FileAuthorizer
./conf/authorizations.xml
./conf/users.xml
CN=bbende, OU=ApacheNiFi

CN=localhost, OU=NIFI 


You need to add the the users to the user-group-provider and then to the 
access-policy-provider...


file-user-group-provider
org.apache.nifi.authorization.FileUserGroupProvider
./conf/users.xml

CN=bbende, OU=Apache 
NiFI
CN=nifi-host-1, 
OU=NIFI
CN=nifi-host-2, 
OU=NIFI



file-access-policy-provider
org.apache.nifi.authorization.FileAccessPolicyProvider
composite-configurable-user-group-provider
./conf/authorizations.xml
CN=bbende, OU=Apache 
NiFI

CN=nifi-host-1, OU=NIFI
CN=nifi-host-2, OU=NIFI



Also, whenever you change any config in the authorizers.xml related to the 
file-based providers, then you will need to remove users.xml and 
authorizations.xml On Mon, Oct 22, 2018 at 12:20 PM Saip, Alexander 
(NIH/CC/BTRIS) [C]  wrote:
>
> Hi Bryan,
>
>
>
> At this point, we don't want to run ZooKeeper on both nodes (as far as I 
> understand, it prefers an odd number of members in the ensemble). Actually, 
> the ZooKeeper running on one of them, sees both NiFi instances, but they 
> don't talk to each other. When we try to make them do so by using a different 
> authorizers.xml file, which is very much just a customized version of the 
> “composite” example from the NiFi Admin Guide, then none of the nodes is able 
> to start at all, throwing the error I mentioned in my previous post.
>
>
>
> Are you saying that we have to run ZooKeeper on both nodes? BTW, do we 
> still need
>
>
>
> nifi.login.identity.provider.configuration.file=./conf/login-identity-
> providers.xml
>
>
>
> in the nifi.properties file when we use that new authorizers.xml? I’m asking 
> since we have the same LDAP authentication/authorization settings in the 
> latter.
>
>
>
> Thank you,
>
>
>
> Alexander
>
>
>
> -Original Message-
> From: Bryan Bende 
> Sent: Monday, October 22, 2018 11:55 AM
> To: users@nifi.apache.org
> Subject: Re: NiFi fails on cluster nodes
>
>
>
> If you are getting separate clusters then each node is likely only using it's 
> own ZooKeeper and therefore doesn't know about the other node.
>
>
>
> In nifi.properties the ZK connect string would need to be something like 
> nifi-node1-hostname:2181,nifi-node2-hostname:2181 and in zoo.properties you 
> would need entries for both ZooKeepers:
>
>
>
> server.1=nifi-node1-hostname:2888:3888
>
> server.2=nifi-node2-hostname:2888:3888
>
> On Mon, Oct 22, 2018 at 11:28 AM Saip, Alexander (NIH/CC/BTRIS) [C] 
>  wrote:
>
> >
>
> > I wonder if anyone has run into the same problem when trying to
>
> > configure composite authentication/authorization (LDAP  and local
>
> > file)? When we use the “stand-alone” authorizers.xml file with the
>
> > addition of two extra properties
>
> >
>
> >
>
> >
>
> > …
>
> >
>
> > …
>
> >
>
> >
>
> >
>
> > and let ZooKeeper start on one on the nodes, we end up with two
>
> > one-node clusters, since apparently, the NiFi instances don’t talk 
> > to
>
> > each other, but at least, they come alive…
>
> >
>
> >
>
> >
>
> > From: Saip, Alexander (NIH/CC/BTRIS) [C] 
>
> > Sent: Friday, October 19, 2018 11:18 AM
>
> > To: users@nifi.apache.org
>
> > Subject: RE: NiFi fails on cluster nodes
>
> >
>
> >
>
> >
>
> > We have managed to get past that error by installing the CA cert in the 
> > truststore. So, we can get a one-node cluster up and running. In order to 
> > add another node, I edited the authorizers.xml file, basically, using the 
> > “example composite implementation loading users and groups from LDAP and a 
> > local file” from the Admin guide as a template. When I re-started the node 
> > running ZooKeeper, though, it crashed with the following error written into 
> > the nifi-app.log file:
>
> >
>
> >
>
> >
>
> > 2018-10-19 08:09:26,992 ERROR [main] o.s.web.context.ContextLoader
>
> > Context initialization failed
>
> >
>
> > org.springframework.beans.factory.UnsatisfiedDependencyException:
>
> > Error 

Re: NiFi fails on cluster nodes

[Bryan Bende]

Sorry I was confused when you said two 1 node clusters and I assumed
they each had their own ZooKeeper.

You don't need to run ZK on both nodes, you can create a 2 node
cluster using the embedded ZK on the first node.

This blog post shows how to setup a secure 2 node cluster:

https://bryanbende.com/development/2016/08/17/apache-nifi-1-0-0-authorization-and-multi-tenancy

The only difference is that the authorizers.xml has changed slightly,
so instead of:


file-provider
org.apache.nifi.authorization.FileAuthorizer
./conf/authorizations.xml
./conf/users.xml
CN=bbende, OU=ApacheNiFi

CN=localhost, OU=NIFI


You need to add the the users to the user-group-provider and then to
the access-policy-provider...


file-user-group-provider
org.apache.nifi.authorization.FileUserGroupProvider
./conf/users.xml

CN=bbende, OU=Apache
NiFI
CN=nifi-host-1,
OU=NIFI
CN=nifi-host-2,
OU=NIFI



file-access-policy-provider
org.apache.nifi.authorization.FileAccessPolicyProvider
composite-configurable-user-group-provider
./conf/authorizations.xml
CN=bbende, OU=Apache
NiFI

CN=nifi-host-1, OU=NIFI
CN=nifi-host-2, OU=NIFI



Also, whenever you change any config in the authorizers.xml related to
the file-based providers, then you will need to remove users.xml and
authorizations.xml
On Mon, Oct 22, 2018 at 12:20 PM Saip, Alexander (NIH/CC/BTRIS) [C]
 wrote:
>
> Hi Bryan,
>
>
>
> At this point, we don't want to run ZooKeeper on both nodes (as far as I 
> understand, it prefers an odd number of members in the ensemble). Actually, 
> the ZooKeeper running on one of them, sees both NiFi instances, but they 
> don't talk to each other. When we try to make them do so by using a different 
> authorizers.xml file, which is very much just a customized version of the 
> “composite” example from the NiFi Admin Guide, then none of the nodes is able 
> to start at all, throwing the error I mentioned in my previous post.
>
>
>
> Are you saying that we have to run ZooKeeper on both nodes? BTW, do we still 
> need
>
>
>
> nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
>
>
>
> in the nifi.properties file when we use that new authorizers.xml? I’m asking 
> since we have the same LDAP authentication/authorization settings in the 
> latter.
>
>
>
> Thank you,
>
>
>
> Alexander
>
>
>
> -Original Message-
> From: Bryan Bende 
> Sent: Monday, October 22, 2018 11:55 AM
> To: users@nifi.apache.org
> Subject: Re: NiFi fails on cluster nodes
>
>
>
> If you are getting separate clusters then each node is likely only using it's 
> own ZooKeeper and therefore doesn't know about the other node.
>
>
>
> In nifi.properties the ZK connect string would need to be something like 
> nifi-node1-hostname:2181,nifi-node2-hostname:2181 and in zoo.properties you 
> would need entries for both ZooKeepers:
>
>
>
> server.1=nifi-node1-hostname:2888:3888
>
> server.2=nifi-node2-hostname:2888:3888
>
> On Mon, Oct 22, 2018 at 11:28 AM Saip, Alexander (NIH/CC/BTRIS) [C] 
>  wrote:
>
> >
>
> > I wonder if anyone has run into the same problem when trying to
>
> > configure composite authentication/authorization (LDAP  and local
>
> > file)? When we use the “stand-alone” authorizers.xml file with the
>
> > addition of two extra properties
>
> >
>
> >
>
> >
>
> > …
>
> >
>
> > …
>
> >
>
> >
>
> >
>
> > and let ZooKeeper start on one on the nodes, we end up with two
>
> > one-node clusters, since apparently, the NiFi instances don’t talk to
>
> > each other, but at least, they come alive…
>
> >
>
> >
>
> >
>
> > From: Saip, Alexander (NIH/CC/BTRIS) [C] 
>
> > Sent: Friday, October 19, 2018 11:18 AM
>
> > To: users@nifi.apache.org
>
> > Subject: RE: NiFi fails on cluster nodes
>
> >
>
> >
>
> >
>
> > We have managed to get past that error by installing the CA cert in the 
> > truststore. So, we can get a one-node cluster up and running. In order to 
> > add another node, I edited the authorizers.xml file, basically, using the 
> > “example composite implementation loading users and groups from LDAP and a 
> > local file” from the Admin guide as a template. When I re-started the node 
> > running ZooKeeper, though, it crashed with the following error written into 
> > the nifi-app.log file:
>
> >
>
> >
>
> >
>
> > 2018-10-19 08:09:26,992 ERROR [main] o.s.web.context.ContextLoader
>
> > Context initialization failed
>
> >
>
> > org.springframework.beans.factory.UnsatisfiedDependencyException:
>
> > Error creating bean with name
>
> > 'org.springframework.security.config.annotation.web.configuration.WebS
>
> > ecurityConfiguration': Unsatisfied dependency expressed through method
>
> > 'setFilterChainProxySecurityConfigurer' parameter 1; nested exception
>
> > is org.springframework.beans.factory.BeanExpressionException:
>
> > Expression parsing failed; nested exception is
>
> > 

Re: NiFi fails on cluster nodes

[Bryan Bende]

If you are getting separate clusters then each node is likely only
using it's own ZooKeeper and therefore doesn't know about the other
node.

In nifi.properties the ZK connect string would need to be something
like nifi-node1-hostname:2181,nifi-node2-hostname:2181 and in
zoo.properties you would need entries for both ZooKeepers:

server.1=nifi-node1-hostname:2888:3888
server.2=nifi-node2-hostname:2888:3888
On Mon, Oct 22, 2018 at 11:28 AM Saip, Alexander (NIH/CC/BTRIS) [C]
 wrote:
>
> I wonder if anyone has run into the same problem when trying to configure 
> composite authentication/authorization (LDAP  and local file)? When we use 
> the “stand-alone” authorizers.xml file with the addition of two extra 
> properties
>
>
>
> …
>
> …
>
>
>
> and let ZooKeeper start on one on the nodes, we end up with two one-node 
> clusters, since apparently, the NiFi instances don’t talk to each other, but 
> at least, they come alive…
>
>
>
> From: Saip, Alexander (NIH/CC/BTRIS) [C] 
> Sent: Friday, October 19, 2018 11:18 AM
> To: users@nifi.apache.org
> Subject: RE: NiFi fails on cluster nodes
>
>
>
> We have managed to get past that error by installing the CA cert in the 
> truststore. So, we can get a one-node cluster up and running. In order to add 
> another node, I edited the authorizers.xml file, basically, using the 
> “example composite implementation loading users and groups from LDAP and a 
> local file” from the Admin guide as a template. When I re-started the node 
> running ZooKeeper, though, it crashed with the following error written into 
> the nifi-app.log file:
>
>
>
> 2018-10-19 08:09:26,992 ERROR [main] o.s.web.context.ContextLoader Context 
> initialization failed
>
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
> creating bean with name 
> 'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':
>  Unsatisfied dependency expressed through method 
> 'setFilterChainProxySecurityConfigurer' parameter 1; nested exception is 
> org.springframework.beans.factory.BeanExpressionException: Expression parsing 
> failed; nested exception is 
> org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
> creating bean with name 
> 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': Unsatisfied dependency 
> expressed through method 'setJwtAuthenticationProvider' parameter 0; nested 
> exception is org.springframework.beans.factory.BeanCreationException: Error 
> creating bean with name 'jwtAuthenticationProvider' defined in class path 
> resource [nifi-web-security-context.xml]: Cannot resolve reference to bean 
> 'authorizer' while setting constructor argument; nested exception is 
> org.springframework.beans.factory.BeanCreationException: Error creating bean 
> with name 'authorizer': FactoryBean threw exception on object creation; 
> nested exception is java.lang.NullPointerException: Name is null
>
> at 
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:667)
>
> at 
> org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
>
> at 
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
>
> at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
>
> at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
>
> at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
>
> at 
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
>
> at 
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
>
> at 
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
>
> at 
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
>
> at 
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
>
> at 
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
>
> at 
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
>
> at 
> org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
>
> at 
> 

RE: NiFi fails on cluster nodes

[Saip, Alexander (NIH/CC/BTRIS) [C]]

I wonder if anyone has run into the same problem when trying to configure 
composite authentication/authorization (LDAP  and local file)? When we use the 
“stand-alone” authorizers.xml file with the addition of two extra properties

…
…

and let ZooKeeper start on one on the nodes, we end up with two one-node 
clusters, since apparently, the NiFi instances don’t talk to each other, but at 
least, they come alive…

From: Saip, Alexander (NIH/CC/BTRIS) [C] 
Sent: Friday, October 19, 2018 11:18 AM
To: users@nifi.apache.org
Subject: RE: NiFi fails on cluster nodes


We have managed to get past that error by installing the CA cert in the 
truststore. So, we can get a one-node cluster up and running. In order to add 
another node, I edited the authorizers.xml file, basically, using the “example 
composite implementation loading users and groups from LDAP and a local file” 
from the Admin 
guide
 as a template. When I re-started the node running ZooKeeper, though, it 
crashed with the following error written into the nifi-app.log file:



2018-10-19 08:09:26,992 ERROR [main] o.s.web.context.ContextLoader Context 
initialization failed

org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 
'org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration':
 Unsatisfied dependency expressed through method 
'setFilterChainProxySecurityConfigurer' parameter 1; nested exception is 
org.springframework.beans.factory.BeanExpressionException: Expression parsing 
failed; nested exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'org.apache.nifi.web.NiFiWebApiSecurityConfiguration': 
Unsatisfied dependency expressed through method 'setJwtAuthenticationProvider' 
parameter 0; nested exception is 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'jwtAuthenticationProvider' defined in class path resource 
[nifi-web-security-context.xml]: Cannot resolve reference to bean 'authorizer' 
while setting constructor argument; nested exception is 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'authorizer': FactoryBean threw exception on object creation; nested 
exception is java.lang.NullPointerException: Name is null

at 
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:667)

at 
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)

at 
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)

at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)

at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)

at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)

at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)

at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)

at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)

at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)

at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)

at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)

at 
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)

at 
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)

at 
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)

at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)

at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)

at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)

at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)

 

Re: Processor not found on loading template. But exists in the AddProcessor menu.

[Bryan Bende]

What was the parent element of that "unversioned" bundle?

I think that entry may be the issue, but hard to say how it got there
since you have other references to the bundle with the correct
information.

Usually that would occur if you had a NAR that was built before NiFi
introduced component versioning (I think before 1.2.x) and then you
deployed that old NAR into 1.2.x or later, it would look for the
version info in the NAR and not find it and then default the group and
version to "default" and "unversioned".

On Fri, Oct 19, 2018 at 4:48 PM Colin Williams
 wrote:
>
> Hi Bryan,
>
> From the template .xml I see the processor listed in a  tag.
>
>
> something like
>
> com.comp.cp.nifi.processors.common.JsonValidateProcessor
>
> I also see
>
>
>  nifi-cp-bundle-nar
>  com.comp.cp.nifi
>  1.0.3
>
>
>
> Then in the UI I see something like
>
>
> Id
> 8dfb7fb5-0166-1000-65e7-b0080cc28ac1
> Type
> JsonValidateProcessor 1.0.3
> Bundle
> com.comp.cp.nifi - nifi-cp-bundle-nar
>
> It looks pretty similar to me.
>
>
>
> In the nifi-app.log I see something like:
>
> 2018-10-19 13:12:40,705 INFO [main] org.apache.nifi.NiFi Controller
> initialization took 19191053655 nanoseconds (19 seconds).
> 2018-10-19 13:12:46,549 INFO [NiFi Web Server-59]
> c.s.j.s.i.application.WebApplicationImpl Initiating Jersey
> application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
> 2018-10-19 13:14:13,740 WARN [NiFi Web Server-18]
> o.a.nifi.web.StandardNiFiServiceFacade Unable to create Processor of
> type com.comp.cp.nifi.processor.JsonValidateProcessor to populate
> default values.
> 2018-10-19 13:14:13,741 WARN [NiFi Web Server-18]
> o.a.nifi.web.StandardNiFiServiceFacade Unable to create Processor of
> type com.comp.cp.nifi.processor.JsonValidateProcessor to populate
> default values.
> 2018-10-19 13:14:13,766 INFO [NiFi Web Server-18]
> o.a.nifi.groups.StandardProcessGroup
> Template[id=a9075360-984e-4788-922d-1072acbc77c1] added to
> StandardProcessGroup[identifier=8df558fc-0166-1000-6368-faec63e764af]
>
>
>
>
>
> Also possibly worth mentioning in the template I do see
>
>
> nifi-cp
> default
> unversioned
>   
>
> But I didn't see it referenced from the processor node.
> On Fri, Oct 19, 2018 at 12:36 PM Colin Williams
>  wrote:
> >
> > Thanks Bryan. I will take a look now. Regarding nifi-app.log, I will
> > look for it. Wasn't sure if I had to configure something for the
> > binary version to write the log, or set it's verbosity.
> > On Fri, Oct 19, 2018 at 12:27 PM Bryan Bende  wrote:
> > >
> > > Can you look in the template using a text editor and find the element
> > > for the processor you are referring to, and look for the "bundle"
> > > elements and see if the bundle info (group, artifact, version) in the
> > > template matches the info for the processor in the UI?
> > >
> > > Also would be helpful to see the stacktrace of the error from nifi-app.log
> > > On Fri, Oct 19, 2018 at 1:36 PM Colin Williams
> > >  wrote:
> > > >
> > > > We've exported a template from a instance running 1.4.0 and also
> > > > copied a custom processor nar bundle from this instance. I'd like to
> > > > load the template to understand the flow without access to the running
> > > > instance we sourced it from.
> > > >
> > > > I get an error that the custom processor is not found, but I see it
> > > > from Add Processor menu. Is there a bug in 1.4.0 which might cause
> > > > this? Is there another issue regarding exporting and importing
> > > > templates in 1.4.0? Can someone suggest a fix for the issue? Also is
> > > > there an easy way to enable debugging logs for the binary version?
> > > >
> > > > Thanks,
> > > >
> > > > Colin Williams

Re: Cluster setup in multiple data centers

[Mark Payne]

Hi Vijay,

You may run into some problems if you try to build a single NiFi cluster that 
spans multiple
data centers. NiFi depends on ZooKeeper to perform leader election to determine 
which node
is the Cluster Coordinator and which is the Primary Node. ZooKeeper guidance 
indicates that you
should not use it for cross-data center communication. Additionally, if 
ZooKeeper does still work
well for you cross-data center you'll likely see UI behaving sluggishly, as all 
of the requests would
have to be replicated across the data centers instead of just across a LAN. 
That said, I've never actually
attempted to do this, so you're welcome to try :)

However, I would lean toward having a second cluster. The NiFi Flow Registry 
[1][2] should help to make
this fairly painless.

Thanks
-Mark

[1] https://nifi.apache.org/registry.html
[2] 
https://bryanbende.com/development/2018/01/19/apache-nifi-how-do-i-deploy-my-flow


On Oct 22, 2018, at 12:36 AM, Vijay Chhipa 
mailto:vchh...@apple.com>> wrote:

Hello

We are putting NiFi in production. To ensure that there is no single point of 
failure we have two options.

a) Setup a single NiFi cluster with nodes distributed across data centers.
b) Setup a cluster in one data center and for failover have a hot stand-by 
cluster in another data center

What are the pros and cons of each approach?

Are there other options?

Any feedback based on your experience will be highly useful.

Thank you,

Vijay

Re: Issue with PutSQL

[Koji Kawamura]

Hi Vyshali,

I was able to connect MySQL Server 8.0.12 with
mysql-connector-java-8.0.12.jar successfully, using username/password.

Here are the SQLs I used to create a MySQL user:
CREATE USER 'nifi'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON * . * TO 'nifi'@'%';

Does your connection url have any configuration? Which NiFi version
are you using?

Thanks,
Koji


On Mon, Oct 22, 2018 at 1:11 PM N, Vyshali  wrote:
>
> Hi,
>
>
>
> I'm using MySQL server(8.0.12) for my development and PutSQL processor in 
> Nifi for inserting records in DB. I have downloaded 
> mysql-connector-java-8.0.12.jar and pointed it in the DBCPConnectionPool 
> setting but I'm getting an error,
>
>
>
>  "Cannot create PoolableConnectionFactory( client doesn't support 
> authentication protocol supported by user, consider upgrading MySQL client) "
>
>
>
> I guess there is no need to upgrade the client as I have downloaded the 
> suitable version of jar file. Also, I have given full permission for the 
> folder containing the jar file. Please let me know where I have gone wrong.
>
>
>
> Thanks
>
> Vyshali
>
>