Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

[Sumanth Chinthagunta]

Hi 
I am following clustering instructions as per the link below:  NCM, Node1 in 
Server1 and Node2 on Server2.
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering
 


I created two copies of Nifi 0.4.0 folders on Server1 and started NCM, then 
tried to start Node1 and got the following error. 
Error: 
org.apache.nifi.bootstrap.Command Apache NiFi is already running, listening to 
Bootstrap on port 43010
 
Am I missing any steps?
Thanks 
Sumo

Re: Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

[Corey Flowers]

If you have a node and cluster on the same box you must be running on
a different ports. Also, if you are not using the default locations of
the repositories, you should set up the repos under different storage
locations. For instance, on ours, the ncms, keep the default locations
and the nodes are changed to larger storage file systems. Part of this
is that the ncm, doesn't actually move data, so the defaults should
suffice. Let me know if you need more info.

Oh and make sure the site to site ports are different too.

Later!

Sent from my iPhone

> On Dec 18, 2015, at 1:00 PM, Sumanth Chinthagunta  wrote:
>
>
> Hi
> I am following clustering instructions as per the link below:  NCM, Node1 in 
> Server1 and Node2 on Server2.
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering
>  
> 
>
> I created two copies of Nifi 0.4.0 folders on Server1 and started NCM, then 
> tried to start Node1 and got the following error.
> Error:
> org.apache.nifi.bootstrap.Command Apache NiFi is already running, listening 
> to Bootstrap on port 43010
>
> Am I missing any steps?
> Thanks
> Sumo
>

Re: Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

[Matt Gilman]

Sumo,

It appears that while getting all the nodes stood up, there is no node that
is assigned the primary role. NiFi will be in safe mode until a primary is
established. If you go to the cluster table (accessed from the cluster icon
in the upper right hand corner) you should be able to assign the primary
role to one of the nodes in your cluster. This is done by clicking the
ribbon icon in the row for the desired node.

The primary node is responsible for running processors that are configured
with the scheduling strategy of 'Primary Node Only'. When scheduled to run,
these will only execute on that Node. This is useful if your using certain
protocols that would be troublesome if multiple nodes executed concurrently
(like [S]FTP for instance).

Matt

On Fri, Dec 18, 2015 at 7:26 PM, Sumanth Chinthagunta 
wrote:

> Issue 1:
> in my NiFi installation I granted only read access to NIFI_HOME/lib and I
> got this error while trying to start NiFi.
>
> This got resolved when I gave read+write access to NIFI_HOME/lib. this
> makes me wonder why NiFi start process needs write access to lib folder!
>
> INFO [main] org.apache.nifi.BootstrapListener Successfully initiated
> communication with Bootstrap
> WARN [main] org.apache.nifi.nar.NarUnpacker Unable to load NAR library
> bundles due to java.io.IOException: /software/nifi-node1/./lib directory
> does not have read/write privilege Will proceed without loading any further
> Nar bundles
> ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to
> java.lang.IllegalStateException: Unable to find the framework NAR
> ClassLoader.
> java.lang.IllegalStateException: Unable to find the framework NAR
> ClassLoader.
> at org.apache.nifi.NiFi.(NiFi.java:116)
> ~[nifi-runtime-0.4.0.jar:0.4.0]
> at org.apache.nifi.NiFi.main(NiFi.java:227)
> ~[nifi-runtime-0.4.0.jar:0.4.0]
> INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web
> server...
> INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed
> (nicely or otherwise).
>
>
> Issue 2:
> Server 1 : NCM , node1
> Server 2 : node2
>
> Case 1: I started NiFi NCM on server 1 and  then node2 on server 2. when
> you access admin UI, you will see error even  node2 and NCM are
> communicating.
> Case 2:  I started NiFi NCM and then node1 on server 1. Admin UI works as
> expected.
> My workaround was, to set nifi.web.http.host to hostname instead of
> leaving to default ‘localhost’  on Server2. this should be documented.
>
> Issue 3:
> After above issues are resolved, now I see both node1 and node2 connected
> to NCM. when I try to add any processors or process group, I am getting
> following error in the browser UI:
>
> Cluster is unable to service request to change flow: Received a mutable
> request [PUT --
> http://xyz:8090/nifi-api/controller/process-groups/77afb88b-5f7d-45d7-a1f4-f9e9269b489b/processors/bceff62d-435d-4c47-a907-bb2a26cd0e56]
> <
> http://xyz:8090/nifi-api/controller/process-groups/77afb88b-5f7d-45d7-a1f4-f9e9269b489b/processors/bceff62d-435d-4c47-a907-bb2a26cd0e56]>
> while in safe mode
>
> please let me know if you find anything wrong with my setup.
>
> Thanks
> Sumo
>
> > On Dec 18, 2015, at 11:37 AM, Sumanth Chinthagunta 
> wrote:
> >
> > Thanks Corey and Mark for quick response.
> >
> > You are right, I was sharing some folders using symbolic links between
> NCM and Node1. After removing sharing for bin folder, it works fine :)
> >
> > NCM/Node1
> > bin -> /software/nifi/latest/bin/
> > conf
> > content_repository
> > database_repository
> > docs -> /software/nifi/latest/docs/
> > flowfile_repository
> > lib -> /software/nifi/latest/lib/
> > logs
> > provenance_repository
> > work
> >
> > Thanks
> > Sumo
> >
> >> On Dec 18, 2015, at 10:05 AM, Mark Payne  wrote:
> >>
> >> Sumo,
> >>
> >> When you say "I created two copies of Nifi 0.4.0 folders on Server1"
> does that mean that you made a copy of the first directory, or
> >> that you untarred the .tar.gz again?
> >>
> >> It looks like the same nifi.pid file is in the bin/ directory of both
> instances. You should be able to delete the nifi.pid file from the node
> >> and then start it up.
> >>
> >> Thanks
> >> -Mark
> >>
> >>
> >>> On Dec 18, 2015, at 12:59 PM, Sumanth Chinthagunta 
> wrote:
> >>>
> >>>
> >>> Hi
> >>> I am following clustering instructions as per the link below:  NCM,
> Node1 in Server1 and Node2 on Server2.
> >>>
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering
> <
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering
> >
> >>>
> >>> I created two copies of Nifi 0.4.0 folders on Server1 and started NCM,
> then tried to start Node1 and got the following error.
> >>> Error:
> >>> org.apache.nifi.bootstrap.Command Apache NiFi is already running,
> listening to Bootstrap on port 43010
> >>>
> >>> Am I missing any steps?
> >>> Thanks
> >>> Sumo
> >>>
> >>
> 

Re: Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

[Sumanth Chinthagunta]

got it. now my cluster is all set :) 
 Thanks 

> On Dec 18, 2015, at 5:04 PM, Matt Gilman  wrote:
> 
> Sumo,
> 
> It appears that while getting all the nodes stood up, there is no node that
> is assigned the primary role. NiFi will be in safe mode until a primary is
> established. If you go to the cluster table (accessed from the cluster icon
> in the upper right hand corner) you should be able to assign the primary
> role to one of the nodes in your cluster. This is done by clicking the
> ribbon icon in the row for the desired node.
> 
> The primary node is responsible for running processors that are configured
> with the scheduling strategy of 'Primary Node Only'. When scheduled to run,
> these will only execute on that Node. This is useful if your using certain
> protocols that would be troublesome if multiple nodes executed concurrently
> (like [S]FTP for instance).
> 
> Matt
> 
> On Fri, Dec 18, 2015 at 7:26 PM, Sumanth Chinthagunta  >
> wrote:
> 
>> Issue 1:
>> in my NiFi installation I granted only read access to NIFI_HOME/lib and I
>> got this error while trying to start NiFi.
>> 
>> This got resolved when I gave read+write access to NIFI_HOME/lib. this
>> makes me wonder why NiFi start process needs write access to lib folder!
>> 
>> INFO [main] org.apache.nifi.BootstrapListener Successfully initiated
>> communication with Bootstrap
>> WARN [main] org.apache.nifi.nar.NarUnpacker Unable to load NAR library
>> bundles due to java.io.IOException: /software/nifi-node1/./lib directory
>> does not have read/write privilege Will proceed without loading any further
>> Nar bundles
>> ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to
>> java.lang.IllegalStateException: Unable to find the framework NAR
>> ClassLoader.
>> java.lang.IllegalStateException: Unable to find the framework NAR
>> ClassLoader.
>>at org.apache.nifi.NiFi.(NiFi.java:116)
>> ~[nifi-runtime-0.4.0.jar:0.4.0]
>>at org.apache.nifi.NiFi.main(NiFi.java:227)
>> ~[nifi-runtime-0.4.0.jar:0.4.0]
>> INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web
>> server...
>> INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed
>> (nicely or otherwise).
>> 
>> 
>> Issue 2:
>> Server 1 : NCM , node1
>> Server 2 : node2
>> 
>> Case 1: I started NiFi NCM on server 1 and  then node2 on server 2. when
>> you access admin UI, you will see error even  node2 and NCM are
>> communicating.
>> Case 2:  I started NiFi NCM and then node1 on server 1. Admin UI works as
>> expected.
>> My workaround was, to set nifi.web.http.host to hostname instead of
>> leaving to default ‘localhost’  on Server2. this should be documented.
>> 
>> Issue 3:
>> After above issues are resolved, now I see both node1 and node2 connected
>> to NCM. when I try to add any processors or process group, I am getting
>> following error in the browser UI:
>> 
>> Cluster is unable to service request to change flow: Received a mutable
>> request [PUT --
>> http://xyz:8090/nifi-api/controller/process-groups/77afb88b-5f7d-45d7-a1f4-f9e9269b489b/processors/bceff62d-435d-4c47-a907-bb2a26cd0e56]
>> <
>> http://xyz:8090/nifi-api/controller/process-groups/77afb88b-5f7d-45d7-a1f4-f9e9269b489b/processors/bceff62d-435d-4c47-a907-bb2a26cd0e56]
>>  
>> >
>> while in safe mode
>> 
>> please let me know if you find anything wrong with my setup.
>> 
>> Thanks
>> Sumo
>> 
>>> On Dec 18, 2015, at 11:37 AM, Sumanth Chinthagunta 
>> wrote:
>>> 
>>> Thanks Corey and Mark for quick response.
>>> 
>>> You are right, I was sharing some folders using symbolic links between
>> NCM and Node1. After removing sharing for bin folder, it works fine :)
>>> 
>>> NCM/Node1
>>> bin -> /software/nifi/latest/bin/
>>> conf
>>> content_repository
>>> database_repository
>>> docs -> /software/nifi/latest/docs/
>>> flowfile_repository
>>> lib -> /software/nifi/latest/lib/
>>> logs
>>> provenance_repository
>>> work
>>> 
>>> Thanks
>>> Sumo
>>> 
 On Dec 18, 2015, at 10:05 AM, Mark Payne  wrote:
 
 Sumo,
 
 When you say "I created two copies of Nifi 0.4.0 folders on Server1"
>> does that mean that you made a copy of the first directory, or
 that you untarred the .tar.gz again?
 
 It looks like the same nifi.pid file is in the bin/ directory of both
>> instances. You should be able to delete the nifi.pid file from the node
 and then start it up.
 
 Thanks
 -Mark
 
 
> On Dec 18, 2015, at 12:59 PM, Sumanth Chinthagunta 
>> wrote:
> 
> 
> Hi
> I am following clustering instructions as per the link below:  NCM,
>> Node1 in Server1 and Node2 on Server2.
> 
>> 

Re: Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

[Sumanth Chinthagunta]

Issue 1:
in my NiFi installation I granted only read access to NIFI_HOME/lib and I got 
this error while trying to start NiFi.

This got resolved when I gave read+write access to NIFI_HOME/lib. this makes me 
wonder why NiFi start process needs write access to lib folder!

INFO [main] org.apache.nifi.BootstrapListener Successfully initiated 
communication with Bootstrap
WARN [main] org.apache.nifi.nar.NarUnpacker Unable to load NAR library bundles 
due to java.io.IOException: /software/nifi-node1/./lib directory does not have 
read/write privilege Will proceed without loading any further Nar bundles
ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to 
java.lang.IllegalStateException: Unable to find the framework NAR ClassLoader.
java.lang.IllegalStateException: Unable to find the framework NAR ClassLoader.
at org.apache.nifi.NiFi.(NiFi.java:116) 
~[nifi-runtime-0.4.0.jar:0.4.0]
at org.apache.nifi.NiFi.main(NiFi.java:227) 
~[nifi-runtime-0.4.0.jar:0.4.0]
INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed 
(nicely or otherwise).


Issue 2:
Server 1 : NCM , node1
Server 2 : node2

Case 1: I started NiFi NCM on server 1 and  then node2 on server 2. when you 
access admin UI, you will see error even  node2 and NCM are communicating. 
Case 2:  I started NiFi NCM and then node1 on server 1. Admin UI works as 
expected. 
My workaround was, to set nifi.web.http.host to hostname instead of leaving to 
default ‘localhost’  on Server2. this should be documented. 

Issue 3:
After above issues are resolved, now I see both node1 and node2 connected to 
NCM. when I try to add any processors or process group, I am getting following 
error in the browser UI: 

Cluster is unable to service request to change flow: Received a mutable request 
[PUT -- 
http://xyz:8090/nifi-api/controller/process-groups/77afb88b-5f7d-45d7-a1f4-f9e9269b489b/processors/bceff62d-435d-4c47-a907-bb2a26cd0e56]
 

 while in safe mode

please let me know if you find anything wrong with my setup.

Thanks 
Sumo

> On Dec 18, 2015, at 11:37 AM, Sumanth Chinthagunta  wrote:
> 
> Thanks Corey and Mark for quick response.
> 
> You are right, I was sharing some folders using symbolic links between  NCM 
> and Node1. After removing sharing for bin folder, it works fine :)
> 
> NCM/Node1
> bin -> /software/nifi/latest/bin/
> conf
> content_repository
> database_repository
> docs -> /software/nifi/latest/docs/
> flowfile_repository
> lib -> /software/nifi/latest/lib/
> logs
> provenance_repository
> work
> 
> Thanks
> Sumo
> 
>> On Dec 18, 2015, at 10:05 AM, Mark Payne  wrote:
>> 
>> Sumo,
>> 
>> When you say "I created two copies of Nifi 0.4.0 folders on Server1" does 
>> that mean that you made a copy of the first directory, or
>> that you untarred the .tar.gz again?
>> 
>> It looks like the same nifi.pid file is in the bin/ directory of both 
>> instances. You should be able to delete the nifi.pid file from the node
>> and then start it up.
>> 
>> Thanks
>> -Mark
>> 
>> 
>>> On Dec 18, 2015, at 12:59 PM, Sumanth Chinthagunta  
>>> wrote:
>>> 
>>> 
>>> Hi 
>>> I am following clustering instructions as per the link below:  NCM, Node1 
>>> in Server1 and Node2 on Server2.
>>> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering
>>>  
>>> 
>>> 
>>> I created two copies of Nifi 0.4.0 folders on Server1 and started NCM, then 
>>> tried to start Node1 and got the following error. 
>>> Error: 
>>> org.apache.nifi.bootstrap.Command Apache NiFi is already running, listening 
>>> to Bootstrap on port 43010
>>> 
>>> Am I missing any steps?
>>> Thanks 
>>> Sumo
>>> 
>> 
>