Re: 3 node of nifi generating 3 different flow files

[Andy LoPresto]

Sanjeet, 

Did you stop every node in the cluster before deleting these files? Can you 
share the actual output of the log files?

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 18, 2020, at 5:15 PM, sanjeet rath  wrote:
> 
> Hi Andy,
> Already tried by deleting flow.xml.gz ,users.xml,authorization.xml file of 3 
> nodes.
> But still getting the same error.
> in nifi-app.log is showing exception that" local flow is differet than 
> cluster flow"
> Also new flow.xml.gz file is having different 'rootGroupid' on each node
> Thanks.
> Sanjeet
> 
> On Fri, 19 Jun 2020, 5:16 am Andy LoPresto,  > wrote:
> Sanjeet, 
> 
> If this is for a new cluster, you can delete the flow.xml.gz file from all 
> nodes and restart NiFi. When the nodes start up again, they will create the 
> new flow definition file on each node respectively with the synced root 
> process group ID. 
> 
> Andy LoPresto
> alopre...@apache.org 
> alopresto.apa...@gmail.com 
> He/Him
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Jun 18, 2020, at 4:44 PM, sanjeet rath > > wrote:
>> 
>> Thanks, Andy for your quick response.
>> I did it as you suggested, now nifi is using its default encryption.
>> 
>> But still gettting the error, in nifi-app.log is showing exception that" 
>> local flow is differet than cluster flow"
>> As per my analysis, The issue is it got 3 different flow.xml.gz files in 
>> each node.The 'rootGroupid' of each flow.xml.gz file is different.
>> As per my understanding ,the flow.xml.gz fie should be same accross all 
>> nodes.so i copied flow.xml.gz file from one node to other two nodes to make 
>> it sync.but still getting same error.
>> The zookeeper is connected properly to all nodes.
>> 
>> Thanks,
>> Sanjeet
>> 
>> 
>> On Thu, 18 Jun 2020, 10:55 pm Andy LoPresto, > > wrote:
>> You do not need to manually run this command, as it migrates the encryption 
>> key used for sensitive processor properties (e.g. database passwords in the 
>> flow) that are stored in your flow.xml.gz file. You only need this command 
>> when you have a cluster which has been using one encryption key to secure 
>> these values and you want to migrate to a new key. 
>> 
>> When starting a new cluster, set the nifi.sensitive.props.key value to the 
>> desired value on all cluster nodes, and NiFi will automatically encrypt and 
>> decrypt the sensitive processor properties with it.  
>> 
>> 
>> Andy LoPresto
>> alopre...@apache.org 
>> alopresto.apa...@gmail.com 
>> He/Him
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>>> On Jun 18, 2020, at 3:55 AM, sanjeet rath >> > wrote:
>>> 
>>> Hi ,
>>> 
>>> I got a issue while starting a 3 node nifi cluster. nifi-app.log is showing 
>>> exception that" local flow file is differet than cluster flow"
>>> The issue is i got 3 different flow files in each node.'rootGroupid' of 
>>> each flow file is different.
>>> 
>>> Previous i was facing no issue in creating 3 node clister.but this time ,i 
>>> made a change here  during creation the clusters i have encripted the flow 
>>> file with using nifi toolkit.(encript-config.sh)
>>> 
>>>  Encript-config.sh -f /nifi/flow.xml.gz -n nifi/nifi.properties 
>>> --bootstrapconf /nifi/bootstrap.conf -s password -x
>>> 
>>> Please note that there is are no flows are availavle in canvas.its a new 
>>> nifi cluster.
>>> 
>>> Could you please help me to understand what i am doing wrong?
>>> 
>>> Thanks a lot in advance for helping me.
>>> 
>>> Regards,
>>> Sanjeet
>> 
> 

Re: 3 node of nifi generating 3 different flow files

[sanjeet rath]

Hi Andy,
Already tried by deleting flow.xml.gz ,users.xml,authorization.xml file of
3 nodes.
But still getting the same error.

in nifi-app.log is showing exception that" local flow is differet than
cluster flow"

Also new flow.xml.gz file is having different 'rootGroupid' on each node
Thanks.
Sanjeet

On Fri, 19 Jun 2020, 5:16 am Andy LoPresto,  wrote:

> Sanjeet,
>
> If this is for a new cluster, you can delete the flow.xml.gz file from all
> nodes and restart NiFi. When the nodes start up again, they will create the
> new flow definition file on each node respectively with the synced root
> process group ID.
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> He/Him
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 18, 2020, at 4:44 PM, sanjeet rath  wrote:
>
> Thanks, Andy for your quick response.
> I did it as you suggested, now nifi is using its default encryption.
>
> But still gettting the error, in nifi-app.log is showing exception that"
> local flow is differet than cluster flow"
>
> As per my analysis, The issue is it got 3 different flow.xml.gz files in
> each node.The 'rootGroupid' of each flow.xml.gz file is different.
>
> As per my understanding ,the flow.xml.gz fie should be same accross all
> nodes.so i copied flow.xml.gz file from one node to other two nodes to make
> it sync.but still getting same error.
> The zookeeper is connected properly to all nodes.
>
> Thanks,
> Sanjeet
>
>
>
> On Thu, 18 Jun 2020, 10:55 pm Andy LoPresto,  wrote:
>
>> You do not need to manually run this command, as it migrates the
>> encryption key used for sensitive processor properties (e.g. database
>> passwords in the flow) that are stored in your flow.xml.gz file. You only
>> need this command when you have a cluster which has been using one
>> encryption key to secure these values and you want to migrate to a new key.
>>
>> When starting a new cluster, set the nifi.sensitive.props.key value to
>> the desired value on all cluster nodes, and NiFi will automatically encrypt
>> and decrypt the sensitive processor properties with it.
>>
>>
>> Andy LoPresto
>> alopre...@apache.org
>> *alopresto.apa...@gmail.com *
>> He/Him
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Jun 18, 2020, at 3:55 AM, sanjeet rath  wrote:
>>
>> Hi ,
>>
>> I got a issue while starting a 3 node nifi cluster. nifi-app.log is
>> showing exception that" local flow file is differet than cluster flow"
>> The issue is i got 3 different flow files in each node.'rootGroupid' of
>> each flow file is different.
>>
>> Previous i was facing no issue in creating 3 node clister.but this time
>> ,i made a change here  during creation the clusters i have encripted the
>> flow file with using nifi toolkit.(encript-config.sh)
>>
>>  Encript-config.sh -f /nifi/flow.xml.gz -n nifi/nifi.properties
>> --bootstrapconf /nifi/bootstrap.conf -s password -x
>>
>> Please note that there is are no flows are availavle in canvas.its a new
>> nifi cluster.
>>
>> Could you please help me to understand what i am doing wrong?
>>
>> Thanks a lot in advance for helping me.
>>
>> Regards,
>> Sanjeet
>>
>>
>>
>

Re: 3 node of nifi generating 3 different flow files

[Andy LoPresto]

Sanjeet, 

If this is for a new cluster, you can delete the flow.xml.gz file from all 
nodes and restart NiFi. When the nodes start up again, they will create the new 
flow definition file on each node respectively with the synced root process 
group ID. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 18, 2020, at 4:44 PM, sanjeet rath  wrote:
> 
> Thanks, Andy for your quick response.
> I did it as you suggested, now nifi is using its default encryption.
> 
> But still gettting the error, in nifi-app.log is showing exception that" 
> local flow is differet than cluster flow"
> As per my analysis, The issue is it got 3 different flow.xml.gz files in each 
> node.The 'rootGroupid' of each flow.xml.gz file is different.
> As per my understanding ,the flow.xml.gz fie should be same accross all 
> nodes.so i copied flow.xml.gz file from one node to other two nodes to make 
> it sync.but still getting same error.
> The zookeeper is connected properly to all nodes.
> 
> Thanks,
> Sanjeet
> 
> 
> On Thu, 18 Jun 2020, 10:55 pm Andy LoPresto,  > wrote:
> You do not need to manually run this command, as it migrates the encryption 
> key used for sensitive processor properties (e.g. database passwords in the 
> flow) that are stored in your flow.xml.gz file. You only need this command 
> when you have a cluster which has been using one encryption key to secure 
> these values and you want to migrate to a new key. 
> 
> When starting a new cluster, set the nifi.sensitive.props.key value to the 
> desired value on all cluster nodes, and NiFi will automatically encrypt and 
> decrypt the sensitive processor properties with it.  
> 
> 
> Andy LoPresto
> alopre...@apache.org 
> alopresto.apa...@gmail.com 
> He/Him
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Jun 18, 2020, at 3:55 AM, sanjeet rath > > wrote:
>> 
>> Hi ,
>> 
>> I got a issue while starting a 3 node nifi cluster. nifi-app.log is showing 
>> exception that" local flow file is differet than cluster flow"
>> The issue is i got 3 different flow files in each node.'rootGroupid' of each 
>> flow file is different.
>> 
>> Previous i was facing no issue in creating 3 node clister.but this time ,i 
>> made a change here  during creation the clusters i have encripted the flow 
>> file with using nifi toolkit.(encript-config.sh)
>> 
>>  Encript-config.sh -f /nifi/flow.xml.gz -n nifi/nifi.properties 
>> --bootstrapconf /nifi/bootstrap.conf -s password -x
>> 
>> Please note that there is are no flows are availavle in canvas.its a new 
>> nifi cluster.
>> 
>> Could you please help me to understand what i am doing wrong?
>> 
>> Thanks a lot in advance for helping me.
>> 
>> Regards,
>> Sanjeet
> 

Re: 3 node of nifi generating 3 different flow files

[sanjeet rath]

Thanks, Andy for your quick response.
I did it as you suggested, now nifi is using its default encryption.

But still gettting the error, in nifi-app.log is showing exception that"
local flow is differet than cluster flow"

As per my analysis, The issue is it got 3 different flow.xml.gz files in
each node.The 'rootGroupid' of each flow.xml.gz file is different.

As per my understanding ,the flow.xml.gz fie should be same accross all
nodes.so i copied flow.xml.gz file from one node to other two nodes to make
it sync.but still getting same error.
The zookeeper is connected properly to all nodes.

Thanks,
Sanjeet



On Thu, 18 Jun 2020, 10:55 pm Andy LoPresto,  wrote:

> You do not need to manually run this command, as it migrates the
> encryption key used for sensitive processor properties (e.g. database
> passwords in the flow) that are stored in your flow.xml.gz file. You only
> need this command when you have a cluster which has been using one
> encryption key to secure these values and you want to migrate to a new key.
>
> When starting a new cluster, set the nifi.sensitive.props.key value to the
> desired value on all cluster nodes, and NiFi will automatically encrypt and
> decrypt the sensitive processor properties with it.
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> He/Him
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 18, 2020, at 3:55 AM, sanjeet rath  wrote:
>
> Hi ,
>
> I got a issue while starting a 3 node nifi cluster. nifi-app.log is
> showing exception that" local flow file is differet than cluster flow"
> The issue is i got 3 different flow files in each node.'rootGroupid' of
> each flow file is different.
>
> Previous i was facing no issue in creating 3 node clister.but this time ,i
> made a change here  during creation the clusters i have encripted the flow
> file with using nifi toolkit.(encript-config.sh)
>
>  Encript-config.sh -f /nifi/flow.xml.gz -n nifi/nifi.properties
> --bootstrapconf /nifi/bootstrap.conf -s password -x
>
> Please note that there is are no flows are availavle in canvas.its a new
> nifi cluster.
>
> Could you please help me to understand what i am doing wrong?
>
> Thanks a lot in advance for helping me.
>
> Regards,
> Sanjeet
>
>
>

Re: 3 node of nifi generating 3 different flow files

[Andy LoPresto]

You do not need to manually run this command, as it migrates the encryption key 
used for sensitive processor properties (e.g. database passwords in the flow) 
that are stored in your flow.xml.gz file. You only need this command when you 
have a cluster which has been using one encryption key to secure these values 
and you want to migrate to a new key. 

When starting a new cluster, set the nifi.sensitive.props.key value to the 
desired value on all cluster nodes, and NiFi will automatically encrypt and 
decrypt the sensitive processor properties with it.  


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 18, 2020, at 3:55 AM, sanjeet rath  wrote:
> 
> Hi ,
> 
> I got a issue while starting a 3 node nifi cluster. nifi-app.log is showing 
> exception that" local flow file is differet than cluster flow"
> The issue is i got 3 different flow files in each node.'rootGroupid' of each 
> flow file is different.
> 
> Previous i was facing no issue in creating 3 node clister.but this time ,i 
> made a change here  during creation the clusters i have encripted the flow 
> file with using nifi toolkit.(encript-config.sh)
> 
>  Encript-config.sh -f /nifi/flow.xml.gz -n nifi/nifi.properties 
> --bootstrapconf /nifi/bootstrap.conf -s password -x
> 
> Please note that there is are no flows are availavle in canvas.its a new nifi 
> cluster.
> 
> Could you please help me to understand what i am doing wrong?
> 
> Thanks a lot in advance for helping me.
> 
> Regards,
> Sanjeet

3 node of nifi generating 3 different flow files

[sanjeet rath]

Hi ,

I got a issue while starting a 3 node nifi cluster. nifi-app.log is showing
exception that" local flow file is differet than cluster flow"
The issue is i got 3 different flow files in each node.'rootGroupid' of
each flow file is different.

Previous i was facing no issue in creating 3 node clister.but this time ,i
made a change here  during creation the clusters i have encripted the flow
file with using nifi toolkit.(encript-config.sh)

 Encript-config.sh -f /nifi/flow.xml.gz -n nifi/nifi.properties
--bootstrapconf /nifi/bootstrap.conf -s password -x

Please note that there is are no flows are availavle in canvas.its a new
nifi cluster.

Could you please help me to understand what i am doing wrong?

Thanks a lot in advance for helping me.

Regards,
Sanjeet