Re: Nifi 1.8 to 1.11.4 migration issue
Thanks Mark for the update. Could you please the jira request number on which this issue is fixed. Regards, Sanjeet On Fri, 31 Jul 2020, 12:51 am Mark Payne, wrote: > OK, I suspect you’re probably hitting an issue that was introduced in > 1.11.x related to fingerprint handling. A change was made to improve how > some things were handled in the flow fingerprinting logic but resulted in > very poor performance in some particular cases. I.e., some flows will load > quickly while others load very slowly. The good news is that this has since > been fixed, though the fix hasn’t yet been released. I would expect the > next release to come pretty soon, though. > > Thanks > -Mark > > On Jul 30, 2020, at 3:15 PM, sanjeet rath wrote: > > Hi Mark, > Thanks for responding. > Sorry, i can't share the log, Due to our company policy. > its keeping printing this 2 warning in nifi-app.log once the server is up > this warning is gone. > 1-> O.apache.nifi.controler.flocontroler failed to send heart beat due to > ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling > 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. > 2-> o.a.nifi.fingerprint.FingerprintFactory unable to get the > controller service of type org.apache.nifi.processor.aws.s3.puts3object;it > is default properties will be fingerprinted instead of being ignored > > Thanks Boris for the response, but i am not that issue, facing node > disconnection issue. > > > > On Fri, Jul 31, 2020 at 12:18 AM Mark Payne wrote: > >> Sanjeet, >> >> I’d recommend grabbing a thread dump while NiFi is starting up (after >> it’s been going for 3 minutes or so) and providing that to understand why >> it’s taking 30 minutes to startup. Specifically, the “main” thread will be >> of interest. >> >> Thanks >> -Mark >> >> >> On Jul 30, 2020, at 2:04 PM, sanjeet rath wrote: >> >> Hi , >> yes i am using external zookeeper 3.5 version.I have already followed all >> the document you have share . >> Now i am seeing after waiting around 30 mins the node gets >> connected after that there is no node disconnection issue. >> when i restarted , again it is taking 30 min for node connection to >> cluster. >> >> not able to figure out where is the issue. >> and same below warning also comes. >> O.apache.nifi.controler.flocontroler failed to send heart beat due to ; >> org.apache.nifi.cluster.protocol.protocalException: Failed marshalling >> 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. >> >> On Thu, Jul 30, 2020 at 8:01 PM Nathan Gough wrote: >> >>> Hi Sanjeet, >>> >>> One thing to do is make sure you're using at least Zookeeper 3.5.5: >>> >>> Migrating from 1.x.x to 1.10.0 - The RPM creation mechanism appears broken for both Java 8 and Java 11 binaries if you wanted to build those yourself. Should be resolved in a later release. - We've removed the following nars from the default convenience binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, druid-controller-service-api-nar, druid-nar, other-graph-services-nar. You can still get them from the various artifact repositories and use them in your flows but we cannot bundle them due to space limitations by default. - The "Auto-Create Partitions" property was removed from the PutHive3Streaming processor, causing existing instances of this processor to become invalid. The property would appear as a unsupported user-defined property and must be removed to return the processor to a valid state. - The Zookeeper dependency that NiFi uses for state management and cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper changed the zookeeper.properties file format and as a result NiFi users using an existing embedded zookeeper will need to adjust their existing zookeeper.properties file accordingly*. More details here: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport . For new deployments of the 1.10.0 release onwards, NiFi will be packaged with an updated template zookeeper.properties file. To update an existing zookeeper.properties file however, edit the conf/zookeeper.properties file: 1. Remove the clientPort=2181 line (or whatever your port number may be) 2. Add the client port to the end of the server string eg: server.1=localhost:2888:3888;2181 >>> You may also need to ensure your server certificates contain a DNS >>> entry in the Subject Alternative Name (SAN) for their respective hosts. >>> >>> On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath >>> wrote: >>> Hi, The error is, O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.proto
Re: Nifi 1.8 to 1.11.4 migration issue
OK, I suspect you’re probably hitting an issue that was introduced in 1.11.x related to fingerprint handling. A change was made to improve how some things were handled in the flow fingerprinting logic but resulted in very poor performance in some particular cases. I.e., some flows will load quickly while others load very slowly. The good news is that this has since been fixed, though the fix hasn’t yet been released. I would expect the next release to come pretty soon, though. Thanks -Mark On Jul 30, 2020, at 3:15 PM, sanjeet rath mailto:rath.sanj...@gmail.com>> wrote: Hi Mark, Thanks for responding. Sorry, i can't share the log, Due to our company policy. its keeping printing this 2 warning in nifi-app.log once the server is up this warning is gone. 1-> O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. 2-> o.a.nifi.fingerprint.FingerprintFactory unable to get the controller service of type org.apache.nifi.processor.aws.s3.puts3object;it is default properties will be fingerprinted instead of being ignored Thanks Boris for the response, but i am not that issue, facing node disconnection issue. On Fri, Jul 31, 2020 at 12:18 AM Mark Payne mailto:marka...@hotmail.com>> wrote: Sanjeet, I’d recommend grabbing a thread dump while NiFi is starting up (after it’s been going for 3 minutes or so) and providing that to understand why it’s taking 30 minutes to startup. Specifically, the “main” thread will be of interest. Thanks -Mark On Jul 30, 2020, at 2:04 PM, sanjeet rath mailto:rath.sanj...@gmail.com>> wrote: Hi , yes i am using external zookeeper 3.5 version.I have already followed all the document you have share . Now i am seeing after waiting around 30 mins the node gets connected after that there is no node disconnection issue. when i restarted , again it is taking 30 min for node connection to cluster. not able to figure out where is the issue. and same below warning also comes. O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. On Thu, Jul 30, 2020 at 8:01 PM Nathan Gough mailto:thena...@gmail.com>> wrote: Hi Sanjeet, One thing to do is make sure you're using at least Zookeeper 3.5.5: Migrating from 1.x.x to 1.10.0 * The RPM creation mechanism appears broken for both Java 8 and Java 11 binaries if you wanted to build those yourself. Should be resolved in a later release. * We've removed the following nars from the default convenience binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, druid-controller-service-api-nar, druid-nar, other-graph-services-nar. You can still get them from the various artifact repositories and use them in your flows but we cannot bundle them due to space limitations by default. * The "Auto-Create Partitions" property was removed from the PutHive3Streaming processor, causing existing instances of this processor to become invalid. The property would appear as a unsupported user-defined property and must be removed to return the processor to a valid state. * The Zookeeper dependency that NiFi uses for state management and cluster elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the zookeeper.properties file format and as a result NiFi users using an existing embedded zookeeper will need to adjust their existing zookeeper.properties file accordingly. More details here: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport. For new deployments of the 1.10.0 release onwards, NiFi will be packaged with an updated template zookeeper.properties file. To update an existing zookeeper.properties file however, edit the conf/zookeeper.properties file: * Remove the clientPort=2181 line (or whatever your port number may be) * Add the client port to the end of the server string eg: server.1=localhost:2888:3888;2181 You may also need to ensure your server certificates contain a DNS entry in the Subject Alternative Name (SAN) for their respective hosts. On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath mailto:rath.sanj...@gmail.com>> wrote: Hi, The error is, O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. Before migration there was no issue in cluster and ckuster are connected. Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 server. Thanks & Regards Sanjeet On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, mailto:rath.sanj...@gmail.com>> wrote: nifi-app.log file , i am getting constant error. Apache.nifi.controler.Flo
Re: Nifi 1.8 to 1.11.4 migration issue
Hi Mark, Thanks for responding. Sorry, i can't share the log, Due to our company policy. its keeping printing this 2 warning in nifi-app.log once the server is up this warning is gone. 1-> O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. 2-> o.a.nifi.fingerprint.FingerprintFactory unable to get the controller service of type org.apache.nifi.processor.aws.s3.puts3object;it is default properties will be fingerprinted instead of being ignored Thanks Boris for the response, but i am not that issue, facing node disconnection issue. On Fri, Jul 31, 2020 at 12:18 AM Mark Payne wrote: > Sanjeet, > > I’d recommend grabbing a thread dump while NiFi is starting up (after it’s > been going for 3 minutes or so) and providing that to understand why it’s > taking 30 minutes to startup. Specifically, the “main” thread will be of > interest. > > Thanks > -Mark > > > On Jul 30, 2020, at 2:04 PM, sanjeet rath wrote: > > Hi , > yes i am using external zookeeper 3.5 version.I have already followed all > the document you have share . > Now i am seeing after waiting around 30 mins the node gets connected after > that there is no node disconnection issue. > when i restarted , again it is taking 30 min for node connection to > cluster. > > not able to figure out where is the issue. > and same below warning also comes. > O.apache.nifi.controler.flocontroler failed to send heart beat due to ; > org.apache.nifi.cluster.protocol.protocalException: Failed marshalling > 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. > > On Thu, Jul 30, 2020 at 8:01 PM Nathan Gough wrote: > >> Hi Sanjeet, >> >> One thing to do is make sure you're using at least Zookeeper 3.5.5: >> >> Migrating from 1.x.x to 1.10.0 >>> >>>- The RPM creation mechanism appears broken for both Java 8 and Java >>>11 binaries if you wanted to build those yourself. Should be resolved >>> in a >>>later release. >>>- We've removed the following nars from the default convenience >>>binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, >>>druid-controller-service-api-nar, druid-nar, other-graph-services-nar. >>> You >>>can still get them from the various artifact repositories and use them in >>>your flows but we cannot bundle them due to space limitations by default. >>>- The "Auto-Create Partitions" property was removed from the >>>PutHive3Streaming processor, causing existing instances of this processor >>>to become invalid. The property would appear as a unsupported >>> user-defined >>>property and must be removed to return the processor to a valid state. >>>- The Zookeeper dependency that NiFi uses for state management and >>>cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper >>>changed the zookeeper.properties file format and as a result NiFi users >>>using an existing embedded zookeeper will need to adjust their existing >>>zookeeper.properties file accordingly*. More details here: >>> >>> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport >>>. >>>For new deployments of the 1.10.0 release onwards, NiFi will be >>>packaged with an updated template zookeeper.properties file. >>>To update an existing zookeeper.properties file however, edit the >>>conf/zookeeper.properties file: >>> 1. Remove the clientPort=2181 line (or whatever your port number >>> may be) >>> 2. Add the client port to the end of the server string eg: >>> server.1=localhost:2888:3888;2181 >>> >>> >> You may also need to ensure your server certificates contain a DNS entry >> in the Subject Alternative Name (SAN) for their respective hosts. >> >> On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath >> wrote: >> >>> Hi, >>> >>> The error is, >>> >>> O.apache.nifi.controler.flocontroler failed to send heart beat due to ; >>> org.apache.nifi.cluster.protocol.protocalException: Failed marshalling >>> 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. >>> >>> Before migration there was no issue in cluster and ckuster are connected. >>> Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 >>> server. >>> >>> Thanks & Regards >>> Sanjeet >>> >>> >>> On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, >>> wrote: >>> nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolException On Thu, 30 Jul 2020, 6:54 pm Mark Payne, wrote: > Sanjeet, > > What error are you receiving? > > Sent from my iPhone > > On Jul 30, 2020, at 9:21 AM, sanjeet rath > wrote: > > > Hi Team, > > Any help on my trailed mail query will be high
Re: Nifi 1.8 to 1.11.4 migration issue
Sanjeet, I’d recommend grabbing a thread dump while NiFi is starting up (after it’s been going for 3 minutes or so) and providing that to understand why it’s taking 30 minutes to startup. Specifically, the “main” thread will be of interest. Thanks -Mark On Jul 30, 2020, at 2:04 PM, sanjeet rath mailto:rath.sanj...@gmail.com>> wrote: Hi , yes i am using external zookeeper 3.5 version.I have already followed all the document you have share . Now i am seeing after waiting around 30 mins the node gets connected after that there is no node disconnection issue. when i restarted , again it is taking 30 min for node connection to cluster. not able to figure out where is the issue. and same below warning also comes. O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. On Thu, Jul 30, 2020 at 8:01 PM Nathan Gough mailto:thena...@gmail.com>> wrote: Hi Sanjeet, One thing to do is make sure you're using at least Zookeeper 3.5.5: Migrating from 1.x.x to 1.10.0 * The RPM creation mechanism appears broken for both Java 8 and Java 11 binaries if you wanted to build those yourself. Should be resolved in a later release. * We've removed the following nars from the default convenience binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, druid-controller-service-api-nar, druid-nar, other-graph-services-nar. You can still get them from the various artifact repositories and use them in your flows but we cannot bundle them due to space limitations by default. * The "Auto-Create Partitions" property was removed from the PutHive3Streaming processor, causing existing instances of this processor to become invalid. The property would appear as a unsupported user-defined property and must be removed to return the processor to a valid state. * The Zookeeper dependency that NiFi uses for state management and cluster elections was upgraded to v3.5.5. From v3.5.x onwards, Zookeeper changed the zookeeper.properties file format and as a result NiFi users using an existing embedded zookeeper will need to adjust their existing zookeeper.properties file accordingly. More details here: https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport. For new deployments of the 1.10.0 release onwards, NiFi will be packaged with an updated template zookeeper.properties file. To update an existing zookeeper.properties file however, edit the conf/zookeeper.properties file: * Remove the clientPort=2181 line (or whatever your port number may be) * Add the client port to the end of the server string eg: server.1=localhost:2888:3888;2181 You may also need to ensure your server certificates contain a DNS entry in the Subject Alternative Name (SAN) for their respective hosts. On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath mailto:rath.sanj...@gmail.com>> wrote: Hi, The error is, O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. Before migration there was no issue in cluster and ckuster are connected. Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 server. Thanks & Regards Sanjeet On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, mailto:rath.sanj...@gmail.com>> wrote: nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolException On Thu, 30 Jul 2020, 6:54 pm Mark Payne, mailto:marka...@hotmail.com>> wrote: Sanjeet, What error are you receiving? Sent from my iPhone On Jul 30, 2020, at 9:21 AM, sanjeet rath mailto:rath.sanj...@gmail.com>> wrote: Hi Team, Any help on my trailed mail query will be highly appreciated.still i am strgling yo fix yhe issue. In nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolEcslception. Thanks & Regards, Sanjeet On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, mailto:rath.sanj...@gmail.com>> wrote: Hi Team, I am facing a wired issue while doing migration from 1.8 to 1.11.4 I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi cluster(1.8 version) to a newly created 1.11.4 version cluster. When i am starting the 1.11.4 cluster with this new files, i am getting node disconnected error (this node is not conncted to cluster pop up ) However with only users.xml, authrization.xml from 1.8 and default 1.11.4 flow.xml.gz. i am getting no error all 3 nodes are getting conncted in the new cluster. Infact when i am bringing another 1.8 env's flow.xml.gz files it working fine. So specific for this flow.xml.gz file only my nodes are getting disconncted.
Re: Nifi 1.8 to 1.11.4 migration issue
Sanjeet, just a heads up as we were in the same boat. We decided to hold off the upgrade because of a critical issue in 1.11.4. not sure if it applies to your environment but we decided to wait for 1.12 more details here https://issues.apache.org/jira/browse/NIFI-7460 On Thu, Jul 30, 2020 at 2:04 PM sanjeet rath wrote: > Hi , > yes i am using external zookeeper 3.5 version.I have already followed all > the document you have share . > Now i am seeing after waiting around 30 mins the node gets connected after > that there is no node disconnection issue. > when i restarted , again it is taking 30 min for node connection to > cluster. > > not able to figure out where is the issue. > and same below warning also comes. > O.apache.nifi.controler.flocontroler failed to send heart beat due to ; > org.apache.nifi.cluster.protocol.protocalException: Failed marshalling > 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. > > On Thu, Jul 30, 2020 at 8:01 PM Nathan Gough wrote: > >> Hi Sanjeet, >> >> One thing to do is make sure you're using at least Zookeeper 3.5.5: >> >> Migrating from 1.x.x to 1.10.0 >>> >>>- The RPM creation mechanism appears broken for both Java 8 and Java >>>11 binaries if you wanted to build those yourself. Should be resolved >>> in a >>>later release. >>>- We've removed the following nars from the default convenience >>>binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, >>>druid-controller-service-api-nar, druid-nar, other-graph-services-nar. >>> You >>>can still get them from the various artifact repositories and use them in >>>your flows but we cannot bundle them due to space limitations by default. >>>- The "Auto-Create Partitions" property was removed from the >>>PutHive3Streaming processor, causing existing instances of this processor >>>to become invalid. The property would appear as a unsupported >>> user-defined >>>property and must be removed to return the processor to a valid state. >>>- The Zookeeper dependency that NiFi uses for state management and >>>cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper >>>changed the zookeeper.properties file format and as a result NiFi users >>>using an existing embedded zookeeper will need to adjust their existing >>>zookeeper.properties file accordingly*. More details here: >>> >>> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport >>>. >>>For new deployments of the 1.10.0 release onwards, NiFi will be >>>packaged with an updated template zookeeper.properties file. >>>To update an existing zookeeper.properties file however, edit the >>>conf/zookeeper.properties file: >>> 1. Remove the clientPort=2181 line (or whatever your port number >>> may be) >>> 2. Add the client port to the end of the server string eg: >>> server.1=localhost:2888:3888;2181 >>> >>> >> You may also need to ensure your server certificates contain a DNS entry >> in the Subject Alternative Name (SAN) for their respective hosts. >> >> On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath >> wrote: >> >>> Hi, >>> >>> The error is, >>> >>> O.apache.nifi.controler.flocontroler failed to send heart beat due to ; >>> org.apache.nifi.cluster.protocol.protocalException: Failed marshalling >>> 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. >>> >>> Before migration there was no issue in cluster and ckuster are connected. >>> Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 >>> server. >>> >>> Thanks & Regards >>> Sanjeet >>> >>> >>> On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, >>> wrote: >>> nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolException On Thu, 30 Jul 2020, 6:54 pm Mark Payne, wrote: > Sanjeet, > > What error are you receiving? > > Sent from my iPhone > > On Jul 30, 2020, at 9:21 AM, sanjeet rath > wrote: > > > Hi Team, > > Any help on my trailed mail query will be highly appreciated.still i > am strgling yo fix yhe issue. > > In nifi-app.log file , i am getting constant error. > > Apache.nifi.controler.FlowControler Failed to send heart beat due to > org.apache.nifi.protocal.protocolEcslception. > > Thanks & Regards, > Sanjeet > > On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, > wrote: > >> Hi Team, >> >> I am facing a wired issue while doing migration from 1.8 to 1.11.4 >> I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi >> cluster(1.8 version) to a newly created 1.11.4 version cluster. >> >> When i am starting the 1.11.4 cluster with this new files, i am >> getting node disconnected error (this node is not conncted t
Re: Nifi 1.8 to 1.11.4 migration issue
Hi , yes i am using external zookeeper 3.5 version.I have already followed all the document you have share . Now i am seeing after waiting around 30 mins the node gets connected after that there is no node disconnection issue. when i restarted , again it is taking 30 min for node connection to cluster. not able to figure out where is the issue. and same below warning also comes. O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. On Thu, Jul 30, 2020 at 8:01 PM Nathan Gough wrote: > Hi Sanjeet, > > One thing to do is make sure you're using at least Zookeeper 3.5.5: > > Migrating from 1.x.x to 1.10.0 >> >>- The RPM creation mechanism appears broken for both Java 8 and Java >>11 binaries if you wanted to build those yourself. Should be resolved in >> a >>later release. >>- We've removed the following nars from the default convenience >>binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, >>druid-controller-service-api-nar, druid-nar, other-graph-services-nar. >> You >>can still get them from the various artifact repositories and use them in >>your flows but we cannot bundle them due to space limitations by default. >>- The "Auto-Create Partitions" property was removed from the >>PutHive3Streaming processor, causing existing instances of this processor >>to become invalid. The property would appear as a unsupported user-defined >>property and must be removed to return the processor to a valid state. >>- The Zookeeper dependency that NiFi uses for state management and >>cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper >>changed the zookeeper.properties file format and as a result NiFi users >>using an existing embedded zookeeper will need to adjust their existing >>zookeeper.properties file accordingly*. More details here: >> >> https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport >>. >>For new deployments of the 1.10.0 release onwards, NiFi will be >>packaged with an updated template zookeeper.properties file. >>To update an existing zookeeper.properties file however, edit the >>conf/zookeeper.properties file: >> 1. Remove the clientPort=2181 line (or whatever your port number >> may be) >> 2. Add the client port to the end of the server string eg: >> server.1=localhost:2888:3888;2181 >> >> > You may also need to ensure your server certificates contain a DNS entry > in the Subject Alternative Name (SAN) for their respective hosts. > > On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath > wrote: > >> Hi, >> >> The error is, >> >> O.apache.nifi.controler.flocontroler failed to send heart beat due to ; >> org.apache.nifi.cluster.protocol.protocalException: Failed marshalling >> 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. >> >> Before migration there was no issue in cluster and ckuster are connected. >> Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 >> server. >> >> Thanks & Regards >> Sanjeet >> >> >> On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, >> wrote: >> >>> nifi-app.log file , i am getting constant error. >>> >>> Apache.nifi.controler.FlowControler Failed to send heart beat due to >>> org.apache.nifi.protocal.protocolException >>> >>> On Thu, 30 Jul 2020, 6:54 pm Mark Payne, wrote: >>> Sanjeet, What error are you receiving? Sent from my iPhone On Jul 30, 2020, at 9:21 AM, sanjeet rath wrote: Hi Team, Any help on my trailed mail query will be highly appreciated.still i am strgling yo fix yhe issue. In nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolEcslception. Thanks & Regards, Sanjeet On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, wrote: > Hi Team, > > I am facing a wired issue while doing migration from 1.8 to 1.11.4 > I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi > cluster(1.8 version) to a newly created 1.11.4 version cluster. > > When i am starting the 1.11.4 cluster with this new files, i am > getting node disconnected error (this node is not conncted to cluster pop > up ) > > However with only users.xml, authrization.xml from 1.8 and default > 1.11.4 flow.xml.gz. i am getting no error all 3 nodes are getting conncted > in the new cluster. > Infact when i am bringing another 1.8 env's flow.xml.gz files it > working fine. > > So specific for this flow.xml.gz file only my nodes are getting > disconncted. > > It will really helpfull if you can help where i need to c
Re: Nifi 1.8 to 1.11.4 migration issue
Hi Sanjeet, One thing to do is make sure you're using at least Zookeeper 3.5.5: Migrating from 1.x.x to 1.10.0 > >- The RPM creation mechanism appears broken for both Java 8 and Java >11 binaries if you wanted to build those yourself. Should be resolved in a >later release. >- We've removed the following nars from the default convenience >binary. These include kite-nar, kafka-0-8-nar, flume-nar, media-nar, >druid-controller-service-api-nar, druid-nar, other-graph-services-nar. You >can still get them from the various artifact repositories and use them in >your flows but we cannot bundle them due to space limitations by default. >- The "Auto-Create Partitions" property was removed from the >PutHive3Streaming processor, causing existing instances of this processor >to become invalid. The property would appear as a unsupported user-defined >property and must be removed to return the processor to a valid state. >- The Zookeeper dependency that NiFi uses for state management and >cluster elections was upgraded to v3.5.5. From v3.5.x onwards, *Zookeeper >changed the zookeeper.properties file format and as a result NiFi users >using an existing embedded zookeeper will need to adjust their existing >zookeeper.properties file accordingly*. More details here: > > https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html#sc_reconfig_clientport >. >For new deployments of the 1.10.0 release onwards, NiFi will be >packaged with an updated template zookeeper.properties file. >To update an existing zookeeper.properties file however, edit the >conf/zookeeper.properties file: > 1. Remove the clientPort=2181 line (or whatever your port number > may be) > 2. Add the client port to the end of the server string eg: > server.1=localhost:2888:3888;2181 > > You may also need to ensure your server certificates contain a DNS entry in the Subject Alternative Name (SAN) for their respective hosts. On Thu, Jul 30, 2020 at 10:17 AM sanjeet rath wrote: > Hi, > > The error is, > > O.apache.nifi.controler.flocontroler failed to send heart beat due to ; > org.apache.nifi.cluster.protocol.protocalException: Failed marshalling > 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. > > Before migration there was no issue in cluster and ckuster are connected. > Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 > server. > > Thanks & Regards > Sanjeet > > > On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, wrote: > >> nifi-app.log file , i am getting constant error. >> >> Apache.nifi.controler.FlowControler Failed to send heart beat due to >> org.apache.nifi.protocal.protocolException >> >> On Thu, 30 Jul 2020, 6:54 pm Mark Payne, wrote: >> >>> Sanjeet, >>> >>> What error are you receiving? >>> >>> Sent from my iPhone >>> >>> On Jul 30, 2020, at 9:21 AM, sanjeet rath >>> wrote: >>> >>> >>> Hi Team, >>> >>> Any help on my trailed mail query will be highly appreciated.still i am >>> strgling yo fix yhe issue. >>> >>> In nifi-app.log file , i am getting constant error. >>> >>> Apache.nifi.controler.FlowControler Failed to send heart beat due to >>> org.apache.nifi.protocal.protocolEcslception. >>> >>> Thanks & Regards, >>> Sanjeet >>> >>> On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, >>> wrote: >>> Hi Team, I am facing a wired issue while doing migration from 1.8 to 1.11.4 I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi cluster(1.8 version) to a newly created 1.11.4 version cluster. When i am starting the 1.11.4 cluster with this new files, i am getting node disconnected error (this node is not conncted to cluster pop up ) However with only users.xml, authrization.xml from 1.8 and default 1.11.4 flow.xml.gz. i am getting no error all 3 nodes are getting conncted in the new cluster. Infact when i am bringing another 1.8 env's flow.xml.gz files it working fine. So specific for this flow.xml.gz file only my nodes are getting disconncted. It will really helpfull if you can help where i need to check to resolve this issue. I am using external zookeeper 3.5 version Thanks in advance, Sanjeet
Re: Nifi 1.8 to 1.11.4 migration issue
Hi, The error is, O.apache.nifi.controler.flocontroler failed to send heart beat due to ; org.apache.nifi.cluster.protocol.protocalException: Failed marshalling 'HEARTBEAT" protocol message due to javax.net.sslException:Read timed out. Before migration there was no issue in cluster and ckuster are connected. Just bought flow.xml.gz ,user.xml & authorization.xml from other 1.8 server. Thanks & Regards Sanjeet On Thu, 30 Jul 2020, 7:37 pm sanjeet rath, wrote: > nifi-app.log file , i am getting constant error. > > Apache.nifi.controler.FlowControler Failed to send heart beat due to > org.apache.nifi.protocal.protocolException > > On Thu, 30 Jul 2020, 6:54 pm Mark Payne, wrote: > >> Sanjeet, >> >> What error are you receiving? >> >> Sent from my iPhone >> >> On Jul 30, 2020, at 9:21 AM, sanjeet rath wrote: >> >> >> Hi Team, >> >> Any help on my trailed mail query will be highly appreciated.still i am >> strgling yo fix yhe issue. >> >> In nifi-app.log file , i am getting constant error. >> >> Apache.nifi.controler.FlowControler Failed to send heart beat due to >> org.apache.nifi.protocal.protocolEcslception. >> >> Thanks & Regards, >> Sanjeet >> >> On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, >> wrote: >> >>> Hi Team, >>> >>> I am facing a wired issue while doing migration from 1.8 to 1.11.4 >>> I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi >>> cluster(1.8 version) to a newly created 1.11.4 version cluster. >>> >>> When i am starting the 1.11.4 cluster with this new files, i am getting >>> node disconnected error (this node is not conncted to cluster pop up ) >>> >>> However with only users.xml, authrization.xml from 1.8 and default >>> 1.11.4 flow.xml.gz. i am getting no error all 3 nodes are getting conncted >>> in the new cluster. >>> Infact when i am bringing another 1.8 env's flow.xml.gz files it working >>> fine. >>> >>> So specific for this flow.xml.gz file only my nodes are getting >>> disconncted. >>> >>> It will really helpfull if you can help where i need to check to >>> resolve this issue. >>> >>> I am using external zookeeper 3.5 version >>> >>> Thanks in advance, >>> Sanjeet >>> >>>
Re: Nifi 1.8 to 1.11.4 migration issue
nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolException On Thu, 30 Jul 2020, 6:54 pm Mark Payne, wrote: > Sanjeet, > > What error are you receiving? > > Sent from my iPhone > > On Jul 30, 2020, at 9:21 AM, sanjeet rath wrote: > > > Hi Team, > > Any help on my trailed mail query will be highly appreciated.still i am > strgling yo fix yhe issue. > > In nifi-app.log file , i am getting constant error. > > Apache.nifi.controler.FlowControler Failed to send heart beat due to > org.apache.nifi.protocal.protocolEcslception. > > Thanks & Regards, > Sanjeet > > On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, > wrote: > >> Hi Team, >> >> I am facing a wired issue while doing migration from 1.8 to 1.11.4 >> I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi >> cluster(1.8 version) to a newly created 1.11.4 version cluster. >> >> When i am starting the 1.11.4 cluster with this new files, i am getting >> node disconnected error (this node is not conncted to cluster pop up ) >> >> However with only users.xml, authrization.xml from 1.8 and default 1.11.4 >> flow.xml.gz. i am getting no error all 3 nodes are getting conncted in the >> new cluster. >> Infact when i am bringing another 1.8 env's flow.xml.gz files it working >> fine. >> >> So specific for this flow.xml.gz file only my nodes are getting >> disconncted. >> >> It will really helpfull if you can help where i need to check to >> resolve this issue. >> >> I am using external zookeeper 3.5 version >> >> Thanks in advance, >> Sanjeet >> >>
Re: Nifi 1.8 to 1.11.4 migration issue
Sanjeet, What error are you receiving? Sent from my iPhone On Jul 30, 2020, at 9:21 AM, sanjeet rath wrote: Hi Team, Any help on my trailed mail query will be highly appreciated.still i am strgling yo fix yhe issue. In nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolEcslception. Thanks & Regards, Sanjeet On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, mailto:rath.sanj...@gmail.com>> wrote: Hi Team, I am facing a wired issue while doing migration from 1.8 to 1.11.4 I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi cluster(1.8 version) to a newly created 1.11.4 version cluster. When i am starting the 1.11.4 cluster with this new files, i am getting node disconnected error (this node is not conncted to cluster pop up ) However with only users.xml, authrization.xml from 1.8 and default 1.11.4 flow.xml.gz. i am getting no error all 3 nodes are getting conncted in the new cluster. Infact when i am bringing another 1.8 env's flow.xml.gz files it working fine. So specific for this flow.xml.gz file only my nodes are getting disconncted. It will really helpfull if you can help where i need to check to resolve this issue. I am using external zookeeper 3.5 version Thanks in advance, Sanjeet
Re: Nifi 1.8 to 1.11.4 migration issue
Hi Team, Any help on my trailed mail query will be highly appreciated.still i am strgling yo fix yhe issue. In nifi-app.log file , i am getting constant error. Apache.nifi.controler.FlowControler Failed to send heart beat due to org.apache.nifi.protocal.protocolEcslception. Thanks & Regards, Sanjeet On Tue, 28 Jul 2020, 10:21 pm sanjeet rath, wrote: > Hi Team, > > I am facing a wired issue while doing migration from 1.8 to 1.11.4 > I am bringing flow.xml.gz,user.xml,authorization.xml from a nifi > cluster(1.8 version) to a newly created 1.11.4 version cluster. > > When i am starting the 1.11.4 cluster with this new files, i am getting > node disconnected error (this node is not conncted to cluster pop up ) > > However with only users.xml, authrization.xml from 1.8 and default 1.11.4 > flow.xml.gz. i am getting no error all 3 nodes are getting conncted in the > new cluster. > Infact when i am bringing another 1.8 env's flow.xml.gz files it working > fine. > > So specific for this flow.xml.gz file only my nodes are getting > disconncted. > > It will really helpfull if you can help where i need to check to > resolve this issue. > > I am using external zookeeper 3.5 version > > Thanks in advance, > Sanjeet > >
