Hi Mark, Thanks for the response. My flow.xml.gz file size is 14 Mb.
Regards, Sanjeet On Thu, 13 Aug 2020, 10:27 pm Mark Payne, <marka...@hotmail.com> wrote: > Actually, I take back what I said. I was a little too quick to jump to > conclusions about what the issue was. There was an issue addressed that > should improve startup time. But what you’re seeing here is unrelated, as > you’re enchanting OutOfMemoryError. The long time is likely related to > garbage collection. How large is your flow.xml.gz file? > > Thanks > -Mark > > > On Aug 13, 2020, at 12:48 PM, Mark Payne <marka...@hotmail.com> wrote: > > Sanjeet, > > I believe this should be addressed in 1.12.0, which should be released > very soon. > > Thanks > -Mark > > On Aug 13, 2020, at 4:13 AM, sanjeet rath <rath.sanj...@gmail.com> wrote: > > Hi Team, > > I have migrated my flows.xml.gz, users.xml, authorization.xml from 1.8 env > to 1.11.4 environment .(12k processor are therr in flow) > There are no error in the log file, the issue is taking 10 mintues to > server up and with all node connected. > > The warning logs , which i am suspecting is causing the delay are > mentioned below > -> I have set max & min jvm to 8gb > ->nifi.cluster.node.connection.timeout=30 sec > >nifi.cluster.node.read timeout=30 sec > ->nifi-zookeeper.connect.timeout=15 sec > ->nifi-zookeeper.session.timeout=15 sec > > When i am making the defaut value 5 & 3 sec respectively, it taking > much longer time to node connected. > > Could you please help me to identify the issue, why it taking so long 10 > minutes time to up the nifi cluster.Thanks In advance. > > 1st suspect warning:(appearing 6 to 7 times in log and dissapears one > server is up with nodes are connected) > > WARN [Process Cluster Protocol Request-10] > o.a.n.c.p.impl.SocketProtocolListener Failed processing protocol message from > “**HOSTIP address**”com due to > org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling > protocol message in response to message type: CONNECTION_REQUEST due to > javax.net.ssl.SSLException: Broken pipe (Write failed) > org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling > protocol message in response to message type: CONNECTION_REQUEST due to > javax.net.ssl.SSLException: Broken pipe (Write failed) > at > org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:184) > at > org.apache.nifi.io.socket.SocketListener$2$1.run(SocketListener.java:136) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > Caused by: javax.net.ssl.SSLException: Broken pipe (Write failed) > at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127) > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326) > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:269) > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264) > at > java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:980) > at java.base/java.io.DataOutputStream.write(DataOutputStream.java:107) > at > java.base/java.io.FilterOutputStream.write(FilterOutputStream.java:108) > at > org.apache.nifi.cluster.protocol.jaxb.JaxbProtocolContext$1.marshal(JaxbProtocolContext.java:86) > at > org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:182) > ... 4 common frames omitted > Suppressed: java.net.SocketException: Broken pipe (Write failed) > at java.base/java.net.SocketOutputStream.socketWrite0(Native > Method) > at > java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) > at > java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150) > at > java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:357) > ... 11 common frames omitted > Caused by: java.net.SocketException: Broken pipe (Write failed) > at java.base/java.net.SocketOutputStream.socketWrite0(Native Method) > at > java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110) > at > java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150) > at > java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:319) > at > java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:975) > ... 8 common frames omitted > 2020-08-13 00:23:51,002 WARN [Process Cluster Protocol Request-6] > org.apache.nifi.io.socket.SocketListener Dispatching socket request > encountered exception due to: java.lang.OutOfMemoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space > > 2nd suspecting: > > WARN [Process Cluster Protocol Request-6] > org.apache.nifi.io.socket.SocketListener Dispatching socket request > encountered exception due to: java.lang.OutOfMemoryError: Java heap space > java.lang.OutOfMemoryError: Java heap space at > java.base/java.util.Arrays.copyOf(Arrays.java:3745) at > java.base/java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120) > at > java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95) > at > java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:156) > at > com.sun.xml.bind.v2.runtime.output.UTF8XmlOutput.flushBuffer(UTF8XmlOutput.java:418) > at > com.sun.xml.bind.v2.runtime.output.UTF8XmlOutput.text(UTF8XmlOutput.java:371) > at > com.sun.xml.bind.v2.runtime.unmarshaller.Base64Data.writeTo(Base64Data.java:314) > at > com.sun.xml.bind.v2.runtime.output.UTF8XmlOutput.text(UTF8XmlOutput.java:312) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.leafElement(XMLSerializer.java:356) > at > com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$PcdataImpl.writeLeafElement(RuntimeBuiltinLeafInfoImpl.java:191) > at > com.sun.xml.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.writeLeafElement(TransducedAccessor.java:254) > at > com.sun.xml.bind.v2.runtime.property.SingleElementLeafProperty.serializeBody(SingleElementLeafProperty.java:130) > at > com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:360) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:696) > at > com.sun.xml.bind.v2.runtime.property.SingleElementNodeProperty.serializeBody(SingleElementNodeProperty.java:158) > at > com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:360) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:696) > at > com.sun.xml.bind.v2.runtime.property.SingleElementNodeProperty.serializeBody(SingleElementNodeProperty.java:158) > at > com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:360) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.childAsSoleContent(XMLSerializer.java:593) > at > com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeRoot(ClassBeanInfoImpl.java:341) > at > com.sun.xml.bind.v2.runtime.XMLSerializer.childAsRoot(XMLSerializer.java:494) > at > com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:328) > at > com.sun.xml.bind.v2.runtime.MarshallerImpl.marshal(MarshallerImpl.java:256) > at > javax.xml.bind.helpers.AbstractMarshallerImpl.marshal(AbstractMarshallerImpl.java:110) > at > org.apache.nifi.cluster.protocol.jaxb.JaxbProtocolContext$1.marshal(JaxbProtocolContext.java:75) > at > org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:182) > at > org.apache.nifi.io.socket.SocketListener$2$1.run(SocketListener.java:136) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > -- > Sanjeet Kumar Rath, > mob- +91 8777577470 > > > >
