[
https://issues.apache.org/jira/browse/NIFI-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062056#comment-17062056
]
Manuel Loayza commented on NIFI-7266:
-------------------------------------
[~joewitt]
Thanks a lot for checking on that.
I see a lot of these exceptions:
{code:java}
2020-03-18 21:09:20,729 WARN [Process Cluster Protocol Request-20]
o.a.n.c.p.impl.SocketProtocolListener Failed processing protocol message from
ip-207-50-126.dqa.capitalone.com due to
org.apache.nifi.cluster.protocol.ProtocolException:
java.security.cert.CertificateException:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
org.apache.nifi.cluster.protocol.ProtocolException:
java.security.cert.CertificateException:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.getRequestorDN(SocketProtocolListener.java:221)
at
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:133)
at org.apache.nifi.io.socket.SocketListener$2$1.run(SocketListener.java:136)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.security.cert.CertificateException:
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at
org.apache.nifi.security.util.CertificateUtils.extractPeerDNFromClientSSLSocket(CertificateUtils.java:314)
at
org.apache.nifi.security.util.CertificateUtils.extractPeerDNFromSSLSocket(CertificateUtils.java:269)
at
org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.getRequestorDN(SocketProtocolListener.java:219)
... 5 common frames omitted
Caused by: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:450)
at
org.apache.nifi.security.util.CertificateUtils.extractPeerDNFromClientSSLSocket(CertificateUtils.java:299)
... 7 common frames omitted{code}
And a post in stackoverflow says something that we already have
[https://stackoverflow.com/questions/43151807/secured-nifi-cluster-setup]
{{}}
{{}}
{code:java}
nifi.security.needClientAuth=true
nifi.cluster.protocol.is.secure=true {code}
That is curious, because the cluster seems to recover the 1st time after 15
minutes, and the 2nd time it recovered after 45 minutes.
> NIFI 1.4.0 gets unresponsive after heavy load
> ---------------------------------------------
>
> Key: NIFI-7266
> URL: https://issues.apache.org/jira/browse/NIFI-7266
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration
> Affects Versions: 1.2.0, 1.3.0, 1.4.0
> Reporter: Manuel Loayza
> Priority: Trivial
> Attachments: NIFI_PERF_LOG_1.log, Screen Shot 2020-03-17 at 3.18.27
> PM.png, image-2020-03-18-16-03-49-351.png, image-2020-03-18-16-35-49-452.png
>
>
> We have 2 clusters (6 instances each one) running with NIFI 1.1.2 + JDK 8u121
> + Linux CentOS
> The traffic get divided between those 2 clusters:
> 1. TPS: 2700 - EAST cluster
> 2. TPS: 980. - WEST cluster
> We have tried to migrate to NIFI 1.2.0, 1.3.0, and 1.4.0, but the cluster
> with higher TPS (EAST) got stuck after 4 hours of intensive traffic. Also it
> web console got unresponsive.
> I've tried many things to fix this thing, but only thing I got was to
> increase the time from 4 to 6 hours before it fails
> Our current instances are running on AWS and each EC2 instances has 8 cpus
> (c5.2xlarge), and 16GB RAM.
> I've tried to use c5.4xlarge (it doubles the cpu and ram), but I got the
> same outcome.
> I don't have a clue to figure it out what the issue is. Also I have a
> datadog dashboard to track some java head metrics but everything looks normal.
> What should I do to find why those new better instances are failing? is it
> memory or disk space or threads got stuck? Why an old NIFI cluster conf
> works better than a new NIFI?
> Hope you can help me with this.
> Thanks
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
