[ https://issues.apache.org/jira/browse/NIFI-8339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315311#comment-17315311 ]
Rene Weidlinger commented on NIFI-8339: --------------------------------------- [~markap14] we changed to RAW and it seems like its working now. Why is RAW the preferred option? We used http, because we thought that RAW isn't encrypted via TLS? It's possible that the dos prevention of jetty is the cause. > Input Threads get Interrupted and stuck indefinitely > ---------------------------------------------------- > > Key: NIFI-8339 > URL: https://issues.apache.org/jira/browse/NIFI-8339 > Project: Apache NiFi > Issue Type: Bug > Affects Versions: 1.13.0 > Reporter: Rene Weidlinger > Priority: Major > Attachments: firefox_Yf6NUeQe5X.png, nifi-app.log, nifi.properties, > td1.txt > > > After some seconds we see this stack trace in nifi on one of our inputs: > {noformat} > 2021-03-18 07:33:34,703 ERROR [NiFi Web Server-18] > o.a.nifi.web.api.ApplicationResource Unexpected exception occurred. > portId=c4d93fb6-5e5b-1382-b39b-66fbc04660f0 > 2021-03-18 07:33:34,703 ERROR [NiFi Web Server-18] > o.a.nifi.web.api.ApplicationResource Exception detail: > org.apache.nifi.processor.exception.ProcessException: > org.apache.nifi.processor.exception.ProcessException: Interrupted while > waiting for site-to-site request to be serviced > at > org.apache.nifi.remote.StandardPublicPort.receiveFlowFiles(StandardPublicPort.java:588) > at > org.apache.nifi.web.api.DataTransferResource.receiveFlowFiles(DataTransferResource.java:277) > at sun.reflect.GeneratedMethodAccessor198.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) > at > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) > at > org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) > at org.glassfish.jersey.internal.Errors.process(Errors.java:316) > at org.glassfish.jersey.internal.Errors.process(Errors.java:298) > at org.glassfish.jersey.internal.Errors.process(Errors.java:268) > at > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) > at > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) > at > org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) > at > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) > at > org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) > at > org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1452) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) > at > org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) > at > org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:61) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:61) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:61) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:61) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100) > at > org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:96) > at > org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:51) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:46) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:487) > at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:336) > at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:301) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.security.headers.StrictTransportSecurityFilter.doFilter(StrictTransportSecurityFilter.java:48) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.security.headers.XContentTypeOptionsFilter.doFilter(XContentTypeOptionsFilter.java:48) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.security.headers.XSSProtectionFilter.doFilter(XSSProtectionFilter.java:48) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.security.headers.ContentSecurityPolicyFilter.doFilter(ContentSecurityPolicyFilter.java:47) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.apache.nifi.web.security.headers.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:48) > at > org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) > at > org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) > at > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > at > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:766) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) > at > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > at org.eclipse.jetty.server.Server.handle(Server.java:516) > at > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > at > org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) > at > org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) > at > org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) > at > org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) > at > org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > at > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > at > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.nifi.processor.exception.ProcessException: Interrupted > while waiting for site-to-site request to be serviced > at > org.apache.nifi.remote.StandardPublicPort.receiveFlowFiles(StandardPublicPort.java:561) > ... 123 common frames omitted > Caused by: java.lang.InterruptedException: null > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:460) > at java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:348) > at > org.apache.nifi.remote.StandardPublicPort$FlowFileRequest.waitForService(StandardPublicPort.java:490) > at > org.apache.nifi.remote.StandardPublicPort.receiveFlowFiles(StandardPublicPort.java:552) > ... 123 common frames omitted{noformat} > The Input threads stuck until we restart nifi. Otherwise we cant receive any > data over this input anymore. > Additional Info: We send Logs and stuff with aprox. 80 Minifis (Java) to > Nifi. The payloads are between some bytes and ~ 10 MB. > I think its related to this issue, because before 1.13 we didnt face this > problem. > https://issues.apache.org/jira/browse/NIFI-7566 -- This message was sent by Atlassian Jira (v8.3.4#803005)