[jira] [Commented] (NIFI-7912) Site-to-Site over HTTP fails for FlowFiles over 100MB
[
https://issues.apache.org/jira/browse/NIFI-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17316292#comment-17316292
]
ASF subversion and git services commented on NIFI-7912:
---
Commit 9da3b1ec01555a5f36047f500b784edffd2f80b3 in nifi's branch
refs/heads/main from Nathan Gough
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9da3b1e ]
NIFI-7912 - Added properties to configure DoSFilter timeout and whitelisted
addresses
- Added nifi.web.request.ip.whitelist property to set DoSFilter.ipWhitelist
- Added nifi.web.request.timeout property to set DoSFilter.maxRequestMs with
default of 60 seconds
This closes #4972
Signed-off-by: David Handermann
> Site-to-Site over HTTP fails for FlowFiles over 100MB
> -
>
> Key: NIFI-7912
> URL: https://issues.apache.org/jira/browse/NIFI-7912
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.12.1
>Reporter: Pierre Villard
>Assignee: Nathan Gough
>Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When using Site-to-Site over HTTP and trying to send FlowFiles larger than
> 100MB, it fails with the following errors.
> {noformat}
> 2020-10-12 16:25:23,132 ERROR [NiFi Web Server-31]
> o.g.j.server.ServerRuntime$Responder An I/O error has occurred while writing
> a response message entity to the container output stream.
> org.glassfish.jersey.server.internal.process.MappableException:
> org.apache.nifi.processor.exception.ProcessException:
> java.lang.InterruptedException
> at
> org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:91)
> at
> org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:163)
> at
> org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1135)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:662)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:395)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:385)
> at
> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:280)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
> at
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
> at
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
> at
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
> at
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
> at
> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
> at
> org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617)
> at
> org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Commented] (NIFI-7912) Site-to-Site over HTTP fails for FlowFiles over 100MB
[
https://issues.apache.org/jira/browse/NIFI-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17311602#comment-17311602
]
Nathan Gough commented on NIFI-7912:
It appears this issue may be related to the Denial of Service Jetty filter
which is limiting connections to 30 seconds. If the data being sent takes
longer than this period, the session will be terminated by the filter. I will
investigate using a NiFi property to configure the maximum timeout for the DoS
filter.
> Site-to-Site over HTTP fails for FlowFiles over 100MB
> -
>
> Key: NIFI-7912
> URL: https://issues.apache.org/jira/browse/NIFI-7912
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.12.1
>Reporter: Pierre Villard
>Priority: Major
>
> When using Site-to-Site over HTTP and trying to send FlowFiles larger than
> 100MB, it fails with the following errors.
> {noformat}
> 2020-10-12 16:25:23,132 ERROR [NiFi Web Server-31]
> o.g.j.server.ServerRuntime$Responder An I/O error has occurred while writing
> a response message entity to the container output stream.
> org.glassfish.jersey.server.internal.process.MappableException:
> org.apache.nifi.processor.exception.ProcessException:
> java.lang.InterruptedException
> at
> org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:91)
> at
> org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:163)
> at
> org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1135)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:662)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:395)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:385)
> at
> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:280)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
> at
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
> at
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
> at
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
> at
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
> at
> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
> at
> org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617)
> at
> org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.apache.nifi.web.security.NiFiAuthenticationFilter.authenticate(NiFiAuthenticationFilter.java:100)
> at
> org.apache.nifi.web.security.NiFiAuthenticationFilter.doFilter(NiFiAuthenticationFilter.java:59)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
>
[jira] [Commented] (NIFI-7912) Site-to-Site over HTTP fails for FlowFiles over 100MB
[
https://issues.apache.org/jira/browse/NIFI-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212466#comment-17212466
]
Pierre Villard commented on NIFI-7912:
--
I confirmed the issue on the following Java versions (to make sure this is no
related to recently reported JDK regressions):
$ java -version
openjdk version "1.8.0_265"
OpenJDK Runtime Environment (Zulu 8.48.0.53-CA-macosx) (build 1.8.0_265-b11)
OpenJDK 64-Bit Server VM (Zulu 8.48.0.53-CA-macosx) (build 25.265-b11, mixed
mode)
$ java -version
java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 1.8.0_231-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode)
$ java -version
openjdk version "11.0.8" 2020-07-14 LTS
OpenJDK Runtime Environment Zulu11.41+23-CA (build 11.0.8+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.41+23-CA (build 11.0.8+10-LTS, mixed mode)
> Site-to-Site over HTTP fails for FlowFiles over 100MB
> -
>
> Key: NIFI-7912
> URL: https://issues.apache.org/jira/browse/NIFI-7912
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.12.1
>Reporter: Pierre Villard
>Priority: Major
>
> When using Site-to-Site over HTTP and trying to send FlowFiles larger than
> 100MB, it fails with the following errors.
> {noformat}
> 2020-10-12 16:25:23,132 ERROR [NiFi Web Server-31]
> o.g.j.server.ServerRuntime$Responder An I/O error has occurred while writing
> a response message entity to the container output stream.
> org.glassfish.jersey.server.internal.process.MappableException:
> org.apache.nifi.processor.exception.ProcessException:
> java.lang.InterruptedException
> at
> org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:91)
> at
> org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:163)
> at
> org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1135)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:662)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:395)
> at
> org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:385)
> at
> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:280)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
> at
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
> at
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
> at
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
> at
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
> at
> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
> at
> org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1395)
> at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1617)
> at
> org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
