[
https://issues.apache.org/jira/browse/CAMEL-20152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790617#comment-17790617
]
Nicolas Filotto edited comment on CAMEL-20152 at 11/29/23 10:35 AM:
--------------------------------------------------------------------
[~acosentino] I don't know what you think of the fix proposal but I think that
it could be backported into the 3.x branch. For 3.20 and 3.21, what I can
propose as a fix is to simply set {{fileSizeThreshold}} to a different default
value, high enough to limit the regressions but not too high to avoid OOME,
something like 10 MB, WDYT?
was (Author: JIRAUSER285918):
[~acosentino] I don't know what you think of the fix proposal but I think that
it could be backported into the 3.x branch. For 3.20 and 3.21, what I can
propose as a fix is to simply set {{fileSizeThreshold}} to a different default
value, high enough to limit the regressions but not too high to avoid OOME,
something like 10 Mbs, WDYT?
> camel-jetty - OutOfMemoryError with big file upload via multipart
> -----------------------------------------------------------------
>
> Key: CAMEL-20152
> URL: https://issues.apache.org/jira/browse/CAMEL-20152
> Project: Camel
> Issue Type: Bug
> Components: camel-jetty
> Affects Versions: 3.21.2
> Reporter: michael elbaz
> Assignee: Nicolas Filotto
> Priority: Major
> Attachments: image-2023-11-28-09-12-11-951.png
>
>
> I don't now if is related to jetty (maybe) but when i use camel with jetty
> and i upload a big file using multipart i get:
> {code:java}
> // espace r23:28:50.262 WARN [qtp1616084922-291] /email/myservice
> java.lang.IllegalStateException: java.lang.OutOfMemoryError: Java heap space
> at
> org.eclipse.jetty.http.MultiPartFormInputStream.throwIfError(MultiPartFormInputStream.java:517)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.http.MultiPartFormInputStream.getParts(MultiPartFormInputStream.java:470)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.MultiParts$MultiPartsHttpParser.getParts(MultiParts.java:74)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.Request.getParts(Request.java:2450)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.Request.getParts(Request.java:2420)
> ~[!/:9.4.52.v20230823]
> at
> org.apache.camel.component.jetty.MultiPartFilter.doFilter(MultiPartFilter.java:52)
> ~[?:?]
> at
> org.apache.camel.component.jetty.CamelFilterWrapper.doFilter(CamelFilterWrapper.java:49)
> ~[?:?]
> at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[?:?]
> at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
> ~[?:?]
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
> ~[?:?]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
> ~[?:?]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.Server.handle(Server.java:516)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
> [!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
> [!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> [!/:9.4.52.v20230823]
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> [!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
> [!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
> [!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
> [!/:9.4.52.v20230823]
> at java.lang.Thread.run(Thread.java:833) [?:?]
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:3537) ~[?:?]
> at
> java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:100)
> ~[?:?]
> at
> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:130) ~[?:?]
> at
> org.eclipse.jetty.http.MultiPartFormInputStream$MultiPart.write(MultiPartFormInputStream.java:178)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.http.MultiPartFormInputStream$Handler.content(MultiPartFormInputStream.java:741)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.http.MultiPartParser.parseOctetContent(MultiPartParser.java:645)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.http.MultiPartParser.parse(MultiPartParser.java:214)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.http.MultiPartFormInputStream.parse(MultiPartFormInputStream.java:587)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.http.MultiPartFormInputStream.getParts(MultiPartFormInputStream.java:469)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.MultiParts$MultiPartsHttpParser.getParts(MultiParts.java:74)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.Request.getParts(Request.java:2450)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.Request.getParts(Request.java:2420)
> ~[!/:9.4.52.v20230823]
> at
> org.apache.camel.component.jetty.MultiPartFilter.doFilter(MultiPartFilter.java:52)
> ~[?:?]
> at
> org.apache.camel.component.jetty.CamelFilterWrapper.doFilter(CamelFilterWrapper.java:49)
> ~[?:?]
> at
> org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[?:?]
> at
> org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
> ~[?:?]
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
> ~[?:?]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
> ~[?:?]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.Server.handle(Server.java:516)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.HttpChannel$$Lambda$828/0x00000007c0935678.dispatch(Unknown
> Source) ~[?:?]
> at
> org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
> ~[!/:9.4.52.v20230823]
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
> ~[!/:9.4.52.v20230823]
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
> ~[?:?]
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> ~[?:?]éservé du code
> {code}
> Here the simple route:
> {code:java}
> public class InputHttpRoute extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from(http())
> .noStreamCaching()
> .process(e -> {
> e.getIn(AttachmentMessage.class).getAttachments()
> .forEach((s, dataHandler) -> {
> InputStream files = null;
> try {
> files = dataHandler.getInputStream();
> byte[] buffer = new byte[10500];
> int b;
> while ((b = files.read(buffer)) != -1) {
> // Convert byte to character
> String data = new String(buffer, 0,
> b);
> System.err.println(data);
> }
> } catch (IOException ex) {
> throw new RuntimeException(ex);
> }
> });;
> }
> private static String http() {
> return new StringBuilder("jetty:http://0.0.0.0:8787/myservice";)
> .append("?httpMethodRestrict=")
> .append("POST")
> .toString();
> }
> }// espace réservé du code
> {code}
>
> The file is fully loaded in memory before i can access it this is a unwanted
> behavior and it will be nice if it will be possible to start processing the
> file during the upload in streaming fashion mode
>
> More about this =>
> [https://camel.zulipchat.com/#narrow/stream/257295-general/topic/camel-jetty.20Large.20file.20upload.20streaming.20mode]
> !image-2023-11-28-09-12-11-951.png|width=770,height=722!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
