Hello,
I'm facing a weird behavior on the Camel-HTTP component (4.2.2) when I want
to use the "disableStreamCache=true" configuration specified in this page :
https://camel.apache.org/components/4.4.x/http-component.html.
When this option is set to true, the stream is consumed first by the Apache
HTTP Client, and then, when Camel tries to convert the InputStream to
CacheOutputStream object by copying the data, it raises a
"StreamAlreadyClosed" exception.
I even tried to get the body in a processor, to convert it into another
object like String.class etc... So, I tried to go deeply in the code to
figure out this error. Please find below my analysis.
When I make an HTTP call, the execution goes through the
org.apache.hc.client5.http.impl.classic.CloseableHttpClient#execute(HttpHost,
ClassicHttpRequest, HttpContext, HttpClientResponseHandler) method. Here,
it calls the handler defined by Camel in
org.apache.camel.component.http.HttpProducer#process(Exchange) - line 248.
In this handler, because I set up the option "disableStreamCache=true" the
handler sets the exchange body as the InputStream provided by the HTTP
Client. So, when it goes back to the execute method, the Apache HTTP Client
consumes the stream (i.e ; he closes it).
This is where the problem starts. In the exchange, I have a closed stream
that can't be used inside my route. For example, if I try to log the body,
it raises the exception pasted below.
I also tried the run my app with the option :
"camel.springboot.stream-caching-enabled: false" in my application.yml but
the bug is caused before this option is valuated (in the HTTP Client) so,
it's not effective.
I don't really know if it's me who does not fully understand this option or
if it's a bug. I think, the HTTP Client expects the user to read the
response inside the handler given in the execute method, then, it can
safely close the InputStream. But, in the Camel case, the handler just put
the InputStream reference inside the exchange. Maybe, when the option
disableStreamCache is set to true, the final user of Camel should be able
to give his own handler's implementation in order to consume it in the way
we need?
Thanks in advance for your inputs.
Regards,
Steps to reproduce :
I noticed this bug by using Camel spring boot 4.2.2 but I think this bug is
reproducible without Spring Boot.
Here find my dependencies in my pom.xml :
<properties>
<camel.version>4.4.2</camel.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-http-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</dependency>
</dependencies>
Here my Main class :
@SpringBootApplication
public class Main {
public static void main(String[] args) {
SpringApplication.run(Main.class, args);
}
}
and found my route here :
@Component
public class MyRoute extends RouteBuilder {
@Override
public void configure() throws Exception {
getContext().getTypeConverterRegistry();
//@formatter:off
from("timer:test?repeatCount=1")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.setHeader(Exchange.HTTP_URI, constant("
https://camel.apache.org/components/4.4.x/http-component.html";))
.to("http:myHttpRequest?disableStreamCache=true")
.log("body : ${body}")
.end();
//@formatter:on
}
}
Find here the log I got after running this project :
org.apache.camel.StreamCacheException: Error during type conversion from
type: org.apache.hc.client5.http.entity.LazyDecompressingInputStream to the
required type: org.apache.camel.StreamCache with value
org.apache.hc.client5.http.entity.LazyDecompressingInputStream@2e74480e due
to org.apache.camel.TypeConversionException: Error during type conversion
from type: org.apache.hc.client5.http.entity.LazyDecompressingInputStream
to the required type: org.apache.camel.StreamCache with value
org.apache.hc.client5.http.entity.LazyDecompressingInputStream@2e74480e due
to java.io.IOException: Attempted read on closed stream.
at
org.apache.camel.impl.engine.StreamCachingHelper.handleException(StreamCachingHelper.java:86)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.StreamCachingHelper.handleException(StreamCachingHelper.java:81)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.StreamCachingHelper.tryStreamCache(StreamCachingHelper.java:74)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.StreamCachingHelper.convertToStreamCache(StreamCachingHelper.java:55)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.CamelInternalProcessor$StreamCachingAdvice.before(CamelInternalProcessor.java:1009)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.CamelInternalProcessor$StreamCachingAdvice.before(CamelInternalProcessor.java:999)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:317)
~[camel-base-engine-4.4.2.jar:4.4.2]
at org.apache.camel.processor.Pipeline$PipelineTask.run(Pipeline.java:102)
~[camel-core-processor-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.doRun(DefaultReactiveExecutor.java:199)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.executeReactiveWork(DefaultReactiveExecutor.java:189)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.tryExecuteReactiveWork(DefaultReactiveExecutor.java:166)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59)
~[camel-base-engine-4.4.2.jar:4.4.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:163)
~[camel-core-processor-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.CamelInternalProcessor.processNonTransacted(CamelInternalProcessor.java:354)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:330)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:293)
~[camel-timer-4.4.2.jar:4.4.2]
at
org.apache.camel.component.timer.TimerConsumer$1.doRun(TimerConsumer.java:164)
~[camel-timer-4.4.2.jar:4.4.2]
at
org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:136)
~[camel-timer-4.4.2.jar:4.4.2]
at java.base/java.util.TimerThread.mainLoop(Timer.java:566) ~[na:na]
at java.base/java.util.TimerThread.run(Timer.java:516) ~[na:na]
Caused by: org.apache.camel.TypeConversionException: Error during type
conversion from type:
org.apache.hc.client5.http.entity.LazyDecompressingInputStream to the
required type: org.apache.camel.StreamCache with value
org.apache.hc.client5.http.entity.LazyDecompressingInputStream@2e74480e due
to java.io.IOException: Attempted read on closed stream.
at
org.apache.camel.converter.stream.StreamCacheBulkConverterLoader.convertTo(StreamCacheBulkConverterLoader.java:62)
~[camel-support-4.4.2.jar:4.4.2]
at
org.apache.camel.spi.BulkTypeConverters.convertTo(BulkTypeConverters.java:122)
~[camel-api-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.tryCachedConverters(CoreTypeConverterRegistry.java:419)
~[camel-base-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.doConvertTo(CoreTypeConverterRegistry.java:378)
~[camel-base-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.doConvertToAndStat(CoreTypeConverterRegistry.java:272)
~[camel-base-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.converter.CoreTypeConverterRegistry.convertTo(CoreTypeConverterRegistry.java:167)
~[camel-base-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultStreamCachingStrategy.doCache(DefaultStreamCachingStrategy.java:275)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.DefaultStreamCachingStrategy.cache(DefaultStreamCachingStrategy.java:252)
~[camel-base-engine-4.4.2.jar:4.4.2]
at
org.apache.camel.impl.engine.StreamCachingHelper.tryStreamCache(StreamCachingHelper.java:68)
~[camel-base-engine-4.4.2.jar:4.4.2]
... 18 common frames omitted
Caused by: java.io.IOException: Attempted read on closed stream.
at
org.apache.hc.core5.http.io.EofSensorInputStream.isReadAllowed(EofSensorInputStream.java:107)
~[httpcore5-5.2.4.jar:5.2.4]
at
org.apache.hc.core5.http.io.EofSensorInputStream.read(EofSensorInputStream.java:116)
~[httpcore5-5.2.4.jar:5.2.4]
at
java.base/java.util.zip.CheckedInputStream.read(CheckedInputStream.java:59)
~[na:na]
at
java.base/java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:266)
~[na:na]
at
java.base/java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)
~[na:na]
at
java.base/java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)
~[na:na]
at java.base/java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
~[na:na]
at java.base/java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
~[na:na]
at
org.apache.hc.client5.http.entity.GZIPInputStreamFactory.create(GZIPInputStreamFactory.java:61)
~[httpclient5-5.2.3.jar:5.2.3]
at
org.apache.hc.client5.http.entity.LazyDecompressingInputStream.initWrapper(LazyDecompressingInputStream.java:51)
~[httpclient5-5.2.3.jar:5.2.3]
at
org.apache.hc.client5.http.entity.LazyDecompressingInputStream.available(LazyDecompressingInputStream.java:86)
~[httpclient5-5.2.3.jar:5.2.3]
at org.apache.camel.util.IOHelper.copy(IOHelper.java:178)
~[camel-util-4.4.2.jar:4.4.2]
at org.apache.camel.util.IOHelper.copy(IOHelper.java:164)
~[camel-util-4.4.2.jar:4.4.2]
at org.apache.camel.util.IOHelper.copy(IOHelper.java:159)
~[camel-util-4.4.2.jar:4.4.2]
at org.apache.camel.util.IOHelper.copyAndCloseInput(IOHelper.java:232)
~[camel-util-4.4.2.jar:4.4.2]
at org.apache.camel.util.IOHelper.copyAndCloseInput(IOHelper.java:228)
~[camel-util-4.4.2.jar:4.4.2]
at
org.apache.camel.converter.stream.StreamCacheConverter.convertToStreamCache(StreamCacheConverter.java:54)
~[camel-support-4.4.2.jar:4.4.2]
at
org.apache.camel.converter.stream.StreamCacheBulkConverterLoader.doConvertTo(StreamCacheBulkConverterLoader.java:80)
~[camel-support-4.4.2.jar:4.4.2]
at
org.apache.camel.converter.stream.StreamCacheBulkConverterLoader.convertTo(StreamCacheBulkConverterLoader.java:53)
~[camel-support-4.4.2.jar:4.4.2]
... 26 common frames omitted
