Hi,
While upgrading Camel from our current verion (3.4.2) to the latest version
(3.18.2) we are running into an issue where the resilience4j circuit breaker is
not working as expected. After the circuit is OPEN it should throw an
exception(CallNotPermittedException) same as 3.4.2 but instead the execution
continues without throwing any exception.
Upon further debugging, we could see the difference in execution logs which are
attached below.
logs from 3.4.2 camel version -
2022-09-13T10:28:59,162 DEBUG [http-nio-8086-exec-8] o.a.c.s.CamelLogger:
Request is routed to service2
2022-09-13T10:28:59,163 DEBUG [ForkJoinPool.commonPool-worker-1]
i.g.r.c.i.CircuitBreakerStateMachine: No Consumers: Event NOT_PERMITTED not
published
2022-09-13T10:28:59,163 DEBUG [http-nio-8086-exec-8] o.a.c.p.PipelineHelper:
Message exchange has failed: so breaking out of pipeline for exchange:
Exchange[ID-WKSWUPUN2455-1663045118166-0-5] Exception:
io.github.resilience4j.circuitbreaker.CallNotPermittedException: CircuitBreaker
'circuitBreaker3' is OPEN and does not permit further calls
2022-09-13T10:28:59,163 DEBUG [http-nio-8086-exec-8] o.a.c.s.CamelLogger:
Failed delivery for (MessageId: ID-WKSWUPUN2455-1663045118166-0-5 on
ExchangeId: ID-WKSWUPUN2455-1663045118166-0-5). On delivery attempt: 0 caught:
io.github.resilience4j.circuitbreaker.CallNotPermittedException: CircuitBreaker
'circuitBreaker3' is OPEN and does not permit further calls
2022-09-13T10:28:59,163 ERROR [http-nio-8086-exec-8]
c.w.c.i.o.c.p.CaseExceptionProcessor:
io.github.resilience4j.circuitbreaker.CallNotPermittedException: CircuitBreaker
'circuitBreaker3' is OPEN and does not permit further calls
at
io.github.resilience4j.circuitbreaker.CallNotPermittedException.createCallNotPermittedException(CallNotPermittedException.java:48)
at
io.github.resilience4j.circuitbreaker.internal.CircuitBreakerStateMachine$OpenState.acquirePermission(CircuitBreakerStateMachine.java:689)
at
io.github.resilience4j.circuitbreaker.internal.CircuitBreakerStateMachine.acquirePermission(CircuitBreakerStateMachine.java:206)
at
io.github.resilience4j.circuitbreaker.CircuitBreaker.lambda$decorateCallable$3(CircuitBreaker.java:168)
at
org.apache.camel.component.resilience4j.ResilienceProcessor$CircuitBreakerTimeoutTask.get(ResilienceProcessor.java:517)
at
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
at
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760)
at
java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at
java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at
java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
2022-09-13T10:28:59,164 DEBUG [http-nio-8086-exec-8] o.a.c.p.PipelineHelper:
Message exchange has failed: so breaking out of pipeline for exchange:
Exchange[ID-WKSWUPUN2455-1663045118166-0-5] Handled by the error handler.
2022-09-13T10:28:59,166 DEBUG [http-nio-8086-exec-8]
o.a.c.h.c.DefaultHttpBinding: Streaming response in chunked mode with buffer
size 8192
2022-09-13T10:29:48,360 DEBUG [CircuitBreakerAutoTransitionThread]
i.g.r.c.i.CircuitBreakerStateMachine: No Consumers: Event STATE_TRANSITION not
published
logs from 3.18.2 camel version -
2022-09-13T10:49:10,036 DEBUG [http-nio-8086-exec-7] o.a.c.s.CamelLogger:
Request is routed to service2
2022-09-13T10:49:10,037 DEBUG [http-nio-8086-exec-7]
i.g.r.c.i.CircuitBreakerStateMachine: No Consumers: Event NOT_PERMITTED not
published
2022-09-13T10:49:10,037 INFO [http-nio-8086-exec-7] o.a.c.s.CamelLogger:
Message sent successfully
2022-09-13T10:49:10,038 DEBUG [http-nio-8086-exec-7] o.a.c.p.FilterProcessor:
Filter matches: true for exchange: Exchange[4A5CC43440BCF38-0000000000000002]
2022-09-13T10:49:10,038 DEBUG [http-nio-8086-exec-7]
o.a.c.h.c.DefaultHttpBinding: Streaming response in chunked mode with buffer
size 8192
Route Code snippet -
from("direct:route24")
.streamCaching()
.log(LoggingLevel.DEBUG, "Request is routed to service2")
.process(requestProcessor)
.marshal().json(JsonLibrary.Jackson)
.circuitBreaker()
.resilience4jConfiguration(resilience4jConfigurationDefinition)
.to("{{url}}")
.end()
.log(LoggingLevel.INFO, "Message sent successfully");
resilience4jConfiguration is as follows:
#circuit breaker
failureRateThreshold=50
automaticTransitionFromOpenToHalfOpenEnabled=true
waitDurationInOpenState=60
minimumNumberOfCalls=4
permittedNumberOfCallsInHalfOpenState=6
slidingWindowType=COUNT_BASED
slidingWindowSize=100
timeoutEnabled=true
timeoutDuration=30000
slowCallRateThreshold=100
slowCallDurationThreshold=60
How do we get the same behavior as previous(3.4.2) of circuit breaker in
version 3.18.2 of camel?
Thank you,
Abhishek
The information transmitted, including any content in this communication is
confidential, is intended only for the use of the intended recipient and is the
property of the Western Union Company or its affiliates and subsidiaries. If
you are not the intended recipient, you are hereby notified that any use of the
information contained in or transmitted with the communication or
dissemination, distribution, or copying of this communication is strictly
prohibited. If you have received this communication in error, please notify the
Western Union sender immediately by replying to this message and delete the
original message
