shashank created CAMEL-25504:
--------------------------------
Summary: camel-jta, camel-spring - a transacted exchange that
completes during a graceful shutdown is rolled back instead of committed
Key: CAMEL-25504
URL: https://issues.apache.org/jira/browse/CAMEL-25504
Project: Camel
Issue Type: Bug
Components: camel-spring
Reporter: shashank
{{DefaultShutdownStrategy}} first stops or suspends the route consumers, then
calls {{prepareShutdown(suspendOnly=false, forced=false)}} on the route
services ("notify the services we intend to shutdown",
{{DefaultShutdownStrategy}} around line 659 at main c578a42a776d), then waits
up to the shutdown timeout for the in-flight exchanges. Only if the timeout
expires does it call {{prepareShutdown(false, forced=true)}}.
Both transaction error handlers set {{preparingShutdown = true}} on both calls:
{{org.apache.camel.jta.TransactionErrorHandler.prepareShutdown}} directly, and
{{org.apache.camel.spring.spi.TransactionErrorHandler}} through
{{RedeliveryErrorHandler.prepareShutdown}}, which it calls first. Since
CAMEL-23234 both transaction callbacks do, after the route work (camel-jta
{{doInTransactionTemplate}} line 205, camel-spring {{doInTransactionTemplate}}
line 215):
{code:java}
// if forced shutdown is in progress, mark the exchange for rollback
if (preparingShutdown) {
exchange.setRollbackOnly(true);
}
{code}
So every transacted exchange that is in flight when a graceful shutdown
(CamelContext stop, or a route stop through the route controller) starts, and
that completes normally inside the grace period, is rolled back. Graceful
shutdown no longer lets in-flight transactions finish. The
{{TransactionRolledbackException}} that forces the rollback is swallowed and no
exception is set on the exchange, so a caller that does not redeliver (direct,
platform-http/REST, timer, a ProducerTemplate) gets a normal reply while the
database work is rolled back; with a transacted JMS consumer the message is
redelivered after the restart and any non-transactional side effects run twice.
The CAMEL-23234 PR description and its tests describe the intended behaviour as
a rollback after the grace period has expired (forced shutdown); the graceful
case was not covered by its tests ({{TransactionErrorHandlerShutdownTest}}
calls {{prepareShutdown(false, true)}}, and the IT only checks the timeout
path).
By reading {{DefaultShutdownStrategy}} (not tested): a route whose consumer is
deferred ({{shutdownRoute(Defer)}}) keeps taking new exchanges during the grace
period, after the graceful {{prepareShutdown}}, so on main every transacted
exchange it processes until the in-flight count reaches zero is rolled back too.
h3. Reproduction
New {{TransactionalClientDataSourceGracefulShutdownTest}} (camel-spring-xml,
next to {{TransactionalClientDataSourceForcedShutdownTest}}; the
{{transactionalClientDataSource.xml}} context: a real
{{DataSourceTransactionManager}} on an embedded H2 database, 1 book inserted at
startup):
* {{inFlightTransactionCompletingDuringContextStopIsCommitted}}:
{{from("direct:graceful").transacted().bean("bookService")}} inserts a book,
then the exchange is held in the route; {{context.stop()}} with a 30 s timeout.
A service in the transacted block releases the exchange from its own graceful
{{prepareShutdown}} call (the strategy prepares the transaction error handler
before it), and the exchange completes long before the timeout. On main:
{noformat}
an exchange that completed during a graceful shutdown must not be marked
rollback only ==> expected: <false> but was: <true>
{noformat}
* {{gracefulPrepareShutdownCommitsInFlightTransaction}}: the same with
{{prepareShutdown(false, false)}} called on the handler directly, as the
existing forced-shutdown test does with {{prepareShutdown(false, true)}}. On
main: {{graceful shutdown must not mark the exchange rollback only ==>
expected: <false> but was: <true>}} (and the insert is rolled back: 1 book
instead of 2).
New {{TransactionErrorHandlerGracefulShutdownTest}} (camel-jta, no container):
* {{inflightExchangeCompletingDuringGracefulShutdownIsCommitted}}:
{{from("direct:start").transacted().process(blocking)}}, a test
{{JtaTransactionPolicy}} counting commits and rollbacks, shutdown timeout 20 s.
The exchange is held in the route, {{camelContext.stop()}} is started on
another thread, the test waits (Awaitility) until the handler received the
graceful {{prepareShutdown}}, then releases the exchange, which completes long
before the timeout. On main:
{noformat}
an exchange that completed during a graceful shutdown must not be marked
rollback only ==> expected: <false> but was: <true>
{noformat}
* {{gracefulPrepareShutdownDoesNotRollBack}}: the same on the handler directly
with {{prepareShutdown(false, false)}}. On main: {{graceful shutdown must not
mark the exchange rollback only ==> expected: <false> but was: <true>}}.
The existing forced-shutdown tests (camel-jta
{{TransactionErrorHandlerShutdownTest}}, camel-spring-xml
{{TransactionalClientDataSourceForcedShutdownTest}}) and the suspend/resume
tests pass with and without the fix.
h3. Proposed fix
In both handlers, a separate {{volatile boolean forcedShutdown}} that
{{prepareShutdown}} sets only when {{forced}} is true (reset in
{{doStart}}/{{doResume}} like {{preparingShutdown}}); the check after the route
work reads it instead of {{preparingShutdown}}. {{preparingShutdown}} itself is
unchanged (in the Spring handler it still stops redeliveries during shutdown).
The forced path of CAMEL-23234 is unchanged: on timeout the in-flight exchanges
are still marked rollback only and an exchange finishing afterwards still rolls
back. One secondary change: with {{shutdownNowOnTimeout=false}} the strategy
never calls {{prepareShutdown(.., forced=true)}}, so an exchange that completes
after the timeout is now committed (as before 4.19). The upgrade guide (4.23)
gets a short note. Tests with the fix: camel-jta 5 tests, 0 failures (the
Postgres IT {{TransactionErrorHandlerGracePeriodShutdownIT}} was skipped: no
Docker; it exercises only the forced path, which the change does not touch);
camel-spring-xml 1184 tests, 0 failures, 26 skipped (camel-spring itself has no
tests).
CAMEL-23234 added the check to both handlers in one change, so this ticket
fixes both.
Found with a TLA+ model of the handler (begin, route work, the flag check,
commit/rollback) and the shutdown strategy (stop consumer, graceful prepare,
wait, timeout, forced marking): "an exchange that completed successfully before
the shutdown timeout is committed" is violated in 7 steps (begin, consumer
stopped, work done, graceful prepare, check sets rollback only, rollback). With
the fix the property holds for 2 and 3 exchanges, and the forced rollback stays
reachable. Then confirmed with the real classes as above (the modelled flag
logic is the same in both handlers; the model does not include redelivery).
Affected: main, camel-4.22.x (LTS) and camel-4.19.x, i.e. 4.19.0 onwards
(CAMEL-23234); camel-4.18.x and camel-4.14.x do not have the check (GitHub
contents API).
Duplicate check (2026-10-09): JIRA text "TransactionErrorHandler" + "shutdown"
(CAMEL-23234 only), "graceful shutdown" + "transacted" (CAMEL-23234,
CAMEL-23260 is Service Bus), "rollbackOnly" + "shutdown" (none); GitHub pull
requests "jta shutdown", "TransactionErrorHandler" (open): only #22204
(CAMEL-23234, merged).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
