Croway opened a new pull request, #30082:
URL: https://github.com/apache/camel/pull/30082

   [CAMEL-26996](https://issues.apache.org/jira/browse/CAMEL-26996)
   
   ### What was wrong
   
   With `outputType=StreamList`, when reading a row fails, 
`ResultSetIterator.next()` closes the iterator and throws, and `hasNext()` is 
`!closed` (camel-jdbc and camel-sql have the same code). After the failure the 
iterator therefore says "no more rows", like an iterator that has read the last 
row.
   
   The failure comes from the splitter's own iteration (not from a 
sub-exchange), so `MulticastTask.doFailed` → `doDone(..., forceExhaust=false)` 
leaves it not exhausted and the route's error handler redelivers to the 
splitter, with the same body: the same iterator, which now has no more rows. 
With a redelivery policy the split completes and the exchange **succeeds with 
only the rows read before the failure**. Found with fault-injection tests 
(Toxiproxy in front of PostgreSQL: connection reset, backend restart).
   
   ### The fix
   
   `ResultSetIterator` (camel-jdbc and camel-sql) remembers the `SQLException`: 
from then on `hasNext()` (and so `next()`) throws a `RuntimeCamelException` 
with that `SQLException` as cause, also after `close()` (the splitter closes a 
`Closeable` iterator body when the split fails). The first failure is unchanged 
(`Cannot process result`). A redelivered split now fails each time, and the 
exchange fails once the redeliveries are exhausted. camel-spring-jdbc uses the 
camel-jdbc iterator, so it gets the fix too.
   
   ### Design choices for reviewers
   
   - **Fix in the iterator, not in the splitter / error handler.** Not 
redelivering a split over an iterator would change every iterator body, 
including ones that can go on after a failure (e.g. a paging iterator that 
retries the failed page). Only the iterator knows that it cannot be read again. 
`IOHelper.closeIterator` already does the same for a `Scanner` (rethrows the 
`IOException` the scanner swallowed, so a failed read does not look like the 
end of the input).
   - **`hasNext()` throws** rather than returning `true` and letting `next()` 
throw: after a failure whether more rows exist is unknown, and a caller that 
catches the exception of `next()` and loops on `hasNext()` would otherwise spin 
forever. The splitter treats an exception from `hasNext()` like one from 
`next()` (both end in `doFailed`).
   - A redelivery of the split can only fail: it cannot read the stream again. 
Running the query again needs the redelivery to happen at the producer (e.g. 
the query and the split in a route called with `to("direct:...")`), which is up 
to the route.
   - Not changed: a row whose `outputClass` mapping fails in camel-jdbc (the 
mapping is in `camel-support`'s `StreamListIterator`, outside the result set 
iterator); the result set itself is fine there.
   
   ### Relation to CAMEL-26991
   
   CAMEL-26991 (camel-sql: `StreamList` takes part in the route transaction, 
branch not opened as a PR yet) also touches camel-sql's `ResultSetIterator`, 
but only its imports, fields, constructors and `safeCloseConnection()` (the 
connection handling). This PR only changes `hasNext()`/`next()` and adds one 
field after `rowNum`, so the two merge without conflict (checked with `git 
merge-tree`). CAMEL-26997 (camel-jdbc commits before a `StreamList` is read, 
which on PostgreSQL with a fetch size fails the stream after the first batch) 
is a separate PR: with it and this one, that scenario no longer truncates 
silently.
   
   ### Relation to #30060 (docs)
   
   #30060 adds a NOTE to `jdbc-component.adoc` and `sql-component.adoc` that 
describes the current behaviour (a redelivered split over a broken `StreamList` 
completes with only the rows read before the failure, so do not configure 
redeliveries for that route). Once this PR is merged that NOTE is no longer 
true: it must be dropped or reworded (a redelivery of the split now fails too; 
to read the rows again, run the query again). Whichever of the two PRs merges 
second adjusts it.
   
   ### Tests
   
   - `JdbcProducerStreamListReadFailureTest` (camel-jdbc) and 
`SqlProducerStreamListReadFailureTest` (camel-sql): H2 behind a 
`DelegatingDataSource` whose result sets fail `next()` (SQLState `08006`, as 
pgjdbc on a lost connection) once the route has handled the first row. A split 
with `maximumRedeliveries(2)` must fail with the `SQLException` as cause after 
2 redeliveries; the iterator must keep throwing on `hasNext()`/`next()`, also 
after `close()`. camel-jdbc also keeps the case without an error handler 
(already failing correctly).
   - **Fail before the fix**: camel-jdbc 2 of 3 fail (the redelivered exchange 
succeeds: `expected: <RuntimeCamelException> but was: <null>`; `hasNext()` does 
not throw), camel-sql 2 of 2 fail.
   - After the fix: the new tests and the full camel-jdbc (68 tests) and 
camel-sql (393 tests) suites pass.
   
   ### Upgrade guide
   
   Yes, 4.24: a `StreamList` that failed to read now keeps failing (a 
redelivered split fails instead of succeeding with part of the rows; 
`hasNext()` throws after a read failure).
   
   _Claude Code on behalf of Croway_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to