allthingssecurity opened a new pull request, #30251: URL: https://github.com/apache/camel/pull/30251
# Description [CAMEL-27067](https://issues.apache.org/jira/browse/CAMEL-27067) With `readLine=false` the stream consumer reads in raw mode. Without `scanStream` it reads the stream once and sends one exchange with the bytes. With `scanStream=true` the loop called `is.readAllBytes()` on each scan but never passed the bytes to `processRaw`, so no exchange was created and the data was dropped: what was in the file when the consumer started, and everything appended later. A route like `from("stream:file?fileName=app.log&scanStream=true&readLine=false")` consumed the file and sent nothing. Two more problems in the same loop: - with `retry=true` it reopened the stream after every read and slept only when nothing was read, so a non-empty file was read again (or the URL requested again) in a tight loop with no pause; - a file that did not exist at start gave a null stream, and the first read threw a `NullPointerException` that ended the consumer thread, with or without `retry` (the line mode handles a null reader). Fix, in `StreamConsumer.readFromStreamRawMode`: - the scan loop routes the bytes it read and keeps the index; - as in the line mode, it reopens the stream for `retry` only at the end of the stream, which removes the busy loop; - a null stream counts as the end of the stream, so with `retry` the file is opened once it exists. Without `retry` a file that is missing at start is never picked up, as in the line mode; - the raw scan exchanges have `CamelStreamComplete=false`, as the line scan mode does (a scanned stream is never complete). The protected `processRaw(byte[], long)` keeps its behaviour and delegates to a private overload with the flag, so there is no API change; - the non-scan raw mode keeps the index returned by `processRaw`, and declares the read buffer inside its loop, so a failed later read does not route the previous bytes again. With `retry` a reopened stream is read again from the beginning, the same as in the line mode: without `fileWatcher` the whole file is routed again at every end of stream, and with `fileWatcher` a change re-reads the file from the start. This change does not alter that; the docs of `retry` already say to use it with `fileWatcher` for files. Docs: the `readLine` option and the consumer body type now say that with `scanStream` each exchange holds the bytes read in one scan (regenerated `stream.json`, catalog copy and endpoint DSL). Tests: - `ScanStreamFileRawModeTest` writes "Hello\nWorld\n" to a file, starts `stream:file?...&scanStream=true&scanStreamDelay=100&readLine=false`, waits for the first exchange, appends "Bye\n", and expects the two bodies with the indexes 0 and 1 and `CamelStreamComplete=false`; - `ScanStreamFileRawModeRetryTest` starts `...&scanStream=true&scanStreamDelay=100&retry=true&readLine=false` with no file, then creates it, and expects at least one exchange with its content. Without the change both fail: the first times out with no exchange, the second gets no exchange because the consumer thread ended with the `NullPointerException`. With the change the camel-stream unit tests pass (33 run, 0 failures, 1 skipped manual test), and the two new tests pass 20 times each in one JVM. Backport: camel-4.18.x has the same code. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-27067](https://issues.apache.org/jira/browse/CAMEL-27067). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built camel-stream and the core modules with the formatter and import-sort plugins and the package plugin, which regenerated `stream.json`; I copied it to the catalog and applied the same text to the generated endpoint DSL, in the generator's wrapping. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
