viirya opened a new pull request, #59439: URL: https://github.com/apache/spark/pull/59439
### What changes were proposed in this pull request? This is a backport of #59327 (SPARK-60109) to branch-3.5. It builds on SPARK-60158 (#59360), which is already on branch-3.5. The change is the same as on master: the History Server end-event reparse aligns to the next line on raw bytes before decoding (`FsHistoryProvider.skipToLineStart`), and replays the tail through the bounded reader with `ReplayListenerBus.replayFromOffset`, so `spark.history.fs.eventLog.maxLineLength` and strict decoding apply, and the diagnostics name the byte offset that their line numbers are relative to. Differences from the master commit, because branch-3.5 has no structured logging and builds with JDK 8: - The log messages in `ReplayListenerBus` and `FsHistoryProvider` keep the branch-3.5 string interpolation style, and `lineOrigin` returns a `String`. The message text is the same as on master. - The `skipToLineStart` test reads the returned stream with Guava `ByteStreams.toByteArray` instead of `InputStream.readAllBytes`. - The tests use the codec short name `"lz4"` instead of `CompressionCodec.LZ4`. ### Why are the changes needed? The same bugs exist on branch-3.5: the end-event reparse bypasses the bounded line reader, a skip target inside a multi-byte UTF-8 character drops a completed application from the listing, the skip loop does not terminate when the decompressed stream is shorter than the target, a line starting exactly at the target is discarded, and the diagnostics on this path print skip-relative line numbers next to the full path. See #59327 for details. ### Does this PR introduce _any_ user-facing change? Yes, in the History Server, the same as #59327: a completed application whose reparse offset lands inside a multi-byte character is listed again, `spark.history.fs.eventLog.maxLineLength` also applies to the end-event reparse, and the replay diagnostics on this path name the byte offset that their line numbers are relative to. ### How was this patch tested? The tests from #59327, adapted as described above. `FsHistoryProviderSuite` (RocksDB backend), `ReplayListenerSuite`, `core/scalastyle` and `core/Test/scalastyle` pass on branch-3.5 with JDK 8. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
