jnioche opened a new pull request, #8440: URL: https://github.com/apache/storm/pull/8440
In
SequenceFileReader.Offset.increment(), line 204 overwrites `currentRecord` (a
record counter) with `newBytePosition` (a byte offset), immediately after
incrementing it on line 202:
```
++currentRecord; // line 202 — correct
prevRecordEndOffset = currRecordEndOffset; // line 203
currentRecord = newBytePosition; // line 204 — BUG: should be
currRecordEndOffset
```
This is a copy-paste error: `currentRecord` should be
`currRecordEndOffset`.
**Impact**
Every call to next() replaces the record counter with the reader's byte
position. This corrupts:
- Offset equality/comparison — two offsets at the same record count
compare as unequal if byte positions differ
- Offset serialization (toString()) — the persisted record= field contains
a byte offset, not a record number
- Resume after restart — the HDFS spout uses the serialized offset to
resume reading; a corrupted value causes records to be skipped or re-processed
- prevRecordEndOffset tracking — since currRecordEndOffset is never
updated, prevRecordEndOffset always copies a stale value, breaking sync point
calculation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
