sungwy opened a new issue, #3952:
URL: https://github.com/apache/iceberg-python/issues/3952
Two places in `pyiceberg/avro/decoder_fast.pyx` advance the read pointer
using a value taken from the stream, without bounding it against `self._end`.
**1. `read_bytes` does not validate the decoded length**
```python
cpdef inline bytes read_bytes(self):
cdef uint64_t length;
if self._current >= self._end: # only confirms 1 byte is available
raise EOFError(f"EOF: read 1 bytes")
decode_zigzag_ints(&self._current, 1, &length)
if length <= 0:
return b""
cdef const unsigned char *r = self._current
self._current += length # not checked against self._end
return r[0:length]
```
The guard confirms one byte is available before decoding the length, but the
decoded `length` is then used to slice and to advance `_current` with no check
that `_current + length <= _end`. A length field larger than the remaining
buffer reads beyond it.
**2. `decode_zigzag_ints` has no end pointer to bound against**
```c
void decode_zigzag_ints(const unsigned char **buffer, const uint64_t count,
uint64_t *result);
```
The signature takes a buffer and a count but no end, so the varint walk
cannot stop at the buffer boundary — a run of bytes with the continuation bit
set keeps advancing. It is called from five sites in the decoder (lines 93,
101, 111, 124, 176), including from `read_bytes` above.
Both are reachable from a malformed or hostile Avro manifest.
---
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
