Rich-T-kid opened a new issue, #12325:
URL: https://github.com/apache/arrow-rs/issues/12325

   ### Is your feature request related to a problem or challenge? Please 
describe what you are trying to do.
   
   We keep finding realistic integer-overflow bugs one callsite at a time. 
Recent examples include Parquet byte-array offsets (#12179 and #12215), 
`MapBuilder` offsets (#12052), `FixedSizeList` validation (#11859), and CSV 
allocation sizing (#9886).
   
   These bugs tend to have the same shape: a length, offset, count, or 
allocation size derived from input is combined with unchecked `+` or `*`; 
overflow then bypasses a bounds check, panics, requests the wrong allocation, 
or violates an unsafe precondition. They are especially easy to miss on 64-bit 
development machines when the failure is reachable on supported 32-bit targets.
   
   Fixing only the reported expression leaves sibling paths with the same 
problem and produces a stream of very small follow-up PRs. We should audit the 
realistic overflow boundaries once and fix them in coordinated batches.
   
   ### Describe the solution you'd like
   
   Audit arithmetic involving user-controlled or format-controlled lengths, 
offsets, counts, and capacities, focusing on:
   
   - parsing and decoding Parquet, IPC, CSV, JSON, and Avro input;
   - arithmetic used before slicing, unchecked indexing, pointer arithmetic, or 
buffer writes;
   - allocation and reserve sizes such as `count * element_size` and `offset + 
length`;
   - builders and kernels that produce `i32`/`u32` offsets, dictionary keys, 
run ends, or other narrow physical values;
   - code whose correctness differs between 32-bit and 64-bit targets.
   
   Use `checked_add`, `checked_mul`, `checked_sub`, or checked conversions 
where overflow is reachable, and return the existing crate error before 
allocation, mutation, or unsafe access. Add boundary tests that fail without 
the check, including a 32-bit-target test when 64-bit arithmetic cannot 
reproduce the overflow.
   
   This should not be a mechanical replacement of every arithmetic operator. 
Loop counters and other arithmetic already bounded by a clear invariant should 
remain simple, and numeric kernels with documented wrapping, saturating, or 
overflow semantics are out of scope.
   
   To avoid another flood of single-callsite PRs, contributors should claim a 
crate or subsystem, inspect sibling implementations and wrapper paths, and 
submit one coherent patch for that area. The issue can track the audited areas 
and any expressions that are intentionally left unchecked with their invariant.
   
   ### Acceptance criteria
   
   - Relevant crates and subsystems have been inventoried and tracked here.
   - Each candidate is fixed or documented as safe because of a concrete 
dominating bound.
   - Fixes cover sibling and wrapper paths, not only the originally reported 
expression.
   - Fallible paths report an error before partial mutation or unsafe access.
   - Applicable overflow boundaries have regression tests, including 32-bit 
coverage where needed.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to