[
https://issues.apache.org/jira/browse/AVRO-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18102762#comment-18102762
]
ASF subversion and git services commented on AVRO-4241:
-------------------------------------------------------
Commit 8fa2067f70e3012cb3fd9a8839cd97e8c7cc1772 in avro's branch
refs/heads/branch-1.12 from Ismaël Mejía
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=8fa2067f70 ]
AVRO-4241: [Java] Bound zero-byte collection elements per datum, not per
collection (#3927)
* AVRO-4241: [Java] Bound zero-byte collection elements per datum, not per
collection
The heap-aware zero-byte-element allocation cap (null, a zero-length fixed, an
all-zero-byte record, or a recursive schema broken with a 0 minimum) was
enforced
per collection: readArray/readCollection and the skip/fast-reader paths each
started counting from zero. Because a container file carries its own schema, an
attacker can declare a record with many array<null> fields, each block
individually under the limit but jointly unbounded, so a tiny payload still
drives
a huge aggregate allocation (e.g. ~16 array<null> fields near the per-array cap
exhaust the heap; a handful burn tens of seconds of CPU).
Track the cumulative zero-byte allocation per decode on a per-thread scope in
SystemLimitException. GenericDatumReader.read and the static skip open the scope
(scopes nest, so a delegated fast reader or a skipped writer field accumulates
into the enclosing datum budget instead of resetting it); only the outermost
scope resets the running total. All zero-byte call sites (GenericDatumReader
read/skip, FastReaderBuilder, ReflectDatumReader) now use the cumulative
checkMaxCollectionAllocation(long). Outside any scope the check falls back to
the
previous per-collection behaviour, so no existing caller becomes stricter.
Positive-size elements are unchanged: they remain bounded per collection by the
bytes-remaining check, which consumes input as it advances.
Adds regression tests for a multi-field record rejected cumulatively and a
within-limit record that still decodes (and confirms the budget resets between
datums), on both the fast and classic reader paths.
* AVRO-4241: [Java] Scope fast array reader so zero-byte cap is cumulative
standalone
Open a collection-allocation scope around the fast array reader's
block-reading loop in a try/finally. When the fast reader is used
standalone via createDatumReader(...), without GenericDatumReader.read
opening the outer datum scope, the zero-byte element cap is now
cumulative across all array blocks instead of degrading to a per-block
stateless check, so a large array<null>-style array split across many
blocks cannot bypass the cap. The scope nests into the outer datum
scope on the normal path, and the finally guarantees it is always
closed so ThreadLocal state cannot leak into later decodes.
> [java] BinaryDecoder should verify available bytes before reading
> -----------------------------------------------------------------
>
> Key: AVRO-4241
> URL: https://issues.apache.org/jira/browse/AVRO-4241
> Project: Apache Avro
> Issue Type: Sub-task
> Components: java
> Affects Versions: 1.11.5, 1.12.1
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.12.2
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
