Hi Marek, On Wed, 18 Feb 2026 at 17:34, Marek Vasut <[email protected]> wrote: > > The current gzwrite() implementation is limited to 4 GiB compressed > input buffer size due to struct z_stream_s { uInt avail_in } member, > which is of type unsigned int. Current gzwrite() implementation sets > the entire input buffer size as avail_in and performs decompression > of the whole compressed input buffer in one round, which limits the > size of input buffer to 4 GiB. > > Rework the decompression loop to use chunked approach, and decompress > the input buffer in up to 4 GiB - 1 kiB avail_in chunks, possibly in > multiple decompression rounds. This way, the compressed input buffer > size is limited by gzwrite() function 'len' parameter type, which is > unsigned long. > > In case of sandbox build, include parsing of 'gzwrite_chunk' > environment variable, so the chunked approach can be thoroughly tested > with non default chunk size. For non-sandbox builds, the chunk size is > 4 GiB - 1 kiB. > > The gzwrite test case is extended to test various chunk sizes during > gzwrite decompression test. > > Signed-off-by: Marek Vasut <[email protected]> > --- > Cc: Alexander Graf <[email protected]> > Cc: Heinrich Schuchardt <[email protected]> > Cc: Ilias Apalodimas <[email protected]> > Cc: Jerome Forissier <[email protected]> > Cc: Mattijs Korpershoek <[email protected]> > Cc: Neil Armstrong <[email protected]> > Cc: Peng Fan <[email protected]> > Cc: Quentin Schulz <[email protected]> > Cc: Simon Glass <[email protected]> > Cc: Tom Rini <[email protected]> > Cc: Yuya Hamamachi <[email protected]> > Cc: [email protected] > --- > V2: Rebase on master > --- > lib/gunzip.c | 76 +++++++++++++++++++++++++++++++++--------------- > test/cmd/unzip.c | 12 +++++++- > 2 files changed, 64 insertions(+), 24 deletions(-) > > diff --git a/lib/gunzip.c b/lib/gunzip.c > index 76f3397fced..20cc14f9688 100644 > --- a/lib/gunzip.c > +++ b/lib/gunzip.c > @@ -8,8 +8,10 @@ > #include <command.h> > #include <console.h> > #include <div64.h> > +#include <env.h> > #include <gzip.h> > #include <image.h> > +#include <linux/sizes.h> > #include <malloc.h> > #include <memalign.h> > #include <u-boot/crc.h> > @@ -119,7 +121,7 @@ void gzwrite_progress_finish(int returnval, > int gzwrite(unsigned char *src, size_t len, struct blk_desc *dev, > size_t szwritebuf, off_t startoffs, size_t szexpected) > { > - int i, flags; > + int flags; > z_stream s; > int r = 0; > unsigned char *writebuf; > @@ -127,13 +129,23 @@ int gzwrite(unsigned char *src, size_t len, struct > blk_desc *dev, > ulong totalfilled = 0; > lbaint_t blksperbuf, outblock; > u32 expected_crc; > - size_t payload_size; > + size_t i, payload_size; > + unsigned long blocks_written; > + lbaint_t writeblocks; > + int numfilled = 0; > int iteration = 0; > - > - if (len > 0xffffffff) { > - log_err("Input size over 4 GiB in size not supported\n"); > - return -1; > - } > + /* > + * Allow runtime configuration of decompression chunk on > + * sandbox to better cover the chunked decompression > + * functionality without having to use > 4 GiB files. > + */ > + const ulong minchunk = 0x400; > + const ulong maxchunk = SZ_4G - minchunk; > + const ulong chunk = > + CONFIG_IS_ENABLED(SANDBOX, > + (clamp(env_get_ulong("gzwrite_chunk", 10, > maxchunk), > + minchunk, maxchunk)), > + (maxchunk)); >
As you mentioned in the last version, there is a compression test in compression.c - could you add a version of this function that takes maxchunk as an argument and call it from that test? Reading an environment variable to avoid passing a parameter seems pretty odd to me :-) [..] Regards, Simon
