PR #24354 opened by yibofang
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24354
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24354.patch
# Summary of changes
The FLAC encoder currently embeds scratch buffers sized for the maximum
FLAC block size and channel count directly in `FlacSubframe` and
`FlacFrame`. As a result, every encoder context requires approximately
7 MiB, regardless of the block size or number of channels actually in
use.
This was observed on embedded devices with small heaps, where allocating
the encoder private context can fail before the encoder is opened. The
same fixed cost is also wasteful for common mono or stereo configurations
using much smaller frames.
For example, with two channels and a block size of 4608, the moved
scratch storage is reduced from approximately 7 MiB to about 154 KiB
(about 190 KiB for 32-bit stereo).
This change allocates the encoder scratch buffers during initialization,
after the actual frame size and channel count are known:
- Sample and residual storage is allocated for the configured number of
channels and frame size.
- The per-channel sample stride is large enough for both the SSE4 LPC
path's minimum 32-sample access and its vector padding of up to 11
samples.
- Rice parameter scratch buffers are moved to the encoder context and
shared between channels. Channels are analyzed and encoded serially,
so separate copies are unnecessary.
- The 33-bit stereo decorrelation buffer is allocated only for stereo
encoding with 32-bit input, which is the only configuration that uses
it.
- All new allocations are released by the encoder close callback, which
is also used by the existing initialization cleanup path.
The encoder continues to support one through eight channels and the full
standard FLAC block-size range. No allocation is performed in the frame
encoding path, and no bitstream change is intended.
## Testing
Tests were run on x86_64 with GCC 13 and x86 assembly enabled. The build
reported `HAVE_SSE4_EXTERNAL=yes`.
### FATE
The following relevant FATE targets were run:
```sh
make -j8 fate-flac \
fate-acodec-flac \
fate-acodec-flac-exact-rice \
fate-api-flac
```
Results:
- Optimized native build: 13/13 tests passed.
- AddressSanitizer build: 13/13 tests passed.
- AddressSanitizer build with `CPUFLAGS=0`: all 12 ffmpeg-based tests
passed using the scalar C paths. `fate-api-flac` also passed, using
runtime-native CPU dispatch.
Each invocation of `fate-api-flac` covers four channel layouts at four
sample rates and 200 frames per configuration, for 16 configurations
and 3,200 frames.
### Bit-exact regression testing
The patched encoder was compared with its parent commit using 27 encoder
configurations, for a total of 54 encodes. The matrix covered:
- Compression levels 0 through 12.
- Exact Rice parameter search.
- Fixed, Cholesky, and Levinson LPC modes.
- Independent, left-side, mid-side, and right-side stereo modes.
- 16-bit, 24-bit, and 32-bit input.
- A 32-bit sample containing wasted bits.
All 27 output FLAC files were byte-for-byte identical to the files
produced by the parent commit.
### Allocation regression testing
A minimal mono encode was run with a 1 MiB per-allocation limit:
```sh
ffmpeg -max_alloc 1048576 \
-f s16le -ar 48000 -ac 1 -i /dev/zero \
-frames:a 1 -c:a flac -frame_size 16 -f null -
```
The parent commit failed to open the encoder with `ENOMEM` because the
fixed-size private context exceeded the allocation limit. The patched
encoder completed successfully.
A second run with `-max_alloc 60000` forced initialization to fail after
partial allocation. It returned `ENOMEM` as expected, with no leak
reported by AddressSanitizer/LeakSanitizer.
### Boundary and sanitizer testing
Additional tests were run under AddressSanitizer for:
- Every supported channel count from one through eight.
- Block sizes 16, 31, 32, 33, 4095, 4096, 4608, and 65535.
- A seven-sample final frame following full-size frames for every tested
block size.
- 24-bit and experimental 32-bit encoding.
- All stereo decorrelation modes.
- The maximum block size with the 32-bit sample path.
- Invalid block sizes 15 and 65536, both rejected as expected.
All eight channel-count cases, eight block-size cases, and eight
bit-depth/stereo-mode cases passed. No out-of-bounds access,
use-after-free, or memory leak was reported.
### API stress testing
`tests/api/api-flac-test` was run ten times under AddressSanitizer. This
covered 160 encoder context configurations and 32,000 encoded and decoded
frames. All runs passed without sanitizer findings.
No new FATE samples are required by this change.
The original allocation failure was observed on an embedded target. The
patch has been validated on the host as described above, but has not yet
been retested on that target.
>From 3cfd4055ddd51ffdce8be0fd5ed2eda514e590c9 Mon Sep 17 00:00:00 2001
From: fangyibo <[email protected]>
Date: Wed, 5 Aug 2026 21:01:43 +0800
Subject: [PATCH] avcodec/flacenc: allocate encoder buffers dynamically
The fixed-size scratch buffers make each encoder context about 7 MiB,
even when a much smaller block size is configured. This can prevent the
encoder from opening on memory-constrained embedded systems.
Allocate the sample and residual buffers based on frame size and channel
count, keeping the padding required by the SSE4 LPC code. Move the Rice
scratch buffers to FlacEncodeContext so they can be shared between
channels, which are encoded serially.
Signed-off-by: fangyibo <[email protected]>
---
libavcodec/flacenc.c | 73 ++++++++++++++++++++++++++++++++++++--------
1 file changed, 60 insertions(+), 13 deletions(-)
diff --git a/libavcodec/flacenc.c b/libavcodec/flacenc.c
index ead2c55f10..e723f71c23 100644
--- a/libavcodec/flacenc.c
+++ b/libavcodec/flacenc.c
@@ -86,16 +86,15 @@ typedef struct FlacSubframe {
int shift;
RiceContext rc;
- uint32_t rc_udata[FLAC_MAX_BLOCKSIZE];
- uint64_t rc_sums[32][MAX_PARTITIONS];
-
- int32_t samples[FLAC_MAX_BLOCKSIZE];
- int32_t residual[FLAC_MAX_BLOCKSIZE+11];
+ int32_t *samples;
+ int32_t *residual;
} FlacSubframe;
typedef struct FlacFrame {
FlacSubframe subframes[FLAC_MAX_CHANNELS];
- int64_t samples_33bps[FLAC_MAX_BLOCKSIZE];
+ int32_t *samples_buffer;
+ int32_t *residual_buffer;
+ int64_t *samples_33bps;
int blocksize;
int bs_code[2];
uint8_t crc8;
@@ -118,6 +117,8 @@ typedef struct FlacEncodeContext {
uint64_t sample_count;
uint8_t md5sum[16];
FlacFrame frame;
+ uint32_t *rc_udata;
+ uint64_t (*rc_sums)[MAX_PARTITIONS];
CompressionOptions options;
AVCodecContext *avctx;
LPCContext lpc_ctx;
@@ -267,6 +268,42 @@ static av_cold void
dprint_compression_options(FlacEncodeContext *s)
}
+static av_cold int allocate_buffers(FlacEncodeContext *s)
+{
+ FlacFrame *frame = &s->frame;
+ size_t blocksize = s->avctx->frame_size;
+ size_t sample_stride = FFMAX(blocksize + 11, 32);
+ int ch;
+
+ /* The SSE4 LPC path unconditionally copies 32 samples and processes
+ * samples in batches of 12, so it may access up to 11 samples past len. */
+ frame->samples_buffer = av_calloc(s->channels * sample_stride,
+ sizeof(*frame->samples_buffer));
+ frame->residual_buffer = av_calloc(s->channels * sample_stride,
+ sizeof(*frame->residual_buffer));
+ s->rc_udata = av_calloc(blocksize, sizeof(*s->rc_udata));
+ s->rc_sums = av_calloc(32, sizeof(*s->rc_sums));
+ if (!frame->samples_buffer || !frame->residual_buffer ||
+ !s->rc_udata || !s->rc_sums)
+ return AVERROR(ENOMEM);
+
+ if (s->channels == 2 && s->avctx->bits_per_raw_sample == 32) {
+ frame->samples_33bps = av_calloc(blocksize,
+ sizeof(*frame->samples_33bps));
+ if (!frame->samples_33bps)
+ return AVERROR(ENOMEM);
+ }
+
+ for (ch = 0; ch < s->channels; ch++) {
+ frame->subframes[ch].samples = frame->samples_buffer + ch *
sample_stride;
+ frame->subframes[ch].residual = frame->residual_buffer +
+ ch * sample_stride;
+ }
+
+ return 0;
+}
+
+
static av_cold int flac_encode_init(AVCodecContext *avctx)
{
int freq = avctx->sample_rate;
@@ -421,6 +458,10 @@ static av_cold int flac_encode_init(AVCodecContext *avctx)
}
s->max_blocksize = s->avctx->frame_size;
+ ret = allocate_buffers(s);
+ if (ret < 0)
+ return ret;
+
/* set maximum encoded frame size in verbatim mode */
s->max_framesize = flac_get_max_frame_size(s->avctx->frame_size,
s->channels,
@@ -710,7 +751,7 @@ static void calc_sum_next(int level, uint64_t
sums[32][MAX_PARTITIONS], int kmax
}
static uint64_t calc_rice_params(RiceContext *rc,
- uint32_t udata[FLAC_MAX_BLOCKSIZE],
+ uint32_t *udata,
uint64_t sums[32][MAX_PARTITIONS],
int pmin, int pmax,
const int32_t *data, int n, int pred_order,
int exact)
@@ -769,7 +810,7 @@ static uint64_t find_subframe_rice_params(FlacEncodeContext
*s,
uint64_t bits = 8 + pred_order * sub->obits + 2 + sub->rc.coding_mode;
if (sub->type == FLAC_SUBFRAME_LPC)
bits += 4 + 5 + pred_order * s->options.lpc_coeff_precision;
- bits += calc_rice_params(&sub->rc, sub->rc_udata, sub->rc_sums, pmin,
pmax, sub->residual,
+ bits += calc_rice_params(&sub->rc, s->rc_udata, s->rc_sums, pmin, pmax,
sub->residual,
s->frame.blocksize, pred_order,
s->options.exact_rice_parameters);
return bits;
}
@@ -1343,17 +1384,18 @@ static void channel_decorrelation(FlacEncodeContext *s)
int64_t *side_33bps;
int n;
- frame = &s->frame;
- n = frame->blocksize;
- left = frame->subframes[0].samples;
- right = frame->subframes[1].samples;
- side_33bps = frame->samples_33bps;
+ frame = &s->frame;
+ n = frame->blocksize;
if (s->channels != 2) {
frame->ch_mode = FLAC_CHMODE_INDEPENDENT;
return;
}
+ left = frame->subframes[0].samples;
+ right = frame->subframes[1].samples;
+ side_33bps = frame->samples_33bps;
+
if (s->options.ch_mode < 0) {
int max_rice_param = (1 << frame->subframes[0].rc.coding_mode) - 2;
frame->ch_mode = estimate_stereo_mode(left, right, n, max_rice_param,
s->avctx->bits_per_raw_sample);
@@ -1703,6 +1745,11 @@ static av_cold int flac_encode_close(AVCodecContext
*avctx)
{
FlacEncodeContext *s = avctx->priv_data;
+ av_freep(&s->frame.samples_buffer);
+ av_freep(&s->frame.residual_buffer);
+ av_freep(&s->frame.samples_33bps);
+ av_freep(&s->rc_udata);
+ av_freep(&s->rc_sums);
av_freep(&s->md5ctx);
av_freep(&s->md5_buffer);
ff_lpc_end(&s->lpc_ctx);
--
2.52.0
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
