PR #25875 opened by Kilian (lusoris) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25875 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25875.patch
This series adds a `qpfile` option to libaom-av1, libsvtav1 and libx264. It reads per-frame quantizer offsets from a text file and passes them to the encoder's own per-block quantizer interface, so an external analysis step can steer the quality per 16x16 block without patching the encoder libraries. ## What changes - `avcodec/libaomenc: add a qpfile option` adds the file format and its reader in `libavcodec/qpfile_parser.c` and passes each record to libaom as a region-of-interest map through `AOME_SET_ROI_MAP`. - `avcodec/libsvtav1: add a qpfile option` passes each record to SVT-AV1 as a region-of-interest map event (ROI map API, SVT-AV1 1.6.0 or newer). - `avcodec/libx264: add a qpfile option` passes the offsets to x264 as per-macroblock `quant_offsets`; libx262 shares the wrapper, so it links the reader too. - `doc/encoders.texi` documents the option for all three encoders. ## File format - Each record names a 0-based input frame, a frame type and a QP, then one line of offsets per row of 16x16 blocks. - Frame numbers must strictly increase. A frame uses the last record whose number is not greater than its own, and a record without offset lines ends the offsets from that frame on. - Offsets are clipped to -12..12. The frame type and QP are checked but not used. - The reader refuses a read error, a line longer than 1 MiB and text after the last value of a line; trailing spaces, tabs and CRLF line ends are accepted. ## Settings the encoders ignore are refused An offset map the encoder silently drops would look like it worked, so each wrapper refuses the option where the library would ignore it: - libaom implements `AOME_SET_ROI_MAP` from 3.13.0 on, the release that redefined `aom_roi_map_t`. `configure` checks for its `ref_frame` member; with an older libaom the option is refused with "qpfile needs libaom 3.13.0 or newer." - libaom applies the map only in one-pass realtime encoding with `-cpu-used` 7 or 8 and without lookahead. Two-pass encoding, other usages, lower speeds and `-lag-in-frames` above 0 are refused. With libaom 3.15.3 and a lag of 10, every packet came out the same size as without a map. Lag 0 is libaom's realtime default, so a command line without `-lag-in-frames` works. - SVT-AV1 older than 1.6.0 has no ROI map API. - x264 applies `quant_offsets` through adaptive quantization, which is off with constant QP, and with `aq-mode` 0 or `aq-strength` 0 unless MB-tree is on. The check reads the parameters x264 reports after `x264_encoder_open()`. - x264 interlaced encoding rounds the macroblock row count up to even, so the per-row offsets would not match; it is refused. ## Relation to region-of-interest side data `AV_FRAME_DATA_REGIONS_OF_INTEREST` (for example from the `addroi` filter) is read by libx264, libx265, libvpx and the hardware encoders, but not by the libaom-av1 or libsvtav1 wrappers. It describes rectangles with a relative offset per frame, while a qpfile gives one offset per 16x16 block for each frame it names. In libx264 the qpfile offsets are added to those of a region of interest, so both can be used together. ## Testing Built on Linux against libaom 3.15.3, SVT-AV1 4.2.0 and x264 0.165.3222: - Each of the three commits builds on its own with no compiler warnings. - `libaomenc.c` also builds without warnings against the libaom 3.12.1 headers, where `configure` finds no `ref_frame` member and the option takes the refusal path. - `make fate-source` passes. - The series merges onto current master 0f89d936b3 without conflicts. Accepted and refused settings, encoding a 64x64 `testsrc` clip to `-f null` with a three-record qpfile, or with a file that repeats a frame number (16 cases, all as expected): - libaom-av1 with `-usage realtime -cpu-used 8`: accepted without `-lag-in-frames` and with `-lag-in-frames 0`. Refused with `-lag-in-frames 10`, with `-cpu-used 6`, with `-usage good` and with a repeated frame number. `-lag-in-frames 10` without a qpfile is still accepted. - libsvtav1: accepted by default; a repeated frame number is refused. - libx264: accepted with `aq-mode` 1, and with `aq-mode` 0 when MB-tree is on. Refused with `aq-mode` 0 or `aq-strength` 0 without MB-tree, with constant QP, with interlaced encoding and with a repeated frame number. The offsets reach the encoder. Encoding `testsrc2` at 320x240, 10 frames, with one record of all +12 offsets (coarser), with none, and with all -12 offsets (finer): | Encoder and settings | +12 | none | -12 | | :--- | ---: | ---: | ---: | | libaom-av1, realtime, cpu-used 8, crf 30 | 15531 bytes | 25756 bytes | 37154 bytes | | libsvtav1, crf 35 | 15520 bytes | 16570 bytes | 18222 bytes | | libx264, crf 23 | 9690 bytes | 22862 bytes | 43307 bytes | >From e0ca390b45d828ed105f5c23625930ccfeca68b3 Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Sun, 8 Nov 2026 17:01:18 +0100 Subject: [PATCH 1/3] avcodec/libaomenc: add a qpfile option Add a qpfile option that reads per-frame QP offsets from a text file and passes them to libaom as a region-of-interest map through AOME_SET_ROI_MAP, so an external analysis step can steer the quality per 16x16 block. The file format and its parser live in qpfile_parser.c, for the SVT-AV1 and x264 wrappers to share. Each record names a 0-based input frame, strictly increasing, a frame type and a QP, which are checked but not used, followed by one line of offsets per row of 16x16 blocks. A frame uses the last record whose number is not greater than its own. The reader refuses a read error, a line longer than 1 MiB and text after the last value of a line; trailing spaces, tabs and CRLF line ends are accepted. Offsets are clipped to -12..12 and stored as int8_t. libaom applies ROI maps only in one-pass encoding with usage realtime, cpu-used 7 or 8 and no lookahead (lag-in-frames 0, the realtime default), so other settings are refused instead of being ignored. libaom takes at most 8 segments per frame; more distinct offsets are quantized to 8 evenly spaced values. The offsets are AV1 quantizer index deltas. Signed-off-by: lusoris <[email protected]> --- configure | 2 + doc/encoders.texi | 21 ++ libavcodec/Makefile | 2 +- libavcodec/libaomenc.c | 278 ++++++++++++++++++++++++ libavcodec/qpfile_parser.c | 427 +++++++++++++++++++++++++++++++++++++ libavcodec/qpfile_parser.h | 66 ++++++ 6 files changed, 795 insertions(+), 1 deletion(-) create mode 100644 libavcodec/qpfile_parser.c create mode 100644 libavcodec/qpfile_parser.h diff --git a/configure b/configure index ccc46895de0..c7d63e1dfbf 100755 --- a/configure +++ b/configure @@ -2623,6 +2623,7 @@ TYPES_LIST=" SecPkgContext_KeyingMaterialInfo socklen_t struct_addrinfo + struct_aom_roi_map_ref_frame struct_group_source_req struct_ip_mreq_source struct_ipv6_mreq @@ -7219,6 +7220,7 @@ enabled jni && { [ $target_os = "android" ] && check_headers jni.h enabled ladspa && require_headers "ladspa.h dlfcn.h" enabled lcms2 && require_pkg_config lcms2 "lcms2 >= 2.13" lcms2.h cmsCreateContext enabled libaom && require_pkg_config libaom "aom >= 2.0.0" aom/aom_codec.h aom_codec_version +enabled libaom && check_struct aom/aomcx.h "struct aom_roi_map" ref_frame enabled liboapv && require_pkg_config liboapv "oapv >= 0.2.0.0" "oapv/oapv.h" oapve_encode enabled libaribb24 && { check_pkg_config libaribb24 "aribb24 > 1.0.3" "aribb24/aribb24.h" arib_instance_new || { enabled gpl && require_pkg_config libaribb24 aribb24 "aribb24/aribb24.h" arib_instance_new; } || diff --git a/doc/encoders.texi b/doc/encoders.texi index d3487b40a5c..5d48e26f637 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -1974,6 +1974,27 @@ Enable interintra compound. Default is true. @item enable-smooth-interintra (@emph{boolean}) (Requires libaom >= v2.0.0) Enable smooth interintra mode. Default is true. +@item qpfile @var{filename} (Requires libaom >= v3.13.0) +Read per-frame QP offsets from @var{filename} and pass them to libaom as +a region-of-interest map. libaom applies the map only in one-pass encoding +with @option{usage} @samp{realtime}, @option{cpu-used} 7 or 8 and +@option{lag-in-frames} 0, the default for realtime usage; other settings +are refused. + +The file is plain text. Each record starts with a header line +@samp{@var{frame} @var{type} @var{qp}}, where @var{frame} is the 0-based +input frame number, strictly increasing from record to record. +@var{type} is @samp{I}, @samp{P} or @samp{B} in either case, and +@var{qp} is an integer from 0 to 63; both are checked but not used. +Zero or more lines of whitespace-separated integer offsets follow, +one line per row of 16x16 blocks and one value per block column; the grid +must match the video size rounded up to 16 pixels. Offsets are clipped to +the range -12 to 12. A frame uses the last record whose number is not +greater than its own; a record without offset lines, or no record yet, +clears the map. libaom takes at most 8 distinct offsets per frame; when a +frame holds more, they are quantized to 8 evenly spaced values. The +offsets are AV1 quantizer index deltas. + @item aom-params Set libaom options using a list of @var{key}=@var{value} pairs separated by ":". For a list of supported options, see @command{aomenc --help} under the diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 4bffc081184..85086b2ad21 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1185,7 +1185,7 @@ OBJS-$(CONFIG_ILBC_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_PCM_ALAW_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_PCM_MULAW_AT_ENCODER) += audiotoolboxenc.o OBJS-$(CONFIG_LIBAOM_AV1_DECODER) += libaomdec.o libaom.o -OBJS-$(CONFIG_LIBAOM_AV1_ENCODER) += libaomenc.o libaom.o +OBJS-$(CONFIG_LIBAOM_AV1_ENCODER) += libaomenc.o libaom.o qpfile_parser.o OBJS-$(CONFIG_LIBARIBB24_DECODER) += libaribb24.o ass.o OBJS-$(CONFIG_LIBARIBCAPTION_DECODER) += libaribcaption.o ass.o OBJS-$(CONFIG_LIBASTCENC_ENCODER) += libastcenc.o diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c index fe68f1f953a..d3f6d6517c3 100644 --- a/libavcodec/libaomenc.c +++ b/libavcodec/libaomenc.c @@ -50,6 +50,7 @@ #include "itut35.h" #include "libaom.h" #include "profiles.h" +#include "qpfile_parser.h" /* * Portion of struct aom_codec_cx_pkt from aom_encoder.h. @@ -69,6 +70,12 @@ struct FrameListData { struct FrameListData *next; }; +/* libaom 3.15.3 applies ROI maps only in realtime usage at cpu-used >= 7 + * without lookahead (measured: lower speeds, good usage and a nonzero + * lag-in-frames ignore the map content). */ +#define QPFILE_AOM_USAGE_REALTIME 1 /* AOM_USAGE_REALTIME */ +#define QPFILE_AOM_MIN_CPU_USED 7 + typedef struct AOMEncoderContext { AVClass *class; AVBSFContext *bsf; @@ -110,6 +117,31 @@ typedef struct AOMEncoderContext { int enable_global_motion; int enable_intrabc; int enable_restoration; + char *qpfile_path; + + /* qpfile -> libaom AOME_SET_ROI_MAP ROI map. + * Populated by ff_qpfile_load() in aom_init() when qpfile_path is set, + * consumed per-frame in aom_encode(), released in aom_free(). + * + * libaom requires the segment-id map to match its mode-info grid + * exactly: rows == mi_rows, cols == mi_cols (4x4-luma cells; see + * av1/encoder/encoder.c::av1_set_roi_map). The qpfile rows are at + * 16x16-MB granularity, so each MB cell expands into a 4x4 block of + * mi cells. delta_q[8] holds up to AOM_MAX_SEGMENTS distinct QP + * deltas; per-frame qp_offsets are quantised into <=8 segment ids + * (uniform binning when the value span exceeds the segment budget). + * + * roi_seg_map is allocated once at the encoder grid size; libaom + * deep-copies it on every aom_codec_control() call, so the buffer + * can be reused across frames. + */ + FFQpFile qpfile; + int qpfile_loaded; + uint8_t *roi_seg_map; + unsigned int roi_mi_rows; + unsigned int roi_mi_cols; + unsigned int roi_input_frame; /* 0-based coded-frame ordinal for qpfile lookup */ + int usage; int tune; int still_picture; @@ -506,6 +538,14 @@ static av_cold int aom_free(AVCodecContext *avctx) aom_codec_destroy(&ctx->encoder); aom_img_remove_metadata(&ctx->rawimg); + if (ctx->qpfile_loaded) { + ff_qpfile_free(&ctx->qpfile); + ctx->qpfile_loaded = 0; + } + av_freep(&ctx->roi_seg_map); + ctx->roi_mi_rows = 0; + ctx->roi_mi_cols = 0; + ctx->roi_input_frame = 0; av_freep(&ctx->twopass_stats.buf); av_freep(&avctx->stats_out); free_frame_list(ctx->coded_frame_list); @@ -797,6 +837,24 @@ static const struct { { AV1E_SET_ENABLE_SMOOTH_INTERINTRA, OFFSET(enable_smooth_interintra) }, }; +#if HAVE_STRUCT_AOM_ROI_MAP_REF_FRAME +/* Load the QP file and check its grid, returning 0 or a negative error code. */ +static av_cold int load_qpfile(AVCodecContext *avctx) +{ + AOMContext *ctx = avctx->priv_data; + int ret = ff_qpfile_load(avctx, ctx->qpfile_path, &ctx->qpfile); + + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "cannot read qpfile %s: %s\n", + ctx->qpfile_path, av_err2str(ret)); + return ret; + } + ctx->qpfile_loaded = 1; + return ff_qpfile_check_grid(avctx, "libaom", &ctx->qpfile, + avctx->width, avctx->height); +} +#endif + static av_cold int aom_config(AVCodecContext *avctx, const struct aom_codec_iface *iface) { @@ -1072,6 +1130,54 @@ static av_cold int aom_init(AVCodecContext *avctx, if (res < 0) return res; + if (ctx->qpfile_path && *ctx->qpfile_path) { +#if HAVE_STRUCT_AOM_ROI_MAP_REF_FRAME + if (avctx->flags & (AV_CODEC_FLAG_PASS1 | AV_CODEC_FLAG_PASS2)) { + av_log(avctx, AV_LOG_ERROR, + "qpfile needs one-pass encoding: libaom applies " + "ROI maps only without recode.\n"); + return AVERROR(EINVAL); + } + if (ctx->usage != QPFILE_AOM_USAGE_REALTIME || ctx->cpu_used < QPFILE_AOM_MIN_CPU_USED) { + av_log(avctx, AV_LOG_ERROR, + "qpfile with libaom-av1 needs -usage realtime " + "-cpu-used 7 or 8; libaom ignores ROI maps otherwise.\n"); + return AVERROR(EINVAL); + } + if (ctx->enccfg.g_lag_in_frames) { + av_log(avctx, AV_LOG_ERROR, + "qpfile with libaom-av1 needs -lag-in-frames 0; " + "libaom ignores ROI maps with lookahead.\n"); + return AVERROR(EINVAL); + } + res = load_qpfile(avctx); + if (res < 0) + return res; + /* libaom's segmentation map is sized at the mode-info grid: + * size_in_mi(x) == ALIGN_POWER_OF_TWO(x, 3) >> MI_SIZE_LOG2, + * i.e. each dim aligned up to a multiple of 8 px and divided + * by 4. Mirror that here so av1_set_roi_map's (rows, cols) + * equality check passes. */ + ctx->roi_mi_cols = ((unsigned)((avctx->width + 7) & ~7)) >> 2; + ctx->roi_mi_rows = ((unsigned)((avctx->height + 7) & ~7)) >> 2; + ctx->roi_seg_map = av_calloc((size_t)ctx->roi_mi_rows * + ctx->roi_mi_cols, + sizeof(*ctx->roi_seg_map)); + if (!ctx->roi_seg_map) + return AVERROR(ENOMEM); + av_log(avctx, AV_LOG_VERBOSE, + "qpfile=%s loaded (frames=%u, %ux%u qpfile blocks " + "-> %ux%u mi grid).\n", + ctx->qpfile_path, ctx->qpfile.frame_count, + ctx->qpfile.block_cols, ctx->qpfile.block_rows, + ctx->roi_mi_cols, ctx->roi_mi_rows); +#else + av_log(avctx, AV_LOG_ERROR, + "qpfile needs libaom 3.13.0 or newer.\n"); + return AVERROR(ENOSYS); +#endif + } + ctx->dovi.logctx = avctx; if ((res = ff_dovi_configure(&ctx->dovi, avctx)) < 0) return res; @@ -1324,6 +1430,162 @@ static enum AVPixelFormat aomfmt_to_pixfmt(struct aom_image *img) return AV_PIX_FMT_NONE; } +#if HAVE_STRUCT_AOM_ROI_MAP_REF_FRAME +/* libaom-av1 ROI map helpers. Convert one FFQpFrame into a + * libaom aom_roi_map_t whose (rows, cols, roi_map, delta_q) match the + * encoder's mode-info grid (4x4-luma cells). + * + * Quantisation: per-MB qp_offsets are clipped to [-12, +12] by the + * shared parser. Per frame, we discover the actual value range and + * either emit one segment per distinct value (if the span is <= + * AOM_MAX_SEGMENTS), or do uniform linear binning across the span + * (lossy: each MB rounds to its nearest segment QP). For libaom's + * 8-segment limit this matches the binning policy used by the + * SVT-AV1 ROI map. */ + +#define QPFILE_AOM_MAX_SEGMENTS AOM_MAX_SEGMENTS /* 8, see aomcx.h */ +#define QPFILE_AOM_MI_PER_MB 4 /* 16 px MB / 4 px mi */ + +/* Pick up to 8 segment QP deltas for a frame, returning 0 or an error. */ +static int qpfile_aom_pick_segments(const FFQpFrame *qf, + int seg_q[QPFILE_AOM_MAX_SEGMENTS], + unsigned *out_n_seg) +{ + unsigned i, j; + unsigned mb_rows = qf->block_rows; + unsigned mb_cols = qf->block_cols; + int min_off = QPFILE_DELTA_MAX, max_off = QPFILE_DELTA_MIN; + unsigned n = 0; + + if (!qf->block_offsets || !mb_rows || !mb_cols) + return AVERROR(EINVAL); + + for (i = 0; i < mb_rows; i++) { + for (j = 0; j < mb_cols; j++) { + int v = qf->block_offsets[i * mb_cols + j]; + min_off = FFMIN(min_off, v); + max_off = FFMAX(max_off, v); + } + } + + if (max_off == min_off) { + seg_q[0] = min_off; + n = 1; + } else { + int span = max_off - min_off + 1; + if (span <= QPFILE_AOM_MAX_SEGMENTS) { + for (int v = min_off; v <= max_off; v++) + seg_q[n++] = v; + } else { + for (i = 0; i < QPFILE_AOM_MAX_SEGMENTS; i++) { + seg_q[i] = min_off + + (int)((unsigned long)(max_off - min_off) * i / + (QPFILE_AOM_MAX_SEGMENTS - 1u)); + } + n = QPFILE_AOM_MAX_SEGMENTS; + } + } + *out_n_seg = n; + return 0; +} + +/* Give each mode-info cell the segment nearest its macroblock's QP offset. */ +static void qpfile_aom_paint_seg_map(const FFQpFrame *qf, + uint8_t *seg_map, + unsigned mi_rows, unsigned mi_cols, + const int *seg_q, unsigned n_seg) +{ + unsigned mi_y, mi_x; + unsigned mb_rows = qf->block_rows; + unsigned mb_cols = qf->block_cols; + /* For each mi cell, pick the qpfile MB it overlaps, then snap that + * MB's QP-offset to the nearest segment id. mi index / 4 = mb idx. */ + for (mi_y = 0; mi_y < mi_rows; mi_y++) { + unsigned mb_y = mi_y / QPFILE_AOM_MI_PER_MB; + mb_y = FFMIN(mb_y, mb_rows - 1u); + for (mi_x = 0; mi_x < mi_cols; mi_x++) { + unsigned mb_x = mi_x / QPFILE_AOM_MI_PER_MB; + int v, best_dist = INT_MAX, best_idx = 0; + unsigned k; + mb_x = FFMIN(mb_x, mb_cols - 1u); + v = qf->block_offsets[mb_y * mb_cols + mb_x]; + for (k = 0; k < n_seg; k++) { + int d = v - seg_q[k]; + d = FFABS(d); + if (d < best_dist) { + best_dist = d; + best_idx = (int)k; + } + } + seg_map[mi_y * mi_cols + mi_x] = (uint8_t)best_idx; + } + } +} + +/* Send an ROI map to libaom via AOME_SET_ROI_MAP, returning 0 or an error. */ +static int qpfile_aom_send_roi(AVCodecContext *avctx, aom_roi_map_t *roi, + unsigned frame_index) +{ + AOMContext *ctx = avctx->priv_data; + int res = aom_codec_control(&ctx->encoder, AOME_SET_ROI_MAP, roi); + + if (res != AOM_CODEC_OK) { + av_log(avctx, AV_LOG_ERROR, + "AOME_SET_ROI_MAP failed for frame %u (res=%d).\n", + frame_index, res); + return AVERROR_EXTERNAL; + } + return 0; +} + +/* Build and apply the ROI map for a frame, returning 0 or a negative error. */ +static int qpfile_aom_apply_roi(AVCodecContext *avctx, unsigned frame_index) +{ + AOMContext *ctx = avctx->priv_data; + aom_roi_map_t roi; + const FFQpFrame *qf; + int seg_q[QPFILE_AOM_MAX_SEGMENTS] = { 0 }; + unsigned n_seg = 0; + int qret; + unsigned i; + + if (!ctx->qpfile_loaded || !ctx->roi_seg_map || + !ctx->roi_mi_rows || !ctx->roi_mi_cols) + return 0; + + qf = ff_qpfile_frame_for_index(&ctx->qpfile, frame_index); + + memset(&roi, 0, sizeof(roi)); + roi.rows = ctx->roi_mi_rows; + roi.cols = ctx->roi_mi_cols; + for (i = 0; i < QPFILE_AOM_MAX_SEGMENTS; i++) + roi.ref_frame[i] = -1; + + if (!qf || !qf->block_offsets || !qf->block_rows || !qf->block_cols) + return qpfile_aom_send_roi(avctx, &roi, frame_index); + + qret = qpfile_aom_pick_segments(qf, seg_q, &n_seg); + if (qret < 0 || !n_seg) + return qret; + + qpfile_aom_paint_seg_map(qf, ctx->roi_seg_map, + ctx->roi_mi_rows, ctx->roi_mi_cols, + seg_q, n_seg); + + roi.enabled = 1; + roi.roi_map = ctx->roi_seg_map; + for (i = 0; i < QPFILE_AOM_MAX_SEGMENTS; i++) { + roi.delta_q[i] = (i < n_seg) ? seg_q[i] : 0; + roi.delta_lf[i] = 0; + roi.skip[i] = 0; + roi.ref_frame[i] = -1; + } + roi.delta_qp_enabled = 1; + + return qpfile_aom_send_roi(avctx, &roi, frame_index); +} +#endif + static int aom_encode(AVCodecContext *avctx, AVPacket *pkt, const AVFrame *frame, int *got_packet) { @@ -1397,6 +1659,14 @@ static int aom_encode(AVCodecContext *avctx, AVPacket *pkt, return res; } +#if HAVE_STRUCT_AOM_ROI_MAP_REF_FRAME + if (ctx->qpfile_loaded && frame) { + int qret = qpfile_aom_apply_roi(avctx, ctx->roi_input_frame++); + if (qret < 0) + return qret; + } +#endif + res = aom_codec_encode(&ctx->encoder, rawimg, timestamp, duration, flags); if (res != AOM_CODEC_OK) { log_encoder_error(avctx, "Error encoding frame"); @@ -1559,6 +1829,13 @@ static av_cold int av1_reconf(AVCodecContext *avctx, AVDictionary **dict) if (res < 0) return res; + if (ctx->qpfile_loaded && ctx->cpu_used < QPFILE_AOM_MIN_CPU_USED) { + av_log(avctx, AV_LOG_ERROR, + "qpfile with libaom-av1 needs -cpu-used 7 or 8; " + "libaom ignores ROI maps otherwise.\n"); + return AVERROR(EINVAL); + } + res = aom_config(avctx, ctx->encoder.iface); if (res < 0) return res; @@ -1621,6 +1898,7 @@ static const AVOption options[] = { { "tune", "The metric that the encoder tunes for. Automatically chosen by the encoder by default", OFFSET(tune), AV_OPT_TYPE_INT, {.i64 = -1}, -1, AOM_TUNE_SSIM, VE, .unit = "tune"}, { "psnr", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, .unit = "tune"}, { "ssim", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, .unit = "tune"}, + { "qpfile", "Read per-frame QP offsets from a file as a region-of-interest map (needs libaom 3.13.0 or newer)", OFFSET(qpfile_path), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, VE }, FF_AV1_PROFILE_OPTS { "still-picture", "Encode in single frame mode (typically used for still AVIF images).", OFFSET(still_picture), AV_OPT_TYPE_BOOL, {.i64 = 0}, -1, 1, VE }, { "dolbyvision", "Enable Dolby Vision RPU coding", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = FF_DOVI_AUTOMATIC }, -1, 1, VE, .unit = "dovi" }, diff --git a/libavcodec/qpfile_parser.c b/libavcodec/qpfile_parser.c new file mode 100644 index 00000000000..e285c38ab6c --- /dev/null +++ b/libavcodec/qpfile_parser.c @@ -0,0 +1,427 @@ +/* + * Copyright 2026 Lusoris + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or modify it under + * the terms of the GNU Lesser General Public License as published by the + * Free Software Foundation; either version 2.1 of the License, or (at + * your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + */ + +/** + * @file + * Shared parser for the per-frame QP-offset file format consumed by libx264, + * libsvtav1, and libaom-av1 encoder adapters. The format consists of + * <frame_idx> <I|P|B> <qp> followed by per-MB-row deltas. + * + * Format (ASCII, one logical record per frame): + * + * <frame_idx> <I|P|B> <baseline_qp> + * <delta_0_0> <delta_0_1> ... <delta_0_(bw-1)> + * ... + * + * Lines are read with fgetc(), not POSIX getline(). + */ + +#include <errno.h> +#include <limits.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +#include "libavutil/common.h" +#include "libavutil/error.h" +#include "libavutil/file_open.h" +#include "libavutil/log.h" +#include "libavutil/macros.h" +#include "libavutil/mem.h" + +#include "qpfile_parser.h" + +#define QPFILE_MAX_FRAMES (1u << 20) +#define QPFILE_MAX_BLOCKS_PER_DIM (1u << 14) +#define QPFILE_MAX_LINE_BYTES (1u << 20) + +/* Parse a base-10 long integer from a string, returning 0 or an error code. */ +static int parse_long(const char *s, char **end, long *out) +{ + long v; + errno = 0; + v = strtol(s, end, 10); + if (errno == ERANGE) + return AVERROR(ERANGE); + if (*end == s) + return AVERROR(EINVAL); + *out = v; + return 0; +} + +/* Clamp a QP delta value to the allowed range, returning the clamped value. */ +static int clamp_offset(long v) +{ + return av_clip64(v, QPFILE_DELTA_MIN, QPFILE_DELTA_MAX); +} + +/* Test whether a line matches the frame header format, returning 1 or 0. */ +static int looks_like_header(const char *line) +{ + const char *p = line; + while (*p == ' ' || *p == '\t') + p++; + if (*p < '0' || *p > '9') + return 0; + while (*p >= '0' && *p <= '9') + p++; + while (*p == ' ' || *p == '\t') + p++; + return (*p == 'I' || *p == 'P' || *p == 'B' || + *p == 'i' || *p == 'p' || *p == 'b'); +} + +/* Return 1 if p contains only whitespace (' ', '\t', '\r', '\n') up to '\0', else 0. */ +static int only_space(const char *p) +{ + while (*p) { + if (*p != ' ' && *p != '\t' && *p != '\r' && *p != '\n') + return 0; + p++; + } + return 1; +} + +/* Parse a frame header line into an FFQpFrame, returning 0 or an error code. */ +static int parse_header(const char *line, FFQpFrame *f) +{ + char *p = NULL; + long idx = 0, qp = 0; + char kind = 0; + const char *cur = line; + int err; + + err = parse_long(cur, &p, &idx); + if (err < 0) + return err; + if (idx < 0 || (unsigned long)idx > QPFILE_MAX_FRAMES) + return AVERROR(EINVAL); + cur = p; + while (*cur == ' ' || *cur == '\t') + cur++; + kind = *cur; + if (kind != 'I' && kind != 'P' && kind != 'B' && + kind != 'i' && kind != 'p' && kind != 'b') + return AVERROR(EINVAL); + cur++; + while (*cur == ' ' || *cur == '\t') + cur++; + err = parse_long(cur, &p, &qp); + if (err < 0) + return err; + if (qp < 0 || qp > 63) + return AVERROR(EINVAL); + if (!only_space(p)) + return AVERROR(EINVAL); + + f->frame_index = (unsigned)idx; + f->block_offsets = NULL; + f->block_cols = 0; + f->block_rows = 0; + return 0; +} + +/* Count whitespace-separated tokens on a line, returning the token count. */ +static unsigned count_tokens(const char *line) +{ + unsigned n = 0; + int in_tok = 0; + for (const char *p = line; *p; p++) { + if (*p == ' ' || *p == '\t' || *p == '\r' || *p == '\n') { + in_tok = 0; + } else if (!in_tok) { + in_tok = 1; + n++; + if (n > QPFILE_MAX_BLOCKS_PER_DIM) + return n; + } + } + return n; +} + +/* Parse a row of clamped block QP deltas, returning 0 or a negative error. */ +static int parse_row(const char *line, int8_t *row_out, unsigned bw) +{ + char *p = NULL; + const char *cur = line; + long v = 0; + for (unsigned i = 0; i < bw; i++) { + int err; + err = parse_long(cur, &p, &v); + if (err < 0) + return err; + row_out[i] = (int8_t)clamp_offset(v); + cur = p; + while (*cur == ' ' || *cur == '\t') + cur++; + } + if (!only_space(cur)) + return AVERROR(EINVAL); + return 0; +} + +/* Portable line reader (no POSIX getline dependency). */ +static int qpfile_getline(char **buf, size_t *cap, FILE *fp, ptrdiff_t *out_len) +{ + size_t pos = 0; + + if (!*cap || !*buf) { + *cap = 4096; + *buf = av_realloc(*buf, *cap); + if (!*buf) + return AVERROR(ENOMEM); + } + while (1) { + if (pos + 1 >= *cap) { + size_t nc = FFMIN((*cap) * 2u, (size_t)QPFILE_MAX_LINE_BYTES + 1); + char *nb = av_realloc(*buf, nc); + if (!nb) + return AVERROR(ENOMEM); + *buf = nb; + *cap = nc; + } + int c = fgetc(fp); + if (c == EOF) { + if (ferror(fp)) + return AVERROR(EIO); + if (!pos) { + *out_len = -1; + return 0; + } + (*buf)[pos] = 0; + *out_len = (ptrdiff_t)pos; + return 0; + } + if (pos >= QPFILE_MAX_LINE_BYTES) + return AVERROR(EINVAL); + (*buf)[pos++] = (char)c; + if (c == '\n') { + (*buf)[pos] = 0; + *out_len = (ptrdiff_t)pos; + return 0; + } + } +} + +typedef struct QpLoadState { + void *log_ctx; + unsigned bw, bh, cap_frames, row_count, line_number; + int8_t *row_buf; + FFQpFrame *cur_frame; +} QpLoadState; + +/* Finalise the current frame's block buffer, returning 0 or an error code. */ +static int qpfile_finish_frame(QpLoadState *st) +{ + FFQpFrame *f = st->cur_frame; + if (f && st->row_count > 0) { + if (!st->bw) { + st->bw = f->block_cols; + st->bh = st->row_count; + } else if (st->row_count != st->bh) { + return AVERROR(EINVAL); + } + f->block_offsets = av_memdup(st->row_buf, (size_t)st->bh * st->bw * sizeof(*st->row_buf)); + if (!f->block_offsets) + return AVERROR(ENOMEM); + f->block_rows = st->bh; + f->block_cols = st->bw; + } + return 0; +} + +/* Append a new frame from a header line, returning 0 or a negative error. */ +static int qpfile_add_header(FFQpFile *out, QpLoadState *st, const char *line) +{ + int ret = qpfile_finish_frame(st); + + if (ret < 0) + return ret; + if (out->frame_count == st->cap_frames) { + unsigned nc = st->cap_frames * 2u; + FFQpFrame *nf; + if (nc > QPFILE_MAX_FRAMES) + return AVERROR(EINVAL); + nf = av_realloc_array(out->frames, nc, sizeof(*nf)); + if (!nf) + return AVERROR(ENOMEM); + memset(nf + st->cap_frames, 0, (nc - st->cap_frames) * sizeof(*nf)); + out->frames = nf; + st->cap_frames = nc; + } + st->cur_frame = &out->frames[out->frame_count]; + ret = parse_header(line, st->cur_frame); + if (ret < 0) + return ret; + if (out->frame_count > 0) { + FFQpFrame *previous; + previous = &out->frames[out->frame_count - 1u]; + if (st->cur_frame->frame_index <= previous->frame_index) { + av_log(st->log_ctx, AV_LOG_ERROR, + "qpfile: line %u frame index %u is not greater than " + "previous index %u\n", st->line_number, + st->cur_frame->frame_index, previous->frame_index); + return AVERROR(EINVAL); + } + } + out->frame_count++; + st->row_count = 0; + return 0; +} + +/* Parse and append a block delta row to the frame, returning 0 or an error. */ +static int qpfile_add_row(QpLoadState *st, const char *line) +{ + unsigned tokens, this_bw, needed; + int8_t *nrow; + int ret; + if (!st->cur_frame) + return AVERROR(EINVAL); + tokens = count_tokens(line); + if (!tokens) + return 0; + if (tokens > QPFILE_MAX_BLOCKS_PER_DIM) + return AVERROR(EINVAL); + if (!st->cur_frame->block_cols) { + st->cur_frame->block_cols = tokens; + if (st->bw && tokens != st->bw) + return AVERROR(EINVAL); + } else if (tokens != st->cur_frame->block_cols) { + return AVERROR(EINVAL); + } + this_bw = st->cur_frame->block_cols; + needed = (st->row_count + 1) * this_bw; + nrow = av_realloc_array(st->row_buf, needed, sizeof(*st->row_buf)); + if (!nrow) + return AVERROR(ENOMEM); + st->row_buf = nrow; + ret = parse_row(line, &st->row_buf[st->row_count * this_bw], this_bw); + if (ret < 0) + return ret; + st->row_count++; + if (st->row_count > QPFILE_MAX_BLOCKS_PER_DIM) + return AVERROR(EINVAL); + return 0; +} + +/* Parse every frame header and row line in a file, returning 0 or an error. */ +static int qpfile_read_frames(FILE *fp, FFQpFile *out, QpLoadState *st, + char **line, size_t *line_cap) +{ + ptrdiff_t got = 0; + while (got >= 0) { + int ret; + ret = qpfile_getline(line, line_cap, fp, &got); + if (ret < 0) + return ret; + if (got < 0) + break; + st->line_number++; + if (!got || (*line)[0] == '#' || (*line)[0] == '\n' || (*line)[0] == '\r') + continue; + if (looks_like_header(*line)) + ret = qpfile_add_header(out, st, *line); + else + ret = qpfile_add_row(st, *line); + if (ret < 0) + return ret; + } + return qpfile_finish_frame(st); +} + +int ff_qpfile_load(void *log_ctx, const char *path, FFQpFile *out) +{ + FILE *fp; + char *line = NULL; + size_t line_cap = 0; + QpLoadState st = { 0 }; + int ret; + + if (!path || !out) + return AVERROR(EINVAL); + memset(out, 0, sizeof(*out)); + st.log_ctx = log_ctx; + fp = avpriv_fopen_utf8(path, "rb"); + if (!fp) + return AVERROR(errno ? errno : ENOENT); + st.cap_frames = 16; + out->frames = av_calloc(st.cap_frames, sizeof(*out->frames)); + if (!out->frames) + ret = AVERROR(ENOMEM); + else + ret = qpfile_read_frames(fp, out, &st, &line, &line_cap); + av_freep(&st.row_buf); + if (ret >= 0) { + out->block_cols = st.bw; + out->block_rows = st.bh; + if (!out->frame_count) + ret = AVERROR(EINVAL); + } + av_free(line); + fclose(fp); + if (ret < 0) + ff_qpfile_free(out); + return ret; +} + +int ff_qpfile_check_grid(void *log_ctx, const char *encoder, const FFQpFile *qf, + int width, int height) +{ + const unsigned mbx = (width + 15) / 16; + const unsigned mby = (height + 15) / 16; + + if (qf->block_cols != mbx || qf->block_rows != mby) { + av_log(log_ctx, AV_LOG_ERROR, + "%s: the qpfile has %ux%u blocks per frame, the %dx%d video has %ux%u " + "macroblocks.\n", encoder, qf->block_cols, qf->block_rows, + width, height, mbx, mby); + return AVERROR(EINVAL); + } + return 0; +} + +void ff_qpfile_free(FFQpFile *qpf) +{ + if (!qpf) + return; + if (qpf->frames) { + for (unsigned i = 0; i < qpf->frame_count; i++) + av_freep(&qpf->frames[i].block_offsets); + av_freep(&qpf->frames); + } + qpf->frame_count = 0; + qpf->block_cols = 0; + qpf->block_rows = 0; +} + +const FFQpFrame *ff_qpfile_frame_for_index(const FFQpFile *qf, + unsigned frame_index) +{ + unsigned low = 0, high; + + if (!qf || !qf->frame_count) + return NULL; + high = qf->frame_count; + while (low < high) { + unsigned mid = low + (high - low) / 2; + + if (qf->frames[mid].frame_index <= frame_index) + low = mid + 1; + else + high = mid; + } + return low ? &qf->frames[low - 1] : NULL; +} diff --git a/libavcodec/qpfile_parser.h b/libavcodec/qpfile_parser.h new file mode 100644 index 00000000000..38c84252920 --- /dev/null +++ b/libavcodec/qpfile_parser.h @@ -0,0 +1,66 @@ +/* + * Copyright 2026 Lusoris + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or modify it under + * the terms of the GNU Lesser General Public License as published by the + * Free Software Foundation; either version 2.1 of the License, or (at + * your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + */ + +/** + * @file + * Shared parser for the per-frame QP-offset file format. Consumed by libx264, + * libsvtav1, and libaom-av1 encoder adapters via the new -qpfile + * AVOption. + * + * Keep the parser shared across encoder adapters so future + * encoder additions inherit the same format handling and + * validation rules. + * + * The parsed block offsets are QP units for libx264. libsvtav1's seg_qp and + * libaom's delta_q use AV1 qindex deltas (0..255, with libaom limiting each + * delta to +/-63), so the same value produces a weaker adjustment on AV1. + */ + +#ifndef AVCODEC_QPFILE_PARSER_H +#define AVCODEC_QPFILE_PARSER_H + +#include <stddef.h> +#include <stdint.h> + +#define QPFILE_DELTA_MIN (-12) +#define QPFILE_DELTA_MAX (12) + +typedef struct FFQpFrame { + unsigned int frame_index; + int8_t *block_offsets; + unsigned int block_cols; + unsigned int block_rows; +} FFQpFrame; + +typedef struct FFQpFile { + FFQpFrame *frames; + unsigned int frame_count; + unsigned int block_cols; + unsigned int block_rows; +} FFQpFile; + +/** Load a per-frame QP-offset file into out, logging errors to log_ctx. */ +int ff_qpfile_load(void *log_ctx, const char *path, FFQpFile *out); +/** Verify that the QP file grid matches the video macroblock dimensions. */ +int ff_qpfile_check_grid(void *log_ctx, const char *encoder, const FFQpFile *qf, + int width, int height); +/** Free all memory allocated inside an FFQpFile structure. */ +void ff_qpfile_free(FFQpFile *qpf); +/** Look up the QP frame record matching or preceding the given frame index. */ +const FFQpFrame *ff_qpfile_frame_for_index(const FFQpFile *qf, + unsigned frame_index); + +#endif /* AVCODEC_QPFILE_PARSER_H */ -- 2.54.0 >From 75cab9b20c50668406071be32aa9b1c73de3f36e Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Sun, 8 Nov 2026 17:01:29 +0100 Subject: [PATCH 2/3] avcodec/libsvtav1: add a qpfile option Add the qpfile option of libaom-av1 to libsvtav1, with the same file format, and pass each record to SVT-AV1 as a region-of-interest map event. An event is sent only on the frames that have a record; a record without offset lines sends a neutral map, so the previous map ends there. The offsets are AV1 quantizer index deltas. SVT-AV1 has the ROI map API from 1.6.0 on; built against an older SVT-AV1, the option is refused. Signed-off-by: lusoris <[email protected]> --- doc/encoders.texi | 8 + libavcodec/Makefile | 2 +- libavcodec/libsvtav1.c | 338 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 347 insertions(+), 1 deletion(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 5d48e26f637..dc267f5e3cb 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2109,6 +2109,14 @@ Set log2 of the number of rows of tiles to use (0-6). @item tile_columns Set log2 of the number of columns of tiles to use (0-4). +@item qpfile @var{filename} +Read per-frame QP offsets from @var{filename}, in the format described for +the libaom-av1 @option{qpfile} option, and pass them to SVT-AV1 as +region-of-interest map events. An event is sent only on the frames that +have a record; a record without offset lines sends a neutral map. The +offsets are AV1 quantizer index deltas. The option is refused when FFmpeg +is built against SVT-AV1 older than 1.6.0. + @item svtav1-params Set SVT-AV1 options using a list of @var{key}=@var{value} pairs separated by ":". See the SVT-AV1 encoder user guide for a list of accepted parameters. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 85086b2ad21..dbfebc61b8d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1226,7 +1226,7 @@ OBJS-$(CONFIG_LIBRSVG_DECODER) += librsvgdec.o OBJS-$(CONFIG_LIBSHINE_ENCODER) += libshine.o OBJS-$(CONFIG_LIBSPEEX_DECODER) += libspeexdec.o OBJS-$(CONFIG_LIBSPEEX_ENCODER) += libspeexenc.o -OBJS-$(CONFIG_LIBSVTAV1_ENCODER) += libsvtav1.o +OBJS-$(CONFIG_LIBSVTAV1_ENCODER) += libsvtav1.o qpfile_parser.o OBJS-$(CONFIG_LIBSVTJPEGXS_DECODER) += libsvtjpegxsdec.o OBJS-$(CONFIG_LIBSVTJPEGXS_ENCODER) += libsvtjpegxsenc.o OBJS-$(CONFIG_LIBTHEORA_ENCODER) += libtheoraenc.o diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c index 1e5732f45d6..3e1d778278b 100644 --- a/libavcodec/libsvtav1.c +++ b/libavcodec/libsvtav1.c @@ -20,6 +20,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include <limits.h> #include <stdint.h> #include <inttypes.h> #include <EbSvtAv1ErrorCodes.h> @@ -43,6 +44,7 @@ #include "encode.h" #include "avcodec.h" #include "profiles.h" +#include "qpfile_parser.h" typedef enum eos_status { EOS_NOT_REACHED = 0, @@ -75,6 +77,31 @@ typedef struct SvtContext { int enc_mode; int crf; int qp; + char *qpfile_path; + + /* qpfile state. Populated by ff_qpfile_load() in + * eb_enc_init() when qpfile_path is set, consumed per-frame in + * eb_send_frame(), released in eb_enc_close(). + * The ROI map requires the versioned ROI ABI and is + * gated on the SVT-AV1 1.6.0+ ROI ABI (enable_roi_map flag and + * the ROI_MAP_EVENT priv-data type); older releases reject qpfile. + * + * roi_evts / roi_seg_maps are heap-owned arrays parallel to + * qpfile.frames; SVT-AV1 keeps the ROI-map pointer alive across + * the asynchronous encode pipeline (copy_private_data_list shallow- + * copies ROI_MAP_EVENT data), so we must preserve these until + * eb_enc_close(). + */ + FFQpFile qpfile; + int qpfile_loaded; +#if SVT_AV1_CHECK_VERSION(1, 6, 0) + SvtAv1RoiMapEvt *roi_evts; + uint8_t **roi_seg_maps; + unsigned int roi_evt_count; + unsigned int roi_b64_cols; + unsigned int roi_b64_rows; + unsigned int roi_input_frame; /* 0-based coded-frame ordinal for qpfile lookup */ +#endif } SvtContext; static const struct { @@ -482,6 +509,237 @@ static int read_in_data(EbSvtAv1EncConfiguration *param, const AVFrame *frame, return 0; } +#if SVT_AV1_CHECK_VERSION(1, 6, 0) +/* Translate a per-frame QP-offset file into an array of per-frame + * SvtAv1RoiMapEvt records. + * + * Strategy: + * 1. Per frame, discover the QP-offset value range; pick up to + * QPFILE_SVTAV1_MAX_SEGMENTS distinct segment QPs (uniform + * binning when the value span exceeds the segment budget). + * 2. Downsample the qpfile's per-16x16-MB grid to SVT-AV1's + * per-64x64-SB grid by averaging the four MBs that overlap + * each SB and assigning the nearest segment id. + * 3. Size b64_seg_map at the encoder's actual SB grid stride + * ((max_input_luma_width+63)/64) so segmentation.c can index + * it correctly even if the qpfile was cut against a smaller + * preview. + * + * Per-SB rectangles fall out naturally -- SVT-AV1 consumes a dense + * per-SB segment map, so explicit rectangle decomposition would + * be redundant. Trade-off: per-MB qpfile resolution is rounded + * to per-SB granularity, losing spatial detail finer than 64x64 + * px (a salient face smaller than that gets averaged with its + * surroundings). Suitable for object-sized ROIs generated by + * an external tool; finer granularity would require an SVT-AV1 ABI + * change. + * + * Lifetime: the library shallow-copies ROI_MAP_EVENT pointers + * (copy_private_data_list in enc_handle.c) and reads them later + * on async pipeline threads, so the SvtAv1RoiMapEvt and its + * b64_seg_map must outlive svt_av1_enc_send_picture() -- they + * are owned by SvtContext and freed in eb_enc_close(). + */ +#define QPFILE_SVTAV1_MAX_SEGMENTS 8 +#define QPFILE_SVTAV1_SB_SIDE 4 /* 4 MBs (16 px each) per SB (64 px) */ + +/* Pick up to 8 segment QP deltas for a frame, returning the segment count. */ +static unsigned roi_segment_values(const FFQpFrame *qf, int *seg_values) +{ + unsigned mb_cols = qf->block_cols, mb_rows = qf->block_rows; + unsigned i, j, n_seg = 0; + int min_off = QPFILE_DELTA_MAX, max_off = QPFILE_DELTA_MIN; + + /* Pass 1: discover the value range. */ + for (i = 0; i < mb_rows; i++) { + for (j = 0; j < mb_cols; j++) { + int v = qf->block_offsets[i * mb_cols + j]; + min_off = FFMIN(min_off, v); + max_off = FFMAX(max_off, v); + } + } + + /* Pass 2: pick segment QPs. */ + if (max_off == min_off) { + seg_values[0] = min_off; + n_seg = 1; + } else { + int span = max_off - min_off + 1; + if (span <= QPFILE_SVTAV1_MAX_SEGMENTS) { + for (int v = min_off; v <= max_off; v++) + seg_values[n_seg++] = v; + } else { + for (i = 0; i < QPFILE_SVTAV1_MAX_SEGMENTS; i++) { + int v = min_off + + (int)((unsigned long)(max_off - min_off) * i / + (QPFILE_SVTAV1_MAX_SEGMENTS - 1u)); + seg_values[i] = v; + } + n_seg = QPFILE_SVTAV1_MAX_SEGMENTS; + } + } + + return n_seg; +} + +/* Downsample macroblock QP offsets into a 64x64 superblock ROI segment map. */ +static void roi_fill_map(const FFQpFrame *qf, unsigned dest_b64_cols, + unsigned dest_b64_rows, const int *seg_values, + unsigned n_seg, uint8_t *seg_map) +{ + unsigned mb_cols = qf->block_cols, mb_rows = qf->block_rows; + unsigned i, j, sb_x, sb_y; + + for (sb_y = 0; sb_y < dest_b64_rows; sb_y++) { + for (sb_x = 0; sb_x < dest_b64_cols; sb_x++) { + long sum = 0; + unsigned count = 0; + unsigned mb_y0 = sb_y * QPFILE_SVTAV1_SB_SIDE; + unsigned mb_x0 = sb_x * QPFILE_SVTAV1_SB_SIDE; + unsigned mb_y1 = mb_y0 + QPFILE_SVTAV1_SB_SIDE; + unsigned mb_x1 = mb_x0 + QPFILE_SVTAV1_SB_SIDE; + int avg, best_idx = 0, best_dist = INT_MAX; + mb_y1 = FFMIN(mb_y1, mb_rows); + mb_x1 = FFMIN(mb_x1, mb_cols); + if (mb_x0 >= mb_cols || mb_y0 >= mb_rows) { + /* SB lies outside the qpfile coverage -- leave at seg 0 + * (zero offset). The qpfile may be smaller than the + * encoder's padded SB grid; the QP-offset map is + * applied where defined and elsewhere we encode at + * the encoder's default rate-control decision. */ + seg_map[sb_y * dest_b64_cols + sb_x] = 0; + continue; + } + for (i = mb_y0; i < mb_y1; i++) { + for (j = mb_x0; j < mb_x1; j++) { + sum += qf->block_offsets[i * mb_cols + j]; + count++; + } + } + avg = count ? (int)(sum / (long)count) : 0; + for (i = 0; i < n_seg; i++) { + int d = avg - seg_values[i]; + d = FFABS(d); + if (d < best_dist) { + best_dist = d; + best_idx = (int)i; + } + } + seg_map[sb_y * dest_b64_cols + sb_x] = (uint8_t)best_idx; + } + } +} + +/* Build an SVT-AV1 ROI event and map for a frame, returning 0 or an error. */ +static int build_one_roi_evt(const FFQpFrame *qf, + unsigned dest_b64_cols, + unsigned dest_b64_rows, + SvtAv1RoiMapEvt *out_evt, + uint8_t **out_seg_map) +{ + unsigned mb_cols = qf->block_cols; + unsigned mb_rows = qf->block_rows; + int seg_values[QPFILE_SVTAV1_MAX_SEGMENTS]; + unsigned n_seg = 0; + unsigned i; + uint8_t *seg_map = NULL; + + if (!dest_b64_cols || !dest_b64_rows) + return AVERROR(EINVAL); + + seg_map = av_calloc((size_t)dest_b64_rows * dest_b64_cols, + sizeof(*seg_map)); + if (!seg_map) + return AVERROR(ENOMEM); + + memset(out_evt, 0, sizeof(*out_evt)); + out_evt->start_picture_number = qf->frame_index; + out_evt->b64_seg_map = seg_map; + out_evt->max_seg_id = 0; + out_evt->next = NULL; + *out_seg_map = seg_map; + if (!mb_cols || !mb_rows || !qf->block_offsets) + return 0; + + n_seg = roi_segment_values(qf, seg_values); + + /* Pass 3: emit the per-SB segment map at the encoder grid. */ + roi_fill_map(qf, dest_b64_cols, dest_b64_rows, seg_values, n_seg, seg_map); + + out_evt->max_seg_id = (int8_t)(n_seg - 1u); + for (i = 0; i < n_seg; i++) + out_evt->seg_qp[i] = (int16_t)seg_values[i]; + return 0; +} + +/* Build ROI events for all qpfile frames, returning 0 or a negative error. */ +static int build_all_roi_evts(AVCodecContext *avctx, SvtContext *svt_enc) +{ + unsigned i; + unsigned b64_cols = ((unsigned)avctx->width + 63u) / 64u; + unsigned b64_rows = ((unsigned)avctx->height + 63u) / 64u; + unsigned n = svt_enc->qpfile.frame_count; + + if (!n || !b64_cols || !b64_rows) + return AVERROR(EINVAL); + + svt_enc->roi_evts = av_calloc(n, sizeof(*svt_enc->roi_evts)); + svt_enc->roi_seg_maps = av_calloc(n, sizeof(*svt_enc->roi_seg_maps)); + if (!svt_enc->roi_evts || !svt_enc->roi_seg_maps) + return AVERROR(ENOMEM); + svt_enc->roi_evt_count = n; + svt_enc->roi_b64_cols = b64_cols; + svt_enc->roi_b64_rows = b64_rows; + + for (i = 0; i < n; i++) { + int qret = build_one_roi_evt(&svt_enc->qpfile.frames[i], + b64_cols, b64_rows, + &svt_enc->roi_evts[i], + &svt_enc->roi_seg_maps[i]); + if (qret < 0) { + av_log(avctx, AV_LOG_ERROR, + "failed to build ROI map for qpfile frame %u (%d)\n", + i, qret); + return qret; + } + } + return 0; +} + +/* Look up the ROI map event for a given frame index, returning it or NULL. */ +static SvtAv1RoiMapEvt *roi_evt_for_index(SvtContext *svt_enc, unsigned frame_index) +{ + const FFQpFrame *qf; + ptrdiff_t index; + + if (!svt_enc->roi_evt_count) + return NULL; + qf = ff_qpfile_frame_for_index(&svt_enc->qpfile, frame_index); + if (!qf || qf->frame_index != frame_index) + return NULL; + index = qf - svt_enc->qpfile.frames; + if ((unsigned)index >= svt_enc->roi_evt_count) + return NULL; + return &svt_enc->roi_evts[index]; +} +#endif /* SVT_AV1_CHECK_VERSION(1, 6, 0) */ + +/* Load the QP file and check its grid, returning 0 or a negative error code. */ +static av_cold int load_qpfile(AVCodecContext *avctx) +{ + SvtContext *svt_enc = avctx->priv_data; + int ret = ff_qpfile_load(avctx, svt_enc->qpfile_path, &svt_enc->qpfile); + + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "cannot read qpfile %s: %s\n", + svt_enc->qpfile_path, av_err2str(ret)); + return ret; + } + svt_enc->qpfile_loaded = 1; + return ff_qpfile_check_grid(avctx, "libsvtav1", &svt_enc->qpfile, + avctx->width, avctx->height); +} + static av_cold int eb_enc_init(AVCodecContext *avctx) { SvtContext *svt_enc = avctx->priv_data; @@ -505,11 +763,41 @@ static av_cold int eb_enc_init(AVCodecContext *avctx) return ret; } + if (svt_enc->qpfile_path && *svt_enc->qpfile_path) { + ret = load_qpfile(avctx); + if (ret < 0) + return ret; +#if SVT_AV1_CHECK_VERSION(1, 6, 0) + /* Build all per-frame ROI map events upfront. SVT-AV1 reads + * ROI_MAP_EVENT data via shallow-copied pointers on async + * pipeline threads, so the events and their b64_seg_maps must + * live for the whole encode session. Then flip enable_roi_map + * before set_parameter so the encoder allocates segmentation + * state. + */ + ret = build_all_roi_evts(avctx, svt_enc); + if (ret < 0) + return ret; + svt_enc->enc_params.enable_roi_map = 1; + av_log(avctx, AV_LOG_VERBOSE, + "qpfile=%s loaded (frames=%u, %ux%u qpfile blocks " + "-> %ux%u SB ROI grid).\n", + svt_enc->qpfile_path, svt_enc->qpfile.frame_count, + svt_enc->qpfile.block_cols, svt_enc->qpfile.block_rows, + svt_enc->roi_b64_cols, svt_enc->roi_b64_rows); +#else + av_log(avctx, AV_LOG_ERROR, + "qpfile needs SVT-AV1 1.6.0 or newer.\n"); + return AVERROR(ENOSYS); +#endif + } + svt_ret = svt_av1_enc_set_parameter(svt_enc->svt_handle, &svt_enc->enc_params); if (svt_ret != EB_ErrorNone) { return svt_print_error(avctx, svt_ret, "Error setting encoder parameters"); } + svt_ret = svt_av1_enc_init(svt_enc->svt_handle); if (svt_ret != EB_ErrorNone) { return svt_print_error(avctx, svt_ret, "Error initializing encoder"); @@ -613,8 +901,40 @@ static int eb_send_frame(AVCodecContext *avctx, const AVFrame *frame) return AVERROR_INVALIDDATA; } +#if SVT_AV1_CHECK_VERSION(1, 6, 0) + /* qpfile -> SVT-AV1 ROI map: attach the (already-built) + * per-frame ROI map event as a stack-local priv-data node. The + * encoder's resource_coordination_process.c expects + * node->size == sizeof(SvtAv1RoiMapEvt*) + * and uses node->data as a pointer that must outlive the async + * pipeline -- so the SvtAv1RoiMapEvt + b64_seg_map live in + * svt_enc->roi_evts/roi_seg_maps until eb_enc_close(). The + * EbPrivDataNode itself is shallow-copied by copy_private_data_list + * in enc_handle.c, so a stack-local node is safe. + */ + EbPrivDataNode roi_node; + SvtAv1RoiMapEvt *roi_evt_ptr = NULL; + if (svt_enc->qpfile_loaded && svt_enc->roi_evt_count > 0) + roi_evt_ptr = roi_evt_for_index(svt_enc, svt_enc->roi_input_frame++); + if (roi_evt_ptr) { + memset(&roi_node, 0, sizeof(roi_node)); + roi_node.node_type = ROI_MAP_EVENT; + roi_node.data = roi_evt_ptr; + roi_node.size = (uint32_t)sizeof(SvtAv1RoiMapEvt *); + roi_node.next = NULL; + headerPtr->p_app_private = &roi_node; + } +#endif svt_ret = svt_av1_enc_send_picture(svt_enc->svt_handle, headerPtr); + +#if SVT_AV1_CHECK_VERSION(1, 6, 0) + /* Restore the slot for the next frame's send-path. The library + * already shallow-copied the priv-data list. */ + if (roi_evt_ptr) + headerPtr->p_app_private = NULL; +#endif + if (svt_ret != EB_ErrorNone) return svt_print_error(avctx, svt_ret, "Error sending a frame to encoder"); @@ -784,6 +1104,23 @@ static av_cold int eb_enc_close(AVCodecContext *avctx) ff_dovi_ctx_unref(&svt_enc->dovi); av_freep(&svt_enc->stats_buf); + if (svt_enc->qpfile_loaded) { +#if SVT_AV1_CHECK_VERSION(1, 6, 0) + if (svt_enc->roi_seg_maps) { + for (unsigned i = 0; i < svt_enc->roi_evt_count; i++) + av_freep(&svt_enc->roi_seg_maps[i]); + av_freep(&svt_enc->roi_seg_maps); + } + av_freep(&svt_enc->roi_evts); + svt_enc->roi_evt_count = 0; + svt_enc->roi_b64_cols = 0; + svt_enc->roi_b64_rows = 0; + svt_enc->roi_input_frame = 0; +#endif + ff_qpfile_free(&svt_enc->qpfile); + svt_enc->qpfile_loaded = 0; + } + return 0; } @@ -828,6 +1165,7 @@ static const AVOption options[] = { { "qp", "Initial Quantizer level value", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 63, VE }, { "svtav1-params", "Set the SVT-AV1 configuration using a :-separated list of key=value parameters", OFFSET(svtav1_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { "qpfile", "Read per-frame QP offsets from a file (needs SVT-AV1 1.6.0 or newer)", OFFSET(qpfile_path), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, VE }, { "dolbyvision", "Enable Dolby Vision RPU coding", OFFSET(dovi.enable), AV_OPT_TYPE_BOOL, {.i64 = FF_DOVI_AUTOMATIC }, -1, 1, VE, .unit = "dovi" }, { "auto", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = FF_DOVI_AUTOMATIC}, .flags = VE, .unit = "dovi" }, -- 2.54.0 >From 9498b42443e2a930d89d55530c2c18e7973e8388 Mon Sep 17 00:00:00 2001 From: lusoris <[email protected]> Date: Sun, 8 Nov 2026 17:01:42 +0100 Subject: [PATCH 3/3] avcodec/libx264: add a qpfile option Add the qpfile option of libaom-av1 to libx264, with the same file format, and pass the offsets to x264 as per-macroblock quantizer offsets. A record without offset lines ends the offsets from that frame on. x264 applies the offsets through adaptive quantization. That is off with constant QP, and with aq-mode 0 or aq-strength 0 unless MB-tree is on, which keeps it running. The option is therefore checked against the parameters x264 reports after x264_encoder_open() and refused when adaptive quantization is off. Interlaced encoding is refused too: x264 rounds the macroblock row count up to even there, so the per-row offsets no longer match. libx262 is built from libx264.c and shares its options, so it links qpfile_parser.o too. Signed-off-by: lusoris <[email protected]> --- doc/encoders.texi | 11 +++++++ libavcodec/Makefile | 4 +-- libavcodec/libx264.c | 77 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 2 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index dc267f5e3cb..676b3ca0d7d 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3193,6 +3193,17 @@ Variable bit rate. Constant bit rate (not allowed in MP4 container). @end table +@item qpfile @var{filename} +Read per-frame macroblock QP offsets from @var{filename}, in the format +described for the libaom-av1 @option{qpfile} option, and pass them to +x264 as quantizer offsets. x264 applies the offsets through adaptive +quantization; the option is refused when x264 runs without adaptive +quantization, that is with constant QP, or with @option{aq-mode} 0 or +@option{aq-strength} 0 while @option{mbtree} is off; it is also refused +with interlaced encoding. A frame uses the last +record whose number is not greater than its own; a record without offset +lines ends the offsets. + @item x264opts @var{opts} @item x264-params @var{opts} Override the x264 configuration using a :-separated list of key=value diff --git a/libavcodec/Makefile b/libavcodec/Makefile index dbfebc61b8d..f05eeae8bea 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1243,8 +1243,8 @@ OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o OBJS-$(CONFIG_LIBVVENC_ENCODER) += libvvenc.o OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_animencoder.o -OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o -OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o +OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o qpfile_parser.o +OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o qpfile_parser.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c index a432c0b50e5..d2dd770c87a 100644 --- a/libavcodec/libx264.c +++ b/libavcodec/libx264.c @@ -39,6 +39,8 @@ #include "sei.h" #include "golomb.h" +#include "qpfile_parser.h" + #include <x264.h> #include <float.h> #include <math.h> @@ -126,6 +128,10 @@ typedef struct X264Context { int roi_warned; int mb_info; + + char *qpfile_path; + FFQpFile qpf; /* the loaded qpfile (frame_count 0: none) */ + unsigned qpf_frames; /* input frames seen: the qpfile's frame index */ } X264Context; static void X264_log(void *p, int level, const char *fmt, va_list args) @@ -473,6 +479,36 @@ static int setup_roi(AVCodecContext *ctx, x264_picture_t *pic, return 0; } +/* Add the qpfile's per-MB QP offsets of input frame `frame_index` to the + * picture's quant_offsets (x264 adds them to adaptive quantization's own, as + * for a region of interest). x264 has no qpfile parameter: its qpfile is a + * feature of the x264 command line, not of libx264. */ +static int setup_qpfile(AVCodecContext *ctx, x264_picture_t *pic, unsigned frame_index) +{ + X264Context *x4 = ctx->priv_data; + const FFQpFrame *qf; + size_t n_mb; + float *qoffsets; + + qf = ff_qpfile_frame_for_index(&x4->qpf, frame_index); + if (!qf || !qf->block_offsets) + return 0; + n_mb = (size_t)qf->block_cols * qf->block_rows; + + qoffsets = pic->prop.quant_offsets; + if (!qoffsets) { + qoffsets = av_calloc(n_mb, sizeof(*qoffsets)); + if (!qoffsets) + return AVERROR(ENOMEM); + pic->prop.quant_offsets = qoffsets; + pic->prop.quant_offsets_free = av_free; + } + for (size_t i = 0; i < n_mb; i++) + qoffsets[i] += (float)qf->block_offsets[i]; + + return 0; +} + static int setup_frame(AVCodecContext *ctx, const AVFrame *frame, x264_picture_t **ppic) { @@ -570,6 +606,12 @@ static int setup_frame(AVCodecContext *ctx, const AVFrame *frame, goto fail; } + if (x4->qpf.frame_count) { + ret = setup_qpfile(ctx, pic, x4->qpf_frames++); + if (ret < 0) + goto fail; + } + mbinfo_sd = av_frame_get_side_data(frame, AV_FRAME_DATA_VIDEO_HINT); if (mbinfo_sd) { int err = setup_mb_info(ctx, pic, frame, (const AVVideoHint *)mbinfo_sd->data); @@ -785,6 +827,7 @@ static av_cold int X264_close(AVCodecContext *avctx) X264Context *x4 = avctx->priv_data; av_freep(&x4->sei); + ff_qpfile_free(&x4->qpf); for (int i = 0; i < x4->nb_reordered_opaque; i++) opaque_uninit(&x4->reordered_opaque[i]); @@ -1418,6 +1461,19 @@ static av_cold int X264_init(AVCodecContext *avctx) x4->params.analyse.b_mb_info = x4->mb_info; + if (x4->qpfile_path && *x4->qpfile_path) { + ret = ff_qpfile_load(avctx, x4->qpfile_path, &x4->qpf); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "cannot read qpfile %s: %s\n", + x4->qpfile_path, av_err2str(ret)); + return ret; + } + ret = ff_qpfile_check_grid(avctx, "libx264", &x4->qpf, + avctx->width, avctx->height); + if (ret < 0) + return ret; + } + // update AVCodecContext with x264 parameters avctx->has_b_frames = x4->params.i_bframe ? x4->params.i_bframe_pyramid ? 2 : 1 : 0; @@ -1430,6 +1486,26 @@ static av_cold int X264_init(AVCodecContext *avctx) if (!x4->enc) return AVERROR_EXTERNAL; + if (x4->qpfile_path && *x4->qpfile_path) { + x264_param_t effective; + x264_encoder_parameters(x4->enc, &effective); + if (effective.rc.i_aq_mode == X264_AQ_NONE) { + av_log(avctx, AV_LOG_ERROR, + "a qpfile needs adaptive quantization, but x264 runs without it (constant QP, or aq-mode 0 or aq-strength 0 without mbtree).\n"); + return AVERROR(EINVAL); + } + if (effective.b_interlaced) { + av_log(avctx, AV_LOG_ERROR, + "a qpfile does not support interlaced encoding.\n"); + return AVERROR(EINVAL); + } + const unsigned mbx = (avctx->width + MB_SIZE - 1) / MB_SIZE; + const unsigned mby = (avctx->height + MB_SIZE - 1) / MB_SIZE; + av_log(avctx, AV_LOG_VERBOSE, "qpfile=%s loaded (%u frames, %ux%u macroblocks); " + "offsets applied as quant_offsets.\n", x4->qpfile_path, x4->qpf.frame_count, + mbx, mby); + } + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) { ret = set_extradata(avctx); if (ret < 0) @@ -1594,6 +1670,7 @@ static const AVOption options[] = { { "udu_sei", "Use user data unregistered SEI if available", OFFSET(udu_sei), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, { "x264-params", "Override the x264 configuration using a :-separated list of key=value parameters", OFFSET(x264_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, { "mb_info", "Set mb_info data through AVSideData, only useful when used from the API", OFFSET(mb_info), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, + { "qpfile", "Read per-frame QP offsets from a file (needs adaptive quantization)", OFFSET(qpfile_path), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, VE }, { NULL }, }; -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
