PR #24731 opened by Jake (jakefineman) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24731 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24731.patch
Problem dnn_detect and dnn_classify convert the incoming YUV frame to the model's RGB/BGR tensor with sws_getContext(). That context assumes limited range for YUV input unless the pixel format itself implies full range (the deprecated YUVJ formats). A plain yuv420p frame tagged color_range=pc is therefore expanded a second time before inference: blacks are crushed, highlights clip, and the model sees a different image than the one in the stream. The frame's color_range tag is ignored entirely. Reproducer (ONNX Runtime backend, D-FINE COCO detector, puppies.ogv): ffmpeg -i puppies.ogv -an -vf "fps=2,scale=1280:720:force_original_aspect_ratio=decrease:out_range=RANGE,pad=1280:720:(ow-iw)/2:(oh-ih)/2,select=eq(n\,330),dnn_detect=dnn_backend=onnx:model=dfine_s_obj2coco_v2.onnx:input=data:output=detection_out:confidence=0.5:labels=coco.txt,showinfo" -frames:v 1 -f null - Results for frame 330: RANGE master patched tv dog 0.6762 dog 0.6762 pc cat 0.6186 dog 0.6322 pc,setrange=tv cat 0.6186 cat 0.6186 On master, full-range pixels produce the same wrong answer whether they are tagged pc or tv, which shows the tag is not consulted. With the patch, the correctly tagged full-range frame is converted properly and agrees with the limited-range result. The mis-tagged case (full-range pixels labelled tv) still gives cat, as it should: the filter now trusts the tag. Fix A small helper, set_sws_src_range(), is called after each of the three frame-to-tensor sws_getContext() calls (the classify crop and both detect paths). When the frame is tagged AVCOL_RANGE_JPEG and the context does not already treat the source as full range, it reads the current details with sws_getColorspaceDetails() and sets srcRange=1 with sws_setColorspaceDetails(), leaving the matrix, destination range and picture adjustments unchanged. For frames tagged limited or unspecified the context is not touched at all, so their output is unchanged. RGB and gray sources are unaffected because swscale ignores the range for them. If setting the range fails, the call now returns an error rather than silently producing wrong input. ff_proc_from_frame_to_dnn() and ff_proc_from_dnn_to_frame() (used by dnn_processing) are left as they are. For YUV frames they only pass the luma plane through as gray, and swscale already treats gray as full range on both sides. For RGB frames they only repack between packed and planar layouts. Neither path does a YUV-to-RGB conversion, so the range does not come into it. Testing (Apple Silicon, macOS SDK 26.5, ONNX Runtime from Homebrew, master at d975849594) - The n330 reproducer above: results as in the table. - Unspecified input unchanged: dnn_detect over the first 30 s of puppies.ogv at fps=2 (60 frames, 385 detections). The showinfo output, including frame checksums and every box and confidence, is byte-identical between master and patched. This run uses -threads 1 -filter_threads 1 because the multi-threaded master build is not deterministic between runs on this input. - dnn_processing with an identity 3-channel float model on rgb24 input: 150 frames of framemd5 output, identical between master and patched. - make fate-source passes. The change adds no compiler warnings. Not covered: on master, dnn_classify only runs on the OpenVINO backend, which is not built here. The classify crop gets the same helper call as the detect paths and compiles cleanly, but it has not been run. Colour matrix (the frame's colorspace) is deliberately left out of this patch. Honouring it would change output for limited-range input tagged BT.709, which is most HD content, so it belongs in a separate patch with its own evaluation. >From 41724899bad69b0493b522c77828cd85d445abd6 Mon Sep 17 00:00:00 2001 From: Jake Fineman <[email protected]> Date: Sat, 26 Sep 2026 17:37:52 -0400 Subject: [PATCH] avfilter/dnn_io_proc: honour the frame colour range in sws conversions The swscale contexts that convert a YUV frame into the RGB/BGR tensor for dnn_detect and dnn_classify are created with sws_getContext(), which assumes limited range for YUV input unless the pixel format itself implies full range (the deprecated YUVJ formats). A yuv420p frame tagged AVCOL_RANGE_JPEG is therefore expanded a second time, crushing blacks and clipping highlights before inference. This changes detection results. With puppies.ogv at fps=2, frame 330, and the D-FINE COCO detector via the ONNX Runtime backend: scale=...:out_range=tv -> dog 0.676 scale=...:out_range=pc -> cat 0.619 scale=...:out_range=pc,setrange=tv -> cat 0.619 The last line shows the filter ignores the tag: full-range pixels give the same wrong answer whether they are tagged pc or tv. Set the source range with sws_setColorspaceDetails() when the frame is tagged full range and the context does not already treat the input as full range. Frames tagged limited or unspecified keep the previous behaviour, and RGB/gray input is unaffected since swscale ignores the range for those. With this patch the out_range=pc case above gives dog 0.632. ff_proc_from_frame_to_dnn() and ff_proc_from_dnn_to_frame() are not changed: for YUV frames they only feed the luma plane through as gray (which swscale already treats as full range on both sides), and for RGB frames they only repack, so no range conversion takes place. Signed-off-by: Jake Fineman <[email protected]> Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --- libavfilter/dnn/dnn_io_proc.c | 45 +++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/libavfilter/dnn/dnn_io_proc.c b/libavfilter/dnn/dnn_io_proc.c index 0a18c56b84..950b09eddf 100644 --- a/libavfilter/dnn/dnn_io_proc.c +++ b/libavfilter/dnn/dnn_io_proc.c @@ -319,6 +319,33 @@ err: return ret; } +/* + * sws_getContext() treats YUV input as limited range unless the pixel format + * itself implies full range (the deprecated YUVJ formats), so a full range + * frame tagged only through color_range would be expanded a second time. + */ +static int set_sws_src_range(struct SwsContext *sws_ctx, const AVFrame *frame, void *log_ctx) +{ + int *inv_table, *table; + int src_range, dst_range, brightness, contrast, saturation; + + if (frame->color_range != AVCOL_RANGE_JPEG) + return 0; + + if (sws_getColorspaceDetails(sws_ctx, &inv_table, &src_range, &table, &dst_range, + &brightness, &contrast, &saturation) >= 0) { + if (src_range) + return 0; + if (sws_setColorspaceDetails(sws_ctx, inv_table, 1, table, dst_range, + brightness, contrast, saturation) >= 0) + return 0; + } + + av_log(log_ctx, AV_LOG_ERROR, "Failed to set full colour range for %s input\n", + av_get_pix_fmt_name(frame->format)); + return AVERROR(EINVAL); +} + static enum AVPixelFormat get_pixel_format(DNNData *data) { if (data->dt == DNN_UINT8) { @@ -393,6 +420,12 @@ int ff_frame_to_dnn_classify(AVFrame *frame, DNNData *input, uint32_t bbox_index return AVERROR(EINVAL); } + ret = set_sws_src_range(sws_ctx, frame, log_ctx); + if (ret < 0) { + sws_freeContext(sws_ctx); + return ret; + } + ret = av_image_fill_linesizes(linesizes, fmt, input->dims[width_idx]); if (ret < 0) { av_log(log_ctx, AV_LOG_ERROR, "unable to get linesizes with av_image_fill_linesizes"); @@ -533,6 +566,12 @@ int ff_frame_to_dnn_detect(AVFrame *frame, DNNData *input, void *log_ctx) return AVERROR(EINVAL); } + ret = set_sws_src_range(sws_ctx, frame, log_ctx); + if (ret < 0) { + sws_freeContext(sws_ctx); + return ret; + } + sws_scale(sws_ctx, (const uint8_t *const *)frame->data, frame->linesize, 0, frame->height, (uint8_t *const [4]){input->data, 0, 0, 0}, linesizes); sws_freeContext(sws_ctx); @@ -565,6 +604,12 @@ int ff_frame_to_dnn_detect(AVFrame *frame, DNNData *input, void *log_ctx) goto end; } + ret = set_sws_src_range(sws_ctx, frame, log_ctx); + if (ret < 0) { + sws_freeContext(sws_ctx); + goto end; + } + sws_scale(sws_ctx, (const uint8_t *const *)frame->data, frame->linesize, 0, frame->height, (uint8_t *const [4]){tmp_buf, 0, 0, 0}, linesizes); sws_freeContext(sws_ctx); -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
