PR #24736 opened by Jake (jakefineman) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24736 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24736.patch
**Problem.** `dnn_classify` never calls `ff_dnn_get_input()`, unlike `dnn_detect` and `dnn_processing`. With the OpenVINO backend that call is where an unset `layout` option gets resolved to NCHW or NHWC from the model input shape (`get_input_ov()` in `dnn_backend_openvino.c`). Without it the layout stays `DL_NONE`, `init_model_ov()` sets no model layout on the pre-processor, and the NHWC u8 frame is fed to an NCHW model without a transpose. The result is silently wrong classifications, with no error, unless the user knows to pass `layout=nchw`. **Fix.** `dnn_classify` gets its own input pad whose `config_props` calls `ff_dnn_get_input()`, the same way `dnn_detect` does. This resolves the layout before the first inference and reports a bad input name at configuration time. An explicit `layout=` is still honoured, since the backend only fills the layout while it is unset. No other backend can be selected for `dnn_classify` on master, so no other backend is affected. **Testing** (OpenVINO 2026.4.0 CPU, macOS arm64): `dnn_detect` then `dnn_classify` with ConvNeXt-B (ImageNet-1k, NCHW ONNX model), on a frame with one dog: | `layout` | master | patched | |---|---|---| | unset | "web site" 0.09 | French bulldog 0.76 | | `nchw` | French bulldog | French bulldog | | `nhwc` | "web site" | "web site" (still obeyed) | `make fate-source` passes. This is independent of the ONNX classify work in #24049/#24353: the change only touches the input pad, and it merges cleanly with both in either order. >From 34075d6f234c4663fd8ce94b9618ebdeb4da235c Mon Sep 17 00:00:00 2001 From: Jake Fineman <[email protected]> Date: Sun, 27 Sep 2026 00:22:14 -0400 Subject: [PATCH] avfilter/vf_dnn_classify: query the model input on input config dnn_classify never called ff_dnn_get_input(), unlike dnn_detect and dnn_processing. With the OpenVINO backend, get_input_ov() is where an unset "layout" option is resolved to NCHW or NHWC from the model input shape. Without that call the layout stays DL_NONE, init_model_ov() sets no model layout on the pre-processor, and the NHWC u8 frame data is fed to an NCHW model without a transpose, producing wrong classifications unless layout=nchw is given explicitly. Query the model input from config_props, as dnn_detect does, so the layout is resolved (and a bad input name is reported) before the first inference. Signed-off-by: Jake Fineman <[email protected]> --- libavfilter/vf_dnn_classify.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/libavfilter/vf_dnn_classify.c b/libavfilter/vf_dnn_classify.c index f92c41ab76..33c5d9f1b2 100644 --- a/libavfilter/vf_dnn_classify.c +++ b/libavfilter/vf_dnn_classify.c @@ -286,6 +286,31 @@ static int dnn_classify_activate(AVFilterContext *filter_ctx) return 0; } +static int config_input(AVFilterLink *inlink) +{ + AVFilterContext *context = inlink->dst; + DnnClassifyContext *ctx = context->priv; + DNNData model_input = { 0 }; + int ret; + + /* Also lets the backend resolve a model input layout left unset by the user. */ + ret = ff_dnn_get_input(&ctx->dnnctx, &model_input); + if (ret != 0) { + av_log(ctx, AV_LOG_ERROR, "could not get input from the model\n"); + return ret; + } + + return 0; +} + +static const AVFilterPad dnn_classify_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_input, + }, +}; + static av_cold void dnn_classify_uninit(AVFilterContext *context) { DnnClassifyContext *ctx = context->priv; @@ -301,7 +326,7 @@ const FFFilter ff_vf_dnn_classify = { .preinit = ff_dnn_filter_init_child_class, .init = dnn_classify_init, .uninit = dnn_classify_uninit, - FILTER_INPUTS(ff_video_default_filterpad), + FILTER_INPUTS(dnn_classify_inputs), FILTER_OUTPUTS(ff_video_default_filterpad), FILTER_PIXFMTS_ARRAY(pix_fmts), .activate = dnn_classify_activate, -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
