PR #25255 opened by stefan-h-singer URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25255 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25255.patch
# Summary of changes The whisper filter emits timestamped JSON segments without the language selected by whisper.cpp. Include the selected chunk-level language code as an optional `detected_language` field in each JSON segment, and document its scope. Other output formats and the configured language are unchanged. # Testing - Minimal `--enable-whisper` build. - `make fate` (without external SAMPLES). - Local WAV transcription with a multilingual tiny model; confirmed the JSON field. >From 98cf2f05591740f4eb9eb4576fd98e57673b4165 Mon Sep 17 00:00:00 2001 From: Stefan Singer <[email protected]> Date: Tue, 20 Oct 2026 15:24:34 +0200 Subject: [PATCH] avfilter/whisper: include detected language in JSON output The whisper filter exposes segment timestamps and text in JSON but omits the language selected by whisper.cpp. Clients consuming multilingual transcripts cannot distinguish the language of each transcription chunk without detecting it again. Read whisper_full_lang_id() after each successful transcription and include its language code in JSON segments when available. This leaves the configured language and audio-frame metadata unchanged. Document that the code applies to the whole chunk, not to individual segments. Tested with a minimal --enable-whisper build, make fate (without external SAMPLES), and a local WAV transcription using a multilingual tiny model. Signed-off-by: Stefan Singer <[email protected]> --- doc/filters.texi | 3 +++ libavfilter/af_whisper.c | 9 ++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/doc/filters.texi b/doc/filters.texi index be3058b1a10..c830d4105b1 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -7792,6 +7792,9 @@ If the destination is a file and it already exists, it will be overwritten. @item format The destination format string; it could be "text" (only the transcribed text will be sent to the destination), "srt" (subtitle format) or "json". +JSON segments include a @code{detected_language} field containing the language +code reported by whisper.cpp for the transcription chunk, when available. All +segments from the same chunk have the same detected language. Default value: @code{"text"} @item max_len diff --git a/libavfilter/af_whisper.c b/libavfilter/af_whisper.c index 33d8a2112af..71217fdfbd8 100644 --- a/libavfilter/af_whisper.c +++ b/libavfilter/af_whisper.c @@ -240,6 +240,8 @@ static void run_transcription(AVFilterContext *ctx, AVFrame *frame, int samples) return; } + const int lang_id = whisper_full_lang_id(wctx->ctx_wsp); + const char *detected_language = lang_id >= 0 ? whisper_lang_str(lang_id) : NULL; const int n_segments = whisper_full_n_segments(wctx->ctx_wsp); char *segments_text = NULL; @@ -292,7 +294,12 @@ static void run_transcription(AVFilterContext *ctx, AVFrame *frame, int samples) wctx->index++; } else if (!av_strcasecmp(wctx->format, "json")) { - buf = av_asprintf("{\"start\":%" PRId64 ",\"end\":%" PRId64 ",\"text\":\"%s\"}\n", start_t, end_t, text_cleaned); + if (detected_language) + buf = av_asprintf("{\"start\":%" PRId64 ",\"end\":%" PRId64 ",\"text\":\"%s\",\"detected_language\":\"%s\"}\n", + start_t, end_t, text_cleaned, detected_language); + else + buf = av_asprintf("{\"start\":%" PRId64 ",\"end\":%" PRId64 ",\"text\":\"%s\"}\n", + start_t, end_t, text_cleaned); } else buf = av_asprintf("%s\n", text_cleaned); -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
