Re: [FFmpeg-devel] [PATCH v2] avfilter/dnn: fix incompatible integer to pointer conversion warning
On Sat, Dec 11, 2021 at 03:06:42AM +, Guo, Yejun wrote:
>
>
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> lance.lmw...@gmail.com
> Sent: 2021年12月9日 22:37
> To: ffmpeg-devel@ffmpeg.org
> Cc: Limin Wang
> Subject: [FFmpeg-devel] [PATCH v2] avfilter/dnn: fix incompatible integer to
> pointer conversion warning
>
> From: Limin Wang
>
> ---
> libavfilter/dnn/dnn_backend_common.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavfilter/dnn/dnn_backend_common.c
> b/libavfilter/dnn/dnn_backend_common.c
> index 6a9c4cc..dd7bdf4 100644
> --- a/libavfilter/dnn/dnn_backend_common.c
> +++ b/libavfilter/dnn/dnn_backend_common.c
> @@ -83,10 +83,10 @@ static void *async_thread_routine(void *args)
> void *request = async_module->args;
>
> if (async_module->start_inference(request) != DNN_SUCCESS) {
> -return DNN_ASYNC_FAIL;
> +return (void*)DNN_ASYNC_FAIL;
>
> there is already (void*) in DNN_ASYNC_FAIL
Sorry, it's my mistake. please ignore the patch.
Can we check the thread_join status with void* and void* error code?
if (status == DNN_ASYNC_FAIL) {
For the compiler doesnt know what type of address the pointer is pointing at,
it doesnt know how many bytes to read from the pointer address and compare.
>
> }
> async_module->callback(request);
> -return DNN_ASYNC_SUCCESS;
> +return (void*)DNN_ASYNC_SUCCESS;
> }
>
> DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
> --
> 1.8.3.1
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org
> with subject "unsubscribe".
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
--
Thanks,
Limin Wang
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] avfilter/dnn: fix incompatible integer to pointer conversion warning
-Original Message-
From: ffmpeg-devel On Behalf Of
lance.lmw...@gmail.com
Sent: 2021年12月9日 22:37
To: ffmpeg-devel@ffmpeg.org
Cc: Limin Wang
Subject: [FFmpeg-devel] [PATCH v2] avfilter/dnn: fix incompatible integer to
pointer conversion warning
From: Limin Wang
---
libavfilter/dnn/dnn_backend_common.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavfilter/dnn/dnn_backend_common.c
b/libavfilter/dnn/dnn_backend_common.c
index 6a9c4cc..dd7bdf4 100644
--- a/libavfilter/dnn/dnn_backend_common.c
+++ b/libavfilter/dnn/dnn_backend_common.c
@@ -83,10 +83,10 @@ static void *async_thread_routine(void *args)
void *request = async_module->args;
if (async_module->start_inference(request) != DNN_SUCCESS) {
-return DNN_ASYNC_FAIL;
+return (void*)DNN_ASYNC_FAIL;
there is already (void*) in DNN_ASYNC_FAIL
}
async_module->callback(request);
-return DNN_ASYNC_SUCCESS;
+return (void*)DNN_ASYNC_SUCCESS;
}
DNNReturnType ff_dnn_async_module_cleanup(DNNAsyncExecModule *async_module)
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with
subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] request to revert all patches that made cfr decoders vfr
fre 2021-12-10 klockan 08:54 +0100 skrev Paul B Mahol: > This is the first warning to revert all patches that made cfr > decoders vfr. Would be good if you named said patches /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 4/5] avformat: introduce AVFormatContext io_close2 which returns an int
On Sat, 4 Dec 2021, Marton Balint wrote:
Otherwise there is no way to detect an error returned by avio_close() because
ff_format_io_close cannot get the return value.
Checking the return value of the close function is important in order to check
if all data was successfully written and the underlying close() operation was
successful.
It can also be useful even for read mode because it can return any pending
AVIOContext error, so the user don't have to manually check AVIOContext->error.
In order to still support if the user overrides io_close, the generic code only
uses io_close2 if io_close is either NULL or the default io_close callback.
Signed-off-by: Marton Balint
---
doc/APIchanges | 3 +++
libavformat/avformat.h | 13 +
libavformat/dashenc.c | 1 +
libavformat/fifo.c | 1 +
libavformat/hlsenc.c | 1 +
libavformat/internal.h | 10 +-
libavformat/options.c | 7 ---
libavformat/segment.c | 1 +
libavformat/tee.c | 1 +
libavformat/utils.c| 17 ++---
libavformat/version.h | 4 ++--
11 files changed, 50 insertions(+), 9 deletions(-)
Will apply the series soon.
Regards,
Marton
diff --git a/doc/APIchanges b/doc/APIchanges
index 2914ad6734..a9092d27ef 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -14,6 +14,9 @@ libavutil: 2021-04-27
API changes, most recent first:
+2021-12-xx - xx - lavf 59.10.100 - avformat.h
+ Add AVFormatContext io_close2 which returns an int
+
2021-11-xx - xx - lavfi 8.19.100 - avfilter.h
Add AVFILTER_FLAG_METADATA_ONLY.
diff --git a/libavformat/avformat.h b/libavformat/avformat.h
index 75699f3a32..70b36d7682 100644
--- a/libavformat/avformat.h
+++ b/libavformat/avformat.h
@@ -1786,6 +1786,19 @@ typedef struct AVFormatContext {
* - decoding: set by user
*/
int max_probe_packets;
+
+/**
+ * A callback for closing the streams opened with
AVFormatContext.io_open().
+ *
+ * Using this is preferred over io_close, because this can return an error.
+ * Therefore this callback is used instead of io_close by the generic
+ * libavformat code if io_close is NULL or the default.
+ *
+ * @param s the format context
+ * @param pb IO context to be closed and freed
+ * @return 0 on success, a negative AVERROR code on failure
+ */
+int (*io_close2)(struct AVFormatContext *s, AVIOContext *pb);
} AVFormatContext;
/**
diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c
index 3f28f5ad71..4709bc6615 100644
--- a/libavformat/dashenc.c
+++ b/libavformat/dashenc.c
@@ -1546,6 +1546,7 @@ static int dash_init(AVFormatContext *s)
ctx->interrupt_callback= s->interrupt_callback;
ctx->opaque= s->opaque;
ctx->io_close = s->io_close;
+ctx->io_close2 = s->io_close2;
ctx->io_open = s->io_open;
ctx->strict_std_compliance = s->strict_std_compliance;
diff --git a/libavformat/fifo.c b/libavformat/fifo.c
index 2ee6dde830..86e5d369b5 100644
--- a/libavformat/fifo.c
+++ b/libavformat/fifo.c
@@ -499,6 +499,7 @@ static int fifo_mux_init(AVFormatContext *avf, const
AVOutputFormat *oformat,
return ret;
avf2->opaque = avf->opaque;
avf2->io_close = avf->io_close;
+avf2->io_close2 = avf->io_close2;
avf2->io_open = avf->io_open;
avf2->flags = avf->flags;
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index cfd0c036d1..ae288e408d 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -867,6 +867,7 @@ static int hls_mux_init(AVFormatContext *s, VariantStream
*vs)
oc->opaque = s->opaque;
oc->io_open = s->io_open;
oc->io_close = s->io_close;
+oc->io_close2= s->io_close2;
oc->strict_std_compliance= s->strict_std_compliance;
av_dict_copy(&oc->metadata, s->metadata, 0);
diff --git a/libavformat/internal.h b/libavformat/internal.h
index 20e93d9267..b0f70c614b 100644
--- a/libavformat/internal.h
+++ b/libavformat/internal.h
@@ -905,8 +905,16 @@ int ff_format_output_open(AVFormatContext *s, const char
*url, AVDictionary **op
/*
* A wrapper around AVFormatContext.io_close that should be used
* instead of calling the pointer directly.
+ *
+ * @param s AVFormatContext
+ * @param *pb the AVIOContext to be closed and freed. Can be NULL.
+ * @return >=0 on success, negative AVERROR in case of failure
*/
-void ff_format_io_close(AVFormatContext *s, AVIOContext **pb);
+int ff_format_io_close(AVFormatContext *s, AVIOContext **pb);
+
+/* Default io_close callback, not to be used directly, use ff_format_io_close
+ * instead. */
+void ff_format_io_close_default(AVFormatContext *s, AVIOContext *pb);
/**
* Utility function to check if the file uses http or https protocol
diff --git a/libavformat/options.c b/libavformat/options.c
index 72c9bdcefe..1634388acb 100644
--- a/libavformat/options.c
+++ b/libavformat/options.c
@@ -146,9 +
[FFmpeg-devel] [PATCH v23 20/21] avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles
Signed-off-by: softworkz
---
configure | 1 +
doc/filters.texi | 164 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_subscale.c | 883 ++
5 files changed, 1050 insertions(+)
create mode 100644 libavfilter/sf_subscale.c
diff --git a/configure b/configure
index 360e91d762..4be4303380 100755
--- a/configure
+++ b/configure
@@ -3706,6 +3706,7 @@ sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
splitcc_filter_deps="avcodec"
+subscale_filter_deps="swscale avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 26bf6014d0..8c49f79f86 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26226,6 +26226,170 @@ Set the rendering margin in pixels.
For rendering, alway use the latest event only, which is covering the given
point in time.
@end table
+@section subscale
+
+Provides high-quality scaling and rearranging functionality for graphical
subtitles.
+
+The subscale filter provides multiple approaches for manipulating
+the size and position of graphical subtitle rectangles wich can
+be combined or used separately.
+Scaling is performed by converting the palettized subtitle bitmaps
+to RGBA and re-quantization to palette colors afterwards via elbg algorithm.
+
+The two major operations are 'scale' and 're-arrange' with the
+latter being separated as 'arrange_h' and 'arrange_v'.
+
+
+Inputs:
+- 0: Subtitles [bitmap]
+
+Outputs:
+- 0: Subtitles [bitmap]
+
+It accepts the following parameters:
+
+@table @option
+
+@item w, width
+Set the width of the output.
+Width and height in case of graphical subtitles are just indicating
+a virtual size for which the output (consisting of 0-n bitmap rectangles)
+is intended to be displayed on.
+
+@item h, height
+Set the height of the output.
+
+@item margin_h
+Sets a horizontal margin to be preserverved when using any
+of the arrange modes.
+
+@item margin_v
+Sets a vertical margin to be preserverved when using any
+of the arrange modes.
+
+@item force_original_aspect_ratio
+Enable decreasing or increasing output video width or height if necessary to
+keep the original aspect ratio. Possible values:
+
+@table @samp
+@item disable
+Scale the video as specified and disable this feature.
+
+@item decrease
+The output video dimensions will automatically be decreased if needed.
+
+@item increase
+The output video dimensions will automatically be increased if needed.
+
+@end table
+
+
+@item scale_mode
+Specifies how subtitle bitmaps should be scaled.
+The scale factor is determined by the the factor between input
+and output size.
+
+@table @samp
+@item none
+Do not apply any common scaling.
+
+@item uniform
+Uniformly scale all subtitle bitmaps including their positions.
+
+@item uniform_no_reposition
+Uniformly scale all subtitle bitmaps without changing positions.
+
+@end table
+
+
+@item arrange_h
+Specifies how subtitle bitmaps should be arranged horizontally.
+
+@item arrange_v
+Specifies how subtitle bitmaps should be arranged vertically.
+
+
+@table @samp
+@item none
+Do not rearrange subtitle bitmaps.
+
+@item margin_no_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) when possible and without scaling.
+
+@item margin_and_scale
+Move subtitle bitmaps to be positioned inside the specified
+margin (margin_h or margin_v) and scale in case it doesn't fit.
+
+@item snapalign_no_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+No scaling will be applied.
+
+@item snapalign_and_scale
+Categorize subtitle bitmap positions as one of left/center/right
+or top/bottom/middle based on original positioning and apply
+these alignments for the target positioning.
+Bitmaps that do not fit inside the margins borders are
+scaled to fit.
+@end table
+
+@item eval
+Set evaluation mode for the expressions (@option{width}, @option{height}).
+
+It accepts the following values:
+@table @samp
+@item init
+Evaluate expressions only once during the filter initialization.
+
+@item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+@samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+@end table
+
+Default value is @samp{init}.
+
+
+@item num_colors
+Set the number of palette colors for output images.
+Choose the maximum (256) when further processing is done (e.g.
+overlaying on a video).
+When subtitles will be encoded as bitmap subtitles (e.g. dvbsub),
+a smaller number of palette colors (e.g. 4-16) might need to be used, depending
+on the target format and codec.
+
+@item bitmap_width_align
+@item bitmap_height_align
+Make sure that subtitle
[FFmpeg-devel] [PATCH v23 21/21] doc/APIchanges: update for subtitle filtering changes
Signed-off-by: softworkz --- doc/APIchanges | 23 +++ libavcodec/version.h | 2 +- libavutil/version.h | 4 ++-- 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index 2914ad6734..bb47faf13d 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -14,6 +14,29 @@ libavutil: 2021-04-27 API changes, most recent first: +2021-12-05 - xx - lavc 59.15.100 - avcodec.h + Deprecate avcodec_encode_subtitle(), use regular encode api now + +2021-12-05 - xx - lavc 59.15.100 - codec_desc.h + Add avcodec_descriptor_get_subtitle_format() + +2021-12-05 - xx - lavc 59.15.100 - avcodec.h + Deprecate avsubtitle_free() + Deprecate avcodec_decode_subtitle2(), use regular decode api now + +2021-12-05 - xx - lavu 57.11.100 - frame.h + Add AVMediaType field to AVFrame + Add Fields for carrying subtitle data to AVFrame + (subtitle_areas, subtitle_header, subtitle_pts, start/end time, etc.) + Add av_frame_get_buffer2() and deprecate av_frame_get_buffer() + +2021-12-05 - xx - lavu 57.11.100 - subfmt.h + Add struct AVSubtitleArea (replaces AVSubtitle) + Add av_get_subtitle_fmt_name() and av_get_subtitle_fmt() + +2021-12-05 - xx - lavu 57.11.100 - subfmt.h + Add enum AVSubtitleType (moved from lavc), add new values, deprecate existing + 2021-11-xx - xx - lavfi 8.19.100 - avfilter.h Add AVFILTER_FLAG_METADATA_ONLY. diff --git a/libavcodec/version.h b/libavcodec/version.h index 8a0b94f5aa..db71fb1130 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 14 +#define LIBAVCODEC_VERSION_MINOR 15 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavutil/version.h b/libavutil/version.h index c6cd0b0d79..cc8caadf50 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 10 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 11 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \ -- 2.30.2.windows.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v23 19/21] avfilter/graphicsub2text: Add new graphicsub2text filter (OCR)
Signed-off-by: softworkz --- configure| 1 + doc/filters.texi | 55 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 1 + libavfilter/sf_graphicsub2text.c | 354 +++ 5 files changed, 413 insertions(+) create mode 100644 libavfilter/sf_graphicsub2text.c diff --git a/configure b/configure index 73a7267cf1..360e91d762 100755 --- a/configure +++ b/configure @@ -3640,6 +3640,7 @@ frei0r_filter_deps="frei0r" frei0r_src_filter_deps="frei0r" fspp_filter_deps="gpl" gblur_vulkan_filter_deps="vulkan spirv_compiler" +graphicsub2text_filter_deps="libtesseract" hflip_vulkan_filter_deps="vulkan spirv_compiler" histeq_filter_deps="gpl" hqdn3d_filter_deps="gpl" diff --git a/doc/filters.texi b/doc/filters.texi index 743c36c432..26bf6014d0 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -25820,6 +25820,61 @@ ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv"; -filter_comple @end example @end itemize +@section graphicsub2text + +Converts graphic subtitles to text subtitles by performing OCR. + +For this filter to be available, ffmpeg needs to be compiled with libtesseract (see https://github.com/tesseract-ocr/tesseract). +Language models need to be downloaded from https://github.com/tesseract-ocr/tessdata and put into as subfolder named 'tessdata' or into a folder specified via the environment variable 'TESSDATA_PREFIX'. +The path can also be specified via filter option (see below). + +Note: These models are including the data for both OCR modes. + +Inputs: +- 0: Subtitles [bitmap] + +Outputs: +- 0: Subtitles [text] + +It accepts the following parameters: + +@table @option +@item ocr_mode +The character recognition mode to use. + +Supported OCR modes are: + +@table @var +@item 0, tesseract +This is the classic libtesseract operation mode. It is fast but less accurate than LSTM. +@item 1, lstm +Newer OCR implementation based on ML models. Provides usually better results, requires more processing resources. +@item 2, both +Use a combination of both modes. +@end table + +@item tessdata_path +The path to a folder containing the language models to be used. + +@item language +The recognition language. It needs to match the first three characters of a language model file in the tessdata path. + +@end table + + +@subsection Examples + +@itemize +@item +Convert DVB graphic subtitles to ASS (text) subtitles + +Note: For this to work, you need to have the data file 'eng.traineddata' in a 'tessdata' subfolder (see above). +@example +ffmpeg ffmpeg -loglevel verbose -i "https://streams.videolan.org/streams/ts/video_subs_ttxt%2Bdvbsub.ts"; -filter_complex "[0:13]graphicsub2text=ocr_mode=both" -c:s ass -y output.mkv +@end example +@end itemize + + @section graphicsub2video Renders graphic subtitles as video frames. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 2224e5fe5f..3b972e134b 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -296,6 +296,8 @@ OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o vulka OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o +OBJS-$(CONFIG_GRAPHICSUB2TEXT_FILTER)+= sf_graphicsub2text.o +OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o framesync.o OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o OBJS-$(CONFIG_GRAYWORLD_FILTER) += vf_grayworld.o OBJS-$(CONFIG_GREYEDGE_FILTER) += vf_colorconstancy.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 6adde2b9f6..f70f08dc5a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -545,6 +545,7 @@ extern const AVFilter ff_avf_showwaves; extern const AVFilter ff_avf_showwavespic; extern const AVFilter ff_vaf_spectrumsynth; extern const AVFilter ff_sf_censor; +extern const AVFilter ff_sf_graphicsub2text; extern const AVFilter ff_sf_showspeaker; extern const AVFilter ff_sf_splitcc; extern const AVFilter ff_sf_stripstyles; diff --git a/libavfilter/sf_graphicsub2text.c b/libavfilter/sf_graphicsub2text.c new file mode 100644 index 00..ef10d60efd --- /dev/null +++ b/libavfilter/sf_graphicsub2text.c @@ -0,0 +1,354 @@ +/* + * Copyright (c) 2021 softworkz + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for mor
[FFmpeg-devel] [PATCH v23 18/21] avfilter/splitcc: Add splitcc filter for closed caption handling
- splitcc {V -> VS)
Extract closed-caption (A53) data from video
frames as subtitle Frames
ffmpeg -y -loglevel verbose -i
"https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v]splitcc[vid1],textmod=mode=remove_chars:find='@',[vid1]overlay_textsubs"
output.mkv
Signed-off-by: softworkz
---
configure| 1 +
doc/filters.texi | 63 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_splitcc.c | 378 +++
5 files changed, 444 insertions(+)
create mode 100644 libavfilter/sf_splitcc.c
diff --git a/configure b/configure
index 16ab6b46c3..73a7267cf1 100755
--- a/configure
+++ b/configure
@@ -3704,6 +3704,7 @@ spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
sr_filter_deps="avformat swscale"
sr_filter_select="dnn"
stereo3d_filter_deps="gpl"
+splitcc_filter_deps="avcodec"
subtitles_filter_deps="avformat avcodec libass"
super2xsai_filter_deps="gpl"
pixfmts_super2xsai_test_deps="super2xsai_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index 241c5d469b..743c36c432 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -26054,6 +26054,69 @@ ffmpeg -i INPUT -filter_complex
"showspeaker=format=colon:style='@{\\c&HDD&\
@end example
@end itemize
+
+@section splitcc
+
+Split-out closed-caption/A53 subtitles from video frame side data.
+
+This filter provides an input and an output for video frames, which are just
passed through without modification.
+The second out provides subtitle frames which are extracted from video frame
side data.
+
+Inputs:
+@itemize
+@item 0: Video [ALL]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@item 1: Subtitles [TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item use_cc_styles
+Emit closed caption style header.
+This will make closed captions appear in white font with a black rectangle
background.
+
+@item real_time
+Emit subtitle events as they are decoded for real-time display.
+
+@item real_time_latency_msec
+Minimum elapsed time between emitting real-time subtitle events.
+Only applies to real_time mode.
+
+@item data_field
+Select data field. Possible values:
+
+@table @samp
+@item auto
+Pick first one that appears.
+@item first
+@item second
+@end table
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Extract closed captions as text subtitle stream and overlay it onto the video
in cc style (black bar background):
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=use_cc_styles=1[vid1][sub1];[vid1][sub1]overlaytextsubs"
output.mkv
+@end example
+
+@item
+A nicer variant, using realtime output from cc_dec and rendering it with the
render_latest_only parameter from overlaytextsubs to avoid ghosting by timely
overlap.
+@example
+ffmpeg -i "https://streams.videolan.org/streams/ts/CC/NewsStream-608-ac3.ts";
-filter_complex
"[0:v:0]splitcc=real_time=1:real_time_latency_msec=200[vid1][sub1];[vid1][sub1]overlaytextsubs=render_latest_only=1"
output.mkv
+@end example
+@end itemize
+
+
@section textsub2video
Converts text subtitles to video frames.
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7d7da0c59a..2224e5fe5f 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_SPLITCC_FILTER)+= sf_splitcc.o
OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 27680a2f00..6adde2b9f6 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_splitcc;
extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
diff --git a/libavfilter/sf_splitcc.c b/libavfilter/sf_splitcc.c
new file mode 100644
index 00..3556d084d9
--- /dev/null
+++ b/libavfilter/sf_splitcc.c
@@ -0,0 +1,378 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABIL
[FFmpeg-devel] [PATCH v23 17/21] avfilter/stripstyles: Add stripstyles filter
- stripstyles {S -> S)
Remove all inline styles from subtitle events
Signed-off-by: softworkz
---
doc/filters.texi | 37 +++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/sf_stripstyles.c | 196 +++
4 files changed, 235 insertions(+)
create mode 100644 libavfilter/sf_stripstyles.c
diff --git a/doc/filters.texi b/doc/filters.texi
index e87d3993df..241c5d469b 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25704,6 +25704,43 @@ ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.
@end example
@end itemize
+@section stripstyles
+
+Remove all inline styles from subtitle events.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item remove_animated
+Also remove text which is subject to animation (default: true)
+Usually, animated text elements are used used in addition to static subtitle
lines for creating effects, so in most cases it is safe to remove the animation
content.
+If subtitle text is missing, try setting this to false.
+
+@item select_layer
+Process only ASS subtitle events from a specific layer. This allows to filter
out certain effects where an ASS author duplicates the text onto multiple
layers.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Remove styles and animations from ASS subtitles and output events from ass
layer 0 only. Then convert asn save as SRT stream:
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:1]stripstyles=select_layer=0" -map 0 -c:s srt output.mkv
+@end example
+@end itemize
+
@section textmod
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index d2995fbec6..7d7da0c59a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -557,6 +557,7 @@ OBJS-$(CONFIG_NULLSINK_FILTER) +=
vsink_nullsink.o
OBJS-$(CONFIG_CENSOR_FILTER) += sf_textmod.o
OBJS-$(CONFIG_SHOW_SPEAKER_FILTER) += sf_textmod.o
OBJS-$(CONFIG_TEXTMOD_FILTER)+= sf_textmod.o
+OBJS-$(CONFIG_STRIPSTYLES_FILTER)+= sf_stripstyles.o
# multimedia filters
OBJS-$(CONFIG_ABITSCOPE_FILTER) += avf_abitscope.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 673f9fb839..27680a2f00 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -546,6 +546,7 @@ extern const AVFilter ff_avf_showwavespic;
extern const AVFilter ff_vaf_spectrumsynth;
extern const AVFilter ff_sf_censor;
extern const AVFilter ff_sf_showspeaker;
+extern const AVFilter ff_sf_stripstyles;
extern const AVFilter ff_sf_textmod;
extern const AVFilter ff_svf_graphicsub2video;
extern const AVFilter ff_svf_textsub2video;
diff --git a/libavfilter/sf_stripstyles.c b/libavfilter/sf_stripstyles.c
new file mode 100644
index 00..82cb9c7647
--- /dev/null
+++ b/libavfilter/sf_stripstyles.c
@@ -0,0 +1,196 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * text subtitle filter which removes inline-styles from subtitles
+ */
+
+#include "libavutil/opt.h"
+#include "internal.h"
+#include "libavutil/ass_split_internal.h"
+#include "libavutil/bprint.h"
+
+typedef struct StripStylesContext {
+const AVClass *class;
+enum AVSubtitleType format;
+int remove_animated;
+int select_layer;
+} StripStylesContext;
+
+typedef struct DialogContext {
+StripStylesContext* ss_ctx;
+AVBPrint buffer;
+int drawing_scale;
+int is_animated;
+} DialogContext;
+
+static void dialog_text_cb(void *priv, const char *text, int len)
+{
+DialogContext *s = priv;
+
+av_log(s->ss_ctx, AV_LOG_DEBUG, "dialog_text_cb: %s\n", text);
+
+if (!s->drawing_scale && (!s->is_animated || !s->ss_ctx->remove_animated))
+av_bprint_append_data(&s->buffer, text, len);
+}
+
+static void dialog_new_line_cb(void *priv, int forced)
+{
+DialogContext *s = priv;
+if (!s->drawing_scale && !s->is_animated)
+av_bprint_append_data(&s->buffer, forced ? "\\N" : "\\n", 2);
+}
+
+static void di
[FFmpeg-devel] [PATCH v23 16/21] avfilter/textmod: Add textmod, censor and show_speaker filters
- textmod {S -> S)
Modify subtitle text in a number of ways
- censor {S -> S)
Censor subtitles using a word list
- show_speaker {S -> S)
Prepend speaker names from ASS subtitles to the visible text lines
Signed-off-by: softworkz
---
doc/filters.texi | 206
libavfilter/Makefile | 5 +
libavfilter/allfilters.c | 3 +
libavfilter/sf_textmod.c | 697 +++
4 files changed, 911 insertions(+)
create mode 100644 libavfilter/sf_textmod.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 7041f9031e..e87d3993df 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25644,6 +25644,145 @@ existing filters using @code{--disable-filters}.
Below is a description of the currently available subtitle filters.
+
+@section censor
+
+Censor selected words in text subtitles.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The censoring mode to apply.
+
+Supported censoring modes are:
+
+@table @var
+@item 0, keep_first_last
+Replace all characters with the 'censor_char' except the first and the last
character of a word.
+For words with less than 4 characters, the last character will be replaced as
well.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 1, keep_first
+Replace all characters with the 'censor_char' except the first character of a
word.
+For words with less than 3 characters, the first character will be replaced as
well.
+@item 2, all
+Replace all characters with the 'censor_char'.
+@end table
+
+@item words
+A list of words to censor, separated by 'separator'.
+
+@item words_file
+Specify a file from which to load the contents for the 'words' parameter.
+
+@item censor_char
+Single character used as replacement for censoring.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Censor a few given words with a pound character.
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex
"[0:1]censor=words='diss,louder,hope,beam,word':censor_char='#'" -map 0 -y
output.mkv
+@end example
+@end itemize
+
+
+@section textmod
+
+Modify subtitle text in a number of ways.
+
+Inputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Subtitles[TEXT]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item mode
+The kind of text modification to apply
+
+Supported operation modes are:
+
+@table @var
+@item 0, leet
+Convert subtitle text to 'leet speak'. It's primarily useful for testing as
the modification will be visible with almost all text lines.
+@item 1, to_upper
+Change all text to upper case. Might improve readability.
+@item 2, to_lower
+Change all text to lower case.
+@item 3, replace_chars
+Replace one or more characters. Requires the find and replace parameters to be
specified.
+Both need to be equal in length.
+The first char in find is replaced by the first char in replace, same for all
subsequent chars.
+@item 4, remove_chars
+Remove certain characters. Requires the find parameter to be specified.
+All chars in the find parameter string will be removed from all subtitle text.
+@item 5, replace_words
+Replace one or more words. Requires the find and replace parameters to be
specified. Multiple words must be separated by the delimiter char specified vie
the separator parameter (default: ',').
+The number of words in the find and replace parameters needs to be equal.
+The first word in find is replaced by the first word in replace, same for all
subsequent words
+@item 6, remove_words
+Remove certain words. Requires the find parameter to be specified. Multiple
words must be separated by the delimiter char specified vie the separator
parameter (default: ',').
+All words in the find parameter string will be removed from all subtitle text.
+@end table
+
+@item find
+Required for replace_chars, remove_chars, replace_words and remove_words.
+
+@item find_file
+Specify a file from which to load the contents for the 'find' parameter.
+
+@item replace
+Required for replace_chars and replace_words.
+
+@item replace_file
+Specify a file from which to load the contents for the 'replace' parameter.
+
+@item separator
+Delimiter character for words. Used with replace_words and remove_words- Must
be a single character.
+The default is '.'.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Change all characters to upper case while keeping all styles and animations:
+@example
+ffmpeg -i "https://streams.videolan.org/ffmpeg/mkv_subtitles.mkv";
-filter_complex "[0:s]textmod=mode=to_upper" -map 0 -y out.mkv
+@end example
+@item
+Remove a set of symbol characters for am improved and smooth
[FFmpeg-devel] [PATCH v23 15/21] avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters
- overlaytextsubs {VS -> V)
Overlay text subtitles onto a video stream.
- textsubs2video {S -> V)
Converts text subtitles to video frames
Signed-off-by: softworkz
---
configure| 2 +
doc/filters.texi | 113 ++
libavfilter/Makefile | 2 +
libavfilter/allfilters.c | 4 +-
libavfilter/vf_overlaytextsubs.c | 646 +++
5 files changed, 766 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_overlaytextsubs.c
diff --git a/configure b/configure
index 7b203d2bf2..16ab6b46c3 100755
--- a/configure
+++ b/configure
@@ -3666,6 +3666,7 @@ overlay_opencl_filter_deps="opencl"
overlay_qsv_filter_deps="libmfx"
overlay_qsv_filter_select="qsvvpp"
overlay_vulkan_filter_deps="vulkan spirv_compiler"
+overlaytextsubs_filter_deps="avcodec libass"
owdenoise_filter_deps="gpl"
pad_opencl_filter_deps="opencl"
pan_filter_deps="swresample"
@@ -3710,6 +3711,7 @@ superequalizer_filter_deps="avcodec"
superequalizer_filter_select="rdft"
surround_filter_deps="avcodec"
surround_filter_select="rdft"
+textsub2video_filter_deps="avcodec libass"
tinterlace_filter_deps="gpl"
tinterlace_merge_test_deps="tinterlace_filter"
tinterlace_pad_test_deps="tinterlace_filter"
diff --git a/doc/filters.texi b/doc/filters.texi
index db4552ba10..7041f9031e 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25752,6 +25752,119 @@ Overlay PGS subtitles
ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
@end example
@end itemize
+
+@section overlaytextsubs
+
+Overlay text subtitles onto a video stream.
+
+This filter supersedes the classic @ref{subtitles} filter opposed to which it
does no longer require to open and access the source stream separately, which
is often causing problems or doesn't even work for non-local or slow sources.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item alpha
+Process alpha channel, by default alpha channel is untouched.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay ASS subtitles with animations:
+@example
+ffmpeg -i
"http://streams.videolan.org/samples/sub/SSA/subtitle_testing_complex.mkv";
-filter_complex "[0:v]overlaytextsubs" -map 0 -y out.mkv
+@end example
+@end itemize
+
+@section textsub2video
+
+Converts text subtitles to video frames.
+
+For overlaying text subtitles onto video frames it is recommended to use the
overlay_textsubs filter.
+The textsub2video is useful for for creating transparent text-frames when
overlay is done via hw acceleration
+
+Inputs:
+@itemize
+@item 0: Subtitles [TEXT]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+
+@item rate, r
+Set the framerate for updating overlay frames.
+Normally, overlay frames will only be updated each time when the subtitles to
display are changing.
+In cases where subtitles include advanced features (like animation), this
parameter determines the frequency by which the overlay frames should be
updated.
+
+@item size, s
+Set the output frame size.
+Allows to override the size of output video frames.
+
+@item fonts_dir
+Set a directory path containing fonts that can be used by the filter.
+These fonts will be used in addition to whatever the font provider uses.
+
+@item default_font_path
+Path to a font file to be used as the default font.
+
+@item font_size
+Set the default font size.
+
+@item fontconfig_file
+Path to ASS fontconfig configuration file.
+
+@item force_style
+Override default style or script info parameters of the subtitles. It accepts a
+string containing ASS style format @code{KEY=VALUE} couples separated by ",".
+
+@item margin
+Set the rendering margin in pixels.
+
+@item render_latest_only
+For rendering, alway use the latest event only, which is covering the given
point in time.
+@end table
+
@c man end SUBTITLE FILTERS
@chapter Multimedia Filters
diff --git a/libavf
[FFmpeg-devel] [PATCH v23 14/21] avfilter/avfilter: Fix hardcoded input index
This fix targets (rare) cases where multiple input pads have a
.filter_frame function. ff_request_frame_to_filter needs
to call ff_request_frame with the correct input pad
instead of the hardcoded first one.
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 75d5e86539..aa9aa71f53 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -463,7 +463,7 @@ static int64_t guess_status_pts(AVFilterContext *ctx, int
status, AVRational lin
return AV_NOPTS_VALUE;
}
-static int ff_request_frame_to_filter(AVFilterLink *link)
+static int ff_request_frame_to_filter(AVFilterLink *link, int input_index)
{
int ret = -1;
@@ -472,8 +472,8 @@ static int ff_request_frame_to_filter(AVFilterLink *link)
link->frame_blocked_in = 1;
if (link->srcpad->request_frame)
ret = link->srcpad->request_frame(link);
-else if (link->src->inputs[0])
-ret = ff_request_frame(link->src->inputs[0]);
+else if (link->src->inputs[input_index])
+ret = ff_request_frame(link->src->inputs[input_index]);
if (ret < 0) {
if (ret != AVERROR(EAGAIN) && ret != link->status_in)
ff_avfilter_link_set_in_status(link, ret,
guess_status_pts(link->src, ret, link->time_base));
@@ -1172,6 +1172,14 @@ static int forward_status_change(AVFilterContext
*filter, AVFilterLink *in)
{
unsigned out = 0, progress = 0;
int ret;
+int input_index = 0;
+
+for (int i = 0; i < in->dst->nb_inputs; i++) {
+if (&in->dst->input_pads[i] == in->dstpad) {
+input_index = i;
+break;
+}
+}
av_assert0(!in->status_out);
if (!filter->nb_outputs) {
@@ -1181,7 +1189,7 @@ static int forward_status_change(AVFilterContext *filter,
AVFilterLink *in)
while (!in->status_out) {
if (!filter->outputs[out]->status_in) {
progress++;
-ret = ff_request_frame_to_filter(filter->outputs[out]);
+ret = ff_request_frame_to_filter(filter->outputs[out],
input_index);
if (ret < 0)
return ret;
}
@@ -1218,7 +1226,7 @@ static int ff_filter_activate_default(AVFilterContext
*filter)
for (i = 0; i < filter->nb_outputs; i++) {
if (filter->outputs[i]->frame_wanted_out &&
!filter->outputs[i]->frame_blocked_in) {
-return ff_request_frame_to_filter(filter->outputs[i]);
+return ff_request_frame_to_filter(filter->outputs[i], 0);
}
}
return FFERROR_NOT_READY;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v23 13/21] fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API
This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.
The heartbeat functionality from the previous sub2video implementation
is retained and applied to all subtitle frames (bitmap, text, ..).
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.
Justification for changed test refs:
- sub2video
The new results are identical excepting the last frame which
is due to the implementation changes
- sub2video_basic
The previous results had some incorrect output because multiple
frames had the same dts
The non-empty content frames are visually identical, the different
CRC is due to the different blending algorithm that is being used.
- sub2video_time_limited
The third frame in the previous ref was a repetition, which doesn't
happen anymore with the new subtitle filtering.
- sub-dvb
Running ffprobe -show_frames on the source file shows that there
are 7 subtitle frames with 0 rects in the source at the start
and 2 at the end. This translates to the 14 and 4 additional
entries in the new test results.
- filter-overlay-dvdsub-2397
Overlay results have slightly different CRCs due to different
blending implementation
Signed-off-by: softworkz
---
fftools/ffmpeg.c | 584 +++---
fftools/ffmpeg.h | 15 +-
fftools/ffmpeg_filter.c | 217 +---
fftools/ffmpeg_hw.c | 2 +-
fftools/ffmpeg_opt.c | 3 +-
tests/ref/fate/filter-overlay-dvdsub-2397 | 181 ---
tests/ref/fate/sub-dvb| 162 +++---
tests/ref/fate/sub2video | 116 ++---
tests/ref/fate/sub2video_basic| 135 ++---
tests/ref/fate/sub2video_time_limited | 4 +-
10 files changed, 725 insertions(+), 694 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 0f1834cdbe..0eee4163ac 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -143,8 +143,6 @@ static int want_sdp = 1;
static BenchmarkTimeStamps current_time;
AVIOContext *progress_avio = NULL;
-static uint8_t *subtitle_out;
-
InputStream **input_streams = NULL;
intnb_input_streams = 0;
InputFile **input_files = NULL;
@@ -169,163 +167,6 @@ static int restore_tty;
static void free_input_threads(void);
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static int sub2video_get_blank_frame(InputStream *ist)
-{
-int ret;
-AVFrame *frame = ist->sub2video.frame;
-
-av_frame_unref(frame);
-ist->sub2video.frame->width = ist->dec_ctx->width ? ist->dec_ctx->width
: ist->sub2video.w;
-ist->sub2video.frame->height = ist->dec_ctx->height ? ist->dec_ctx->height
: ist->sub2video.h;
-ist->sub2video.frame->format = AV_PIX_FMT_RGB32;
-if ((ret = av_frame_get_buffer(frame, 0)) < 0)
-return ret;
-memset(frame->data[0], 0, frame->height * frame->linesize[0]);
-return 0;
-}
-
-static void sub2video_copy_rect(uint8_t *dst, int dst_linesize, int w, int h,
-AVSubtitleRect *r)
-{
-uint32_t *pal, *dst2;
-uint8_t *src, *src2;
-int x, y;
-
-if (r->type != SUBTITLE_BITMAP) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: non-bitmap subtitle\n");
-return;
-}
-if (r->x < 0 || r->x + r->w > w || r->y < 0 || r->y + r->h > h) {
-av_log(NULL, AV_LOG_WARNING, "sub2video: rectangle (%d %d %d %d)
overflowing %d %d\n",
-r->x, r->y, r->w, r->h, w, h
-);
-return;
-}
-
-dst += r->y * dst_linesize + r->x * 4;
-src = r->data[0];
-pal = (uint32_t *)r->data[1];
-for (y = 0; y < r->h; y++) {
-dst2 = (uint32_t *)dst;
-src2 = src;
-for (x = 0; x < r->w; x++)
-*(dst2++) = pal[*(src2++)];
-dst += dst_linesize;
-src += r->linesize[0];
-}
-}
-
-static void sub2video_push_ref(InputStream *ist, int64_t pts)
-{
-AVFrame *frame = ist->sub2video.frame;
-int i;
-int ret;
-
-av_assert1(frame->data[0]);
-ist->sub2video.last_pts = frame->pts = pts;
-for (i = 0; i < ist->nb_filters; i++) {
-ret = av_buffersrc_add_frame_flags(ist->filters[i]->filter, frame,
- AV_BUFFERSRC_FLAG_KEEP_REF |
- AV_BUFFERSRC_FLAG_PUSH);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Error while add the frame to buffer
source(%s).\n",
- av_err2str(ret));
-}
-}
-
-void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
-{
-AVFrame *frame = ist->sub2video.frame;
-int8_t *dst;
-int dst_linesize;
-int num_rects, i;
-int64_t pts, end_pts;
-
-
[FFmpeg-devel] [PATCH v23 12/21] avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters
- overlaygraphicsubs (VS -> V)
Overlay graphic subtitles onto a video stream
- graphicsub2video {S -> V)
Converts graphic subtitles to video frames (with alpha)
Gets auto-inserted for retaining compatibility with
sub2video command lines
Signed-off-by: softworkz
---
doc/filters.texi| 118 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 2 +
libavfilter/vf_overlaygraphicsubs.c | 737
4 files changed, 859 insertions(+)
create mode 100644 libavfilter/vf_overlaygraphicsubs.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 8eff460cd9..db4552ba10 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -25636,6 +25636,124 @@ tools.
@c man end VIDEO SINKS
+@chapter Subtitle Filters
+@c man begin SUBTITLE FILTERS
+
+When you configure your FFmpeg build, you can disable any of the
+existing filters using @code{--disable-filters}.
+
+Below is a description of the currently available subtitle filters.
+
+@section graphicsub2video
+
+Renders graphic subtitles as video frames.
+
+This filter replaces the previous "sub2video" hack which did the conversion
implicitly and up-front as subtitle filtering wasn't possible at that time.
+To retain compatibility with earlier sub2video command lines, this filter is
being auto-inserted in those cases.
+
+For overlaying graphicsal subtitles it is recommended to use the
'overlay_graphicsubs' filter which is more efficient and takes less processing
resources.
+
+This filter is still useful in cases where the overlay is done with hardware
acceleration (e.g. overlay_qsv, overlay_vaapi, overlay_cuda) for preparing the
overlay frames.
+
+Inputs:
+@itemize
+@item 0: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video [RGB32]
+@end itemize
+
+
+It accepts the following parameters:
+
+@table @option
+@item size, s
+Set the size of the output video frame.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+(not recommended - better use overlay_graphicsubs)
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:1]graphicsub2video[subs];[0:0][subs]overlay" output.mp4
+@end example
+
+@item
+Overlay PGS subtitles implicitly
+The graphicsub2video is inserted automatically for compatibility with legacy
command lines.
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlay" output.mp4
+@end example
+@end itemize
+
+@section overlaygraphicsubs
+
+Overlay graphic subtitles onto a video stream.
+
+This filter can blend graphical subtitles on a video stream directly, i.e.
without creating full-size alpha images first.
+The blending operation is limited to the area of the subtitle rectangles,
which also means that no processing is done at times where no subtitles are to
be displayed.
+
+Inputs:
+@itemize
+@item 0: Video [YUV420P, YUV422P, YUV444P, ARGB, RGBA, ABGR, BGRA, RGB24,
BGR24]
+@item 1: Subtitles [BITMAP]
+@end itemize
+
+Outputs:
+@itemize
+@item 0: Video (same as input)
+@end itemize
+
+It accepts the following parameters:
+
+@table @option
+@item x
+@item y
+Set the expression for the x and y coordinates of the overlaid video
+on the main video. Default value is "0" for both expressions. In case
+the expression is invalid, it is set to a huge value (meaning that the
+overlay will not be displayed within the output visible area).
+
+@item eof_action
+See @ref{framesync}.
+
+@item eval
+Set when the expressions for @option{x}, and @option{y} are evaluated.
+
+It accepts the following values:
+@table @samp
+@item init
+only evaluate expressions once during the filter initialization or
+when a command is processed
+
+@item frame
+evaluate expressions for each incoming frame
+@end table
+
+Default value is @samp{frame}.
+
+@item shortest
+See @ref{framesync}.
+
+@end table
+
+@subsection Examples
+
+@itemize
+@item
+Overlay PGS subtitles
+@example
+ffmpeg -i
"https://streams.videolan.org/samples/sub/PGS/Girl_With_The_Dragon_Tattoo_2%3A23%3A56.mkv";
-filter_complex "[0:0][0:1]overlaygraphicsubs" output.mp4
+@end example
+@end itemize
+@c man end SUBTITLE FILTERS
+
@chapter Multimedia Filters
@c man begin MULTIMEDIA FILTERS
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 664566a18a..525b3a6e3c 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -295,6 +295,7 @@ OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
vulkan_filter.o
OBJS-$(CONFIG_GEQ_FILTER)+= vf_geq.o
OBJS-$(CONFIG_GRADFUN_FILTER)+= vf_gradfun.o
+OBJS-$(CONFIG_GRAPHICSUB2VIDEO_FILTER) += vf_overlaygraphicsubs.o
framesync.o
OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
OBJS-$(CONFIG_GRAYWORLD_FILTER)
[FFmpeg-devel] [PATCH v23 11/21] avfilter/sbuffer: Add sbuffersrc and sbuffersink filters
Signed-off-by: softworkz
---
configure| 2 +-
libavfilter/allfilters.c | 2 ++
libavfilter/buffersink.c | 54 ++
libavfilter/buffersink.h | 7
libavfilter/buffersrc.c | 72
libavfilter/buffersrc.h | 1 +
6 files changed, 137 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index 04f319b197..7b203d2bf2 100755
--- a/configure
+++ b/configure
@@ -7808,7 +7808,7 @@ print_enabled_components(){
fi
done
if [ "$name" = "filter_list" ]; then
-for c in asrc_abuffer vsrc_buffer asink_abuffer vsink_buffer; do
+for c in asrc_abuffer vsrc_buffer ssrc_sbuffer asink_abuffer
vsink_buffer ssink_sbuffer; do
printf "&ff_%s,\n" $c >> $TMPH
done
fi
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index b1af2cbcc8..e134ac8059 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -553,8 +553,10 @@ extern const AVFilter ff_avsrc_movie;
* being the same while having different 'types'). */
extern const AVFilter ff_asrc_abuffer;
extern const AVFilter ff_vsrc_buffer;
+extern const AVFilter ff_ssrc_sbuffer;
extern const AVFilter ff_asink_abuffer;
extern const AVFilter ff_vsink_buffer;
+extern const AVFilter ff_ssink_sbuffer;
extern const AVFilter ff_af_afifo;
extern const AVFilter ff_vf_fifo;
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index c0215669e7..0b268c2fa4 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -29,6 +29,8 @@
#include "libavutil/internal.h"
#include "libavutil/opt.h"
+#include "libavcodec/avcodec.h"
+
#define FF_INTERNAL_FIELDS 1
#include "framequeue.h"
@@ -57,6 +59,10 @@ typedef struct BufferSinkContext {
int *sample_rates; ///< list of accepted sample rates
int sample_rates_size;
+/* only used for subtitles */
+enum AVSubtitleType *subtitle_types; ///< list of accepted subtitle
types, must be terminated with -1
+int subtitle_types_size;
+
AVFrame *peeked_frame;
} BufferSinkContext;
@@ -305,6 +311,28 @@ static int asink_query_formats(AVFilterContext *ctx)
return 0;
}
+static int ssink_query_formats(AVFilterContext *ctx)
+{
+BufferSinkContext *buf = ctx->priv;
+AVFilterFormats *formats = NULL;
+unsigned i;
+int ret;
+
+CHECK_LIST_SIZE(subtitle_types)
+if (buf->subtitle_types_size) {
+for (i = 0; i < NB_ITEMS(buf->subtitle_types); i++)
+if ((ret = ff_add_subtitle_type(&formats, buf->subtitle_types[i]))
< 0)
+return ret;
+if ((ret = ff_set_common_formats(ctx, formats)) < 0)
+return ret;
+} else {
+if ((ret = ff_default_query_formats(ctx)) < 0)
+return ret;
+}
+
+return 0;
+}
+
#define OFFSET(x) offsetof(BufferSinkContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption buffersink_options[] = {
@@ -322,9 +350,16 @@ static const AVOption abuffersink_options[] = {
{ NULL },
};
#undef FLAGS
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_SUBTITLE_PARAM
+static const AVOption sbuffersink_options[] = {
+{ "subtitle_types", "set the supported subtitle formats",
OFFSET(subtitle_types), AV_OPT_TYPE_BINARY, .flags = FLAGS },
+{ NULL },
+};
+#undef FLAGS
AVFILTER_DEFINE_CLASS(buffersink);
AVFILTER_DEFINE_CLASS(abuffersink);
+AVFILTER_DEFINE_CLASS(sbuffersink);
static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
{
@@ -363,3 +398,22 @@ const AVFilter ff_asink_abuffer = {
.outputs = NULL,
FILTER_QUERY_FUNC(asink_query_formats),
};
+
+static const AVFilterPad avfilter_ssink_sbuffer_inputs[] = {
+{
+.name = "default",
+.type = AVMEDIA_TYPE_SUBTITLE,
+},
+};
+
+const AVFilter ff_ssink_sbuffer = {
+.name = "sbuffersink",
+.description = NULL_IF_CONFIG_SMALL("Buffer subtitle frames, and make
them available to the end of the filter graph."),
+.priv_class= &sbuffersink_class,
+.priv_size = sizeof(BufferSinkContext),
+.init = common_init,
+.activate = activate,
+FILTER_INPUTS(avfilter_ssink_sbuffer_inputs),
+.outputs = NULL,
+FILTER_QUERY_FUNC(ssink_query_formats),
+};
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 69ed0f29a8..11905abdc5 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -129,6 +129,13 @@ typedef struct AVABufferSinkParams {
*/
attribute_deprecated
AVABufferSinkParams *av_abuffersink_params_alloc(void);
+
+/**
+ * Deprecated and unused struct to use for initializing an sbuffersink context.
+ */
+typedef struct AVSBufferSinkParams {
+const int *subtitle_type;
+} AVSBufferSinkParams;
#endif
/**
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0611872f1..d2362999a2 100644
--- a/libavfilter/buffersrc.
[FFmpeg-devel] [PATCH v23 10/21] avfilter/avfilter: Handle subtitle frames
Signed-off-by: softworkz
---
libavfilter/avfilter.c | 8 +---
libavfilter/avfilter.h | 11 +++
libavfilter/avfiltergraph.c | 5 +
libavfilter/formats.c | 22 ++
libavfilter/formats.h | 3 +++
libavfilter/internal.h | 18 +++---
6 files changed, 61 insertions(+), 6 deletions(-)
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index df5b8f483c..75d5e86539 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -56,7 +56,8 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->linesize[0], ref->linesize[1], ref->linesize[2],
ref->linesize[3],
ref->pts, ref->pkt_pos);
-if (ref->width) {
+switch(ref->type) {
+case AVMEDIA_TYPE_VIDEO:
ff_tlog(ctx, " a:%d/%d s:%dx%d i:%c iskey:%d type:%c",
ref->sample_aspect_ratio.num, ref->sample_aspect_ratio.den,
ref->width, ref->height,
@@ -64,12 +65,13 @@ static void tlog_ref(void *ctx, AVFrame *ref, int end)
ref->top_field_first ? 'T' : 'B',/* Top / Bottom */
ref->key_frame,
av_get_picture_type_char(ref->pict_type));
-}
-if (ref->nb_samples) {
+break;
+case AVMEDIA_TYPE_AUDIO:
ff_tlog(ctx, " cl:%"PRId64"d n:%d r:%d",
ref->channel_layout,
ref->nb_samples,
ref->sample_rate);
+break;
}
ff_tlog(ctx, "]%s", end ? "\n" : "");
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index b105dc3159..9f917deb41 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -45,6 +45,7 @@
#include "libavutil/log.h"
#include "libavutil/samplefmt.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "libavfilter/version.h"
@@ -343,6 +344,12 @@ typedef struct AVFilter {
* and outputs use the same sample rate and channel count/layout.
*/
const enum AVSampleFormat *samples_list;
+/**
+ * Analogous to pixels, but delimited by AV_SUBTITLE_FMT_NONE
+ * and restricted to filters that only have AVMEDIA_TYPE_SUBTITLE
+ * inputs and outputs.
+ */
+const enum AVSubtitleType *subs_list;
/**
* Equivalent to { pix_fmt, AV_PIX_FMT_NONE } as pixels_list.
*/
@@ -351,6 +358,10 @@ typedef struct AVFilter {
* Equivalent to { sample_fmt, AV_SAMPLE_FMT_NONE } as samples_list.
*/
enum AVSampleFormat sample_fmt;
+/**
+ * Equivalent to { sub_fmt, AV_SUBTITLE_FMT_NONE } as subs_list.
+ */
+enum AVSubtitleType sub_fmt;
} formats;
int priv_size; ///< size of private data to allocate for the filter
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index b8b432e98b..f4987654af 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -311,6 +311,8 @@ static int filter_link_check_formats(void *log,
AVFilterLink *link, AVFilterForm
return ret;
break;
+case AVMEDIA_TYPE_SUBTITLE:
+return 0;
default:
av_assert0(!"reached");
}
@@ -441,6 +443,9 @@ static int query_formats(AVFilterGraph *graph, void
*log_ctx)
if (!link)
continue;
+if (link->type == AVMEDIA_TYPE_SUBTITLE)
+continue;
+
neg = ff_filter_get_negotiation(link);
av_assert0(neg);
for (neg_step = 1; neg_step < neg->nb_mergers; neg_step++) {
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index ba62f73248..5c972bb183 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -19,6 +19,7 @@
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include "libavutil/subfmt.h"
#include "libavutil/avassert.h"
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
@@ -431,6 +432,12 @@ int ff_add_channel_layout(AVFilterChannelLayouts **l,
uint64_t channel_layout)
return 0;
}
+int ff_add_subtitle_type(AVFilterFormats **avff, int64_t fmt)
+{
+ADD_FORMAT(avff, fmt, ff_formats_unref, int, formats, nb_formats);
+return 0;
+}
+
AVFilterFormats *ff_make_formats_list_singleton(int fmt)
{
int fmts[2] = { fmt, -1 };
@@ -450,6 +457,13 @@ AVFilterFormats *ff_all_formats(enum AVMediaType type)
return NULL;
fmt++;
}
+} else if (type == AVMEDIA_TYPE_SUBTITLE) {
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_BITMAP) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_ASS) < 0)
+return NULL;
+if (ff_add_subtitle_type(&ret, AV_SUBTITLE_FMT_TEXT) < 0)
+return NULL;
}
return ret;
@@ -724,6 +738,10 @@ int ff_default_query_formats(AVFilterContext *ctx)
type= AVMEDIA_TYPE_AUDIO
[FFmpeg-devel] [PATCH v23 09/21] avfilter/subtitles: Add subtitles.c for subtitle frame allocation
Analog to avfilter/video.c and avfilter/audio.c
Signed-off-by: softworkz
---
libavfilter/Makefile| 1 +
libavfilter/avfilter.c | 4 +++
libavfilter/internal.h | 1 +
libavfilter/subtitles.c | 63 +
libavfilter/subtitles.h | 44
5 files changed, 113 insertions(+)
create mode 100644 libavfilter/subtitles.c
create mode 100644 libavfilter/subtitles.h
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index c8082c4a2f..664566a18a 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -19,6 +19,7 @@ OBJS = allfilters.o
\
framequeue.o \
graphdump.o \
graphparser.o\
+ subtitles.o \
video.o \
OBJS-$(HAVE_THREADS) += pthread.o
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index 7362bcdab5..df5b8f483c 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -43,6 +43,7 @@
#include "formats.h"
#include "framepool.h"
#include "internal.h"
+#include "subtitles.h"
#include "libavutil/ffversion.h"
const char av_filter_ffversion[] = "FFmpeg version " FFMPEG_VERSION;
@@ -1475,6 +1476,9 @@ int ff_inlink_make_frame_writable(AVFilterLink *link,
AVFrame **rframe)
case AVMEDIA_TYPE_AUDIO:
out = ff_get_audio_buffer(link, frame->nb_samples);
break;
+case AVMEDIA_TYPE_SUBTITLE:
+out = ff_get_subtitles_buffer(link, link->format);
+break;
default:
return AVERROR(EINVAL);
}
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 1099b82b4b..fc09ef574c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -90,6 +90,7 @@ struct AVFilterPad {
union {
AVFrame *(*video)(AVFilterLink *link, int w, int h);
AVFrame *(*audio)(AVFilterLink *link, int nb_samples);
+AVFrame *(*subtitle)(AVFilterLink *link, int format);
} get_buffer;
/**
diff --git a/libavfilter/subtitles.c b/libavfilter/subtitles.c
new file mode 100644
index 00..951bfd612c
--- /dev/null
+++ b/libavfilter/subtitles.c
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+
+#include "subtitles.h"
+#include "avfilter.h"
+#include "internal.h"
+
+
+AVFrame *ff_null_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+return ff_get_subtitles_buffer(link->dst->outputs[0], format);
+}
+
+AVFrame *ff_default_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *frame;
+
+frame = av_frame_alloc();
+if (!frame)
+return NULL;
+
+frame->format = format;
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+
+if (av_frame_get_buffer2(frame, 0) < 0) {
+av_frame_free(&frame);
+return NULL;
+}
+
+return frame;
+}
+
+AVFrame *ff_get_subtitles_buffer(AVFilterLink *link, int format)
+{
+AVFrame *ret = NULL;
+
+if (link->dstpad->get_buffer.subtitle)
+ret = link->dstpad->get_buffer.subtitle(link, format);
+
+if (!ret)
+ret = ff_default_get_subtitles_buffer(link, format);
+
+return ret;
+}
diff --git a/libavfilter/subtitles.h b/libavfilter/subtitles.h
new file mode 100644
index 00..4a9115126e
--- /dev/null
+++ b/libavfilter/subtitles.h
@@ -0,0 +1,44 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have recei
[FFmpeg-devel] [PATCH v23 08/21] fftools/play, probe: Adjust for subtitle changes
Signed-off-by: softworkz
---
fftools/ffplay.c | 102 +-
fftools/ffprobe.c | 48 ++
2 files changed, 78 insertions(+), 72 deletions(-)
diff --git a/fftools/ffplay.c b/fftools/ffplay.c
index e7b20be76b..0af32888da 100644
--- a/fftools/ffplay.c
+++ b/fftools/ffplay.c
@@ -152,7 +152,6 @@ typedef struct Clock {
/* Common struct for handling all types of decoded data and allocated render
buffers. */
typedef struct Frame {
AVFrame *frame;
-AVSubtitle sub;
int serial;
double pts; /* presentation timestamp for the frame */
double duration; /* estimated duration of the frame */
@@ -586,7 +585,7 @@ static int decoder_init(Decoder *d, AVCodecContext *avctx,
PacketQueue *queue, S
return 0;
}
-static int decoder_decode_frame(Decoder *d, AVFrame *frame, AVSubtitle *sub) {
+static int decoder_decode_frame(Decoder *d, AVFrame *frame) {
int ret = AVERROR(EAGAIN);
for (;;) {
@@ -620,6 +619,9 @@ static int decoder_decode_frame(Decoder *d, AVFrame *frame,
AVSubtitle *sub) {
}
}
break;
+case AVMEDIA_TYPE_SUBTITLE:
+ret = avcodec_receive_frame(d->avctx, frame);
+break;
}
if (ret == AVERROR_EOF) {
d->finished = d->pkt_serial;
@@ -652,25 +654,11 @@ static int decoder_decode_frame(Decoder *d, AVFrame
*frame, AVSubtitle *sub) {
av_packet_unref(d->pkt);
} while (1);
-if (d->avctx->codec_type == AVMEDIA_TYPE_SUBTITLE) {
-int got_frame = 0;
-ret = avcodec_decode_subtitle2(d->avctx, sub, &got_frame, d->pkt);
-if (ret < 0) {
-ret = AVERROR(EAGAIN);
-} else {
-if (got_frame && !d->pkt->data) {
-d->packet_pending = 1;
-}
-ret = got_frame ? 0 : (d->pkt->data ? AVERROR(EAGAIN) :
AVERROR_EOF);
-}
-av_packet_unref(d->pkt);
+if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
+av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet both
returned EAGAIN, which is an API violation.\n");
+d->packet_pending = 1;
} else {
-if (avcodec_send_packet(d->avctx, d->pkt) == AVERROR(EAGAIN)) {
-av_log(d->avctx, AV_LOG_ERROR, "Receive_frame and send_packet
both returned EAGAIN, which is an API violation.\n");
-d->packet_pending = 1;
-} else {
-av_packet_unref(d->pkt);
-}
+av_packet_unref(d->pkt);
}
}
}
@@ -683,7 +671,6 @@ static void decoder_destroy(Decoder *d) {
static void frame_queue_unref_item(Frame *vp)
{
av_frame_unref(vp->frame);
-avsubtitle_free(&vp->sub);
}
static int frame_queue_init(FrameQueue *f, PacketQueue *pktq, int max_size,
int keep_last)
@@ -981,7 +968,7 @@ static void video_image_display(VideoState *is)
if (frame_queue_nb_remaining(&is->subpq) > 0) {
sp = frame_queue_peek(&is->subpq);
-if (vp->pts >= sp->pts + ((float) sp->sub.start_display_time /
1000)) {
+if (vp->pts >= sp->pts + ((float) sp->frame->subtitle_start_time /
1000)) {
if (!sp->uploaded) {
uint8_t* pixels[4];
int pitch[4];
@@ -993,25 +980,27 @@ static void video_image_display(VideoState *is)
if (realloc_texture(&is->sub_texture,
SDL_PIXELFORMAT_ARGB, sp->width, sp->height, SDL_BLENDMODE_BLEND, 1) < 0)
return;
-for (i = 0; i < sp->sub.num_rects; i++) {
-AVSubtitleRect *sub_rect = sp->sub.rects[i];
+for (i = 0; i < sp->frame->num_subtitle_areas; i++) {
+AVSubtitleArea *area = sp->frame->subtitle_areas[i];
+SDL_Rect sdl_rect = { .x = area->x, .y = area->y, .w =
area->w, .h = area->h };
-sub_rect->x = av_clip(sub_rect->x, 0, sp->width );
-sub_rect->y = av_clip(sub_rect->y, 0, sp->height);
-sub_rect->w = av_clip(sub_rect->w, 0, sp->width -
sub_rect->x);
-sub_rect->h = av_clip(sub_rect->h, 0, sp->height -
sub_rect->y);
+area->x = av_clip(area->x, 0, sp->width );
+area->y = av_clip(area->y, 0, sp->height);
+area->w = av_clip(area->w, 0, sp->width - area->x);
+area->h = av_clip(area->h, 0, sp->height - area->y);
is->sub_convert_ctx =
sws_getCachedContext(is->sub_convert_ctx,
-sub_rect->w, sub_rect->h, AV_PIX_FMT_PAL8,
-sub_rect->w, sub_rec
[FFmpeg-devel] [PATCH v23 07/21] avcodec/subtitles: Replace deprecated enum values
Signed-off-by: softworkz
---
libavcodec/ass.h | 2 +-
libavcodec/assdec.c| 2 +-
libavcodec/dvbsubdec.c | 2 +-
libavcodec/dvdsubdec.c | 2 +-
libavcodec/dvdsubenc.c | 2 +-
libavcodec/pgssubdec.c | 2 +-
libavcodec/xsubdec.c | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/libavcodec/ass.h b/libavcodec/ass.h
index 9c9cb01c8a..8e885b3ae2 100644
--- a/libavcodec/ass.h
+++ b/libavcodec/ass.h
@@ -82,7 +82,7 @@ static inline int avpriv_ass_add_rect(AVSubtitle *sub, const
char *dialog,
rects[sub->num_rects] = av_mallocz(sizeof(*rects[0]));
if (!rects[sub->num_rects])
return AVERROR(ENOMEM);
-rects[sub->num_rects]->type = SUBTITLE_ASS;
+rects[sub->num_rects]->type = AV_SUBTITLE_FMT_ASS;
ass_str = avpriv_ass_get_dialog(readorder, layer, style, speaker, dialog);
if (!ass_str)
return AVERROR(ENOMEM);
diff --git a/libavcodec/assdec.c b/libavcodec/assdec.c
index 7802a44e71..fd321e7004 100644
--- a/libavcodec/assdec.c
+++ b/libavcodec/assdec.c
@@ -54,7 +54,7 @@ static int ass_decode_frame(AVCodecContext *avctx, void
*data, int *got_sub_ptr,
if (!sub->rects[0])
return AVERROR(ENOMEM);
sub->num_rects = 1;
-sub->rects[0]->type = SUBTITLE_ASS;
+sub->rects[0]->type = AV_SUBTITLE_FMT_ASS;
sub->rects[0]->ass = av_strdup(avpkt->data);
if (!sub->rects[0]->ass)
return AVERROR(ENOMEM);
diff --git a/libavcodec/dvbsubdec.c b/libavcodec/dvbsubdec.c
index 81ccaf4c57..b13244c803 100644
--- a/libavcodec/dvbsubdec.c
+++ b/libavcodec/dvbsubdec.c
@@ -795,7 +795,7 @@ static int save_subtitle_set(AVCodecContext *avctx,
AVSubtitle *sub, int *got_ou
rect->w = region->width;
rect->h = region->height;
rect->nb_colors = (1 << region->depth);
-rect->type = SUBTITLE_BITMAP;
+rect->type = AV_SUBTITLE_FMT_BITMAP;
rect->linesize[0] = region->width;
clut = get_clut(ctx, region->clut);
diff --git a/libavcodec/dvdsubdec.c b/libavcodec/dvdsubdec.c
index 52259f0730..b39b3d1838 100644
--- a/libavcodec/dvdsubdec.c
+++ b/libavcodec/dvdsubdec.c
@@ -406,7 +406,7 @@ static int decode_dvd_subtitles(DVDSubContext *ctx,
AVSubtitle *sub_header,
sub_header->rects[0]->y = y1;
sub_header->rects[0]->w = w;
sub_header->rects[0]->h = h;
-sub_header->rects[0]->type = SUBTITLE_BITMAP;
+sub_header->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub_header->rects[0]->linesize[0] = w;
sub_header->rects[0]->flags = is_menu ?
AV_SUBTITLE_FLAG_FORCED : 0;
}
diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
index 4916410fc8..fbab88a992 100644
--- a/libavcodec/dvdsubenc.c
+++ b/libavcodec/dvdsubenc.c
@@ -273,7 +273,7 @@ static int encode_dvd_subtitles(AVCodecContext* avctx,
AVPacket* avpkt,
return AVERROR(EINVAL);
for (i = 0; i < rects; i++)
-if (frame->subtitle_areas[i]->type != SUBTITLE_BITMAP) {
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_BITMAP) {
av_log(avctx, AV_LOG_ERROR, "Bitmap subtitle required\n");
return AVERROR(EINVAL);
}
diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index 388639a110..4829babb7c 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -539,7 +539,7 @@ static int display_end_segment(AVCodecContext *avctx, void
*data,
return AVERROR(ENOMEM);
}
sub->num_rects++;
-sub->rects[i]->type = SUBTITLE_BITMAP;
+sub->rects[i]->type = AV_SUBTITLE_FMT_BITMAP;
/* Process bitmap */
object = find_object(ctx->presentation.objects[i].id, &ctx->objects);
diff --git a/libavcodec/xsubdec.c b/libavcodec/xsubdec.c
index 979399bae6..5c9e65f737 100644
--- a/libavcodec/xsubdec.c
+++ b/libavcodec/xsubdec.c
@@ -107,7 +107,7 @@ static int decode_frame(AVCodecContext *avctx, void *data,
int *got_sub_ptr,
}
sub->rects[0]->x = x; sub->rects[0]->y = y;
sub->rects[0]->w = w; sub->rects[0]->h = h;
-sub->rects[0]->type = SUBTITLE_BITMAP;
+sub->rects[0]->type = AV_SUBTITLE_FMT_BITMAP;
sub->rects[0]->linesize[0] = w;
sub->rects[0]->data[0] = av_malloc(w * h);
sub->rects[0]->nb_colors = 4;
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v23 03/21] avcodec/subtitles: Introduce new frame-based subtitle decoding API
- Add avcodec_decode_subtitle3 which takes subtitle frames,
serving as compatibility shim to legacy subtitle decoding
- Add additional methods for conversion between old and new API
Signed-off-by: softworkz
---
libavcodec/avcodec.h| 8 +-
libavcodec/codec_desc.c | 11 +++
libavcodec/codec_desc.h | 8 ++
libavcodec/decode.c | 54 ++--
libavcodec/internal.h | 16
libavcodec/utils.c | 182
6 files changed, 270 insertions(+), 9 deletions(-)
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index b05c12e47e..3e734d3003 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -1675,7 +1675,7 @@ typedef struct AVCodecContext {
/**
* Header containing style information for text subtitles.
- * For SUBTITLE_ASS subtitle type, it should contain the whole ASS
+ * For AV_SUBTITLE_FMT_ASS subtitle type, it should contain the whole ASS
* [Script Info] and [V4+ Styles] section, plus the [Events] line and
* the Format line following. It shouldn't include any Dialogue line.
* - encoding: Set/allocated/freed by user (before avcodec_open2())
@@ -2417,7 +2417,10 @@ int avcodec_close(AVCodecContext *avctx);
* Free all allocated data in the given subtitle struct.
*
* @param sub AVSubtitle to free.
+ *
+ * @deprecated Use the regular frame based encode and decode APIs instead.
*/
+attribute_deprecated
void avsubtitle_free(AVSubtitle *sub);
/**
@@ -2510,7 +2513,10 @@ enum AVChromaLocation avcodec_chroma_pos_to_enum(int
xpos, int ypos);
* must be freed with avsubtitle_free if *got_sub_ptr is set.
* @param[in,out] got_sub_ptr Zero if no subtitle could be decompressed,
otherwise, it is nonzero.
* @param[in] avpkt The input AVPacket containing the input buffer.
+ *
+ * @deprecated Use the new decode API (avcodec_send_packet,
avcodec_receive_frame) instead.
*/
+attribute_deprecated
int avcodec_decode_subtitle2(AVCodecContext *avctx, AVSubtitle *sub,
int *got_sub_ptr,
AVPacket *avpkt);
diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c
index 0974ee03de..e48e4532ba 100644
--- a/libavcodec/codec_desc.c
+++ b/libavcodec/codec_desc.c
@@ -3548,3 +3548,14 @@ enum AVMediaType avcodec_get_type(enum AVCodecID
codec_id)
const AVCodecDescriptor *desc = avcodec_descriptor_get(codec_id);
return desc ? desc->type : AVMEDIA_TYPE_UNKNOWN;
}
+
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor)
+{
+if(codec_descriptor->props & AV_CODEC_PROP_BITMAP_SUB)
+return AV_SUBTITLE_FMT_BITMAP;
+
+if(codec_descriptor->props & AV_CODEC_PROP_TEXT_SUB)
+return AV_SUBTITLE_FMT_ASS;
+
+return AV_SUBTITLE_FMT_UNKNOWN;
+}
diff --git a/libavcodec/codec_desc.h b/libavcodec/codec_desc.h
index 126b52df47..ba68d24e0e 100644
--- a/libavcodec/codec_desc.h
+++ b/libavcodec/codec_desc.h
@@ -121,6 +121,14 @@ const AVCodecDescriptor *avcodec_descriptor_next(const
AVCodecDescriptor *prev);
*/
const AVCodecDescriptor *avcodec_descriptor_get_by_name(const char *name);
+/**
+ * Return subtitle format from a codec descriptor
+ *
+ * @param codec_descriptor codec descriptor
+ * @return the subtitle type (e.g. bitmap, text)
+ */
+enum AVSubtitleType avcodec_descriptor_get_subtitle_format(const
AVCodecDescriptor *codec_descriptor);
+
/**
* @}
*/
diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 52bf5dcd33..ac267f0df6 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -576,6 +576,37 @@ static int decode_receive_frame_internal(AVCodecContext
*avctx, AVFrame *frame)
return ret;
}
+static int decode_subtitle2_priv(AVCodecContext *avctx, AVSubtitle *sub,
+ int *got_sub_ptr, AVPacket *avpkt);
+
+static int decode_subtitle_shim(AVCodecContext *avctx, AVFrame *frame,
AVPacket *avpkt)
+{
+int ret, got_sub_ptr = 0;
+AVSubtitle subtitle = { 0 };
+
+if (frame->buf[0])
+return AVERROR(EAGAIN);
+
+av_frame_unref(frame);
+
+ret = decode_subtitle2_priv(avctx, &subtitle, &got_sub_ptr, avpkt);
+
+if (ret >= 0 && got_sub_ptr) {
+frame->type = AVMEDIA_TYPE_SUBTITLE;
+frame->format = subtitle.format;
+ret = av_frame_get_buffer2(frame, 0);
+
+if (ret >= 0)
+ret = ff_frame_put_subtitle(frame, &subtitle);
+
+frame->pkt_dts = avpkt->dts;
+}
+
+avsubtitle_free(&subtitle);
+
+return ret;
+}
+
int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const
AVPacket *avpkt)
{
AVCodecInternal *avci = avctx->internal;
@@ -590,6 +621,9 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext
*avctx, const AVPacke
if (avpkt && !avpkt->size && avpkt->data)
return AVERROR(EINVAL);
+if (avctx->codec_type == AVMEDIA_TYPE_SUBTITLE)
+return decode_subtitle_shim(a
[FFmpeg-devel] [PATCH v23 06/21] avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api
Signed-off-by: softworkz
---
libavcodec/assenc.c| 90 +
libavcodec/avcodec.h | 5 +-
libavcodec/dvbsubenc.c | 96 +--
libavcodec/dvdsubenc.c | 100 +++--
libavcodec/encode.c| 63 -
libavcodec/movtextenc.c| 112 +++--
libavcodec/srtenc.c| 104 ++
libavcodec/tests/avcodec.c | 2 -
libavcodec/ttmlenc.c | 98
libavcodec/webvttenc.c | 82 +++
libavcodec/xsubenc.c | 87 +---
11 files changed, 584 insertions(+), 255 deletions(-)
diff --git a/libavcodec/assenc.c b/libavcodec/assenc.c
index b0e475834b..94601bba68 100644
--- a/libavcodec/assenc.c
+++ b/libavcodec/assenc.c
@@ -22,48 +22,92 @@
#include
#include "avcodec.h"
+#include "encode.h"
#include "libavutil/ass_internal.h"
#include "internal.h"
#include "libavutil/avstring.h"
#include "libavutil/internal.h"
#include "libavutil/mem.h"
+static void check_write_header(AVCodecContext* avctx, const AVFrame* frame)
+{
+if (avctx->extradata_size)
+return;
+
+if (frame->subtitle_header && frame->subtitle_header->size > 0) {
+const char* subtitle_header = (char*)frame->subtitle_header->data;
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(frame->subtitle_header->size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
+if (!avctx->extradata_size) {
+const char* subtitle_header =
avpriv_ass_get_subtitle_header_default(0);
+if (!subtitle_header)
+return;
+
+avctx->extradata_size = strlen(subtitle_header);
+avctx->extradata = av_mallocz(avctx->extradata_size + 1);
+memcpy(avctx->extradata, subtitle_header, avctx->extradata_size);
+avctx->extradata[avctx->extradata_size] = 0;
+av_freep(&subtitle_header);
+}
+}
+
static av_cold int ass_encode_init(AVCodecContext *avctx)
{
-avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
-if (!avctx->extradata)
-return AVERROR(ENOMEM);
-memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
-avctx->extradata_size = avctx->subtitle_header_size;
-avctx->extradata[avctx->extradata_size] = 0;
+if (avctx->subtitle_header_size) {
+avctx->extradata = av_malloc(avctx->subtitle_header_size + 1);
+if (!avctx->extradata)
+return AVERROR(ENOMEM);
+memcpy(avctx->extradata, avctx->subtitle_header,
avctx->subtitle_header_size);
+avctx->extradata_size = avctx->subtitle_header_size;
+avctx->extradata[avctx->extradata_size] = 0;
+}
+
return 0;
}
-static int ass_encode_frame(AVCodecContext *avctx,
-unsigned char *buf, int bufsize,
-const AVSubtitle *sub)
+static int ass_encode_frame(AVCodecContext* avctx, AVPacket* avpkt,
+const AVFrame* frame, int* got_packet)
{
-int i, len, total_len = 0;
+int ret;
+size_t req_len = 0, total_len = 0;
+
+check_write_header(avctx, frame);
-for (i=0; inum_rects; i++) {
-const char *ass = sub->rects[i]->ass;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
-if (sub->rects[i]->type != SUBTITLE_ASS) {
-av_log(avctx, AV_LOG_ERROR, "Only SUBTITLE_ASS type supported.\n");
+if (frame->subtitle_areas[i]->type != AV_SUBTITLE_FMT_ASS) {
+av_log(avctx, AV_LOG_ERROR, "Only AV_SUBTITLE_FMT_ASS type
supported.\n");
return AVERROR(EINVAL);
}
-len = av_strlcpy(buf+total_len, ass, bufsize-total_len);
+if (ass)
+req_len += strlen(ass);
+}
-if (len > bufsize-total_len-1) {
-av_log(avctx, AV_LOG_ERROR, "Buffer too small for ASS event.\n");
-return AVERROR_BUFFER_TOO_SMALL;
-}
+ret = ff_get_encode_buffer(avctx, avpkt, req_len + 1, 0);
+if (ret < 0) {
+av_log(avctx, AV_LOG_ERROR, "Error getting output packet.\n");
+return ret;
+}
-total_len += len;
+for (unsigned i = 0; i < frame->num_subtitle_areas; i++) {
+const char *ass = frame->subtitle_areas[i]->ass;
+
+if (ass) {
+size_t len = av_strlcpy((char *)avpkt->data + total_len, ass,
avpkt->size - total_len);
+total_len += len;
+}
}
-return total_len;
+avpkt->size = total_len;
+*got_packet = total_len > 0;
+
+return 0;
}
#if CONFIG_SSA_ENCODER
@@ -73,7 +117,7 @@ const AVCodec ff_ssa_encoder = {
.type = AVMEDIA_TYPE_SUBTITLE,
.id
[FFmpeg-devel] [PATCH v23 05/21] avcodec, avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing
Also add
- hard_space callback (for upcoming fix)
- extensible callback (for future extension)
Signed-off-by: softworkz
---
libavcodec/Makefile | 56 +++
libavcodec/ass.h | 147 ++
libavcodec/assdec.c | 2 +-
libavcodec/assenc.c | 2 +-
libavcodec/ccaption_dec.c | 19 +--
libavcodec/jacosubdec.c | 2 +-
libavcodec/libaribb24.c | 2 +-
libavcodec/libzvbi-teletextdec.c | 14 +-
libavcodec/microdvddec.c | 7 +-
libavcodec/movtextdec.c | 3 +-
libavcodec/movtextenc.c | 20 +--
libavcodec/mpl2dec.c | 2 +-
libavcodec/realtextdec.c | 2 +-
libavcodec/samidec.c | 2 +-
libavcodec/srtdec.c | 2 +-
libavcodec/srtenc.c | 16 +-
libavcodec/subviewerdec.c | 2 +-
libavcodec/textdec.c | 4 +-
libavcodec/ttmlenc.c | 15 +-
libavcodec/webvttdec.c| 2 +-
libavcodec/webvttenc.c| 16 +-
libavutil/Makefile| 2 +
{libavcodec => libavutil}/ass.c | 87 ---
libavutil/ass_internal.h | 133
{libavcodec => libavutil}/ass_split.c | 30 ++--
.../ass_split_internal.h | 32 ++--
26 files changed, 347 insertions(+), 274 deletions(-)
rename {libavcodec => libavutil}/ass.c (65%)
create mode 100644 libavutil/ass_internal.h
rename {libavcodec => libavutil}/ass_split.c (94%)
rename libavcodec/ass_split.h => libavutil/ass_split_internal.h (86%)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 4122a9b144..b12638da5e 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -209,10 +209,10 @@ OBJS-$(CONFIG_APNG_DECODER)+= png.o pngdec.o
pngdsp.o
OBJS-$(CONFIG_APNG_ENCODER)+= png.o pngenc.o
OBJS-$(CONFIG_ARBC_DECODER)+= arbc.o
OBJS-$(CONFIG_ARGO_DECODER)+= argo.o
-OBJS-$(CONFIG_SSA_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_SSA_ENCODER) += assenc.o ass.o
-OBJS-$(CONFIG_ASS_DECODER) += assdec.o ass.o
-OBJS-$(CONFIG_ASS_ENCODER) += assenc.o ass.o
+OBJS-$(CONFIG_SSA_DECODER) += assdec.o
+OBJS-$(CONFIG_SSA_ENCODER) += assenc.o
+OBJS-$(CONFIG_ASS_DECODER) += assdec.o
+OBJS-$(CONFIG_ASS_ENCODER) += assenc.o
OBJS-$(CONFIG_ASV1_DECODER)+= asvdec.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV1_ENCODER)+= asvenc.o asv.o mpeg12data.o
OBJS-$(CONFIG_ASV2_DECODER)+= asvdec.o asv.o mpeg12data.o
@@ -253,7 +253,7 @@ OBJS-$(CONFIG_BRENDER_PIX_DECODER) += brenderpix.o
OBJS-$(CONFIG_C93_DECODER) += c93.o
OBJS-$(CONFIG_CAVS_DECODER)+= cavs.o cavsdec.o cavsdsp.o \
cavsdata.o
-OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o ass.o
+OBJS-$(CONFIG_CCAPTION_DECODER)+= ccaption_dec.o
OBJS-$(CONFIG_CDGRAPHICS_DECODER) += cdgraphics.o
OBJS-$(CONFIG_CDTOONS_DECODER) += cdtoons.o
OBJS-$(CONFIG_CDXL_DECODER)+= cdxl.o
@@ -425,7 +425,7 @@ OBJS-$(CONFIG_INTERPLAY_ACM_DECODER) += interplayacm.o
OBJS-$(CONFIG_INTERPLAY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_INTERPLAY_VIDEO_DECODER) += interplayvideo.o
OBJS-$(CONFIG_IPU_DECODER) += mpeg12dec.o mpeg12.o mpeg12data.o
-OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o ass.o
+OBJS-$(CONFIG_JACOSUB_DECODER) += jacosubdec.o
OBJS-$(CONFIG_JPEG2000_ENCODER)+= j2kenc.o mqcenc.o mqc.o jpeg2000.o \
jpeg2000dwt.o
OBJS-$(CONFIG_JPEG2000_DECODER)+= jpeg2000dec.o jpeg2000.o
jpeg2000dsp.o \
@@ -447,7 +447,7 @@ OBJS-$(CONFIG_MAGICYUV_ENCODER)+= magicyuvenc.o
OBJS-$(CONFIG_MDEC_DECODER)+= mdec.o mpeg12.o mpeg12data.o
OBJS-$(CONFIG_METASOUND_DECODER) += metasound.o metasound_data.o \
twinvq.o
-OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o ass.o
+OBJS-$(CONFIG_MICRODVD_DECODER)+= microdvddec.o
OBJS-$(CONFIG_MIMIC_DECODER) += mimic.o
OBJS-$(CONFIG_MJPEG_DECODER) += mjpegdec.o mjpegdec_common.o
OBJS-$(CONFIG_MJPEG_QSV_DECODER) += qsvdec.o
@@ -462,8 +462,8 @@ OBJS-$(CONFIG_MLP_ENCODER) += mlpenc.o mlp.o
OBJS-$(CONFIG_MMVIDEO_DECODER) += mmvideo.o
OBJS-$(CONFIG_MOBICLIP_DECODER)+= mobiclip.o
OBJS-$(CONFIG_MOTIONPIXELS_DECODER)+= motionpixels.o
-OBJS-$(CONFIG_MOVTEXT_DECODER) += movtextdec.o ass.o
-OBJS-$(CONFIG_MOVTEXT_ENCODER) += mov
[FFmpeg-devel] [PATCH v23 04/21] avfilter/subtitles: Update vf_subtitles to use new decoding api
Signed-off-by: softworkz
---
libavfilter/vf_subtitles.c | 54 +-
1 file changed, 42 insertions(+), 12 deletions(-)
diff --git a/libavfilter/vf_subtitles.c b/libavfilter/vf_subtitles.c
index 3fc4eeb63d..66b2082894 100644
--- a/libavfilter/vf_subtitles.c
+++ b/libavfilter/vf_subtitles.c
@@ -35,14 +35,12 @@
# include "libavformat/avformat.h"
#endif
#include "libavutil/avstring.h"
-#include "libavutil/imgutils.h"
#include "libavutil/opt.h"
#include "libavutil/parseutils.h"
#include "drawutils.h"
#include "avfilter.h"
#include "internal.h"
#include "formats.h"
-#include "video.h"
typedef struct AssContext {
const AVClass *class;
@@ -292,6 +290,29 @@ static int attachment_is_font(AVStream * st)
return 0;
}
+static int decode(AVCodecContext *avctx, AVFrame *frame, int *got_frame,
AVPacket *pkt)
+{
+int ret;
+
+*got_frame = 0;
+
+if (pkt) {
+ret = avcodec_send_packet(avctx, pkt);
+// In particular, we don't expect AVERROR(EAGAIN), because we read all
+// decoded frames with avcodec_receive_frame() until done.
+if (ret < 0 && ret != AVERROR_EOF)
+return ret;
+}
+
+ret = avcodec_receive_frame(avctx, frame);
+if (ret < 0 && ret != AVERROR(EAGAIN))
+return ret;
+if (ret >= 0)
+*got_frame = 1;
+
+return 0;
+}
+
AVFILTER_DEFINE_CLASS(subtitles);
static av_cold int init_subtitles(AVFilterContext *ctx)
@@ -306,6 +327,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
AVStream *st;
AVPacket pkt;
AssContext *ass = ctx->priv;
+enum AVSubtitleType subtitle_format;
/* Init libass */
ret = init(ctx);
@@ -386,13 +408,17 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
ret = AVERROR_DECODER_NOT_FOUND;
goto end;
}
+
dec_desc = avcodec_descriptor_get(st->codecpar->codec_id);
-if (dec_desc && !(dec_desc->props & AV_CODEC_PROP_TEXT_SUB)) {
+subtitle_format = avcodec_descriptor_get_subtitle_format(dec_desc);
+
+if (subtitle_format != AV_SUBTITLE_FMT_ASS) {
av_log(ctx, AV_LOG_ERROR,
- "Only text based subtitles are currently supported\n");
-ret = AVERROR_PATCHWELCOME;
+ "Only text based subtitles are supported by this filter\n");
+ret = AVERROR_INVALIDDATA;
goto end;
}
+
if (ass->charenc)
av_dict_set(&codec_opts, "sub_charenc", ass->charenc, 0);
@@ -448,18 +474,22 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
dec_ctx->subtitle_header_size);
while (av_read_frame(fmt, &pkt) >= 0) {
int i, got_subtitle;
-AVSubtitle sub = {0};
+AVFrame *sub = av_frame_alloc();
+if (!sub) {
+ret = AVERROR(ENOMEM);
+goto end;
+}
if (pkt.stream_index == sid) {
-ret = avcodec_decode_subtitle2(dec_ctx, &sub, &got_subtitle, &pkt);
+ret = decode(dec_ctx, sub, &got_subtitle, &pkt);
if (ret < 0) {
av_log(ctx, AV_LOG_WARNING, "Error decoding: %s (ignored)\n",
av_err2str(ret));
} else if (got_subtitle) {
-const int64_t start_time = av_rescale_q(sub.pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
-const int64_t duration = sub.end_display_time;
-for (i = 0; i < sub.num_rects; i++) {
-char *ass_line = sub.rects[i]->ass;
+const int64_t start_time = av_rescale_q(sub->subtitle_pts,
AV_TIME_BASE_Q, av_make_q(1, 1000));
+const int64_t duration = sub->subtitle_end_time;
+for (i = 0; i < sub->num_subtitle_areas; i++) {
+char *ass_line = sub->subtitle_areas[i]->ass;
if (!ass_line)
break;
ass_process_chunk(ass->track, ass_line, strlen(ass_line),
@@ -468,7 +498,7 @@ static av_cold int init_subtitles(AVFilterContext *ctx)
}
}
av_packet_unref(&pkt);
-avsubtitle_free(&sub);
+av_frame_free(&sub);
}
end:
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v23 02/21] avutil/frame: Prepare AVFrame for subtitle handling
Root commit for adding subtitle filtering capabilities.
In detail:
- Add type (AVMediaType) field to AVFrame
Replaces previous way of distinction which was based on checking
width and height to determine whether a frame is audio or video
- Add subtitle fields to AVFrame
- Add new struct AVSubtitleArea, similar to AVSubtitleRect, but different
allocation logic. Cannot and must not be used interchangeably, hence
the new struct
Signed-off-by: softworkz
---
libavutil/Makefile | 1 +
libavutil/frame.c | 211 -
libavutil/frame.h | 78 -
libavutil/subfmt.c | 45 ++
libavutil/subfmt.h | 47 ++
5 files changed, 358 insertions(+), 24 deletions(-)
create mode 100644 libavutil/subfmt.c
diff --git a/libavutil/Makefile b/libavutil/Makefile
index c7843db1e4..7e79936876 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -160,6 +160,7 @@ OBJS = adler32.o
\
slicethread.o\
spherical.o \
stereo3d.o \
+ subfmt.o \
threadmessage.o \
time.o \
timecode.o \
diff --git a/libavutil/frame.c b/libavutil/frame.c
index 0912ad9131..62d29ae7d6 100644
--- a/libavutil/frame.c
+++ b/libavutil/frame.c
@@ -26,6 +26,7 @@
#include "imgutils.h"
#include "mem.h"
#include "samplefmt.h"
+#include "subfmt.h"
#include "hwcontext.h"
#define CHECK_CHANNELS_CONSISTENCY(frame) \
@@ -50,6 +51,9 @@ const char *av_get_colorspace_name(enum AVColorSpace val)
return name[val];
}
#endif
+
+static int frame_copy_subtitles(AVFrame *dst, const AVFrame *src, int
copy_data);
+
static void get_frame_defaults(AVFrame *frame)
{
memset(frame, 0, sizeof(*frame));
@@ -70,7 +74,12 @@ static void get_frame_defaults(AVFrame *frame)
frame->colorspace = AVCOL_SPC_UNSPECIFIED;
frame->color_range = AVCOL_RANGE_UNSPECIFIED;
frame->chroma_location = AVCHROMA_LOC_UNSPECIFIED;
-frame->flags = 0;
+frame->subtitle_start_time = 0;
+frame->subtitle_end_time = 0;
+frame->num_subtitle_areas = 0;
+frame->subtitle_areas = NULL;
+frame->subtitle_pts= 0;
+frame->subtitle_header = NULL;
}
static void free_side_data(AVFrameSideData **ptr_sd)
@@ -240,23 +249,55 @@ static int get_audio_buffer(AVFrame *frame, int align)
}
+static int get_subtitle_buffer(AVFrame *frame)
+{
+// Buffers in AVFrame->buf[] are not used in case of subtitle frames.
+// To accomodate with existing code, checking ->buf[0] to determine
+// whether a frame is ref-counted or has data, we're adding a 1-byte
+// buffer here, which marks the subtitle frame to contain data.
+frame->buf[0] = av_buffer_alloc(1);
+if (!frame->buf[0]) {
+av_frame_unref(frame);
+return AVERROR(ENOMEM);
+}
+
+frame->extended_data = frame->data;
+
+return 0;
+}
+
int av_frame_get_buffer(AVFrame *frame, int align)
+{
+if (frame->width > 0 && frame->height > 0)
+frame->type = AVMEDIA_TYPE_VIDEO;
+else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+frame->type = AVMEDIA_TYPE_AUDIO;
+
+return av_frame_get_buffer2(frame, align);
+}
+
+int av_frame_get_buffer2(AVFrame *frame, int align)
{
if (frame->format < 0)
return AVERROR(EINVAL);
-if (frame->width > 0 && frame->height > 0)
+switch(frame->type) {
+case AVMEDIA_TYPE_VIDEO:
return get_video_buffer(frame, align);
-else if (frame->nb_samples > 0 && (frame->channel_layout ||
frame->channels > 0))
+case AVMEDIA_TYPE_AUDIO:
return get_audio_buffer(frame, align);
-
-return AVERROR(EINVAL);
+case AVMEDIA_TYPE_SUBTITLE:
+return get_subtitle_buffer(frame);
+default:
+return AVERROR(EINVAL);
+}
}
static int frame_copy_props(AVFrame *dst, const AVFrame *src, int force_copy)
{
int ret, i;
+dst->type = src->type;
dst->key_frame = src->key_frame;
dst->pict_type = src->pict_type;
dst->sample_aspect_ratio= src->sample_aspect_ratio;
@@ -288,6 +329,12 @@ static int frame_copy_props(AVFrame *dst, const AVFrame
*src, int force_copy)
dst->colorspace = src->colorspace;
dst->color_range= src->color_range;
dst->chroma_location= src->chroma_location;
+dst->subtitle_start_time= src->subtitle_start_time;
+dst->subtitle_end_time = src->subtitle_end_time;
+dst->subtitle_pts = s
[FFmpeg-devel] [PATCH v23 01/21] avcodec, avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values
Signed-off-by: softworkz
---
libavcodec/avcodec.h | 19 +
libavutil/Makefile | 1 +
libavutil/subfmt.h | 68
libavutil/version.h | 1 +
4 files changed, 71 insertions(+), 18 deletions(-)
create mode 100644 libavutil/subfmt.h
diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 7ee8bc2b7c..b05c12e47e 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -35,6 +35,7 @@
#include "libavutil/frame.h"
#include "libavutil/log.h"
#include "libavutil/pixfmt.h"
+#include "libavutil/subfmt.h"
#include "libavutil/rational.h"
#include "codec.h"
@@ -2238,24 +2239,6 @@ typedef struct AVHWAccel {
* @}
*/
-enum AVSubtitleType {
-SUBTITLE_NONE,
-
-SUBTITLE_BITMAP,///< A bitmap, pict will be set
-
-/**
- * Plain text, the text field must be set by the decoder and is
- * authoritative. ass and pict fields may contain approximations.
- */
-SUBTITLE_TEXT,
-
-/**
- * Formatted text, the ass field must be set by the decoder and is
- * authoritative. pict and text fields may contain approximations.
- */
-SUBTITLE_ASS,
-};
-
#define AV_SUBTITLE_FLAG_FORCED 0x0001
typedef struct AVSubtitleRect {
diff --git a/libavutil/Makefile b/libavutil/Makefile
index 529046dbc8..c7843db1e4 100644
--- a/libavutil/Makefile
+++ b/libavutil/Makefile
@@ -74,6 +74,7 @@ HEADERS = adler32.h
\
sha512.h \
spherical.h \
stereo3d.h\
+ subfmt.h \
threadmessage.h \
time.h\
timecode.h\
diff --git a/libavutil/subfmt.h b/libavutil/subfmt.h
new file mode 100644
index 00..791b45519f
--- /dev/null
+++ b/libavutil/subfmt.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright (c) 2021 softworkz
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef AVUTIL_SUBFMT_H
+#define AVUTIL_SUBFMT_H
+
+#include "version.h"
+
+enum AVSubtitleType {
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_NONE = -1,
+
+/**
+ * Subtitle format unknown.
+ */
+AV_SUBTITLE_FMT_UNKNOWN = 0,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_NONE = 0, ///< Deprecated, use AV_SUBTITLE_FMT_NONE
instead.
+#endif
+
+/**
+ * Bitmap area in AVSubtitleRect.data, pixfmt AV_PIX_FMT_PAL8.
+ */
+AV_SUBTITLE_FMT_BITMAP = 1,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_BITMAP = 1,///< Deprecated, use AV_SUBTITLE_FMT_BITMAP
instead.
+#endif
+
+/**
+ * Plain text in AVSubtitleRect.text.
+ */
+AV_SUBTITLE_FMT_TEXT = 2,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_TEXT = 2, ///< Deprecated, use AV_SUBTITLE_FMT_TEXT
instead.
+#endif
+
+/**
+ * Text Formatted as per ASS specification, contained AVSubtitleRect.ass.
+ */
+AV_SUBTITLE_FMT_ASS = 3,
+#if FF_API_OLD_SUBTITLES
+SUBTITLE_ASS = 3, ///< Deprecated, use AV_SUBTITLE_FMT_ASS
instead.
+#endif
+
+AV_SUBTITLE_FMT_NB, ///< number of subtitle formats, DO NOT USE
THIS if you want to link with shared libav* because the number of formats might
differ between versions.
+};
+
+#endif /* AVUTIL_SUBFMT_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 017fc277a6..c6cd0b0d79 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -109,6 +109,7 @@
#define FF_API_DECLARE_ALIGNED (LIBAVUTIL_VERSION_MAJOR < 58)
#define FF_API_COLORSPACE_NAME (LIBAVUTIL_VERSION_MAJOR < 58)
#define FF_API_AV_MALLOCZ_ARRAY (LIBAVUTIL_VERSION_MAJOR < 58)
+#define FF_API_OLD_SUBTITLES(LIBAVUTIL_VERSION_MAJOR < 58)
/**
* @}
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-
[FFmpeg-devel] [PATCH v23 00/21] Subtitle Filtering
New in V22 - subfmt.h: Fix includes, use #if rather than #ifdef - subfmt.c: Remove unused struct - move addition of ass hard_space callback to commit 5 - also add ass extensibility callback - remove bugfix commits regarding ass hard-space due to being incompatible with GIT e-mail workflow New in V22 - Two new commits, implementing parsing of hard-space ASS tags (\h} The ccaption_dec decoder emits a lot of \h tags via splitcc which the encoders (all using ass_split) didn't handle properly - Combined APIchanges and version bumps into a single commit at the end - Moved new fields to end of AVFrame - Changed version bumps as suggested => no more major bumps now! New in V21 - Rebased. Patchset was broken again. - Don't declare AVFrame.subtitle_pts as deprecated and clarify doc text - Remove AVFilter.subs_list and AVFilter.sub_fmt from APIchanges - Change include in formats.c New in V20 - Rebased. V19 didn't apply cleanly anymore New in V19 - Document API changes in all relevant commits - Move enum AVSubtitleType: Improved commit message - Put deprecated enum values under removal guards - Document AV_SUBTITLE_FMT_NB - Fixed all trailing whitespace - splitcc: Add scatter_realtime_output option - splitcc: Emit empty subtitle frames instead of repeating - New commit: Replace deprecated enum values New in V18 - rebased to latest head (to fix merge conflict in allfilters.c) - textmod: Removed two trailing blanks from sf_textmod.c - xsubdec: fix xsubdec regression found by Michael; packet size estimation as flawed - splitcc: Fix uninitialzed locals - splitcc: Do not propagate hwcontext on secondary output - testsub2video: fix output timeing, re-add mutex locking and implement render_latest_only parameter New in V17 - Applied almost all suggestions that were made (thanks everybody!) - Split the initial commit, no longer moving the legacy structs (AVSubtitle..) - Use the regular encoding API for subtitles - Updated subtitle encoders to handle packet allocation - Removed compatibility layer for accessing legacy encoders - Updated the compatibility implementation for the legacy subtitle encode api - Reordered commits and merged ffmpeg.c changes into a single commit New in V16 - Added missing reference to subfmt.h iun libzvbi-teletextdec.c - Fixed Fate error in patch 15/16 - Removed all trsiling whitespace in tilers.texi New in V15 - Rebased to upstream changes - avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide compatibility shim for legacy api - fftools/ffmpeg: Use new frame-based subtitle encoding API - AVSubtitleArea: copy flags field, make params const - graphicsubs2text: Don't emit duplicate frames - graphicsubs2text: Combined OCR output into a single AVSubtitleArea (I have a prototype for detecting text colors and positions, but it's not ready at this point) - splitcc: cleanup local subtitle_header ref - stripstyles: add parameter for ass layer selection - avcodec/subtitles: deferred loading of ass header for text subtitle encoders - verified all example command lines in the docs are working, added somre more Kind regards, softworkz softworkz (21): avcodec,avutil: Move enum AVSubtitleType to avutil, add new and deprecate old values avutil/frame: Prepare AVFrame for subtitle handling avcodec/subtitles: Introduce new frame-based subtitle decoding API avfilter/subtitles: Update vf_subtitles to use new decoding api avcodec,avutil: Move ass helper functions to avutil as avpriv_ and extend ass dialog parsing avcodec/subtitles: Migrate subtitle encoders to frame-based API and provide a compatibility shim for the legacy api avcodec/subtitles: Replace deprecated enum values fftools/play,probe: Adjust for subtitle changes avfilter/subtitles: Add subtitles.c for subtitle frame allocation avfilter/avfilter: Handle subtitle frames avfilter/sbuffer: Add sbuffersrc and sbuffersink filters avfilter/overlaygraphicsubs: Add overlaygraphicsubs and graphicsub2video filters fftools/ffmpeg: Replace sub2video with subtitle frame filtering and use new frame-based subtitle encoding API avfilter/avfilter: Fix hardcoded input index avfilter/overlaytextsubs: Add overlaytextsubs and textsubs2video filters avfilter/textmod: Add textmod, censor and show_speaker filters avfilter/stripstyles: Add stripstyles filter avfilter/splitcc: Add splitcc filter for closed caption handling avfilter/graphicsub2text: Add new graphicsub2text filter (OCR) avfilter/subscale: Add filter for scaling and/or re-arranging graphical subtitles doc/APIchanges: update for subtitle filtering changes configure | 7 +- doc/APIchanges| 23 + doc/filters.texi | 756 +++ fftools/ffmpeg.c | 584 ++-- fftools/ffmpeg.h | 15 +- fftools/ffmpeg_filter.c
[FFmpeg-devel] [PATCH 1/1] Test ref file change as attachment
Signed-off-by: softworkz
---
tests/ref/fate/sub-textenc | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/tests/ref/fate/sub-textenc b/tests/ref/fate/sub-textenc
index 3ea56b38f0..2310227c3e 100644
--- a/tests/ref/fate/sub-textenc
+++ b/tests/ref/fate/sub-textenc
@@ -160,18 +160,18 @@ but show this: {normal text}
\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard spaces are not trimmed.
-The\hline\hwill\hnever\hbreak\hautomatically\hright\hbefore\hor\hafter\ha\hhard\hspace.\h:-D
+Thexlinexwillxneverxbreakxautomaticallyxrightxbeforexorxafterxaxhardxspace.x:-D
31
00:00:54,501 --> 00:00:56,500
-\h\h\h\h\hA (05 hard spaces followed by a letter)
+xA (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
32
00:00:56,501 --> 00:00:58,500
-\h\h\h\h\hA (05 hard spaces followed by a letter)
+xA (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
Show this: \TEST and this: \-)
@@ -179,10 +179,10 @@ Show this: \TEST and this: \-)
33
00:00:58,501 --> 00:01:00,500
-A letter followed by 05 hard spaces: A\h\h\h\h\h
+A letter followed by 05 hard spaces: Ax
A letter followed by normal spaces: A
A letter followed by no hard spaces: A
-05 hard spaces between letters: A\h\h\h\h\hA
+05 hard spaces between letters: AxA
5 normal spaces between letters: A A
^--Forced line break___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] FATE-Suite Data Test Data
Soft Works: > > >> -Original Message- >> From: ffmpeg-devel On Behalf Of Andreas >> Rheinhardt >> Sent: Friday, December 10, 2021 8:21 PM >> To: ffmpeg-devel@ffmpeg.org >> Subject: Re: [FFmpeg-devel] FATE-Suite Data Test Data >> >> Soft Works: >>> >>> -Original Message- From: ffmpeg-devel On Behalf Of Andreas Rheinhardt Sent: Friday, December 10, 2021 8:00 PM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] FATE-Suite Data Test Data Soft Works: > Hi, > > can we add the attached file (SubRip_capability_tester2.srt) to the fate- suite data? > > Compared to SubRip_capability_tester.srt, it has one line removed that would > cause (legitimate) trailing whitespace in some ref data files. > > It hasn’t been an issue so far due to a bug in the encoders which didn’t properly > convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” >> text. > After fixing this, encoders to formats which don’t have a concept of >> “hard spaces”, > will output \h as regular spaces, and in the case of that single source line > that I have removed in the source file, this would lead to trailing >> spaces in the > ref files. > > As I don’t know how, when or whether at all this can be fixed in a way >> that it > could work with patches transported via e-mail, this single line change seems > to be the best quick solution to the problem. > > Thanks, > softworkz > I don't think that this will help. Trailing whitespace should actually only lead to a warning when applying (depending upon core.whitespace). Your patch is broken: There is a line "\ N is a forced line break" in the current ref file which ends with \r\n. This line is not marked as changed in your patchfile, but it is part of the surrounding context of a diff; and said line of context uses only \n and is therefore not recognized. And when I create a diff where the patch file has the proper context, it still fails; I need to add the "--keep-cr" option to git am to make it work. But then it works, whereas this option didn't help if the patch file only has \n instead of \r\n. - Andreas >>> >>> Thanks for looking into this. >>> >>> I'm simply going after the patchwork error from my recent submnission: >>> >>> https://patchwork.ffmpeg.org/check/47132/ >>> -- >>> .git/rebase-apply/patch:131: trailing whitespace. >>> A letter followed by 05 hard spaces: A >>> .git/rebase-apply/patch:192: trailing whitespace. >>> A letter followed by 05 hard spaces: A >>> error: patch failed: tests/ref/fate/sub-textenc:160 >>> error: tests/ref/fate/sub-textenc: patch does not apply >>> error: Did you hand edit your patch? >>> It does not apply to blobs recorded in its index. >>> -- >>> >>> It fails exactly at the line that I have removed in >>> SubRip_capability_tester2.srt >> >> No, it does not. Look at the error: "error: patch failed: >> tests/ref/fate/sub-textenc:160" >> Line 160 is "\ N is a forced line break", the first line of context with >> \r\n. It has nothing to do with the line you removed. The >> trailing-whitespace stuff just leads to warnings, not errors. > > OK, seems you are right. I understood the 160 as indication > of the hunk start line which failed to apply rather than the line of failure. > > To summarize: > - The file in the repo has mixed endings > - My patch contains all CRLF > - The MBOX has all LF > > When we look at the previous patch submitted for the file: > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20170729192751.26379-...@pkh.me/ > > It also as only LF (the MBOX), but the file I the repo, > with the sequences that were changed by the patch have > mixed line endings. > > So, that's kind of a miracle then, how that patch got > applied to have mixed endings in the repo? > That's easy: Just don't let it be touched as email. git push/pull/fetch/clone would have no issue at all. You also had no problem committing the updated file locally. (git am uses git mailsplit to split mbox files into smaller files (one per email, I think, even if the mbox file contained only one email); during this process, \r\n is just treated as newline and replaced by \n if it is run without --keep-cr. And therefore the patch does no longer apply later when the file to be changed really uses \r\n. So the fix for this issue would be for patchwork to use said option (which I think doesn't have any downside (patches using \r\n somewhere except a +-line could no longer be applied to files using \n, but that is not a downside as it indicates wrong patches)); furthermore, the eventual committer would also have to use said option if he receives the mail by email.) - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel
Re: [FFmpeg-devel] FATE-Suite Data Test Data
> -Original Message- > From: ffmpeg-devel On Behalf Of Andreas > Rheinhardt > Sent: Friday, December 10, 2021 8:21 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] FATE-Suite Data Test Data > > Soft Works: > > > > > >> -Original Message- > >> From: ffmpeg-devel On Behalf Of Andreas > >> Rheinhardt > >> Sent: Friday, December 10, 2021 8:00 PM > >> To: ffmpeg-devel@ffmpeg.org > >> Subject: Re: [FFmpeg-devel] FATE-Suite Data Test Data > >> > >> Soft Works: > >>> Hi, > >>> > >>> can we add the attached file (SubRip_capability_tester2.srt) to the fate- > >> suite data? > >>> > >>> Compared to SubRip_capability_tester.srt, it has one line removed that > >> would > >>> cause (legitimate) trailing whitespace in some ref data files. > >>> > >>> It hasn’t been an issue so far due to a bug in the encoders which didn’t > >> properly > >>> convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” > text. > >>> After fixing this, encoders to formats which don’t have a concept of > “hard > >> spaces”, > >>> will output \h as regular spaces, and in the case of that single source > >> line > >>> that I have removed in the source file, this would lead to trailing > spaces > >> in the > >>> ref files. > >>> > >>> As I don’t know how, when or whether at all this can be fixed in a way > that > >> it > >>> could work with patches transported via e-mail, this single line change > >> seems > >>> to be the best quick solution to the problem. > >>> > >>> Thanks, > >>> softworkz > >>> > >> > >> I don't think that this will help. Trailing whitespace should actually > >> only lead to a warning when applying (depending upon core.whitespace). > >> > >> Your patch is broken: There is a line "\ N is a forced line break" in > >> the current ref file which ends with \r\n. This line is not marked as > >> changed in your patchfile, but it is part of the surrounding context of > >> a diff; and said line of context uses only \n and is therefore not > >> recognized. > >> > >> And when I create a diff where the patch file has the proper context, it > >> still fails; I need to add the "--keep-cr" option to git am to make it > >> work. But then it works, whereas this option didn't help if the patch > >> file only has \n instead of \r\n. > >> > >> - Andreas > > > > Thanks for looking into this. > > > > I'm simply going after the patchwork error from my recent submnission: > > > > https://patchwork.ffmpeg.org/check/47132/ > > -- > > .git/rebase-apply/patch:131: trailing whitespace. > > A letter followed by 05 hard spaces: A > > .git/rebase-apply/patch:192: trailing whitespace. > > A letter followed by 05 hard spaces: A > > error: patch failed: tests/ref/fate/sub-textenc:160 > > error: tests/ref/fate/sub-textenc: patch does not apply > > error: Did you hand edit your patch? > > It does not apply to blobs recorded in its index. > > -- > > > > It fails exactly at the line that I have removed in > > SubRip_capability_tester2.srt > > No, it does not. Look at the error: "error: patch failed: > tests/ref/fate/sub-textenc:160" > Line 160 is "\ N is a forced line break", the first line of context with > \r\n. It has nothing to do with the line you removed. The > trailing-whitespace stuff just leads to warnings, not errors. OK, seems you are right. I understood the 160 as indication of the hunk start line which failed to apply rather than the line of failure. To summarize: - The file in the repo has mixed endings - My patch contains all CRLF - The MBOX has all LF When we look at the previous patch submitted for the file: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20170729192751.26379-...@pkh.me/ It also as only LF (the MBOX), but the file I the repo, with the sequences that were changed by the patch have mixed line endings. So, that's kind of a miracle then, how that patch got applied to have mixed endings in the repo? softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_transpose_vulkan: simplify config_props_output function
10 Dec 2021, 20:05 by jianhua...@intel.com:
> It's no need to assign outlink here, which has been done in
> ff_vk_filter_config_output already.
>
> Signed-off-by: Wu Jianhua
> ---
> libavfilter/vf_transpose_vulkan.c | 11 +--
> 1 file changed, 1 insertion(+), 10 deletions(-)
>
> diff --git a/libavfilter/vf_transpose_vulkan.c
> b/libavfilter/vf_transpose_vulkan.c
> index eceb9b9011..ce83cf0fd7 100644
> --- a/libavfilter/vf_transpose_vulkan.c
> +++ b/libavfilter/vf_transpose_vulkan.c
> @@ -262,7 +262,6 @@ static av_cold void
> transpose_vulkan_uninit(AVFilterContext *avctx)
>
> static int config_props_output(AVFilterLink *outlink)
> {
> -int err = 0;
> AVFilterContext *avctx = outlink->src;
> TransposeVulkanContext *s = avctx->priv;
> FFVulkanContext *vkctx = &s->vkctx;
> @@ -271,21 +270,13 @@ static int config_props_output(AVFilterLink *outlink)
> vkctx->output_width = inlink->h;
> vkctx->output_height = inlink->w;
>
> -RET(ff_vk_filter_config_output(outlink));
> -
> -outlink->w = inlink->h;
> -outlink->h = inlink->w;
> -
> if (inlink->sample_aspect_ratio.num)
> outlink->sample_aspect_ratio = av_div_q((AVRational) { 1, 1 },
> inlink->sample_aspect_ratio);
> else
> outlink->sample_aspect_ratio = inlink->sample_aspect_ratio;
>
> -err = 0;
> -
> -fail:
> -return err;
> +return ff_vk_filter_config_output(outlink);
> }
>
> #define OFFSET(x) offsetof(TransposeVulkanContext, x)
>
Tested, squashed the first 3 patches into one, and pushed this along with that
one.
Thanks
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/1] Test ref file change
Signed-off-by: softworkz
---
tests/ref/fate/sub-textenc | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/tests/ref/fate/sub-textenc b/tests/ref/fate/sub-textenc
index 3ea56b38f0..2310227c3e 100644
--- a/tests/ref/fate/sub-textenc
+++ b/tests/ref/fate/sub-textenc
@@ -160,18 +160,18 @@ but show this: {normal text}
\ N is a forced line break
\ h is a hard space
Normal spaces at the start and at the end of the line are trimmed while hard
spaces are not trimmed.
-The\hline\hwill\hnever\hbreak\hautomatically\hright\hbefore\hor\hafter\ha\hhard\hspace.\h:-D
+Thexlinexwillxneverxbreakxautomaticallyxrightxbeforexorxafterxaxhardxspace.x:-D
31
00:00:54,501 --> 00:00:56,500
-\h\h\h\h\hA (05 hard spaces followed by a letter)
+xA (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
32
00:00:56,501 --> 00:00:58,500
-\h\h\h\h\hA (05 hard spaces followed by a letter)
+xA (05 hard spaces followed by a letter)
A (Normal spaces followed by a letter)
A (No hard spaces followed by a letter)
Show this: \TEST and this: \-)
@@ -179,10 +179,10 @@ Show this: \TEST and this: \-)
33
00:00:58,501 --> 00:01:00,500
-A letter followed by 05 hard spaces: A\h\h\h\h\h
+A letter followed by 05 hard spaces: Ax
A letter followed by normal spaces: A
A letter followed by no hard spaces: A
-05 hard spaces between letters: A\h\h\h\h\hA
+05 hard spaces between letters: AxA
5 normal spaces between letters: A A
^--Forced line break
--
2.30.2.windows.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] FATE-Suite Data Test Data
Soft Works: > > >> -Original Message- >> From: ffmpeg-devel On Behalf Of Andreas >> Rheinhardt >> Sent: Friday, December 10, 2021 8:00 PM >> To: ffmpeg-devel@ffmpeg.org >> Subject: Re: [FFmpeg-devel] FATE-Suite Data Test Data >> >> Soft Works: >>> Hi, >>> >>> can we add the attached file (SubRip_capability_tester2.srt) to the fate- >> suite data? >>> >>> Compared to SubRip_capability_tester.srt, it has one line removed that >> would >>> cause (legitimate) trailing whitespace in some ref data files. >>> >>> It hasn’t been an issue so far due to a bug in the encoders which didn’t >> properly >>> convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” text. >>> After fixing this, encoders to formats which don’t have a concept of “hard >> spaces”, >>> will output \h as regular spaces, and in the case of that single source >> line >>> that I have removed in the source file, this would lead to trailing spaces >> in the >>> ref files. >>> >>> As I don’t know how, when or whether at all this can be fixed in a way that >> it >>> could work with patches transported via e-mail, this single line change >> seems >>> to be the best quick solution to the problem. >>> >>> Thanks, >>> softworkz >>> >> >> I don't think that this will help. Trailing whitespace should actually >> only lead to a warning when applying (depending upon core.whitespace). >> >> Your patch is broken: There is a line "\ N is a forced line break" in >> the current ref file which ends with \r\n. This line is not marked as >> changed in your patchfile, but it is part of the surrounding context of >> a diff; and said line of context uses only \n and is therefore not >> recognized. >> >> And when I create a diff where the patch file has the proper context, it >> still fails; I need to add the "--keep-cr" option to git am to make it >> work. But then it works, whereas this option didn't help if the patch >> file only has \n instead of \r\n. >> >> - Andreas > > Thanks for looking into this. > > I'm simply going after the patchwork error from my recent submnission: > > https://patchwork.ffmpeg.org/check/47132/ > -- > .git/rebase-apply/patch:131: trailing whitespace. > A letter followed by 05 hard spaces: A > .git/rebase-apply/patch:192: trailing whitespace. > A letter followed by 05 hard spaces: A > error: patch failed: tests/ref/fate/sub-textenc:160 > error: tests/ref/fate/sub-textenc: patch does not apply > error: Did you hand edit your patch? > It does not apply to blobs recorded in its index. > -- > > It fails exactly at the line that I have removed in > SubRip_capability_tester2.srt No, it does not. Look at the error: "error: patch failed: tests/ref/fate/sub-textenc:160" Line 160 is "\ N is a forced line break", the first line of context with \r\n. It has nothing to do with the line you removed. The trailing-whitespace stuff just leads to warnings, not errors. - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] FATE-Suite Data Test Data
> -Original Message- > From: ffmpeg-devel On Behalf Of Andreas > Rheinhardt > Sent: Friday, December 10, 2021 8:00 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] FATE-Suite Data Test Data > > Soft Works: > > Hi, > > > > can we add the attached file (SubRip_capability_tester2.srt) to the fate- > suite data? > > > > Compared to SubRip_capability_tester.srt, it has one line removed that > would > > cause (legitimate) trailing whitespace in some ref data files. > > > > It hasn’t been an issue so far due to a bug in the encoders which didn’t > properly > > convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” text. > > After fixing this, encoders to formats which don’t have a concept of “hard > spaces”, > > will output \h as regular spaces, and in the case of that single source > line > > that I have removed in the source file, this would lead to trailing spaces > in the > > ref files. > > > > As I don’t know how, when or whether at all this can be fixed in a way that > it > > could work with patches transported via e-mail, this single line change > seems > > to be the best quick solution to the problem. > > > > Thanks, > > softworkz > > > > I don't think that this will help. Trailing whitespace should actually > only lead to a warning when applying (depending upon core.whitespace). > > Your patch is broken: There is a line "\ N is a forced line break" in > the current ref file which ends with \r\n. This line is not marked as > changed in your patchfile, but it is part of the surrounding context of > a diff; and said line of context uses only \n and is therefore not > recognized. > > And when I create a diff where the patch file has the proper context, it > still fails; I need to add the "--keep-cr" option to git am to make it > work. But then it works, whereas this option didn't help if the patch > file only has \n instead of \r\n. > > - Andreas Thanks for looking into this. I'm simply going after the patchwork error from my recent submnission: https://patchwork.ffmpeg.org/check/47132/ -- .git/rebase-apply/patch:131: trailing whitespace. A letter followed by 05 hard spaces: A .git/rebase-apply/patch:192: trailing whitespace. A letter followed by 05 hard spaces: A error: patch failed: tests/ref/fate/sub-textenc:160 error: tests/ref/fate/sub-textenc: patch does not apply error: Did you hand edit your patch? It does not apply to blobs recorded in its index. -- It fails exactly at the line that I have removed in SubRip_capability_tester2.srt > PS: The intra-subtitle newlines in said files are \r\n, yet the > inter-subtitle delimiters are \n only. This inconsistency might be > considered a bug. You may be right, but fixing this would be a different task, I think. Kind regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] FATE-Suite Data Test Data
Andreas Rheinhardt: > Soft Works: >> Hi, >> >> can we add the attached file (SubRip_capability_tester2.srt) to the >> fate-suite data? >> >> Compared to SubRip_capability_tester.srt, it has one line removed that would >> cause (legitimate) trailing whitespace in some ref data files. >> >> It hasn’t been an issue so far due to a bug in the encoders which didn’t >> properly >> convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” text. >> After fixing this, encoders to formats which don’t have a concept of “hard >> spaces”, >> will output \h as regular spaces, and in the case of that single source line >> that I have removed in the source file, this would lead to trailing spaces >> in the >> ref files. >> >> As I don’t know how, when or whether at all this can be fixed in a way that >> it >> could work with patches transported via e-mail, this single line change seems >> to be the best quick solution to the problem. >> >> Thanks, >> softworkz >> > > I don't think that this will help. Trailing whitespace should actually > only lead to a warning when applying (depending upon core.whitespace). > > Your patch is broken: There is a line "\ N is a forced line break" in > the current ref file which ends with \r\n. This line is not marked as > changed in your patchfile, but it is part of the surrounding context of > a diff; and said line of context uses only \n and is therefore not > recognized. I based this part on a patch (mbox) downloaded from patchwork; your original patch seems to have correct line-endings, so that you (or rather: patchwork) run into the following issue: > > And when I create a diff where the patch file has the proper context, it > still fails; I need to add the "--keep-cr" option to git am to make it > work. But then it works, whereas this option didn't help if the patch > file only has \n instead of \r\n. > > - Andreas > > PS: The intra-subtitle newlines in said files are \r\n, yet the > inter-subtitle delimiters are \n only. This inconsistency might be > considered a bug. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/4] avfilter/vf_transpose_vulkan: simplify config_props_output function
It's no need to assign outlink here, which has been done in
ff_vk_filter_config_output already.
Signed-off-by: Wu Jianhua
---
libavfilter/vf_transpose_vulkan.c | 11 +--
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/libavfilter/vf_transpose_vulkan.c
b/libavfilter/vf_transpose_vulkan.c
index eceb9b9011..ce83cf0fd7 100644
--- a/libavfilter/vf_transpose_vulkan.c
+++ b/libavfilter/vf_transpose_vulkan.c
@@ -262,7 +262,6 @@ static av_cold void transpose_vulkan_uninit(AVFilterContext
*avctx)
static int config_props_output(AVFilterLink *outlink)
{
-int err = 0;
AVFilterContext *avctx = outlink->src;
TransposeVulkanContext *s = avctx->priv;
FFVulkanContext *vkctx = &s->vkctx;
@@ -271,21 +270,13 @@ static int config_props_output(AVFilterLink *outlink)
vkctx->output_width = inlink->h;
vkctx->output_height = inlink->w;
-RET(ff_vk_filter_config_output(outlink));
-
-outlink->w = inlink->h;
-outlink->h = inlink->w;
-
if (inlink->sample_aspect_ratio.num)
outlink->sample_aspect_ratio = av_div_q((AVRational) { 1, 1 },
inlink->sample_aspect_ratio);
else
outlink->sample_aspect_ratio = inlink->sample_aspect_ratio;
-err = 0;
-
-fail:
-return err;
+return ff_vk_filter_config_output(outlink);
}
#define OFFSET(x) offsetof(TransposeVulkanContext, x)
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/4] avfilter/vf_transpose_vulkan: add clock option
The following command is on how to apply clock option:
ffmpeg -init_hw_device vulkan -i input.264 -vf \
hwupload=extra_hw_frames=16,transpose_vulkan=dir=clock,hwdownload,format=yuv420p
\
output.264
Signed-off-by: Wu Jianhua
---
libavfilter/vf_transpose_vulkan.c | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/libavfilter/vf_transpose_vulkan.c
b/libavfilter/vf_transpose_vulkan.c
index 4c20becb5c..eceb9b9011 100644
--- a/libavfilter/vf_transpose_vulkan.c
+++ b/libavfilter/vf_transpose_vulkan.c
@@ -88,16 +88,18 @@ static av_cold int init_filter(AVFilterContext *ctx,
AVFrame *in)
GLSLC(0, void main() );
GLSLC(0, { );
GLSLC(1, ivec2 size; );
-GLSLC(1, const ivec2 pos = ivec2(gl_GlobalInvocationID.xy););
+GLSLC(1, ivec2 pos = ivec2(gl_GlobalInvocationID.xy); );
for (int i = 0; i < planes; i++) {
GLSLC(0, );
GLSLF(1, size = imageSize(output_images[%i]);,i);
GLSLC(1, if (IS_WITHIN(pos, size)) { );
if (s->dir == TRANSPOSE_CCLOCK)
GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.y -
pos.y, pos.x)); ,i);
-else if (s->dir == TRANSPOSE_CLOCK_FLIP)
+else if (s->dir == TRANSPOSE_CLOCK_FLIP || s->dir ==
TRANSPOSE_CLOCK) {
GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.yx -
pos.yx)); ,i);
-else
+if (s->dir == TRANSPOSE_CLOCK)
+GLSLC(2, pos = ivec2(pos.x, size.y - pos.y); );
+} else
GLSLF(2, vec4 res = texture(input_images[%i], pos.yx); ,i);
GLSLF(2, imageStore(output_images[%i], pos, res);,i);
GLSLC(1, } );
@@ -292,6 +294,7 @@ fail:
static const AVOption transpose_vulkan_options[] = {
{ "dir", "set transpose direction", OFFSET(dir), AV_OPT_TYPE_INT, { .i64 =
TRANSPOSE_CCLOCK_FLIP }, 0, 7, FLAGS, "dir" },
{ "cclock_flip", "rotate counter-clockwise with vertical flip", 0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK_FLIP }, .flags=FLAGS, .unit =
"dir" },
+{ "clock", "rotate clockwise",0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK }, .flags=FLAGS, .unit =
"dir" },
{ "cclock", "rotate counter-clockwise",0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK }, .flags=FLAGS, .unit =
"dir" },
{ "clock_flip", "rotate clockwise with vertical flip", 0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK_FLIP }, .flags=FLAGS, .unit =
"dir" },
{ NULL }
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/4] avfilter/vf_transpose_vulkan: add clock_flip option
The following command is on how to apply clock_flip option:
ffmpeg -init_hw_device vulkan -i input.264 -vf \
hwupload=extra_hw_frames=16,transpose_vulkan=dir=clock_flip,hwdownload,format=yuv420p
\
output.264
Signed-off-by: Wu Jianhua
---
libavfilter/vf_transpose_vulkan.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/libavfilter/vf_transpose_vulkan.c
b/libavfilter/vf_transpose_vulkan.c
index 59a548a12f..4c20becb5c 100644
--- a/libavfilter/vf_transpose_vulkan.c
+++ b/libavfilter/vf_transpose_vulkan.c
@@ -95,6 +95,8 @@ static av_cold int init_filter(AVFilterContext *ctx, AVFrame
*in)
GLSLC(1, if (IS_WITHIN(pos, size)) { );
if (s->dir == TRANSPOSE_CCLOCK)
GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.y -
pos.y, pos.x)); ,i);
+else if (s->dir == TRANSPOSE_CLOCK_FLIP)
+GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.yx -
pos.yx)); ,i);
else
GLSLF(2, vec4 res = texture(input_images[%i], pos.yx); ,i);
GLSLF(2, imageStore(output_images[%i], pos, res);,i);
@@ -291,6 +293,7 @@ static const AVOption transpose_vulkan_options[] = {
{ "dir", "set transpose direction", OFFSET(dir), AV_OPT_TYPE_INT, { .i64 =
TRANSPOSE_CCLOCK_FLIP }, 0, 7, FLAGS, "dir" },
{ "cclock_flip", "rotate counter-clockwise with vertical flip", 0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK_FLIP }, .flags=FLAGS, .unit =
"dir" },
{ "cclock", "rotate counter-clockwise",0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK }, .flags=FLAGS, .unit =
"dir" },
+{ "clock_flip", "rotate clockwise with vertical flip", 0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CLOCK_FLIP }, .flags=FLAGS, .unit =
"dir" },
{ NULL }
};
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/4] avfilter/vf_transpose_vulkan: add cclock option
The following command is on how to apply cclock option:
ffmpeg -init_hw_device vulkan -i input.264 -vf \
hwupload=extra_hw_frames=16,transpose_vulkan=dir=cclock,hwdownload,format=yuv420p
\
output.264
Signed-off-by: Wu Jianhua
---
libavfilter/vf_transpose_vulkan.c | 17 ++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/libavfilter/vf_transpose_vulkan.c
b/libavfilter/vf_transpose_vulkan.c
index c9bae413c3..59a548a12f 100644
--- a/libavfilter/vf_transpose_vulkan.c
+++ b/libavfilter/vf_transpose_vulkan.c
@@ -21,6 +21,7 @@
#include "libavutil/opt.h"
#include "vulkan_filter.h"
#include "internal.h"
+#include "transpose.h"
#define CGS 32
@@ -33,6 +34,7 @@ typedef struct TransposeVulkanContext {
VkDescriptorImageInfo input_images[3];
VkDescriptorImageInfo output_images[3];
+int dir;
int initialized;
} TransposeVulkanContext;
@@ -89,10 +91,13 @@ static av_cold int init_filter(AVFilterContext *ctx,
AVFrame *in)
GLSLC(1, const ivec2 pos = ivec2(gl_GlobalInvocationID.xy););
for (int i = 0; i < planes; i++) {
GLSLC(0, );
-GLSLF(1, size = imageSize(output_images[%i]); ,i);
+GLSLF(1, size = imageSize(output_images[%i]);,i);
GLSLC(1, if (IS_WITHIN(pos, size)) { );
-GLSLF(2, vec4 res = texture(input_images[%i], pos.yx);,i);
-GLSLF(2, imageStore(output_images[%i], pos, res); ,i);
+if (s->dir == TRANSPOSE_CCLOCK)
+GLSLF(2, vec4 res = texture(input_images[%i], ivec2(size.y -
pos.y, pos.x)); ,i);
+else
+GLSLF(2, vec4 res = texture(input_images[%i], pos.yx); ,i);
+GLSLF(2, imageStore(output_images[%i], pos, res);,i);
GLSLC(1, } );
}
GLSLC(0, } );
@@ -279,7 +284,13 @@ fail:
return err;
}
+#define OFFSET(x) offsetof(TransposeVulkanContext, x)
+#define FLAGS (AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM)
+
static const AVOption transpose_vulkan_options[] = {
+{ "dir", "set transpose direction", OFFSET(dir), AV_OPT_TYPE_INT, { .i64 =
TRANSPOSE_CCLOCK_FLIP }, 0, 7, FLAGS, "dir" },
+{ "cclock_flip", "rotate counter-clockwise with vertical flip", 0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK_FLIP }, .flags=FLAGS, .unit =
"dir" },
+{ "cclock", "rotate counter-clockwise",0,
AV_OPT_TYPE_CONST, { .i64 = TRANSPOSE_CCLOCK }, .flags=FLAGS, .unit =
"dir" },
{ NULL }
};
--
2.25.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] FATE-Suite Data Test Data
Soft Works: > Hi, > > can we add the attached file (SubRip_capability_tester2.srt) to the > fate-suite data? > > Compared to SubRip_capability_tester.srt, it has one line removed that would > cause (legitimate) trailing whitespace in some ref data files. > > It hasn’t been an issue so far due to a bug in the encoders which didn’t > properly > convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” text. > After fixing this, encoders to formats which don’t have a concept of “hard > spaces”, > will output \h as regular spaces, and in the case of that single source line > that I have removed in the source file, this would lead to trailing spaces in > the > ref files. > > As I don’t know how, when or whether at all this can be fixed in a way that it > could work with patches transported via e-mail, this single line change seems > to be the best quick solution to the problem. > > Thanks, > softworkz > I don't think that this will help. Trailing whitespace should actually only lead to a warning when applying (depending upon core.whitespace). Your patch is broken: There is a line "\ N is a forced line break" in the current ref file which ends with \r\n. This line is not marked as changed in your patchfile, but it is part of the surrounding context of a diff; and said line of context uses only \n and is therefore not recognized. And when I create a diff where the patch file has the proper context, it still fails; I need to add the "--keep-cr" option to git am to make it work. But then it works, whereas this option didn't help if the patch file only has \n instead of \r\n. - Andreas PS: The intra-subtitle newlines in said files are \r\n, yet the inter-subtitle delimiters are \n only. This inconsistency might be considered a bug. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/4] lavu/frame: Add Dolby Vision metadata side data type
On Thu, 9 Dec 2021, Niklas Haas wrote:
On Thu, 09 Dec 2021 17:21:58 + Derek Buitenhuis
wrote:
On 12/8/2021 10:12 AM, Niklas Haas wrote:
+/* based on guesswork, see mkvtoolnix and dovi_tool */
+int av_dovi_profile(const AVDOVIRpuDataHeader *hdr)
+{
The correct way to find the profile is from the stream level
DOVI configuration record side data, if available.
So, I changed the API to accept the DOVI configuration record, but
actually getting that record into hevcdec isimpossible given the current
design of FFmpeg where stream-level properties are not mode available to
the decoder. Possibilities:
1. Add patch to propagate stream-level properties to
AVCodecContext.coded_side_data automatically
2. Add patch to automatically propagate stream-level properties to each
AVPacket somehow?
Don't we already have av_format_inject_global_side_data() for something
like this? It is not enabled by default however, only for ffplay, but I am
not sure why.
Regards,
Marton
3. Have the code read the profile from the AVPacket even though the side
data doesn't exist for them, and let this just be an open bug.
4. Something else?
Thoughts?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 1/4] lavu/frame: Add Dolby Vision metadata side data type
On 12/9/2021 9:46 PM, Niklas Haas wrote: > So, I changed the API to accept the DOVI configuration record, but > actually getting that record into hevcdec isimpossible given the current > design of FFmpeg where stream-level properties are not mode available to > the decoder. Possibilities: Ugh, yes, now I remember - I an into this same issue when adding RPU buffer export. > 1. Add patch to propagate stream-level properties to >AVCodecContext.coded_side_data automatically I don't know enough about what coded_side_data is to comment, I think. > 2. Add patch to automatically propagate stream-level properties to each >AVPacket somehow? This seems excessive if it means attaching stream side data to every single packet... > 3. Have the code read the profile from the AVPacket even though the side >data doesn't exist for them, and let this just be an open bug. If you mean just leaving the level guessing in, I guess it is somehow the least bad idea... somehow. > 4. Something else? I hope someone else does have an idea ;) - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] lavfi/drawtext: Add localtime_ms for millisecond precision
On Fri, Dec 10, 2021 at 12:36:21PM +0100, Thilo Borgmann wrote:
>
>
> On 10 Dec 2021, at 3:47, zhilizhao(赵志立) wrote:
>
> > > On Dec 10, 2021, at 3:11 AM, Thilo Borgmann
> > > wrote:
> > >
> > > Hi,
> > >
> > > add %{localtime_ms}, %{gmtime_ms} functions to the drawtext filter.
> > > Same as %{localtime}/%{gmtime} but with additional millisecond part.
> > >
> > > sorry for delay, second version including review remarks:
> > >
> > > -get timing once
> > > -also add gmtime_ms instead of just localtime_ms
> > >
> >
> > > +if (tag == 'M' || tag == 'm') {
> > > +char ms[5] = {0};
> > > +int64_t dnow = (unow - ((int64_t)now) * 100) / 1000;
> > > +snprintf(ms, 5, ".%03d", (int)dnow);
> > > +av_bprint_append_data(bp, ms, 4);
> > > +}
> > >
> >
> >
> > How about
> >
> > av_bprintf(&bp, ".%03d", (int)(unow % 100) / 1000);
>
> Makes way too much sense. I need holidays…
>
> Attached v3.
>
> Thanks!
> -Thilo
> doc/filters.texi |8
> libavfilter/vf_drawtext.c | 12 ++--
> 2 files changed, 18 insertions(+), 2 deletions(-)
> 87d34e4106b829d42c5e57c847c28bed08bf3a81
> v3-0001-lavfi-drawtext-Add-localtime_ms-for-millisecond-p.patch
> From fd34d1434e2243a881c24f6db4cc0db92289f4bb Mon Sep 17 00:00:00 2001
> From: Thilo Borgmann
> Date: Fri, 10 Dec 2021 12:34:23 +0100
> Subject: [PATCH v3] lavfi/drawtext: Add localtime_ms for millisecond precision
Iam missining something here as it doesnt build
AR libavdevice/libavdevice.a
CC libavfilter/vf_drawtext.o
libavfilter/vf_drawtext.c: In function ‘func_strftime’:
libavfilter/vf_drawtext.c:1052:12: error: implicit declaration of function
‘av_gettime’; did you mean ‘av_get_token’?
[-Werror=implicit-function-declaration]
unow = av_gettime();
^~
av_get_token
libavfilter/vf_drawtext.c:1061:20: warning: passing argument 1 of ‘av_bprintf’
from incompatible pointer type [-Wincompatible-pointer-types]
av_bprintf(&bp, ".%03d", (int)(unow % 100) / 1000);
^
In file included from libavfilter/vf_drawtext.c:47:0:
./libavutil/bprint.h:127:6: note: expected ‘AVBPrint * {aka struct AVBPrint *}’
but argument is of type ‘AVBPrint ** {aka struct AVBPrint **}’
void av_bprintf(AVBPrint *buf, const char *fmt, ...) av_printf_format(2, 3);
^~
cc1: some warnings being treated as errors
ffbuild/common.mak:70: recipe for target 'libavfilter/vf_drawtext.o' failed
make: *** [libavfilter/vf_drawtext.o] Error 1
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Republics decline into democracies and democracies degenerate into
despotisms. -- Aristotle
signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] FATE-Suite Data Test Data
Hi, can we add the attached file (SubRip_capability_tester2.srt) to the fate-suite data? Compared to SubRip_capability_tester.srt, it has one line removed that would cause (legitimate) trailing whitespace in some ref data files. It hasn’t been an issue so far due to a bug in the encoders which didn’t properly convert ASS hard-space tags (\h) and incorrectly emitted them as “\h” text. After fixing this, encoders to formats which don’t have a concept of “hard spaces”, will output \h as regular spaces, and in the case of that single source line that I have removed in the source file, this would lead to trailing spaces in the ref files. As I don’t know how, when or whether at all this can be fixed in a way that it could work with patches transported via e-mail, this single line change seems to be the best quick solution to the problem. Thanks, softworkz <> ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH V6 5/5] libavutil/hwcontext_vulkan: specify the modifier to create VKImage
7 Dec 2021, 10:05 by wenbin.c...@intel.com: > When vulkan image exports to drm, the tilling need to be > VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT. Now add code to create vulkan > image using this format. > > Now the following command line works: > > ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 > -hwaccel_output_format \ > vaapi -i input_1080p.264 -vf "hwmap=derive_device=vulkan,format=vulkan, \ > scale_vulkan=1920:1080,hwmap=derive_device=vaapi,format=vaapi" -c:v > h264_vaapi output.264 > > Signed-off-by: Wenbin Chen > --- > libavutil/hwcontext_vulkan.c | 133 +-- > 1 file changed, 127 insertions(+), 6 deletions(-) > Pushed the patchset, along with the transpose filter. Tested on all 3 vendors, works fine. AMD requires dedicated allocation, for which I added an error condition. The code was quite ugly in quite a lot of places, so I did touch mostly every line of it in some way. Please fix your quirky, limited hardware next time. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of first input to set output format for overlay_qsv
> -Original Message-
> From: ffmpeg-devel On Behalf Of Wenbin Chen
> Sent: Friday, December 10, 2021 3:22 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Wenbin Chen
> Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of
> first input to set output format for overlay_qsv
>
> overlay_qsv hard coded to use nv12 as output format. Now use the format
> of the first input to set output format. Now the following command
> works:
>
> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v
> hevc_qsv \
> -i input2_p010.265 -filter_complex "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255,
> \
> hwdownload,format=p010le" -f rawvideo -y output_p010.yuv
>
> Signed-off-by: Wenbin Chen
> ---
> libavfilter/vf_overlay_qsv.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c
> index 7e76b39aa9..d947a1faa1 100644
> --- a/libavfilter/vf_overlay_qsv.c
> +++ b/libavfilter/vf_overlay_qsv.c
> @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink)
> int ret;
>
> av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n",
> av_get_pix_fmt_name(outlink->format));
> +vpp->qsv_param.out_sw_format = in0->format;
> if ((in0->format == AV_PIX_FMT_QSV && in1->format != AV_PIX_FMT_QSV) ||
> (in0->format != AV_PIX_FMT_QSV && in1->format == AV_PIX_FMT_QSV)) {
> av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel
> formats is not supported.\n");
> @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink)
> av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV
> devices are forbidden.\n");
> return AVERROR(EINVAL);
> }
> +vpp->qsv_param.out_sw_format = hw_frame0->sw_format;
> }
>
> outlink->w = vpp->var_values[VAR_MW];
> --
When you're already looking at that area:
Do you have any idea why it sets the output time_base to the reciprocal
input0 framerate?
I think it should rather set it to the same as the input0 time_base.
https://github.com/FFmpeg/FFmpeg/blob/4f44a218e53cd92e64ba10a935bc1e7583c3e218/libavfilter/vf_overlay_qsv.c#L293-L296
What do you think? (also @Haihao)
Thanks,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of first input to set output format for overlay_qsv
> -Original Message-
> From: ffmpeg-devel On Behalf Of Wenbin Chen
> Sent: Friday, December 10, 2021 3:22 AM
> To: ffmpeg-devel@ffmpeg.org
> Cc: Wenbin Chen
> Subject: [FFmpeg-devel] [PATCH] libavfilter/vf_overlay_qsv: Use format of
> first input to set output format for overlay_qsv
>
> overlay_qsv hard coded to use nv12 as output format. Now use the format
> of the first input to set output format. Now the following command
> works:
>
> ffmpeg -hwaccel qsv -c:v hevc_qsv -i input_p010.265 -hwaccel qsv -c:v
> hevc_qsv \
> -i input2_p010.265 -filter_complex "[0:v][1:v]overlay_qsv=x=0:y=0:alpha=255,
> \
> hwdownload,format=p010le" -f rawvideo -y output_p010.yuv
>
> Signed-off-by: Wenbin Chen
> ---
> libavfilter/vf_overlay_qsv.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/libavfilter/vf_overlay_qsv.c b/libavfilter/vf_overlay_qsv.c
> index 7e76b39aa9..d947a1faa1 100644
> --- a/libavfilter/vf_overlay_qsv.c
> +++ b/libavfilter/vf_overlay_qsv.c
> @@ -276,6 +276,7 @@ static int config_output(AVFilterLink *outlink)
> int ret;
>
> av_log(ctx, AV_LOG_DEBUG, "Output is of %s.\n",
> av_get_pix_fmt_name(outlink->format));
> +vpp->qsv_param.out_sw_format = in0->format;
> if ((in0->format == AV_PIX_FMT_QSV && in1->format != AV_PIX_FMT_QSV) ||
> (in0->format != AV_PIX_FMT_QSV && in1->format == AV_PIX_FMT_QSV)) {
> av_log(ctx, AV_LOG_ERROR, "Mixing hardware and software pixel
> formats is not supported.\n");
> @@ -288,6 +289,7 @@ static int config_output(AVFilterLink *outlink)
> av_log(ctx, AV_LOG_ERROR, "Inputs with different underlying QSV
> devices are forbidden.\n");
> return AVERROR(EINVAL);
> }
> +vpp->qsv_param.out_sw_format = hw_frame0->sw_format;
> }
>
> outlink->w = vpp->var_values[VAR_MW];
> --
Isn't this a bit too optimistic?
The doc says:
The only supported combinations of input and output color formats are:
RGB to RGB,
NV12 to NV12,
RGB and NV12 to NV12, for per pixel alpha blending use case.
https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextvppcomposite
Kind regards,
softworkz
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame\n for subtitle handling
> -Original Message- > From: ffmpeg-devel On Behalf Of Daniel > Cantarín > Sent: Thursday, December 9, 2021 10:33 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame\n > for subtitle handling > > Hi there. > This is my first message to this list, so please excuse me if I > unintendedly break some rule. > > I've read the debate between Soft Works and others, and would like to > add something to it. > I don't have a deep knowledge of the libs as other people here show. My > knowledge comes from working with live streams for some years now. And I > do understand the issue about modifying a public API for some use case > under debate: I believe it's a legit line of questioning to Soft Works > patches. However, I also feel we live streaming people are often let > aside as "border case" when it comes to ffmpeg/libav usage, and this > bias is present in many subtitles/captions debates. > > I work with Digital TV signals as input, and several different target > outputs more related to live streaming (mobiles, PCs, and so on). The > target location is Latin America, and thus I need subtitles/captions for > when we use english spoken audio (we speak mostly Spanish in LATAM). TV > people send you TV subtitle formats: scte-27, dvb subs, and so on. And > live streaming people uses other subtitles formats, mostly vtt and ttml. > I've found that CEA-608 captions are the most compatible caption format, > as it's understood natively by smart tvs and other devices, as well as > non-natively by any other device using popular player-side libraries. > So, I've made my own filter for generating CEA-608 captions for live > streams, using ffmpeg with the previously available OCR filter. Tried > VTT first, but it was problematic for live-streaming packaging, and with > CEA-608 I could just ignore that part of the process. > > While doing those filters, besides the whole deal of implementing the > conversion from text to CEA-608, I struggled with stuff like this: > - the sparseness of input subtitles, leading to OOM in servers and > stalled players. > - the "libavfilter doesn't take subtitle frames" and "it's all ASS > internally" issues. > - the "captions timings vs video frame timings vs audio timings" > problems (people talk a lot about syncing subs with video frames, but > rarely against actual dialogue audio). > - other (meta)data problems, like screen positioning or text encoding. > > This are all problems Soft Works seems to have faced as well. > > But of all the problems regarding live streaming subtitles with ffmpeg > (and there are LOTS of it), the most annoying problem is always this: > almost every time someone talked about implementing subtitles in filters > (in mail lists, in tickets, in other places like stack overflow, > etcetera), they always asumed input files. When the people specifically > talked about live streams, their peers always reasoned with files > mindset, and stated live streaming subtitles/captions as "border case". > > Let me be clear: this are not "border case" issues, but actually appear > in the most common use cases of live streaming transcoding. They all > appear *inmediatelly* when you try to use subtitles/captions in live > streams. > > I got here (I mean this thread) while looking for ways to fixing some > issues in my setup. I was reconsidering VTT/TTML generation instead of > CEA-608 (as rendering behave significantly different from device to > device), and thus I was about to generate subtitle type output from some > filter, was about to create my own standalone "heartbeat" filter to > normalize the sparseness, and so on and so on: again, all stuff Soft > Works seems to be handling as well. So I was quite happy to find someone > working on this again; last time I've seen it in ffmpeg's > mailing/patchwork > (https://patchwork.ffmpeg.org/project/ffmpeg/patch/20161102220934.26010-1- > u...@pkh.me) > the code there seemed to die, and I was already late to say anything > about it. However, reading the other devs reaction to Soft Works work > was worrying, as it felt as history wanted to repeat itself (take a look > at discussions back then). > > It has been years so far of this situation. This time I wanted to > annotate this, as this conversation is still warm, in order to help Soft > Works's code survive. So, dear devs: I love and respect your work, and > your opinion is very important to me. I do not claim to know better than > you do ffmpeg's code. I do not claim to know better what to do with > libavfilter's API. Please understand: I'm not here to be right, but to > note my point of view. I'm not better than you; quite on the contrary > most likely. But I also need to solve some very real problems, and can't > wait until everything else is in wonderful shape to do it. I can't also > add lots of conditions in order to just fix the most immediate issues; > like it's the case with sparseness and heartbeat frames,
Re: [FFmpeg-devel] [PATCH] Fix comparing NULL string when processing certain subtitle files.
Charlie Monroe:
> Hi,
>
> I hope I'm submitting this fix correctly - some subtitles will cause FFmpeg
> to
> crash when searching for a font in movtextencode. One such subtitles file is
> appended. The crash occurrs in find_font_id where the name argument passed is
> NULL.
>
> Here is an example command for reproducing the issue:
>
> ffmpeg -i test.mp4 -i test.ass -c:v copy -c:a copy -c:s 'mov_text' out.mp4
>
> Cheers,
>
> Charlie
>
>
>
> Signed-off-by: Charlie Monroe
> ---
> libavcodec/movtextenc.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c
> index 5869942ec0..df3a2dd97a 100644
> --- a/libavcodec/movtextenc.c
> +++ b/libavcodec/movtextenc.c
> @@ -494,6 +494,9 @@ static void mov_text_alpha_cb(void *priv, int alpha, int
> alpha_id)
>
> static uint16_t find_font_id(MovTextContext *s, const char *name)
> {
> +if (name == NULL)
> +return 1;
> +
> int i;
> for (i = 0; i < s->font_count; i++) {
> if (!strcmp(name, s->fonts[i]))
> --
> 2.30.1 (Apple Git-130)
>
Thanks for this patch. I will apply this (with the
declaration-after-statement issue fixed) tomorrow unless someone objects
to it or beats me to it.
- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] lavfi/drawtext: Add localtime_ms for millisecond precision
On 10 Dec 2021, at 3:47, zhilizhao(赵志立) wrote:
On Dec 10, 2021, at 3:11 AM, Thilo Borgmann
wrote:
Hi,
add %{localtime_ms}, %{gmtime_ms} functions to the drawtext filter.
Same as %{localtime}/%{gmtime} but with additional millisecond part.
sorry for delay, second version including review remarks:
-get timing once
-also add gmtime_ms instead of just localtime_ms
+if (tag == 'M' || tag == 'm') {
+char ms[5] = {0};
+int64_t dnow = (unow - ((int64_t)now) * 100) / 1000;
+snprintf(ms, 5, ".%03d", (int)dnow);
+av_bprint_append_data(bp, ms, 4);
+}
How about
av_bprintf(&bp, ".%03d", (int)(unow % 100) / 1000);
Makes way too much sense. I need holidays…
Attached v3.
Thanks!
-Thilo
From fd34d1434e2243a881c24f6db4cc0db92289f4bb Mon Sep 17 00:00:00 2001
From: Thilo Borgmann
Date: Fri, 10 Dec 2021 12:34:23 +0100
Subject: [PATCH v3] lavfi/drawtext: Add localtime_ms for millisecond precision
Suggested-By: ffm...@fb.com
---
doc/filters.texi | 8
libavfilter/vf_drawtext.c | 12 ++--
2 files changed, 18 insertions(+), 2 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 78faf76..db75632 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -10949,10 +10949,18 @@ It can be used to add padding with zeros from the
left.
The time at which the filter is running, expressed in UTC.
It can accept an argument: a strftime() format string.
+@item gmtime_ms
+Same as @code{gmtime} but with millisecond precision.
+It can accept an argument: a strftime() format string.
+
@item localtime
The time at which the filter is running, expressed in the local time zone.
It can accept an argument: a strftime() format string.
+@item localtime_ms
+Same as @code{localtime} but with millisecond precision.
+It can accept an argument: a strftime() format string.
+
@item metadata
Frame metadata. Takes one or two arguments.
diff --git a/libavfilter/vf_drawtext.c b/libavfilter/vf_drawtext.c
index 382d589..36c9103 100644
--- a/libavfilter/vf_drawtext.c
+++ b/libavfilter/vf_drawtext.c
@@ -1045,15 +1045,21 @@ static int func_strftime(AVFilterContext *ctx, AVBPrint
*bp,
char *fct, unsigned argc, char **argv, int tag)
{
const char *fmt = argc ? argv[0] : "%Y-%m-%d %H:%M:%S";
+int64_t unow;
time_t now;
struct tm tm;
-time(&now);
-if (tag == 'L')
+unow = av_gettime();
+now = unow / 100;
+if (tag == 'L' || tag == 'm')
localtime_r(&now, &tm);
else
tm = *gmtime_r(&now, &tm);
av_bprint_strftime(bp, fmt, &tm);
+
+if (tag == 'M' || tag == 'm') {
+av_bprintf(&bp, ".%03d", (int)(unow % 100) / 1000);
+}
return 0;
}
@@ -1152,7 +1158,9 @@ static const struct drawtext_function {
{ "pict_type", 0, 0, 0, func_pict_type },
{ "pts", 0, 3, 0, func_pts },
{ "gmtime",0, 1, 'G', func_strftime },
+{ "gmtime_ms", 0, 1, 'M', func_strftime },
{ "localtime", 0, 1, 'L', func_strftime },
+{ "localtime_ms", 0, 1, 'm', func_strftime },
{ "frame_num", 0, 0, 0, func_frame_num },
{ "n", 0, 0, 0, func_frame_num },
{ "metadata", 1, 2, 0, func_metadata },
--
1.8.3.2
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] libavdevice/avfoundation.m: use AudioConvert, extend supported formats
Hi, On 8 Dec 2021, at 19:29, Romain Beauxis wrote: On Dec 7, 2021, at 7:21 AM, Thilo Borgmann wrote: Hi, will look at this soon (tm), ping me if I don‘t. Great, thank you! Would it be a good use of your time to send the two other patches that I have pending as well? yes, best to send a patchset of all three of them in one thread. Expecting to investigate this next week, would be good if it would have hit the ML by then :) -Thilo ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] Fix comparing NULL string when processing certain subtitle files.
Hi,
I hope I'm submitting this fix correctly - some subtitles will cause FFmpeg to
crash when searching for a font in movtextencode. One such subtitles file is
appended. The crash occurrs in find_font_id where the name argument passed is
NULL.
Here is an example command for reproducing the issue:
ffmpeg -i test.mp4 -i test.ass -c:v copy -c:a copy -c:s 'mov_text' out.mp4
Cheers,
Charlie
Signed-off-by: Charlie Monroe
---
libavcodec/movtextenc.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/libavcodec/movtextenc.c b/libavcodec/movtextenc.c
index 5869942ec0..df3a2dd97a 100644
--- a/libavcodec/movtextenc.c
+++ b/libavcodec/movtextenc.c
@@ -494,6 +494,9 @@ static void mov_text_alpha_cb(void *priv, int alpha, int
alpha_id)
static uint16_t find_font_id(MovTextContext *s, const char *name)
{
+if (name == NULL)
+return 1;
+
int i;
for (i = 0; i < s->font_count; i++) {
if (!strcmp(name, s->fonts[i]))
--
2.30.1 (Apple Git-130)
test.ass
Description: Binary data
patch.diff
Description: Binary data
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v3 2/6] avfilter/af_astats: improve options descriptions
From: Limin Wang
Signed-off-by: Limin Wang
---
doc/filters.texi| 7 ---
libavfilter/af_astats.c | 6 +++---
2 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 8eff460..e5059f2 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -2886,16 +2886,17 @@ this @code{lavfi.astats.Overall.Peak_count}.
For description what each key means read below.
@item reset
-Set number of frame after which stats are going to be recalculated.
+Set the number of frames over which cumulative stats are calculated before
+being reset
Default is disabled.
@item measure_perchannel
-Select the entries which need to be measured per channel. The metadata keys can
+Select the parameters which are measured per channel. The metadata keys can
be used as flags, default is @option{all} which measures everything.
@option{none} disables all per channel measurement.
@item measure_overall
-Select the entries which need to be measured overall. The metadata keys can
+Select the parameters which are measured overall. The metadata keys can
be used as flags, default is @option{all} which measures everything.
@option{none} disables all overall measurement.
diff --git a/libavfilter/af_astats.c b/libavfilter/af_astats.c
index d0810b5..6c79257 100644
--- a/libavfilter/af_astats.c
+++ b/libavfilter/af_astats.c
@@ -114,8 +114,8 @@ typedef struct AudioStatsContext {
static const AVOption astats_options[] = {
{ "length", "set the window length", OFFSET(time_constant),
AV_OPT_TYPE_DOUBLE, {.dbl=.05}, 0, 10, FLAGS },
{ "metadata", "inject metadata in the filtergraph", OFFSET(metadata),
AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, FLAGS },
-{ "reset", "recalculate stats after this many frames",
OFFSET(reset_count), AV_OPT_TYPE_INT, {.i64=0}, 0, INT_MAX, FLAGS },
-{ "measure_perchannel", "only measure_perchannel these per-channel
statistics", OFFSET(measure_perchannel), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL},
0, UINT_MAX, FLAGS, "measure" },
+{ "reset", "Set the number of frames over which cumulative stats are
calculated before being reset", OFFSET(reset_count), AV_OPT_TYPE_INT, {.i64=0},
0, INT_MAX, FLAGS },
+{ "measure_perchannel", "Select the parameters which are measured per
channel", OFFSET(measure_perchannel), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0,
UINT_MAX, FLAGS, "measure" },
{ "none" , "", 0, AV_OPT_TYPE_CONST,
{.i64=MEASURE_NONE}, 0, 0, FLAGS, "measure" },
{ "all" , "", 0, AV_OPT_TYPE_CONST,
{.i64=MEASURE_ALL }, 0, 0, FLAGS, "measure" },
{ "DC_offset" , "", 0, AV_OPT_TYPE_CONST,
{.i64=MEASURE_DC_OFFSET }, 0, 0, FLAGS, "measure" },
@@ -143,7 +143,7 @@ static const AVOption astats_options[] = {
{ "Number_of_NaNs", "", 0, AV_OPT_TYPE_CONST,
{.i64=MEASURE_NUMBER_OF_NANS }, 0, 0, FLAGS, "measure" },
{ "Number_of_Infs", "", 0, AV_OPT_TYPE_CONST,
{.i64=MEASURE_NUMBER_OF_INFS }, 0, 0, FLAGS, "measure" },
{ "Number_of_denormals" , "", 0, AV_OPT_TYPE_CONST,
{.i64=MEASURE_NUMBER_OF_DENORMALS }, 0, 0, FLAGS, "measure" },
-{ "measure_overall", "only measure_perchannel these overall statistics",
OFFSET(measure_overall), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0, UINT_MAX,
FLAGS, "measure" },
+{ "measure_overall", "Select the parameters which are measured overall",
OFFSET(measure_overall), AV_OPT_TYPE_FLAGS, {.i64=MEASURE_ALL}, 0, UINT_MAX,
FLAGS, "measure" },
{ NULL }
};
--
1.8.3.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 000/279] New channel layout API
On Thu, Dec 09, 2021 at 04:47:48PM +0100, Lynne wrote: [...] > So I'm fine with your proposal to have 16-bit enum for the channel > ID and a 16-bit opaque. Though I'd like the opaque to be an > uint16_t instead of int opaque : 16. > And 16-bits does sound like enough for many channels and quite a > few flags, though the silent flag should be moved to 1 << 15 instead > of 64, and any new flags could be added beneath so as to not conflict > with channels. in how many cases where we use flags have 16 been enough ? thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I have never wished to cater to the crowd; for what I know they do not approve, and what they approve I do not know. -- Epicurus signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v20 02/20] avutil/frame: Prepare AVFrame\n for subtitle handling
On Thu, Dec 9, 2021 at 10:33 PM Daniel Cantarín wrote: > Hi there. > This is my first message to this list, so please excuse me if I > unintendedly break some rule. > > I've read the debate between Soft Works and others, and would like to > add something to it. > I don't have a deep knowledge of the libs as other people here show. My > knowledge comes from working with live streams for some years now. And I > do understand the issue about modifying a public API for some use case > under debate: I believe it's a legit line of questioning to Soft Works > patches. However, I also feel we live streaming people are often let > aside as "border case" when it comes to ffmpeg/libav usage, and this > bias is present in many subtitles/captions debates. > > I work with Digital TV signals as input, and several different target > outputs more related to live streaming (mobiles, PCs, and so on). The > target location is Latin America, and thus I need subtitles/captions for > when we use english spoken audio (we speak mostly Spanish in LATAM). TV > people send you TV subtitle formats: scte-27, dvb subs, and so on. And > live streaming people uses other subtitles formats, mostly vtt and ttml. > I've found that CEA-608 captions are the most compatible caption format, > as it's understood natively by smart tvs and other devices, as well as > non-natively by any other device using popular player-side libraries. > So, I've made my own filter for generating CEA-608 captions for live > streams, using ffmpeg with the previously available OCR filter. Tried > VTT first, but it was problematic for live-streaming packaging, and with > CEA-608 I could just ignore that part of the process. > > While doing those filters, besides the whole deal of implementing the > conversion from text to CEA-608, I struggled with stuff like this: > - the sparseness of input subtitles, leading to OOM in servers and > stalled players. > - the "libavfilter doesn't take subtitle frames" and "it's all ASS > internally" issues. > - the "captions timings vs video frame timings vs audio timings" > problems (people talk a lot about syncing subs with video frames, but > rarely against actual dialogue audio). > - other (meta)data problems, like screen positioning or text encoding. > > This are all problems Soft Works seems to have faced as well. > > But of all the problems regarding live streaming subtitles with ffmpeg > (and there are LOTS of it), the most annoying problem is always this: > almost every time someone talked about implementing subtitles in filters > (in mail lists, in tickets, in other places like stack overflow, > etcetera), they always asumed input files. When the people specifically > talked about live streams, their peers always reasoned with files > mindset, and stated live streaming subtitles/captions as "border case". > > Let me be clear: this are not "border case" issues, but actually appear > in the most common use cases of live streaming transcoding. They all > appear *inmediatelly* when you try to use subtitles/captions in live > streams. > > I got here (I mean this thread) while looking for ways to fixing some > issues in my setup. I was reconsidering VTT/TTML generation instead of > CEA-608 (as rendering behave significantly different from device to > device), and thus I was about to generate subtitle type output from some > filter, was about to create my own standalone "heartbeat" filter to > normalize the sparseness, and so on and so on: again, all stuff Soft > Works seems to be handling as well. So I was quite happy to find someone > working on this again; last time I've seen it in ffmpeg's > mailing/patchwork > ( > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20161102220934.26010-...@pkh.me) > > the code there seemed to die, and I was already late to say anything > about it. However, reading the other devs reaction to Soft Works work > was worrying, as it felt as history wanted to repeat itself (take a look > at discussions back then). > > It has been years so far of this situation. This time I wanted to > annotate this, as this conversation is still warm, in order to help Soft > Works's code survive. So, dear devs: I love and respect your work, and > your opinion is very important to me. I do not claim to know better than > you do ffmpeg's code. I do not claim to know better what to do with > libavfilter's API. Please understand: I'm not here to be right, but to > note my point of view. I'm not better than you; quite on the contrary > most likely. But I also need to solve some very real problems, and can't > wait until everything else is in wonderful shape to do it. I can't also > add lots of conditions in order to just fix the most immediate issues; > like it's the case with sparseness and heartbeat frames, which was a > heated debate years ago and seems to still be one, while I find it to be > the most obvious common sense backwards-compatible solution > implementation. Stuff like "clean" or "well designed" can't be mor
