Re: [FFmpeg-devel] [PATCH] bwdif_vulkan: fix artifacts on vulkan decode images
On Sun, 29 Oct 2023 07:22:46 +0100 (CET) Lynne wrote: > Due to making the decode frames context use the coded size, the > filter started to display those artifacts as it reused the input > frame's size. > > Change it to instead output the real image size for images, not the > input. > > Patch attached. > LGTM. Thanks. --phil ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
Nov 5, 2023, 00:02 by mar...@martin.st: > Hi, > > Just following up on this - I'm sorry I haven't been able to look at the > proposed patchset myself quite in detail yet. > > My prime concern is about the requests to have this merged into the upcoming > 6.1 release; that's way too soon IMO. > > These patches do change aspects of how these things behave, that have been > working the same for a very long time, so there are all sorts of potential > subtle breakage, or (incorrect or not) assumptions being broken, across > libavcodec and its users. > > On Sat, 4 Nov 2023, Derek Buitenhuis wrote: > >> Next, a quick breakdown of the AAC situation, in terms of both how this it >> is stored, >> what we support, and the state of the ecosystem and types of files that >> exist: >> * 'Raw' ADTS streams have no way to store any of this. The best we can do >> is guess >> the pre-roll. We should not guess priming or end padding, as no matter what >> we do, >> it'll be wrong, and any value will be a cargo culted hack value. >> > > I share this concern; all the various encoders I've seen have used a > different amount of priming samples, so guessing it will be bound to be wrong > in a lot of the cases. > >> * MP4 - there are two places to store this metadata - one standard, and one >> proprietary >> Apple way. There are, separately, two ways to signal priming length when >> SBR is present. >> * MP4s may contain a user data box with 'iTunSMPB' which contains priming, >> pre-roll, >> and end padding data. We support reading only priming data from this at the >> moment, >> and we set skip samples based on this. This is 'iTunes style' metadata. >> * The standards compliant (read: non-iTunes) way is to use an edit list to >> trim the >> priming samples, and, opionally end padding. End padding may also be >> trimmed by reducing >> the sample duration of the last packet in the stts box. Pre-roll is store >> in the sgpb >> box with the 'roll', type, which signals the roll distance as a number of >> packets; >> for example, -1 indicates you should decode an discard the samples of 1 >> packet before >> beginning plaback. Notably, this allows the sgpd box to also be use for >> video like >> periodic intra refresh H.264. libavformat does not current parse or export >> this info, >> but even if we did, converting number of packets to audio samples can get >> hairy. >> * Notably, since in MP4, the edit list represents the exact >> presentation-level info, >> when no edit list, or an edit list startiing at 0 is present, no samples, >> not even >> pre-roll should be trimmed - all players in the wild handle this properly, >> and it >> has been standard practice among streaming services for >10 years to not >> output >> the AAC frames representing priming samples at all (even if there is a >> small hit >> quality). This is what the patch at [0] is addressing. >> > > FWIW, MP4 isn't the only container where this might be relevant; AAC is > frequently used in muxes together with video in FLV and MKV and others as > well. > > In the case of FLV, I'm not aware of any metadata that signals how much to > trim off, so essentially we can't do it by guessing. On the producing side, > this is handled by shifting the timestamps so the audio track, which would be > starting at -, ends up starting at 0, and the video track ends up > starting at + instead. > > In that case, if we trim off the priming samples (based on a guess as that's > all we have?), I guess that'd lead us to both tracks starting at + > (i.e. not affecting sync). As long as it doesn't change sync, I guess it can > be tolerable. > > To avoid all these effects, producers of muxed files can work around this in > many ways. For many years, I've been doing the trick of skipping the first > samples of input to the audio encoder, so that after accounting for > that, I have both audio and video tracks starting at 0.0, without the decoder > needing to do anything - working the same across all players, good and bad. > > If we suddenly start decoding such files with the audio track starting at > +, I guess it'll be ok for sync, but it's a mildly surprising change, > but hopefully any reasonable player based on libavcodec would still not freak > out by it. > >> * As noted above, I don't think we should apply any guessed priming to >> initial samples (pre-roll, >> or 'algorithmic delay, included). No other decoders or players do this, in >> the world, to my >> knowledge, and violating the principal of least surpise because we think >> we're slightly more >> correct isn't great. I also think trying to 'fix' raw ADTS is destined to >> always be a hack, >> an we shouldn't. YMMV. I'd like to hear views from others here. This would >> make the patch in >> [0] redundant. >> > > Yes, with raw ADTS there's really no good way of getting this right, other > than plain guessing, and there's no single universally correct guess AFAIK.
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
Hi, Just following up on this - I'm sorry I haven't been able to look at the proposed patchset myself quite in detail yet. My prime concern is about the requests to have this merged into the upcoming 6.1 release; that's way too soon IMO. These patches do change aspects of how these things behave, that have been working the same for a very long time, so there are all sorts of potential subtle breakage, or (incorrect or not) assumptions being broken, across libavcodec and its users. On Sat, 4 Nov 2023, Derek Buitenhuis wrote: Next, a quick breakdown of the AAC situation, in terms of both how this it is stored, what we support, and the state of the ecosystem and types of files that exist: * 'Raw' ADTS streams have no way to store any of this. The best we can do is guess the pre-roll. We should not guess priming or end padding, as no matter what we do, it'll be wrong, and any value will be a cargo culted hack value. I share this concern; all the various encoders I've seen have used a different amount of priming samples, so guessing it will be bound to be wrong in a lot of the cases. * MP4 - there are two places to store this metadata - one standard, and one proprietary Apple way. There are, separately, two ways to signal priming length when SBR is present. * MP4s may contain a user data box with 'iTunSMPB' which contains priming, pre-roll, and end padding data. We support reading only priming data from this at the moment, and we set skip samples based on this. This is 'iTunes style' metadata. * The standards compliant (read: non-iTunes) way is to use an edit list to trim the priming samples, and, opionally end padding. End padding may also be trimmed by reducing the sample duration of the last packet in the stts box. Pre-roll is store in the sgpb box with the 'roll', type, which signals the roll distance as a number of packets; for example, -1 indicates you should decode an discard the samples of 1 packet before beginning plaback. Notably, this allows the sgpd box to also be use for video like periodic intra refresh H.264. libavformat does not current parse or export this info, but even if we did, converting number of packets to audio samples can get hairy. * Notably, since in MP4, the edit list represents the exact presentation-level info, when no edit list, or an edit list startiing at 0 is present, no samples, not even pre-roll should be trimmed - all players in the wild handle this properly, and it has been standard practice among streaming services for >10 years to not output the AAC frames representing priming samples at all (even if there is a small hit quality). This is what the patch at [0] is addressing. FWIW, MP4 isn't the only container where this might be relevant; AAC is frequently used in muxes together with video in FLV and MKV and others as well. In the case of FLV, I'm not aware of any metadata that signals how much to trim off, so essentially we can't do it by guessing. On the producing side, this is handled by shifting the timestamps so the audio track, which would be starting at -, ends up starting at 0, and the video track ends up starting at + instead. In that case, if we trim off the priming samples (based on a guess as that's all we have?), I guess that'd lead us to both tracks starting at + (i.e. not affecting sync). As long as it doesn't change sync, I guess it can be tolerable. To avoid all these effects, producers of muxed files can work around this in many ways. For many years, I've been doing the trick of skipping the first samples of input to the audio encoder, so that after accounting for that, I have both audio and video tracks starting at 0.0, without the decoder needing to do anything - working the same across all players, good and bad. If we suddenly start decoding such files with the audio track starting at +, I guess it'll be ok for sync, but it's a mildly surprising change, but hopefully any reasonable player based on libavcodec would still not freak out by it. * As noted above, I don't think we should apply any guessed priming to initial samples (pre-roll, or 'algorithmic delay, included). No other decoders or players do this, in the world, to my knowledge, and violating the principal of least surpise because we think we're slightly more correct isn't great. I also think trying to 'fix' raw ADTS is destined to always be a hack, an we shouldn't. YMMV. I'd like to hear views from others here. This would make the patch in [0] redundant. Yes, with raw ADTS there's really no good way of getting this right, other than plain guessing, and there's no single universally correct guess AFAIK. (And even if we have a qualified guess for the amount of encoder priming, we have even less knowledge about
Re: [FFmpeg-devel] [ANNOUNCE] upcoming GA vote
Yo, On Fri, 3 Nov 2023, at 18:33, Michael Niedermayer wrote: >> It will take a bit of time, but I'll do it. Aman Andreas Andriy Anton Lynne Baptiste Jun Zhao Lauri Kasanen Carl Marton Dale Derek Diego Gyan foo86 Gautam Ramakrishnan Nicolas hwrenx James Jeeb Jerome Borsboom Josh Kaustubh Raste Karthick Jeyapal Limin Wang Linjie Fu Steven Liu Li Zhong Martin Storsjo Martin Vignali Matthieu Bouron Michael Jacob Trimble wm4 Paul Philip Peter Ruiling Song John Stebbins Mark Thilo Timo Ting Fu Tobias Clément Vishwanath Dixit Vittorio Guo Yejun Shiyou Yin Zane Zhao > and how long will it take you ? Sorry, I got quite sick, like many people who came from Demuxed. -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] lsws/swscale.h: introduce sws_get_gaussian_vec
On date Wednesday 2023-09-06 13:13:25 +0200, Anton Khirnov wrote: > Quoting Stefano Sabatini (2023-09-06 00:59:44) [...] > A simple rename is a trivial API change. Almost everything else is not. > > > and therefore should not be committed. > > Yes, the baseline for every API change is that it is undesirable and you > must supply sufficiently strong arguments to overcome that. > > > Also there is no evidence that external components are using this > > function. > > > > Besides the naming change, there are ergonomic and functional changes > > making the behavior of the code more correct. > > I do not see the code being made more correct, but Michael observed in > this thread that it becomes longer and more convoluted. It is a different construct, not necessarily more convoluted. Also keep in mind this is needed in order to achive two goals: 1. return a proper error code since we can have two types of failures here (memalloc or invalid argument) 2. log the invalid argument reason so that the failure reason is in the log, without having to repeat the validation logic in the caller > I am not convinced that adding logging to this function is an > improvement. You have to pass an extra parameter to every call, making > the code longer and less readable. We do not need a dedicated error > message for every malloc. See above, in this case we can have ENOMEM/EINVAL and it should be possible to distinguish between the two. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/3] doc/encoders/libx264: review and extend option description
On date Wednesday 2023-09-06 11:04:49 +0530, Gyan Doshi wrote:
[...]
Sorry for the slow reply.
> > -Argument is a list of @var{key}=@var{value} couples separated by
> > -":". In @var{filter} and @var{psy-rd} options that use ":" as a separator
> > -themselves, use "," instead. They accept it as well since long ago but this
> > -is kept undocumented for some reason.
> > +@option{x264-param} is functionally the same as the @option{x264opts},
> > +but is duplicated for compatibility with the Libav fork.
>
> There is a difference, in that x264opts accepts boolean args without a
> value.
Right.
> > +
> > +The argument for both options is a list of @var{key}=@var{value}
> > +couples separated by ":". In @var{filter} and @var{psy-rd} options
> > +that use ":" as a separator themselves, use "," instead. They accept
> > +it as well since long ago but this is kept undocumented for some
> > +reason.
> > +
> > +For example, the options might be provided as:
> > +@example
> > +level=30:bframes=0:weightp=0:cabac=0:ref=1:vbv-maxrate=768:vbv-bufsize=2000:analyse=all:me=umh:no-fast-pskip=1:subq=6:8x8dct=0:trellis=0
> > +@end example
> > For example to specify libx264 encoding options with @command{ffmpeg}:
> > @example
> > ffmpeg -i foo.mpg -c:v libx264 -x264opts keyint=123:min-keyint=20 -an
> > out.mkv
> > @end example
> > +To get the complete list of the libx264 options, invoke the command
> > +@command{x264 --fullhelp} or consult the libx264 documentation.
>
> May want to make it clear that x264 refers to the standalone CLI tool.
Isn't that clear enough? (we use the same formula in other places,
e.g. in x265 docs).
>From a11eafe5a61a572f4e9ab9e4ab9907bba98300e4 Mon Sep 17 00:00:00 2001
From: Stefano Sabatini
Date: Sat, 26 Aug 2023 01:10:09 +0200
Subject: [PATCH] doc/encoders/libx264: review and extend option description
Also, merge x264opts and x264-opts option docs to avoid duplication
and make it clearer that they provide mostly the same functionality.
---
doc/encoders.texi | 65 ++-
1 file changed, 36 insertions(+), 29 deletions(-)
diff --git a/doc/encoders.texi b/doc/encoders.texi
index 8b2ab937d1..f2aba10627 100644
--- a/doc/encoders.texi
+++ b/doc/encoders.texi
@@ -2421,6 +2421,10 @@ To get a more accurate and extensive documentation of the libx264
options, invoke the command @command{x264 --fullhelp} or consult
the libx264 documentation.
+In the list below, note that the @command{x264} option name is shown
+in parentheses after the libavcodec corresponding name, in case there
+is a direct mapping.
+
@table @option
@item b (@emph{bitrate})
Set bitrate in bits/s. Note that FFmpeg's @option{b} option is
@@ -2428,17 +2432,19 @@ expressed in bits/s, while @command{x264}'s @option{bitrate} is in
kilobits/s.
@item bf (@emph{bframes})
+Number of B-frames between I and P-frames
@item g (@emph{keyint})
+Maximum GOP size
@item qmin (@emph{qpmin})
-Minimum quantizer scale.
+Minimum quantizer scale
@item qmax (@emph{qpmax})
-Maximum quantizer scale.
+Maximum quantizer scale
@item qdiff (@emph{qpstep})
-Maximum difference between quantizer scales.
+Maximum difference between quantizer scales
@item qblur (@emph{qblur})
Quantizer curve blur
@@ -2447,7 +2453,7 @@ Quantizer curve blur
Quantizer curve compression factor
@item refs (@emph{ref})
-Number of reference frames each P-frame can use. The range is from @var{0-16}.
+Number of reference frames each P-frame can use. The range is @var{0-16}.
@item level (@emph{level})
Set the @code{x264_param_t.i_level_idc} value in case the value is
@@ -2469,7 +2475,8 @@ Sets the threshold for the scene change detection.
@item trellis (@emph{trellis})
Performs Trellis quantization to increase efficiency. Enabled by default.
-@item nr (@emph{nr})
+@item nr (@emph{nr})
+Noise reduction
@item me_range (@emph{merange})
Maximum range of the motion search in pixels.
@@ -2550,6 +2557,7 @@ open GOP by setting it to @code{-cgop}. The result is similar to
the behavior of @command{x264}'s @option{--open-gop} option.
@item rc_init_occupancy (@emph{vbv-init})
+Initial VBV buffer occupancy
@item preset (@emph{preset})
Set the encoding preset.
@@ -2595,7 +2603,7 @@ Set AQ strength, reduce blocking and blurring in flat and textured areas.
Use psychovisual optimizations when set to 1. When set to 0, it has the
same effect as @command{x264}'s @option{--no-psy} option.
-@item psy-rd (@emph{psy-rd})
+@item psy-rd (@emph{psy-rd})
Set strength of psychovisual optimization, in
@var{psy-rd}:@var{psy-trellis} format.
@@ -2627,7 +2635,7 @@ to 1.
@item avcintra-class (@emph{class})
Configure the encoder to generate AVC-Intra.
-Valid values are 50,100 and 200
+Valid values are 50, 100 and 200
@item bluray-compat (@emph{bluray-compat})
Configure the encoder to be compatible with the bluray standard.
@@ -2678,8 +2686,8 @@ Set loop filter parameters, in @var{alpha}:@var{beta} form.
Set fluc
Re: [FFmpeg-devel] [PATCH v3 7/8] avfilter/vf_scale: tag output color space
On Tue, Oct 31, 2023 at 03:54:49PM +0100, Niklas Haas wrote: > From: Niklas Haas > > When using vf_scale to force a specific output color space, also tag > this on the AVFrame. (Mirroring existing logic for output range) > --- > libavfilter/vf_scale.c | 3 +++ > 1 file changed, 3 insertions(+) LGTM thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you drop bombs on a foreign country and kill a hundred thousand innocent people, expect your government to call the consequence "unprovoked inhuman terrorist attacks" and use it to justify dropping more bombs and killing more people. The technology changed, the idea is old. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v3 8/8] avcodec/pnm: explicitly tag color range
On Tue, Oct 31, 2023 at 03:54:50PM +0100, Niklas Haas wrote: > From: Niklas Haas > > PGMYUV seems to be always limited range. This was a format originally > invented by FFmpeg at a time when YUVJ distinguished limited from full > range YUV, and this codec never appeared to output YUVJ in any > circumstance, so hard-coding limited range preserves the status quo. > > The other formats are explicitly documented to be full range RGB/gray > formats. That said, don't tag them yet, due to outstanding bugs w.r.t > grayscale formats and color range handling. > > This change in behavior updates a bunch of FATE tests in trivial ways > (added tagging being the only difference). > --- > libavcodec/pnm.c | 7 -- > tests/ref/lavf/mkv| 4 ++-- > tests/ref/lavf/mkv_attachment | 4 ++-- > tests/ref/lavf/mxf| 6 ++--- > tests/ref/lavf/y4m| 4 ++-- > tests/ref/seek/lavf-mkv | 44 +-- > tests/ref/seek/lavf-y4m | 22 +- > 7 files changed, 47 insertions(+), 44 deletions(-) > > diff --git a/libavcodec/pnm.c b/libavcodec/pnm.c > index 77d24eeaf7..796807da23 100644 > --- a/libavcodec/pnm.c > +++ b/libavcodec/pnm.c [...] > @@ -240,5 +242,6 @@ int ff_pnm_decode_header(AVCodecContext *avctx, > PNMContext * const s) > h /= 3; > avctx->height = h; > } > + > return 0; > } stray change otherwise LGTM thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Old school: Use the lowest level language in which you can solve the problem conveniently. New school: Use the highest level language in which the latest supercomputer can solve the problem without the user falling asleep waiting. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
Nov 4, 2023, 17:22 by derek.buitenh...@gmail.com: > Hi, > > I'm going to opine a bit here, and also comment on the mov/MP4 patch[0] that > accompanies > this set. > > This is for both historical purposes, and to distill IRC logs into something > more > digestible for others on the list to gain context on the issue, so apologies > for > re-treading ground. > > On 10/30/2023 5:09 AM, Lynne wrote: > >> This is a convenience function, which is required to be called by decoders >> needing to skip samples every time. >> It automatically creates and increments side data. >> >> The idea is to get rid of skip_samples eventually and replace it with this >> function. >> > > So there is a lot to cover here, and lot of nuance to how things should be > handled, > that I want to list out, before discussing the specific changes of these two > sets of > patches. A bit of of a 'state of the world'. > > The goal of this patchset seems to be to aid in gapless playback and correct > seeking > in AAC streams (or later, more generally MDCT-styl audio codecs in general), > but properly > skipping initial priming samples (which include pre-roll), pre-roll (both > normal, and extra > incurred due to SBR) when seeking, and and, though not covered in these sets, > I'll mention, > end padding. > > First a note on terminology: 'Algorithmic delay' as it is being used here is > not quite > correct, for two reasons: > * Latency and pre-roll (or roll distance) are separate things. Opus, for > example, > can have a latency as low as 2.5ms, but pre-roll is always at least 80ms - > they > are different things which serve different purposes, and I confirmed this > with > people who definitely know more about audio than me[1]. Pre-roll is often > larger > than latency, and the values stored in file metadata reflect this. > Algorithmic delay is an industry standard term for those that do codec research, and is accurate. Algorithmic delay == your definition of latency. (latency is not a technically completely valid term, as latency has blurry definition outside of real-time situations, which decoding may or may not be). It is not equal to pre-roll. It is a codec-dependent value, defined by the sum of all delay that happens within each algorithm (block) of a codec. It is exactly defined to: - 1024 samples for AAC (21.3̅ milliseconds), due to the MDCT overlap process. - 120 samples for Opus (2.5 milliseconds), due to the MDCT overlap process - 320 samples for Siren (G.719, 6.6̅ milliseconds), due to the MDCT overlap process - Equal to the frame size for Vorbis, due to it having flexible blocks. - 1024 samples for ATRAC9 (21.3̅ milliseconds), due to the MDCT overlap process. - 256 samples for AC-3 (5.3̅ milliseconds), due to the MDCT overlap process. Quoting from a few official documents: RFC6716 "Definition of the Opus Audio Codec": "The inverse MDCT implementation has no special characteristics. The input is N frequency-domain samples and the output is 2*N time-domain samples, while scaling by 1/2. A "low-overlap" window reduces the algorithmic delay." ISO/IEC 14496-3, the bible itself on the very problem we're trying to solve: "To enable coding of general audio signals with an algorithmic delay not exceeding 20 ms at 48 kHz, it uses a frame length of 512 or 480 samples (compared to the 1024 or 960 samples used in standard MPEG-2/4 AAC)." G.719: "The codec operates on 20 ms frames, and the algorithmic delay end-to-end is 40 ms. The encoder input and decoder output are sampled at 48 kHz." (this one gives the algorithmic delay as the sum of both the decoder and the encoder's delays during real-time operation). > * Pre-roll, or roll distance, are the industry standard terms. Making up out > own > terms because we disagree is silly and stubborn, and makes it harder on API > users trying to use the API correctly, or understnd our code. > You call me silly and stubborn for trying to use a standard term for a mathematically defined value upon which to base other values (your definition of pre-roll, which contains the algorithmic delay), so that there is no disagreement in what we talk about. I defined the algorithmic delay earlier in our discussion and outlined the reason to do so. And disagreement in what we talk about is exactly what happened. With that out of the way, I will not stop calling what I have just defined as algorithmic delay. Nor will I want an explanation on why after saying algorithmic delay so many times, you keep reading pre-roll, and you keep trying to interpret every value as pre-roll, even in the chat log you posted. Now, let me begin by defining that the amount of samples you have to necessarily strip from any MDCT-based audio codec at the start is exactly equal to its algorithmic delay. **Including Opus**. The output audio signal will be timing-accurate compared to the input signal, as long as the encoder did not add any extra samples. Continuing, let me define the exact
Re: [FFmpeg-devel] [PATCH] avfilter/buffersrc: switch to activate
On Sat, Nov 4, 2023 at 8:07 PM Nicolas George wrote: > Paul B Mahol (12023-11-04): > > From 31a6a78ebc3a3f8785ec7c8e5ffd4257c7eadec3 Mon Sep 17 00:00:00 2001 > > From: Paul B Mahol > > Date: Fri, 27 Oct 2023 14:26:50 +0200 > > Subject: [PATCH] avfilter/buffersrc: switch to activate > > > > Fixes error when caller keeps adding frames into filtergraph > > that reached EOF by other means, for example EOF is signalled > > by other filter in filtergraph or by buffersink. > > Filters with no more than one input and one output do not need to > implement activate. If it seems they need, that indicates a bug in the > framework. Then that bug needs to be found and fixed. This patch just > adds code to work around it, it is not acceptable. > Sorry to hear your deep feelings and failed realizations for this topic and in FFmpeg in general. Certainly you are not, never was, and never will be central absolute authority to say what is acceptable and what is not to do in FFmpeg code. Now on technical side of story: buffersrc can be closed by calling av_buffersrc_close(). And currently in master that is the only way how buffersrc can end processing of sending more frames to filter chain. There is possibility for filters in filtergraph to signal EOF back down (backward) to buffersrc filter in filtergraph chain, but that same thing is ignored by buffersrc filter. This patch address this issue/bug. Now why switch to .activate() is proper solution for buffersrc filter? buffersrc filter uses request_frame(), and that one is used to count number of failed requests when no frame is available for next filters in filtergraph. Inside request_frame() one can check only for EOF from inlinks, but as buffersrc filter is source filter it have no inlinks. Other function filter_frame() is not applicable to source filter types. Also you forgot fact that buffersrc filter have 0 inputs and 1 output, so to correct your claim, following holds: Any filter with only one input and only one output of same media type is not forced to use .activate(). Above may not hold if such 1->1 filter does modify timeline of frames. (Drops/inserts frames/samples, like obvious example is fps filter) Similar applies to buffersink filter, with 1 input and 0 outputs, by your standards such filter do not need .activate(), which is obviously false. For more insight into this subject and groundbreaking theory of filtering, please do consult: 912969a33e313c57c906e87a7e2367b78a2160f4 lavfi/buffersink: move to the new design. Note: the OOM happens even without split filter usage, which means its not fault of .activate() in split filter code. > > -- > Nicolas George > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 10/24] fftools/ffmpeg_filter: move filtering to a separate thread
On Sat, Nov 04, 2023 at 08:56:19AM +0100, Anton Khirnov wrote: > As previously for decoding, this is merely "scaffolding" for moving to a > fully threaded architecture and does not yet make filtering truly > parallel - the main thread will currently wait for the filtering thread > to finish its work before continuing. That will change in future commits > after encoders are also moved to threads and a thread-aware scheduler is > added. > --- > fftools/ffmpeg.h| 9 +- > fftools/ffmpeg_dec.c| 39 +- > fftools/ffmpeg_filter.c | 825 ++-- > 3 files changed, 730 insertions(+), 143 deletions(-) causes assertion failure in: make -j32 && echo 'Call 0 ping' | ./ffmpeg -nostats -i ~/videos/mm-short.mpg -vf nullsink,color=green -bitexact -vframes 1 -f null - Assertion stream_idx < tq->nb_streams failed at fftools/thread_queue.c:125 thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB He who knows, does not speak. He who speaks, does not know. -- Lao Tsu signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter/buffersrc: switch to activate
Paul B Mahol (12023-11-04): > From 31a6a78ebc3a3f8785ec7c8e5ffd4257c7eadec3 Mon Sep 17 00:00:00 2001 > From: Paul B Mahol > Date: Fri, 27 Oct 2023 14:26:50 +0200 > Subject: [PATCH] avfilter/buffersrc: switch to activate > > Fixes error when caller keeps adding frames into filtergraph > that reached EOF by other means, for example EOF is signalled > by other filter in filtergraph or by buffersink. Filters with no more than one input and one output do not need to implement activate. If it seems they need, that indicates a bug in the framework. Then that bug needs to be found and fixed. This patch just adds code to work around it, it is not acceptable. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfillter/buffersrc: activate and EOF fix
Paul B Mahol (12023-11-03): > Also I think that forward and/or backward EOF direction status checking is > not correctly handled at all for any filters not using .activate(), and I'm > not aware that it was ever working correctly in all cases. Could you not start with that?!? If you are right, that means there is a bug in the framework code. Then the patch forward is to fix that bug, not needlessly single every simple filter to activate. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avfilter/buffersrc: switch to activate
Attached.
From 31a6a78ebc3a3f8785ec7c8e5ffd4257c7eadec3 Mon Sep 17 00:00:00 2001
From: Paul B Mahol
Date: Fri, 27 Oct 2023 14:26:50 +0200
Subject: [PATCH] avfilter/buffersrc: switch to activate
Fixes error when caller keeps adding frames into filtergraph
that reached EOF by other means, for example EOF is signalled
by other filter in filtergraph or by buffersink.
Signed-off-by: Paul B Mahol
---
libavfilter/buffersrc.c | 27 +--
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index 453fc0fd5c..b0a905d455 100644
--- a/libavfilter/buffersrc.c
+++ b/libavfilter/buffersrc.c
@@ -36,6 +36,7 @@
#include "audio.h"
#include "avfilter.h"
#include "buffersrc.h"
+#include "filters.h"
#include "formats.h"
#include "internal.h"
#include "video.h"
@@ -194,7 +195,7 @@ FF_ENABLE_DEPRECATION_WARNINGS
if (!frame)
return av_buffersrc_close(ctx, s->last_pts, flags);
if (s->eof)
-return AVERROR(EINVAL);
+return AVERROR_EOF;
s->last_pts = frame->pts + frame->duration;
@@ -484,21 +485,28 @@ static int config_props(AVFilterLink *link)
return 0;
}
-static int request_frame(AVFilterLink *link)
+static int activate(AVFilterContext *ctx)
{
-BufferSourceContext *c = link->src->priv;
+AVFilterLink *outlink = ctx->outputs[0];
+BufferSourceContext *c = ctx->priv;
-if (c->eof)
-return AVERROR_EOF;
+if (!c->eof && ff_outlink_get_status(outlink)) {
+c->eof = 1;
+return 0;
+}
+
+if (c->eof) {
+ff_outlink_set_status(outlink, AVERROR_EOF, c->last_pts);
+return 0;
+}
c->nb_failed_requests++;
-return AVERROR(EAGAIN);
+return FFERROR_NOT_READY;
}
static const AVFilterPad avfilter_vsrc_buffer_outputs[] = {
{
.name = "default",
.type = AVMEDIA_TYPE_VIDEO,
-.request_frame = request_frame,
.config_props = config_props,
},
};
@@ -507,7 +515,7 @@ const AVFilter ff_vsrc_buffer = {
.name = "buffer",
.description = NULL_IF_CONFIG_SMALL("Buffer video frames, and make them accessible to the filterchain."),
.priv_size = sizeof(BufferSourceContext),
-
+.activate = activate,
.init = init_video,
.uninit= uninit,
@@ -521,7 +529,6 @@ static const AVFilterPad avfilter_asrc_abuffer_outputs[] = {
{
.name = "default",
.type = AVMEDIA_TYPE_AUDIO,
-.request_frame = request_frame,
.config_props = config_props,
},
};
@@ -530,7 +537,7 @@ const AVFilter ff_asrc_abuffer = {
.name = "abuffer",
.description = NULL_IF_CONFIG_SMALL("Buffer audio frames, and make them accessible to the filterchain."),
.priv_size = sizeof(BufferSourceContext),
-
+.activate = activate,
.init = init_audio,
.uninit= uninit,
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 18/24] fftools/ffmpeg: add thread-aware transcode scheduling infrastructure
On Sat, Nov 04, 2023 at 08:56:27AM +0100, Anton Khirnov wrote: > See the comment block at the top of fftools/ffmpeg_sched.h for more > details on what this scheduler is for. > > This commit adds the scheduling code itself, along with minimal > integration with the rest of the program: > * allocating and freeing the scheduler > * passing it throughout the call stack in order to register the > individual components (demuxers/decoders/filtergraphs/encoders/muxers) > with the scheduler > > The scheduler is not actually used as of this commit, so it should not > result in any change in behavior. That will change in future commits. > --- > fftools/Makefile |1 + > fftools/ffmpeg.c | 18 +- > fftools/ffmpeg.h | 24 +- > fftools/ffmpeg_dec.c | 10 +- > fftools/ffmpeg_demux.c| 46 +- > fftools/ffmpeg_enc.c | 13 +- > fftools/ffmpeg_filter.c | 37 +- > fftools/ffmpeg_mux.c | 17 +- > fftools/ffmpeg_mux.h | 11 + > fftools/ffmpeg_mux_init.c | 82 +- > fftools/ffmpeg_opt.c | 22 +- > fftools/ffmpeg_sched.c| 2072 + > fftools/ffmpeg_sched.h| 461 + > 13 files changed, 2758 insertions(+), 56 deletions(-) > create mode 100644 fftools/ffmpeg_sched.c > create mode 100644 fftools/ffmpeg_sched.h Is fate intended to pass for each point in the patchset ? make -j32 fate-h264-skip-nointra seems to fail after this [h264 @ 0x5598133bccc0] non-existing PPS 0 referenced Last message repeated 5 times [h264 @ 0x5598133bccc0] decode_slice_header error [h264 @ 0x5598133bccc0] non-existing PPS 0 referenced Last message repeated 1 times [h264 @ 0x5598133bccc0] decode_slice_header error [h264 @ 0x5598133bccc0] non-existing PPS 0 referenced Last message repeated 1 times [h264 @ 0x5598133bccc0] decode_slice_header error [h264 @ 0x5598133bccc0] non-existing PPS 0 referenced Last message repeated 1 times [h264 @ 0x5598133bccc0] decode_slice_header error [h264 @ 0x5598133bccc0] non-existing PPS 0 referenced Last message repeated 3 times [h264 @ 0x5598133bccc0] mmco: unref short failure Last message repeated 1 times [h264 @ 0x5598133bccc0] number of reference frames (0+4) exceeds max (3; probably corrupt input), discarding one [mpegts @ 0x5598133b7080] PES packet size mismatch [mpegts @ 0x5598133b7080] Packet corrupt (stream = 2, dts = 212088417). [mpegts @ 0x5598133b7080] PES packet size mismatch [mpegts @ 0x5598133b7080] Packet corrupt (stream = 1, dts = 212089857). [mpegts @ 0x5598133b7080] PES packet size mismatch [mpegts @ 0x5598133b7080] Packet corrupt (stream = 2, dts = 212088417). [mpegts @ 0x5598133b7080] PES packet size mismatch [mpegts @ 0x5598133b7080] Packet corrupt (stream = 1, dts = 212089857). Input #0, mpegts, from 'fatesamples/fate/fate-suite//h264/h264_intra_first-small.ts': Duration: 00:00:02.48, start: 2355.577967, bitrate: 4961 kb/s Program 41 Stream #0:0[0x12b5]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn Stream #0:1[0x12b4](swe): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 224 kb/s Stream #0:2[0x12b2](nor): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 224 kb/s Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native)) Stream #0:1 -> #0:1 (mp2 (native) -> pcm_s16le (native)) [aost#0:1/pcm_s16le @ 0x5598134536c0] Too many packets buffered for output stream 0:1. [aost#0:1/pcm_s16le @ 0x5598134536c0] Error submitting a packet to the muxer: No space left on deviceError while filtering: No space left on device [out#0/framecrc @ 0x5598133bedc0] Nothing was written into output file, because at least one of its streams received no packets. frame=0 fps=0.0 q=0.0 Lsize= 0kB time=00:00:00.19 bitrate= 0.0kbits/s speed=68.4x Conversion failed! threads=1 tests/Makefile:307: recipe for target 'fate-h264-skip-nointra' failed make: *** [fate-h264-skip-nointra] Error 228 [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Dictatorship: All citizens are under surveillance, all their steps and actions recorded, for the politicians to enforce control. Democracy: All politicians are under surveillance, all their steps and actions recorded, for the citizens to enforce control. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 20/24] fftools/ffmpeg_dec: convert to the scheduler
On Sat, Nov 04, 2023 at 08:56:29AM +0100, Anton Khirnov wrote: > --- > fftools/ffmpeg.c | 22 --- > fftools/ffmpeg.h | 13 +- > fftools/ffmpeg_dec.c | 315 ++- > 3 files changed, 70 insertions(+), 280 deletions(-) This or the previous commit (which doesnt build without this) breaks: ffmpeg -y -i tickets//1714/fake_mjpeg_stream.wmv -t 1 /tmp/file1714.avi (infinite loop) it can be quit with a "q" press here but with all patches applied it seems "q" is not enough to kill it seems on the samples server here: ffmpeg-bugs/trac/ticket1714/fake_mjpeg_stream.wmv thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If a bugfix only changes things apparently unrelated to the bug with no further explanation, that is a good sign that the bugfix is wrong. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [ANNOUNCE] upcoming GA vote
A reminder to everyone who has not voted yet that the vote will close tomorrow around midnight UTC. Please vote as soon as possible. -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [ANNOUNCE] upcoming GA vote
Quoting Michael Niedermayer (2023-11-02 20:53:46) > On Sun, Oct 29, 2023 at 10:33:12AM +0100, Anton Khirnov wrote: > > Hi all, > > updating the answers according to received comments: > > > > vote question (unchanged): > > How do we update the list of active members of the general assembly? > > > > Available answers: > > * twice a year (1st Jan & 1st July, 0:00 UTC); > > as an exception, the list will also be updated immediately after this > > vote > > (suggested at VDD, added time and the exceptional update clause) > > > * before each vote > > (suggested at VDD) > > This is not well defined > https://en.wikipedia.org/wiki/Well-defined_expression > > "before each vote" is not a single unique point in time > > First what is meant by "vote", theres a annoucement of some > intend to make a vote, there would be a discussion, there could > be a 2nd annoucement theres a period where people can propose > new options. Then the vote start is announced at some point > the actual vote start then also happens. > > So we have many reference points > then "before" is everything before, even a year is "before" > > and exactly at the time when the vote actually starts is impossible > because at that time the list must already be known and entered > into the vote system > > I guess this doesnt matter if the option doesnt win ... It seems obvious to me that the intent is "generate the list immediately before the starting the vote". Since the script generating the voter list has a granularity of one day, it does not matter at what precise instant is the list generated, as long as it is on the same day as starting the vote. -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
On Sat, Nov 04, 2023 at 04:22:11PM +, Derek Buitenhuis wrote: [...] > I am almost certain I missed even more nuance, and hopefully Martin or Anton > can chime in, or I remember more. > > But also, given all of this, I think we need to deeply consider how we > approach this, so we don't end up with > something that only covers certain cases (and I am sure I forgot more cases). > To that end, I do not think rushing > to get a patchset that can change sync on all AAC files in existence into 6.1 > is wise. Even when this does go in, > it should be able to sit in master for a good long time before being in a > release. As I understand it, FATE is > already unhappy, and it shouldn't be treated as being a problem with FATE vs > the set. > > Lastly, some time this weekend/week, I will labour to create a more extensive > set of AAC files we can use to > test, and use in FATE. I think thats a very good idea. Ideally we would have a set of test samples encoded with all major encoders samples where start, end and sync can be easily determined unambigously once we have such a set, it becomes easy to check that we get the start/end/sync right for every file by default. I think the guiding principle should be that what comes out of the decoder is as close to what went into the encoder. And of course to comply to specifications > > Hope that all made sense and I didn't forget any details in my Covid-induced > haze. i hope you get better soon! thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Good people do not need laws to tell them to act responsibly, while bad people will find a way around the laws. -- Plato signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
On 11/4/2023 4:22 PM, Derek Buitenhuis wrote: > the sample duration of the last packet in the stts box. Pre-roll is > store in the sgpb Bah. Please try and ignore my various typos of 'sgpd'. - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
Hi, I'm going to opine a bit here, and also comment on the mov/MP4 patch[0] that accompanies this set. This is for both historical purposes, and to distill IRC logs into something more digestible for others on the list to gain context on the issue, so apologies for re-treading ground. On 10/30/2023 5:09 AM, Lynne wrote: > This is a convenience function, which is required to be called by decoders > needing to skip samples every time. > It automatically creates and increments side data. > > The idea is to get rid of skip_samples eventually and replace it with this > function. So there is a lot to cover here, and lot of nuance to how things should be handled, that I want to list out, before discussing the specific changes of these two sets of patches. A bit of of a 'state of the world'. The goal of this patchset seems to be to aid in gapless playback and correct seeking in AAC streams (or later, more generally MDCT-styl audio codecs in general), but properly skipping initial priming samples (which include pre-roll), pre-roll (both normal, and extra incurred due to SBR) when seeking, and and, though not covered in these sets, I'll mention, end padding. First a note on terminology: 'Algorithmic delay' as it is being used here is not quite correct, for two reasons: * Latency and pre-roll (or roll distance) are separate things. Opus, for example, can have a latency as low as 2.5ms, but pre-roll is always at least 80ms - they are different things which serve different purposes, and I confirmed this with people who definitely know more about audio than me[1]. Pre-roll is often larger than latency, and the values stored in file metadata reflect this. * Pre-roll, or roll distance, are the industry standard terms. Making up out own terms because we disagree is silly and stubborn, and makes it harder on API users trying to use the API correctly, or understnd our code. Next, a quick breakdown of the AAC situation, in terms of both how this it is stored, what we support, and the state of the ecosystem and types of files that exist: * 'Raw' ADTS streams have no way to store any of this. The best we can do is guess the pre-roll. We should not guess priming or end padding, as no matter what we do, it'll be wrong, and any value will be a cargo culted hack value. * MP4 - there are two places to store this metadata - one standard, and one proprietary Apple way. There are, separately, two ways to signal priming length when SBR is present. * MP4s may contain a user data box with 'iTunSMPB' which contains priming, pre-roll, and end padding data. We support reading only priming data from this at the moment, and we set skip samples based on this. This is 'iTunes style' metadata. * The standards compliant (read: non-iTunes) way is to use an edit list to trim the priming samples, and, opionally end padding. End padding may also be trimmed by reducing the sample duration of the last packet in the stts box. Pre-roll is store in the sgpb box with the 'roll', type, which signals the roll distance as a number of packets; for example, -1 indicates you should decode an discard the samples of 1 packet before beginning plaback. Notably, this allows the sgpd box to also be use for video like periodic intra refresh H.264. libavformat does not current parse or export this info, but even if we did, converting number of packets to audio samples can get hairy. * Notably, since in MP4, the edit list represents the exact presentation-level info, when no edit list, or an edit list startiing at 0 is present, no samples, not even pre-roll should be trimmed - all players in the wild handle this properly, and it has been standard practice among streaming services for >10 years to not output the AAC frames representing priming samples at all (even if there is a small hit quality). This is what the patch at [0] is addressing. * My personal opinion is that since priming samples include any inherent delay already, that if we do not know how many priming samples there are, we should not trim anything from the start of the file, regardless of format. I am keen on hearing others Opinions(TM) here, particularily Anton and Martin (sorry for name dropping :)). * Further complicating matters is the fact that, again thanks to Apple, there are a lot of broken files around, since iTunes expects files to *not* include addition delay incurred by SBR in their edit list / priming info, even though, by spec, you are suppose to. This leads to the unfortunate case where you have tons of files in the wild that both do, and do not include SBR delay in their edit lists, and there is n
Re: [FFmpeg-devel] [PATCH 08/24] fftools/ffmpeg_filter: remove an unnecessary sub2video_push_ref() call
Anton Khirnov (12023-11-04): > It only seems to produce duplicate frames. Yes it does, that is the point of it. Without the duplicated frame, a filter with synchronized inputs (like overlay) will accumulate video frames while waiting for the next sub2video frame. It is actually super easy to test: any file with both video and bitmap subtitles will cause a huge memory consumption if there is a long enough time without subtitles. Since I am feeling super helpful, here is how I just tested: ./ffmpeg_g -lavfi testsrc2=s=720x480:d=150 -preset ultrafast -y /tmp/dummy.mkv mkvmerge -o /tmp/dummy_with_sub.mkv /tmp/dummy.mkv $ffmpeg_fate/sub/vobsub.idx limit addressspace 2G ./ffmpeg_g -xerror -i /tmp/dummy_with_sub.mkv -preset ultrafast -lavfi '[0:v][0:s]overlay' -y /tmp/dummy_with_hardsub.mkv mplayer /tmp/dummy_with_hardsub.mkv -ss 2:05 The limit addressspace 2G is there so that the huge memory consumption will hit something. Another way is to log the number of queued frames: --- a/libavfilter/framequeue.c +++ b/libavfilter/framequeue.c @@ -90,2 +90,3 @@ int ff_framequeue_add(FFFrameQueue *fq, AVFrame *frame) fq->queued++; +av_log(0, 16, "queued = %zd\n", fq->queued); fq->total_frames_head++; ffmpeg version N-112710-g86e0dea620 Copyright (c) 2000-2023 the FFmpeg developers built with gcc 13 (Debian 13.2.0-5) configuration: --enable-shared --disable-static --enable-gpl --enable-libx264 --enable-libopus --enable-libass --enable-libfreetype --enable-opengl --assert-level=2 libavutil 58. 31.100 / 58. 31.100 libavcodec 60. 32.102 / 60. 32.102 libavformat60. 17.100 / 60. 17.100 libavdevice60. 4.100 / 60. 4.100 libavfilter 9. 13.100 / 9. 13.100 libswscale 7. 6.100 / 7. 6.100 libswresample 4. 13.100 / 4. 13.100 libpostproc57. 4.100 / 57. 4.100 Input #0, matroska,webm, from '/tmp/dummy_with_sub.mkv': […] queued = 2210 queued = 2211 queued = 2212 queued = 2213 [h264 @ 0x562874ac4f40] get_buffer() failed [h264 @ 0x562874ac4f40] thread_get_buffer() failed [h264 @ 0x562874ac4f40] decode_slice_header error [h264 @ 0x562874ac4f40] no frame! […] -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 18/24] fftools/ffmpeg: add thread-aware transcode scheduling infrastructure
On 11/4/2023 4:56 AM, Anton Khirnov wrote:
+static void *task_wrapper(void *arg)
+{
+SchTask *task = arg;
+Scheduler *sch = task->parent;
+int ret;
+int err = 0;
+
+ret = (intptr_t)task->func(task->func_arg);
+if (ret < 0)
+av_log(task->func_arg, AV_LOG_ERROR,
+ "Task finished with error code: %d (%s)\n", ret,
av_err2str(ret));
+
+switch (task->node.type) {
+case SCH_NODE_TYPE_DEMUX: err = demux_done (sch, task->node.idx);
break;
+case SCH_NODE_TYPE_MUX: err = mux_done (sch, task->node.idx);
break;
+case SCH_NODE_TYPE_DEC: err = dec_done (sch, task->node.idx);
break;
+case SCH_NODE_TYPE_ENC: err = enc_done (sch, task->node.idx);
break;
+case SCH_NODE_TYPE_FILTER_IN: err = filter_done(sch, task->node.idx);
break;
task->node.type seems to be constant, so wouldn't it be faster, or at
least cleaner looking here, to use a function pointer assigned in
task_init()?
+default: av_assert0(0);
+}
+
+ret = err_merge(ret, err);
+
+// EOF is considered normal termination
+if (ret == AVERROR_EOF)
+ret = 0;
+if (ret < 0)
+atomic_store(&sch->task_failed, 1);
+
+av_log(task->func_arg, ret < 0 ? AV_LOG_ERROR : AV_LOG_VERBOSE,
+ "Terminating thread with return code %d (%s)\n", ret,
+ ret < 0 ? av_err2str(ret) : "success");
+
+return (void*)(intptr_t)ret;
+}
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 14/24] fftools/ffmpeg_mux: move bitstream filtering to the muxer thread
On 11/4/2023 4:56 AM, Anton Khirnov wrote:
This will be the appropriate place for it after the rest of transcoding
is switched to a threaded architecture.
---
fftools/ffmpeg_mux.c | 112 ++-
1 file changed, 67 insertions(+), 45 deletions(-)
diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c
index 82352b7981..57fb8a8413 100644
--- a/fftools/ffmpeg_mux.c
+++ b/fftools/ffmpeg_mux.c
@@ -207,6 +207,67 @@ static int sync_queue_process(Muxer *mux, OutputStream
*ost, AVPacket *pkt, int
return 0;
}
+/* apply the output bitstream filters */
+static int mux_packet_filter(Muxer *mux, OutputStream *ost,
+ AVPacket *pkt, int *stream_eof)
+{
+MuxStream *ms = ms_from_ost(ost);
+const char *err_msg;
+int ret = 0;
+
+if (ms->bsf_ctx) {
+int bsf_eof = 0;
+
+if (pkt)
+av_packet_rescale_ts(pkt, pkt->time_base,
ms->bsf_ctx->time_base_in);
+
+ret = av_bsf_send_packet(ms->bsf_ctx, pkt);
+if (ret < 0) {
Unrelated to this patch, but this should probably include a comment
about the reason we're not checking for EAGAIN, like we do for
avcodec_send_packet().
I'll send a patch for that after this is pushed, so you don't have to
solve conflicts after a rebase.
+err_msg = "submitting a packet for bitstream filtering";
+goto fail;
+}
+
+while (!bsf_eof) {
+ret = av_bsf_receive_packet(ms->bsf_ctx, ms->bsf_pkt);
+if (ret == AVERROR(EAGAIN))
+return 0;
+else if (ret == AVERROR_EOF)
+bsf_eof = 1;
+else if (ret < 0) {
+av_log(ost, AV_LOG_ERROR,
+ "Error applying bitstream filters to a packet: %s",
+ av_err2str(ret));
+if (exit_on_error)
+return ret;
+continue;
+}
+
+if (!bsf_eof)
+ms->bsf_pkt->time_base = ms->bsf_ctx->time_base_out;
+
+ret = sync_queue_process(mux, ost, bsf_eof ? NULL : ms->bsf_pkt,
stream_eof);
+if (ret < 0)
+goto mux_fail;
+}
+*stream_eof = 1;
+return AVERROR_EOF;
+} else {
+ret = sync_queue_process(mux, ost, pkt, stream_eof);
+if (ret < 0)
+goto mux_fail;
+}
+
+return 0;
+
+mux_fail:
+err_msg = "submitting a packet to the muxer";
+
+fail:
+if (ret != AVERROR_EOF)
+av_log(ost, AV_LOG_ERROR, "Error %s: %s\n", err_msg, av_err2str(ret));
+return ret;
+}
+
static void thread_set_name(OutputFile *of)
{
char name[16];
@@ -263,7 +324,7 @@ static void *muxer_thread(void *arg)
}
ost = of->streams[stream_idx];
-ret = sync_queue_process(mux, ost, ret < 0 ? NULL : mt.pkt,
&stream_eof);
+ret = mux_packet_filter(mux, ost, ret < 0 ? NULL : mt.pkt,
&stream_eof);
av_packet_unref(mt.pkt);
if (ret == AVERROR_EOF) {
if (stream_eof) {
@@ -376,58 +437,19 @@ static int submit_packet(Muxer *mux, AVPacket *pkt,
OutputStream *ost)
int of_output_packet(OutputFile *of, OutputStream *ost, AVPacket *pkt)
{
Muxer *mux = mux_from_of(of);
-MuxStream *ms = ms_from_ost(ost);
-const char *err_msg;
int ret = 0;
if (pkt && pkt->dts != AV_NOPTS_VALUE)
ost->last_mux_dts = av_rescale_q(pkt->dts, pkt->time_base,
AV_TIME_BASE_Q);
-/* apply the output bitstream filters */
-if (ms->bsf_ctx) {
-int bsf_eof = 0;
-
-if (pkt)
-av_packet_rescale_ts(pkt, pkt->time_base,
ms->bsf_ctx->time_base_in);
-
-ret = av_bsf_send_packet(ms->bsf_ctx, pkt);
-if (ret < 0) {
-err_msg = "submitting a packet for bitstream filtering";
-goto fail;
-}
-
-while (!bsf_eof) {
-ret = av_bsf_receive_packet(ms->bsf_ctx, ms->bsf_pkt);
-if (ret == AVERROR(EAGAIN))
-return 0;
-else if (ret == AVERROR_EOF)
-bsf_eof = 1;
-else if (ret < 0) {
-err_msg = "applying bitstream filters to a packet";
-goto fail;
-}
-
-if (!bsf_eof)
-ms->bsf_pkt->time_base = ms->bsf_ctx->time_base_out;
-
-ret = submit_packet(mux, bsf_eof ? NULL : ms->bsf_pkt, ost);
-if (ret < 0)
-goto mux_fail;
-}
-} else {
-ret = submit_packet(mux, pkt, ost);
-if (ret < 0)
-goto mux_fail;
+ret = submit_packet(mux, pkt, ost);
+if (ret < 0) {
+av_log(ost, AV_LOG_ERROR, "Error submitting a packet to the muxer: %s",
+ av_err2str(ret));
+return ret;
}
return 0;
-
-mux_fail:
-err_msg = "submitting a packet to the muxer";
-
-fail:
-av_log(ost, AV_LOG_ERROR, "Error %s\n", err_msg);
-retu
Re: [FFmpeg-devel] [PATCH 1/2] decode: add ff_decode_skip_samples function
Quoting Lynne (2023-10-30 06:09:28) > This is a convenience function, which is required to be called by decoders > needing to skip samples every time. > It automatically creates and increments side data. > > The idea is to get rid of skip_samples eventually and replace it with this > function. > > Patch attached. > > > From 41dfcbbacfa9232d2308d0229dcd172309b32f9f Mon Sep 17 00:00:00 2001 > From: Lynne > Date: Mon, 30 Oct 2023 05:38:17 +0100 > Subject: [PATCH 1/2] decode: add ff_decode_skip_samples function > > This is a convenience function, which is required to be called by decoders > needing to skip samples every time. > It automatically creates and increments side data. > > The idea is to get rid of skip_samples eventually and replace it with this > function. > --- > libavcodec/decode.c | 18 ++ > libavcodec/decode.h | 9 + > 2 files changed, 27 insertions(+) > > diff --git a/libavcodec/decode.c b/libavcodec/decode.c > index ad39021354..f971723ff7 100644 > --- a/libavcodec/decode.c > +++ b/libavcodec/decode.c > @@ -299,6 +299,24 @@ static int64_t guess_correct_pts(AVCodecContext *ctx, > return pts; > } > > +int ff_decode_skip_samples(AVCodecContext *avctx, AVFrame *frame, uint32_t > base_skip, uint32_t additional) avctx seems unused. -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 21/24] fftools/ffmpeg_filter: convert to the scheduler
---
fftools/ffmpeg.c| 44 +--
fftools/ffmpeg.h| 32 +-
fftools/ffmpeg_filter.c | 720 +++-
3 files changed, 204 insertions(+), 592 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index bd783fe674..1f21008588 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -138,30 +138,6 @@ static struct termios oldtty;
static int restore_tty;
#endif
-/* sub2video hack:
- Convert subtitles to video with alpha to insert them in filter graphs.
- This is a temporary solution until libavfilter gets real subtitles support.
- */
-
-static void sub2video_heartbeat(InputFile *infile, int64_t pts, AVRational tb)
-{
-/* When a frame is read from a file, examine all sub2video streams in
- the same file and send the sub2video frame again. Otherwise, decoded
- video frames could be accumulating in the filter graph while a filter
- (possibly overlay) is desperately waiting for a subtitle frame. */
-for (int i = 0; i < infile->nb_streams; i++) {
-InputStream *ist = infile->streams[i];
-
-if (ist->dec_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE)
-continue;
-
-for (int j = 0; j < ist->nb_filters; j++)
-ifilter_sub2video_heartbeat(ist->filters[j], pts, tb);
-}
-}
-
-/* end of sub2video hack */
-
static void term_exit_sigsafe(void)
{
#if HAVE_TERMIOS_H
@@ -552,8 +528,8 @@ static void print_report(int is_last_report, int64_t
timer_start, int64_t cur_ti
if (is_last_report)
av_bprintf(&buf, "L");
-nb_frames_dup = ost->filter->nb_frames_dup;
-nb_frames_drop = ost->filter->nb_frames_drop;
+nb_frames_dup = atomic_load(&ost->filter->nb_frames_dup);
+nb_frames_drop = atomic_load(&ost->filter->nb_frames_drop);
vid = 1;
}
@@ -890,9 +866,7 @@ static int choose_output(OutputStream **post)
for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) {
int64_t opts;
-if (ost->filter && ost->filter->last_pts != AV_NOPTS_VALUE) {
-opts = ost->filter->last_pts;
-} else {
+{
opts = ost->last_mux_dts == AV_NOPTS_VALUE ?
INT64_MIN : ost->last_mux_dts;
}
@@ -1041,8 +1015,6 @@ static int process_input(int file_index, AVPacket *pkt)
ist = ifile->streams[pkt->stream_index];
-sub2video_heartbeat(ifile, pkt->pts, pkt->time_base);
-
ret = process_input_packet(ist, pkt, 0);
av_packet_unref(pkt);
@@ -1061,8 +1033,6 @@ static int transcode_step(OutputStream *ost, AVPacket
*demux_pkt)
int ret;
if (ost->filter) {
-if ((ret = fg_transcode_step(ost->filter->graph, &ist)) < 0)
-return ret;
if (!ist)
return 0;
} else {
@@ -1078,14 +1048,6 @@ static int transcode_step(OutputStream *ost, AVPacket
*demux_pkt)
if (ret < 0)
return ret == AVERROR_EOF ? 0 : ret;
-// process_input() above might have caused output to become available
-// in multiple filtergraphs, so we process all of them
-for (int i = 0; i < nb_filtergraphs; i++) {
-ret = reap_filters(filtergraphs[i], 0);
-if (ret < 0)
-return ret;
-}
-
return 0;
}
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 975d8b737e..c1b61c83e7 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -84,9 +84,7 @@ enum HWAccelID {
};
enum FrameOpaque {
-FRAME_OPAQUE_REAP_FILTERS = 1,
-FRAME_OPAQUE_CHOOSE_INPUT,
-FRAME_OPAQUE_SUB_HEARTBEAT,
+FRAME_OPAQUE_SUB_HEARTBEAT = 1,
FRAME_OPAQUE_EOF,
FRAME_OPAQUE_SEND_COMMAND,
};
@@ -313,11 +311,8 @@ typedef struct OutputFilter {
enum AVMediaType type;
-/* pts of the last frame received from this filter, in AV_TIME_BASE_Q */
-int64_t last_pts;
-
-uint64_t nb_frames_dup;
-uint64_t nb_frames_drop;
+atomic_uint_least64_t nb_frames_dup;
+atomic_uint_least64_t nb_frames_drop;
} OutputFilter;
typedef struct FilterGraph {
@@ -728,10 +723,6 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle
*subtitle, int copy);
*/
FrameData *frame_data(AVFrame *frame);
-int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int
keep_reference);
-int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb);
-void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational
tb);
-
/**
* Set up fallback filtering parameters from a decoder context. They will only
* be used if no frames are ever sent on this input, otherwise the actual
@@ -752,26 +743,9 @@ int fg_create(FilterGraph **pfg, char *graph_desc,
Scheduler *sch);
void fg_free(FilterGraph **pfg);
-/**
- * Perform a step of transcoding for the specified filter graph.
- *
- * @param[in] graph filter graph to consider
- * @param[out] best_ist input stream where a frame would allow to continue
- * @return 0 for success, <0 for err
[FFmpeg-devel] [PATCH 13/24] fftools/ffmpeg_mux: add muxing thread private data
To be used for data that never needs to be visible outside of the muxer
thread. Start by moving the muxed AVPacket in there.
---
fftools/ffmpeg_mux.c | 44 +++-
1 file changed, 35 insertions(+), 9 deletions(-)
diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c
index 30c033036d..82352b7981 100644
--- a/fftools/ffmpeg_mux.c
+++ b/fftools/ffmpeg_mux.c
@@ -39,6 +39,10 @@
#include "libavformat/avformat.h"
#include "libavformat/avio.h"
+typedef struct MuxThreadContext {
+AVPacket *pkt;
+} MuxThreadContext;
+
int want_sdp = 1;
static Muxer *mux_from_of(OutputFile *of)
@@ -210,18 +214,40 @@ static void thread_set_name(OutputFile *of)
ff_thread_setname(name);
}
+static void mux_thread_uninit(MuxThreadContext *mt)
+{
+av_packet_free(&mt->pkt);
+
+memset(mt, 0, sizeof(*mt));
+}
+
+static int mux_thread_init(MuxThreadContext *mt)
+{
+memset(mt, 0, sizeof(*mt));
+
+mt->pkt = av_packet_alloc();
+if (!mt->pkt)
+goto fail;
+
+return 0;
+
+fail:
+mux_thread_uninit(mt);
+return AVERROR(ENOMEM);
+}
+
static void *muxer_thread(void *arg)
{
Muxer *mux = arg;
OutputFile *of = &mux->of;
-AVPacket *pkt = NULL;
+
+MuxThreadContext mt;
+
intret = 0;
-pkt = av_packet_alloc();
-if (!pkt) {
-ret = AVERROR(ENOMEM);
+ret = mux_thread_init(&mt);
+if (ret < 0)
goto finish;
-}
thread_set_name(of);
@@ -229,7 +255,7 @@ static void *muxer_thread(void *arg)
OutputStream *ost;
int stream_idx, stream_eof = 0;
-ret = tq_receive(mux->tq, &stream_idx, pkt);
+ret = tq_receive(mux->tq, &stream_idx, mt.pkt);
if (stream_idx < 0) {
av_log(mux, AV_LOG_VERBOSE, "All streams finished\n");
ret = 0;
@@ -237,8 +263,8 @@ static void *muxer_thread(void *arg)
}
ost = of->streams[stream_idx];
-ret = sync_queue_process(mux, ost, ret < 0 ? NULL : pkt, &stream_eof);
-av_packet_unref(pkt);
+ret = sync_queue_process(mux, ost, ret < 0 ? NULL : mt.pkt,
&stream_eof);
+av_packet_unref(mt.pkt);
if (ret == AVERROR_EOF) {
if (stream_eof) {
tq_receive_finish(mux->tq, stream_idx);
@@ -254,7 +280,7 @@ static void *muxer_thread(void *arg)
}
finish:
-av_packet_free(&pkt);
+mux_thread_uninit(&mt);
for (unsigned int i = 0; i < mux->fc->nb_streams; i++)
tq_receive_finish(mux->tq, i);
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 11/24] fftools/ffmpeg_filter: buffer sub2video heartbeat frames like other frames
Otherwise they'd be silently ignored if received by the filtering thread
before the filtergraph can be initialized, which would make the output
dependent on the order in which frames from different inputs arrive.
---
fftools/ffmpeg_filter.c | 43 -
1 file changed, 25 insertions(+), 18 deletions(-)
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 384cdedcd0..9d86c29ebd 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -1761,6 +1761,8 @@ static int graph_is_meta(AVFilterGraph *graph)
return 1;
}
+static int sub2video_frame(InputFilter *ifilter, AVFrame *frame);
+
static int configure_filtergraph(FilterGraph *fg, const FilterGraphThread *fgt)
{
FilterGraphPriv *fgp = fgp_from_fg(fg);
@@ -1872,7 +1874,7 @@ static int configure_filtergraph(FilterGraph *fg, const
FilterGraphThread *fgt)
AVFrame *tmp;
while (av_fifo_read(ifp->frame_queue, &tmp, 1) >= 0) {
if (ifp->type_src == AVMEDIA_TYPE_SUBTITLE) {
-sub2video_update(ifp, INT64_MIN, (const
AVSubtitle*)tmp->buf[0]->data);
+sub2video_frame(&ifp->ifilter, tmp);
} else {
ret = av_buffersrc_add_frame(ifp->filter, tmp);
}
@@ -2467,9 +2469,6 @@ static void sub2video_heartbeat(InputFilter *ifilter,
int64_t pts, AVRational tb
InputFilterPriv *ifp = ifp_from_ifilter(ifilter);
int64_t pts2;
-if (!ifilter->graph->graph)
-return;
-
/* subtitles seem to be usually muxed ahead of other streams;
if not, subtracting a larger time here is necessary */
pts2 = av_rescale_q(pts, tb, ifp->time_base) - 1;
@@ -2485,18 +2484,38 @@ static void sub2video_heartbeat(InputFilter *ifilter,
int64_t pts, AVRational tb
sub2video_update(ifp, pts2 + 1, NULL);
}
-static int sub2video_frame(InputFilter *ifilter, const AVFrame *frame)
+static int sub2video_frame(InputFilter *ifilter, AVFrame *frame)
{
InputFilterPriv *ifp = ifp_from_ifilter(ifilter);
int ret;
+if (!ifilter->graph->graph) {
+AVFrame *tmp;
+
+if (!frame)
+return 0;
+
+tmp = av_frame_alloc();
+if (!tmp)
+return AVERROR(ENOMEM);
+
+av_frame_move_ref(tmp, frame);
+
+ret = av_fifo_write(ifp->frame_queue, &tmp, 1);
+if (ret < 0) {
+av_frame_free(&tmp);
+return ret;
+}
+
+return 0;
+}
+
// heartbeat frame
if (frame && !frame->buf[0]) {
sub2video_heartbeat(ifilter, frame->pts, frame->time_base);
return 0;
}
-if (ifilter->graph->graph) {
if (!frame) {
if (ifp->sub2video.end_pts < INT64_MAX)
sub2video_update(ifp, INT64_MAX, NULL);
@@ -2508,18 +2527,6 @@ static int sub2video_frame(InputFilter *ifilter, const
AVFrame *frame)
ifp->height = frame->height ? frame->height : ifp->height;
sub2video_update(ifp, INT64_MIN, (const
AVSubtitle*)frame->buf[0]->data);
-} else if (frame) {
-AVFrame *tmp = av_frame_clone(frame);
-
-if (!tmp)
-return AVERROR(ENOMEM);
-
-ret = av_fifo_write(ifp->frame_queue, &tmp, 1);
-if (ret < 0) {
-av_frame_free(&tmp);
-return ret;
-}
-}
return 0;
}
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 14/24] fftools/ffmpeg_mux: move bitstream filtering to the muxer thread
This will be the appropriate place for it after the rest of transcoding
is switched to a threaded architecture.
---
fftools/ffmpeg_mux.c | 112 ++-
1 file changed, 67 insertions(+), 45 deletions(-)
diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c
index 82352b7981..57fb8a8413 100644
--- a/fftools/ffmpeg_mux.c
+++ b/fftools/ffmpeg_mux.c
@@ -207,6 +207,67 @@ static int sync_queue_process(Muxer *mux, OutputStream
*ost, AVPacket *pkt, int
return 0;
}
+/* apply the output bitstream filters */
+static int mux_packet_filter(Muxer *mux, OutputStream *ost,
+ AVPacket *pkt, int *stream_eof)
+{
+MuxStream *ms = ms_from_ost(ost);
+const char *err_msg;
+int ret = 0;
+
+if (ms->bsf_ctx) {
+int bsf_eof = 0;
+
+if (pkt)
+av_packet_rescale_ts(pkt, pkt->time_base,
ms->bsf_ctx->time_base_in);
+
+ret = av_bsf_send_packet(ms->bsf_ctx, pkt);
+if (ret < 0) {
+err_msg = "submitting a packet for bitstream filtering";
+goto fail;
+}
+
+while (!bsf_eof) {
+ret = av_bsf_receive_packet(ms->bsf_ctx, ms->bsf_pkt);
+if (ret == AVERROR(EAGAIN))
+return 0;
+else if (ret == AVERROR_EOF)
+bsf_eof = 1;
+else if (ret < 0) {
+av_log(ost, AV_LOG_ERROR,
+ "Error applying bitstream filters to a packet: %s",
+ av_err2str(ret));
+if (exit_on_error)
+return ret;
+continue;
+}
+
+if (!bsf_eof)
+ms->bsf_pkt->time_base = ms->bsf_ctx->time_base_out;
+
+ret = sync_queue_process(mux, ost, bsf_eof ? NULL : ms->bsf_pkt,
stream_eof);
+if (ret < 0)
+goto mux_fail;
+}
+*stream_eof = 1;
+return AVERROR_EOF;
+} else {
+ret = sync_queue_process(mux, ost, pkt, stream_eof);
+if (ret < 0)
+goto mux_fail;
+}
+
+return 0;
+
+mux_fail:
+err_msg = "submitting a packet to the muxer";
+
+fail:
+if (ret != AVERROR_EOF)
+av_log(ost, AV_LOG_ERROR, "Error %s: %s\n", err_msg, av_err2str(ret));
+return ret;
+}
+
static void thread_set_name(OutputFile *of)
{
char name[16];
@@ -263,7 +324,7 @@ static void *muxer_thread(void *arg)
}
ost = of->streams[stream_idx];
-ret = sync_queue_process(mux, ost, ret < 0 ? NULL : mt.pkt,
&stream_eof);
+ret = mux_packet_filter(mux, ost, ret < 0 ? NULL : mt.pkt,
&stream_eof);
av_packet_unref(mt.pkt);
if (ret == AVERROR_EOF) {
if (stream_eof) {
@@ -376,58 +437,19 @@ static int submit_packet(Muxer *mux, AVPacket *pkt,
OutputStream *ost)
int of_output_packet(OutputFile *of, OutputStream *ost, AVPacket *pkt)
{
Muxer *mux = mux_from_of(of);
-MuxStream *ms = ms_from_ost(ost);
-const char *err_msg;
int ret = 0;
if (pkt && pkt->dts != AV_NOPTS_VALUE)
ost->last_mux_dts = av_rescale_q(pkt->dts, pkt->time_base,
AV_TIME_BASE_Q);
-/* apply the output bitstream filters */
-if (ms->bsf_ctx) {
-int bsf_eof = 0;
-
-if (pkt)
-av_packet_rescale_ts(pkt, pkt->time_base,
ms->bsf_ctx->time_base_in);
-
-ret = av_bsf_send_packet(ms->bsf_ctx, pkt);
-if (ret < 0) {
-err_msg = "submitting a packet for bitstream filtering";
-goto fail;
-}
-
-while (!bsf_eof) {
-ret = av_bsf_receive_packet(ms->bsf_ctx, ms->bsf_pkt);
-if (ret == AVERROR(EAGAIN))
-return 0;
-else if (ret == AVERROR_EOF)
-bsf_eof = 1;
-else if (ret < 0) {
-err_msg = "applying bitstream filters to a packet";
-goto fail;
-}
-
-if (!bsf_eof)
-ms->bsf_pkt->time_base = ms->bsf_ctx->time_base_out;
-
-ret = submit_packet(mux, bsf_eof ? NULL : ms->bsf_pkt, ost);
-if (ret < 0)
-goto mux_fail;
-}
-} else {
-ret = submit_packet(mux, pkt, ost);
-if (ret < 0)
-goto mux_fail;
+ret = submit_packet(mux, pkt, ost);
+if (ret < 0) {
+av_log(ost, AV_LOG_ERROR, "Error submitting a packet to the muxer: %s",
+ av_err2str(ret));
+return ret;
}
return 0;
-
-mux_fail:
-err_msg = "submitting a packet to the muxer";
-
-fail:
-av_log(ost, AV_LOG_ERROR, "Error %s\n", err_msg);
-return exit_on_error ? ret : 0;
}
int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts)
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org w
[FFmpeg-devel] [PATCH 16/24] fftools/ffmpeg: disable -fix_sub_duration_heartbeat
As it causes subtitle packets processed by encoders/muxers to signal
back to decoding, it depends on packets being processed in a specific
order and is thus in its current form fundamentally incompatible with
threading architecture.
---
fftools/ffmpeg.c | 31 ---
fftools/ffmpeg.h | 10 --
fftools/ffmpeg_dec.c | 23 ---
fftools/ffmpeg_enc.c | 7 ---
fftools/ffmpeg_mux.c | 10 --
fftools/ffmpeg_mux_init.c | 4
fftools/ffmpeg_opt.c | 9 +++--
tests/fate/ffmpeg.mak | 24
8 files changed, 19 insertions(+), 99 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 038649d9b5..f2293e0250 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -769,37 +769,6 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle
*subtitle, int copy)
return 0;
}
-int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt)
-{
-OutputFile *of = output_files[ost->file_index];
-int64_t signal_pts = av_rescale_q(pkt->pts, pkt->time_base,
- AV_TIME_BASE_Q);
-
-if (!ost->fix_sub_duration_heartbeat || !(pkt->flags & AV_PKT_FLAG_KEY))
-// we are only interested in heartbeats on streams configured, and
-// only on random access points.
-return 0;
-
-for (int i = 0; i < of->nb_streams; i++) {
-OutputStream *iter_ost = of->streams[i];
-InputStream *ist = iter_ost->ist;
-int ret = AVERROR_BUG;
-
-if (iter_ost == ost || !ist || !ist->decoding_needed ||
-ist->dec_ctx->codec_type != AVMEDIA_TYPE_SUBTITLE)
-// We wish to skip the stream that causes the heartbeat,
-// output streams without an input stream, streams not decoded
-// (as fix_sub_duration is only done for decoded subtitles) as
-// well as non-subtitle streams.
-continue;
-
-if ((ret = fix_sub_duration_heartbeat(ist, signal_pts)) < 0)
-return ret;
-}
-
-return 0;
-}
-
/* pkt = NULL means EOF (needed to flush decoder buffers) */
static int process_input_packet(InputStream *ist, const AVPacket *pkt, int
no_eof)
{
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 8de91ab85a..c954ed5ebf 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -248,8 +248,6 @@ typedef struct OptionsContext {
intnb_reinit_filters;
SpecifierOpt *fix_sub_duration;
intnb_fix_sub_duration;
-SpecifierOpt *fix_sub_duration_heartbeat;
-intnb_fix_sub_duration_heartbeat;
SpecifierOpt *canvas_sizes;
intnb_canvas_sizes;
SpecifierOpt *pass;
@@ -604,12 +602,6 @@ typedef struct OutputStream {
EncStats enc_stats_pre;
EncStats enc_stats_post;
-
-/*
- * bool on whether this stream should be utilized for splitting
- * subtitles utilizing fix_sub_duration at random access points.
- */
-unsigned int fix_sub_duration_heartbeat;
} OutputStream;
typedef struct OutputFile {
@@ -875,8 +867,6 @@ InputStream *ist_iter(InputStream *prev);
OutputStream *ost_iter(OutputStream *prev);
void close_output_stream(OutputStream *ost);
-int trigger_fix_sub_duration_heartbeat(OutputStream *ost, const AVPacket *pkt);
-int fix_sub_duration_heartbeat(InputStream *ist, int64_t signal_pts);
void update_benchmark(const char *fmt, ...);
#define SPECIFIER_OPT_FMT_str "%s"
diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index b60bad1220..798ddc25b3 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -439,29 +439,6 @@ static int process_subtitle(InputStream *ist, AVFrame
*frame)
return 0;
}
-int fix_sub_duration_heartbeat(InputStream *ist, int64_t signal_pts)
-{
-Decoder *d = ist->decoder;
-int ret = AVERROR_BUG;
-AVSubtitle *prev_subtitle = d->sub_prev[0]->buf[0] ?
-(AVSubtitle*)d->sub_prev[0]->buf[0]->data : NULL;
-AVSubtitle *subtitle;
-
-if (!ist->fix_sub_duration || !prev_subtitle ||
-!prev_subtitle->num_rects || signal_pts <= prev_subtitle->pts)
-return 0;
-
-av_frame_unref(d->sub_heartbeat);
-ret = subtitle_wrap_frame(d->sub_heartbeat, prev_subtitle, 1);
-if (ret < 0)
-return ret;
-
-subtitle = (AVSubtitle*)d->sub_heartbeat->buf[0]->data;
-subtitle->pts = signal_pts;
-
-return process_subtitle(ist, d->sub_heartbeat);
-}
-
static int transcode_subtitles(InputStream *ist, const AVPacket *pkt,
AVFrame *frame)
{
diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c
index fa4539664f..aae0ba7a73 100644
--- a/fftools/ffmpeg_enc.c
+++ b/fftools/ffmpeg_enc.c
@@ -692,13 +692,6 @@ static int encode_frame(OutputFile *of, OutputStream *ost,
AVFrame *frame)
av_ts2str(pkt->duration), av_ts2timestr(pkt->duration,
&enc->time_base));
}
-if ((ret = trigger_fix_sub_duration
[FFmpeg-devel] [PATCH 06/24] fftools/thread_queue: count receive-finished streams as finished
This ensures that tq_receive() will always return EOF after all streams
were receive-finished, even though the sending side might not have
closed them yet. This may allow the receiver to avoid manually tracking
which streams it has already closed.
---
fftools/thread_queue.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fftools/thread_queue.c b/fftools/thread_queue.c
index feac6a7748..fd73cc0a9b 100644
--- a/fftools/thread_queue.c
+++ b/fftools/thread_queue.c
@@ -177,7 +177,7 @@ static int receive_locked(ThreadQueue *tq, int *stream_idx,
}
for (unsigned int i = 0; i < tq->nb_streams; i++) {
-if (!(tq->finished[i] & FINISHED_SEND))
+if (!tq->finished[i])
continue;
/* return EOF to the consumer at most once for each stream */
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 19/24] fftools/ffmpeg_demux: convert to the scheduler
---
fftools/ffmpeg.c | 12 +-
fftools/ffmpeg.h | 21 +---
fftools/ffmpeg_demux.c | 268 -
3 files changed, 134 insertions(+), 167 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 1a58bf98cf..611ac4621d 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -791,16 +791,6 @@ static int process_input_packet(InputStream *ist, const
AVPacket *pkt, int no_eo
dts_est = pd->dts_est;
}
-if (f->recording_time != INT64_MAX) {
-int64_t start_time = 0;
-if (copy_ts) {
-start_time += f->start_time != AV_NOPTS_VALUE ? f->start_time : 0;
-start_time += start_at_zero ? 0 : f->start_time_effective;
-}
-if (dts_est >= f->recording_time + start_time)
-pkt = NULL;
-}
-
for (int oidx = 0; oidx < ist->nb_outputs; oidx++) {
OutputStream *ost = ist->outputs[oidx];
if (ost->enc || (!pkt && no_eof))
@@ -1029,7 +1019,7 @@ static int process_input(int file_index, AVPacket *pkt)
InputStream *ist;
int ret, i;
-ret = ifile_get_packet(ifile, pkt);
+ret = 0;
if (ret == 1) {
/* the input file is looped: flush the decoders */
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 5833f85ab5..73b3e54fb0 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -89,6 +89,10 @@ enum FrameOpaque {
FRAME_OPAQUE_SEND_COMMAND,
};
+enum PacketOpaque {
+PKT_OPAQUE_SUB_HEARTBEAT = 1,
+};
+
typedef struct HWDevice {
const char *name;
enum AVHWDeviceType type;
@@ -424,11 +428,6 @@ typedef struct InputFile {
float readrate;
int accurate_seek;
-
-/* when looping the input file, this queue is used by decoders to report
- * the last frame timestamp back to the demuxer thread */
-AVThreadMessageQueue *audio_ts_queue;
-int audio_ts_queue_size;
} InputFile;
enum forced_keyframes_const {
@@ -842,18 +841,6 @@ int64_t of_filesize(OutputFile *of);
int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch);
void ifile_close(InputFile **f);
-/**
- * Get next input packet from the demuxer.
- *
- * @param pkt the packet is written here when this function returns 0
- * @return
- * - 0 when a packet has been read successfully
- * - 1 when stream end was reached, but the stream is looped;
- * caller should flush decoders and read from this demuxer again
- * - a negative error code on failure
- */
-int ifile_get_packet(InputFile *f, AVPacket *pkt);
-
int ist_output_add(InputStream *ist, OutputStream *ost);
int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple);
diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c
index 2234dbe076..91cd7a1125 100644
--- a/fftools/ffmpeg_demux.c
+++ b/fftools/ffmpeg_demux.c
@@ -22,8 +22,6 @@
#include "ffmpeg.h"
#include "ffmpeg_sched.h"
#include "ffmpeg_utils.h"
-#include "objpool.h"
-#include "thread_queue.h"
#include "libavutil/avassert.h"
#include "libavutil/avstring.h"
@@ -35,7 +33,6 @@
#include "libavutil/pixdesc.h"
#include "libavutil/time.h"
#include "libavutil/timestamp.h"
-#include "libavutil/thread.h"
#include "libavcodec/packet.h"
@@ -66,7 +63,11 @@ typedef struct DemuxStream {
double ts_scale;
+// scheduler returned EOF for this stream
+int finished;
+
int streamcopy_needed;
+int have_sub2video;
int wrap_correction_done;
int saw_first_ts;
@@ -101,6 +102,7 @@ typedef struct Demuxer {
/* number of times input stream should be looped */
int loop;
+int have_audio_dec;
/* duration of the looped segment of the input file */
Timestamp duration;
/* pts with the smallest/largest values ever seen */
@@ -113,11 +115,12 @@ typedef struct Demuxer {
double readrate_initial_burst;
Scheduler*sch;
-ThreadQueue *thread_queue;
-int thread_queue_size;
-pthread_t thread;
+
+AVPacket *pkt_heartbeat;
int read_started;
+int nb_streams_used;
+int nb_streams_finished;
} Demuxer;
static DemuxStream *ds_from_ist(InputStream *ist)
@@ -153,7 +156,7 @@ static void report_new_stream(Demuxer *d, const AVPacket
*pkt)
d->nb_streams_warn = pkt->stream_index + 1;
}
-static int seek_to_start(Demuxer *d)
+static int seek_to_start(Demuxer *d, Timestamp end_pts)
{
InputFile*ifile = &d->f;
AVFormatContext *is = ifile->ctx;
@@ -163,21 +166,10 @@ static int seek_to_start(Demuxer *d)
if (ret < 0)
return ret;
-if (ifile->audio_ts_queue_size) {
-int got_ts = 0;
-
-while (got_ts < ifile->audio_ts_queue_size) {
-Timestamp ts;
-ret = av_thread_message_queue_recv(ifile->audio_ts_queue, &ts, 0);
-if (ret < 0)
-return ret;
-got_ts++;
-
-if (d->max_pts.t
[FFmpeg-devel] [PATCH 18/24] fftools/ffmpeg: add thread-aware transcode scheduling infrastructure
See the comment block at the top of fftools/ffmpeg_sched.h for more
details on what this scheduler is for.
This commit adds the scheduling code itself, along with minimal
integration with the rest of the program:
* allocating and freeing the scheduler
* passing it throughout the call stack in order to register the
individual components (demuxers/decoders/filtergraphs/encoders/muxers)
with the scheduler
The scheduler is not actually used as of this commit, so it should not
result in any change in behavior. That will change in future commits.
---
fftools/Makefile |1 +
fftools/ffmpeg.c | 18 +-
fftools/ffmpeg.h | 24 +-
fftools/ffmpeg_dec.c | 10 +-
fftools/ffmpeg_demux.c| 46 +-
fftools/ffmpeg_enc.c | 13 +-
fftools/ffmpeg_filter.c | 37 +-
fftools/ffmpeg_mux.c | 17 +-
fftools/ffmpeg_mux.h | 11 +
fftools/ffmpeg_mux_init.c | 82 +-
fftools/ffmpeg_opt.c | 22 +-
fftools/ffmpeg_sched.c| 2072 +
fftools/ffmpeg_sched.h| 461 +
13 files changed, 2758 insertions(+), 56 deletions(-)
create mode 100644 fftools/ffmpeg_sched.c
create mode 100644 fftools/ffmpeg_sched.h
diff --git a/fftools/Makefile b/fftools/Makefile
index 56820e6bc8..d6a8913a7f 100644
--- a/fftools/Makefile
+++ b/fftools/Makefile
@@ -18,6 +18,7 @@ OBJS-ffmpeg += \
fftools/ffmpeg_mux.o\
fftools/ffmpeg_mux_init.o \
fftools/ffmpeg_opt.o\
+fftools/ffmpeg_sched.o \
fftools/objpool.o \
fftools/sync_queue.o\
fftools/thread_queue.o \
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index f2293e0250..1a58bf98cf 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -99,6 +99,7 @@
#include "cmdutils.h"
#include "ffmpeg.h"
+#include "ffmpeg_sched.h"
#include "ffmpeg_utils.h"
#include "sync_queue.h"
@@ -1123,7 +1124,7 @@ static int transcode_step(OutputStream *ost, AVPacket
*demux_pkt)
/*
* The following code is the main loop of the file converter
*/
-static int transcode(int *err_rate_exceeded)
+static int transcode(Scheduler *sch, int *err_rate_exceeded)
{
int ret = 0, i;
InputStream *ist;
@@ -1261,6 +1262,8 @@ static int64_t getmaxrss(void)
int main(int argc, char **argv)
{
+Scheduler *sch = NULL;
+
int ret, err_rate_exceeded;
BenchmarkTimeStamps ti;
@@ -1278,8 +1281,14 @@ int main(int argc, char **argv)
show_banner(argc, argv, options);
+sch = sch_alloc();
+if (!sch) {
+ret = AVERROR(ENOMEM);
+goto finish;
+}
+
/* parse options and open all input/output files */
-ret = ffmpeg_parse_options(argc, argv);
+ret = ffmpeg_parse_options(argc, argv, sch);
if (ret < 0)
goto finish;
@@ -1297,7 +1306,7 @@ int main(int argc, char **argv)
}
current_time = ti = get_benchmark_time_stamps();
-ret = transcode(&err_rate_exceeded);
+ret = transcode(sch, &err_rate_exceeded);
if (ret >= 0 && do_benchmark) {
int64_t utime, stime, rtime;
current_time = get_benchmark_time_stamps();
@@ -1317,5 +1326,8 @@ finish:
ret = 0;
ffmpeg_cleanup(ret);
+
+sch_free(&sch);
+
return ret;
}
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index c954ed5ebf..5833f85ab5 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -27,6 +27,7 @@
#include
#include "cmdutils.h"
+#include "ffmpeg_sched.h"
#include "sync_queue.h"
#include "libavformat/avformat.h"
@@ -713,7 +714,8 @@ int parse_and_set_vsync(const char *arg, int *vsync_var,
int file_idx, int st_id
int check_filter_outputs(void);
int filtergraph_is_simple(const FilterGraph *fg);
int init_simple_filtergraph(InputStream *ist, OutputStream *ost,
-char *graph_desc);
+char *graph_desc,
+Scheduler *sch, unsigned sch_idx_enc);
int init_complex_filtergraph(FilterGraph *fg);
int copy_av_subtitle(AVSubtitle *dst, const AVSubtitle *src);
@@ -736,7 +738,8 @@ void ifilter_sub2video_heartbeat(InputFilter *ifilter,
int64_t pts, AVRational t
*/
int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext
*dec);
-int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost);
+int ofilter_bind_ost(OutputFilter *ofilter, OutputStream *ost,
+ unsigned sched_idx_enc);
/**
* Create a new filtergraph in the global filtergraph list.
@@ -744,7 +747,7 @@ int ofilter_bind_ost(OutputFilter *ofilter, OutputStream
*ost);
* @param graph_desc Graph description; an av_malloc()ed string, filtergraph
* takes ownership of it.
*/
-int fg_create(FilterGraph **pfg, char *graph_desc);
+int fg_create(FilterGraph **pfg, char *graph_desc, Scheduler *sch);
void fg_free(FilterGraph **pfg);
@@ -768,7 +771,7 @@ void fg_send_command(FilterGraph *fg, double time, const
char *target,
*/
int reap_filte
[FFmpeg-devel] [PATCH 23/24] fftools/ffmpeg_mux: convert to the scheduler
---
fftools/ffmpeg.c | 30 +---
fftools/ffmpeg.h | 11 +-
fftools/ffmpeg_mux.c | 290 ++
fftools/ffmpeg_mux.h | 24 +---
fftools/ffmpeg_mux_init.c | 40 ++
fftools/ffmpeg_opt.c | 6 +-
6 files changed, 61 insertions(+), 340 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 122424a0e1..5d1560b891 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -117,7 +117,7 @@ typedef struct BenchmarkTimeStamps {
static BenchmarkTimeStamps get_benchmark_time_stamps(void);
static int64_t getmaxrss(void);
-unsigned nb_output_dumped = 0;
+atomic_uint nb_output_dumped = 0;
static BenchmarkTimeStamps current_time;
AVIOContext *progress_avio = NULL;
@@ -496,7 +496,7 @@ static void print_report(int is_last_report, int64_t
timer_start, int64_t cur_ti
last_time = cur_time;
}
if (((cur_time - last_time) < stats_period && !first_report) ||
-(first_report && nb_output_dumped < nb_output_files))
+(first_report && atomic_load(&nb_output_dumped) < nb_output_files))
return;
last_time = cur_time;
}
@@ -750,28 +750,12 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle
*subtitle, int copy)
static int process_input_packet(InputStream *ist, const AVPacket *pkt, int
no_eof)
{
InputFile *f = input_files[ist->file_index];
-int64_t dts_est = AV_NOPTS_VALUE;
int ret = 0;
int eof_reached = 0;
if (ret == AVERROR_EOF || (!pkt && !ist->decoding_needed))
eof_reached = 1;
-if (pkt && pkt->opaque_ref) {
-DemuxPktData *pd = (DemuxPktData*)pkt->opaque_ref->data;
-dts_est = pd->dts_est;
-}
-
-for (int oidx = 0; oidx < ist->nb_outputs; oidx++) {
-OutputStream *ost = ist->outputs[oidx];
-if (ost->enc || (!pkt && no_eof))
-continue;
-
-ret = of_streamcopy(ost, pkt, dts_est);
-if (ret < 0)
-return ret;
-}
-
return !eof_reached;
}
@@ -995,16 +979,6 @@ static int process_input(int file_index, AVPacket *pkt)
else if (ret < 0)
return ret;
}
-
-/* mark all outputs that don't go through lavfi as finished */
-for (int oidx = 0; oidx < ist->nb_outputs; oidx++) {
-OutputStream *ost = ist->outputs[oidx];
-OutputFile*of = output_files[ost->file_index];
-
-ret = of_output_packet(of, ost, NULL);
-if (ret < 0)
-return ret;
-}
}
ifile->eof_reached = 1;
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 20abd5e772..afc4496bd6 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -594,7 +594,6 @@ typedef struct OutputStream {
/* packet quality factor */
atomic_int quality;
-int sq_idx_encode;
int sq_idx_mux;
EncStats enc_stats_pre;
@@ -646,7 +645,6 @@ extern FilterGraph **filtergraphs;
extern intnb_filtergraphs;
extern char *vstats_filename;
-extern char *sdp_filename;
extern float dts_delta_threshold;
extern float dts_error_threshold;
@@ -679,7 +677,7 @@ extern const AVIOInterruptCB int_cb;
extern const OptionDef options[];
extern HWDevice *filter_hw_device;
-extern unsigned nb_output_dumped;
+extern atomic_uint nb_output_dumped;
extern int ignore_unknown_streams;
extern int copy_unknown_streams;
@@ -791,13 +789,6 @@ void of_free(OutputFile **pof);
void of_enc_stats_close(void);
-int of_output_packet(OutputFile *of, OutputStream *ost, AVPacket *pkt);
-
-/**
- * @param dts predicted packet dts in AV_TIME_BASE_Q
- */
-int of_streamcopy(OutputStream *ost, const AVPacket *pkt, int64_t dts);
-
int64_t of_filesize(OutputFile *of);
int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch);
diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c
index 7dd8e8c848..815bc883ea 100644
--- a/fftools/ffmpeg_mux.c
+++ b/fftools/ffmpeg_mux.c
@@ -23,16 +23,13 @@
#include "ffmpeg.h"
#include "ffmpeg_mux.h"
#include "ffmpeg_utils.h"
-#include "objpool.h"
#include "sync_queue.h"
-#include "thread_queue.h"
#include "libavutil/fifo.h"
#include "libavutil/intreadwrite.h"
#include "libavutil/log.h"
#include "libavutil/mem.h"
#include "libavutil/timestamp.h"
-#include "libavutil/thread.h"
#include "libavcodec/packet.h"
@@ -43,8 +40,6 @@ typedef struct MuxThreadContext {
AVPacket *pkt;
} MuxThreadContext;
-int want_sdp = 1;
-
static Muxer *mux_from_of(OutputFile *of)
{
return (Muxer*)of;
@@ -207,6 +202,8 @@ static int sync_queue_process(Muxer *mux, OutputStream
*ost, AVPacket *pkt, int
return 0;
}
+static int of_streamcopy(OutputStream *ost, AVPacket *pkt);
+
/* apply the output bitstream filters */
static int mux_packet_filter(Muxer *mux, OutputStream *ost,
AVPacket *pkt, int *stream_eof)
@@ -215,6 +212,18 @@ static int mux
[FFmpeg-devel] [PATCH 24/24] ffmpeg: switch to scheduler
---
fftools/ffmpeg.c | 236 +--
fftools/ffmpeg.h | 10 +-
2 files changed, 25 insertions(+), 221 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 5d1560b891..aae680f052 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -462,23 +462,13 @@ void update_benchmark(const char *fmt, ...)
}
}
-void close_output_stream(OutputStream *ost)
-{
-OutputFile *of = output_files[ost->file_index];
-ost->finished |= ENCODER_FINISHED;
-
-if (ost->sq_idx_encode >= 0)
-sq_send(of->sq_encode, ost->sq_idx_encode, SQFRAME(NULL));
-}
-
-static void print_report(int is_last_report, int64_t timer_start, int64_t
cur_time)
+static void print_report(int is_last_report, int64_t timer_start, int64_t
cur_time, int64_t pts)
{
AVBPrint buf, buf_script;
int64_t total_size = of_filesize(output_files[0]);
int vid;
double bitrate;
double speed;
-int64_t pts = AV_NOPTS_VALUE;
static int64_t last_time = -1;
static int first_report = 1;
uint64_t nb_frames_dup = 0, nb_frames_drop = 0;
@@ -533,17 +523,13 @@ static void print_report(int is_last_report, int64_t
timer_start, int64_t cur_ti
vid = 1;
}
-/* compute min output value */
-if (ost->last_mux_dts != AV_NOPTS_VALUE) {
-if (pts == AV_NOPTS_VALUE || ost->last_mux_dts > pts)
-pts = ost->last_mux_dts;
-if (copy_ts) {
-if (copy_ts_first_pts == AV_NOPTS_VALUE && pts > 1)
-copy_ts_first_pts = pts;
-if (copy_ts_first_pts != AV_NOPTS_VALUE)
-pts -= copy_ts_first_pts;
-}
-}
+}
+
+if (copy_ts) {
+if (copy_ts_first_pts == AV_NOPTS_VALUE && pts > 1)
+copy_ts_first_pts = pts;
+if (copy_ts_first_pts != AV_NOPTS_VALUE)
+pts -= copy_ts_first_pts;
}
us= FFABS64U(pts) % AV_TIME_BASE;
@@ -746,19 +732,6 @@ int subtitle_wrap_frame(AVFrame *frame, AVSubtitle
*subtitle, int copy)
return 0;
}
-/* pkt = NULL means EOF (needed to flush decoder buffers) */
-static int process_input_packet(InputStream *ist, const AVPacket *pkt, int
no_eof)
-{
-InputFile *f = input_files[ist->file_index];
-int ret = 0;
-int eof_reached = 0;
-
-if (ret == AVERROR_EOF || (!pkt && !ist->decoding_needed))
-eof_reached = 1;
-
-return !eof_reached;
-}
-
static void print_stream_maps(void)
{
av_log(NULL, AV_LOG_INFO, "Stream mapping:\n");
@@ -835,41 +808,6 @@ static void print_stream_maps(void)
}
}
-/**
- * Select the output stream to process.
- *
- * @retval 0 an output stream was selected
- * @retval AVERROR(EAGAIN) need to wait until more input is available
- * @retval AVERROR_EOF no more streams need output
- */
-static int choose_output(OutputStream **post)
-{
-int64_t opts_min = INT64_MAX;
-OutputStream *ost_min = NULL;
-
-for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) {
-int64_t opts;
-
-{
-opts = ost->last_mux_dts == AV_NOPTS_VALUE ?
- INT64_MIN : ost->last_mux_dts;
-}
-
-if (!ost->initialized && !ost->finished) {
-ost_min = ost;
-break;
-}
-if (!ost->finished && opts < opts_min) {
-opts_min = opts;
-ost_min = ost;
-}
-}
-if (!ost_min)
-return AVERROR_EOF;
-*post = ost_min;
-return ost_min->unavailable ? AVERROR(EAGAIN) : 0;
-}
-
static void set_tty_echo(int on)
{
#if HAVE_TERMIOS_H
@@ -941,110 +879,21 @@ static int check_keyboard_interaction(int64_t cur_time)
return 0;
}
-static void reset_eagain(void)
-{
-for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost))
-ost->unavailable = 0;
-}
-
-/*
- * Return
- * - 0 -- one packet was read and processed
- * - AVERROR(EAGAIN) -- no packets were available for selected file,
- * this function should be called again
- * - AVERROR_EOF -- this function should not be called again
- */
-static int process_input(int file_index, AVPacket *pkt)
-{
-InputFile *ifile = input_files[file_index];
-InputStream *ist;
-int ret, i;
-
-ret = 0;
-
-if (ret < 0) {
-if (ret != AVERROR_EOF) {
-av_log(ifile, AV_LOG_ERROR,
- "Error retrieving a packet from demuxer: %s\n",
av_err2str(ret));
-if (exit_on_error)
-return ret;
-}
-
-for (i = 0; i < ifile->nb_streams; i++) {
-ist = ifile->streams[i];
-if (!ist->discard) {
-ret = process_input_packet(ist, NULL, 0);
-if (ret>0)
-return 0;
-else if (ret < 0)
-return ret;
-}
-}
-
-ifile->eof_reached = 1;
-return AVERROR(EAGAIN);
-}
-
-reset_eagain();
-
-ist = ifile->streams
[FFmpeg-devel] [PATCH 22/24] fftools/ffmpeg_enc: convert to the scheduler
---
fftools/ffmpeg.c | 3 +-
fftools/ffmpeg.h | 7 +-
fftools/ffmpeg_enc.c | 361 ++
fftools/ffmpeg_mux_init.c | 43 +
4 files changed, 66 insertions(+), 348 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 1f21008588..122424a0e1 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -507,7 +507,7 @@ static void print_report(int is_last_report, int64_t
timer_start, int64_t cur_ti
av_bprint_init(&buf, 0, AV_BPRINT_SIZE_AUTOMATIC);
av_bprint_init(&buf_script, 0, AV_BPRINT_SIZE_AUTOMATIC);
for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost)) {
-const float q = ost->enc ? ost->quality / (float) FF_QP2LAMBDA : -1;
+const float q = ost->enc ? atomic_load(&ost->quality) / (float)
FF_QP2LAMBDA : -1;
if (vid && ost->type == AVMEDIA_TYPE_VIDEO) {
av_bprintf(&buf, "q=%2.1f ", q);
@@ -1127,7 +1127,6 @@ static int transcode(Scheduler *sch, int
*err_rate_exceeded)
} else if (err_rate)
av_log(ist, AV_LOG_VERBOSE, "Decode error rate %g\n", err_rate);
}
-ret = err_merge(ret, enc_flush());
term_exit();
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index c1b61c83e7..20abd5e772 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -592,7 +592,7 @@ typedef struct OutputStream {
uint64_t samples_encoded;
/* packet quality factor */
-int quality;
+atomic_int quality;
int sq_idx_encode;
int sq_idx_mux;
@@ -776,10 +776,7 @@ int enc_alloc(Encoder **penc, const AVCodec *codec,
Scheduler *sch, unsigned sch_idx);
void enc_free(Encoder **penc);
-int enc_open(OutputStream *ost, const AVFrame *frame);
-int enc_subtitle(OutputFile *of, OutputStream *ost, const AVSubtitle *sub);
-int enc_frame(OutputStream *ost, AVFrame *frame);
-int enc_flush(void);
+int enc_open(void *opaque, const AVFrame *frame);
/*
* Initialize muxing state for the given stream, should be called
diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c
index fbfe592f20..9383b167f7 100644
--- a/fftools/ffmpeg_enc.c
+++ b/fftools/ffmpeg_enc.c
@@ -41,12 +41,6 @@
#include "libavformat/avformat.h"
struct Encoder {
-AVFrame *sq_frame;
-
-// packet for receiving encoded output
-AVPacket *pkt;
-AVFrame *sub_frame;
-
// combined size of all the packets received from the encoder
uint64_t data_size;
@@ -54,25 +48,9 @@ struct Encoder {
uint64_t packets_encoded;
int opened;
-int finished;
Scheduler *sch;
unsignedsch_idx;
-
-pthread_t thread;
-/**
- * Queue for sending frames from the main thread to
- * the encoder thread.
- */
-ThreadQueue*queue_in;
-/**
- * Queue for sending encoded packets from the encoder thread
- * to the main thread.
- *
- * An empty packet is sent to signal that a previously sent
- * frame has been fully processed.
- */
-ThreadQueue*queue_out;
};
// data that is local to the decoder thread and not visible outside of it
@@ -81,24 +59,6 @@ typedef struct EncoderThread {
AVPacket *pkt;
} EncoderThread;
-static int enc_thread_stop(Encoder *e)
-{
-void *ret;
-
-if (!e->queue_in)
-return 0;
-
-tq_send_finish(e->queue_in, 0);
-tq_receive_finish(e->queue_out, 0);
-
-pthread_join(e->thread, &ret);
-
-tq_free(&e->queue_in);
-tq_free(&e->queue_out);
-
-return (int)(intptr_t)ret;
-}
-
void enc_free(Encoder **penc)
{
Encoder *enc = *penc;
@@ -106,13 +66,6 @@ void enc_free(Encoder **penc)
if (!enc)
return;
-enc_thread_stop(enc);
-
-av_frame_free(&enc->sq_frame);
-av_frame_free(&enc->sub_frame);
-
-av_packet_free(&enc->pkt);
-
av_freep(penc);
}
@@ -127,25 +80,12 @@ int enc_alloc(Encoder **penc, const AVCodec *codec,
if (!enc)
return AVERROR(ENOMEM);
-if (codec->type == AVMEDIA_TYPE_SUBTITLE) {
-enc->sub_frame = av_frame_alloc();
-if (!enc->sub_frame)
-goto fail;
-}
-
-enc->pkt = av_packet_alloc();
-if (!enc->pkt)
-goto fail;
-
enc->sch = sch;
enc->sch_idx = sch_idx;
*penc = enc;
return 0;
-fail:
-enc_free(&enc);
-return AVERROR(ENOMEM);
}
static int hw_device_setup_for_encode(OutputStream *ost, AVBufferRef
*frames_ref)
@@ -224,52 +164,9 @@ static int set_encoder_id(OutputFile *of, OutputStream
*ost)
return 0;
}
-static int enc_thread_start(OutputStream *ost)
-{
-Encoder *e = ost->enc;
-ObjPool *op;
-int ret = 0;
-
-op = objpool_alloc_frames();
-if (!op)
-return AVERROR(ENOMEM);
-
-e->queue_in = tq_alloc(1, 1, op, frame_move);
-if (!e->queue_in) {
-objpool_free(&op);
-return AVERROR(ENOMEM);
-}
-
-op = objpool_alloc_packets();
-if (!op)
-goto fail;
-
-e->queue_out = tq_alloc(1, 4, op, pkt
[FFmpeg-devel] [PATCH 12/24] fftools/ffmpeg_filter: reindent
---
fftools/ffmpeg_filter.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 9d86c29ebd..e288ea4b80 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -2516,17 +2516,17 @@ static int sub2video_frame(InputFilter *ifilter,
AVFrame *frame)
return 0;
}
-if (!frame) {
-if (ifp->sub2video.end_pts < INT64_MAX)
-sub2video_update(ifp, INT64_MAX, NULL);
+if (!frame) {
+if (ifp->sub2video.end_pts < INT64_MAX)
+sub2video_update(ifp, INT64_MAX, NULL);
-return av_buffersrc_add_frame(ifp->filter, NULL);
-}
+return av_buffersrc_add_frame(ifp->filter, NULL);
+}
-ifp->width = frame->width ? frame->width : ifp->width;
-ifp->height = frame->height ? frame->height : ifp->height;
+ifp->width = frame->width ? frame->width : ifp->width;
+ifp->height = frame->height ? frame->height : ifp->height;
-sub2video_update(ifp, INT64_MIN, (const
AVSubtitle*)frame->buf[0]->data);
+sub2video_update(ifp, INT64_MIN, (const AVSubtitle*)frame->buf[0]->data);
return 0;
}
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 17/24] fftools/ffmpeg_enc: move encoding to a separate thread
As for the analogous decoding change, this is only a preparatory step to
a fully threaded architecture and does not yet make encoding truly
parallel. The main thread will currently submit a frame and wait until
it has been fully processed by the encoder before moving on. That will
change in future commits after filters are moved to threads and a
thread-aware scheduler is added.
This code suffers from a known issue - if an encoder with a sync queue
receives EOF it will terminate after processing everything it currently
has, even though the sync queue might still be triggered by other
threads. That will be fixed in following commits.
---
fftools/ffmpeg_enc.c | 360 ++-
1 file changed, 320 insertions(+), 40 deletions(-)
diff --git a/fftools/ffmpeg_enc.c b/fftools/ffmpeg_enc.c
index aae0ba7a73..f1c41272b0 100644
--- a/fftools/ffmpeg_enc.c
+++ b/fftools/ffmpeg_enc.c
@@ -20,6 +20,8 @@
#include
#include "ffmpeg.h"
+#include "ffmpeg_utils.h"
+#include "thread_queue.h"
#include "libavutil/avassert.h"
#include "libavutil/avstring.h"
@@ -43,6 +45,7 @@ struct Encoder {
// packet for receiving encoded output
AVPacket *pkt;
+AVFrame *sub_frame;
// combined size of all the packets received from the encoder
uint64_t data_size;
@@ -51,8 +54,48 @@ struct Encoder {
uint64_t packets_encoded;
int opened;
+int finished;
+
+pthread_t thread;
+/**
+ * Queue for sending frames from the main thread to
+ * the encoder thread.
+ */
+ThreadQueue*queue_in;
+/**
+ * Queue for sending encoded packets from the encoder thread
+ * to the main thread.
+ *
+ * An empty packet is sent to signal that a previously sent
+ * frame has been fully processed.
+ */
+ThreadQueue*queue_out;
};
+// data that is local to the decoder thread and not visible outside of it
+typedef struct EncoderThread {
+AVFrame *frame;
+AVPacket *pkt;
+} EncoderThread;
+
+static int enc_thread_stop(Encoder *e)
+{
+void *ret;
+
+if (!e->queue_in)
+return 0;
+
+tq_send_finish(e->queue_in, 0);
+tq_receive_finish(e->queue_out, 0);
+
+pthread_join(e->thread, &ret);
+
+tq_free(&e->queue_in);
+tq_free(&e->queue_out);
+
+return (int)(intptr_t)ret;
+}
+
void enc_free(Encoder **penc)
{
Encoder *enc = *penc;
@@ -60,7 +103,10 @@ void enc_free(Encoder **penc)
if (!enc)
return;
+enc_thread_stop(enc);
+
av_frame_free(&enc->sq_frame);
+av_frame_free(&enc->sub_frame);
av_packet_free(&enc->pkt);
@@ -77,6 +123,12 @@ int enc_alloc(Encoder **penc, const AVCodec *codec)
if (!enc)
return AVERROR(ENOMEM);
+if (codec->type == AVMEDIA_TYPE_SUBTITLE) {
+enc->sub_frame = av_frame_alloc();
+if (!enc->sub_frame)
+goto fail;
+}
+
enc->pkt = av_packet_alloc();
if (!enc->pkt)
goto fail;
@@ -165,6 +217,52 @@ static int set_encoder_id(OutputFile *of, OutputStream
*ost)
return 0;
}
+static void *encoder_thread(void *arg);
+
+static int enc_thread_start(OutputStream *ost)
+{
+Encoder *e = ost->enc;
+ObjPool *op;
+int ret = 0;
+
+op = objpool_alloc_frames();
+if (!op)
+return AVERROR(ENOMEM);
+
+e->queue_in = tq_alloc(1, 1, op, frame_move);
+if (!e->queue_in) {
+objpool_free(&op);
+return AVERROR(ENOMEM);
+}
+
+op = objpool_alloc_packets();
+if (!op)
+goto fail;
+
+e->queue_out = tq_alloc(1, 4, op, pkt_move);
+if (!e->queue_out) {
+objpool_free(&op);
+goto fail;
+}
+
+ret = pthread_create(&e->thread, NULL, encoder_thread, ost);
+if (ret) {
+ret = AVERROR(ret);
+av_log(ost, AV_LOG_ERROR, "pthread_create() failed: %s\n",
+ av_err2str(ret));
+goto fail;
+}
+
+return 0;
+fail:
+if (ret >= 0)
+ret = AVERROR(ENOMEM);
+
+tq_free(&e->queue_in);
+tq_free(&e->queue_out);
+return ret;
+}
+
int enc_open(OutputStream *ost, const AVFrame *frame)
{
InputStream *ist = ost->ist;
@@ -373,6 +471,13 @@ int enc_open(OutputStream *ost, const AVFrame *frame)
if (ost->st->time_base.num <= 0 || ost->st->time_base.den <= 0)
ost->st->time_base = av_add_q(ost->enc_ctx->time_base, (AVRational){0,
1});
+ret = enc_thread_start(ost);
+if (ret < 0) {
+av_log(ost, AV_LOG_ERROR, "Error starting encoder thread: %s\n",
+ av_err2str(ret));
+return ret;
+}
+
ret = of_stream_init(of, ost);
if (ret < 0)
return ret;
@@ -386,19 +491,18 @@ static int check_recording_time(OutputStream *ost,
int64_t ts, AVRational tb)
if (of->recording_time != INT64_MAX &&
av_compare_ts(ts, tb, of->recording_time, AV_TIME_BASE_Q) >= 0) {
-close_output_stream(ost);
return 0;
}
return 1;
}
-int enc_subtitle(OutputFile *of, OutputStr
[FFmpeg-devel] [PATCH 20/24] fftools/ffmpeg_dec: convert to the scheduler
---
fftools/ffmpeg.c | 22 ---
fftools/ffmpeg.h | 13 +-
fftools/ffmpeg_dec.c | 315 ++-
3 files changed, 70 insertions(+), 280 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 611ac4621d..bd783fe674 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -778,11 +778,6 @@ static int process_input_packet(InputStream *ist, const
AVPacket *pkt, int no_eo
int ret = 0;
int eof_reached = 0;
-if (ist->decoding_needed) {
-ret = dec_packet(ist, pkt, no_eof);
-if (ret < 0 && ret != AVERROR_EOF)
-return ret;
-}
if (ret == AVERROR_EOF || (!pkt && !ist->decoding_needed))
eof_reached = 1;
@@ -994,18 +989,6 @@ static void reset_eagain(void)
ost->unavailable = 0;
}
-static void decode_flush(InputFile *ifile)
-{
-for (int i = 0; i < ifile->nb_streams; i++) {
-InputStream *ist = ifile->streams[i];
-
-if (ist->discard || !ist->decoding_needed)
-continue;
-
-dec_packet(ist, NULL, 1);
-}
-}
-
/*
* Return
* - 0 -- one packet was read and processed
@@ -1021,11 +1004,6 @@ static int process_input(int file_index, AVPacket *pkt)
ret = 0;
-if (ret == 1) {
-/* the input file is looped: flush the decoders */
-decode_flush(ifile);
-return AVERROR(EAGAIN);
-}
if (ret < 0) {
if (ret != AVERROR_EOF) {
av_log(ifile, AV_LOG_ERROR,
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 73b3e54fb0..975d8b737e 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -61,6 +61,8 @@
#define FFMPEG_OPT_TOP 1
#define FFMPEG_OPT_FORCE_KF_SOURCE_NO_DROP 1
+#define FFMPEG_ERROR_RATE_EXCEEDED FFERRTAG('E', 'R', 'E', 'D')
+
enum VideoSyncMethod {
VSYNC_AUTO = -1,
VSYNC_PASSTHROUGH,
@@ -796,17 +798,6 @@ int hwaccel_retrieve_data(AVCodecContext *avctx, AVFrame
*input);
int dec_open(InputStream *ist, Scheduler *sch, unsigned sch_idx);
void dec_free(Decoder **pdec);
-/**
- * Submit a packet for decoding
- *
- * When pkt==NULL and no_eof=0, there will be no more input. Flush decoders and
- * mark all downstreams as finished.
- *
- * When pkt==NULL and no_eof=1, the stream was reset (e.g. after a seek). Flush
- * decoders and await further input.
- */
-int dec_packet(InputStream *ist, const AVPacket *pkt, int no_eof);
-
int enc_alloc(Encoder **penc, const AVCodec *codec,
Scheduler *sch, unsigned sch_idx);
void enc_free(Encoder **penc);
diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index 53e14f061e..a81f83fc92 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -54,24 +54,6 @@ struct Decoder {
Scheduler *sch;
unsignedsch_idx;
-
-pthread_t thread;
-/**
- * Queue for sending coded packets from the main thread to
- * the decoder thread.
- *
- * An empty packet is sent to flush the decoder without terminating
- * decoding.
- */
-ThreadQueue*queue_in;
-/**
- * Queue for sending decoded frames from the decoder thread
- * to the main thread.
- *
- * An empty frame is sent to signal that a single packet has been fully
- * processed.
- */
-ThreadQueue*queue_out;
};
// data that is local to the decoder thread and not visible outside of it
@@ -80,24 +62,6 @@ typedef struct DecThreadContext {
AVPacket*pkt;
} DecThreadContext;
-static int dec_thread_stop(Decoder *d)
-{
-void *ret;
-
-if (!d->queue_in)
-return 0;
-
-tq_send_finish(d->queue_in, 0);
-tq_receive_finish(d->queue_out, 0);
-
-pthread_join(d->thread, &ret);
-
-tq_free(&d->queue_in);
-tq_free(&d->queue_out);
-
-return (intptr_t)ret;
-}
-
void dec_free(Decoder **pdec)
{
Decoder *dec = *pdec;
@@ -105,8 +69,6 @@ void dec_free(Decoder **pdec)
if (!dec)
return;
-dec_thread_stop(dec);
-
av_frame_free(&dec->frame);
av_packet_free(&dec->pkt);
@@ -148,25 +110,6 @@ fail:
return AVERROR(ENOMEM);
}
-static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame)
-{
-int i, ret = 0;
-
-for (i = 0; i < ist->nb_filters; i++) {
-ret = ifilter_send_frame(ist->filters[i], decoded_frame,
- i < ist->nb_filters - 1 ||
- ist->dec->type == AVMEDIA_TYPE_SUBTITLE);
-if (ret == AVERROR_EOF)
-ret = 0; /* ignore */
-if (ret < 0) {
-av_log(NULL, AV_LOG_ERROR,
- "Failed to inject frame into filter network: %s\n",
av_err2str(ret));
-break;
-}
-}
-return ret;
-}
-
static AVRational audio_samplerate_update(void *logctx, Decoder *d,
const AVFrame *frame)
{
@@ -421,36 +364,31 @@ static int process_subtitle(InputStream *ist, AVFrame
*frame)
if (!subtitle)
return 0;
-ret = send_fram
[FFmpeg-devel] [PATCH 15/24] fftools/ffmpeg_demux: switch from AVThreadMessageQueue to ThreadQueue
* the code is made shorter and simpler
* avoids constantly allocating and freeing AVPackets, thanks to
ThreadQueue integration with ObjPool
* is consistent with decoding/filtering/muxing
* reduces the diff in the future switch to thread-aware scheduling
This makes ifile_get_packet() always block. Any potential issues caused
by this will be resolved by the switch to thread-aware scheduling in
future commits.
---
fftools/ffmpeg.c| 32 ++--
fftools/ffmpeg.h| 3 +-
fftools/ffmpeg_demux.c | 108 ++--
fftools/ffmpeg_filter.c | 5 +-
4 files changed, 58 insertions(+), 90 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index cdb16ef90b..038649d9b5 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -1030,9 +1030,6 @@ static int check_keyboard_interaction(int64_t cur_time)
static void reset_eagain(void)
{
-int i;
-for (i = 0; i < nb_input_files; i++)
-input_files[i]->eagain = 0;
for (OutputStream *ost = ost_iter(NULL); ost; ost = ost_iter(ost))
ost->unavailable = 0;
}
@@ -1056,19 +1053,14 @@ static void decode_flush(InputFile *ifile)
* this function should be called again
* - AVERROR_EOF -- this function should not be called again
*/
-static int process_input(int file_index)
+static int process_input(int file_index, AVPacket *pkt)
{
InputFile *ifile = input_files[file_index];
InputStream *ist;
-AVPacket *pkt;
int ret, i;
-ret = ifile_get_packet(ifile, &pkt);
+ret = ifile_get_packet(ifile, pkt);
-if (ret == AVERROR(EAGAIN)) {
-ifile->eagain = 1;
-return ret;
-}
if (ret == 1) {
/* the input file is looped: flush the decoders */
decode_flush(ifile);
@@ -1115,7 +1107,7 @@ static int process_input(int file_index)
ret = process_input_packet(ist, pkt, 0);
-av_packet_free(&pkt);
+av_packet_unref(pkt);
return ret < 0 ? ret : 0;
}
@@ -1125,7 +1117,7 @@ static int process_input(int file_index)
*
* @return 0 for success, <0 for error
*/
-static int transcode_step(OutputStream *ost)
+static int transcode_step(OutputStream *ost, AVPacket *demux_pkt)
{
InputStream *ist = NULL;
int ret;
@@ -1140,10 +1132,8 @@ static int transcode_step(OutputStream *ost)
av_assert0(ist);
}
-ret = process_input(ist->file_index);
+ret = process_input(ist->file_index, demux_pkt);
if (ret == AVERROR(EAGAIN)) {
-if (input_files[ist->file_index]->eagain)
-ost->unavailable = 1;
return 0;
}
@@ -1169,12 +1159,19 @@ static int transcode(int *err_rate_exceeded)
int ret = 0, i;
InputStream *ist;
int64_t timer_start;
+AVPacket *demux_pkt = NULL;
print_stream_maps();
*err_rate_exceeded = 0;
atomic_store(&transcode_init_done, 1);
+demux_pkt = av_packet_alloc();
+if (!demux_pkt) {
+ret = AVERROR(ENOMEM);
+goto fail;
+}
+
if (stdin_interaction) {
av_log(NULL, AV_LOG_INFO, "Press [q] to stop, [?] for help\n");
}
@@ -1202,7 +1199,7 @@ static int transcode(int *err_rate_exceeded)
break;
}
-ret = transcode_step(ost);
+ret = transcode_step(ost, demux_pkt);
if (ret < 0 && ret != AVERROR_EOF) {
av_log(NULL, AV_LOG_ERROR, "Error while filtering: %s\n",
av_err2str(ret));
break;
@@ -1243,6 +1240,9 @@ static int transcode(int *err_rate_exceeded)
/* dump report by using the first video and audio streams */
print_report(1, timer_start, av_gettime_relative());
+fail:
+av_packet_free(&demux_pkt);
+
return ret;
}
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 9852df8320..8de91ab85a 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -407,7 +407,6 @@ typedef struct InputFile {
AVFormatContext *ctx;
int eof_reached; /* true if eof reached */
-int eagain; /* true if last read attempt returned EAGAIN */
int64_t input_ts_offset;
int input_sync_ref;
/**
@@ -857,7 +856,7 @@ void ifile_close(InputFile **f);
* caller should flush decoders and read from this demuxer again
* - a negative error code on failure
*/
-int ifile_get_packet(InputFile *f, AVPacket **pkt);
+int ifile_get_packet(InputFile *f, AVPacket *pkt);
int ist_output_add(InputStream *ist, OutputStream *ost);
int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple);
diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c
index 791952f120..65a5e08ca5 100644
--- a/fftools/ffmpeg_demux.c
+++ b/fftools/ffmpeg_demux.c
@@ -21,6 +21,8 @@
#include "ffmpeg.h"
#include "ffmpeg_utils.h"
+#include "objpool.h"
+#include "thread_queue.h"
#include "libavutil/avassert.h"
#include "libavutil/avstring.h"
@@ -33,7 +35,6 @@
#include "libavutil/time.h"
#include "libavutil/timestamp.h"
#include "libavutil/thread.h"
-#include "libavutil/threadmessage.h
[FFmpeg-devel] [PATCH 07/24] fftools/ffmpeg: rework keeping track of file duration for -stream_loop
Current code tracks min/max pts for each stream separately; then when
the file ends it combines them with last frame's duration to compute the
total duration of each stream; finally it selects the longest of those
durations as the file duration.
This is incorrect - the total file duration is the largest timestamp
difference between any frames, regardless of the stream.
Also change the way the last frame information is reported from decoders
to the muxer - previously it would be just the last frame's duration,
now the end timestamp is sent, which is simpler.
Changes the result of the fate-ffmpeg-streamloop-transcode-av test,
where the timestamps are shifted slightly forward. Note that the
matroska demuxer does not return the first audio packet after seeking
(due to buggy interaction betwen the generic code and the demuxer), so
there is a gap in audio.
---
fftools/ffmpeg.h | 13 +-
fftools/ffmpeg_dec.c | 16 +-
fftools/ffmpeg_demux.c| 113 +
fftools/ffmpeg_utils.h| 6 +
tests/ref/fate/ffmpeg-streamloop-transcode-av | 160 +-
5 files changed, 141 insertions(+), 167 deletions(-)
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index d52c954df5..41935d39d5 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -347,8 +347,6 @@ typedef struct InputStream {
AVRational framerate_guessed;
-int64_t nb_samples; /* number of samples in the last decoded audio frame
before looping */
-
AVDictionary *decoder_opts;
AVRational framerate; /* framerate forced with -r */
#if FFMPEG_OPT_TOP
@@ -391,11 +389,6 @@ typedef struct InputStream {
uint64_t decode_errors;
} InputStream;
-typedef struct LastFrameDuration {
-int stream_idx;
-int64_t duration;
-} LastFrameDuration;
-
typedef struct InputFile {
const AVClass *class;
@@ -427,9 +420,9 @@ typedef struct InputFile {
int accurate_seek;
/* when looping the input file, this queue is used by decoders to report
- * the last frame duration back to the demuxer thread */
-AVThreadMessageQueue *audio_duration_queue;
-int audio_duration_queue_size;
+ * the last frame timestamp back to the demuxer thread */
+AVThreadMessageQueue *audio_ts_queue;
+int audio_ts_queue_size;
} InputFile;
enum forced_keyframes_const {
diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index 8795a94c1a..517d6b3ced 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -632,7 +632,6 @@ static int packet_decode(InputStream *ist, AVPacket *pkt,
AVFrame *frame)
if (dec->codec_type == AVMEDIA_TYPE_AUDIO) {
ist->samples_decoded += frame->nb_samples;
-ist->nb_samples = frame->nb_samples;
audio_ts_process(ist, ist->decoder, frame);
} else {
@@ -724,14 +723,9 @@ static void *decoder_thread(void *arg)
/* report last frame duration to the demuxer thread */
if (ist->dec->type == AVMEDIA_TYPE_AUDIO) {
-LastFrameDuration dur;
-
-dur.stream_idx = ist->index;
-dur.duration = av_rescale_q(ist->nb_samples,
- (AVRational){ 1,
ist->dec_ctx->sample_rate},
- ist->st->time_base);
-
-av_thread_message_queue_send(ifile->audio_duration_queue,
&dur, 0);
+Timestamp ts = { .ts = d->last_frame_pts +
d->last_frame_duration_est,
+ .tb = d->last_frame_tb };
+av_thread_message_queue_send(ifile->audio_ts_queue, &ts, 0);
}
avcodec_flush_buffers(ist->dec_ctx);
@@ -760,8 +754,8 @@ finish:
// make sure the demuxer does not get stuck waiting for audio durations
// that will never arrive
-if (ifile->audio_duration_queue && ist->dec->type == AVMEDIA_TYPE_AUDIO)
-av_thread_message_queue_set_err_recv(ifile->audio_duration_queue,
AVERROR_EOF);
+if (ifile->audio_ts_queue && ist->dec->type == AVMEDIA_TYPE_AUDIO)
+av_thread_message_queue_set_err_recv(ifile->audio_ts_queue,
AVERROR_EOF);
dec_thread_uninit(&dt);
diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c
index ec96daf26b..791952f120 100644
--- a/fftools/ffmpeg_demux.c
+++ b/fftools/ffmpeg_demux.c
@@ -74,9 +74,6 @@ typedef struct DemuxStream {
///< dts of the last packet read for this stream (in AV_TIME_BASE units)
int64_t dts;
-int64_t min_pts; /* pts with the smallest value in a current stream */
-int64_t max_pts; /* pts with the higher value in a current stream */
-
/* number of packets successfully read for this stream */
uint64_t nb_packets;
// combined size of all the packets read
@@ -99,11 +96,11 @@ typedef struct Demuxer {
/* number of times input stream shoul
[FFmpeg-devel] [PATCH 10/24] fftools/ffmpeg_filter: move filtering to a separate thread
As previously for decoding, this is merely "scaffolding" for moving to a
fully threaded architecture and does not yet make filtering truly
parallel - the main thread will currently wait for the filtering thread
to finish its work before continuing. That will change in future commits
after encoders are also moved to threads and a thread-aware scheduler is
added.
---
fftools/ffmpeg.h| 9 +-
fftools/ffmpeg_dec.c| 39 +-
fftools/ffmpeg_filter.c | 825 ++--
3 files changed, 730 insertions(+), 143 deletions(-)
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 41935d39d5..9852df8320 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -80,6 +80,14 @@ enum HWAccelID {
HWACCEL_GENERIC,
};
+enum FrameOpaque {
+FRAME_OPAQUE_REAP_FILTERS = 1,
+FRAME_OPAQUE_CHOOSE_INPUT,
+FRAME_OPAQUE_SUB_HEARTBEAT,
+FRAME_OPAQUE_EOF,
+FRAME_OPAQUE_SEND_COMMAND,
+};
+
typedef struct HWDevice {
const char *name;
enum AVHWDeviceType type;
@@ -728,7 +736,6 @@ FrameData *frame_data(AVFrame *frame);
int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame, int
keep_reference);
int ifilter_send_eof(InputFilter *ifilter, int64_t pts, AVRational tb);
-int ifilter_sub2video(InputFilter *ifilter, const AVFrame *frame);
void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational
tb);
/**
diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index 517d6b3ced..b60bad1220 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -147,11 +147,12 @@ fail:
static int send_frame_to_filters(InputStream *ist, AVFrame *decoded_frame)
{
-int i, ret;
+int i, ret = 0;
-av_assert1(ist->nb_filters > 0); /* ensure ret is initialized */
for (i = 0; i < ist->nb_filters; i++) {
-ret = ifilter_send_frame(ist->filters[i], decoded_frame, i <
ist->nb_filters - 1);
+ret = ifilter_send_frame(ist->filters[i], decoded_frame,
+ i < ist->nb_filters - 1 ||
+ ist->dec->type == AVMEDIA_TYPE_SUBTITLE);
if (ret == AVERROR_EOF)
ret = 0; /* ignore */
if (ret < 0) {
@@ -380,15 +381,6 @@ static int video_frame_process(InputStream *ist, AVFrame
*frame)
return 0;
}
-static void sub2video_flush(InputStream *ist)
-{
-for (int i = 0; i < ist->nb_filters; i++) {
-int ret = ifilter_sub2video(ist->filters[i], NULL);
-if (ret != AVERROR_EOF && ret < 0)
-av_log(NULL, AV_LOG_WARNING, "Flush the frame error.\n");
-}
-}
-
static int process_subtitle(InputStream *ist, AVFrame *frame)
{
Decoder *d = ist->decoder;
@@ -426,14 +418,9 @@ static int process_subtitle(InputStream *ist, AVFrame
*frame)
if (!subtitle)
return 0;
-for (int i = 0; i < ist->nb_filters; i++) {
-ret = ifilter_sub2video(ist->filters[i], frame);
-if (ret < 0) {
-av_log(ist, AV_LOG_ERROR, "Error sending a subtitle for filtering:
%s\n",
- av_err2str(ret));
-return ret;
-}
-}
+ret = send_frame_to_filters(ist, frame);
+if (ret < 0)
+return ret;
subtitle = (AVSubtitle*)frame->buf[0]->data;
if (!subtitle->num_rects)
@@ -824,14 +811,10 @@ finish:
return ret;
// signal EOF to our downstreams
-if (ist->dec->type == AVMEDIA_TYPE_SUBTITLE)
-sub2video_flush(ist);
-else {
-ret = send_filter_eof(ist);
-if (ret < 0) {
-av_log(NULL, AV_LOG_FATAL, "Error marking filters as finished\n");
-return ret;
-}
+ret = send_filter_eof(ist);
+if (ret < 0) {
+av_log(NULL, AV_LOG_FATAL, "Error marking filters as finished\n");
+return ret;
}
return AVERROR_EOF;
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 4edf634b26..384cdedcd0 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -21,6 +21,8 @@
#include
#include "ffmpeg.h"
+#include "ffmpeg_utils.h"
+#include "thread_queue.h"
#include "libavfilter/avfilter.h"
#include "libavfilter/buffersink.h"
@@ -53,12 +55,50 @@ typedef struct FilterGraphPriv {
int is_meta;
int disable_conversions;
+int nb_inputs_bound;
+int nb_outputs_bound;
+
const char *graph_desc;
// frame for temporarily holding output from the filtergraph
AVFrame *frame;
// frame for sending output to the encoder
AVFrame *frame_enc;
+
+pthread_tthread;
+/**
+ * Queue for sending frames from the main thread to the filtergraph. Has
+ * nb_inputs+1 streams - the first nb_inputs stream correspond to
+ * filtergraph inputs. Frames on those streams may have their opaque set to
+ * - FRAME_OPAQUE_EOF: frame contains no data, but pts+timebase of the
+ * EOF event for the correspondint stream. Will be immediately followed
by
+ * this stream being send-closed.
+ * - F
[FFmpeg-devel] [PATCH 09/24] fftools/ffmpeg_filter: track input/output index in {Input, Output}FilterPriv
Will be useful in following commits.
---
fftools/ffmpeg_filter.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index d48974581b..4edf634b26 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -74,6 +74,8 @@ static const FilterGraphPriv *cfgp_from_cfg(const FilterGraph
*fg)
typedef struct InputFilterPriv {
InputFilter ifilter;
+int index;
+
AVFilterContext *filter;
InputStream *ist;
@@ -162,6 +164,8 @@ typedef struct FPSConvContext {
typedef struct OutputFilterPriv {
OutputFilterofilter;
+int index;
+
AVFilterContext*filter;
/* desired output stream properties */
@@ -594,6 +598,7 @@ static OutputFilter *ofilter_alloc(FilterGraph *fg)
ofilter = &ofp->ofilter;
ofilter->graph= fg;
ofp->format = -1;
+ofp->index= fg->nb_outputs - 1;
ofilter->last_pts = AV_NOPTS_VALUE;
return ofilter;
@@ -787,6 +792,7 @@ static InputFilter *ifilter_alloc(FilterGraph *fg)
if (!ifp->frame)
return NULL;
+ifp->index = fg->nb_inputs - 1;
ifp->format = -1;
ifp->fallback.format = -1;
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] ffmpeg CLI multithreading
Hi, after ~2 years of work and ~700+ preparatory commits, here is finally the first "fully functional" version of multithreaded ffmpeg CLI. In quotes because due to the scale of the changes I'm sure some things got broken and I didn't notice - more testing very much welcome. One thing which is most definitely broken is the -fix_sub_duration_heartbeat option, which in its current form makes assumptions about synchronization between distant components (encoders/muxers -> decoders), which makes it fundamentally non-deterministic when each of these components runs asynchronously (note that its behaviour is unpredictable even now, it's just deterministic across runs with the same options). I'm currently disabling the option in 16/24, better suggestions on what to do with it are welcome. Runtime overhead of the threading seems to be negligible in typical cases, though it may become significant when there is very little work per packet. You should see significantly better CPU utilization/wallclock speedup on multicore systems whenever transcoding isn't dominated by a a single component and the components aren't themselves already multithreaded. The set is structured similarly to the previous RFC: * 01-09/24 are preparatory fixes and could be pushed on their own well before the rest of the series; thanks to Paul for writing 02/24 * 10-17/24 move the two remaining components (encoding and filtering) into threads (as before it's "fake" threading where the main thread waits for the other thread to be done, and is thus not truly parallel) and do other preparatory changes - these may introduce significant temporary overhead and/or break some corner cases, so should be pushed when the whole set is ready; * 18/24, the biggest individual patch, adds the transcode scheduler; as of that patch it does not yet do anything; * 19-24/24 convert the individual components to use the scheduler; the conversion is split for review purposes, but will have to be squashed for the final push. Some more information is in my recent VDD talk [1]. You can also fetch the code from the 'ffmpeg_threading' branch in [2]. Reviews, testing, and constructive comments are all very much welcome. [1] https://youtu.be/Z4DS3jiZhfo?t=1221 [2] git://git.khirnov.net/libav Cheers, -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 08/24] fftools/ffmpeg_filter: remove an unnecessary sub2video_push_ref() call
It only seems to produce duplicate frames. --- fftools/ffmpeg_filter.c | 3 --- tests/ref/fate/sub2video_basic| 3 --- tests/ref/fate/sub2video_time_limited | 1 - 3 files changed, 7 deletions(-) diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index c738fc3397..d48974581b 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -2266,9 +2266,6 @@ void ifilter_sub2video_heartbeat(InputFilter *ifilter, int64_t pts, AVRational t or if we need to initialize the system, update the overlayed subpicture and its start/end times */ sub2video_update(ifp, pts2 + 1, NULL); - -if (av_buffersrc_get_nb_failed_requests(ifp->filter)) -sub2video_push_ref(ifp, pts2); } int ifilter_sub2video(InputFilter *ifilter, const AVFrame *frame) diff --git a/tests/ref/fate/sub2video_basic b/tests/ref/fate/sub2video_basic index a6eb1a34ea..314b7a07dd 100644 --- a/tests/ref/fate/sub2video_basic +++ b/tests/ref/fate/sub2video_basic @@ -12,7 +12,6 @@ 0, 183357, 183357,0, 1382400, 0x 0, 183433, 183433,0, 1382400, 0x85547fd1 0, 185799, 185799,0, 1382400, 0x -0, 185909, 185909,0, 1382400, 0x 0, 185910, 185910,0, 1382400, 0xb6a8f181 0, 188606, 188606,0, 1382400, 0x 0, 188663, 188663,0, 1382400, 0xb64d1a2c @@ -59,7 +58,6 @@ 0, 296776, 296776,0, 1382400, 0x 0, 300049, 300049,0, 1382400, 0xaf08b10d 0, 301949, 301949,0, 1382400, 0x -0, 302034, 302034,0, 1382400, 0x 0, 302035, 302035,0, 1382400, 0x853a9d93 0, 303559, 303559,0, 1382400, 0x 0, 304203, 304203,0, 1382400, 0x7491a87d @@ -76,7 +74,6 @@ 0, 326403, 326403,0, 1382400, 0x 0, 327193, 327193,0, 1382400, 0x35b85f2e 0, 328285, 328285,0, 1382400, 0x -0, 328360, 328360,0, 1382400, 0x 0, 328361, 328361,0, 1382400, 0x83f103e5 0, 329885, 329885,0, 1382400, 0x 0, 329946, 329946,0, 1382400, 0xbc1ca9b3 diff --git a/tests/ref/fate/sub2video_time_limited b/tests/ref/fate/sub2video_time_limited index c7d48d639f..0634d5857e 100644 --- a/tests/ref/fate/sub2video_time_limited +++ b/tests/ref/fate/sub2video_time_limited @@ -5,4 +5,3 @@ #sar 0: 0/1 0, 6072, 6072,0, 8294400, 0x 0, 6072, 6072,0, 8294400, 0xa87c518f -0, 36101, 36101,0, 8294400, 0xa87c518f -- 2.42.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 05/24] fftools/thread_queue: do not return elements for receive-finished streams
It does not cause any issues in current callers, but still should not
happen.
---
fftools/thread_queue.c | 17 +
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/fftools/thread_queue.c b/fftools/thread_queue.c
index a1ab4ce92e..feac6a7748 100644
--- a/fftools/thread_queue.c
+++ b/fftools/thread_queue.c
@@ -164,7 +164,12 @@ static int receive_locked(ThreadQueue *tq, int *stream_idx,
FifoElem elem;
unsigned int nb_finished = 0;
-if (av_fifo_read(tq->fifo, &elem, 1) >= 0) {
+while (av_fifo_read(tq->fifo, &elem, 1) >= 0) {
+if (tq->finished[elem.stream_idx] & FINISHED_RECV) {
+objpool_release(tq->obj_pool, &elem.obj);
+continue;
+}
+
tq->obj_move(data, elem.obj);
objpool_release(tq->obj_pool, &elem.obj);
*stream_idx = elem.stream_idx;
@@ -197,7 +202,14 @@ int tq_receive(ThreadQueue *tq, int *stream_idx, void
*data)
pthread_mutex_lock(&tq->lock);
while (1) {
+size_t can_read = av_fifo_can_read(tq->fifo);
+
ret = receive_locked(tq, stream_idx, data);
+
+// signal other threads if the fifo state changed
+if (can_read != av_fifo_can_read(tq->fifo))
+pthread_cond_broadcast(&tq->cond);
+
if (ret == AVERROR(EAGAIN)) {
pthread_cond_wait(&tq->cond, &tq->lock);
continue;
@@ -206,9 +218,6 @@ int tq_receive(ThreadQueue *tq, int *stream_idx, void *data)
break;
}
-if (ret == 0)
-pthread_cond_broadcast(&tq->cond);
-
pthread_mutex_unlock(&tq->lock);
return ret;
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 01/24] lavf/mux: do not apply max_interleave_delta to subtitles
It is common for subtitle streams to have large gaps between packets. When the caller is interleaving packets from multiple files, it can easily happen that two successive subtitle packets trigger this limit, even though no excessive buffering is happening. --- libavformat/mux.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavformat/mux.c b/libavformat/mux.c index c7877c5d98..de10d2c008 100644 --- a/libavformat/mux.c +++ b/libavformat/mux.c @@ -995,7 +995,7 @@ int ff_interleave_packet_per_dts(AVFormatContext *s, AVPacket *pkt, const PacketListEntry *const last = sti->last_in_packet_buffer; int64_t last_dts; -if (!last) +if (!last || st->codecpar->codec_type == AVMEDIA_TYPE_SUBTITLE) continue; last_dts = av_rescale_q(last->pkt.dts, -- 2.42.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 03/24] lavc/8bps: fix exporting palette after 63767b79a570404628b2521b83104108b7b6884c
It would be left empty on each frame whose packet does not come with
palette attached.
---
libavcodec/8bps.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/libavcodec/8bps.c b/libavcodec/8bps.c
index 0becaa9320..11b9f526b7 100644
--- a/libavcodec/8bps.c
+++ b/libavcodec/8bps.c
@@ -43,6 +43,8 @@ typedef struct EightBpsContext {
uint8_t planes;
uint8_t planemap[4];
+
+uint32_t pal[256];
} EightBpsContext;
static int decode_frame(AVCodecContext *avctx, AVFrame *frame,
@@ -116,10 +118,12 @@ static int decode_frame(AVCodecContext *avctx, AVFrame
*frame,
FF_DISABLE_DEPRECATION_WARNINGS
frame->palette_has_changed =
#endif
-ff_copy_palette(frame->data[1], avpkt, avctx);
+ff_copy_palette(c->pal, avpkt, avctx);
#if FF_API_PALETTE_HAS_CHANGED
FF_ENABLE_DEPRECATION_WARNINGS
#endif
+
+memcpy(frame->data[1], c->pal, AVPALETTE_SIZE);
}
*got_frame = 1;
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 04/24] fftools/ffmpeg: move a few inline function into a new header
Will allow to use them in future commits without including the whole
ffmpeg.h.
---
fftools/ffmpeg.c | 1 +
fftools/ffmpeg.h | 21 --
fftools/ffmpeg_dec.c | 1 +
fftools/ffmpeg_demux.c | 1 +
fftools/ffmpeg_mux.c | 1 +
fftools/ffmpeg_utils.h | 50 ++
6 files changed, 54 insertions(+), 21 deletions(-)
create mode 100644 fftools/ffmpeg_utils.h
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 46a85b41a8..cdb16ef90b 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -99,6 +99,7 @@
#include "cmdutils.h"
#include "ffmpeg.h"
+#include "ffmpeg_utils.h"
#include "sync_queue.h"
const char program_name[] = "ffmpeg";
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index 0983d026cd..d52c954df5 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -880,17 +880,6 @@ int trigger_fix_sub_duration_heartbeat(OutputStream *ost,
const AVPacket *pkt);
int fix_sub_duration_heartbeat(InputStream *ist, int64_t signal_pts);
void update_benchmark(const char *fmt, ...);
-/**
- * Merge two return codes - return one of the error codes if at least one of
- * them was negative, 0 otherwise.
- * Currently just picks the first one, eventually we might want to do something
- * more sophisticated, like sorting them by priority.
- */
-static inline int err_merge(int err0, int err1)
-{
-return (err0 < 0) ? err0 : FFMIN(err1, 0);
-}
-
#define SPECIFIER_OPT_FMT_str "%s"
#define SPECIFIER_OPT_FMT_i"%i"
#define SPECIFIER_OPT_FMT_i64 "%"PRId64
@@ -942,14 +931,4 @@ extern const char * const opt_name_frame_rates[];
extern const char * const opt_name_top_field_first[];
#endif
-static inline void pkt_move(void *dst, void *src)
-{
-av_packet_move_ref(dst, src);
-}
-
-static inline void frame_move(void *dst, void *src)
-{
-av_frame_move_ref(dst, src);
-}
-
#endif /* FFTOOLS_FFMPEG_H */
diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c
index fcee8b65ac..8795a94c1a 100644
--- a/fftools/ffmpeg_dec.c
+++ b/fftools/ffmpeg_dec.c
@@ -30,6 +30,7 @@
#include "libavfilter/buffersrc.h"
#include "ffmpeg.h"
+#include "ffmpeg_utils.h"
#include "thread_queue.h"
struct Decoder {
diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c
index 350f233ab7..ec96daf26b 100644
--- a/fftools/ffmpeg_demux.c
+++ b/fftools/ffmpeg_demux.c
@@ -20,6 +20,7 @@
#include
#include "ffmpeg.h"
+#include "ffmpeg_utils.h"
#include "libavutil/avassert.h"
#include "libavutil/avstring.h"
diff --git a/fftools/ffmpeg_mux.c b/fftools/ffmpeg_mux.c
index 7a924dba6c..30c033036d 100644
--- a/fftools/ffmpeg_mux.c
+++ b/fftools/ffmpeg_mux.c
@@ -22,6 +22,7 @@
#include "ffmpeg.h"
#include "ffmpeg_mux.h"
+#include "ffmpeg_utils.h"
#include "objpool.h"
#include "sync_queue.h"
#include "thread_queue.h"
diff --git a/fftools/ffmpeg_utils.h b/fftools/ffmpeg_utils.h
new file mode 100644
index 00..20cde94969
--- /dev/null
+++ b/fftools/ffmpeg_utils.h
@@ -0,0 +1,50 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#ifndef FFTOOLS_FFMPEG_UTILS_H
+#define FFTOOLS_FFMPEG_UTILS_H
+
+#include
+
+#include "libavutil/common.h"
+#include "libavutil/frame.h"
+
+#include "libavcodec/packet.h"
+
+/**
+ * Merge two return codes - return one of the error codes if at least one of
+ * them was negative, 0 otherwise.
+ * Currently just picks the first one, eventually we might want to do something
+ * more sophisticated, like sorting them by priority.
+ */
+static inline int err_merge(int err0, int err1)
+{
+return (err0 < 0) ? err0 : FFMIN(err1, 0);
+}
+
+static inline void pkt_move(void *dst, void *src)
+{
+av_packet_move_ref(dst, src);
+}
+
+static inline void frame_move(void *dst, void *src)
+{
+av_frame_move_ref(dst, src);
+}
+
+#endif // FFTOOLS_FFMPEG_UTILS_H
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 02/24] lavfi/af_amix: make sure the output does not depend on input ordering
From: Paul B Mahol
Signed-off-by: Anton Khirnov
---
libavfilter/af_amix.c | 13 ++---
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/libavfilter/af_amix.c b/libavfilter/af_amix.c
index aa42514106..120a97f48c 100644
--- a/libavfilter/af_amix.c
+++ b/libavfilter/af_amix.c
@@ -394,6 +394,8 @@ static int request_samples(AVFilterContext *ctx, int
min_samples)
int i;
av_assert0(s->nb_inputs > 1);
+if (min_samples == 1 && s->duration_mode == DURATION_FIRST)
+min_samples = av_audio_fifo_size(s->fifos[0]);
for (i = 1; i < s->nb_inputs; i++) {
if (!(s->input_state[i] & INPUT_ON) ||
@@ -402,6 +404,7 @@ static int request_samples(AVFilterContext *ctx, int
min_samples)
if (av_audio_fifo_size(s->fifos[i]) >= min_samples)
continue;
ff_inlink_request_frame(ctx->inputs[i]);
+return 0;
}
return output_frame(ctx->outputs[0]);
}
@@ -471,17 +474,13 @@ static int activate(AVFilterContext *ctx)
if (ff_inlink_acknowledge_status(ctx->inputs[i], &status, &pts)) {
if (status == AVERROR_EOF) {
-if (i == 0) {
-s->input_state[i] = 0;
+s->input_state[i] |= INPUT_EOF;
+if (av_audio_fifo_size(s->fifos[i]) == 0) {
+s->input_state[i] &= ~INPUT_ON;
if (s->nb_inputs == 1) {
ff_outlink_set_status(outlink, status, pts);
return 0;
}
-} else {
-s->input_state[i] |= INPUT_EOF;
-if (av_audio_fifo_size(s->fifos[i]) == 0) {
-s->input_state[i] = 0;
-}
}
}
}
--
2.42.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
