On Fri, Apr 19, 2024 at 1:19 AM Michael Niedermayer <mich...@niedermayer.cc> wrote:
> On Thu, Apr 18, 2024 at 06:13:40PM -0300, James Almer wrote: > > On 4/18/2024 5:53 PM, Michael Niedermayer wrote: > > > On Thu, Apr 18, 2024 at 04:02:07PM +0200, Niklas Haas wrote: > > > > On Wed, 17 Apr 2024 15:58:32 +0200 Michael Niedermayer < > mich...@niedermayer.cc> wrote: > > > > > Hi all > > > > > > > > > > The pace of inovation in FFmpeg has been slowing down. > > > > > Most work is concentarted nowadays on code refactoring, and adding > > > > > support for new codecs and formats. > > > > > > > > > > Should we > > > > > * make a list of longer term goals > > > > > * vote on them > > > > > * and then together work towards implementing them > > > > > ? > > > > > > > > > > (The idea here is to increase the success of larger efforts > > > > > than adding codecs and refactoring code) > > > > > It would then also not be possible for individuals to object > > > > > to a previously agreed goal. > > > > > And it would add ideas for which we can try to get funding/grants > for > > > > > > > > > > (larger scale changes need consensus first that we as a whole want > > > > > them before we would be able to ask for funding/grants for them) > > > > > > > > > > Some ideas and why they would help FFmpeg: > > > > > > > > > > * Switch to a plugin architecture > > > > > (Increase the number of developers willing to contribute and > reduce > > > > > friction as the team and community grows) > > > > > > > > This would almost surely hurt productivity > > > > > > i dont agree, ill elaborae below > > > > > > > > > > as it will require exposing, > > > > versioning, documenting and maintaining a vast number of internal > APIs > > > > > > yes to some extend that is needed. It can be more or less depending > > > on how things are implemented > > > > > > > > > > which we are currently at the liberty of modifying freely. > > > > > > we are modifying these APIs for decades. That modification of APIs > > > their documentation and all code using them costs time. > > > > The AVCodec hooks being internal allowed us to add autoinserted bsfs and > to > > painlessly rewrite the decouple I/O callbacks to work as a pure pull > based > > interface. Were said internals public, that wouldn't have been possible. > > A decoder API needs packet in, frame out. > AVPacket and AVFrame are public. > (and a bunch of key-value data like width / height / timebase / > pixelformat / aspect / ... > teh struct for that is also public since a long time) > I dont see the problem. > > you have the decoder API facing the user, that causes no problems, > i dont agree that having a decoder API (or encoder, muxer, demuxer, ...) > facing a plugin would be a bigger problem than the APIs we already have > public > sure there are details, sure there are things that need one to think about > and sleep over and all that but these are from a high level point of > view simple and also the same interfaces to what we already have public > > > > > > > > > > More so we have issues with patch-management. And i claim this is > > > not the mailing list but a lack of time and in some cases a lack of > > > interrest in many areas. > > > > > > A plugin system moves this patch-management to people who actually > > > care, that is the authors of the codecs and (de)muxers. > > > > > > Our productivity as is, is not good, many patches are ignored. > > > > A lot of patches fall through the cracks rather than being ignored. > > There are people that send patchsets unthreaded (Because of misconfigured > > git send-email i suppose), and it makes browsing your mailbox hard. > > well i can say that i dont review many patches anymore because i just dont > have > the time, its not that i cant keep track of what i wanted to review. > > either i make a note in a TODO list or i keep the patch marked as NEW > in my mail user agent. > > trac in some sense or patchwork are just more public TODO lists > that can be shared between people so if one doesnt do it another > developer sees it and can do it. > > > > > > > The people caring about these patches are their Authors and yet they > > > are powerless as they must sometimes wait many months for reviews > > > Besides that, developers are leaving for various reasons and they > > > are forced to setup full forks not being able to maintain their > > > code in any other way. > > > IMO A plugin system would improve productivity as everyone could work > > > on their own terms. > > > > You say the ML is not the problem, but it sort of is. An MR based > > development would greatly improve this problem. > > People historically where very opposed to merge requests > > But, having a public git repo (that people already have) > asking for it to be merged. You get a merge commit and someone will > probably > feel offended by that. (thats not what you meant i guess) > but i would 100% support doing git merge requests. > (there are good arguments from people much smarter than me why > merging is better than rebasing) > > > > > > > No week or month long delays and weekly pinging patches > > > No risk about patches being rejected or ignored > > > No need to read every other discussion on the ML. One can just > > > simply work on their own plugin looking just at the API documentation > > > > I don't personally have a strong opinion one way or another on this > subject > > at this moment, but any such approach would need to be carefully thought > and > > designed, to prevent all the potential problems people have expressed so > > far. > > of course this would require carefull design as any public API > would. > > > [...] > > > > > > > > > > * AI / neural network filters and codecs > > > > > The future seems to be AI based. Future Filters and Codecs > will use > > > > > neural networks. FFmpeg can be at the forefront, developing > these > > > > > > > > We already have TensorFlow support, no? (vf_sr etc.) What more is > > > > needed? > > > > > > more of that AND more convenience > > > > > > lets pick a comparission > > > to run fate > > > you write "make fate" > > > if you want to do it with the samples its > > > make fate-rsync ; make fate > > > > > > if you want to use vf_sr, its reading the docs, looking at some > scripts reading their docs > > > and i presume selecting a training set ? creating a model ? .... > > > > By no means we could ever ship a custom AI model for the sake of a "git > > fetch, compile, and run" scenario. This was already a problem when > > tensorflow was first committed. So this can't be avoided. > > I am not sure i understand you, but training a model on lets say the > fate samples or some public domain images. Why would we not be able > to ship that in a similar way to the fate samples ? > > or why not ship a standarized script to build such a model from > user data ? > (i mean by standarized, the same for every filter, like ffmpeg is the same > for every format not link to different papers and random off site scripts) > > > > > > > > > > how many people do that ? > > > > Every external dependency has its documented requirements... > > vf_sr doesnt have a example one can copy and paste that would > work on a fresh checkout of ffmpeg. That alone is a fail IMHO > > Current AI tech is just inflated hype, for both audio and video processing. > thx > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Concerning the gods, I have no means of knowing whether they exist or not > or of what sort they may be, because of the obscurity of the subject, and > the brevity of human life -- Protagoras > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".