Hi Lynne On Fri, Mar 20, 2026 at 12:06:55AM +0100, Lynne via ffmpeg-devel wrote: > On 19/03/2026 22:52, Michael Niedermayer via ffmpeg-devel wrote: [...] > > > > > > And if that is true, what can we do to make Bayer coding fit into FFv1's > > > architecture? > > > > > I think given those constraints, we can put a limit on what we could do. > > > > > > Given this, would you agree that needing a linecache larger than what we > > > currently use is not acceptable? > > > > "larger than what we currently use" is a bit too strict > > > > I think > > we want all memory needs to be minimal (but its not the most important goal) > > Addditional memory needs would require proportional gains elsewhere > > I think I understand memory limits a bit more clearly, since I had to work > with the limitations that Vulkan exposed.
certainly but while vulkan is important, the design of ffv1 bayer should not be guided only by vulkan. > Requiring extra memory implies more copies, which means more bandwidth. FFv1 > is already very bandwidth-starved, as decoding each sample means needing to > do 6 loads from a potentially massive backing buffer. > > Seeing as I need to also create a Vulkan version of the algorithm, I would > very much prefer to avoid needing to increase the buffer size. ATM with RGBA you need 2 lines * 4 planes = 8 and 4 range coders with their whole state the current PR IIRC requires 4 lines and 4 range coders with full state. my suggestion to not split the green plane would reduce that to 3 range coders with their state and also a reduction of the lines needed as the single green place needs less lines than 2 interleaved ones. Sadly i dont have the time ATM to properly read all the papers but There are some apparent low complexity transforms like the star tetrix transforms https://sci-hub.box/10.1109/TIP.2021.3095421 This here compares it for lossless images with some other transforms https://www.cs.tsukuba.ac.jp/~taizo/pdf/journal/202209_TIP.pdf Also advances in color image demosaicing (not compression specific IIUC) can be seen in this series of papers, maybe some ideas can be used from these ? https://sci-hub.box/10.1109/TIP.2016.2518082 MLRI (compares 12 algorithms) https://www.mdpi.com/1424-8220/17/12/2787 ARI (compares MANY algorithms) More recent (2019 & 2025 comparissions of demosaicing algorithms these show that NN beats non NN) https://www.ipol.im/pub/art/2019/274/article_lr.pdf https://www.researchgate.net/publication/396809360_Color_Image_Demosaicking_A_Systematic_Survey_of_Algorithms_Performance_Evaluation_and_Open_Challenges > > > > I expect that we will end with a O(slice_width) and not O(slice_width * > > slice_height) > > but i dont want to discard anything without knowing what iam discarding > > > > > > > > > > > > > > We need to see how alternatives perform > > > > But what is clear is that this will not be resolved in 2 weeks > > > > > > I can certainly make comparisons between other coding method alternatives, > > > as long as they are a reasonable fit for the codec. > > > > great > > > > > > [...] > > > > > > > I will prepare performance and compression figures too, of course. > > > > > > > > that would be very good > > > > > > > > > > > > > > > > > > You are mainly focusing on the use of Haar to decompose Bayer into > > > > > median + > > > > > difference for the green samples. Would you prefer a more general > > > > > interpolation or predictor instead? Like I mentioned, I wouldn't mind > > > > > this, > > > > > though it would be better if it benefitted RGB coding as well. > > > > > > > > My concern is not about a specific part of this, rather that we dont > > > > know how the surroundings of the algorithm landscape perform. > > > > > > > > We dont even know if this choice is a local minimum in the landscape > > > > of algorithms. Maybe we can make a small change and it will perform > > > > better. > > > > Maybe we can make a larger change and we would be in a much deeper local > > > > minimum. > > > > > > > > What iam asking for is that the landscape of choices / algorithms is > > > > investigated more before we settle on what choice FFv1 would use for > > > > decades > > > > > That's good, but you should keep in mind that this is an unstable feature > > > that can be changed in the future, rather than the final design. > > > > I have to push back on this > > the STF project says > > "Completing this milestone will expand FFv1's archival capabilities to > > handle not just produced video, but also direct camera footage, as well as > > film scanners and scientific applications." > > > > I dont think "an unstable feature" can be used for archival. > > > > archival requires features to be long term supported and stable > > All new features need to be introduced as unstable features until version 4 > of the codec is released. It is implied that this extension to the codec > will also go through the same procedure. > > I think that merging support for it (or approving the PR) is enough to > satisfy the requirement of the milestone, rather than standardization. > The feature would be in the next version of the codec whenever it is frozen > and specified. The formal IETF standardization isnt the same thing as designing a feature that we _believe_ will not need to be changed For example te float compression i designed to the best of my knowledge at that time was in a local minimum. I tried finding better ways to store floats in the FFv1 design (tried many differennt ways to store floats, many diffeerenet context models, quantziation methods and i found nothing short of things that should go to a FFv2 I also belive i asked on ffmpeg-devel and IETF-cellar about float ideas in that sense you maybe want to ask on IETF-cellar about bayer CFA coding ideas before this is finalized. and thats why i said i dont think this can be done in the timeline for the current STF milestones I mean asking on IETF-cellar then waiting and discussing and considering any ideas brought up will take some time > > > > > I think starting with a simple but proven design, and improving it, is the > > > most optimal path. It's how all other codecs were developed, including > > > AV1. > > > > sure but when we have bayer support behind "-strict -2" and change the > > algorithm > > over time then it cant really be used by users. Its more for developers > > colaborating > > That's my point. I found some ways that floatmap can be optimized, and we > can collaborate to improve it. I don't want to be the single person who can > touch Bayer coding. > > Right now, users who want to encode floating-point images losslessly via > FFv1 also have to use -strict -2 -level 4, and that's okay. yes, and we can change floatmap still if theres something that improves it so can we with bayer but writing well optimized CPU and GPU implementations should happen ideally after we have settled on a design because otherwise someone will have to keep changing the implementations for both CPU and GPU thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety -- Benjamin Franklin
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
