On 23/03/2026 18:20, Michael Niedermayer via ffmpeg-devel wrote:
Hi Lynne
On Fri, Mar 20, 2026 at 12:06:55AM +0100, Lynne via ffmpeg-devel wrote:
On 19/03/2026 22:52, Michael Niedermayer via ffmpeg-devel wrote:
[...]
And if that is true, what can we do to make Bayer coding fit into FFv1's
architecture?
I think given those constraints, we can put a limit on what we could do.
Given this, would you agree that needing a linecache larger than what we
currently use is not acceptable?
"larger than what we currently use" is a bit too strict
I think
we want all memory needs to be minimal (but its not the most important goal)
Addditional memory needs would require proportional gains elsewhere
I think I understand memory limits a bit more clearly, since I had to work
with the limitations that Vulkan exposed.
certainly but while vulkan is important, the design of ffv1 bayer should not
be guided only by vulkan.
Requiring extra memory implies more copies, which means more bandwidth. FFv1
is already very bandwidth-starved, as decoding each sample means needing to
do 6 loads from a potentially massive backing buffer.
Seeing as I need to also create a Vulkan version of the algorithm, I would
very much prefer to avoid needing to increase the buffer size.
ATM with RGBA you need 2 lines * 4 planes = 8 and 4 range coders with their
whole state
the current PR IIRC requires 4 lines and 4 range coders with full state.
my suggestion to not split the green plane would reduce that to 3 range coders
with their state
and also a reduction of the lines needed as the single green place needs
less lines than 2 interleaved ones.
Like I mentioned, we currently require 3 half-width lines per component
(4 components in total), not 4 full lines.
Sadly i dont have the time ATM to properly read all the papers but
There are some apparent low complexity transforms like the star tetrix
transforms
https://sci-hub.box/10.1109/TIP.2021.3095421
This here compares it for lossless images with some other transforms
https://www.cs.tsukuba.ac.jp/~taizo/pdf/journal/202209_TIP.pdf
Also advances in color image demosaicing (not compression specific IIUC)
can be seen in this series of papers, maybe some ideas can be used from these ?
https://sci-hub.box/10.1109/TIP.2016.2518082 MLRI (compares 12 algorithms)
Debayering is a solved problem IMO, I agree that neural networks are the
future.
I don't see why you're linking it, though.
https://www.mdpi.com/1424-8220/17/12/2787 ARI (compares MANY algorithms)
More recent (2019 & 2025 comparissions of demosaicing algorithms these show
that NN beats non NN)
https://www.ipol.im/pub/art/2019/274/article_lr.pdf
https://www.researchgate.net/publication/396809360_Color_Image_Demosaicking_A_Systematic_Survey_of_Algorithms_Performance_Evaluation_and_Open_Challenges
I expect that we will end with a O(slice_width) and not O(slice_width *
slice_height)
but i dont want to discard anything without knowing what iam discarding
We need to see how alternatives perform
But what is clear is that this will not be resolved in 2 weeks
I can certainly make comparisons between other coding method alternatives,
as long as they are a reasonable fit for the codec.
great
[...]
I will prepare performance and compression figures too, of course.
that would be very good
You are mainly focusing on the use of Haar to decompose Bayer into median +
difference for the green samples. Would you prefer a more general
interpolation or predictor instead? Like I mentioned, I wouldn't mind this,
though it would be better if it benefitted RGB coding as well.
My concern is not about a specific part of this, rather that we dont
know how the surroundings of the algorithm landscape perform.
We dont even know if this choice is a local minimum in the landscape
of algorithms. Maybe we can make a small change and it will perform better.
Maybe we can make a larger change and we would be in a much deeper local
minimum.
What iam asking for is that the landscape of choices / algorithms is
investigated more before we settle on what choice FFv1 would use for decades
That's good, but you should keep in mind that this is an unstable feature
that can be changed in the future, rather than the final design.
I have to push back on this
the STF project says
"Completing this milestone will expand FFv1's archival capabilities to handle not
just produced video, but also direct camera footage, as well as film scanners and
scientific applications."
I dont think "an unstable feature" can be used for archival.
archival requires features to be long term supported and stable
All new features need to be introduced as unstable features until version 4
of the codec is released. It is implied that this extension to the codec
will also go through the same procedure.
I think that merging support for it (or approving the PR) is enough to
satisfy the requirement of the milestone, rather than standardization.
The feature would be in the next version of the codec whenever it is frozen
and specified.
The formal IETF standardization isnt the same thing as designing a feature
that we _believe_ will not need to be changed
For example te float compression i designed to the best of my knowledge
at that time was in a local minimum. I tried finding better ways to store
floats in the FFv1 design (tried many differennt ways to store floats,
many diffeerenet context models, quantziation methods and i found nothing
short of things that should go to a FFv2
I also belive i asked on ffmpeg-devel and IETF-cellar about float ideas
in that sense you maybe want to ask on IETF-cellar about bayer CFA coding
ideas before this is finalized.
and thats why i said i dont think this can be done in the timeline for
the current STF milestones
I don't foresee the base of the coding mechanism needing to be
fundamentally changed here either.
In my opinion, we should not change the prediction structure or the
basic coding (no extension of the codec has). Which means the only
feasible solution I see we have is in the PR.
I don't consider changing whether Haar is used, or what wavelet type is
used, or not used a major difference. It is a 2-line change, after all.
Speaking of floating point encoding, I'd like to draw a parallel with
it, since it's close to what I am trying to accomplish here too.
Support for it was merged 4 days after the RFC was posted. There were
some comments left outstanding, mainly from me, since I never really
like the way paletization was used, but overall, I found no reason to
block it, since I did think it can be worked on and improved.
In the case here, I'm asked to do much more rigor and benchmarking to
merge, most of which I see no practical point in - they either do not
apply, or cannot be feasibly implemented without substantial changes,
which I myself would not agree with merging. Going through IETF/CELLAR
is something that float encoding never had to go through either.
Why does this need to be treated differently, when it's so much simpler
than floating-point coding
Furthermore, I do think that Vulkan should guide the design somewhat -
encoding and decoding of FFv1 on CPUs is still slow to the point where
the main application of this extension would be through the GPU
implementations.
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
