On 19/03/2026 14:13, Michael Niedermayer via ffmpeg-devel wrote:
Hi
On Wed, Mar 18, 2026 at 02:44:34PM +0100, Lynne via ffmpeg-devel wrote:
On 18/03/2026 14:04, Michael Niedermayer via ffmpeg-devel wrote:
Hi Lynne
On Wed, Mar 18, 2026 at 06:06:22AM +0100, Lynne via ffmpeg-devel wrote:
On 18/03/2026 01:11, Michael Niedermayer via ffmpeg-devel wrote:
[...]
[...]
when it comes to Decorrelating, the proposed filters are asymetric,
how would symmetric ones perform ? and how do adaptive ones perform?
The current implementation does not prohibit adaptive decorrelation. The RCT
search can still be performed to find more optimal coefficients than (1, 1).
While it can be extended with more expensive search, I think that extending
the decorrelation process should be done in a way in which RGBA coding would
also benefit, than tuning it specifically for Bayer. Perhaps that's
something we can investigate in the future, since custom RCT coefficients
are still marked as unstable.
G B G B G B G B
R G R G R G R G
G B G B G B G B
R G R G R G R G
G B G B G B G B
each blue and red sample has 4 adjacent green samples
That is each sample can be decorrelated with an average of 4, 3, 2 or 1
of these samples. Thats without considering more distant samples.
(this could be attempted in a local edge direction dependant way)
PR22528 uses the left and top green samples for blue and the bottom and right
green
samples for red.
Such a asymetric fixed choice "feels" strange
The PR is implemented for RGGB16 (R G\n G B), not GBRG16.
The pattern we get is:
R G R G R G R G
G B G B G B G B
R G R G R G R G
G B G B G B G B
R G R G R G R G
G B G B G B G B
The names we use for the samples should have also been a hint.
My text and analysis refers to RGGB.
If the CFA example starts at a even or odd line is something i did not
pay attention to
Also you effectively use a haar transform to split the green into 2 green planes
why that and not the 5/3 one from j2k. But either feels odd, putting a wavelet
style transform into ffv1 feels odd.
I really would like to see some numbers that show that this is the best choice
5/3 is a Wavelet transform that requires 6 samples in both directions, and
would impose a restriction on the slice width and height. Haar can operate
on just a single Bayer quad.
you can apply a 5/3 transform on a 1x1 image or a 2x2 image, if thats what you
want to do.
You could zero-pad the data, but then you're essentially not far off from
what Haar does by interpolating, but with an offset basis. Is this what you
were referring to?
What codecs do is mirror the samples (or filter), that introduces
no offsets
[...]
And this is the core problem here.
We need to know what these different choices do to compression (and speed),
to be able to make a choice
We also need the "state of teh art CFA compression algorithms" (only
decorrelation
and teh full ones or whatever part we can use) compared
And then make a choice what we should use.
To me this looks a bit like "heres my favorite algorithm, take it"
I performed literature review, provided references, as well as a review of
existing codecs, and weighted in feasibility of approaches to arrive at this
proposal.
When i asked chatgpt a few days ago (yes, please forgive me)
... "Please list all algorithms that we should consider. " ...
It gave me a long blob of academic stuff, here just 2 of it
The one reminding me of what you implemeted is this:
https://sci-hub.box/10.1109/ICIP.2019.8803376 (short and simple paper)
And heres a comparission of this against 5 other methods
https://www.mdpi.com/1424-8220/22/21/8362
Noticably, its never the best compressing nor the fastest
The issue I see with those is they are not lossless.
I much more would liek to see "heres the compression my PR achieves, heres the
compression it achieves when this and that is changed, heres the compression
the best state of the art alternatives achieve"
What I was hoping for is feedback about whether this path - e.g. modelling
Bayer coding from the RGB path, including a decorrelation filter like RCT -
is agreeable.
I cannot awnser this in a vaccuum
I think you can at least agree that writing a fundamentally different
codec is not what we would like to end up with, right? At least not on
this stage
And if that is true, what can we do to make Bayer coding fit into FFv1's
architecture?
I think given those constraints, we can put a limit on what we could do.
Given this, would you agree that needing a linecache larger than what we
currently use is not acceptable?
We need to see how alternatives perform
But what is clear is that this will not be resolved in 2 weeks
I can certainly make comparisons between other coding method
alternatives, as long as they are a reasonable fit for the codec.
We have an existing codec, and while we could invent a
completely new coding scheme for Bayer, I think that making it fit in is
conceptually what makes sense.
Thats all good and well but that doesnt mean that this specific method
is best nor that there isnt a faster and better one thats a simple
but "new coding scheme"
Indeed. Like all codecs, it's a technological compromise, between what
we can build upon, and what's optimal.
I will prepare performance and compression figures too, of course.
that would be very good
You are mainly focusing on the use of Haar to decompose Bayer into median +
difference for the green samples. Would you prefer a more general
interpolation or predictor instead? Like I mentioned, I wouldn't mind this,
though it would be better if it benefitted RGB coding as well.
My concern is not about a specific part of this, rather that we dont
know how the surroundings of the algorithm landscape perform.
We dont even know if this choice is a local minimum in the landscape
of algorithms. Maybe we can make a small change and it will perform better.
Maybe we can make a larger change and we would be in a much deeper local
minimum.
What iam asking for is that the landscape of choices / algorithms is
investigated more before we settle on what choice FFv1 would use for decades
That's good, but you should keep in mind that this is an unstable
feature that can be changed in the future, rather than the final design.
I think starting with a simple but proven design, and improving it, is
the most optimal path. It's how all other codecs were developed,
including AV1.
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
