PR #25892 opened by Ramiro Polla (ramiro)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25892
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25892.patch

The rounding constant for alpha was the one for 8-bit output regardless
of depth, which biased alpha by 2, 8 and 32 codes for 10, 12 and 14 bits.

Tested with:
  ./libswscale/tests/swscale -scaler none -src yuva444p -dst gbrap12le

```
Before: SSIM={Y=1.000000 U=0.999999 V=0.999999 A=0.999949} loss=5.149841e-05
After:  SSIM={Y=1.000000 U=0.999999 V=0.999999 A=1.000000} loss=5.364418e-07
```

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Ramiro Polla <[email protected]>



>From 3b66e38039534bddb90417ecd0662b464f01967e Mon Sep 17 00:00:00 2001
From: Ramiro Polla <[email protected]>
Date: Mon, 9 Nov 2026 20:05:49 +0100
Subject: [PATCH] swscale/output: fix alpha rounding in yuv2gbrp_full_X_c()
 above 8 bits

The rounding constant for alpha was the one for 8-bit output regardless
of depth, which biased alpha by 2, 8 and 32 codes for 10, 12 and 14 bits.

Tested with:
  ./libswscale/tests/swscale -scaler none -src yuva444p -dst gbrap12le

Before: SSIM={Y=1.000000 U=0.999999 V=0.999999 A=0.999949} loss=5.149841e-05
After:  SSIM={Y=1.000000 U=0.999999 V=0.999999 A=1.000000} loss=5.364418e-07

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Ramiro Polla <[email protected]>
---
 libswscale/output.c       | 2 +-
 libswscale/x86/output.asm | 6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/libswscale/output.c b/libswscale/output.c
index 55cba1ea5f1..bd4fe04b4ca 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -2434,7 +2434,7 @@ yuv2gbrp_full_X_c(SwsInternal *c, const int16_t 
*lumFilter,
         V >>= 10;
 
         if (hasAlpha) {
-            A = 1 << 18;
+            A = 1 << (SH - 4);
 
             for (j = 0; j < lumFilterSize; j++)
                 A += alpSrc[j][i] * (unsigned)lumFilter[j];
diff --git a/libswscale/x86/output.asm b/libswscale/x86/output.asm
index e1b369c551d..8f749b713af 100644
--- a/libswscale/x86/output.asm
+++ b/libswscale/x86/output.asm
@@ -42,7 +42,9 @@ pd_65535_invf:             times 8 dd 0x37800080 ;1.0/65535.0
 pd_yuv2gbrp16_start:       times 8 dd -0x40000000
 pd_yuv2gbrp_y_start:       times 8 dd  (1 << 9)
 pd_yuv2gbrp_uv_start:      times 8 dd  ((1 << 9) - (128 << 19))
-pd_yuv2gbrp_a_start:       times 8 dd  (1 << 18)
+pd_yuv2gbrp_a_start_8:     times 8 dd  (1 << 18)
+pd_yuv2gbrp_a_start_10:    times 8 dd  (1 << 16)
+pd_yuv2gbrp_a_start_12:    times 8 dd  (1 << 14)
 pd_yuv2gbrp16_offset:      times 8 dd  0x10000  ;(1 << 16)
 pd_yuv2gbrp16_round13:     times 8 dd  0xE0002000  ;(1 << 13) - (1 << 29)
 pd_yuv2gbrp16_a_offset:    times 8 dd  0x20002000
@@ -758,7 +760,7 @@ endstruc
     %define Y_START  [pd_yuv2gbrp_y_start]
     %define Y_ROUND  m9
     %define UV_START [pd_yuv2gbrp_uv_start]
-    %define A_START  [pd_yuv2gbrp_a_start]
+    %define A_START  [pd_yuv2gbrp_a_start_ %+ DEPTH]
     %define A_CLIP2P 27
 %endif
 
-- 
2.54.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]

Reply via email to