PR #25315 opened by Forgejo_Fairy URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25315 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/25315.patch
LLM-GPT-6-ASTRA — verified fix for #25106 using oittaa’s three Vulkan patches from the [attachment comment on #25105](https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/25105#issuecomment-71415), preserving authorship and existing attribution. `bwdif_vulkan` reads beyond visible plane boundaries, stores values above the meaningful bit-depth maximum, and makes different no-motion and equal-difference decisions from CPU `bwdif`. The supplied 10-bit block example produces 480 samples above 1023; the noisy stripe example differs from CPU output by up to 876 codes. The series supplies visible plane heights through specialization constants, implements the CPU boundary substitutions and temporal edge path, clips low-aligned integer formats, and makes the decision comparisons aware of the sample step. Floating-point interpolation is retained. Validation on Ubuntu 24.04 x86-64, NVIDIA RTX 5090, driver 595.84, glslang 15.1.0: - **Reported examples: reproduced before, within one code after.** Both supplied examples fail on master e2798d5cad06 and release/9.0 c89d6ee1ca84. Patched master has no above-range samples and a maximum all-plane difference of one code, including both field orders and both output modes for the block example. - **Additional format and height coverage: 68 comparisons pass.** NV12/P010 heights 8, 56, 64, 72 and 1080; planar 4:2:2 at 10/12/16 bits; P012/P016; YUVA420P10 and GBRP10; both field orders and output modes. Every comparison stays within one meaningful sample code. - **Independent equal-difference test: corrected.** With both integer differences equal to 198, unpatched Vulkan produces 367 versus CPU 331 at the selected pixel. Patched master matches the entire tested row. - **Build and patch checks: pass.** Shader compilation and `git diff --check` succeed. Standalone generators, comparison scripts, configuration details and measured results are preserved separately on `fairy/issue25106-tooling` under `tools/issue25106/`. These are GPU validation scripts, not FATE tests. No release-branch backports are included. Fixes #25106. From 8bafc4cd5edfe62abba27ed2a03eab079ddad4c9 Mon Sep 17 00:00:00 2001 From: oittaa <[email protected]> Date: Thu, 15 Oct 2026 20:19:37 +0200 Subject: [PATCH 1/3] lavfi/vf_bwdif_vulkan: filter the top and bottom edges like bwdif The shader has no case for the rows near the top and bottom edges. It loads the rows up to 4 above and below the rebuilt row, and a load outside the image is undefined. On an RTX 4080 SUPER it reads 0. So the outermost rebuilt row comes out at half of bwdif's value or less, and the rebuilt rows next to the edges overshoot. bwdif replaces a row past an edge with the nearest row on the other side. It filters the 4 rows nearest each edge with filter_edge, which averages the rows next to it and uses the rows 2 away only when both are in the picture. Do the same here, in the intra path too, and load no rows outside the image. This needs the planes' real heights, and imageSize() doesn't always give them. On an RTX 4080 SUPER it rounds the height of 2-plane images up to 16: the luma of a 1080-row P010 frame reports 1088, and NV12 frames behave the same. The heights now come from the frame, as specialization constants. Tested on an RTX 4080 SUPER with interlaced 10-bit 4:2:2 ProRes clips decoded by Vulkan, white blocks on black touching both edges at 128x64 and 1920x1080, top and bottom field first, send_frame and send_field. The 4 rows at each edge now match bwdif within a code in every plane. So do NV12 frames of 56, 64, 72 and 1080 rows, uploaded. Before, a white (940) block touching the bottom edge gave 470 in row 63 and 1053 in row 61, where bwdif gives 940. Co-Authored-By: Claude Opus 5.5 <[email protected]> Assisted-by: Fairy --- libavfilter/vf_bwdif_vulkan.c | 8 ++- libavfilter/vulkan/bwdif.comp.glsl | 102 +++++++++++++++++++++++++++-- 2 files changed, 102 insertions(+), 8 deletions(-) diff --git a/libavfilter/vf_bwdif_vulkan.c b/libavfilter/vf_bwdif_vulkan.c index 4ddb77a8404..61bf56f43b7 100644 --- a/libavfilter/vf_bwdif_vulkan.c +++ b/libavfilter/vf_bwdif_vulkan.c @@ -51,6 +51,7 @@ static av_cold int init_filter(AVFilterContext *ctx) BWDIFVulkanContext *s = ctx->priv; FFVulkanContext *vkctx = &s->vkctx; const int planes = av_pix_fmt_count_planes(s->vkctx.output_format); + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(s->vkctx.output_format); s->qf = ff_vk_qf_find(vkctx, VK_QUEUE_COMPUTE_BIT, 0); if (!s->qf) { @@ -61,7 +62,12 @@ static av_cold int init_filter(AVFilterContext *ctx) RET(ff_vk_exec_pool_init(vkctx, s->qf, &s->e, FF_VK_DEFAULT_EXEC_CONTEXTS, 0, 0, 0, NULL)); - ff_vk_shader_load(&s->shd, VK_SHADER_STAGE_COMPUTE_BIT, NULL, + /* The planes' heights, for the edges */ + SPEC_LIST_CREATE(sl, 2, 2*sizeof(int)) + SPEC_LIST_ADD(sl, 0, 32, vkctx->output_height); + SPEC_LIST_ADD(sl, 1, 32, AV_CEIL_RSHIFT(vkctx->output_height, desc->log2_chroma_h)); + + ff_vk_shader_load(&s->shd, VK_SHADER_STAGE_COMPUTE_BIT, sl, (uint32_t [3]) { 1, 64, planes }, 0); const FFVulkanDescriptorSetBinding desc_set[] = { diff --git a/libavfilter/vulkan/bwdif.comp.glsl b/libavfilter/vulkan/bwdif.comp.glsl index 81eb93c8d71..3d53b0bdffc 100644 --- a/libavfilter/vulkan/bwdif.comp.glsl +++ b/libavfilter/vulkan/bwdif.comp.glsl @@ -26,6 +26,9 @@ layout (local_size_x_id = 253, local_size_y_id = 254, local_size_z_id = 255) in; +layout (constant_id = 0) const int height = 0; +layout (constant_id = 1) const int chroma_height = 0; + layout (set = 0, binding = 0) uniform writeonly image2D dst[]; layout (set = 0, binding = 1) uniform readonly image2D prev[]; layout (set = 0, binding = 2) uniform readonly image2D cur[]; @@ -46,15 +49,17 @@ vec4 process_intra(vec4 cur[4]) return (coef_sp[0]*(cur[1] + cur[2]) - coef_sp[1]*(cur[0] + cur[3])) / (1 << 13); } -void process_plane_intra(ivec2 pos) +void process_plane_intra(ivec2 pos, int h) { nonuniformEXT uint idx = gl_LocalInvocationID.z; + /* As in bwdif, a row past an edge is replaced by the nearest row on the + * other side */ vec4 dcur[4]; - dcur[0] = imageLoad(cur[idx], pos - ivec2(0, 3)); - dcur[1] = imageLoad(cur[idx], pos - ivec2(0, 1)); - dcur[2] = imageLoad(cur[idx], pos + ivec2(0, 1)); - dcur[3] = imageLoad(cur[idx], pos + ivec2(0, 3)); + dcur[0] = imageLoad(cur[idx], pos - ivec2(0, pos.y > 2 ? 3 : -1)); + dcur[1] = imageLoad(cur[idx], pos - ivec2(0, pos.y > 0 ? 1 : -1)); + dcur[2] = imageLoad(cur[idx], pos + ivec2(0, pos.y + 1 < h ? 1 : -1)); + dcur[3] = imageLoad(cur[idx], pos + ivec2(0, pos.y + 3 < h ? 3 : -1)); imageStore(dst[idx], pos, process_intra(dcur)); } @@ -93,6 +98,33 @@ vec4 process_line(vec4 prev2[5], vec4 prev1[2], vec4 cur[4], vec4 next1[2], vec4 return mix(interpol, fd, diff_mask); } +vec4 process_edge(vec4 prev2[3], vec4 prev1[2], vec4 cur[2], vec4 next1[2], vec4 next2[3], bool spat) +{ + vec4 fc = cur[0]; + vec4 fe = cur[1]; + vec4 fd = (prev2[1] + next2[1]) / 2; + + vec4 temp_diff[3]; + temp_diff[0] = abs(prev2[1] - next2[1]); + temp_diff[1] = (abs(prev1[0] - fc) + abs(prev1[1] - fe)) / 2; + temp_diff[2] = (abs(next1[0] - fc) + abs(next1[1] - fe)) / 2; + vec4 diff = max(temp_diff[0] / 2, max(temp_diff[1], temp_diff[2])); + bvec4 diff_mask = equal(diff, vec4(0)); + + if (spat) { + vec4 fb = ((prev2[0] + next2[0]) / 2) - fc; + vec4 ff = ((prev2[2] + next2[2]) / 2) - fe; + vec4 dc = fd - fc; + vec4 de = fd - fe; + vec4 mmax = max(de, max(dc, min(fb, ff))); + vec4 mmin = min(de, min(dc, max(fb, ff))); + diff = max(diff, max(mmin, -mmax)); + } + + vec4 interpol = clamp((fc + fe) / 2, fd - diff, fd + diff); + return mix(interpol, fd, diff_mask); +} + void process_plane(const ivec2 pos) { nonuniformEXT uint idx = gl_LocalInvocationID.z; @@ -144,17 +176,73 @@ void process_plane(const ivec2 pos) imageStore(dst[idx], pos, process_line(prev2, prev1, dcur, next1, next2)); } +void process_plane_edge(const ivec2 pos, int h) +{ + nonuniformEXT uint idx = gl_LocalInvocationID.z; + bool field_parity = (parity ^ tff) != 0; + + /* As in bwdif, a row past an edge is replaced by the nearest row on the + * other side. The rows 2 away are only read when both are in the + * picture. */ + bool spat = !(pos.y < 2 || pos.y + 3 > h); + ivec2 refs2 = ivec2(0, spat ? 2 : 0); + ivec2 mrefs = ivec2(0, pos.y > 0 ? -1 : 1); + ivec2 prefs = ivec2(0, pos.y + 1 < h ? 1 : -1); + + vec4 dcur[2]; + vec4 prev1[2]; + vec4 next1[2]; + vec4 prev2[3]; + vec4 next2[3]; + + dcur[0] = imageLoad(cur[idx], pos + mrefs); + dcur[1] = imageLoad(cur[idx], pos + prefs); + + prev1[0] = imageLoad(prev[idx], pos + mrefs); + prev1[1] = imageLoad(prev[idx], pos + prefs); + + next1[0] = imageLoad(next[idx], pos + mrefs); + next1[1] = imageLoad(next[idx], pos + prefs); + + if (field_parity) { + prev2[0] = imageLoad(prev[idx], pos - refs2); + prev2[1] = imageLoad(prev[idx], pos); + prev2[2] = imageLoad(prev[idx], pos + refs2); + + next2[0] = imageLoad(cur[idx], pos - refs2); + next2[1] = imageLoad(cur[idx], pos); + next2[2] = imageLoad(cur[idx], pos + refs2); + } else { + prev2[0] = imageLoad(cur[idx], pos - refs2); + prev2[1] = imageLoad(cur[idx], pos); + prev2[2] = imageLoad(cur[idx], pos + refs2); + + next2[0] = imageLoad(next[idx], pos - refs2); + next2[1] = imageLoad(next[idx], pos); + next2[2] = imageLoad(next[idx], pos + refs2); + } + + imageStore(dst[idx], pos, process_edge(prev2, prev1, dcur, next1, next2, spat)); +} + void main() { const ivec2 pos = ivec2(gl_GlobalInvocationID.xy); bool filter_field = ((pos.y ^ parity) & 1) == 1; bool is_intra = filter_field && (current_field == 0); + /* imageSize() can include padding: NVIDIA rounds the height of 2-plane + * images up to 16. So the plane's height comes from the frame. */ + uint plane = gl_LocalInvocationID.z; + int h = plane == 1 || plane == 2 ? chroma_height : height; ivec2 size = imageSize(dst[nonuniformEXT(gl_LocalInvocationID.z)]); - if (any(greaterThanEqual(pos, size))) { + bool is_edge = pos.y < 4 || pos.y + 5 > h; + if (pos.x >= size.x || pos.y >= h) { return; } else if (is_intra) { - process_plane_intra(pos); + process_plane_intra(pos, h); + } else if (filter_field && is_edge) { + process_plane_edge(pos, h); } else if (filter_field) { process_plane(pos); } else { -- 2.54.0 From 3badd28b097c93bef16e3b82fbfcefd3689e4bc4 Mon Sep 17 00:00:00 2001 From: oittaa <[email protected]> Date: Thu, 15 Oct 2026 20:19:37 +0200 Subject: [PATCH 2/3] lavfi/vf_bwdif_vulkan: clip to the format's bit depth bwdif clips its output to (1 << depth) - 1. The shader works on samples scaled to their storage, where a 10-bit sample in a 16-bit plane is a fraction of 65535. So its results above 1023 were stored as they were. A white (940) block on black gave 1045 next to its inner edge, where bwdif gives 1023. Pass the largest value to the shader as a specialization constant and clip to it. Only formats that keep fewer than 16 bits in 16-bit storage need it. The others are left as they were, so float formats are not clipped either. Tested with the clips of the previous commit. No sample is above 1023 now, and on the clips of white blocks on black the whole output matches bwdif within a code. Co-Authored-By: Claude Opus 5.5 <[email protected]> Assisted-by: Fairy --- libavfilter/vf_bwdif_vulkan.c | 15 ++++++++++++++- libavfilter/vulkan/bwdif.comp.glsl | 7 ++++--- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/libavfilter/vf_bwdif_vulkan.c b/libavfilter/vf_bwdif_vulkan.c index 61bf56f43b7..caaee342c6b 100644 --- a/libavfilter/vf_bwdif_vulkan.c +++ b/libavfilter/vf_bwdif_vulkan.c @@ -20,6 +20,9 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include <float.h> + +#include "libavutil/intfloat.h" #include "libavutil/random_seed.h" #include "libavutil/opt.h" #include "vulkan_filter.h" @@ -52,6 +55,8 @@ static av_cold int init_filter(AVFilterContext *ctx) FFVulkanContext *vkctx = &s->vkctx; const int planes = av_pix_fmt_count_planes(s->vkctx.output_format); const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(s->vkctx.output_format); + const VkFormat *vkfmts = av_vkfmt_from_pixfmt(s->vkctx.output_format); + float clip_max = FLT_MAX; s->qf = ff_vk_qf_find(vkctx, VK_QUEUE_COMPUTE_BIT, 0); if (!s->qf) { @@ -62,10 +67,18 @@ static av_cold int init_filter(AVFilterContext *ctx) RET(ff_vk_exec_pool_init(vkctx, s->qf, &s->e, FF_VK_DEFAULT_EXEC_CONTEXTS, 0, 0, 0, NULL)); + /* Clip to the largest sample value, as bwdif does. The shader sees + * samples scaled to their storage, so only formats that keep fewer than + * 16 bits in 16-bit storage need it. */ + if (vkfmts[0] == VK_FORMAT_R16_UNORM && + desc->comp[0].depth + desc->comp[0].shift < 16) + clip_max = ((1 << desc->comp[0].depth) - 1) / 65535.0f; + /* The planes' heights, for the edges */ - SPEC_LIST_CREATE(sl, 2, 2*sizeof(int)) + SPEC_LIST_CREATE(sl, 3, 2*sizeof(int) + sizeof(float)) SPEC_LIST_ADD(sl, 0, 32, vkctx->output_height); SPEC_LIST_ADD(sl, 1, 32, AV_CEIL_RSHIFT(vkctx->output_height, desc->log2_chroma_h)); + SPEC_LIST_ADD(sl, 2, 32, av_float2int(clip_max)); ff_vk_shader_load(&s->shd, VK_SHADER_STAGE_COMPUTE_BIT, sl, (uint32_t [3]) { 1, 64, planes }, 0); diff --git a/libavfilter/vulkan/bwdif.comp.glsl b/libavfilter/vulkan/bwdif.comp.glsl index 3d53b0bdffc..a54da311074 100644 --- a/libavfilter/vulkan/bwdif.comp.glsl +++ b/libavfilter/vulkan/bwdif.comp.glsl @@ -28,6 +28,7 @@ layout (local_size_x_id = 253, local_size_y_id = 254, local_size_z_id = 255) in; layout (constant_id = 0) const int height = 0; layout (constant_id = 1) const int chroma_height = 0; +layout (constant_id = 2) const float clip_max = 1.0; layout (set = 0, binding = 0) uniform writeonly image2D dst[]; layout (set = 0, binding = 1) uniform readonly image2D prev[]; @@ -60,7 +61,7 @@ void process_plane_intra(ivec2 pos, int h) dcur[1] = imageLoad(cur[idx], pos - ivec2(0, pos.y > 0 ? 1 : -1)); dcur[2] = imageLoad(cur[idx], pos + ivec2(0, pos.y + 1 < h ? 1 : -1)); dcur[3] = imageLoad(cur[idx], pos + ivec2(0, pos.y + 3 < h ? 3 : -1)); - imageStore(dst[idx], pos, process_intra(dcur)); + imageStore(dst[idx], pos, min(process_intra(dcur), clip_max)); } vec4 process_line(vec4 prev2[5], vec4 prev1[2], vec4 cur[4], vec4 next1[2], vec4 next2[5]) @@ -173,7 +174,7 @@ void process_plane(const ivec2 pos) next2[4] = imageLoad(next[idx], pos + ivec2(0, 4)); } - imageStore(dst[idx], pos, process_line(prev2, prev1, dcur, next1, next2)); + imageStore(dst[idx], pos, min(process_line(prev2, prev1, dcur, next1, next2), clip_max)); } void process_plane_edge(const ivec2 pos, int h) @@ -222,7 +223,7 @@ void process_plane_edge(const ivec2 pos, int h) next2[2] = imageLoad(next[idx], pos + refs2); } - imageStore(dst[idx], pos, process_edge(prev2, prev1, dcur, next1, next2, spat)); + imageStore(dst[idx], pos, min(process_edge(prev2, prev1, dcur, next1, next2, spat), clip_max)); } void main() -- 2.54.0 From a86bf4a80207d74d8a54072031103ca76c09a98e Mon Sep 17 00:00:00 2001 From: oittaa <[email protected]> Date: Thu, 15 Oct 2026 20:57:33 +0200 Subject: [PATCH 3/3] lavfi/vf_bwdif_vulkan: decide on whole sample values like bwdif bwdif makes two decisions on whole sample values. A sample has no motion when its halved temporal differences, rounded down by the shifts, are all 0, so a change of one step counts as none. And it takes interpolate_cur unless abs(c - e) is above temporal_diff0. The shader works on floats, fractions of the storage's largest value, which aren't exact and whose halves aren't rounded down. So both decisions can go the other way. On still 10-bit 1-row stripes with one step of noise, the no-motion test fails. The spatial check then widens diff, and a bright row (940) between dark ones comes out as 64, where bwdif keeps 940. On moving pictures, equal differences go either way: c 230, e 428 and temporal_diff0 198 gave 399, where bwdif gives 318. Pass the size of one sample step to the shader as a specialization constant, and make both decisions in whole steps. In formats that shift their samples up, like P010, a step is one value of the format's depth, so P010 decides as yuv420p10 does. Float formats have no steps and keep the plain comparisons. Tested on an RTX 4080 SUPER with interlaced 1920x1080 clips: white blocks over moving waves as 10-bit 4:2:2 ProRes decoded by Vulkan, and the blocks over moving waves and over still stripes with one step of noise, uploaded as yuv422p10, yuv420p10, p010, yuv420p and nv12. Top and bottom field first, send_frame and send_field. Some of them also ran on an AMD Radeon iGPU. The whole output now matches bwdif within a code. Before, it was off by up to 876 on the stripes and 107 on the moving waves. Co-Authored-By: Claude Opus 5.5 <[email protected]> Assisted-by: Fairy --- libavfilter/vf_bwdif_vulkan.c | 14 ++++++++++++-- libavfilter/vulkan/bwdif.comp.glsl | 13 ++++++++++--- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/libavfilter/vf_bwdif_vulkan.c b/libavfilter/vf_bwdif_vulkan.c index caaee342c6b..77dfb4fcdb3 100644 --- a/libavfilter/vf_bwdif_vulkan.c +++ b/libavfilter/vf_bwdif_vulkan.c @@ -56,7 +56,7 @@ static av_cold int init_filter(AVFilterContext *ctx) const int planes = av_pix_fmt_count_planes(s->vkctx.output_format); const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(s->vkctx.output_format); const VkFormat *vkfmts = av_vkfmt_from_pixfmt(s->vkctx.output_format); - float clip_max = FLT_MAX; + float clip_max = FLT_MAX, unit = 0.0f; s->qf = ff_vk_qf_find(vkctx, VK_QUEUE_COMPUTE_BIT, 0); if (!s->qf) { @@ -74,11 +74,21 @@ static av_cold int init_filter(AVFilterContext *ctx) desc->comp[0].depth + desc->comp[0].shift < 16) clip_max = ((1 << desc->comp[0].depth) - 1) / 65535.0f; + /* bwdif decides on whole sample values, so the shader needs the size of + * one step as it sees them. Float formats have no steps. */ + if (vkfmts[0] == VK_FORMAT_A2R10G10B10_UNORM_PACK32 || + vkfmts[0] == VK_FORMAT_A2B10G10R10_UNORM_PACK32) + unit = 1.0f / 1023; + else if (!(desc->flags & AV_PIX_FMT_FLAG_FLOAT)) + unit = (1 << desc->comp[0].shift) / + (desc->comp[0].depth + desc->comp[0].shift > 8 ? 65535.0f : 255.0f); + /* The planes' heights, for the edges */ - SPEC_LIST_CREATE(sl, 3, 2*sizeof(int) + sizeof(float)) + SPEC_LIST_CREATE(sl, 4, 2*sizeof(int) + 2*sizeof(float)) SPEC_LIST_ADD(sl, 0, 32, vkctx->output_height); SPEC_LIST_ADD(sl, 1, 32, AV_CEIL_RSHIFT(vkctx->output_height, desc->log2_chroma_h)); SPEC_LIST_ADD(sl, 2, 32, av_float2int(clip_max)); + SPEC_LIST_ADD(sl, 3, 32, av_float2int(unit)); ff_vk_shader_load(&s->shd, VK_SHADER_STAGE_COMPUTE_BIT, sl, (uint32_t [3]) { 1, 64, planes }, 0); diff --git a/libavfilter/vulkan/bwdif.comp.glsl b/libavfilter/vulkan/bwdif.comp.glsl index a54da311074..d64ab8f6982 100644 --- a/libavfilter/vulkan/bwdif.comp.glsl +++ b/libavfilter/vulkan/bwdif.comp.glsl @@ -30,6 +30,11 @@ layout (constant_id = 0) const int height = 0; layout (constant_id = 1) const int chroma_height = 0; layout (constant_id = 2) const float clip_max = 1.0; +/* bwdif decides on whole sample values. Here they are floats, which aren't + * exact, so the decisions are made on steps of this size, 0 for float + * formats. */ +layout (constant_id = 3) const float unit = 0.0; + layout (set = 0, binding = 0) uniform writeonly image2D dst[]; layout (set = 0, binding = 1) uniform readonly image2D prev[]; layout (set = 0, binding = 2) uniform readonly image2D cur[]; @@ -76,7 +81,8 @@ vec4 process_line(vec4 prev2[5], vec4 prev1[2], vec4 cur[4], vec4 next1[2], vec4 temp_diff[1] = (abs(prev1[0] - fc) + abs(prev1[1] - fe)) / 2; temp_diff[2] = (abs(next1[0] - fc) + abs(next1[1] - fe)) / 2; vec4 diff = max(temp_diff[0] / 2, max(temp_diff[1], temp_diff[2])); - bvec4 diff_mask = equal(diff, vec4(0)); + /* bwdif halves with a shift, so a change of one step is no motion */ + bvec4 diff_mask = lessThanEqual(diff, vec4(unit * 0.75)); vec4 fbs = prev2[1] + next2[1]; vec4 ffs = prev2[3] + next2[3]; @@ -93,7 +99,8 @@ vec4 process_line(vec4 prev2[5], vec4 prev1[2], vec4 cur[4], vec4 next1[2], vec4 coef_lf[0]*(fc + fe) - coef_lf[1]*(cur[0] + cur[3])) / (1 << 13); vec4 interpolate_cur = (coef_sp[0]*(fc + fe) - coef_sp[1]*(cur[0] + cur[3])) / (1 << 13); - bvec4 interpolate_cnd1 = greaterThan(abs(fc - fe), temp_diff[0]); + /* bwdif takes interpolate_cur when the differences are equal */ + bvec4 interpolate_cnd1 = greaterThan(abs(fc - fe), temp_diff[0] + unit * 0.5); vec4 interpol = mix(interpolate_cur, interpolate_all, interpolate_cnd1); interpol = clamp(interpol, fd - diff, fd + diff); return mix(interpol, fd, diff_mask); @@ -110,7 +117,7 @@ vec4 process_edge(vec4 prev2[3], vec4 prev1[2], vec4 cur[2], vec4 next1[2], vec4 temp_diff[1] = (abs(prev1[0] - fc) + abs(prev1[1] - fe)) / 2; temp_diff[2] = (abs(next1[0] - fc) + abs(next1[1] - fe)) / 2; vec4 diff = max(temp_diff[0] / 2, max(temp_diff[1], temp_diff[2])); - bvec4 diff_mask = equal(diff, vec4(0)); + bvec4 diff_mask = lessThanEqual(diff, vec4(unit * 0.75)); if (spat) { vec4 fb = ((prev2[0] + next2[0]) / 2) - fc; -- 2.54.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
