Re: [FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
On 25/07/18 13:13, Danil Iashchenko wrote: > Add lumakey_opencl filter. Behaves like existing lumakey filter. > > --- > configure | 1 + > libavfilter/Makefile| 2 + > libavfilter/allfilters.c| 1 + > libavfilter/opencl/lumakey.cl | 43 +++ > libavfilter/opencl_source.h | 1 + > libavfilter/vf_lumakey_opencl.c | 243 > > 6 files changed, 291 insertions(+) > create mode 100644 libavfilter/opencl/lumakey.cl > create mode 100644 libavfilter/vf_lumakey_opencl.c I think you need a bit more configuration to make this work without needing to force the formats externally. I guess you did something like 'format=yuv420p,hwupload,lumakey_opencl,hwdownload,format=yuva420p'? It should probably be able to work with something like: ./ffmpeg_g -y -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device opencl=cl@va -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device va -i in1.mp4 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device va -i in2.mp4 -an -filter_hw_device cl -filter_complex '[0:v]scale_vaapi=format=yuv420p,hwmap,lumakey_opencl[a]; [1:v]scale_vaapi=format=yuv420p,hwmap[b]; [a][b]overlay_opencl,hwmap=derive_device=vaapi:reverse=1,scale_vaapi=format=nv12' -c:v h264_vaapi out.mp4 to composite two videos together keyed by the luma of the first (I'm not sure that command-line is exactly right, but something like that). In this case there isn't any way to make the output format implicitly do the right thing, so the filter needs to deal with the formats internally. (And extra points if you can avoid the yuv420p conversion :) Everything else in the patch looks fine. Thanks, - Mark ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
On Fri, Jul 27, 2018 at 11:42:57AM -0300, James Almer wrote: > On 7/27/2018 7:22 AM, Danil Iashchenko wrote: > > Add lumakey_opencl filter. Behaves like existing lumakey filter. > > > > --- > > > > On Wed, Jul 25, 2018 at 10:50:43AM -0300, James Almer wrote: > >>> On 7/25/2018 9:13 AM, Danil Iashchenko wrote: > Add lumakey_opencl filter. Behaves like existing lumakey filter. > >>> > >>> Isn't it possible to keep each of these new OpenCL filters as an > >>> optional codepath within the C version, using an AVOption like "opencl" > >>> or "hwaccel" to toggle one or another? Or maybe autodetected depending > >>> on the filter chain and/or input pix_fmt? > >>> > >>> I'm asking because it's getting too crowded. We also have some vaapi and > >>> qsv duplicate filters, and once we start committing filters using the > >>> upcoming Vulkan hwcontext the same way, we may also end up introducing > >>> yet another hardware specific variant for each of these. > >>> > > > >>> In libavcodec the hwaccels are seamlessly integrated into supported > >>> decoders. This has been favored over separate full stream hardware > >>> decoders where possible for the above reasons. It would be ideal to > >>> achieve the same with libavfilter. > > > >> i am in favor of this design as well. The user should not need to have > >> to know about and manage manually GPU optimizations. > > > >> thx > > > > Hi! I am GSoC student and I still have some tasks before the program ends. > > Also my mentor said: > > IMO don't think about it now, there isn't that much time left. > > I looked at doing last year (when converting to the current > > structure) and concluded that it's not really sane to do. > > The _opencl versions of filters operate completely differently, so > > while some code for setup can be shared putting them in the same filter > > doesn't really make sense. > > > > Thanks, Danil. > > Fair enough, i wasn't aware this was for GSoC, and this kind of change > (assuming it's doable) would definitely take a while, so it can wait. +1 [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Modern terrorism, a quick summary: Need oil, start war with country that has oil, kill hundread thousand in war. Let country fall into chaos, be surprised about raise of fundamantalists. Drop more bombs, kill more people, be surprised about them taking revenge and drop even more bombs and strip your own citizens of their rights and freedoms. to be continued signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
On 7/27/2018 7:22 AM, Danil Iashchenko wrote: > Add lumakey_opencl filter. Behaves like existing lumakey filter. > > --- > > On Wed, Jul 25, 2018 at 10:50:43AM -0300, James Almer wrote: >>> On 7/25/2018 9:13 AM, Danil Iashchenko wrote: Add lumakey_opencl filter. Behaves like existing lumakey filter. >>> >>> Isn't it possible to keep each of these new OpenCL filters as an >>> optional codepath within the C version, using an AVOption like "opencl" >>> or "hwaccel" to toggle one or another? Or maybe autodetected depending >>> on the filter chain and/or input pix_fmt? >>> >>> I'm asking because it's getting too crowded. We also have some vaapi and >>> qsv duplicate filters, and once we start committing filters using the >>> upcoming Vulkan hwcontext the same way, we may also end up introducing >>> yet another hardware specific variant for each of these. >>> > >>> In libavcodec the hwaccels are seamlessly integrated into supported >>> decoders. This has been favored over separate full stream hardware >>> decoders where possible for the above reasons. It would be ideal to >>> achieve the same with libavfilter. > >> i am in favor of this design as well. The user should not need to have >> to know about and manage manually GPU optimizations. > >> thx > > Hi! I am GSoC student and I still have some tasks before the program ends. > Also my mentor said: > IMO don't think about it now, there isn't that much time left. > I looked at doing last year (when converting to the current > structure) and concluded that it's not really sane to do. > The _opencl versions of filters operate completely differently, so > while some code for setup can be shared putting them in the same filter > doesn't really make sense. > > Thanks, Danil. Fair enough, i wasn't aware this was for GSoC, and this kind of change (assuming it's doable) would definitely take a while, so it can wait. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
Add lumakey_opencl filter. Behaves like existing lumakey filter. --- On Wed, Jul 25, 2018 at 10:50:43AM -0300, James Almer wrote: >> On 7/25/2018 9:13 AM, Danil Iashchenko wrote: >> > Add lumakey_opencl filter. Behaves like existing lumakey filter. >> >> Isn't it possible to keep each of these new OpenCL filters as an >> optional codepath within the C version, using an AVOption like "opencl" >> or "hwaccel" to toggle one or another? Or maybe autodetected depending >> on the filter chain and/or input pix_fmt? >> >> I'm asking because it's getting too crowded. We also have some vaapi and >> qsv duplicate filters, and once we start committing filters using the >> upcoming Vulkan hwcontext the same way, we may also end up introducing >> yet another hardware specific variant for each of these. >> >> In libavcodec the hwaccels are seamlessly integrated into supported >> decoders. This has been favored over separate full stream hardware >> decoders where possible for the above reasons. It would be ideal to >> achieve the same with libavfilter. >i am in favor of this design as well. The user should not need to have >to know about and manage manually GPU optimizations. >thx Hi! I am GSoC student and I still have some tasks before the program ends. Also my mentor said: IMO don't think about it now, there isn't that much time left. I looked at doing last year (when converting to the current structure) and concluded that it's not really sane to do. The _opencl versions of filters operate completely differently, so while some code for setup can be shared putting them in the same filter doesn't really make sense. Thanks, Danil. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
On Wed, Jul 25, 2018 at 10:50:43AM -0300, James Almer wrote: > On 7/25/2018 9:13 AM, Danil Iashchenko wrote: > > Add lumakey_opencl filter. Behaves like existing lumakey filter. > > Isn't it possible to keep each of these new OpenCL filters as an > optional codepath within the C version, using an AVOption like "opencl" > or "hwaccel" to toggle one or another? Or maybe autodetected depending > on the filter chain and/or input pix_fmt? > > I'm asking because it's getting too crowded. We also have some vaapi and > qsv duplicate filters, and once we start committing filters using the > upcoming Vulkan hwcontext the same way, we may also end up introducing > yet another hardware specific variant for each of these. > > In libavcodec the hwaccels are seamlessly integrated into supported > decoders. This has been favored over separate full stream hardware > decoders where possible for the above reasons. It would be ideal to > achieve the same with libavfilter. i am in favor of this design as well. The user should not need to have to know about and manage manually GPU optimizations. thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Dictatorship: All citizens are under surveillance, all their steps and actions recorded, for the politicians to enforce control. Democracy: All politicians are under surveillance, all their steps and actions recorded, for the citizens to enforce control. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
On 7/25/2018 9:13 AM, Danil Iashchenko wrote: > Add lumakey_opencl filter. Behaves like existing lumakey filter. Isn't it possible to keep each of these new OpenCL filters as an optional codepath within the C version, using an AVOption like "opencl" or "hwaccel" to toggle one or another? Or maybe autodetected depending on the filter chain and/or input pix_fmt? I'm asking because it's getting too crowded. We also have some vaapi and qsv duplicate filters, and once we start committing filters using the upcoming Vulkan hwcontext the same way, we may also end up introducing yet another hardware specific variant for each of these. In libavcodec the hwaccels are seamlessly integrated into supported decoders. This has been favored over separate full stream hardware decoders where possible for the above reasons. It would be ideal to achieve the same with libavfilter. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] lavfi: add lumakey_opencl filter
Add lumakey_opencl filter. Behaves like existing lumakey filter.
---
configure | 1 +
libavfilter/Makefile| 2 +
libavfilter/allfilters.c| 1 +
libavfilter/opencl/lumakey.cl | 43 +++
libavfilter/opencl_source.h | 1 +
libavfilter/vf_lumakey_opencl.c | 243
6 files changed, 291 insertions(+)
create mode 100644 libavfilter/opencl/lumakey.cl
create mode 100644 libavfilter/vf_lumakey_opencl.c
diff --git a/configure b/configure
index 5783407..9816ebb 100755
--- a/configure
+++ b/configure
@@ -3356,6 +3356,7 @@ interlace_filter_deps="gpl"
kerndeint_filter_deps="gpl"
ladspa_filter_deps="ladspa libdl"
lensfun_filter_deps="liblensfun version3"
+lumakey_opencl_filter_deps="opencl"
lv2_filter_deps="lv2"
mcdeint_filter_deps="avcodec gpl"
movie_filter_deps="avcodec avformat"
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 5d4549e..2a01bf3 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -253,6 +253,8 @@ OBJS-$(CONFIG_LIBVMAF_FILTER)+=
vf_libvmaf.o framesync.o
OBJS-$(CONFIG_LIMITER_FILTER)+= vf_limiter.o
OBJS-$(CONFIG_LOOP_FILTER) += f_loop.o
OBJS-$(CONFIG_LUMAKEY_FILTER)+= vf_lumakey.o
+OBJS-$(CONFIG_LUMAKEY_OPENCL_FILTER) += vf_lumakey_opencl.o opencl.o \
+opencl/lumakey.o
OBJS-$(CONFIG_LUT_FILTER)+= vf_lut.o
OBJS-$(CONFIG_LUT2_FILTER) += vf_lut2.o framesync.o
OBJS-$(CONFIG_LUT3D_FILTER) += vf_lut3d.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 521bc53..065ad9f 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -242,6 +242,7 @@ extern AVFilter ff_vf_libvmaf;
extern AVFilter ff_vf_limiter;
extern AVFilter ff_vf_loop;
extern AVFilter ff_vf_lumakey;
+extern AVFilter ff_vf_lumakey_opencl;
extern AVFilter ff_vf_lut;
extern AVFilter ff_vf_lut2;
extern AVFilter ff_vf_lut3d;
diff --git a/libavfilter/opencl/lumakey.cl b/libavfilter/opencl/lumakey.cl
new file mode 100644
index 000..dbee63e
--- /dev/null
+++ b/libavfilter/opencl/lumakey.cl
@@ -0,0 +1,43 @@
+/*
+ * Copyright (c) 2018 Danil Iashchenko
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+__kernel void lumakey_global(__write_only image2d_t dstAlpha,
+ __read_only image2d_t srcLuma,
+ float w,
+ float b,
+ int so)
+{
+const sampler_t sampler = (CLK_NORMALIZED_COORDS_FALSE |
+ CLK_ADDRESS_CLAMP_TO_EDGE |
+ CLK_FILTER_NEAREST);
+
+int2 loc = (int2)(get_global_id(0), get_global_id(1));
+float4 lumaPix = read_imagef(srcLuma, sampler, loc) * 255;
+
+if (lumaPix.x >= b && lumaPix.x <= w) {
+write_imagef(dstAlpha, loc, (float4)(0.0f));
+} else if (lumaPix.x > b - so && lumaPix.x < w + so) {
+if (lumaPix.x < b) {
+write_imagef(dstAlpha, loc, (float4)((1 - (lumaPix.x - b + so) /
so)));
+} else {
+write_imagef(dstAlpha, loc, (float4)(((lumaPix.x - w) / so)));
+}
+}
+}
diff --git a/libavfilter/opencl_source.h b/libavfilter/opencl_source.h
index de4e66e..dba701e 100644
--- a/libavfilter/opencl_source.h
+++ b/libavfilter/opencl_source.h
@@ -22,6 +22,7 @@
extern const char *ff_opencl_source_avgblur;
extern const char *ff_opencl_source_colorspace_common;
extern const char *ff_opencl_source_convolution;
+extern const char *ff_opencl_source_lumakey;
extern const char *ff_opencl_source_overlay;
extern const char *ff_opencl_source_tonemap;
extern const char *ff_opencl_source_unsharp;
diff --git a/libavfilter/vf_lumakey_opencl.c b/libavfilter/vf_lumakey_opencl.c
new file mode 100644
index 000..8879a88
--- /dev/null
+++ b/libavfilter/vf_lumakey_opencl.c
@@ -0,0 +1,243 @@
+/*
+ * Copyright (c) 2018 Danil Iashchenko
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software
