On Thu, Oct 30, 2014 at 9:34 PM, Stefano Sabatini <stefa...@gmail.com>
wrote:
> On date Thursday 2014-10-30 18:23:35 +0530, arwa arif encoded:
> > On Wed, Oct 29, 2014 at 3:36 AM, Clément Bœsch <u...@pkh.me> wrote:
> >
> > > On Tue, Oct 28, 2014 at 10:51:27PM +0100, Michael Niedermayer wrote:
> > > > On Tue, Oct 28, 2014 at 07:16:45PM +0100, Clément Bœsch wrote:
> > > > > On Tue, Oct 28, 2014 at 06:30:34PM +0100, Stefano Sabatini wrote:
> > > > > [...]
> > > > > > How much effort would it take to implement the remaining scaling
> > > modes?
> > > > > >
> > > > >
> > > > > According to
> > > > > https://ffmpeg.org/pipermail/ffmpeg-devel/2014-October/164574.html
> > > > >
> > > > > "I think 4x can be done fast enough, but 3x will take time."
> > > > >
> > > > > [...]
> > > > > > > +typedef struct {
> > > > > > > + uint32_t rgbtoyuv[1<<24];
> > > > > >
> > > > > > We should avoid this 64MiB. Also the table should be possibly
> static,
> > > > > > so you don't have to fill it per each xBR instance.
> > > > > >
> > > > >
> > > > > So, I requested to do it exactly the same as HQx because this part
> is
> > > > > common according to the specifications. This should be kept the
> same
> > > > > vf_hqx, and then factorized.
> > > > >
> > > >
> > > > > Now about removing this allocation, I did benchmark this LUT vs
> > > > > computation (see attached patch for comp. version). And the
> problem is
> > > > > that it's slightly slower, probably due to the /1000.
> > > >
> > > > why do you divide at all ?
> > > > cant you do the computations with full precission ?
> > >
> > > I wasn't able to... but I was probably doing it wrong.
> > >
> > > And anyway, so far I observed this:
> > > lut: 127fps
> > > nolut+div: 119fps
> > > nolut+nodiv: 123fps
> > >
> > > So even with "fast" computation, it's still slower than the LUT. It
> > > probably
> > > doesn't matter that much in practice, and dropping that huge table
> might be
> > > worth the performance impact (feel free to discuss).
> > >
> > > Note that looking at the original code (which was working on rgb565
> only),
> > > it was bitexact. The rgb 24-bit was added in the "modern" hqx with
> float
> > > point. So we can probably tolerate the inaccuracy. Still, if you find a
> > > way of keeping full accuracy with the modern implementation...
> > >
> > > Typically, I tried stuff like this:
> > >
> > > const uint32_t y = (uint32_t)((1225*r + 2404*g + 467*b + (1<<11)) >>
> > > 12);
> > > const uint32_t u = (uint32_t)((-692*r - 1356*g + 2048*b + (1<<11)) >>
> > > 12) + 128;
> > > const uint32_t v = (uint32_t)((2048*r - 1716*g - 332*b + (1<<11)) >>
> > > 12) + 128;
> > >
> > > ...but I'm probably doing it very wrong somewhere (sign issue maybe?),
> > > haven't
> > > looked deeper. I went up to 15 bits, still didn't match, so I was
> probably
> > > doing something stupid.
> > >
> > > > also instead of doing 2 rgb2yuv and then taking their difference you
> > > > can do the difference in rgb space and convert the rgb difference to
> > > > a yuv difference
> > > > its just aM - bM = (a-b)M
> > >
> > > Ah, sounds like a good idea, I guess I'll try that.
> > >
> > > [...]
> > >
> > > --
> > > Clément B.
> > >
> > > Updated the patch. How should I finally go about converting rgb to yuv?
> >
> >
> > > _______________________________________________
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > >
>
> > From 905fb0f15033e447fc344097dd649ca671fa1074 Mon Sep 17 00:00:00 2001
> > From: Arwa Arif <arwaarif1...@gmail.com>
> > Date: Thu, 30 Oct 2014 18:05:45 +0530
> > Subject: [PATCH] [PATCH]lvafi: add xbr filter
>
> lavfi: add xbr filter
>
> >
> > ---
>
> > doc/filters.texi | 81 ++----------
>
> what happened with this? Did you edit the patch by hand? I can't apply
> the patch...
>
> > libavfilter/Makefile | 1 +
> > libavfilter/allfilters.c | 1 +
> > libavfilter/vf_xbr.c | 319
> ++++++++++++++++++++++++++++++++++++++++++++++
> > 4 files changed, 335 insertions(+), 67 deletions(-)
> > create mode 100644 libavfilter/vf_xbr.c
> >
> [...]
> --
> FFmpeg = Fostering Fundamental Minimal Plastic Enhanced Geisha
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
From b9b56b594f856fef8b113b283df2d2045e8357f7 Mon Sep 17 00:00:00 2001
From: Arwa Arif <arwaarif1...@gmail.com>
Date: Thu, 30 Oct 2014 22:06:20 +0530
Subject: [PATCH] [PATCH]lvafi: add xbr filter
---
doc/filters.texi | 5 +
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/vf_xbr.c | 319 ++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 326 insertions(+)
create mode 100644 libavfilter/vf_xbr.c
diff --git a/doc/filters.texi b/doc/filters.texi
index 7be29de..2905e5d 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -9163,6 +9163,11 @@ Only deinterlace frames marked as interlaced.
Default value is @samp{all}.
@end table
+@section xbr
+Apply high-quality magnification filter which is designed for pixel art. It follows a set
+of edge-detection rules @url{http://www.libretro.com/forums/viewtopic.php?f=6&t=134}.
+This filter was originally created by Hyllian.
+
@anchor{yadif}
@section yadif
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 6d868e7..2c56e38 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -198,6 +198,7 @@ OBJS-$(CONFIG_VIDSTABDETECT_FILTER) += vidstabutils.o vf_vidstabdetect.
OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER) += vidstabutils.o vf_vidstabtransform.o
OBJS-$(CONFIG_VIGNETTE_FILTER) += vf_vignette.o
OBJS-$(CONFIG_W3FDIF_FILTER) += vf_w3fdif.o
+OBJS-$(CONFIG_XBR_FILTER) += vf_xbr.o
OBJS-$(CONFIG_YADIF_FILTER) += vf_yadif.o
OBJS-$(CONFIG_ZMQ_FILTER) += f_zmq.o
OBJS-$(CONFIG_ZOOMPAN_FILTER) += vf_zoompan.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index d88a9ad..2352d44 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -213,6 +213,7 @@ void avfilter_register_all(void)
REGISTER_FILTER(VIDSTABTRANSFORM, vidstabtransform, vf);
REGISTER_FILTER(VIGNETTE, vignette, vf);
REGISTER_FILTER(W3FDIF, w3fdif, vf);
+ REGISTER_FILTER(XBR, xbr, vf);
REGISTER_FILTER(YADIF, yadif, vf);
REGISTER_FILTER(ZMQ, zmq, vf);
REGISTER_FILTER(ZOOMPAN, zoompan, vf);
diff --git a/libavfilter/vf_xbr.c b/libavfilter/vf_xbr.c
new file mode 100644
index 0000000..1a828d8
--- /dev/null
+++ b/libavfilter/vf_xbr.c
@@ -0,0 +1,319 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * Copyright (c) 2014 Arwa Arif <arwaarif1...@gmail.com>
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * XBR Filter is used for depixelization of image.
+ * This is based on Hyllian's 2xBR shader.
+ * 2xBR Filter v0.2.5
+ * @see : http://www.libretro.com/forums/viewtopic.php?f=6&t=134
+ * Future work : To implement x3 and x4 scale, and threading.
+ */
+
+#include "libavutil/opt.h"
+#include "libavutil/avassert.h"
+#include "libavutil/pixdesc.h"
+#include "internal.h"
+
+typedef struct {
+ uint32_t rgbtoyuv[1<<24];
+} xBRContext;
+
+/**
+* Calculates the weight of difference of the pixels, by transforming these
+* pixels into their Y'UV parts. It then uses the threshold used by HQx filters:
+* 48*Y + 7*U + 6*V, to give it those smooth looking edges.
+**/
+static int d(AVFrame *in, int x1, int y1, int x2, int y2, const uint32_t *r2y)
+{
+
+#define YMASK 0xff0000
+#define UMASK 0x00ff00
+#define VMASK 0x0000ff
+
+ int r1 = *(in->data[0] + y1 * in->linesize[0] + x1*3);
+ int g1 = *(in->data[0] + y1 * in->linesize[0] + x1*3 + 1);
+ int b1 = *(in->data[0] + y1 * in->linesize[0] + x1*3 + 2);
+
+ int r2 = *(in->data[0] + y2 * in->linesize[0] + x2*3);
+ int g2 = *(in->data[0] + y2 * in->linesize[0] + x2*3 + 1);
+ int b2 = *(in->data[0] + y2 * in->linesize[0] + x2*3 + 2);
+
+ uint32_t c1 = r1 | g1<<8 | b1<<16 ;
+ uint32_t c2 = r2 | g2<<8 | b2<<16 ;
+
+ uint32_t yuv1 = r2y[c1 & 0xffffff];
+ uint32_t yuv2 = r2y[c2 & 0xffffff];
+
+ return abs((yuv1 & YMASK) - (yuv2 & YMASK)) > (48 << 16) ||
+ abs((yuv1 & UMASK) - (yuv2 & UMASK)) > ( 7 << 8) ||
+ abs((yuv1 & VMASK) - (yuv2 & VMASK)) > ( 6 << 0);
+}
+
+/**
+* Mixes a pixel A, with pixel B, with B's transparency set to 'a'
+* In other words, A is a solid color (bottom) and B is a transparent color (top)
+**/
+static int mix(AVFrame *in, int x1, int y1, int x2, int y2, int a, int color)
+{
+ int col1,col2;
+ col1 = *(in->data[0] + y1 * in->linesize[0] + x1*3 + color);
+ col2 = *(in->data[0] + y2 * in->linesize[0] + x2*3 + color);
+
+ return (a*col2 + (2-a)*col1)/2;
+};
+
+/**
+* Fills the output matrix
+**/
+static void fill(AVFrame *in, AVFrame *out, int u, int v, int x, int y, int mode, int new_x, int new_y)
+{
+ int r,g,b;
+ /*mix colors if they are not on boundary*/
+ if (mode!=0 && u>=0 && v>=0 && u<in->width && v<in->height) {
+
+ r = mix(in, u, v, x, y, 1, 0);
+ g = mix(in, u, v, x, y, 1, 1);
+ b = mix(in, u, v, x, y, 1, 2);
+
+ } else {
+
+ r = *(in->data[0] + y*in->linesize[0] + x*3);
+ g = *(in->data[0] + y*in->linesize[0] + x*3 + 1);
+ b = *(in->data[0] + y*in->linesize[0] + x*3 + 2);
+ }
+
+ /*Insert blended color into scaledImageData*/
+ *(out->data[0] + (new_y)*out->linesize[0] + (new_x)*3) = r;
+ *(out->data[0] + (new_y)*out->linesize[0] + (new_x)*3 + 1) = g;
+ *(out->data[0] + (new_y)*out->linesize[0] + (new_x)*3 + 2) = b;
+
+ return;
+}
+
+/**
+* Applies the xBR filter rules.
+**/
+static void apply_edge_detection_rules(AVFrame *in, AVFrame *out, int x, int y, const uint32_t *r2y)
+{
+ /* Matrix: (E is 0,0 i.e: current pixel)
+ -2 | -1| 0| +1| +2 (x)
+ ______________________________
+ -2 | [A1][B1][C1]
+ -1 | [A0][ A][ B][ C][C4]
+ 0 | [D0][ D][ E][ F][F4]
+ +1 | [G0][ G][ H][ I][I4]
+ +2 | [G5][H5][I5]
+ |(y)|
+ */
+
+ /*Cached Pixel Weight Difference*/
+ int d_E_D = d(in, x, y, x-1, y, r2y);
+ int d_E_B = d(in, x, y, x, y-1, r2y);
+ int d_E_F = d(in, x, y, x+1, y, r2y);
+ int d_E_H = d(in, x, y, x, y+1, r2y);
+ int d_E_G = d(in, x, y, x-1, y+1, r2y);
+ int d_E_C = d(in, x, y, x+1, y-1, r2y);
+ int d_A_D0 = d(in, x-1, y-1, x-2, y, r2y);
+ int d_A_B1 = d(in, x-1, y-1, x, y-2, r2y);
+ int d_D_B = d(in, x-1, y, x, y-1, r2y);
+ int d_D_H = d(in, x-1, y, x, y+1, r2y);
+ int d_D_A0 = d(in, x-1, y, x-2, y-1, r2y);
+ int d_B_F = d(in, x, y-1, x+1, y, r2y);
+ int d_B_A1 = d(in, x, y-1, x-1, y-2, r2y);
+ int d_E_A = d(in, x, y, x-1, y-1, r2y);
+ int d_E_I = d(in, x, y, x+1, y+1, r2y);
+ int d_C_F4 = d(in, x+1, y-1, x+2, y, r2y);
+ int d_C_B1 = d(in, x+1, y-1, x, y-2, r2y);
+ int d_F_H = d(in, x+1, y, x, y+1, r2y);
+ int d_F_C4 = d(in, x+1, y, x+2, y-1, r2y);
+ int d_B_C1 = d(in, x, y-1, x+1, y-2, r2y);
+ int d_G_D0 = d(in, x-1, y+1, x-2, y, r2y);
+ int d_G_H5 = d(in, x-1, y+1, x, y+2, r2y);
+ int d_H_G5 = d(in, x, y+1, x-1, y+2, r2y);
+ int d_D_G0 = d(in, x-1, y, x-2, y+1, r2y);
+ int d_I_F4 = d(in, x+1, y+1, x+2, y, r2y);
+ int d_I_H5 = d(in, x+1, y+1, x, y+2, r2y);
+ int d_H_I5 = d(in, x, y+1, x+1, y+2, r2y);
+ int d_H_I4 = d(in, x, y+1, x+2, y+1, r2y);
+
+ /**
+ * Note: On reading edge detection rules
+ *
+ * Each edge rule is an if..else statement, everytime on else, the
+ * current pixel color pointed to by matrix[0] is used to color it's edge.
+ *
+ * Each if statement checks wether the sum of weight difference on the left is
+ * lesser than that of the right weight differece.
+ */
+
+ /**
+ * Top Left Edge Detection Rule
+ **/
+ if ((d_E_G+d_E_C+d_A_D0+d_A_B1+(4*d_D_B)) < (d_D_H+d_D_A0+d_B_F+d_B_A1+(4*d_E_A))) {
+ // Figure what color to blend with current pixel -->E
+ if (d_E_D <= d_E_B)
+ fill(in, out, x-1, y, x, y, 1, x*2, y*2);
+ else
+ fill(in, out, x, y-1, x, y, 1, x*2, y*2);
+ } else {
+ /*Insert current pixel color into output frame*/
+ fill(in, out, x, y, x, y, 0, x*2, y*2);
+ }
+ /**
+ * Top Right Edge Detection Rule
+ **/
+ if ((d_E_I+d_E_A+d_C_F4+d_C_B1+(4*d_B_F)) < (d_F_H+d_F_C4+d_D_B+d_B_C1+(4*d_E_C))) {
+ // Figure what color to blend with current pixel --> E
+ if (d_E_B <= d_E_F)
+ fill(in, out, x, y-1, x, y, 1, (x*2)+1, y*2);
+ else
+ fill(in, out, x+1, y, x, y, 1, (x*2)+1, y*2);
+ } else {
+ /*Insert current pixel color into output frame*/
+ fill(in, out, x, y, x, y, 0, (x*2)+1, y*2);
+ }
+
+ /**
+ * Bottom Left Edge Detection Rule
+ **/
+ if ((d_E_A+d_E_I+d_G_D0+d_G_H5+(4*d_D_H)) < (d_D_B+d_D_G0+d_F_H+d_H_G5+(4*d_E_G))) {
+ // Figure what color to blend with current pixel --> E
+ if (d_E_D <= d_E_H)
+ fill(in, out, x-1, y, x, y, 1, x*2, (y*2)+1);
+ else
+ fill(in, out, x, y+1, x, y, 1, x*2, (y*2)+1);
+
+ } else {
+ /*Insert current pixel color into output frame*/
+ fill(in, out, x, y, x, y, 0, x*2, (y*2)+1);
+ }
+
+ /**
+ * Bottom Right Edge Detection Rule
+ **/
+ if ((d_E_C+d_E_G+d_I_F4+d_I_H5+(4*d_F_H)) < (d_D_H+d_H_I5+d_H_I4+d_B_F+(4*d_E_I))) {
+ // Figure what color to blend with current pixel --> E
+ if (d_E_F <= d_E_H)
+ fill(in, out, x+1, y, x, y, 1, (x*2)+1, (y*2)+1);
+ else
+ fill(in, out, x, y+1, x, y, 1, (x*2)+1, (y*2)+1);
+
+ } else {
+ /*Insert current pixel color into output frame*/
+ fill(in, out, x, y, x, y, 0, (x*2)+1, (y*2)+1);
+ }
+}
+
+static int config_output(AVFilterLink *outlink)
+{
+ AVFilterContext *ctx = outlink->src;
+ AVFilterLink *inlink = ctx->inputs[0];
+
+ outlink->w = inlink->w * 2 ;
+ outlink->h = inlink->h * 2 ;
+ return 0;
+}
+
+static int query_formats(AVFilterContext *ctx)
+{
+ static const enum AVPixelFormat pix_fmts[] = {
+ AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24,AV_PIX_FMT_NONE,
+ };
+
+ ff_set_common_formats(ctx, ff_make_format_list(pix_fmts));
+ return 0;
+}
+
+static int filter_frame(AVFilterLink *inlink, AVFrame *in)
+{
+ AVFilterContext *ctx = inlink->dst;
+ AVFilterLink *outlink = ctx->outputs[0];
+ int i,j;
+ xBRContext *xBR = ctx->priv;
+ const uint32_t *r2y = xBR->rgbtoyuv;
+ AVFrame *out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
+ if (!out) {
+ av_frame_free(&in);
+ return AVERROR(ENOMEM);
+ }
+
+ av_frame_copy_props(out, in);
+ for (i=0; i<inlink->w; i++)
+ for (j=0; j<inlink->h; j++)
+ apply_edge_detection_rules(in, out, i, j, r2y);
+
+ out->width = outlink->w;
+ out->height = outlink->h;
+
+ av_frame_free(&in);
+ return ff_filter_frame(outlink, out);
+}
+
+static int init(AVFilterContext *ctx)
+{
+ xBRContext *xbr = ctx->priv;
+ uint32_t c;
+ int bg, rg, g;
+
+ for (bg=-255; bg<256; bg++) {
+ for (rg=-255; rg<256; rg++) {
+ const uint32_t u = (uint32_t)((-169*rg + 500*bg)/1000) + 128;
+ const uint32_t v = (uint32_t)(( 500*rg - 81*bg)/1000) + 128;
+ int startg = FFMAX3(-bg, -rg, 0);
+ int endg = FFMIN3(255-bg, 255-rg, 255);
+ uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000);
+ c = bg + (rg<<16) + 0x010101 * startg;
+ for (g = startg; g <= endg; g++) {
+ xbr->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v;
+ c+= 0x010101;
+ }
+ }
+ }
+ return 0;
+}
+
+static const AVFilterPad xbr_inputs[] = {
+ {
+ .name = "default",
+ .type = AVMEDIA_TYPE_VIDEO,
+ .filter_frame = filter_frame,
+ },
+ { NULL }
+};
+
+static const AVFilterPad xbr_outputs[] = {
+ {
+ .name = "default",
+ .type = AVMEDIA_TYPE_VIDEO,
+ .config_props = config_output,
+ },
+ { NULL }
+};
+
+AVFilter ff_vf_xbr = {
+ .name = "xbr",
+ .description = NULL_IF_CONFIG_SMALL("Scale the input by 2 using xbr algorithm."),
+ .priv_size = sizeof(xBRContext),
+ .inputs = xbr_inputs,
+ .outputs = xbr_outputs,
+ .query_formats = query_formats,
+ .init = init,
+};
--
1.7.9.5
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
