Re: [libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-23 Thread Janne Grunau
On 2014-02-22 21:34:19 +0100, Guillaume Martres wrote: On Wednesday 19 February 2014 22:43:13 Luca Barbato wrote: +if (x 0) +x_pos = -((-x / 2) s-sps-pixel_shift); This seems very weird to me, x and y are used to get an index inside the

Re: [libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-22 Thread Guillaume Martres
On Wednesday 19 February 2014 22:43:13 Luca Barbato wrote: +if (x 0) +x_pos = -((-x / 2) s-sps-pixel_shift); This seems very weird to me, x and y are used to get an index inside the chroma frame so x should never be negative.

Re: [libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-22 Thread Luca Barbato
On 22/02/14 21:34, Guillaume Martres wrote: On Wednesday 19 February 2014 22:43:13 Luca Barbato wrote: +if (x 0) +x_pos = -((-x / 2) s-sps-pixel_shift); This seems very weird to me, x and y are used to get an index inside the chroma frame so x

Re: [libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-21 Thread Janne Grunau
On 2014-02-21 00:10:05 +0100, Luca Barbato wrote: On 20/02/14 23:58, Janne Grunau wrote: Also after a cursory glance x can only be smaller than 0 if x0 is smaller than 0. There are some right shifts of x0 i the same function which are implementation defined. IIRC x0 should be at least

Re: [libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-20 Thread Janne Grunau
On 2014-02-19 22:43:13 +0100, Luca Barbato wrote: --- libavcodec/hevc_filter.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/libavcodec/hevc_filter.c b/libavcodec/hevc_filter.c index eb3f5f3..8f425d8 100644 --- a/libavcodec/hevc_filter.c +++

Re: [libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-20 Thread Luca Barbato
On 20/02/14 23:58, Janne Grunau wrote: On 2014-02-19 22:43:13 +0100, Luca Barbato wrote: --- libavcodec/hevc_filter.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/libavcodec/hevc_filter.c b/libavcodec/hevc_filter.c index eb3f5f3..8f425d8 100644 ---

[libav-devel] [PATCH 7/7] hevc: Do not left shift a negative value

2014-02-19 Thread Luca Barbato
--- libavcodec/hevc_filter.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/libavcodec/hevc_filter.c b/libavcodec/hevc_filter.c index eb3f5f3..8f425d8 100644 --- a/libavcodec/hevc_filter.c +++ b/libavcodec/hevc_filter.c @@ -485,11 +485,17 @@ static void