Re: [FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Move reslevelno check before use in case JPEG2000_PGOD_RPCL

2020-07-27 Thread Gautam Ramakrishnan
On Mon, Jul 27, 2020 at 1:46 PM Michael Niedermayer
 wrote:
>
> On Mon, Jul 27, 2020 at 09:55:31AM +0530, Gautam Ramakrishnan wrote:
> > On Mon, Jul 27, 2020 at 9:48 AM Gautam Ramakrishnan
> >  wrote:
> > >
> > > On Mon, Jul 27, 2020 at 3:16 AM Michael Niedermayer
> > >  wrote:
> > > >
> > > > Fixes: division by zero
> > > > Fixes: 
> > > > 24201/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5665813827420160
> > > > Fixes: 
> > > > 24245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6285831682392064
> > > >
> > > > Found-by: continuous fuzzing process 
> > > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > > Signed-off-by: Michael Niedermayer 
> > > > ---
> > > >  libavcodec/jpeg2000dec.c | 6 +++---
> > > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
> > > > index a470cf47da..b168e52db6 100644
> > > > --- a/libavcodec/jpeg2000dec.c
> > > > +++ b/libavcodec/jpeg2000dec.c
> > > > @@ -1401,12 +1401,12 @@ static int 
> > > > jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2
> > > >  if (!s->cdx[compno] || !s->cdy[compno])
> > > >  return AVERROR_INVALIDDATA;
> > > >
> > > > -trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> > > > s->cdx[compno] << reducedresno);
> > > > -try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> > > > s->cdy[compno] << reducedresno);
> > > > -
> > > >  if (reslevelno >= codsty->nreslevels)
> > > >  continue;
> > > >
> > > > +trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> > > > s->cdx[compno] << reducedresno);
> > > > +try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> > > > s->cdy[compno] << reducedresno);
> > > > +
> > > >  if (!(y % ((uint64_t)s->cdy[compno] << 
> > > > (rlevel->log2_prec_height + reducedresno)) == 0 ||
> > > >   (y == tile->coord[1][0] && (try0 << 
> > > > reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height)
> > > >  continue;
> > > > --
> > > > 2.17.1
> > > >
> > > > ___
> > > > ffmpeg-devel mailing list
> > > > ffmpeg-devel@ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> > >
> > > Looks good to me.
> > > I guess the division by zero happens because shifting by reducedresno
> > > overflows?
>
> yes
>
> > >
> > > --
> > > -
> > > Gautam |
> > Just realized, this might happen in one more place, shall send a
> > patch for the same.
>
> yes
Went through the code again, looks like this would not occur in other cases.
The other loops take into account whether reslvlno < codsty->nreslevels

I think this might be the only patch necessary
>
> thx
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Dictatorship naturally arises out of democracy, and the most aggravated
> form of tyranny and slavery out of the most extreme liberty. -- Plato
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".



-- 
-
Gautam |
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Move reslevelno check before use in case JPEG2000_PGOD_RPCL

2020-07-27 Thread Michael Niedermayer
On Mon, Jul 27, 2020 at 09:55:31AM +0530, Gautam Ramakrishnan wrote:
> On Mon, Jul 27, 2020 at 9:48 AM Gautam Ramakrishnan
>  wrote:
> >
> > On Mon, Jul 27, 2020 at 3:16 AM Michael Niedermayer
> >  wrote:
> > >
> > > Fixes: division by zero
> > > Fixes: 
> > > 24201/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5665813827420160
> > > Fixes: 
> > > 24245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6285831682392064
> > >
> > > Found-by: continuous fuzzing process 
> > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > Signed-off-by: Michael Niedermayer 
> > > ---
> > >  libavcodec/jpeg2000dec.c | 6 +++---
> > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
> > > index a470cf47da..b168e52db6 100644
> > > --- a/libavcodec/jpeg2000dec.c
> > > +++ b/libavcodec/jpeg2000dec.c
> > > @@ -1401,12 +1401,12 @@ static int 
> > > jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2
> > >  if (!s->cdx[compno] || !s->cdy[compno])
> > >  return AVERROR_INVALIDDATA;
> > >
> > > -trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> > > s->cdx[compno] << reducedresno);
> > > -try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> > > s->cdy[compno] << reducedresno);
> > > -
> > >  if (reslevelno >= codsty->nreslevels)
> > >  continue;
> > >
> > > +trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> > > s->cdx[compno] << reducedresno);
> > > +try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> > > s->cdy[compno] << reducedresno);
> > > +
> > >  if (!(y % ((uint64_t)s->cdy[compno] << 
> > > (rlevel->log2_prec_height + reducedresno)) == 0 ||
> > >   (y == tile->coord[1][0] && (try0 << 
> > > reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height)
> > >  continue;
> > > --
> > > 2.17.1
> > >
> > > ___
> > > ffmpeg-devel mailing list
> > > ffmpeg-devel@ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
> >
> > Looks good to me.
> > I guess the division by zero happens because shifting by reducedresno
> > overflows?

yes

> >
> > --
> > -
> > Gautam |
> Just realized, this might happen in one more place, shall send a
> patch for the same.

yes

thx

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Dictatorship naturally arises out of democracy, and the most aggravated
form of tyranny and slavery out of the most extreme liberty. -- Plato


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Move reslevelno check before use in case JPEG2000_PGOD_RPCL

2020-07-26 Thread Gautam Ramakrishnan
On Mon, Jul 27, 2020 at 3:16 AM Michael Niedermayer
 wrote:
>
> Fixes: division by zero
> Fixes: 
> 24201/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5665813827420160
> Fixes: 
> 24245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6285831682392064
>
> Found-by: continuous fuzzing process 
> https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer 
> ---
>  libavcodec/jpeg2000dec.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
> index a470cf47da..b168e52db6 100644
> --- a/libavcodec/jpeg2000dec.c
> +++ b/libavcodec/jpeg2000dec.c
> @@ -1401,12 +1401,12 @@ static int 
> jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2
>  if (!s->cdx[compno] || !s->cdy[compno])
>  return AVERROR_INVALIDDATA;
>
> -trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> s->cdx[compno] << reducedresno);
> -try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> s->cdy[compno] << reducedresno);
> -
>  if (reslevelno >= codsty->nreslevels)
>  continue;
>
> +trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> s->cdx[compno] << reducedresno);
> +try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> s->cdy[compno] << reducedresno);
> +
>  if (!(y % ((uint64_t)s->cdy[compno] << 
> (rlevel->log2_prec_height + reducedresno)) == 0 ||
>   (y == tile->coord[1][0] && (try0 << 
> reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height)
>  continue;
> --
> 2.17.1
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Looks good to me.
I guess the division by zero happens because shifting by reducedresno
overflows?

-- 
-
Gautam |
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Move reslevelno check before use in case JPEG2000_PGOD_RPCL

2020-07-26 Thread Gautam Ramakrishnan
On Mon, Jul 27, 2020 at 9:48 AM Gautam Ramakrishnan
 wrote:
>
> On Mon, Jul 27, 2020 at 3:16 AM Michael Niedermayer
>  wrote:
> >
> > Fixes: division by zero
> > Fixes: 
> > 24201/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5665813827420160
> > Fixes: 
> > 24245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6285831682392064
> >
> > Found-by: continuous fuzzing process 
> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > Signed-off-by: Michael Niedermayer 
> > ---
> >  libavcodec/jpeg2000dec.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
> > index a470cf47da..b168e52db6 100644
> > --- a/libavcodec/jpeg2000dec.c
> > +++ b/libavcodec/jpeg2000dec.c
> > @@ -1401,12 +1401,12 @@ static int 
> > jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2
> >  if (!s->cdx[compno] || !s->cdy[compno])
> >  return AVERROR_INVALIDDATA;
> >
> > -trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> > s->cdx[compno] << reducedresno);
> > -try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> > s->cdy[compno] << reducedresno);
> > -
> >  if (reslevelno >= codsty->nreslevels)
> >  continue;
> >
> > +trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
> > s->cdx[compno] << reducedresno);
> > +try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
> > s->cdy[compno] << reducedresno);
> > +
> >  if (!(y % ((uint64_t)s->cdy[compno] << 
> > (rlevel->log2_prec_height + reducedresno)) == 0 ||
> >   (y == tile->coord[1][0] && (try0 << 
> > reducedresno) % (1U << (reducedresno + rlevel->log2_prec_height)
> >  continue;
> > --
> > 2.17.1
> >
> > ___
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
> Looks good to me.
> I guess the division by zero happens because shifting by reducedresno
> overflows?
>
> --
> -
> Gautam |
Just realized, this might happen in one more place, shall send a
patch for the same.


-- 
-
Gautam |
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Move reslevelno check before use in case JPEG2000_PGOD_RPCL

2020-07-26 Thread Michael Niedermayer
Fixes: division by zero
Fixes: 
24201/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5665813827420160
Fixes: 
24245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6285831682392064

Found-by: continuous fuzzing process 
https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer 
---
 libavcodec/jpeg2000dec.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
index a470cf47da..b168e52db6 100644
--- a/libavcodec/jpeg2000dec.c
+++ b/libavcodec/jpeg2000dec.c
@@ -1401,12 +1401,12 @@ static int 
jpeg2000_decode_packets_po_iteration(Jpeg2000DecoderContext *s, Jpeg2
 if (!s->cdx[compno] || !s->cdy[compno])
 return AVERROR_INVALIDDATA;
 
-trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
s->cdx[compno] << reducedresno);
-try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
s->cdy[compno] << reducedresno);
-
 if (reslevelno >= codsty->nreslevels)
 continue;
 
+trx0 = ff_jpeg2000_ceildiv(tile->coord[0][0], 
s->cdx[compno] << reducedresno);
+try0 = ff_jpeg2000_ceildiv(tile->coord[1][0], 
s->cdy[compno] << reducedresno);
+
 if (!(y % ((uint64_t)s->cdy[compno] << 
(rlevel->log2_prec_height + reducedresno)) == 0 ||
  (y == tile->coord[1][0] && (try0 << reducedresno) 
% (1U << (reducedresno + rlevel->log2_prec_height)
 continue;
-- 
2.17.1

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".