[FFmpeg-cvslog] swscale/x86/rgb2rgb_template: Do not crash on misaligend stride

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: master | Michael Niedermayer  | Tue 
Dec 15 02:06:04 2015 +0100| [80bfce35ccd11458e97f68f417fc094c5347070c] | 
committer: Michael Niedermayer

swscale/x86/rgb2rgb_template: Do not crash on misaligend stride

Fixes Ticket5013

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=80bfce35ccd11458e97f68f417fc094c5347070c
---

 libswscale/x86/rgb2rgb_template.c |5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libswscale/x86/rgb2rgb_template.c 
b/libswscale/x86/rgb2rgb_template.c
index e97ba4f..6524461 100644
--- a/libswscale/x86/rgb2rgb_template.c
+++ b/libswscale/x86/rgb2rgb_template.c
@@ -1887,8 +1887,10 @@ static void RENAME(interleaveBytes)(const uint8_t *src1, 
const uint8_t *src2, ui
 for (h=0; h < height; h++) {
 int w;
 
-if (width >= 16)
+if (width >= 16
 #if COMPILE_TEMPLATE_SSE2
+&& !intptr_t)src1) | ((intptr_t)src2) | ((intptr_t)dest))&15)
+)
 __asm__(
 "xor  %%"REG_a", %%"REG_a"  \n\t"
 "1: \n\t"
@@ -1908,6 +1910,7 @@ static void RENAME(interleaveBytes)(const uint8_t *src1, 
const uint8_t *src2, ui
 : "memory", XMM_CLOBBERS("xmm0", "xmm1", "xmm2",) "%"REG_a
 );
 #else
+)
 __asm__(
 "xor %%"REG_a", %%"REG_a"   \n\t"
 "1: \n\t"

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avfilter/vf_decimate: fix typo in fraction

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: master | Michael Niedermayer  | Mon 
Dec 14 22:59:38 2015 +0100| [06987dab972e275a70ea961be27bc12ff531da75] | 
committer: Michael Niedermayer

avfilter/vf_decimate: fix typo in fraction

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=06987dab972e275a70ea961be27bc12ff531da75
---

 libavfilter/vf_decimate.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavfilter/vf_decimate.c b/libavfilter/vf_decimate.c
index a79fc02..cd374c3 100644
--- a/libavfilter/vf_decimate.c
+++ b/libavfilter/vf_decimate.c
@@ -217,7 +217,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in)
 av_frame_free(&frame);
 frame = dm->clean_src[i];
 }
-frame->pts = av_rescale_q(outlink->frame_count, dm->ts_unit, 
(AVRational){1,0}) +
+frame->pts = av_rescale_q(outlink->frame_count, dm->ts_unit, 
(AVRational){1,1}) +
  (dm->start_pts == AV_NOPTS_VALUE ? 0 : dm->start_pts);
 ret = ff_filter_frame(outlink, frame);
 if (ret < 0)

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] configure: add conditional library dependency for sofalizer and showfreqs

2015-12-14 Thread Paul B Mahol
ffmpeg | branch: master | Paul B Mahol  | Mon Dec 14 22:41:52 
2015 +0100| [e8586ecb86f461aa7194e6882e699a336d92d2a1] | committer: Paul B Mahol

configure: add conditional library dependency for sofalizer and showfreqs

Signed-off-by: Paul B Mahol 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e8586ecb86f461aa7194e6882e699a336d92d2a1
---

 configure |2 ++
 1 file changed, 2 insertions(+)

diff --git a/configure b/configure
index 04deb2a..4ada587 100755
--- a/configure
+++ b/configure
@@ -6020,6 +6020,8 @@ enabled resample_filter && prepend avfilter_deps 
"avresample"
 enabled sab_filter  && prepend avfilter_deps "swscale"
 enabled scale_filter&& prepend avfilter_deps "swscale"
 enabled scale2ref_filter&& prepend avfilter_deps "swscale"
+enabled sofalizer_filter&& prepend avfilter_deps "avcodec"
+enabled showfreqs_filter&& prepend avfilter_deps "avcodec"
 enabled showspectrum_filter && prepend avfilter_deps "avcodec"
 enabled smartblur_filter&& prepend avfilter_deps "swscale"
 enabled subtitles_filter&& prepend avfilter_deps "avformat avcodec"

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avfilter/af_sofalizer: add frequency domain processing and use it by default

2015-12-14 Thread Paul B Mahol
ffmpeg | branch: master | Paul B Mahol  | Sun Dec 13 23:05:09 
2015 +0100| [2f12172d670996ff8f18b80ebdee7d0a8c230ac3] | committer: Paul B Mahol

avfilter/af_sofalizer: add frequency domain processing and use it by default

Code ported from SOFAlizer patch for VLC.

Signed-off-by: Paul B Mahol 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=2f12172d670996ff8f18b80ebdee7d0a8c230ac3
---

 configure  |3 +-
 doc/filters.texi   |6 +
 libavfilter/af_sofalizer.c |  297 +++-
 3 files changed, 277 insertions(+), 29 deletions(-)

diff --git a/configure b/configure
index 43fa9a6..04deb2a 100755
--- a/configure
+++ b/configure
@@ -2892,7 +2892,8 @@ showfreqs_filter_deps="avcodec"
 showfreqs_filter_select="fft"
 showspectrum_filter_deps="avcodec"
 showspectrum_filter_select="rdft"
-sofalizer_filter_deps="netcdf"
+sofalizer_filter_deps="netcdf avcodec"
+sofalizer_filter_select="fft"
 spp_filter_deps="gpl avcodec"
 spp_filter_select="fft idctdsp fdctdsp me_cmp pixblockdsp"
 stereo3d_filter_deps="gpl"
diff --git a/doc/filters.texi b/doc/filters.texi
index ba2ffc4..78fbd47 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -2916,6 +2916,12 @@ Set elevation of virtual speakers in deg. Default is 0.
 @item radius
 Set distance in meters between loudspeakers and the listener with near-field
 HRTFs. Default is 1.
+
+@item type
+Set processing type. Can be @var{time} or @var{freq}. @var{time} is
+processing audio in time domain which is slow but gives high quality output.
+@var{freq} is processing audio in frequency domain which is fast but gives
+mediocre output. Default is @var{freq}.
 @end table
 
 @section stereotools
diff --git a/libavfilter/af_sofalizer.c b/libavfilter/af_sofalizer.c
index bcb3519..0aaae4b 100644
--- a/libavfilter/af_sofalizer.c
+++ b/libavfilter/af_sofalizer.c
@@ -28,12 +28,16 @@
 #include 
 #include 
 
+#include "libavcodec/avfft.h"
 #include "libavutil/float_dsp.h"
 #include "libavutil/opt.h"
 #include "avfilter.h"
 #include "internal.h"
 #include "audio.h"
 
+#define TIME_DOMAIN  0
+#define FREQUENCY_DOMAIN 1
+
 typedef struct NCSofa {  /* contains data of one SOFA file */
 int ncid;/* netCDF ID of the opened SOFA file */
 int n_samples;   /* length of one impulse response (IR) */
@@ -67,6 +71,7 @@ typedef struct SOFAlizerContext {
 int write[2];   /* current write position to ringbuffer */
 int buffer_length;  /* is: longest IR plus max. delay in all SOFA 
files */
 /* then choose next power of 2 */
+int n_fft;  /* number of samples in one FFT block */
 
 /* netCDF variables */
 int *delay[2];  /* broadband delay for each channel/IR to be 
convolved */
@@ -74,12 +79,17 @@ typedef struct SOFAlizerContext {
 float *data_ir[2];  /* IRs for all channels to be convolved */
 /* (this excludes the LFE) */
 float *temp_src[2];
+FFTComplex *temp_fft[2];
 
  /* control variables */
 float gain;  /* filter gain (in dB) */
 float rotation;  /* rotation of virtual loudspeakers (in degrees)  */
 float elevation; /* elevation of virtual loudspeakers (in deg.) */
 float radius;/* distance virtual loudspeakers to listener (in 
metres) */
+int type;/* processing type */
+
+FFTContext *fft[2], *ifft[2];
+FFTComplex *data_hrtf[2];
 
 AVFloatDSPContext *fdsp;
 } SOFAlizerContext;
@@ -259,11 +269,8 @@ static int load_sofa(AVFilterContext *ctx, char *filename, 
int *samplingrate)
 /* delay and IR values required for each ear and measurement position: */
 data_delay = s->sofa.data_delay = av_calloc(m_dim, 2 * sizeof(int));
 data_ir = s->sofa.data_ir = av_malloc_array(m_dim * n_samples, 
sizeof(float) * 2);
-s->temp_src[0] = av_calloc(FFALIGN(n_samples, 16), sizeof(float));
-s->temp_src[1] = av_calloc(FFALIGN(n_samples, 16), sizeof(float));
 
-if (!data_delay || !sp_a || !sp_e || !sp_r || !data_ir ||
-!s->temp_src[0] || !s->temp_src[1]) {
+if (!data_delay || !sp_a || !sp_e || !sp_r || !data_ir) {
 /* if memory could not be allocated */
 close_sofa(&s->sofa);
 return AVERROR(ENOMEM);
@@ -590,6 +597,7 @@ typedef struct ThreadData {
 int *n_clippings;
 float **ringbuffer;
 float **temp_src;
+FFTComplex **temp_fft;
 } ThreadData;
 
 static int sofalizer_convolute(AVFilterContext *ctx, void *arg, int jobnr, int 
nb_jobs)
@@ -678,6 +686,120 @@ static int sofalizer_convolute(AVFilterContext *ctx, void 
*arg, int jobnr, int n
 return 0;
 }
 
+static int sofalizer_fast_convolute(AVFilterContext *ctx, void *arg, int 
jobnr, int nb_jobs)
+{
+SOFAlizerContext *s = ctx->priv;
+ThreadData *td = arg;
+AVFrame *in = td->in, *out = td->out;
+int offset = jobnr;
+  

[FFmpeg-cvslog] avfilter/vf_decimate: Check that input parameters match

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: master | Michael Niedermayer  | Mon 
Dec 14 21:18:39 2015 +0100| [30fe3fd52721c8c6566001192cd16be423ffc92b] | 
committer: Michael Niedermayer

avfilter/vf_decimate: Check that input parameters match

Fixes Ticket4964

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=30fe3fd52721c8c6566001192cd16be423ffc92b
---

 libavfilter/vf_decimate.c |9 +
 1 file changed, 9 insertions(+)

diff --git a/libavfilter/vf_decimate.c b/libavfilter/vf_decimate.c
index e580d05..a79fc02 100644
--- a/libavfilter/vf_decimate.c
+++ b/libavfilter/vf_decimate.c
@@ -362,6 +362,8 @@ static int config_output(AVFilterLink *outlink)
 DecimateContext *dm = ctx->priv;
 const AVFilterLink *inlink =
 ctx->inputs[dm->ppsrc ? INPUT_CLEANSRC : INPUT_MAIN];
+const AVFilterLink *inlink_main =
+ctx->inputs[INPUT_MAIN];
 AVRational fps = inlink->frame_rate;
 
 if (!fps.num || !fps.den) {
@@ -369,6 +371,13 @@ static int config_output(AVFilterLink *outlink)
"current rate of %d/%d is invalid\n", fps.num, fps.den);
 return AVERROR(EINVAL);
 }
+
+if (inlink->w != inlink_main->w ||
+inlink->h != inlink_main->h ||
+inlink->format != inlink_main->format) {
+av_log(ctx, AV_LOG_ERROR, "frame parameters differ between inputs\n");
+return AVERROR_PATCHWELCOME;
+}
 fps = av_mul_q(fps, (AVRational){dm->cycle - 1, dm->cycle});
 av_log(ctx, AV_LOG_VERBOSE, "FPS: %d/%d -> %d/%d\n",
inlink->frame_rate.num, inlink->frame_rate.den, fps.num, fps.den);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] golomb: always check for invalid UE golomb codes in get_ue_golomb

2015-12-14 Thread Andreas Cadhalpun
ffmpeg | branch: master | Andreas Cadhalpun  
| Sun Dec 13 21:02:16 2015 +0100| [22e960ad478e568f4094971a58c6ad8f549c0180] | 
committer: Andreas Cadhalpun

golomb: always check for invalid UE golomb codes in get_ue_golomb

Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.

This fixes undefined behavior:
runtime error: shift exponent is negative

Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:

With the check in the B branch:
629 decicycles in get_ue_golomb B, 4194260 runs, 44 skips
433 decicycles in get_ue_golomb A,268434102 runs,   1354 skips

Without the check:
624 decicycles in get_ue_golomb B, 4194273 runs, 31 skips
433 decicycles in get_ue_golomb A,268434203 runs,   1253 skips

Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.

Fixes: mozilla bug 1230239
Fixes: 
fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit

Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer 
Signed-off-by: Andreas Cadhalpun 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=22e960ad478e568f4094971a58c6ad8f549c0180
---

 libavcodec/golomb.h |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/golomb.h b/libavcodec/golomb.h
index d30bb6b..5136a04 100644
--- a/libavcodec/golomb.h
+++ b/libavcodec/golomb.h
@@ -68,7 +68,7 @@ static inline int get_ue_golomb(GetBitContext *gb)
 int log = 2 * av_log2(buf) - 31;
 LAST_SKIP_BITS(re, gb, 32 - log);
 CLOSE_READER(re, gb);
-if (CONFIG_FTRAPV && log < 0) {
+if (log < 7) {
 av_log(NULL, AV_LOG_ERROR, "Invalid UE golomb code\n");
 return AVERROR_INVALIDDATA;
 }

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avfilter/vf_decimate: change ts_unit to AVRational

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: master | Michael Niedermayer  | Mon 
Dec 14 20:24:21 2015 +0100| [1925eaeaa6f2582660571701d4f20d102b960e4a] | 
committer: Michael Niedermayer

avfilter/vf_decimate: change ts_unit to AVRational

This might help with rounding differences between platforms

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=1925eaeaa6f2582660571701d4f20d102b960e4a
---

 libavfilter/vf_decimate.c |6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavfilter/vf_decimate.c b/libavfilter/vf_decimate.c
index 26f3ce0..e580d05 100644
--- a/libavfilter/vf_decimate.c
+++ b/libavfilter/vf_decimate.c
@@ -42,7 +42,7 @@ typedef struct {
 AVFrame *last;  ///< last frame from the previous queue
 AVFrame **clean_src;///< frame queue for the clean source
 int got_frame[2];   ///< frame request flag for each input stream
-double ts_unit; ///< timestamp units for the output frames
+AVRational ts_unit; ///< timestamp units for the output frames
 int64_t start_pts;  ///< base for output timestamps
 uint32_t eof;   ///< bitmask for end of stream
 int hsub, vsub; ///< chroma subsampling values
@@ -217,7 +217,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in)
 av_frame_free(&frame);
 frame = dm->clean_src[i];
 }
-frame->pts = outlink->frame_count * dm->ts_unit +
+frame->pts = av_rescale_q(outlink->frame_count, dm->ts_unit, 
(AVRational){1,0}) +
  (dm->start_pts == AV_NOPTS_VALUE ? 0 : dm->start_pts);
 ret = ff_filter_frame(outlink, frame);
 if (ret < 0)
@@ -377,7 +377,7 @@ static int config_output(AVFilterLink *outlink)
 outlink->sample_aspect_ratio = inlink->sample_aspect_ratio;
 outlink->w = inlink->w;
 outlink->h = inlink->h;
-dm->ts_unit = av_q2d(av_inv_q(av_mul_q(fps, outlink->time_base)));
+dm->ts_unit = av_inv_q(av_mul_q(fps, outlink->time_base));
 return 0;
 }
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avfilter/vf_mpdecimate: Add missing emms_c()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: master | Michael Niedermayer  | Mon 
Dec 14 18:56:13 2015 +0100| [997de2e8107cc4256e50611463d609b18fe9619f] | 
committer: Michael Niedermayer

avfilter/vf_mpdecimate: Add missing emms_c()

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=997de2e8107cc4256e50611463d609b18fe9619f
---

 libavfilter/vf_mpdecimate.c |5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libavfilter/vf_mpdecimate.c b/libavfilter/vf_mpdecimate.c
index 25efacf..20b15a2 100644
--- a/libavfilter/vf_mpdecimate.c
+++ b/libavfilter/vf_mpdecimate.c
@@ -131,10 +131,13 @@ static int decimate_frame(AVFilterContext *ctx,
 cur->data[plane], cur->linesize[plane],
 ref->data[plane], ref->linesize[plane],
 FF_CEIL_RSHIFT(ref->width,  hsub),
-FF_CEIL_RSHIFT(ref->height, vsub)))
+FF_CEIL_RSHIFT(ref->height, vsub))) {
+emms_c();
 return 0;
+}
 }
 
+emms_c();
 return 1;
 }
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] aacenc: switch to using the RNG from libavutil

2015-12-14 Thread Rostislav Pehlivanov
ffmpeg | branch: master | Rostislav Pehlivanov  | Mon Dec 
14 18:53:09 2015 +| [ade31b9424f6bb8f70f277b1acb4575d312ed955] | committer: 
Rostislav Pehlivanov

aacenc: switch to using the RNG from libavutil

PSNR doesn't change as expected. The AAC spec doesn't really say
anything about how exactly to generate noise.

Signed-off-by: Rostislav Pehlivanov 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ade31b9424f6bb8f70f277b1acb4575d312ed955
---

 libavcodec/aaccoder.c |8 ++--
 libavcodec/aacenc.c   |2 +-
 libavcodec/aacenc.h   |2 ++
 libavcodec/aacenc_utils.h |   10 --
 4 files changed, 9 insertions(+), 13 deletions(-)

diff --git a/libavcodec/aaccoder.c b/libavcodec/aaccoder.c
index 7e55494..15d467b 100644
--- a/libavcodec/aaccoder.c
+++ b/libavcodec/aaccoder.c
@@ -693,8 +693,12 @@ static void search_for_pns(AACEncContext *s, 
AVCodecContext *avctx, SingleChanne
 float band_energy, scale, pns_senergy;
 const int start_c = (w+w2)*128+sce->ics.swb_offset[g];
 band = &s->psy.ch[s->cur_channel].psy_bands[(w+w2)*16+g];
-for (i = 0; i < sce->ics.swb_sizes[g]; i++)
-PNS[i] = s->random_state = lcg_random(s->random_state);
+for (i = 0; i < sce->ics.swb_sizes[g]; i+=2) {
+double rnd[2];
+av_bmg_get(&s->lfg, rnd);
+PNS[i+0] = (float)rnd[0];
+PNS[i+1] = (float)rnd[1];
+}
 band_energy = s->fdsp->scalarproduct_float(PNS, PNS, 
sce->ics.swb_sizes[g]);
 scale = noise_amp/sqrtf(band_energy);
 s->fdsp->vector_fmul_scalar(PNS, PNS, scale, 
sce->ics.swb_sizes[g]);
diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c
index 3406f43..ec09063 100644
--- a/libavcodec/aacenc.c
+++ b/libavcodec/aacenc.c
@@ -904,7 +904,6 @@ static av_cold int aac_encode_init(AVCodecContext *avctx)
 
 s->channels = avctx->channels;
 s->chan_map = aac_chan_configs[s->channels-1];
-s->random_state = 0x1f2e3d4c;
 s->lambda = avctx->global_quality > 0 ? avctx->global_quality : 120;
 avctx->extradata_size = 5;
 avctx->frame_size = 1024;
@@ -997,6 +996,7 @@ static av_cold int aac_encode_init(AVCodecContext *avctx)
 goto fail;
 s->psypp = ff_psy_preprocess_init(avctx);
 ff_lpc_init(&s->lpc, 2*avctx->frame_size, TNS_MAX_ORDER, 
FF_LPC_TYPE_LEVINSON);
+av_lfg_init(&s->lfg, 0x72adca55);
 
 if (HAVE_MIPSDSP)
 ff_aac_coder_init_mips(s);
diff --git a/libavcodec/aacenc.h b/libavcodec/aacenc.h
index d8bed82..2b721d3 100644
--- a/libavcodec/aacenc.h
+++ b/libavcodec/aacenc.h
@@ -23,6 +23,7 @@
 #define AVCODEC_AACENC_H
 
 #include "libavutil/float_dsp.h"
+#include "libavutil/lfg.h"
 #include "avcodec.h"
 #include "put_bits.h"
 
@@ -100,6 +101,7 @@ typedef struct AACEncContext {
 FFTContext mdct1024; ///< long (1024 samples) 
frame transform context
 FFTContext mdct128;  ///< short (128 samples) 
frame transform context
 AVFloatDSPContext *fdsp;
+AVLFG lfg;   ///< PRNG needed for PNS
 float *planar_samples[8];///< saved preprocessed input
 
 int profile; ///< copied from avctx
diff --git a/libavcodec/aacenc_utils.h b/libavcodec/aacenc_utils.h
index 5b308f2..736e4a0 100644
--- a/libavcodec/aacenc_utils.h
+++ b/libavcodec/aacenc_utils.h
@@ -183,16 +183,6 @@ static av_always_inline float bval2bmax(float b)
 }
 
 /*
- * linear congruential pseudorandom number generator, copied from the decoder
- */
-static inline int lcg_random(unsigned previous_val)
-{
-union { unsigned u; int s; } v = { previous_val * 1664525u + 1013904223 };
-return v.s;
-}
-
-
-/*
  * Compute a nextband map to be used with SF delta constraint utilities.
  * The nextband array should contain 128 elements, and positions that don't
  * map to valid, nonzero bands of the form w*16+g (with w being the initial

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Tue Dec  1 12:44:23 2015 +0100| [88ccca204ab67d8eda5745a94eed69882af6a3dc] | 
committer: Michael Niedermayer

avutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd

The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative

Signed-off-by: Michael Niedermayer 
(cherry picked from commit 25e37f5ea92d4201976a59ae306ce848d257a7e6)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=88ccca204ab67d8eda5745a94eed69882af6a3dc
---

 libavutil/mathematics.c |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavutil/mathematics.c b/libavutil/mathematics.c
index 126cffc..b1ffd65 100644
--- a/libavutil/mathematics.c
+++ b/libavutil/mathematics.c
@@ -76,8 +76,8 @@ int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, enum 
AVRounding rnd)
 rnd -= AV_ROUND_PASS_MINMAX;
 }
 
-if (a < 0 && a != INT64_MIN)
-return -av_rescale_rnd(-a, b, c, rnd ^ ((rnd >> 1) & 1));
+if (a < 0)
+return -av_rescale_rnd(-FFMAX(a, -INT64_MAX), b, c, rnd ^ ((rnd >> 1) 
& 1));
 
 if (rnd == AV_ROUND_NEAR_INF)
 r = c / 2;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/apedec: Check length in long_filter_high_3800()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Wed Dec  2 21:16:27 2015 +0100| [d259a0534ee2f89b103cfc52b988b452c6411662] | 
committer: Michael Niedermayer

avcodec/apedec: Check length in long_filter_high_3800()

Fixes out of array read
Fixes: 
0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit cd7524fdd13dc8d0cf22e2cfd8300a245542b13a)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d259a0534ee2f89b103cfc52b988b452c6411662
---

 libavcodec/apedec.c |3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
index 03afd75..9984b40 100644
--- a/libavcodec/apedec.c
+++ b/libavcodec/apedec.c
@@ -892,6 +892,9 @@ static void long_filter_high_3800(int32_t *buffer, int 
order, int shift, int len
 int32_t dotprod, sign;
 int32_t coeffs[256], delay[256];
 
+if (order >= length)
+return;
+
 memset(coeffs, 0, order * sizeof(*coeffs));
 for (i = 0; i < order; i++)
 delay[i] = buffer[i];

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avformat/dump: Fix integer overflow in av_dump_format()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Tue Dec  1 12:40:32 2015 +0100| [cf1f615b67a4a486176b9aeaedeceef0337023e9] | 
committer: Michael Niedermayer

avformat/dump: Fix integer overflow in av_dump_format()

Fixes part of mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 8e7f4520226d2d9ad6a58ad6c32d1455a8b244b2)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=cf1f615b67a4a486176b9aeaedeceef0337023e9
---

 libavformat/dump.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/dump.c b/libavformat/dump.c
index 7bb593c..7a746b3 100644
--- a/libavformat/dump.c
+++ b/libavformat/dump.c
@@ -493,7 +493,7 @@ void av_dump_format(AVFormatContext *ic, int index,
 av_log(NULL, AV_LOG_INFO, "  Duration: ");
 if (ic->duration != AV_NOPTS_VALUE) {
 int hours, mins, secs, us;
-int64_t duration = ic->duration + 5000;
+int64_t duration = ic->duration + (ic->duration <= INT64_MAX - 
5000 ? 5000 : 0);
 secs  = duration / AV_TIME_BASE;
 us= duration % AV_TIME_BASE;
 mins  = secs / 60;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avutil/mathematics: return INT64_MIN (=AV_NOPTS_VALUE) from av_rescale_rnd () for overflows

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Tue Dec  1 13:32:31 2015 +0100| [86a52988bd50b16d7f4c52e610de00d5354c5174] | 
committer: Michael Niedermayer

avutil/mathematics: return INT64_MIN (=AV_NOPTS_VALUE) from av_rescale_rnd() 
for overflows

Fixes integer overflow
Fixes: mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer 
(cherry picked from commit f03c2ceec174877e03bb302f5971fbe9ffbe4856)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=86a52988bd50b16d7f4c52e610de00d5354c5174
---

 libavutil/mathematics.c |   13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavutil/mathematics.c b/libavutil/mathematics.c
index b1ffd65..4d8467b 100644
--- a/libavutil/mathematics.c
+++ b/libavutil/mathematics.c
@@ -77,7 +77,7 @@ int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, enum 
AVRounding rnd)
 }
 
 if (a < 0)
-return -av_rescale_rnd(-FFMAX(a, -INT64_MAX), b, c, rnd ^ ((rnd >> 1) 
& 1));
+return -(uint64_t)av_rescale_rnd(-FFMAX(a, -INT64_MAX), b, c, rnd ^ 
((rnd >> 1) & 1));
 
 if (rnd == AV_ROUND_NEAR_INF)
 r = c / 2;
@@ -87,8 +87,13 @@ int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, enum 
AVRounding rnd)
 if (b <= INT_MAX && c <= INT_MAX) {
 if (a <= INT_MAX)
 return (a * b + r) / c;
-else
-return a / c * b + (a % c * b + r) / c;
+else {
+int64_t ad = a / c;
+int64_t a2 = (a % c * b + r) / c;
+if (ad >= INT32_MAX && ad > (INT64_MAX - a2) / b)
+return INT64_MIN;
+return ad * b + a2;
+}
 } else {
 #if 1
 uint64_t a0  = a & 0x;
@@ -112,6 +117,8 @@ int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, 
enum AVRounding rnd)
 t1++;
 }
 }
+if (t1 > INT64_MAX)
+return INT64_MIN;
 return t1;
 }
 #else

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] mpegencts: Fix overflow in cbr mode period calculations

2015-12-14 Thread Timo Teräs
ffmpeg | branch: release/2.7 | Timo Teräs  | Sat Nov 28 
08:27:39 2015 +0200| [76cb34f7f5b8608670d1998b587ef9e12856ab99] | committer: 
Michael Niedermayer

mpegencts: Fix overflow in cbr mode period calculations

ts->mux_rate is int (signed 32-bit) type. The period calculations
will start to overflow when mux_rate > 5mbps. This fixes overflows
by converting first to 64-bit type.

Fixes #5044.

Signed-off-by: Timo Teräs 
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 64f7db554ee83846f207e82a08946a6a5a6acfe2)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=76cb34f7f5b8608670d1998b587ef9e12856ab99
---

 libavformat/mpegtsenc.c |6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavformat/mpegtsenc.c b/libavformat/mpegtsenc.c
index 9efa9fc..043a80f 100644
--- a/libavformat/mpegtsenc.c
+++ b/libavformat/mpegtsenc.c
@@ -751,11 +751,11 @@ static int mpegts_write_header(AVFormatContext *s)
 ts_st = pcr_st->priv_data;
 
 if (ts->mux_rate > 1) {
-service->pcr_packet_period = (ts->mux_rate * ts->pcr_period) /
+service->pcr_packet_period = (int64_t)ts->mux_rate * ts->pcr_period /
  (TS_PACKET_SIZE * 8 * 1000);
-ts->sdt_packet_period  = (ts->mux_rate * SDT_RETRANS_TIME) /
+ts->sdt_packet_period  = (int64_t)ts->mux_rate * SDT_RETRANS_TIME /
  (TS_PACKET_SIZE * 8 * 1000);
-ts->pat_packet_period  = (ts->mux_rate * PAT_RETRANS_TIME) /
+ts->pat_packet_period  = (int64_t)ts->mux_rate * PAT_RETRANS_TIME /
  (TS_PACKET_SIZE * 8 * 1000);
 
 if (ts->copyts < 1)

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/mpeg4videodec: Check available data before reading custom matrix

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sun Nov 29 23:44:40 2015 +0100| [fc69fa8474ae2a3b3420074c17c908e0204c82af] | 
committer: Michael Niedermayer

avcodec/mpeg4videodec: Check available data before reading custom matrix

Fixes: out of array read
Fixes: 
76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 891dc8f87536ac2ec695c70d081345224524ad99)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=fc69fa8474ae2a3b3420074c17c908e0204c82af
---

 libavcodec/mpeg4videodec.c |8 
 1 file changed, 8 insertions(+)

diff --git a/libavcodec/mpeg4videodec.c b/libavcodec/mpeg4videodec.c
index e151f9e..dd2414b 100644
--- a/libavcodec/mpeg4videodec.c
+++ b/libavcodec/mpeg4videodec.c
@@ -1875,6 +1875,10 @@ static int decode_vol_header(Mpeg4DecContext *ctx, 
GetBitContext *gb)
 int last = 0;
 for (i = 0; i < 64; i++) {
 int j;
+if (get_bits_left(gb) < 8) {
+av_log(s->avctx, AV_LOG_ERROR, "insufficient data for 
custom matrix\n");
+return AVERROR_INVALIDDATA;
+}
 v = get_bits(gb, 8);
 if (v == 0)
 break;
@@ -1898,6 +1902,10 @@ static int decode_vol_header(Mpeg4DecContext *ctx, 
GetBitContext *gb)
 int last = 0;
 for (i = 0; i < 64; i++) {
 int j;
+if (get_bits_left(gb) < 8) {
+av_log(s->avctx, AV_LOG_ERROR, "insufficient data for 
custom matrix\n");
+return AVERROR_INVALIDDATA;
+}
 v = get_bits(gb, 8);
 if (v == 0)
 break;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avformat/smacker: fix integer overflow with pts_inc

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 13:06:16 2015 +0100| [4e80d4bf25589f3beecc5e0633b148cc113fe708] | 
committer: Michael Niedermayer

avformat/smacker: fix integer overflow with pts_inc

Fixes: 
ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 7ed47e97297fd5ef473d0cc93f0455adbadaac83)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=4e80d4bf25589f3beecc5e0633b148cc113fe708
---

 libavformat/smacker.c |5 +
 1 file changed, 5 insertions(+)

diff --git a/libavformat/smacker.c b/libavformat/smacker.c
index 5dcf4ad..de8bbdb 100644
--- a/libavformat/smacker.c
+++ b/libavformat/smacker.c
@@ -120,6 +120,11 @@ static int smacker_read_header(AVFormatContext *s)
 smk->height = avio_rl32(pb);
 smk->frames = avio_rl32(pb);
 smk->pts_inc = (int32_t)avio_rl32(pb);
+if (smk->pts_inc > INT_MAX / 100) {
+av_log(s, AV_LOG_ERROR, "pts_inc %d is too large\n", smk->pts_inc);
+return AVERROR_INVALIDDATA;
+}
+
 smk->flags = avio_rl32(pb);
 if(smk->flags & SMACKER_FLAG_RING_FRAME)
 smk->frames++;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/utils: Clear dimensions in ff_get_buffer() on failure

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Nov 28 20:08:46 2015 +0100| [fad0748b9221fbc7ca806e2ca9aac84927cd7234] | 
committer: Michael Niedermayer

avcodec/utils: Clear dimensions in ff_get_buffer() on failure

Fixes out of array access
Fixes: 
482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit abee0a1c60612e8638640a8a3738fffb65e16dbf)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=fad0748b9221fbc7ca806e2ca9aac84927cd7234
---

 libavcodec/utils.c |4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index f960295..0e8e466 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -1037,8 +1037,10 @@ end:
 int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags)
 {
 int ret = get_buffer_internal(avctx, frame, flags);
-if (ret < 0)
+if (ret < 0) {
 av_log(avctx, AV_LOG_ERROR, "get_buffer() failed\n");
+frame->width = frame->height = 0;
+}
 return ret;
 }
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/vp3: always set pix_fmt in theora_decode_header()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Mon Nov 30 03:32:36 2015 +0100| [710dccf0360baa7375130b31c7ccae17a5aaa601] | 
committer: Michael Niedermayer

avcodec/vp3: always set pix_fmt in theora_decode_header()

Fixes assertion failure
Fixes: 
d0bb0662da342ec65f8f2a081222e6b9/signal_sigabrt_76ae7cc9_5471_82964f0a9ac2f4d3d59390c15473f6f7.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit a814f1d364ba912adf61adef158168c5f7604e93)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=710dccf0360baa7375130b31c7ccae17a5aaa601
---

 libavcodec/vp3.c |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c
index 26c2ea7..33c3e39 100644
--- a/libavcodec/vp3.c
+++ b/libavcodec/vp3.c
@@ -2321,7 +2321,8 @@ static int theora_decode_header(AVCodecContext *avctx, 
GetBitContext *gb)
 return AVERROR_INVALIDDATA;
 }
 skip_bits(gb, 3); /* reserved */
-}
+} else
+avctx->pix_fmt = AV_PIX_FMT_YUV420P;
 
 ret = ff_set_dimensions(avctx, s->width, s->height);
 if (ret < 0)

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/vp3: Clear context on reinitialization failure

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Nov 28 00:23:54 2015 +0100| [c1d29678f1ca993e201b32b8e4af3f9846c5db32] | 
committer: Michael Niedermayer

avcodec/vp3: Clear context on reinitialization failure

Fixes null pointer dereference
Fixes: 
1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 6105b7219a90438deae71b0dc5a034c71ee30fc0)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=c1d29678f1ca993e201b32b8e4af3f9846c5db32
---

 libavcodec/vp3.c |   10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c
index 2c8a8fe..26c2ea7 100644
--- a/libavcodec/vp3.c
+++ b/libavcodec/vp3.c
@@ -2014,17 +2014,19 @@ static int vp3_decode_frame(AVCodecContext *avctx,
 vp3_decode_end(avctx);
 ret = theora_decode_header(avctx, &gb);
 
+if (ret >= 0)
+ret = vp3_decode_init(avctx);
 if (ret < 0) {
 vp3_decode_end(avctx);
-} else
-ret = vp3_decode_init(avctx);
+}
 return ret;
 } else if (type == 2) {
 ret = theora_decode_tables(avctx, &gb);
+if (ret >= 0)
+ret = vp3_decode_init(avctx);
 if (ret < 0) {
 vp3_decode_end(avctx);
-} else
-ret = vp3_decode_init(avctx);
+}
 return ret;
 }
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/hevc: allocate entries unconditionally

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 23:33:03 2015 +0100| [4fe6f9f6271c3e33ec9fddc3dbac17c0436c0f9a] | 
committer: Michael Niedermayer

avcodec/hevc: allocate entries unconditionally

Fixes out of array access
Fixes: 
08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit d85aa76115214183e7e3b7d65e950da61474959a)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=4fe6f9f6271c3e33ec9fddc3dbac17c0436c0f9a
---

 libavcodec/hevc.c |4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/libavcodec/hevc.c b/libavcodec/hevc.c
index 8e7e736..b52a6d1 100644
--- a/libavcodec/hevc.c
+++ b/libavcodec/hevc.c
@@ -2436,11 +2436,9 @@ static int hls_slice_data_wpp(HEVCContext *s, const 
uint8_t *nal, int length)
 return AVERROR(ENOMEM);
 }
 
+ff_alloc_entries(s->avctx, s->sh.num_entry_point_offsets + 1);
 
 if (!s->sList[1]) {
-ff_alloc_entries(s->avctx, s->sh.num_entry_point_offsets + 1);
-
-
 for (i = 1; i < s->threads_number; i++) {
 s->sList[i] = av_malloc(sizeof(HEVCContext));
 memcpy(s->sList[i], s, sizeof(HEVCContext));

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] mjpegdec: consider chroma subsampling in size check

2015-12-14 Thread Andreas Cadhalpun
ffmpeg | branch: release/2.7 | Andreas Cadhalpun 
 | Wed Dec  2 21:52:23 2015 +0100| 
[9a8d2f51cf0548aa3724e2a46e58416b333c755f] | committer: Michael Niedermayer

mjpegdec: consider chroma subsampling in size check

If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer 
Signed-off-by: Andreas Cadhalpun 
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=9a8d2f51cf0548aa3724e2a46e58416b333c755f
---

 libavcodec/mjpegdec.c |   11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libavcodec/mjpegdec.c b/libavcodec/mjpegdec.c
index 30fc99a..48f5ca4 100644
--- a/libavcodec/mjpegdec.c
+++ b/libavcodec/mjpegdec.c
@@ -1241,7 +1241,7 @@ static int mjpeg_decode_scan(MJpegDecodeContext *s, int 
nb_components, int Ah,
  int mb_bitmask_size,
  const AVFrame *reference)
 {
-int i, mb_x, mb_y;
+int i, mb_x, mb_y, chroma_h_shift, chroma_v_shift, chroma_width, 
chroma_height;
 uint8_t *data[MAX_COMPONENTS];
 const uint8_t *reference_data[MAX_COMPONENTS];
 int linesize[MAX_COMPONENTS];
@@ -1258,6 +1258,11 @@ static int mjpeg_decode_scan(MJpegDecodeContext *s, int 
nb_components, int Ah,
 
 s->restart_count = 0;
 
+av_pix_fmt_get_chroma_sub_sample(s->avctx->pix_fmt, &chroma_h_shift,
+ &chroma_v_shift);
+chroma_width  = FF_CEIL_RSHIFT(s->width,  chroma_h_shift);
+chroma_height = FF_CEIL_RSHIFT(s->height, chroma_v_shift);
+
 for (i = 0; i < nb_components; i++) {
 int c   = s->comp_index[i];
 data[c] = s->picture_ptr->data[c];
@@ -1294,8 +1299,8 @@ static int mjpeg_decode_scan(MJpegDecodeContext *s, int 
nb_components, int Ah,
 
 if (s->interlaced && s->bottom_field)
 block_offset += linesize[c] >> 1;
-if (   8*(h * mb_x + x) < s->width
-&& 8*(v * mb_y + y) < s->height) {
+if (   8*(h * mb_x + x) < ((c == 1) || (c == 2) ? 
chroma_width  : s->width)
+&& 8*(v * mb_y + y) < ((c == 1) || (c == 2) ? 
chroma_height : s->height)) {
 ptr = data[c] + block_offset;
 } else
 ptr = NULL;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/h264_refs: Check that long references match before use

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sun Nov 29 03:25:41 2015 +0100| [900039e7dc321ca33db696f7ae11b4058d6494ba] | 
committer: Michael Niedermayer

avcodec/h264_refs: Check that long references match before use

Fixes out of array read
Fixes: 
59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit aa427537b529cd584cd73222980286d36a00fe28)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=900039e7dc321ca33db696f7ae11b4058d6494ba
---

 libavcodec/h264_refs.c |   15 ++-
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/libavcodec/h264_refs.c b/libavcodec/h264_refs.c
index 548a3ba..2d787a5 100644
--- a/libavcodec/h264_refs.c
+++ b/libavcodec/h264_refs.c
@@ -122,6 +122,14 @@ static int add_sorted(H264Picture **sorted, H264Picture 
**src, int len, int limi
 return out_i;
 }
 
+static int mismatches_ref(H264Context *h, H264Picture *pic)
+{
+AVFrame *f = pic->f;
+return (h->cur_pic_ptr->f->width  != f->width ||
+h->cur_pic_ptr->f->height != f->height ||
+h->cur_pic_ptr->f->format != f->format);
+}
+
 int ff_h264_fill_default_ref_list(H264Context *h, H264SliceContext *sl)
 {
 int i, len;
@@ -193,10 +201,7 @@ int ff_h264_fill_default_ref_list(H264Context *h, 
H264SliceContext *sl)
 for (j = 0; j<1+(sl->slice_type_nos == AV_PICTURE_TYPE_B); j++) {
 for (i = 0; i < sl->ref_count[j]; i++) {
 if (h->default_ref_list[j][i].parent) {
-AVFrame *f = h->default_ref_list[j][i].parent->f;
-if (h->cur_pic_ptr->f->width  != f->width ||
-h->cur_pic_ptr->f->height != f->height ||
-h->cur_pic_ptr->f->format != f->format) {
+if (mismatches_ref(h, h->default_ref_list[j][i].parent)) {
 av_log(h->avctx, AV_LOG_ERROR, "Discarding mismatching 
reference\n");
 memset(&h->default_ref_list[j][i], 0, 
sizeof(h->default_ref_list[j][i]));
 }
@@ -305,7 +310,7 @@ int ff_h264_decode_ref_pic_list_reordering(H264Context *h, 
H264SliceContext *sl)
 }
 ref = h->long_ref[long_idx];
 assert(!(ref && !ref->reference));
-if (ref && (ref->reference & pic_structure)) {
+if (ref && (ref->reference & pic_structure) && 
!mismatches_ref(h, ref)) {
 ref->pic_id = pic_id;
 assert(ref->long_ref);
 i = 0;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 21:02:13 2015 +0100| [c1db1a5ff47fd3b99cb074c761d76fd0bf05dd82] | 
committer: Michael Niedermayer

avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()

Signed-off-by: Michael Niedermayer 
(cherry picked from commit feb3f39614b88c113211a98dda1bc2fe5c3c6957)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=c1db1a5ff47fd3b99cb074c761d76fd0bf05dd82
---

 libavcodec/jpeg2000dwt.c |3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/jpeg2000dwt.c b/libavcodec/jpeg2000dwt.c
index 015a4fe..925adea 100644
--- a/libavcodec/jpeg2000dwt.c
+++ b/libavcodec/jpeg2000dwt.c
@@ -540,6 +540,9 @@ int ff_jpeg2000_dwt_init(DWTContext *s, uint16_t 
border[2][2],
 
 int ff_dwt_encode(DWTContext *s, void *t)
 {
+if (s->ndeclevels == 0)
+return 0;
+
 switch(s->type){
 case FF_DWT97:
 dwt_encode97_float(s, t); break;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avutil/timecode: Fix fps check

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Thu Dec  3 03:14:11 2015 +0100| [da87a699ea9c1ba0b330413528dbca29bcc54ef8] | 
committer: Michael Niedermayer

avutil/timecode: Fix fps check

The fps variable is explicitly set to -1 in case of some errors, the check must
thus be signed or the code setting it needs to use 0 as error code
the type of the field could be changed as well but its in an installed header

Fixes: integer overflow
Fixes: 
9982cc157b1ea90429435640a989122f/asan_generic_3ad004a_3799_22cf198d9cd09928e2d9ad250474fa58.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit b46dcd5209a77254345ae098b83a872634c5591b)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=da87a699ea9c1ba0b330413528dbca29bcc54ef8
---

 libavutil/timecode.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavutil/timecode.c b/libavutil/timecode.c
index 1dfd040..bf463ed 100644
--- a/libavutil/timecode.c
+++ b/libavutil/timecode.c
@@ -151,7 +151,7 @@ static int check_fps(int fps)
 
 static int check_timecode(void *log_ctx, AVTimecode *tc)
 {
-if (tc->fps <= 0) {
+if ((int)tc->fps <= 0) {
 av_log(log_ctx, AV_LOG_ERROR, "Timecode frame rate must be 
specified\n");
 return AVERROR(EINVAL);
 }

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 20:52:39 2015 +0100| [8ef86669ca25b71a61e4b69d428b69b5d71ec7b7] | 
committer: Michael Niedermayer

avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()

Fixes out of array access
Fixes: 
01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 75422280fbcdfbe9dc56bde5525b4d8b280f1bc5)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8ef86669ca25b71a61e4b69d428b69b5d71ec7b7
---

 libavcodec/jpeg2000dwt.c |3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/jpeg2000dwt.c b/libavcodec/jpeg2000dwt.c
index ceceda3..015a4fe 100644
--- a/libavcodec/jpeg2000dwt.c
+++ b/libavcodec/jpeg2000dwt.c
@@ -555,6 +555,9 @@ int ff_dwt_encode(DWTContext *s, void *t)
 
 int ff_dwt_decode(DWTContext *s, void *t)
 {
+if (s->ndeclevels == 0)
+return 0;
+
 switch (s->type) {
 case FF_DWT97:
 dwt_decode97_float(s, t);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/dirac_parser: Fix potential overflows in pointer checks

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 17:11:54 2015 +0100| [d092b7f04ca8dfb977343aa12289b6b6123260ea] | 
committer: Michael Niedermayer

avcodec/dirac_parser: Fix potential overflows in pointer checks

Signed-off-by: Michael Niedermayer 
(cherry picked from commit 79798f7c57b098c78e0bbc6becd64b9888b013d1)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d092b7f04ca8dfb977343aa12289b6b6123260ea
---

 libavcodec/dirac_parser.c |8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libavcodec/dirac_parser.c b/libavcodec/dirac_parser.c
index 83c35a2..12f1a60 100644
--- a/libavcodec/dirac_parser.c
+++ b/libavcodec/dirac_parser.c
@@ -100,10 +100,12 @@ typedef struct DiracParseUnit {
 static int unpack_parse_unit(DiracParseUnit *pu, DiracParseContext *pc,
  int offset)
 {
-uint8_t *start = pc->buffer + offset;
-uint8_t *end   = pc->buffer + pc->index;
-if (start < pc->buffer || (start + 13 > end))
+int8_t *start;
+
+if (offset < 0 || pc->index - 13 < offset)
 return 0;
+
+start = pc->buffer + offset;
 pu->pu_type = start[4];
 
 pu->next_pu_offset = AV_RB32(start + 5);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Tue Nov 24 22:12:37 2015 +0100| [54e94522b899111dd4b9f93386d582d26859ead5] | 
committer: Michael Niedermayer

avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized

Fixes out of array access
Fixes: 
1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 4ea4d2f438c9a7eba37980c9a87be4b34943e4d5)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=54e94522b899111dd4b9f93386d582d26859ead5
---

 libavcodec/h264_slice.c |1 +
 1 file changed, 1 insertion(+)

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index 10f4d77..9cbe8d2 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -1086,6 +1086,7 @@ static int h264_slice_header_init(H264Context *h)
 nb_slices = max_slices;
 }
 h->slice_context_count = nb_slices;
+h->max_contexts = FFMIN(h->max_contexts, nb_slices);
 
 if (!HAVE_THREADS || !(h->avctx->active_thread_type & FF_THREAD_SLICE)) {
 ret = ff_h264_slice_context_init(h, &h->slice_ctx[0]);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/vp8: Do not use num_coeff_partitions in thread/buffer setup

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Wed Sep 30 13:10:48 2015 +0200| [8405b6329463af8ac2e5dac4fcdda03888388f78] | 
committer: Michael Niedermayer

avcodec/vp8: Do not use num_coeff_partitions in thread/buffer setup

The variable is not a constant and can lead to race conditions

Fixes: repro.webm (not reproducable with FFmpeg alone)

Found-by: Dale Curtis 
Tested-by: Dale Curtis 
Signed-off-by: Michael Niedermayer 
(cherry picked from commit dabea74d0e82ea80cd344f630497cafcb3ef872c)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8405b6329463af8ac2e5dac4fcdda03888388f78
---

 libavcodec/vp8.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vp8.c b/libavcodec/vp8.c
index 25fe70a..29379f4 100644
--- a/libavcodec/vp8.c
+++ b/libavcodec/vp8.c
@@ -164,7 +164,7 @@ int update_dimensions(VP8Context *s, int width, int height, 
int is_vp7)
 s->mb_height = (s->avctx->coded_height + 15) / 16;
 
 s->mb_layout = is_vp7 || avctx->active_thread_type == FF_THREAD_SLICE &&
-   FFMIN(s->num_coeff_partitions, avctx->thread_count) > 1;
+   avctx->thread_count > 1;
 if (!s->mb_layout) { // Frame threading and one thread
 s->macroblocks_base   = av_mallocz((s->mb_width + s->mb_height * 2 
+ 1) *
sizeof(*s->macroblocks));

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/vp3: Fix "runtime error: left shift of negative value"

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Dec  4 12:47:20 2015 +0100| [e3ffc7ab4a07cfaa25b6a36e78e91cf47a47c942] | 
committer: Michael Niedermayer

avcodec/vp3: Fix "runtime error: left shift of negative value"

Fixes: 
5c6129154b356b80bcab86f9e3ee5d29/signal_sigabrt_76ae7cc9_7322_d26ac6d7cb6567db1b8be0159b387d0b.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 18268f761bffb37552f59f87542fef3d5c80618c)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e3ffc7ab4a07cfaa25b6a36e78e91cf47a47c942
---

 libavcodec/vp3.c |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c
index 33c3e39..fbb7da7 100644
--- a/libavcodec/vp3.c
+++ b/libavcodec/vp3.c
@@ -209,8 +209,8 @@ typedef struct Vp3DecodeContext {
 int16_t *dct_tokens[3][64];
 int16_t *dct_tokens_base;
 #define TOKEN_EOB(eob_run)  ((eob_run) << 2)
-#define TOKEN_ZERO_RUN(coeff, zero_run) (((coeff) << 9) + ((zero_run) << 2) + 
1)
-#define TOKEN_COEFF(coeff)  (((coeff) << 2) + 2)
+#define TOKEN_ZERO_RUN(coeff, zero_run) (((coeff) * 512) + ((zero_run) << 2) + 
1)
+#define TOKEN_COEFF(coeff)  (((coeff) * 4) + 2)
 
 /**
  * number of blocks that contain DCT coefficients at

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Nov 28 17:26:05 2015 +0100| [6c25411c06a9867eb993ad2c0ee477b4573516c1] | 
committer: Michael Niedermayer

avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()

Fixes integer overflow
Fixes: 
3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 4f03bebc79f76df3a3e5bb9e1bc32baabfb7797c)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=6c25411c06a9867eb993ad2c0ee477b4573516c1
---

 libavcodec/utils.c |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/utils.c b/libavcodec/utils.c
index cc689d8..f960295 100644
--- a/libavcodec/utils.c
+++ b/libavcodec/utils.c
@@ -3115,8 +3115,8 @@ void avcodec_string(char *buf, int buf_size, 
AVCodecContext *enc, int encode)
 
 if (enc->sample_aspect_ratio.num) {
 av_reduce(&display_aspect_ratio.num, &display_aspect_ratio.den,
-  enc->width * enc->sample_aspect_ratio.num,
-  enc->height * enc->sample_aspect_ratio.den,
+  enc->width * (int64_t)enc->sample_aspect_ratio.num,
+  enc->height * (int64_t)enc->sample_aspect_ratio.den,
   1024 * 1024);
 snprintf(buf + strlen(buf), buf_size - strlen(buf),
  " [SAR %d:%d DAR %d:%d]",

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/hevc: Check entry_point_offsets

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 18:30:05 2015 +0100| [0cd23e0d1e89c0f7faecea997ade67b56abbc83d] | 
committer: Michael Niedermayer

avcodec/hevc: Check entry_point_offsets

Fixes out of array read
Fixes: 
007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit ef9f7bbfa47317f9d46bf46982a394d2be78503c)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=0cd23e0d1e89c0f7faecea997ade67b56abbc83d
---

 libavcodec/hevc.c |8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavcodec/hevc.c b/libavcodec/hevc.c
index 05e7f12..8e7e736 100644
--- a/libavcodec/hevc.c
+++ b/libavcodec/hevc.c
@@ -2426,7 +2426,7 @@ static int hls_slice_data_wpp(HEVCContext *s, const 
uint8_t *nal, int length)
 HEVCLocalContext *lc = s->HEVClc;
 int *ret = av_malloc_array(s->sh.num_entry_point_offsets + 1, sizeof(int));
 int *arg = av_malloc_array(s->sh.num_entry_point_offsets + 1, sizeof(int));
-int offset;
+int64_t offset;
 int startheader, cmpt = 0;
 int i, j, res = 0;
 
@@ -2473,6 +2473,11 @@ static int hls_slice_data_wpp(HEVCContext *s, const 
uint8_t *nal, int length)
 }
 if (s->sh.num_entry_point_offsets != 0) {
 offset += s->sh.entry_point_offset[s->sh.num_entry_point_offsets - 1] 
- cmpt;
+if (length < offset) {
+av_log(s->avctx, AV_LOG_ERROR, "entry_point_offset table is 
corrupted\n");
+res = AVERROR_INVALIDDATA;
+goto error;
+}
 s->sh.size[s->sh.num_entry_point_offsets - 1] = length - offset;
 s->sh.offset[s->sh.num_entry_point_offsets - 1] = offset;
 
@@ -2499,6 +2504,7 @@ static int hls_slice_data_wpp(HEVCContext *s, const 
uint8_t *nal, int length)
 
 for (i = 0; i <= s->sh.num_entry_point_offsets; i++)
 res += ret[i];
+error:
 av_free(ret);
 av_free(arg);
 return res;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/jpeg2000dec: Check bpno in decode_cblk()

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Dec  4 16:23:24 2015 +0100| [0f331f94c0db42647bb950c8ae529e43215c32df] | 
committer: Michael Niedermayer

avcodec/jpeg2000dec: Check bpno in decode_cblk()

Fixes: undefined shift
Fixes: 
c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit a85b02dcf70f62a6a433a607143f1f78fa5648bb)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=0f331f94c0db42647bb950c8ae529e43215c32df
---

 libavcodec/jpeg2000dec.c |4 
 1 file changed, 4 insertions(+)

diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
index caaf2f7..54006af 100644
--- a/libavcodec/jpeg2000dec.c
+++ b/libavcodec/jpeg2000dec.c
@@ -1106,6 +1106,10 @@ static int decode_cblk(Jpeg2000DecoderContext *s, 
Jpeg2000CodingStyle *codsty,
 ff_mqc_initdec(&t1->mqc, cblk->data);
 
 while (passno--) {
+if (bpno < 0) {
+av_log(s->avctx, AV_LOG_ERROR, "bpno became negative\n");
+return AVERROR_INVALIDDATA;
+}
 switch(pass_t) {
 case 0:
 decode_sigpass(t1, width, height, bpno + 1, bandpos,

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/hevc: Fix integer overflow of entry_point_offset

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 22:08:59 2015 +0100| [5af5396970b7cbaf22bd8754257f9bf9ffb36297] | 
committer: Michael Niedermayer

avcodec/hevc: Fix integer overflow of entry_point_offset

Fixes out of array read
Fixes: 
d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=5af5396970b7cbaf22bd8754257f9bf9ffb36297
---

 libavcodec/hevc.c |4 ++--
 libavcodec/hevc.h |2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/hevc.c b/libavcodec/hevc.c
index b52a6d1..c520878 100644
--- a/libavcodec/hevc.c
+++ b/libavcodec/hevc.c
@@ -730,7 +730,7 @@ static int hls_slice_header(HEVCContext *s)
 av_freep(&sh->entry_point_offset);
 av_freep(&sh->offset);
 av_freep(&sh->size);
-sh->entry_point_offset = 
av_malloc_array(sh->num_entry_point_offsets, sizeof(int));
+sh->entry_point_offset = 
av_malloc_array(sh->num_entry_point_offsets, sizeof(unsigned));
 sh->offset = av_malloc_array(sh->num_entry_point_offsets, 
sizeof(int));
 sh->size = av_malloc_array(sh->num_entry_point_offsets, 
sizeof(int));
 if (!sh->entry_point_offset || !sh->offset || !sh->size) {
@@ -2427,7 +2427,7 @@ static int hls_slice_data_wpp(HEVCContext *s, const 
uint8_t *nal, int length)
 int *ret = av_malloc_array(s->sh.num_entry_point_offsets + 1, sizeof(int));
 int *arg = av_malloc_array(s->sh.num_entry_point_offsets + 1, sizeof(int));
 int64_t offset;
-int startheader, cmpt = 0;
+int64_t startheader, cmpt = 0;
 int i, j, res = 0;
 
 if (!ret || !arg) {
diff --git a/libavcodec/hevc.h b/libavcodec/hevc.h
index 2b47de2..96fc258 100644
--- a/libavcodec/hevc.h
+++ b/libavcodec/hevc.h
@@ -609,7 +609,7 @@ typedef struct SliceHeader {
 
 unsigned int max_num_merge_cand; ///< 5 - 5_minus_max_num_merge_cand
 
-int *entry_point_offset;
+unsigned *entry_point_offset;
 int * offset;
 int * size;
 int num_entry_point_offsets;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] swscale/utils: Fix for runtime error: left shift of negative value -1

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Dec  4 21:44:05 2015 +0100| [dc0bc71471bff6dedd371d97c0b1a45d0ea59042] | 
committer: Michael Niedermayer

swscale/utils: Fix for runtime error: left shift of negative value -1

Fixes: 
c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 325b59368dae3c3f2f5cc39873002b4cf133ccbc)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=dc0bc71471bff6dedd371d97c0b1a45d0ea59042
---

 libswscale/utils.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libswscale/utils.c b/libswscale/utils.c
index 63fb05c..93cbad7 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -384,7 +384,7 @@ static av_cold int initFilter(int16_t **outFilter, int32_t 
**filterPos,
 
 xDstInSrc = ((dstPos*(int64_t)xInc)>>7) - ((srcPos*0x1LL)>>7);
 for (i = 0; i < dstW; i++) {
-int xx = (xDstInSrc - ((int64_t)(filterSize - 2) << 16)) / (1 << 
17);
+int xx = (xDstInSrc - (filterSize - 2) * (1LL<<16)) / (1 << 17);
 int j;
 (*filterPos)[i] = xx;
 for (j = 0; j < filterSize; j++) {

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H

2015-12-14 Thread zjh8890
ffmpeg | branch: release/2.7 | zjh8890 <243186...@qq.com> | Sun Nov 22 00:07:35 
2015 +0800| [a9ce4583dfbb65dc2d0d7483f11c6e71b413bac8] | committer: Michael 
Niedermayer

avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H

The transpose_4x4H is wrong which cost me much time to find this bug. The 
orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the 
function.
(cherry picked from commit c18176bd551b4616757080376707637e30547fd0)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=a9ce4583dfbb65dc2d0d7483f11c6e71b413bac8
---

 libavcodec/aarch64/neon.S |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/aarch64/neon.S b/libavcodec/aarch64/neon.S
index 619aec6..a227cbd 100644
--- a/libavcodec/aarch64/neon.S
+++ b/libavcodec/aarch64/neon.S
@@ -107,8 +107,8 @@
 .macro  transpose_4x4H  r0, r1, r2, r3, r4, r5, r6, r7
 trn1\r4\().4H,  \r0\().4H,  \r1\().4H
 trn2\r5\().4H,  \r0\().4H,  \r1\().4H
-trn1\r7\().4H,  \r3\().4H,  \r2\().4H
-trn2\r6\().4H,  \r3\().4H,  \r2\().4H
+trn1\r7\().4H,  \r2\().4H,  \r3\().4H
+trn2\r6\().4H,  \r2\().4H,  \r3\().4H
 trn1\r0\().2S,  \r4\().2S,  \r7\().2S
 trn2\r3\().2S,  \r4\().2S,  \r7\().2S
 trn1\r1\().2S,  \r5\().2S,  \r6\().2S

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/cabac_functions: Fix "left shift of negative value -31767"

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 12:11:29 2015 +0100| [1258bdf7f0985ec489e2767a1a633bb47ceecc92] | 
committer: Michael Niedermayer

avcodec/cabac_functions: Fix "left shift of negative value -31767"

Fixes: 
1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer 
(cherry picked from commit a1f6b05f5228979dab0e149deca7a30d22e98af5)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=1258bdf7f0985ec489e2767a1a633bb47ceecc92
---

 libavcodec/cabac_functions.h |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavcodec/cabac_functions.h b/libavcodec/cabac_functions.h
index 15dba29..4e13253 100644
--- a/libavcodec/cabac_functions.h
+++ b/libavcodec/cabac_functions.h
@@ -74,7 +74,8 @@ static inline void renorm_cabac_decoder_once(CABACContext *c){
 
 #ifndef get_cabac_inline
 static void refill2(CABACContext *c){
-int i, x;
+int i;
+unsigned x;
 
 x= c->low ^ (c->low-1);
 i= 7 - ff_h264_norm_shift[x>>(CABAC_BITS-1)];

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Dec  4 21:38:12 2015 +0100| [0a06e2824a675fb778cd98ccab35a8936db569b9] | 
committer: Michael Niedermayer

avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in 
type int

Fixes: 
b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 4f2419888ba49245761f4ab343679c38e7880cfe)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=0a06e2824a675fb778cd98ccab35a8936db569b9
---

 libavcodec/pgssubdec.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/pgssubdec.c b/libavcodec/pgssubdec.c
index 0d307f5..e567f53 100644
--- a/libavcodec/pgssubdec.c
+++ b/libavcodec/pgssubdec.c
@@ -33,7 +33,7 @@
 #include "libavutil/imgutils.h"
 #include "libavutil/opt.h"
 
-#define RGBA(r,g,b,a) (((a) << 24) | ((r) << 16) | ((g) << 8) | (b))
+#define RGBA(r,g,b,a) (((unsigned)(a) << 24) | ((r) << 16) | ((g) << 8) | (b))
 #define MAX_EPOCH_PALETTES 8   // Max 8 allowed per PGS epoch
 #define MAX_EPOCH_OBJECTS  64  // Max 64 allowed per PGS epoch
 #define MAX_OBJECT_REFS2   // Max objects per display set

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/hevc: Check max ctb addresses for WPP

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Nov 28 13:42:05 2015 +0100| [d013f51303de27126f92dfa4357c14243019037a] | 
committer: Michael Niedermayer

avcodec/hevc: Check max ctb addresses for WPP

Fixes out of array read
Fixes: 
2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d013f51303de27126f92dfa4357c14243019037a
---

 libavcodec/hevc.c |9 +
 1 file changed, 9 insertions(+)

diff --git a/libavcodec/hevc.c b/libavcodec/hevc.c
index c520878..7d1565e 100644
--- a/libavcodec/hevc.c
+++ b/libavcodec/hevc.c
@@ -2436,6 +2436,15 @@ static int hls_slice_data_wpp(HEVCContext *s, const 
uint8_t *nal, int length)
 return AVERROR(ENOMEM);
 }
 
+if (s->sh.slice_ctb_addr_rs + s->sh.num_entry_point_offsets * 
s->sps->ctb_width >= s->sps->ctb_width * s->sps->ctb_height) {
+av_log(s->avctx, AV_LOG_ERROR, "WPP ctb addresses are wrong (%d %d %d 
%d)\n",
+s->sh.slice_ctb_addr_rs, s->sh.num_entry_point_offsets,
+s->sps->ctb_width, s->sps->ctb_height
+);
+res = AVERROR_INVALIDDATA;
+goto error;
+}
+
 ff_alloc_entries(s->avctx, s->sh.num_entry_point_offsets + 1);
 
 if (!s->sList[1]) {

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avutil/integer: Fix av_mod_i() with negative dividend

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Tue Dec  1 12:41:43 2015 +0100| [07a30246315b568fc2155e17a4e78529082657ed] | 
committer: Michael Niedermayer

avutil/integer: Fix av_mod_i() with negative dividend

Signed-off-by: Michael Niedermayer 
(cherry picked from commit 3a9cb18855d29c96a5d9d2f5ad30448cae3a2ddf)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=07a30246315b568fc2155e17a4e78529082657ed
---

 libavutil/integer.c |8 
 1 file changed, 8 insertions(+)

diff --git a/libavutil/integer.c b/libavutil/integer.c
index 5bcde0d..6d6855f 100644
--- a/libavutil/integer.c
+++ b/libavutil/integer.c
@@ -29,6 +29,8 @@
 #include "integer.h"
 #include "avassert.h"
 
+static const AVInteger zero_i;
+
 AVInteger av_add_i(AVInteger a, AVInteger b){
 int i, carry=0;
 
@@ -111,6 +113,12 @@ AVInteger av_mod_i(AVInteger *quot, AVInteger a, AVInteger 
b){
 AVInteger quot_temp;
 if(!quot) quot = "_temp;
 
+if ((int16_t)a.v[AV_INTEGER_SIZE-1] < 0) {
+a = av_mod_i(quot, av_sub_i(zero_i, a), b);
+*quot = av_sub_i(zero_i, *quot);
+return av_sub_i(zero_i, a);
+}
+
 av_assert2((int16_t)a.v[AV_INTEGER_SIZE-1] >= 0 && 
(int16_t)b.v[AV_INTEGER_SIZE-1] >= 0);
 av_assert2(av_log2_i(b)>=0);
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/wmaprodec: Check bits per sample to be within the range not causing integer overflows

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 13:48:06 2015 +0100| [1601420be4b505d4b9f8f2d92d37e19a44cc42bd] | 
committer: Michael Niedermayer

avcodec/wmaprodec: Check bits per sample to be within the range not causing 
integer overflows

Fixes: 
549d5aab1480d10f2a775ed90b0342f1/signal_sigabrt_76ae7cc9_5643_96bbb0cfe3e28be1dadfce1075016345.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 66e05f6ff5e5c105bdd7bf3a49234ddac1b592c5)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=1601420be4b505d4b9f8f2d92d37e19a44cc42bd
---

 libavcodec/wmaprodec.c |6 ++
 1 file changed, 6 insertions(+)

diff --git a/libavcodec/wmaprodec.c b/libavcodec/wmaprodec.c
index 5b59da5..8653bda 100644
--- a/libavcodec/wmaprodec.c
+++ b/libavcodec/wmaprodec.c
@@ -300,6 +300,12 @@ static av_cold int decode_init(AVCodecContext *avctx)
 s->decode_flags= AV_RL16(edata_ptr+14);
 channel_mask   = AV_RL32(edata_ptr+2);
 s->bits_per_sample = AV_RL16(edata_ptr);
+
+if (s->bits_per_sample > 32 || s->bits_per_sample < 1) {
+avpriv_request_sample(avctx, "bits per sample is %d", 
s->bits_per_sample);
+return AVERROR_PATCHWELCOME;
+}
+
 /** dump the extradata */
 for (i = 0; i < avctx->extradata_size; i++)
 ff_dlog(avctx, "[%x] ", avctx->extradata[i]);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/wmaprodec: Fix overflow of cutoff

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 13:11:23 2015 +0100| [bdf79f29db6ff5ebd7d09082734c6e39f5afc39f] | 
committer: Michael Niedermayer

avcodec/wmaprodec: Fix overflow of cutoff

Fixes: 
129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_76ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 0c56f8303e676556ea09bfac73d881c6c9057259)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bdf79f29db6ff5ebd7d09082734c6e39f5afc39f
---

 libavcodec/wmaprodec.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/wmaprodec.c b/libavcodec/wmaprodec.c
index ecc3aae..5b59da5 100644
--- a/libavcodec/wmaprodec.c
+++ b/libavcodec/wmaprodec.c
@@ -477,7 +477,7 @@ static av_cold int decode_init(AVCodecContext *avctx)
 /** calculate subwoofer cutoff values */
 for (i = 0; i < num_possible_block_sizes; i++) {
 int block_size = s->samples_per_frame >> i;
-int cutoff = (440*block_size + 3 * (s->avctx->sample_rate >> 1) - 1)
+int cutoff = (440*block_size + 3LL * (s->avctx->sample_rate >> 1) - 1)
  / s->avctx->sample_rate;
 s->subwoofer_cutoffs[i] = av_clip(cutoff, 4, block_size);
 }

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/hevc_cabac: Fix multiple integer overflows

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 22:45:46 2015 +0100| [694416e327c2fb6495cbc62f63acec65d8c5446c] | 
committer: Michael Niedermayer

avcodec/hevc_cabac: Fix multiple integer overflows

Fixes: 
04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit d5028f61e44b7607b6a547f218f7d85217490a5b)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=694416e327c2fb6495cbc62f63acec65d8c5446c
---

 libavcodec/hevc_cabac.c |   10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/libavcodec/hevc_cabac.c b/libavcodec/hevc_cabac.c
index 3d16896..c1c9f20 100644
--- a/libavcodec/hevc_cabac.c
+++ b/libavcodec/hevc_cabac.c
@@ -883,11 +883,13 @@ static av_always_inline int mvd_decode(HEVCContext *s)
 int k = 1;
 
 while (k < CABAC_MAX_BIN && get_cabac_bypass(&s->HEVClc->cc)) {
-ret += 1 << k;
+ret += 1U << k;
 k++;
 }
-if (k == CABAC_MAX_BIN)
+if (k == CABAC_MAX_BIN) {
 av_log(s->avctx, AV_LOG_ERROR, "CABAC_MAX_BIN : %d\n", k);
+return 0;
+}
 while (k--)
 ret += get_cabac_bypass(&s->HEVClc->cc) << k;
 return get_cabac_bypass_sign(&s->HEVClc->cc, -ret);
@@ -1025,8 +1027,10 @@ static av_always_inline int 
coeff_abs_level_remaining_decode(HEVCContext *s, int
 
 while (prefix < CABAC_MAX_BIN && get_cabac_bypass(&s->HEVClc->cc))
 prefix++;
-if (prefix == CABAC_MAX_BIN)
+if (prefix == CABAC_MAX_BIN) {
 av_log(s->avctx, AV_LOG_ERROR, "CABAC_MAX_BIN : %d\n", prefix);
+return 0;
+}
 if (prefix < 3) {
 for (i = 0; i < rc_rice_param; i++)
 suffix = (suffix << 1) | get_cabac_bypass(&s->HEVClc->cc);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avformat/mxfenc: Do not crash if there is no packet in the first stream

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sun Dec 13 16:13:22 2015 +0100| [4aa876f2880c0252b547e0de9c2ca1034d6120d9] | 
committer: Michael Niedermayer

avformat/mxfenc: Do not crash if there is no packet in the first stream

Fixes: Ticket4914

Signed-off-by: Michael Niedermayer 
(cherry picked from commit b51e7554e74cbf007a1cab83c7bed3ad9fa2793a)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=4aa876f2880c0252b547e0de9c2ca1034d6120d9
---

 libavformat/mxfenc.c |4 
 1 file changed, 4 insertions(+)

diff --git a/libavformat/mxfenc.c b/libavformat/mxfenc.c
index db7d2bf..319d3c7 100644
--- a/libavformat/mxfenc.c
+++ b/libavformat/mxfenc.c
@@ -2448,6 +2448,10 @@ static int mxf_write_packet(AVFormatContext *s, AVPacket 
*pkt)
 }
 mxf->edit_units_count++;
 } else if (!mxf->edit_unit_byte_count && st->index == 1) {
+if (!mxf->edit_units_count) {
+av_log(s, AV_LOG_ERROR, "No packets in first stream\n");
+return AVERROR_PATCHWELCOME;
+}
 mxf->index_entries[mxf->edit_units_count-1].slice_offset =
 mxf->body_offset - 
mxf->index_entries[mxf->edit_units_count-1].offset;
 }

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avutil/mathematics: Fix division by 0

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Wed Dec  9 17:39:38 2015 +0100| [8be41ad2bb3201aac6ec608e860ecb3e4ff02c26] | 
committer: Michael Niedermayer

avutil/mathematics: Fix division by 0

Fixes: CID1341571

Signed-off-by: Michael Niedermayer 
(cherry picked from commit bc8b1e694cc395fdf5e2917377ef11263c937d85)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8be41ad2bb3201aac6ec608e860ecb3e4ff02c26
---

 libavutil/mathematics.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavutil/mathematics.c b/libavutil/mathematics.c
index 4d8467b..78a87d8 100644
--- a/libavutil/mathematics.c
+++ b/libavutil/mathematics.c
@@ -90,7 +90,7 @@ int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, enum 
AVRounding rnd)
 else {
 int64_t ad = a / c;
 int64_t a2 = (a % c * b + r) / c;
-if (ad >= INT32_MAX && ad > (INT64_MAX - a2) / b)
+if (ad >= INT32_MAX && b && ad > (INT64_MAX - a2) / b)
 return INT64_MIN;
 return ad * b + a2;
 }

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/cabac: Check initial cabac decoder state

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Fri Nov 27 13:37:50 2015 +0100| [ed3d4336769425912b925dc46c8d647fbfb4a400] | 
committer: Michael Niedermayer

avcodec/cabac: Check initial cabac decoder state

Fixes integer overflows
Fixes: 
1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 8000d484b83aafa752d84fbdbfb352ffe0dc64f8)

Conflicts:

libavcodec/cabac.h

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ed3d4336769425912b925dc46c8d647fbfb4a400
---

 libavcodec/cabac.c   |5 -
 libavcodec/cabac.h   |2 +-
 libavcodec/cabac_functions.h |3 ++-
 libavcodec/h264_cabac.c  |5 -
 libavcodec/h264_slice.c  |4 +++-
 5 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/libavcodec/cabac.c b/libavcodec/cabac.c
index 8cc9333..f298336 100644
--- a/libavcodec/cabac.c
+++ b/libavcodec/cabac.c
@@ -51,7 +51,7 @@ void ff_init_cabac_encoder(CABACContext *c, uint8_t *buf, int 
buf_size){
  *
  * @param buf_size size of buf in bits
  */
-void ff_init_cabac_decoder(CABACContext *c, const uint8_t *buf, int buf_size){
+int ff_init_cabac_decoder(CABACContext *c, const uint8_t *buf, int buf_size){
 c->bytestream_start=
 c->bytestream= buf;
 c->bytestream_end= buf + buf_size;
@@ -64,6 +64,9 @@ void ff_init_cabac_decoder(CABACContext *c, const uint8_t 
*buf, int buf_size){
 #endif
 c->low+= ((*c->bytestream++)<<2) + 2;
 c->range= 0x1FE;
+if ((c->range<<(CABAC_BITS+1)) < c->low)
+return AVERROR_INVALIDDATA;
+return 0;
 }
 
 void ff_init_cabac_states(void)
diff --git a/libavcodec/cabac.h b/libavcodec/cabac.h
index f9eafed..857211c 100644
--- a/libavcodec/cabac.h
+++ b/libavcodec/cabac.h
@@ -56,7 +56,7 @@ typedef struct CABACContext{
 }CABACContext;
 
 void ff_init_cabac_encoder(CABACContext *c, uint8_t *buf, int buf_size);
-void ff_init_cabac_decoder(CABACContext *c, const uint8_t *buf, int buf_size);
+int ff_init_cabac_decoder(CABACContext *c, const uint8_t *buf, int buf_size);
 void ff_init_cabac_states(void);
 
 #endif /* AVCODEC_CABAC_H */
diff --git a/libavcodec/cabac_functions.h b/libavcodec/cabac_functions.h
index 4e13253..2d1d2a6 100644
--- a/libavcodec/cabac_functions.h
+++ b/libavcodec/cabac_functions.h
@@ -191,7 +191,8 @@ static av_unused const uint8_t* skip_bytes(CABACContext *c, 
int n) {
 #endif
 if ((int) (c->bytestream_end - ptr) < n)
 return NULL;
-ff_init_cabac_decoder(c, ptr + n, c->bytestream_end - ptr - n);
+if (ff_init_cabac_decoder(c, ptr + n, c->bytestream_end - ptr - n) < 0)
+return NULL;
 
 return ptr;
 }
diff --git a/libavcodec/h264_cabac.c b/libavcodec/h264_cabac.c
index c1c8b80..04d412b 100644
--- a/libavcodec/h264_cabac.c
+++ b/libavcodec/h264_cabac.c
@@ -2026,6 +2026,7 @@ decode_intra_mb:
 const int mb_size = ff_h264_mb_sizes[h->sps.chroma_format_idc] *
 h->sps.bit_depth_luma >> 3;
 const uint8_t *ptr;
+int ret;
 
 // We assume these blocks are very rare so we do not optimize it.
 // FIXME The two following lines get the bitstream position in the 
cabac
@@ -2042,7 +2043,9 @@ decode_intra_mb:
 sl->intra_pcm_ptr = ptr;
 ptr += mb_size;
 
-ff_init_cabac_decoder(&sl->cabac, ptr, sl->cabac.bytestream_end - ptr);
+ret = ff_init_cabac_decoder(&sl->cabac, ptr, sl->cabac.bytestream_end 
- ptr);
+if (ret < 0)
+return ret;
 
 // All blocks are present
 h->cbp_table[mb_xy] = 0xf7ef;
diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index 9cbe8d2..2f32948 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -2319,9 +2319,11 @@ static int decode_slice(struct AVCodecContext *avctx, 
void *arg)
 align_get_bits(&sl->gb);
 
 /* init cabac */
-ff_init_cabac_decoder(&sl->cabac,
+ret = ff_init_cabac_decoder(&sl->cabac,
   sl->gb.buffer + get_bits_count(&sl->gb) / 8,
   (get_bits_left(&sl->gb) + 7) / 8);
+if (ret < 0)
+return ret;
 
 ff_h264_init_cabac_states(h, sl);
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avformat/matroskaenc: Check codecdelay before use

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Wed Dec  9 16:16:46 2015 +0100| [5c7ffbbda3e7addc63702ff51cee95c33457f194] | 
committer: Michael Niedermayer

avformat/matroskaenc: Check codecdelay before use

Fixes CID1238790

Signed-off-by: Michael Niedermayer 
(cherry picked from commit e6971db12b8ae49712b77378fa8141de4904082b)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=5c7ffbbda3e7addc63702ff51cee95c33457f194
---

 libavformat/matroskaenc.c |   12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c
index 3b525ad..dd4f356 100644
--- a/libavformat/matroskaenc.c
+++ b/libavformat/matroskaenc.c
@@ -904,14 +904,18 @@ static int mkv_write_track(AVFormatContext *s, 
MatroskaMuxContext *mkv,
 }
 
 if (codec->codec_type == AVMEDIA_TYPE_AUDIO && codec->initial_padding && 
codec->codec_id == AV_CODEC_ID_OPUS) {
+int64_t codecdelay = av_rescale_q(codec->initial_padding,
+  (AVRational){ 1, codec->sample_rate 
},
+  (AVRational){ 1, 10 });
+if (codecdelay < 0) {
+av_log(s, AV_LOG_ERROR, "Initial padding is invalid\n");
+return AVERROR(EINVAL);
+}
 // mkv->tracks[i].ts_offset = av_rescale_q(codec->initial_padding,
 // (AVRational){ 1, 
codec->sample_rate },
 // st->time_base);
 
-put_ebml_uint(pb, MATROSKA_ID_CODECDELAY,
-  av_rescale_q(codec->initial_padding,
-   (AVRational){ 1, codec->sample_rate },
-   (AVRational){ 1, 10 }));
+put_ebml_uint(pb, MATROSKA_ID_CODECDELAY, codecdelay);
 }
 if (codec->codec_id == AV_CODEC_ID_OPUS) {
 put_ebml_uint(pb, MATROSKA_ID_SEEKPREROLL, OPUS_SEEK_PREROLL);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avformat/utils: estimate_timings_from_pts - increase retry counter, fixes invalid duration for ts files with hevc codec

2015-12-14 Thread Rainer Hochecker
ffmpeg | branch: release/2.7 | Rainer Hochecker  | Sun 
Nov 15 13:58:50 2015 +0100| [93fa19addf6dcf503989709de083b33443dc65c5] | 
committer: Michael Niedermayer

avformat/utils: estimate_timings_from_pts - increase retry counter, fixes 
invalid duration for ts files with hevc codec

Fixes a mpegts file with hevc that fails estimating duration. Increasing number 
of
retries fixes the issue.

Signed-off-by: Michael Niedermayer 
(cherry picked from commit 2d8c2f1a28073d451c7db31291c333cb15ca3d0b)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=93fa19addf6dcf503989709de083b33443dc65c5
---

 libavformat/utils.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/utils.c b/libavformat/utils.c
index 5729b0b..47d0707 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -2391,7 +2391,7 @@ static void 
estimate_timings_from_bit_rate(AVFormatContext *ic)
 }
 
 #define DURATION_MAX_READ_SIZE 25LL
-#define DURATION_MAX_RETRY 4
+#define DURATION_MAX_RETRY 6
 
 /* only usable for MPEG-PS streams */
 static void estimate_timings_from_pts(AVFormatContext *ic, int64_t old_offset)

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/vp3: ensure header is parsed successfully before tables

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Wed Dec  2 22:59:56 2015 +0100| [548a07cdc439def22a24d64a9092830af9ea7e2c] | 
committer: Michael Niedermayer

avcodec/vp3: ensure header is parsed successfully before tables

Fixes assertion failure
Fixes: 
266ee543812e934f7b4a72923a2701d4/signal_sigabrt_76ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit 26379d4fddc17cac853ef297ff327b58c44edbad)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=548a07cdc439def22a24d64a9092830af9ea7e2c
---

 libavcodec/vp3.c |7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c
index fbb7da7..1670aba 100644
--- a/libavcodec/vp3.c
+++ b/libavcodec/vp3.c
@@ -131,7 +131,7 @@ static const uint8_t hilbert_offset[16][2] = {
 
 typedef struct Vp3DecodeContext {
 AVCodecContext *avctx;
-int theora, theora_tables;
+int theora, theora_tables, theora_header;
 int version;
 int width, height;
 int chroma_x_shift, chroma_y_shift;
@@ -2251,6 +2251,7 @@ static int theora_decode_header(AVCodecContext *avctx, 
GetBitContext *gb)
 int ret;
 AVRational fps, aspect;
 
+s->theora_header = 0;
 s->theora = get_bits_long(gb, 24);
 av_log(avctx, AV_LOG_DEBUG, "Theora bitstream version %X\n", s->theora);
 
@@ -2356,6 +2357,7 @@ static int theora_decode_header(AVCodecContext *avctx, 
GetBitContext *gb)
 avctx->color_trc  = AVCOL_TRC_BT709;
 }
 
+s->theora_header = 1;
 return 0;
 }
 
@@ -2364,6 +2366,9 @@ static int theora_decode_tables(AVCodecContext *avctx, 
GetBitContext *gb)
 Vp3DecodeContext *s = avctx->priv_data;
 int i, n, matrices, inter, plane;
 
+if (!s->theora_header)
+return AVERROR_INVALIDDATA;
+
 if (s->theora >= 0x030200) {
 n = get_bits(gb, 3);
 /* loop filter limit values table */

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 17:14:36 2015 +0100| [ece3912daf656651b092f2362f62203594418a3a] | 
committer: Michael Niedermayer

avcodec/dirac_parser: Add basic validity checks for next_pu_offset and 
prev_pu_offset

Signed-off-by: Michael Niedermayer 
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ece3912daf656651b092f2362f62203594418a3a
---

 libavcodec/dirac_parser.c |9 +
 1 file changed, 9 insertions(+)

diff --git a/libavcodec/dirac_parser.c b/libavcodec/dirac_parser.c
index 12f1a60..c7c4b69 100644
--- a/libavcodec/dirac_parser.c
+++ b/libavcodec/dirac_parser.c
@@ -114,6 +114,15 @@ static int unpack_parse_unit(DiracParseUnit *pu, 
DiracParseContext *pc,
 if (pu->pu_type == 0x10 && pu->next_pu_offset == 0)
 pu->next_pu_offset = 13;
 
+if (pu->next_pu_offset && pu->next_pu_offset < 13) {
+av_log(NULL, AV_LOG_ERROR, "next_pu_offset %d is invalid\n", 
pu->next_pu_offset);
+return 0;
+}
+if (pu->prev_pu_offset && pu->prev_pu_offset < 13) {
+av_log(NULL, AV_LOG_ERROR, "prev_pu_offset %d is invalid\n", 
pu->prev_pu_offset);
+return 0;
+}
+
 return 1;
 }
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/dirac_parser: Check that there is a previous PU before accessing it

2015-12-14 Thread Michael Niedermayer
ffmpeg | branch: release/2.7 | Michael Niedermayer  | 
Sat Dec  5 17:15:38 2015 +0100| [d17298b666bcbaba2931d98a292c866c29cc94da] | 
committer: Michael Niedermayer

avcodec/dirac_parser: Check that there is a previous PU before accessing it

Fixes out of array read
Fixes: 
99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer 
(cherry picked from commit a08681f1e614152184615e2bcd71c3d63835f810)

Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d17298b666bcbaba2931d98a292c866c29cc94da
---

 libavcodec/dirac_parser.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/dirac_parser.c b/libavcodec/dirac_parser.c
index c7c4b69..1ca7e31 100644
--- a/libavcodec/dirac_parser.c
+++ b/libavcodec/dirac_parser.c
@@ -201,7 +201,7 @@ static int dirac_combine_frame(AVCodecParserContext *s, 
AVCodecContext *avctx,
 }
 
 /* Get the picture number to set the pts and dts*/
-if (parse_timing_info) {
+if (parse_timing_info && pu1.prev_pu_offset >= 13) {
 uint8_t *cur_pu = pc->buffer +
   pc->index - 13 - pu1.prev_pu_offset;
 int pts = AV_RB32(cur_pu + 13);

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avfilter/vf_delogo: fix show option when clipping

2015-12-14 Thread Jean Delvare
ffmpeg | branch: master | Jean Delvare  | Mon Dec 14 11:23:05 
2015 +0100| [932cbc846f5574ed6b775a0fd586e70b5c8f84a2] | committer: Michael 
Niedermayer

avfilter/vf_delogo: fix show option when clipping

The show option did not take clipping into account, so the borders on
the clipped side wouldn't show up. Fix it.

Signed-off-by: Jean Delvare 
Signed-off-by: Michael Niedermayer 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=932cbc846f5574ed6b775a0fd586e70b5c8f84a2
---

 libavfilter/vf_delogo.c |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavfilter/vf_delogo.c b/libavfilter/vf_delogo.c
index 168af89..8058ea9 100644
--- a/libavfilter/vf_delogo.c
+++ b/libavfilter/vf_delogo.c
@@ -101,8 +101,8 @@ static void apply_delogo(uint8_t *dst, int dst_linesize,
  xdst = dst+logo_x1+1,
  xsrc = src+logo_x1+1; x < logo_x2-1; x++, xdst++, xsrc++) {
 
-if (show && (y == logo_y+1 || y == logo_y+logo_h-2 ||
- x == logo_x+1 || x == logo_x+logo_w-2)) {
+if (show && (y == logo_y1+1 || y == logo_y2-2 ||
+ x == logo_x1+1 || x == logo_x2-2)) {
 *xdst = 0;
 continue;
 }

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/nvenc: clamp initial qp value to [1, 51]

2015-12-14 Thread Agatha Hu
ffmpeg | branch: master | Agatha Hu  | Mon Dec 14 10:34:59 
2015 +0100| [758be457564983b879122f6e08701aa236a7f3bf] | committer: Timo 
Rothenpieler

avcodec/nvenc: clamp initial qp value to [1, 51]

Signed-off-by: Timo Rothenpieler 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=758be457564983b879122f6e08701aa236a7f3bf
---

 libavcodec/nvenc.c |8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
index 4ab56fe..31f2dfd 100644
--- a/libavcodec/nvenc.c
+++ b/libavcodec/nvenc.c
@@ -843,10 +843,10 @@ static av_cold int nvenc_encode_init(AVCodecContext 
*avctx)
 ctx->encode_config.rcParams.initialRCQP.qpInterP  = qp_inter_p;
 
 if(avctx->i_quant_factor != 0.0 && avctx->b_quant_factor != 0.0) {
-ctx->encode_config.rcParams.initialRCQP.qpIntra = qp_inter_p * 
fabs(avctx->i_quant_factor);
-ctx->encode_config.rcParams.initialRCQP.qpIntra += 
avctx->i_quant_offset;
-ctx->encode_config.rcParams.initialRCQP.qpInterB = qp_inter_p * 
fabs(avctx->b_quant_factor);
-ctx->encode_config.rcParams.initialRCQP.qpInterB += 
avctx->b_quant_offset;
+ctx->encode_config.rcParams.initialRCQP.qpIntra = av_clip(
+qp_inter_p * fabs(avctx->i_quant_factor) + 
avctx->i_quant_offset, 0, 51);
+ctx->encode_config.rcParams.initialRCQP.qpInterB = av_clip(
+qp_inter_p * fabs(avctx->b_quant_factor) + 
avctx->b_quant_offset, 0, 51);
 } else {
 ctx->encode_config.rcParams.initialRCQP.qpIntra = qp_inter_p;
 ctx->encode_config.rcParams.initialRCQP.qpInterB = qp_inter_p;

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog


[FFmpeg-cvslog] avcodec/nvenc: set slice number to 1 to improve encoding quality

2015-12-14 Thread Agatha Hu
ffmpeg | branch: master | Agatha Hu  | Mon Dec 14 10:27:36 
2015 +0100| [f1a889737532ad8696574cdb95b844371f74c864] | committer: Timo 
Rothenpieler

avcodec/nvenc: set slice number to 1 to improve encoding quality

Signed-off-by: Timo Rothenpieler 

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=f1a889737532ad8696574cdb95b844371f74c864
---

 libavcodec/nvenc.c |6 ++
 1 file changed, 6 insertions(+)

diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
index 43b8e78..4ab56fe 100644
--- a/libavcodec/nvenc.c
+++ b/libavcodec/nvenc.c
@@ -876,6 +876,9 @@ static av_cold int nvenc_encode_init(AVCodecContext *avctx)
 
 
ctx->encode_config.encodeCodecConfig.h264Config.h264VUIParameters.videoFullRangeFlag
 = avctx->color_range == AVCOL_RANGE_JPEG;
 
+ctx->encode_config.encodeCodecConfig.h264Config.sliceMode = 3;
+ctx->encode_config.encodeCodecConfig.h264Config.sliceModeData = 1;
+
 ctx->encode_config.encodeCodecConfig.h264Config.disableSPSPPS = 
(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) ? 1 : 0;
 ctx->encode_config.encodeCodecConfig.h264Config.repeatSPSPPS = 
(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) ? 0 : 1;
 
@@ -940,6 +943,9 @@ static av_cold int nvenc_encode_init(AVCodecContext *avctx)
 
 break;
 case AV_CODEC_ID_H265:
+ctx->encode_config.encodeCodecConfig.hevcConfig.sliceMode = 3;
+ctx->encode_config.encodeCodecConfig.hevcConfig.sliceModeData = 1;
+
 ctx->encode_config.encodeCodecConfig.hevcConfig.disableSPSPPS = 
(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) ? 1 : 0;
 ctx->encode_config.encodeCodecConfig.hevcConfig.repeatSPSPPS = 
(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) ? 0 : 1;
 

___
ffmpeg-cvslog mailing list
ffmpeg-cvslog@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-cvslog