From: Matthew Fearnley
score_tab[] was only declared/initialised for elements 0..255, but with
block sizes set to 16*16, it was possible to reach 256.
This limit could also be overflowed in the histogram, because it was
declared with a uint8_t type.
This can be fixed, and also allow different Z
From: Matthew Fearnley
All the values in score_tab are positive or 0, and so should be the sum
returned by block_cmp().
The logic in zmbv_me() assumes that all 'bv' values will be non-negative,
in order to guarantee an early return if ever bv==0.
---
libavcodec/zmbvenc.c | 7 ---
1 file cha
From: Matthew Fearnley
Store the value of *xored computed within block_cmp() in a local variable,
and only update the *xored parameter at the same time as *mx,*my are set.
This ensures that the value of *xored is accurate for the value of *mx,*my
whenever the function ends.
Note that the local v
From: Matthew Fearnley
If *xored is 0, then histogram[0]==bw*bh and histogram[1..255]==0.
Because histogram[0] is skipped over for the entropy calculation, the
return value is always 0 when *xored==0, so we don't need to waste time
calculating it.
This addition both clarifies the behaviour of t
From: Matthew Fearnley
The maximum allowable range for ZMBV motion estimation is [-64..63], since
the dx,dy values are each stored in the upper 7 bits of a signed char.
(Previously, the range was capped in the code to 127, resulting in an
effective range of [-127..126])
Also fix a range error i
