[PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[Ian Romanick]

On 04/05/2014 02:45 AM, Daniel Vetter wrote:
> This is C standard hair-splitting, but afaict
> - sum will be promoted to signed int in computation since
>   uint8_t fits
> - signed overflow is undefined.
> 
> No we need to add up an awful lot of bytes to actually make it
  ^^
Now

> overflow. But I guess the real risk is gcc spotting this and going
> bananas. Fix this by simply using unsigned in to force all computations
> to use the well-defined unsigned behaviour.

Seems reasonable... it also seems impossible (ha!) to break anything.

Reviewed-by: Ian Romanick 

> Spotted by coverity.
> 
> Cc: Russell King 
> Cc: Rob Clark 
> Cc: Jean-Francois Moine 
> Signed-off-by: Daniel Vetter 
> ---
>  drivers/gpu/drm/i2c/tda998x_drv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c 
> b/drivers/gpu/drm/i2c/tda998x_drv.c
> index 48af5cac1902..ae2754760d77 100644
> --- a/drivers/gpu/drm/i2c/tda998x_drv.c
> +++ b/drivers/gpu/drm/i2c/tda998x_drv.c
> @@ -568,7 +568,7 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)
>  
>  static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
>  {
> - uint8_t sum = 0;
> + unsigned sum = 0;
>  
>   while (bytes--)
>   sum += *buf++;
> 

[PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[Jean-Francois Moine]

On Sat Apr 5 02:45:01 PDT 2014,
Daniel Vetter  wrote:

> This is C standard hair-splitting, but afaict
> - sum will be promoted to signed int in computation since
>   uint8_t fits
> - signed overflow is undefined.
[snip]
>  drivers/gpu/drm/i2c/tda998x_drv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c 
> b/drivers/gpu/drm/i2c/tda998x_drv.c
> index 48af5cac1902..ae2754760d77 100644
> --- a/drivers/gpu/drm/i2c/tda998x_drv.c
> +++ b/drivers/gpu/drm/i2c/tda998x_drv.c
> @@ -568,7 +568,7 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)
>  
>  static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
>  {
> - uint8_t sum = 0;
> + unsigned sum = 0;
>  
>   while (bytes--)
>   sum += *buf++;

This function may be simplified by:

--- tda998x_drv.c~
+++ tda998x_drv.c
@@ -568,11 +568,11 @@

 static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
 {
-   uint8_t sum = 0;
+   int sum = 0;

while (bytes--)
-   sum += *buf++;
-   return (255 - sum) + 1;
+   sum -= *buf++;
+   return sum;
 }

 #define HB(x) (x)


and the same may be done in hdmi.c:

diff --git a/drivers/video/hdmi.c b/drivers/video/hdmi.c
index 9e758a8..b6c9030 100644
--- a/drivers/video/hdmi.c
+++ b/drivers/video/hdmi.c
@@ -31,14 +31,14 @@
 static void hdmi_infoframe_checksum(void *buffer, size_t size)
 {
u8 *ptr = buffer;
-   u8 csum = 0;
+   int csum = 0;
size_t i;

/* compute checksum */
for (i = 0; i < size; i++)
-   csum += ptr[i];
+   csum -= ptr[i];

-   ptr[3] = 256 - csum;
+   ptr[3] = csum;
 }

 /**


-- 
Ken ar c'henta? | ** Breizh ha Linux atav! **
Jef |   http://moinejf.free.fr/

[PATCH 12/13] drm/i2c/tda998x: Fix signed overflow issue

[Daniel Vetter]

This is C standard hair-splitting, but afaict
- sum will be promoted to signed int in computation since
  uint8_t fits
- signed overflow is undefined.

No we need to add up an awful lot of bytes to actually make it
overflow. But I guess the real risk is gcc spotting this and going
bananas. Fix this by simply using unsigned in to force all computations
to use the well-defined unsigned behaviour.

Spotted by coverity.

Cc: Russell King 
Cc: Rob Clark 
Cc: Jean-Francois Moine 
Signed-off-by: Daniel Vetter 
---
 drivers/gpu/drm/i2c/tda998x_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i2c/tda998x_drv.c 
b/drivers/gpu/drm/i2c/tda998x_drv.c
index 48af5cac1902..ae2754760d77 100644
--- a/drivers/gpu/drm/i2c/tda998x_drv.c
+++ b/drivers/gpu/drm/i2c/tda998x_drv.c
@@ -568,7 +568,7 @@ static irqreturn_t tda998x_irq_thread(int irq, void *data)

 static uint8_t tda998x_cksum(uint8_t *buf, size_t bytes)
 {
-   uint8_t sum = 0;
+   unsigned sum = 0;

while (bytes--)
sum += *buf++;
-- 
1.8.5.2