Re: [PATCH 1/1] lib: rsa: fix padding_pss_verify

[Heinrich Schuchardt]

On 8/31/22 15:46, Simon Glass wrote:

Hi Heinrich,

On Wed, 31 Aug 2022 at 03:32, Heinrich Schuchardt
<heinrich.schucha...@canonical.com> wrote:

Check the that the hash length is shorter than the message length. This
avoids:

     ./tools/../lib/rsa/rsa-verify.c:275:11: warning:
     ‘*db’ may be used uninitialized [-Wmaybe-uninitialized]
       275 |         db[0] &= 0xff >> leftmost_bits;

Signed-off-by: Heinrich Schuchardt <heinrich.schucha...@canonical.com>
---
  lib/rsa/rsa-verify.c | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/lib/rsa/rsa-verify.c b/lib/rsa/rsa-verify.c
index 1d95cfbdee..255f99be24 100644
--- a/lib/rsa/rsa-verify.c
+++ b/lib/rsa/rsa-verify.c
@@ -234,6 +234,9 @@ int padding_pss_verify(struct image_sign_info *info,
         uint8_t leftmost_mask;
         struct checksum_algo *checksum = info->checksum;

+       if (db_len <= 0)
+               return 1;
+
         /* first, allocate everything */
         db_mask = malloc(db_len);
         db = malloc(db_len);
--
2.37.2

While you are here can you please add a 'Returns' comment for this
function? I think in this case you are return 1, meaning 'different' ?

Looking at the rest of the code 0 means ok and non-zero means not ok.

At another code position ret = EINVAL. So let me update the patch.

Best regards

Heinrich

Regards,
Simon