On 18.06.20 16:23, Heiko Stuebner wrote:
> From: Heiko Stuebner <heiko.stueb...@theobroma-systems.com>
>
> When calculating rrtmp/rr rsa_gen_key_prop() tries to make
> (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and
> (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[]
> with rlen being num_bits * 2
>
> On a 4096bit key this comes down to to 257 uint32_t elements
> in rr and 256 elements in rrtmp but with the current allocation
> rr and rrtmp only have 129 uint32_t elements.
>
> On 2048bit keys this works by chance as the defined max_rsa_size=4096
> allocates a suitable number of elements, but with an actual 4096bit key
> this results in other memory parts getting overwritten.
>
> So as suggested by Heinrich Schuchardt just use the actual bis-size
> of the key as base for the size calculation, in turn making the code
> compatible to any future keysizes.
>
> Suggested-by: Heinrich Schuchardt <xypron.deb...@gmx.de>
> Signed-off-by: Heiko Stuebner <heiko.stueb...@theobroma-systems.com>
> ---
> changes in v6:
> - drop max_rsa_size and use the keysize as base
> changes in v4:
> - new patch
>
>  lib/rsa/rsa-keyprop.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/lib/rsa/rsa-keyprop.c b/lib/rsa/rsa-keyprop.c
> index 4b54db44c4..83b942615f 100644
> --- a/lib/rsa/rsa-keyprop.c
> +++ b/lib/rsa/rsa-keyprop.c
> @@ -654,14 +654,10 @@ int rsa_gen_key_prop(const void *key, uint32_t keylen, 
> struct key_prop **prop)
>  {
>       struct rsa_key rsa_key;
>       uint32_t *n = NULL, *rr = NULL, *rrtmp = NULL;
> -     const int max_rsa_size = 4096;
>       int rlen, i, ret;
>
>       *prop = calloc(sizeof(**prop), 1);
> -     n = calloc(sizeof(uint32_t), 1 + (max_rsa_size >> 5));
> -     rr = calloc(sizeof(uint32_t), 1 + (max_rsa_size >> 5));
> -     rrtmp = calloc(sizeof(uint32_t), 1 + (max_rsa_size >> 5));
> -     if (!(*prop) || !n || !rr || !rrtmp) {
> +     if (!(*prop)) {
>               ret = -ENOMEM;
>               goto err;
>       }
> @@ -682,6 +678,14 @@ int rsa_gen_key_prop(const void *key, uint32_t keylen, 
> struct key_prop **prop)
>       }
>       memcpy((void *)(*prop)->modulus, &rsa_key.n[i], rsa_key.n_sz - i);
>
> +     n = calloc(sizeof(uint32_t), 1 + ((*prop)->num_bits >> 5));
> +     rr = calloc(sizeof(uint32_t), 1 + (((*prop)->num_bits * 2) >> 5));
> +     rrtmp = calloc(sizeof(uint32_t), 1 + (((*prop)->num_bits * 2) >> 5));

At least for rrtmp you need

2 + (((*prop)->num_bits * 2) >> 5)

For num_bits = 2048 br_i32_decode() writes to indices 0..129.

@Takahiro
This is why the code crashes even with an RSA 2048 key on
qemu_arm_defconfig.

Best regards

Heinrich

> +     if (!n || !rr || !rrtmp) {
> +             ret = -ENOMEM;
> +             goto out;
> +     }
> +
>       /* exponent */
>       (*prop)->public_exponent = calloc(1, sizeof(uint64_t));
>       if (!(*prop)->public_exponent) {
>

Reply via email to