On Thu, 18 Jun 2020 at 08:23, Heiko Stuebner <he...@sntech.de> wrote: > > From: Heiko Stuebner <heiko.stueb...@theobroma-systems.com> > > When calculating rrtmp/rr rsa_gen_key_prop() tries to make > (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and > (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[] > with rlen being num_bits * 2 > > On a 4096bit key this comes down to to 257 uint32_t elements > in rr and 256 elements in rrtmp but with the current allocation > rr and rrtmp only have 129 uint32_t elements. > > On 2048bit keys this works by chance as the defined max_rsa_size=4096 > allocates a suitable number of elements, but with an actual 4096bit key > this results in other memory parts getting overwritten. > > So as suggested by Heinrich Schuchardt just use the actual bis-size > of the key as base for the size calculation, in turn making the code > compatible to any future keysizes. > > Suggested-by: Heinrich Schuchardt <xypron.deb...@gmx.de> > Signed-off-by: Heiko Stuebner <heiko.stueb...@theobroma-systems.com> > --- > changes in v6: > - drop max_rsa_size and use the keysize as base > changes in v4: > - new patch > > lib/rsa/rsa-keyprop.c | 14 +++++++++----- > 1 file changed, 9 insertions(+), 5 deletions(-)
Reviewed-by: Simon Glass <s...@chromium.org>