Hi,
I've built openssl-1.0.2-stable-SNAP-20131205 with openssl-fips-ecp-2.0.5
on Linux (x86 arch). I specified the no-threads option while building
openssl-1.0.2.
I wrote the following test program:
#include <openssl/rand.h>
int main()
{
RAND_seed("12", 2);
return 0;
}
When I compile and execute this program, it aborts with the following error:
md_rand.c:325: ssleay_rand_add: Assertion `md_c[1] == md_count[1]'
failed.
Aborted (core dumped)
The stack trace from the core is:
(gdb) bt
#0 0x00007f5c63308f77 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f5c6330c5e8 in __GI_abort () at abort.c:90
#2 0x00007f5c63301d43 in __assert_fail_base (fmt=0x7f5c63458f58
"%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x7f5c638cbf5a
"md_c[1] == md_count[1]",
file=file@entry=0x7f5c638cbf50 "md_rand.c", line=line@entry=325,
function=function@entry=0x7f5c638cbfe0 <__PRETTY_FUNCTION__.8039>
"ssleay_rand_add") at assert.c:92
#3 0x00007f5c63301df2 in __GI___assert_fail (assertion=0x7f5c638cbf5a
"md_c[1] == md_count[1]", file=0x7f5c638cbf50 "md_rand.c", line=325,
function=0x7f5c638cbfe0 <__PRETTY_FUNCTION__.8039> "ssleay_rand_add")
at assert.c:101
#4 0x00007f5c63813df2 in ssleay_rand_add (buf=0x400786, num=2, add=2) at
md_rand.c:325
#5 0x00007f5c63813e2d in ssleay_rand_seed (buf=0x400784, num=2) at
md_rand.c:331
#6 0x00007f5c63814cc7 in RAND_seed (buf=0x400784, num=2) at rand_lib.c:150
#7 0x00000000004006d0 in main () at testCrypto.c:4
The code in ssleay_rand_add (see below) seems to assume that the condition
(md_c[1] == md_count[1]) will always hold for single threaded consumers of
the API.
#if !defined(OPENSSL_THREADS) && !defined(OPENSSL_SYS_WIN32)
assert(md_c[1] == md_count[1]);
#endif
This assumption does not hold when OpenSSL is compiled in fips mode because
a call to ssleay_rand_add() can call itself recursively. This global
md_count will be incremented by these recursive calls and the assert fails
when the control eventually returns to the first call of ssleay_rand_add().
Here's a stack trace as evidence of the recursive calls (see #12 and #0).
(gdb) bt
#0 ssleay_rand_add (buf=0x7fffffffdb70, num=32, add=32) at md_rand.c:325
#1 0x00007ffff7aa9d1b in RAND_add (buf=0x7fffffffdb50, num=32, entropy=32)
at rand_lib.c:157
#2 0x00007ffff7aaa9a6 in RAND_poll () at rand_unix.c:393
#3 0x00007ffff7aa8f95 in ssleay_rand_bytes (buf=0x602010 "\212\003",
num=60, pseudo=0) at md_rand.c:394
#4 0x00007ffff7aa9477 in ssleay_rand_nopseudo_bytes (buf=0x602010
"\212\003", num=60) at md_rand.c:536
#5 0x00007ffff7aa9e7f in drbg_get_entropy (ctx=0x7ffff7dd56e0 <ossl_dctx>,
pout=0x7fffffffdcc0, entropy=276, min_len=60, max_len=2147483652) at
rand_lib.c:202
#6 0x00007ffff79e1ca3 in fips_get_entropy () from
/tmp/openssl/build/lib/libcrypto.so.1.0.0
#7 0x00007ffff79e225f in FIPS_drbg_instantiate () from
/tmp/openssl/build/lib/libcrypto.so.1.0.0
#8 0x00007ffff7aaa10d in RAND_init_fips () at rand_lib.c:297
#9 0x00007ffff79f15d9 in OPENSSL_init () at o_init.c:76
#10 0x00007ffff79f1516 in FIPS_mode () at o_fips.c:67
#11 0x00007ffff7aad8ab in EVP_DigestInit_ex (ctx=0x7fffffffdea0,
type=0x7ffff7dc0180 <sha1_md>, impl=0x0) at digest.c:150
#12 0x00007ffff7aa8b9e in ssleay_rand_add (buf=0x400784, num=2, add=2) at
md_rand.c:268
#13 0x00007ffff7aa8e2d in ssleay_rand_seed (buf=0x400784, num=2) at
md_rand.c:331
#14 0x00007ffff7aa9cc7 in RAND_seed (buf=0x400784, num=2) at rand_lib.c:150
#15 0x00000000004006d0 in main () at testCrypto.c:4
I've also verified that this issue exists in 1.0.1e but not in 0.9.8y. A
fix that comes to mind is to change the code from:
#if !defined(OPENSSL_THREADS) && !defined(OPENSSL_SYS_WIN32)
assert(md_c[1] == md_count[1]);
#endif
to
#if !defined(OPENSSL_THREADS) && !defined(OPENSSL_SYS_WIN32) &&
!defined(OPENSSL_FIPS)
assert(md_c[1] == md_count[1]);
#endif
Is this something that should be fixed and can I do anything to help?
Please advise.
Thanks and Regards,
Gaurav Malhotra
----------------------------------------------------
Detailed build steps:
$ cd /tmp/openssl
$ tar -zxf openssl-fips-ecp-2.0.5.tar.gz
$ cd /tmp/openssl/openssl-fips-ecp-2.0.5
$ ./config --prefix=/tmp/openssl/fips
$ make
$ make install
$ cd /tmp/openssl
$ tar -zxf openssl-1.0.2-stable-SNAP-20131205.tar.gz
$ cd /tmp/openssl/openssl-1.0.2-stable-SNAP-20131205/
$ ./config fips -d no-threads shared --prefix=/tmp/openssl/build
--with-fipsdir=/tmp/openssl/fips no-ec2m
$ make depend
$ make
$ make test
---snip---
This test will take some time....123456789ABCDEF ok
../util/shlib_wrap.sh ./randtest
randtest: md_rand.c:325: ssleay_rand_add: Assertion `md_c[1] ==
md_count[1]' failed.
make[1]: *** [test_rand] Aborted (core dumped)
make[1]: Leaving directory
`/tmp/openssl/openssl-1.0.2-stable-SNAP-20131205/test'
make: *** [tests] Error 2
