Re: dc(1) fails with big number failure on 2^64

2012-11-08 Thread Jakub Lach
I can confirm something similar.

$ dc
18446744073709551616 18446744073709551616 / ps
dc: big number failure 306b06b: No error: 0

FreeBSD 9.1-PRERELEASE #0 r242513 amd64, clang 



--
View this message in context: 
http://freebsd.1045724.n5.nabble.com/dc-1-fails-with-big-number-failure-on-2-64-tp5759049p5759215.html
Sent from the freebsd-stable mailing list archive at Nabble.com.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: dc(1) fails with big number failure on 2^64

2012-11-08 Thread Michiel Boland

On 11/08/2012 03:06, Jan Mikkelsen wrote:

Hi,

I am seeing this in dc:

janm@gray: dc $ dc
18446744073709551616 18446744073709551616 / ps
dc: big number failure 306b06b: No such file or directory

That number is 2^64. The error is coming from BN_check in bdiv(), which is complaining about the 
number at the top of the stack being uninitialised. Looking at the data, after the second pop in 
bdiv() in bdata.c, b-number-d[b-number-top - 1] == 0. After a while poking around 
in a debugger, it looks like the first word of the second number (a-number-d) is being 
allocated at the same location as the last word of the second number, it gets zeroed, and then 
looks uninitialised.

All of this seems to be happening in the BN_* routines in openssl.

I am seeing this on my builds for 9.1-RC3 and 9.0-p3, as well as the CDROM 
shell on the 9.1-RC3 ISO, so I'm pretty sure it isn't my build process or 
compiler flags. I have checked an OpenBSD 5.2 installation, and it works fine.

Can anyone confirm this? Am I just seeing things? Is there an obvious fix?


No fix, but I see a problem in the BN_add_word function in
/usr/src/crypto/openssl/crypto/bn/bn_word.c

/* Only expand (and risk failing) if it's possibly necessary */
if (((BN_ULONG)(a-d[a-top - 1] + 1) == 0) 
(bn_wexpand(a,a-top+1) == NULL))
return(0);

This can't possibly work, since it never calls bn_wexpand if, for example, you 
add 6 to the number 18446744073709551610


Cheers
Michiel
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: dc(1) fails with big number failure on 2^64

2012-11-08 Thread Michiel Boland

On 11/08/2012 19:32, Michiel Boland wrote:
[...]

No fix, but I see a problem in the BN_add_word function in
/usr/src/crypto/openssl/crypto/bn/bn_word.c


Small test case:-

#include openssl/bn.h
#include limits.h

int main()
{
BIGNUM *n;

n = BN_new();
BN_set_word(n, ULONG_MAX - 1);
BN_add_word(n, 2);
BN_free(n);
return 0;
}


$ gcc x.c -lcrypto
$ valgrind ./a.out
==30682== Memcheck, a memory error detector
==30682== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==30682== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==30682== Command: ./a.out
==30682==
==30682== Invalid write of size 8
==30682==at 0x1328EA8: BN_add_word (bn_word.c:158)
==30682==by 0x40076E: main (in /usr/home/boland/a.out)
==30682==  Address 0x18fc0a8 is 0 bytes after a block of size 8 alloc'd
==30682==at 0x100410B: malloc (in 
/usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)

==30682==by 0x1331B82: CRYPTO_malloc (mem.c:328)
==30682==by 0x1330F76: ??? (bn_lib.c:317)
==30682==by 0x13310C7: bn_expand2 (bn_lib.c:432)
==30682==by 0x133121C: BN_set_word (bn_lib.c:570)
==30682==by 0x400760: main (in /usr/home/boland/a.out)

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


Re: dc(1) fails with big number failure on 2^64

2012-11-08 Thread Jan Mikkelsen
Hi,

Great, the test case is very useful.

I have applied the following patch to crypto/bn/bn_word.c, which fixes the 
problem for me.

--- //depot/vendor/freebsd/9.1-local/src/crypto/openssl/crypto/bn/bn_word.c 
2012-08-13 00:32:35.0 1000
+++ 
/data/scratch/janm/p4/freebsd-image-std-2011.1/FreeBSD/src/crypto/openssl/crypto/bn/bn_word.c
   2012-08-13 00:32:35.0 1000
@@ -145,9 +145,11 @@
return(i);
}
/* Only expand (and risk failing) if it's possibly necessary */
-   if (((BN_ULONG)(a-d[a-top - 1] + 1) == 0) 
-   (bn_wexpand(a,a-top+1) == NULL))
-   return(0);
+   if (
+   (((a-top == 1)  (BN_MASK2 - w  a-d[0])) ||
+((a-top  1)  ((BN_ULONG)(a-d[a-top - 1] + 1) == 0))) 
+   (bn_wexpand(a,a-top+1) == NULL))
+   return(0);
i=0;
for (;;)
{

This is a heap overflow in BN_add_word. I don't know what other problems this 
could cause beyond bc and dc returning crap ...

Regards,

Jan.



On 09/11/2012, at 6:39 AM, Michiel Boland mich...@boland.org wrote:

 On 11/08/2012 19:32, Michiel Boland wrote:
 [...]
 No fix, but I see a problem in the BN_add_word function in
 /usr/src/crypto/openssl/crypto/bn/bn_word.c
 
 Small test case:-
 
 #include openssl/bn.h
 #include limits.h
 
 int main()
 {
BIGNUM *n;
 
n = BN_new();
BN_set_word(n, ULONG_MAX - 1);
BN_add_word(n, 2);
BN_free(n);
return 0;
 }
 
 
 $ gcc x.c -lcrypto
 $ valgrind ./a.out
 ==30682== Memcheck, a memory error detector
 ==30682== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
 ==30682== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
 ==30682== Command: ./a.out
 ==30682==
 ==30682== Invalid write of size 8
 ==30682==at 0x1328EA8: BN_add_word (bn_word.c:158)
 ==30682==by 0x40076E: main (in /usr/home/boland/a.out)
 ==30682==  Address 0x18fc0a8 is 0 bytes after a block of size 8 alloc'd
 ==30682==at 0x100410B: malloc (in 
 /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so)
 ==30682==by 0x1331B82: CRYPTO_malloc (mem.c:328)
 ==30682==by 0x1330F76: ??? (bn_lib.c:317)
 ==30682==by 0x13310C7: bn_expand2 (bn_lib.c:432)
 ==30682==by 0x133121C: BN_set_word (bn_lib.c:570)
 ==30682==by 0x400760: main (in /usr/home/boland/a.out)
 
 ___
 freebsd-stable@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-stable
 To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org


dc(1) fails with big number failure on 2^64

2012-11-07 Thread Jan Mikkelsen
Hi,

I am seeing this in dc:

janm@gray: dc $ dc
18446744073709551616 18446744073709551616 / ps
dc: big number failure 306b06b: No such file or directory

That number is 2^64. The error is coming from BN_check in bdiv(), which is 
complaining about the number at the top of the stack being uninitialised. 
Looking at the data, after the second pop in bdiv() in bdata.c, 
b-number-d[b-number-top - 1] == 0. After a while poking around in a 
debugger, it looks like the first word of the second number (a-number-d) is 
being allocated at the same location as the last word of the second number, it 
gets zeroed, and then looks uninitialised.

All of this seems to be happening in the BN_* routines in openssl.

I am seeing this on my builds for 9.1-RC3 and 9.0-p3, as well as the CDROM 
shell on the 9.1-RC3 ISO, so I'm pretty sure it isn't my build process or 
compiler flags. I have checked an OpenBSD 5.2 installation, and it works fine.

Can anyone confirm this? Am I just seeing things? Is there an obvious fix?

Thanks,

Jan Mikkelsen

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org