RE: OpenSSL and multiple threads

2006-06-26 Thread Ambarish Mitra
Some days back, we had a riot on select call usage. You may revisit those posts to see if it is helpful. One thing I do not get is: Each thread has it's own SSL context . I also had a mult-threaded application, and for the entire process, there was only one context created with SSL_CTX_new. The

Re: OpenSSL and multiple threads

2006-06-26 Thread Krishna M Singh
Hi We are using the multiple contexts (although not same as thread count i.e. 10 Contexts for 3 threads).. Select call may be failing as the default FD_SET_SIZE is 255 on most systems and thus in case u want to handle 1000 sockets u need to increase the limit.. There is #def in some Windows

Re: Accessing Manual Pages in openssl

2006-06-26 Thread Dave Pawson
On 26/06/06, Simon [EMAIL PROTECTED] wrote: Another option may be using http://www.openssl.org/docs/crypto/, http://www.openssl.org/docs/ssl and http://www.openssl.org/docs/apps Is that the same as the generated files please? Seems so, however, my only comment is there is no

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Bodo Moeller
On Thu, Jun 22, 2006 at 10:41:14PM +0100, Darryl Miles wrote: SSL_CTX_set_mode(3) SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER Make it possible to retry SSL_write() with changed buffer location (the buffer contents must stay the same). This is not the default to avoid the mis-

RE: OpenSSL and multiple threads

2006-06-26 Thread Leon
Thanks for your reply! Some days back, we had a riot on select call usage. You may revisit those posts to see if it is helpful. Well, I do not think it is select() since it works for a 1000 threads. The part that fails is also part of the standard OpenSSL code so I would not like to change it

Re: OpenSSL and multiple threads

2006-06-26 Thread Bernhard Froehlich
Leon wrote: Thanks for your reply! Some days back, we had a riot on select call usage. You may revisit those posts to see if it is helpful. Well, I do not think it is select() since it works for a 1000 threads. The part that fails is also part of the standard OpenSSL code so I would

RE: OpenSSL and multiple threads

2006-06-26 Thread Ambarish Mitra
One thing I do not get is: Each thread has it's own SSL context . Yes I setup the SSL_CTX in each thread. I have also taken it out of the threads into main() creating one global context BUT this gives the same error. - Even if it gives the same error, I think you should persue the route of

Re: OpenSSL and multiple threads

2006-06-26 Thread Bodo Moeller
On Mon, Jun 26, 2006 at 08:49:19AM +0200, Leon wrote: I tracked the bug with gdb and found that it fails in RAND_poll(), called from SSL_accept(), when a new session key is generated. The strange thing is that after the file descriptor set is zeroed [(FD_ZERO(fset)] the call [FDSET(fd,fset)]

RE: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 14:31 +0530, Ambarish Mitra wrote: One thing I do not get is: Each thread has it's own SSL context . Yes I setup the SSL_CTX in each thread. I have also taken it out of the threads into main() creating one global context BUT this gives the same error. - Even if it

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 10:46 +0200, Bernhard Froehlich wrote: Hmm, another wild shot, could it be that /dev/random runs out of entropy and blocks? What exactly are the symtoms? Does your application crash or is it just hanging? Sorry but I did post a follow up directly after the initial post

Re: OpenSSL and multiple threads

2006-06-26 Thread Kyle Hamilton
Does changing the quota of file descriptors available to the program modify its behavior? 'ulimit -n -H' should give you your maximum number allowed. Remember that network sockets are considered files as well. From your description, I'd think that your file limit is around 2048. Also, your

Re: OpenSSL and multiple threads

2006-06-26 Thread Darryl Miles
Krishna M Singh wrote: We are using the multiple contexts (although not same as thread count i.e. 10 Contexts for 3 threads).. Select call may be failing as the default FD_SET_SIZE is 255 on most systems and thus in case u want to handle 1000 sockets u need to increase the limit.. There is #def

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 03:00 -0700, Kyle Hamilton wrote: Does changing the quota of file descriptors available to the program modify its behavior? 'ulimit -n -H' should give you your maximum number allowed. Made it more from the start. Set at 10. Also, your system may have a limit on the

Re: OpenSSL and multiple threads

2006-06-26 Thread Marek Marcola
Hello, This is the maximum number of fd's the fd_set type holds by default. Maybe it would be possible to stop the crashes and override this with some ugly stack paddings: #defined EXTRA_FDS 500 char padd0[(EXTRA_FDS/8)+1]; fd_set fdread; char padd1[(EXTRA_FDS/8)+1]; Or resolution for

Re: OpenSSL and multiple threads

2006-06-26 Thread Girish Venkatachalam
Right. If I were you I would use kqueue() on *BSD or epoll() which is avail only on 2.6 linux kernels. I am not sure what you are trying to achieve but it may be worthwhile to take a look at libevent by Neils Provos as well. It abstracts out select(), kqueue() and epoll() thus making ur app

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
$ grep __FD_SETSIZE /usr/include/bits/*.h /usr/include/bits/typesizes.h:#define __FD_SETSIZE1024 This is the maximum number of fd's the fd_set type holds by default. Maybe it would be possible to stop the crashes and override this with some ugly stack paddings: If I follow

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 12:46 +0200, Marek Marcola wrote: Or resolution for this problem may be defining new data type my_fd_set, replacing FD_* macros, and use this new data type in select() with cast to fd_set. The select is part of the OpenSSL implementation. I specifically avoided the

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 03:53 -0700, Girish Venkatachalam wrote: Right. If I were you I would use kqueue() on *BSD or epoll() which is avail only on 2.6 linux kernels. I am not sure what you are trying to achieve but it may be worthwhile to take a look at libevent by Neils Provos as well. It

Re: OpenSSL and multiple threads

2006-06-26 Thread Dr. Stephen Henson
On Mon, Jun 26, 2006, Leon wrote: $ grep __FD_SETSIZE /usr/include/bits/*.h /usr/include/bits/typesizes.h:#define __FD_SETSIZE1024 This is the maximum number of fd's the fd_set type holds by default. Maybe it would be possible to stop the crashes and override this with

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Darryl Miles
Bodo Moeller wrote: On Thu, Jun 22, 2006 at 10:41:14PM +0100, Darryl Miles wrote: SSL_CTX_set_mode(3) SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER Make it possible to retry SSL_write() with changed buffer location (the buffer contents must stay the same). This is not the default to avoid

Root certificates usage and distibution

2006-06-26 Thread Oleksandr Darchuk
Hello. Sorry for possible off topic, but may be openssl users and gurus will be able to help me in SSL related issue. I want to receive as much root certificates as possible for internal application (SSL gateway). For me the simples way to do it: export all of them from Internet Explorer. But as

Re: OpenSSL and multiple threads

2006-06-26 Thread Darryl Miles
Leon wrote: $ grep __FD_SETSIZE /usr/include/bits/*.h /usr/include/bits/typesizes.h:#define __FD_SETSIZE1024 This is the maximum number of fd's the fd_set type holds by default. Maybe it would be possible to stop the crashes and override this with some ugly stack paddings: If

Re: OpenSSL and multiple threads

2006-06-26 Thread Darryl Miles
Darryl Miles wrote: But I can see your point now, if it is an OpenSSL problem you are pretty much stuck. For example if OpenSSL uses select() to sleep for /dev/random but your application is already into the 1500th active file descritor. Then OpenSSL is pretty much hosed for using select()

Re: OpenSSL and multiple threads

2006-06-26 Thread Bodo Moeller
On Mon, Jun 26, 2006 at 12:25:09PM +0200, Leon wrote: On Mon, 2006-06-26 at 11:44 +0200, Bodo Moeller wrote: What is the file descriptor number that you observe during these calls? The file descriptor is 1507 which seems correct since each thread opened a socket. Can a single-threaded

Re: OpenSSL and multiple threads

2006-06-26 Thread Marek Marcola
Hello, On Mon, 2006-06-26 at 12:46 +0200, Marek Marcola wrote: Or resolution for this problem may be defining new data type my_fd_set, replacing FD_* macros, and use this new data type in select() with cast to fd_set. The select is part of the OpenSSL implementation. I specifically

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Bodo Moeller
On Mon, Jun 26, 2006 at 12:35:57PM +0100, Darryl Miles wrote: Some of the calls to SSL_write() may read some of your data, I am still not such how the reading of data impacts the write operation. Are you saying that when WANT_READ is returned from SSL_write() the OpenSSL library has

Re: OpenSSL and multiple threads

2006-06-26 Thread Marek Marcola
Hello, The select is part of the OpenSSL implementation. I specifically avoided the select() by going multi threaded and here I am sitting with a select problem (I think) due to the OpenSSL library. I want to stay away from hacking the OpenSSL library. Sorry for misundestanding.

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Kyle Hamilton
On 6/26/06, Darryl Miles [EMAIL PROTECTED] wrote: Bodo Moeller wrote: On Thu, Jun 22, 2006 at 10:41:14PM +0100, Darryl Miles wrote: SSL_CTX_set_mode(3) SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER Make it possible to retry SSL_write() with changed buffer location (the buffer contents

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Darryl Miles
Bodo Moeller wrote: On Mon, Jun 26, 2006 at 12:35:57PM +0100, Darryl Miles wrote: Yes. During the first call to SSL_write(), OpenSSL may take as many bytes as fit into one TLS record, and encrypt this for transport. Then SSL_write() may fail with WANT_WRITE or WANT_READ both before and after

Re: How to verify OpenSSL lib version from autoconf?

2006-06-26 Thread Kyle Hamilton
autoconf compiles small programs to check the expected behavior. If you wrote an m4 macro that checked against the value of the version constant, you could check it. (return 0 on success, 1 on error, I believe is autoconf's concept.) Note that I haven't looked into autoconf for about six

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Bodo Moeller
On Mon, Jun 26, 2006 at 02:04:47PM +0100, Darryl Miles wrote: Bodo Moeller wrote: On Mon, Jun 26, 2006 at 12:35:57PM +0100, Darryl Miles wrote: Yes. During the first call to SSL_write(), OpenSSL may take as many bytes as fit into one TLS record, and encrypt this for transport. Then

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Darryl Miles
Kyle Hamilton wrote: On 6/26/06, Darryl Miles [EMAIL PROTECTED] wrote: I still have not gotten to the bottom of the entire scope of situation(s) can cause an SSL_write() to return -1 WANT_READ. If its only renegotiation that can; then this is always instigated by a SSL_renegotiate() (from my

[no subject]

2006-06-26 Thread jean-luc . le-poupon
Hello, I try to install openssl-0.9.7i onto Cygwin 5.6.xxx. Windows XP SP2 French The ./config command aborts with the following error DES_PTR used DES_RISC1 used DES_UNROLL used BN_LLONG mode RC4_INDEX mode RC4_CHUNK is undefined 'make' n'est pas reconnu en tant que commande interne ou externe,

Re: OpenSSL and multiple threads

2006-06-26 Thread Richard Salz
select() has a limit on how big the descriptors can be, otherwise it crashes. /r$ -- SOA Appliances Application Integration Middleware __ OpenSSL Project http://www.openssl.org User

Re: OpenSSL and multiple threads

2006-06-26 Thread Richard Salz
You may look at poll() and epoll() as alternative event wake mechanisms for IO with large numbers of fds in the working set. Yes. Either rebuild your entire system and fix this value: /usr/include/bits/typesizes.h:#define __FD_SETSIZE1024 or use poll. You'll probably find

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Darryl Miles
Bodo Moeller wrote: On Mon, Jun 26, 2006 at 02:04:47PM +0100, Darryl Miles wrote: Bodo Moeller wrote: On Mon, Jun 26, 2006 at 12:35:57PM +0100, Darryl Miles wrote: Yes. During the first call to SSL_write(), OpenSSL may take as many bytes as fit into one TLS record, and encrypt this for

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 14:24 +0200, Marek Marcola wrote: For me seems that if you properly initialize PRNG (before creating threads) this may resolve problem. I think something like: RAND_load_file(/dev/urandom, 1024); should be enough. OK weirdness going on here. I've added the

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Marek Marcola
Hello, * TLS header/protocol overhead * Cipher blocks and chaining modes (picking the most commonly used) * Blocking mode padding overhead * Ethernet 1500 MTUs I presume the minimum is 1 byte, to be send and flushed at the receiver. But maximum block size I read somewhere maybe

Re: SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER

2006-06-26 Thread Marek Marcola
Hello, For example if you have 8 data bytes to send: 8(data) + 20(MAC) + 8(padding) = 36 and 5 bytes for SSL3/TLS record header = 41. Sorry, mistake, should be: For example if you have 12 data bytes to send: (12(data) + 20(MAC)) + 8(padding) = 40 and 5 bytes for SSL3/TLS record

Re: OpenSSL and multiple threads

2006-06-26 Thread Darryl Miles
Leon wrote: On Mon, 2006-06-26 at 14:24 +0200, Marek Marcola wrote: OK weirdness going on here. I've added the RAND_load_file() command to the beginning of my program and it does not make a difference. With a 1000 threads I get a call to RAND_poll() only with the first connection and not with

SSL Compile Problem II

2006-06-26 Thread Randy
This code compiles and links fine. If I uncomment the SSL_new line I get undefined symbol: ssl_x SSL_CTX *ctx; // SSL_new *ssl_x; /* Initializing OpenSSL */ SSL_load_error_strings();/* readable error messages */ SSL_library_init();

Re: SSL Compile Problem II

2006-06-26 Thread Darryl Miles
Randy wrote: This code compiles and links fine. If I uncomment the SSL_new line I get undefined symbol: ssl_x SSL_CTX *ctx; // SSL_new *ssl_x; SSL_new is not a type, but a function: int SSL_new(SSL *); #include openssl/ssl.h SSL *ssl_x; should work. HTH Darryl

Re: SSL Compile Problem II

2006-06-26 Thread Darryl Miles
Darryl Miles wrote: SSL_new is not a type, but a function: int SSL_new(SSL *); Opps: SSL_new is not a type, but a function: SSL *SSL_new(SSL_CTX *); So then you'd use: ssl_x = SSL_new(ctx); __ OpenSSL Project

OpenSSL and Windows XP Install Issues

2006-06-26 Thread Joseph Mierwa
Hi All, I'm new to open SSL and have been out of the unix realm for several years now. I have cygwin installed on my machine and am trying to get openssl to build. Assuming I'm reading the install directions correctly, the configure is executing OK, but the ms\do_ms fails with

RE: SSL Compile Problem II

2006-06-26 Thread Randy
Unfortunately that is how I orginally had it but that generates the same error. SSL *ssl_x; Generates UX:acomp: ERROR: curl_dip.c, line 271: undefined symbol: ssl_x UX:acomp: WARNING: curl_dip.c, line 457: assignment type mismatch *** Error code 1 (bu21) -Original

Re: SSL Compile Problem II

2006-06-26 Thread Darryl Miles
Randy wrote: Unfortunately that is how I orginally had it but that generates the same error. SSL *ssl_x; Generates UX:acomp: ERROR: curl_dip.c, line 271: undefined symbol: ssl_x UX:acomp: WARNING: curl_dip.c, line 457: assignment type mismatch *** Error code 1 (bu21)

RE: SSL Compile Problem II

2006-06-26 Thread Randy
Got it, had two different ssl.h files on the system. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Randy Sent: Monday, June 26, 2006 10:12 AM To: openssl-users@openssl.org Subject: SSL Compile Problem II This code compiles and links fine. If I

Re: OpenSSL and multiple threads

2006-06-26 Thread Leon
On Mon, 2006-06-26 at 15:59 +0100, Darryl Miles wrote: I do not believe the kernel has any problem with super-large fd_set's being passed to it. I believe the kernel will use whatever size its given and attempt to access the memory based on the 'maxfd' argument to select. If the kernel

Re: Creating compatible PKCS12 files

2006-06-26 Thread Jason K. Resch
Dr. Henson, I wanted to thank you for your suggestions, it is working now. I had to change the code: int res = i2d_PKCS8PrivateKey_fp (fp, clave, EVP_des_ede3_cbc(), NULL, 0, NULL, pwd); TO int pbe_nid = OBJ_txt2nid(PBE-SHA1-3DES); int res = i2d_PKCS8PrivateKey_nid_fp (fp, clave, pbe_nid,

Re: Cygwin 5.6.xxx. Windows XP SP2 French

2006-06-26 Thread Marek Marcola
Hello, I try to install openssl-0.9.7i onto Cygwin 5.6.xxx. Windows XP SP2 French The ./config command aborts with the following error DES_PTR used DES_RISC1 used DES_UNROLL used BN_LLONG mode RC4_INDEX mode RC4_CHUNK is undefined 'make' n'est pas reconnu en tant que commande interne

Re: Creating compatible PKCS12 files

2006-06-26 Thread Dr. Stephen Henson
On Mon, Jun 26, 2006, Jason K. Resch wrote: I wanted to thank you for your suggestions, it is working now. I had to change the code: int res = i2d_PKCS8PrivateKey_fp (fp, clave, EVP_des_ede3_cbc(), NULL, 0, NULL, pwd); TO int pbe_nid = OBJ_txt2nid(PBE-SHA1-3DES); int res =

Re: sigsegv in BN_BLINDING_free 0.9.8a

2006-06-26 Thread Robin Bowes
Matthew L Daniel wrote: I am experiencing a SIGSEGV in BN_BLINDING_free because mt_blinding appears to be 0x11 instead of a pointer to some memory. We had an identical issue reported here: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=193633 which is somehow caused by the use of Zimbra