RE: segfault when using crypto library inside netscape plugin (Solaris 2.6/Sparc/openssl-0.9.5a)

[Jussi Kohonen]

There is a public function named SHA1_Update in netscape httpd itself (yes, I wasted some time with this too!), and that function of course does not have same parameters as the OpenSSL SHA1_Update.

Because you hardly have access to netscape's source code, you have to bend over and rename the SHA1_Update function in OpenSSL to something else -> lots of files to update. Or link the OpenSSL crypto library using -Wl,-Bsymbolic option. This should prevent the unix loader from resolving calls from crypto library to netscape httpd's code (which in my opinion is a really strange if not stupid feature of solaris/linux).

 

Jussi Kohonen

-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Steve Bazyl
Sent: Wednesday, June 14, 2000 6:21 AM
To: [EMAIL PROTECTED]
Subject: RE: segfault when using crypto library inside netscape plugin (Solaris 2.6/Sparc/openssl-0.9.5a)

One more thing...I also tried adding lock callbacks to make sure its not a threading problem.  Made no difference (was getting lock requests as I should, and only from a single thread as expected).

 

 -----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Steve Bazyl
Sent: Tuesday, June 13, 2000 7:41 PM
To: [EMAIL PROTECTED]
Subject: segfault when using crypto library inside netscape plugin (Solaris 2.6/Sparc/openssl-0.9.5a)

We're having a really strange problem with the openssl crypto library -- it keeps segfaulting down in SHA1_Update when called from an NSAPI plugin (running in NES 3.6). 

 

I've tried building the library with optimizations off and all that fun stuff, and have run the test suite which it passes with flying colors.  I've also written various pieces of test code which drive the crypto lib with both static and dynamic linking, all works fine.  However, every time we run it inside NES, it crashes.

 

I've reduced it down to a simple piece of test code which promptly crashes the web server when invoked.

 

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <nsapi.h> 

 

NSAPI_PUBLIC int openssl_test( pblock *param, Session *sn, Request *rq )
{
  char seed[] = "some arbitrary data to seed the random number generator";

 

  printf ( "seeding..." );
  RAND_seed( seed, sizeof seed - 1); // Probably don't need the -1, but I'm getting paranoid :)
  printf ( "done\n" );
  return REQ_PROCEED;
}

 

To build (assuming you have openssl and netscape libs in appropriate places...):

gcc -G -o test_plugin.so test_plugin.c -lcrypto -lnsl -lsocket -DUNIX -DXP_UNIX -D_REENTRANT

 

Add to the server's obj.conf:

 

Init fn="load-modules" funcs="openssl_test" shlib="wherever_you_put_your_library"

Init fn="openssl_test"

 

Interestingly, if I link against the old SSLeay crypto library it works fine!!!!! (ok...I have an old binary, not quite sure how it was built...maybe something in the build options...probably not since all the tests pass fine?).

 

Any and all help is greatly appreciated :)