Thanks for the detailed response and the tips. I think I am going to give it a go and cross my fingers.
The scenario at my company is we have abstract file I/O stream classes called file_read_random and another that derives from that called file_read_write. These are C++ classes with the following pure virtual functions: read(), eof(), seek(), tell(), size(), clone(), and for file_read_write there is also write(), set_eof(), and reopen_read_only(). We use our I/O classes everywhere throughout our code, so our functions can handle any kind of stream (could be a file stream, in-memory stream, socket stream, substream, etc.). Because all of our code expects and operates on file_read_random et al, in order to use them with OpenSSL we are currently stuck with the less desirable method of copying the stream to a temp file on disk and then using that with BIO_new_fp(). This isn't a big problem when working with PEM and PFX certs which are relatively small, but now I am writing some code to parse PKCS7 files with enveloped data (S/MIME encrypted emails), so the stream could be fairly large and I don't want to have to copy it to a temp file. So the goal is to implement a BIO interface that can "wrap" a file_read_random or file_read_write object. I don't think it will be too tough, but after reading your email I'm sure there will be a couple tricky parts. Thanks for the hint about using the bio->ptr. I'm pretty sure that is where I will store a pointer to my stream, but I hope OpenSSL doesn't go mucking with it... Phillip On Mon, Apr 12, 2010 at 12:14 PM, Ray Satiro <[email protected]> wrote: > Yes you can implement your own BIO. I've done it but I don't recommend it. > > I had started writing something similar to osslsigncode but that could > verify Microsoft Authenticode. When hashing the file certain fields must be > excluded. PKCS7_verify() will not take a hash for comparison, only the > content to be hashed, so I couldn't use my own hash function and skip the > fields to be excluded. PKCS7_verify() accepts a BIO for detached content so > instead what I did is a custom BIO that reads in the data but will skip any > excluded field. So when OpenSSL goes to hash the file it calls my custom > BIO's read which skips the excluded fields. > > There are a few things you should remember such that BIOs are an > abstraction layer and can have more than one instance so stay away from > static variables (think cpp classes). Instead what I did was use bio->ptr > (pointer) to an internal structure that contains my variables > > struct class_variables > { > int fd; > uint64_t filesize; > } > etc > then on construct (ie create function) do something like > assert( ( bio->ptr = malloc( sizeof( struct class_variables ) ) ) ); > then on destruct (ie destroy function) free it > so then in your BIO read/whatever function you can do something like this > struct class_variables *variables = (struct class_variables *)bio->ptr; > then variables->whatever > > In the case of bytes read BIOs actually have a specific member that > openssl's functions can use internally: > unsigned long num_read; (i.e. bio->num_read) > but the problem with that might be that it's only 32bit. with what i was > doing it was ok to ignore this issue > > If that's not enough for you you'll have to watch out for things like where > OpenSSL internal functions copy the pointer to bio data but not the struct. > There's an example of this in PKCS7_verify() where if you pass a memory BIO > it creates a secondary memory BIO and then copies over only the memory > pointer and len from the first BIO, and nothing else, and uses that > secondary BIO for some work. The problem being say you have a callback on > the first BIO to monitor for a read well forget it ever being called. > > Probably other stuff I'm not thinking of, but yes it's possible. Can you > share with us what you are trying to do? > > --- On *Mon, 4/12/10, Phillip Hellewell <[email protected]>* wrote: > > > From: Phillip Hellewell <[email protected]> > > Subject: Re: Custom user-defined BIO > To: "Howard Chu" <[email protected]> > Cc: [email protected] > Date: Monday, April 12, 2010, 12:27 PM > > > Thanks for the example. Yeah, my best idea so far was to look at existing > examples, such as bss_mem.c. One question I still have is which of the > BIO_CTRL_* commands does my ctrl function need to handle? bss_mem.c handles > about a dozen of them, but the example you gave me only handles > BIO_CTRL_FLUSH, so is that the only one I'm required to handle? > > Phillip > > On Mon, Apr 12, 2010 at 10:15 AM, Howard Chu > <[email protected]<http://mc/[email protected]> > > wrote: > >> Phillip Hellewell wrote: >> >>> Hi all, >>> >>> How do I implement my own BIO? Do I just create a function that returns >>> a >>> BIO_METHOD* with pointers to my custom functions? Is there any >>> documentation >>> on how to do this? >>> >>> I searched the mailing list and found that this question has been asked >>> before >>> but no one ever responds. The lack of documentation and responses leads >>> me to >>> believe that the BIO interface is not really meant to be used for >>> user-defined >>> I/O. >>> >>> 1999: http://marc.info/?l=ssl-users&m=91639275613495&w=2 >>> <http://marc.info/?l=ssl-users&m=91639275613495&w=2> >>> 2005: http://marc.info/?l=openssl-users&m=110624296809686&w=2 >>> <http://marc.info/?l=openssl-users&m=110624296809686&w=2> >>> 2008: http://marc.info/?l=openssl-dev&m=122959672906832&w=2 >>> <http://marc.info/?l=openssl-dev&m=122959672906832&w=2> >>> >>> Please, at least will someone respond with one of the following: >>> 1. "No, don't even try to do it. You're not supposed to." >>> 2. "Yes, you can do it but no one is going to help you. Just look at >>> some of >>> the files in crypto/bio/ and see if you can figure it out." >>> >> >> Some things are probably too easy to justify the time it takes to respond. >> In this case, define a BIO_METHOD struct with your handlers and just use it. >> >> One example is in OpenLDAP's libldap/tls_o.c. >> >> >> http://www.openldap.org/devel/cvsweb.cgi/~checkout~/libraries/libldap/tls_o.c?rev=1.16<http://www.openldap.org/devel/cvsweb.cgi/%7Echeckout%7E/libraries/libldap/tls_o.c?rev=1.16> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ >> > > >
