Thanks Howard. That is good to know. With permission from my employer, I'll post my code once it's completed for the benefit of the whole community (and for my own selfish reasons hoping that if any of you see any glaring mistakes you can point them out).
Sound good? Phillip On Mon, Apr 12, 2010 at 1:13 PM, Howard Chu <[email protected]> wrote: > Phillip Hellewell wrote: > >> Thanks for the detailed response and the tips. I think I am going to give >> it >> a go and cross my fingers. >> >> The scenario at my company is we have abstract file I/O stream classes >> called >> file_read_random and another that derives from that called >> file_read_write. >> These are C++ classes with the following pure virtual functions: read(), >> eof(), seek(), tell(), size(), clone(), and for file_read_write there is >> also >> write(), set_eof(), and reopen_read_only(). We use our I/O classes >> everywhere >> throughout our code, so our functions can handle any kind of stream (could >> be >> a file stream, in-memory stream, socket stream, substream, etc.). >> >> Because all of our code expects and operates on file_read_random et al, in >> order to use them with OpenSSL we are currently stuck with the less >> desirable >> method of copying the stream to a temp file on disk and then using that >> with >> BIO_new_fp(). This isn't a big problem when working with PEM and PFX >> certs >> which are relatively small, but now I am writing some code to parse PKCS7 >> files with enveloped data (S/MIME encrypted emails), so the stream could >> be >> fairly large and I don't want to have to copy it to a temp file. >> >> So the goal is to implement a BIO interface that can "wrap" a >> file_read_random >> or file_read_write object. I don't think it will be too tough, but after >> reading your email I'm sure there will be a couple tricky parts. Thanks >> for >> the hint about using the bio->ptr. I'm pretty sure that is where I will >> store >> a pointer to my stream, but I hope OpenSSL doesn't go mucking with it... >> > > It won't, the ptr is reserved for the BIO implementation's use. > >> >> Phillip >> >> On Mon, Apr 12, 2010 at 12:14 PM, Ray Satiro <[email protected] >> <mailto:[email protected]>> wrote: >> >> Yes you can implement your own BIO. I've done it but I don't recommend >> it. >> >> I had started writing something similar to osslsigncode but that could >> verify Microsoft Authenticode. When hashing the file certain fields >> must >> be excluded. PKCS7_verify() will not take a hash for comparison, only >> the >> content to be hashed, so I couldn't use my own hash function and skip >> the >> fields to be excluded. PKCS7_verify() accepts a BIO for detached >> content >> so instead what I did is a custom BIO that reads in the data but will >> skip >> any excluded field. So when OpenSSL goes to hash the file it calls my >> custom BIO's read which skips the excluded fields. >> >> There are a few things you should remember such that BIOs are an >> abstraction layer and can have more than one instance so stay away from >> static variables (think cpp classes). Instead what I did was use >> bio->ptr >> (pointer) to an internal structure that contains my variables >> >> struct class_variables >> { >> int fd; >> uint64_t filesize; >> } >> etc >> then on construct (ie create function) do something like >> assert( ( bio->ptr = malloc( sizeof( struct class_variables ) ) ) ); >> then on destruct (ie destroy function) free it >> so then in your BIO read/whatever function you can do something like >> this >> struct class_variables *variables = (struct class_variables *)bio->ptr; >> then variables->whatever >> >> In the case of bytes read BIOs actually have a specific member that >> openssl's functions can use internally: >> unsigned long num_read; (i.e. bio->num_read) >> but the problem with that might be that it's only 32bit. with what i >> was >> doing it was ok to ignore this issue >> >> If that's not enough for you you'll have to watch out for things like >> where OpenSSL internal functions copy the pointer to bio data but not >> the >> struct. There's an example of this in PKCS7_verify() where if you pass >> a >> memory BIO it creates a secondary memory BIO and then copies over only >> the >> memory pointer and len from the first BIO, and nothing else, and uses >> that >> secondary BIO for some work. The problem being say you have a callback >> on >> the first BIO to monitor for a read well forget it ever being called. >> >> Probably other stuff I'm not thinking of, but yes it's possible. Can >> you >> share with us what you are trying to do? >> >> --- On *Mon, 4/12/10, Phillip Hellewell /<[email protected] >> <mailto:[email protected]>>/* wrote: >> >> >> From: Phillip Hellewell <[email protected] <mailto:[email protected] >> >> >> >> >> Subject: Re: Custom user-defined BIO >> To: "Howard Chu" <[email protected] <mailto:[email protected] >> >> >> Cc: [email protected] <mailto:[email protected]> >> >> Date: Monday, April 12, 2010, 12:27 PM >> >> >> Thanks for the example. Yeah, my best idea so far was to look at >> existing examples, such as bss_mem.c. One question I still have is >> which of the BIO_CTRL_* commands does my ctrl function need to >> handle? bss_mem.c handles about a dozen of them, but the example >> you >> gave me only handles BIO_CTRL_FLUSH, so is that the only one I'm >> required to handle? >> >> Phillip >> >> On Mon, Apr 12, 2010 at 10:15 AM, Howard Chu <[email protected] >> <http://mc/[email protected]>> wrote: >> >> Phillip Hellewell wrote: >> >> Hi all, >> >> How do I implement my own BIO? Do I just create a function >> that returns a >> BIO_METHOD* with pointers to my custom functions? Is there >> any documentation >> on how to do this? >> >> I searched the mailing list and found that this question >> has >> been asked before >> but no one ever responds. The lack of documentation and >> responses leads me to >> believe that the BIO interface is not really meant to be >> used >> for user-defined >> I/O. >> >> 1999: http://marc.info/?l=ssl-users&m=91639275613495&w=2 >> <http://marc.info/?l=ssl-users&m=91639275613495&w=2> >> <http://marc.info/?l=ssl-users&m=91639275613495&w=2 >> <http://marc.info/?l=ssl-users&m=91639275613495&w=2>> >> 2005: >> http://marc.info/?l=openssl-users&m=110624296809686&w=2 >> <http://marc.info/?l=openssl-users&m=110624296809686&w=2> >> <http://marc.info/?l=openssl-users&m=110624296809686&w=2 >> <http://marc.info/?l=openssl-users&m=110624296809686&w=2>> >> 2008: >> http://marc.info/?l=openssl-dev&m=122959672906832&w=2 >> <http://marc.info/?l=openssl-dev&m=122959672906832&w=2> >> <http://marc.info/?l=openssl-dev&m=122959672906832&w=2 >> <http://marc.info/?l=openssl-dev&m=122959672906832&w=2>> >> >> Please, at least will someone respond with one of the >> following: >> 1. "No, don't even try to do it. You're not supposed to." >> 2. "Yes, you can do it but no one is going to help you. >> Just >> look at some of >> the files in crypto/bio/ and see if you can figure it out." >> >> >> Some things are probably too easy to justify the time it takes >> to >> respond. In this case, define a BIO_METHOD struct with your >> handlers and just use it. >> >> One example is in OpenLDAP's libldap/tls_o.c. >> >> >> http://www.openldap.org/devel/cvsweb.cgi/~checkout~/libraries/libldap/tls_o.c?rev=1.16<http://www.openldap.org/devel/cvsweb.cgi/%7Echeckout%7E/libraries/libldap/tls_o.c?rev=1.16> >> < >> http://www.openldap.org/devel/cvsweb.cgi/%7Echeckout%7E/libraries/libldap/tls_o.c?rev=1.16 >> > >> > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ >
