On Tue, Oct 6, 2009 at 10:51 PM, Patel, Anand <anand.pa...@cobham.com> wrote:
> I would like to know how can I implement general purpose
> encryption/decryption filter that can be used with BIO objects.
>
> Basically, filter should get the data before it is written out to the
> stream/socket/memory. This allows my filter to encrypt/decrypt the data
> for any kind of source/sink object.
>
> Furthermore, is it possible to implement devices also, if so how can I
> do that. Before I knew BIO concept I had implemented device and filter
> using BOOST::Iostream libraries.
>
> BIO seems very attractive concept so far, but I don not know how to
> attach my version of device and filter to it.
>
> Thank you,
> -Anand
Your question is a bit ambiguous to me; the first part seems to
request how to use BIOs as general purpose encryption/decryption data
filters while the second seems to request how one code BIOs of their
own, such as custom filters and devices (source/sinks in OpenSSL BIO
parlance).
The first is the shortest and easiest answer:
use a BIO_f_cipher() BIO and set it up with the cipher you require.
BIO_f_cipher wraps the EVP routines so anything that can be done
through them can be done through BIO_f_cipher().
There's also a BIO_f_md() filter for generic secure hashing of your
data, but you should take care to read it's documentation page before
using as you'll need to know how to extract the calculated hash.
You can combine such BIO filters in a chain with any BIO source/sink
device at the end; often used types are supported in OpenSSL off the
shelf, i.e. BIO_s_fd() for 'file handle' based file I/O,
BIO_s_socket() for (TCP) socket-based I/O, BIO_s_mem() for special
purpose memory-buffered I/O (BIO_s_mem() has a few caveats, see also
second part below. In usage, such can be overcome by using a
bidirectional buffering device instead, called a 'BIO pair': see
BIO_s_bio())
As BIOs are essentially 'C' structures and callbacks invoked through
OpenSSL API functions, such can be simply used in C++ classes as well.
Use the OpenSSL provided APIs to create them, link them in a chain
(BIO_push()), access them (BIO_read(), etc.) and clean them up
(BIO_close(), BIO_free()).
Then there's the answer to the second part: how to create your own
BIOs... very doable, but it requires a bit more studying of existing
code and documentation.
For generic encrypt/decrypt BIOs see the sources
crypto/evp/bio_enc.*
and
crypto/evp/bio_ok.*
we use the BIO_f_cipher() one around here most of the time -- that
would be the one in bio_enc.c; we have an edited BIO_f_reliable() over
here so can't say anything about that one in the official distro.
The BIO system is indeed very attractive indeed, and we use it a /lot/
around here, but a few caveats apply:
- the BIO system is a 'C' design. Which means you'll need some extra
handywork to have it behave like C++ objects or include C++ objects in
a BIO filter chain.
- some quirks in very advanced stuff regarding detaching/attaching
chains and pushing control messages; I should find/make time to prep
and submit to rt@ :-( Don't let that bother you, it won't matter
unless you try some quite fancy stuff.
As the BIO system is 'C' based, your own, custom BIO filters and/or
devices should have a 'C' interface. That is: you /can/ code BIO's in
C++, but you'll need to supply an
extern 'C'
facade with those to make it behave.
Your BIO structure (as visible to 'C') should then be made to carry
the C++ 'this' pointer around so you can properly invoke your C++
virtual and non-virtual class methods from the 'extern "C"' facade
functions and callbacks that should come with your BIO filter /
device.
On the other hand, you can code them entirely in 'C' (a C++ compiler
can generally also compile 'C' code after all); apart from a few quite
advanced filters we took that approach over here.
I find that reading the crypto/BIO/* source code files is most helpful
(but then I don't have a problem grokking code); take heed and note
that a few of these are a little light on the error/failure checking
and handling part (I'm rather a*** retentive in that regard ;-) )
For a filter example, look how evp/bio_enc.c was done, then compare
with a different, slightly more complex filter, such as the
BIO_f_buffer() bidirectional buffering filter found in
crypto/BIO/bf_buff.c
Note that some filters are unidirectional by design/implementation,
despite the fact that the basic BIO structure design facilitates
bidirectional I/O.
For clarity the definition of 'bidirectional I/O capable' as I use it:
A filter or device is 'bidirectional I/O capable' when read(I) and
write(O) data transmissions do not influence one another within the
BIO implementation. [Mark the use of the word 'within' here] In other
words: IN and OUT data directions are independent of one another as
far as the BIO is concerned.'
'UNIdirectional' filters include BIO_f_base64(), which won't play nice
when you pump partial data snippets through it in both directions at
the same time: both ends will receive garbled content then. Such
filters (and devices: see BIO_s_mem()) are easily recognizable in
their implementation as read and write direction methods share common
buffer(s) and/or state (BIO-attached flags and counters which are thus
persisted beyond the callback invocation).
I mention this so you are warned and don't copy&paste an arbitrary
filter source when you need to create a bidirectional I/O capable BIO
filter. Examples of various levels of 'bidirectional I/O support' in
existing BIO filters and devices:
- BIO_f_buffer() (crypto/bio/bf_buff.c) is bidirectional I/O capable
(notice the separate buffers and state for read and write direction)
- BIO_f_linebuffer() (crypto/bio/bf_lbuf.c) is a bidirectional I/O
capable filter which only 'works' in one direction: the linebuffering
is only performed in /write/ direction. Yet it is 100% bidirectional
I/O capable.
- BIO_f_base64() (crypto/evp/bio_b64.c) is a /non/-bidirectional
filter: notice how the read and write methods both meddle with the
single 'shared' base64<->raw text buffer. It only 'works' as expected
when you feed it entire messages, explicitly flush the buffer and only
then switch data direction (I<->O) in the filter chain. Wicked
surprises lie this way, especially when mixing this kind of filter
with implicitly bidirectional usage filters such as, e.g., the
BIO_f_ssl() SSL protocol filter in the same chain.
- BIO_s_socket() (crypto/BIO/bss_sock.c) is fully bidirectional I/O
capable: read and write activity passes to the operating system
without the one influencing the other within the BIO.
- BIO_s_mem() (crypto/BIO/bss_mem.c) is not bidirectional I/O capable
as per the definition above; it /can/ be used to reverse data
direction from write to read, as the writes will fill the shared
buffer and the read calls will extract data from that same buffer, but
it has a few peculiarities that don't make it a good example to base
your own BIOs on, in my opinion.
For a device, the simplest is of course the NULL BIO device
BIO_s_null() (which acts as the any-platform BIO equivalent of UNIX
/dev/null) in
crypto/BIO/bss_null.c
and maybe you'ld have a look at the file-handle based source/sink BIO
BIO_s_fd source code too to get an idea about how, what, goes where
for a source/sink ('device') BIO.
crypto/BIO/bss_fd.c
Note that in BIO land, a 'device' is called a source/sink.
Also note that the BIO structure and callback set must be constructed
for each BIO: yes, a BIO is defined by the configuration structure and
the ID plus function callbacks listed in there, as returned by the
various BIO_f_xyz (for filter xyz) and BIO_s_xyz (for source/sink
device xyz) functions.
These above two parts must be accessible in 'C', i.e. must be done as
'extern "C"' code pieces.
Advise: make sure your BIO has a unique identifier for its
BIO_TYPE_xyz, so you can use the BIO chain traversal and BIO lookup
functions provided by OpenSSL as they are.
There is no problem 'attaching' a C++ object instance to such a BIO,
as each BIO comes with it's own internal structure, where the 'this'
pointer of the C++ instance can be placed inside the BIO instance.
A very nice example and a hint how one could do this can be found by
having a look at the BIO_s_file() source/sink, i.e. the BIO which
represents a 'FILE*' pointer (crypto/BIO/bss_file.c).
Here, note once more that each BIO instance is allocated on the heap
through a call to BIO_new() and the BIO-defined allocator callback.
Note that any 'custom data' which is specific to the BIO can be
allocated any which way you like -- as long as you clean up
accordingly afterwards in your own 'free' callback! and put a pointer
to this custom data in BIO->data for best practices. See in bss_file.c
how the external/internal 'FILE*' pointer is attached to the fresh BIO
instance. The hint here is that attaching a C++ class instance can be
done quite similarly, by doing with its 'this' pointer what bss_file.c
does with a 'FILE*' pointer.
Lastly note that your BIO (filter or device) is completely free to
define and allocate any custom data it deems necessary (such as a C++
class instance of type X); anything you allocate must be freed from
the heap using the matching heap management function: malloc() vs.
free(), new and new[] vs. delete and delete[] respectively.
Of course all BIOs are cleaned up themselves by invoking BIO_free() --
or it's chain-oriented big brother BIO_free_all() - both of which will
call your BIOs 'free' callback when the time comes.
OpenSSL chains and manages BIOs, it does not care what is inside them:
your callbacks have to provide for that.
--
Met vriendelijke groeten / Best regards,
Ger Hobbelt
--------------------------------------------------
web: http://www.hobbelt.com/
http://www.hebbut.net/
mail: g...@hobbelt.com
mobile: +31-6-11 120 978
--------------------------------------------------
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
