Re: New Engine
From: "Diarmuid Oneill" [EMAIL PROTECTED] diarmuidoneill When adding a new engine to openssl, is it necessary diarmuidoneill to add an entry to engine_internal_check in diarmuidoneill engine_list.c and rebuild, or is there some way to diarmuidoneill dynamically add a new engine. ENGINE_new() followed by a number of ENGINE_set_*() followed by ENGINE_add(engine). -- Richard Levitte \ Spannvägen 38, II \ [EMAIL PROTECTED] Chairman@Stacken \ S-168 35 BROMMA \ T: +46-8-26 52 47 Redakteur@Stacken \ SWEDEN \ or +46-709-50 36 10 Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED] Member of the OpenSSL development team: http://www.openssl.org/ Software Engineer, Celo Communications: http://www.celocom.com/ Unsolicited commercial email is subject to an archival fee of $400. See http://www.stacken.kth.se/~levitte/mail/ for more info. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Geoff Thorpe wrote: On Tue, 31 Oct 2000, Ben Laurie wrote: Yes, your answer is satisfactory. If I understand then I can't use openssl.exe main application for testing my new engine (of course after compilation of openssl with new engine features). Exactly, and this is wrong and bad. We should fix it. Enough of the one liners already - go ahead and fix it! Sheesh! Well, the main point is to decide how it should be fixed - once that's done it should be easy, right? This is not *that* big a deal anyhow. Anything that could be assumed as far as default values for initialising an ENGINE can be set by the author of the implementation. Anything that needs to be pulled out of some kind of environment specific to the ENGINE (eg. registry or configs relating to the engine-specific "thing") can also be pulled out by the implementation (and its author). Which leaves us with the only worthwhile point made in this little exchange so far, namely, that it should be possible to set various things inside OpenSSL config files that could be passed into the ENGINEs like a batch file. This *can* be done, but it depends how you choose syntatically to do it ... numerically isn't so hot except for "generic" parameter types that can be defined in OpenSSL once and stay fixed (which is not likely to be the case often IMHO). On the other hand, doing it string based might be ugly too. Anyway, some way of batching a series of ENGINE_ctrl() functions by way of config files could be done. The case that started this deals with some kind of engine implementation that requires a hostname, username, and password. That's hardly typical for engines (so far) - and if any such thing as "default values" for those settings exists on that system, it's very likely they're stored somewhere in the environment that controls (or administers) the stuff that engine deals with. So the "requirement" that they be specified by OpenSSL config files is at best a nice-to-have, but hardly a showstopper. Your comment that; Leaves me with one - if specific engines have to be initialised "by hand", then we haven't got much in the way of abstraction, have we? Is there a better way? Config files? External programs? is a little simplisitic, as have been most of the bite(yte)-sized comments thus far; "to make the engine go" (as you said in another post) requires whatever the engine's purpose dictates and whatever the environment it operates in dictates. If defaults can be assumed, then by definition they can be used inside the engine implementation to obviate the need for ENGINE_ctrl()ing. Likewise if the defaults *exist* somewhere, they can equally be pulled out automatically. The only issue is whether this can or should be driven by generic OpenSSL-based settings rather than ENGINE-specific settings. Well, the point is that if I have some application that uses OpenSSL to provide crypto, I shouldn't have to modify the application in order to be able to use a specific engine, in an ideal world. Other possible cases of engines requiring settings are equally likely to require smart cards, paths, one-time-pad keys, retina-scans, filenames, network addresses, challenge-response variants, hardware settings, system settings, system limitations (eg. RAM, disk caching, whatever), UIDs, PINs, IRQs, DMAs, and probably a tonne of other things. There should be some way to feed arbitrary name-value pairs into an ENGINE perhaps, maybe an ENGINE_ctrl_ext() or some such variant, but the lack of that feature thus far is hardly "not much in the way of abstraction", and likewise the addition of that feature is not going to make such a general problem go away. BTW: Right now, all the existing engine implementations typically work immediately without any "setup" beyond what they work out for themselves before, during, or after initialisation. This is true. But clearly there are other engines where this isn't the case. Anyway, I think the idea of driving ENGINE_ctrl()s from the configuration isn't so good, because it leads to obscure configuration. Probably its better to pass a configuration handle into the ENGINE_init() and let it pick up whatever it needs from there itself? Cheers, Ben. -- http://www.apache-ssl.org/ben.html "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Geoff Thorpe wrote: On Tue, 31 Oct 2000, Ben Laurie wrote: BTW: Right now, all the existing engine implementations typically work immediately without any "setup" beyond what they work out for themselves before, during, or after initialisation. Indeed, but its possible to imagine various engines that require rather complex settings possibly on a per instance basis. Lets say for example we have a PKCS#11 engine. It might need to know the location of the PKCS#11 library, whether to login immediately and probably a bunch of other things I haven't thought of yet. We might also have multiple instances using separate PKCS#11 libraries too. IMHO a situation to avoid is one where this kind of thing is handled only via engine specific ENGINE_ctrl() functions such that an engine aware application needs to be modified to handle newer engines. I can think of several ways of handling this. One is to just give the engine access to a config file and let it decide how to interpret it. At an application level we could have (say) an ENGINE_load_config() high level function that can be used to decide which engines to load, some things like default methods to override and sections containing engine specific configuration. The idea behind this is that a simple engine aware application could then just call ENGINE_load_config("filename.cnf") and forget about any other details. Steve. -- Dr Stephen N. Henson. http://www.drh-consultancy.demon.co.uk/ Personal Email: [EMAIL PROTECTED] Senior crypto engineer, Celo Communications: http://www.celocom.com/ Core developer of the OpenSSL project: http://www.openssl.org/ Business Email: [EMAIL PROTECTED] PGP key: via homepage. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Dr S N Henson wrote: The idea behind this is that a simple engine aware application could then just call ENGINE_load_config("filename.cnf") and forget about any other details. Would carve the way to store the engine configuration in stone... By Goetz -- Goetz Babin-Ebell, TC TrustCenter GmbH, http://www.trustcenter.de Sonninstr. 24-28, 20097 Hamburg, Germany Tel.: +49-(0)40 80 80 26 -0, Fax: +49-(0)40 80 80 26 -126 __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
The idea behind this is that a simple engine aware application could then just call ENGINE_load_config("filename.cnf") and forget about any other details. or you can encode the parameters into string and pass this string around. file-based configuration is not always the best. every engine will have two APIs: common and private. private API is for configuration. for example in case of pkcs#11 engine the private API allows one to select pkcs#11 library to be loaded, enumerate slots, tokens and keys. finally there is function which will encode the parameters into string (like d:\program files\smart card vendor\pkcs11\api.dll,0,0,"the key"). this string is then passed to application which will use it to access private key in uniform fashion by passing the string through common API to engine, which will decode it and act appropriately. the private API is useful for GUI applications which may want to allow browsing through your keys and it is optional. if the selector-string encoding is documented then you can create this string manualy. arne __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Dr S N Henson wrote: The idea behind this is that a simple engine aware application could then just call ENGINE_load_config("filename.cnf") and forget about any other details. The reason I suggested a handle instead of a filename was so that the data could come from elsewhere. Cheers, Ben. -- http://www.apache-ssl.org/ben.html "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Ben Laurie wrote: Dr S N Henson wrote: The idea behind this is that a simple engine aware application could then just call ENGINE_load_config("filename.cnf") and forget about any other details. The reason I suggested a handle instead of a filename was so that the data could come from elsewhere. Well whatever... I was just using that as an example. The whole point is that if an application writer doesn't want or have time to look into how ENGINEs work they can make a simple call and some basic functionality (crypto acceleration for example) is supported. Arne Ansper wrote: The idea behind this is that a simple engine aware application could then just call ENGINE_load_config("filename.cnf") and forget about any other details. or you can encode the parameters into string and pass this string around. file-based configuration is not always the best. No it isn't always the best but having a simple string could get very messy if lots of information needs to be passed around. Maybe something a bit like the extension code. You can pass a simple string but if that isn't sufficient the engine can access other information from the CONF pointer (which need not refer to a config file). Steve. -- Dr Stephen N. Henson. http://www.drh-consultancy.demon.co.uk/ Personal Email: [EMAIL PROTECTED] Senior crypto engineer, Celo Communications: http://www.celocom.com/ Core developer of the OpenSSL project: http://www.openssl.org/ Business Email: [EMAIL PROTECTED] PGP key: via homepage. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
On Tue, 31 Oct 2000, Libor Krystek wrote: I'am creating new engine for other hardware. This hardware must be initialized before using but for its initialization I need input some parameters (e.g. hostname, username, password). Function ENGINE_init(ENGINE *e) call engine function init() and it have no input parameters. How can I do it? Do you have any conception? ENGINE_ctrl(); make your "init()" handler fail (setting an explanatory error) unless ENGINE_ctrl() has already been called to pass in everything you need. ENGINE_init() is used to get a "functional" reference, not just a "structural" one. Eg. ENGINE_by_id() returns only a structural reference - so using that you can call ENGINE_ctrl() to initialise whatever you need. It is only when a structural reference is required that the ENGINE's "init" handler will be called to try and make the ENGINE actually get ready to work. This typically happens by calling ENGINE_init() directly, or something implicit like "ENGINE_set_default()" that requires the ENGINE be "working". For structural references, the ENGINE's own functions are never called - that's so that you can still look around the various ENGINEs available even though many of them won't be supported on your host system (because of lack of drivers, vendor libraries, etc). Does that answer your question? Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Geoff Thorpe wrote: ENGINE_ctrl(); make your "init()" handler fail (setting an explanatory error) unless ENGINE_ctrl() has already been called to pass in everything you need. ENGINE_init() is used to get a "functional" reference, not just a "structural" one. Eg. ENGINE_by_id() returns only a structural reference - so using that you can call ENGINE_ctrl() to initialise whatever you need. It is only when a structural reference is required that the ENGINE's "init" handler will be called to try and make the ENGINE actually get ready to work. This typically happens by calling ENGINE_init() directly, or something implicit like "ENGINE_set_default()" that requires the ENGINE be "working". For structural references, the ENGINE's own functions are never called - that's so that you can still look around the various ENGINEs available even though many of them won't be supported on your host system (because of lack of drivers, vendor libraries, etc). Does that answer your question? Yes, your answer is satisfactory. If I understand then I can't use openssl.exe main application for testing my new engine (of course after compilation of openssl with new engine features). Bye. -- Libor Krystek PVT, a.s., Radnicka 14, 602 00 Brno, Czech Republic email: [EMAIL PROTECTED] tel.: +420-5-42217380 __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Geoff Thorpe wrote: On Tue, 31 Oct 2000, Libor Krystek wrote: I'am creating new engine for other hardware. This hardware must be initialized before using but for its initialization I need input some parameters (e.g. hostname, username, password). Function ENGINE_init(ENGINE *e) call engine function init() and it have no input parameters. How can I do it? Do you have any conception? ENGINE_ctrl(); make your "init()" handler fail (setting an explanatory error) unless ENGINE_ctrl() has already been called to pass in everything you need. ENGINE_init() is used to get a "functional" reference, not just a "structural" one. Eg. ENGINE_by_id() returns only a structural reference - so using that you can call ENGINE_ctrl() to initialise whatever you need. It is only when a structural reference is required that the ENGINE's "init" handler will be called to try and make the ENGINE actually get ready to work. This typically happens by calling ENGINE_init() directly, or something implicit like "ENGINE_set_default()" that requires the ENGINE be "working". For structural references, the ENGINE's own functions are never called - that's so that you can still look around the various ENGINEs available even though many of them won't be supported on your host system (because of lack of drivers, vendor libraries, etc). Does that answer your question? Leaves me with one - if specific engines have to be initialised "by hand", then we haven't got much in the way of abstraction, have we? Is there a better way? Config files? External programs? Cheers, Ben. -- http://www.apache-ssl.org/ben.html "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
From: Ben Laurie [EMAIL PROTECTED] ben Does that answer your question? ben ben Leaves me with one - if specific engines have to be initialised "by ben hand", then we haven't got much in the way of abstraction, have we? Is ben there a better way? Config files? External programs? Depends on what you mean. Each engine will probably come with it's own set of programs to handle it (for example, with CHIL, you have to create private keys in the box with the programs that come with said box, there's no way OpenSSL can support that operation), so external programs is one answer... -- Richard Levitte \ Spannvägen 38, II \ [EMAIL PROTECTED] Chairman@Stacken \ S-168 35 BROMMA \ T: +46-8-26 52 47 Redakteur@Stacken \ SWEDEN \ or +46-709-50 36 10 Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED] Member of the OpenSSL development team: http://www.openssl.org/ Software Engineer, Celo Communications: http://www.celocom.com/ Unsolicited commercial email is subject to an archival fee of $400. See http://www.stacken.kth.se/~levitte/mail/ for more info. __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Ben Laurie wrote: Geoff Thorpe wrote: On Tue, 31 Oct 2000, Libor Krystek wrote: I'am creating new engine for other hardware. This hardware must be initialized before using but for its initialization I need input some parameters (e.g. hostname, username, password). Function ENGINE_init(ENGINE *e) call engine function init() and it have no input parameters. How can I do it? Do you have any conception? ENGINE_ctrl(); make your "init()" handler fail (setting an explanatory error) unless ENGINE_ctrl() has already been called to pass in everything you need. [...] Leaves me with one - if specific engines have to be initialised "by hand", then we haven't got much in the way of abstraction, have we? Is there a better way? Config files? External programs? something like [ENGINE_ENTRY_xyz] ctrl_0 = 1234,LONG:42 ctrl_1 = 0xff,DATA:hex_data [...] resulting in something like: ... ENGINE_ctrl(engine,1234,42,0,0); ENGINE_ctrl(engine,0xff,0,data,0); ... By Goetz -- Goetz Babin-Ebell, TC TrustCenter GmbH, http://www.trustcenter.de Sonninstr. 24-28, 20097 Hamburg, Germany Tel.: +49-(0)40 80 80 26 -0, Fax: +49-(0)40 80 80 26 -126 __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Goetz Babin-Ebell wrote: Ben Laurie wrote: Geoff Thorpe wrote: On Tue, 31 Oct 2000, Libor Krystek wrote: I'am creating new engine for other hardware. This hardware must be initialized before using but for its initialization I need input some parameters (e.g. hostname, username, password). Function ENGINE_init(ENGINE *e) call engine function init() and it have no input parameters. How can I do it? Do you have any conception? ENGINE_ctrl(); make your "init()" handler fail (setting an explanatory error) unless ENGINE_ctrl() has already been called to pass in everything you need. [...] Leaves me with one - if specific engines have to be initialised "by hand", then we haven't got much in the way of abstraction, have we? Is there a better way? Config files? External programs? something like [ENGINE_ENTRY_xyz] ctrl_0 = 1234,LONG:42 ctrl_1 = 0xff,DATA:hex_data [...] resulting in something like: ... ENGINE_ctrl(engine,1234,42,0,0); ENGINE_ctrl(engine,0xff,0,data,0); ... That's the kind of thing I had in mind - or, better, engines could define their own config fields. Cheers, Ben. -- http://www.apache-ssl.org/ben.html "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Richard Levitte - VMS Whacker wrote: From: Ben Laurie [EMAIL PROTECTED] ben Does that answer your question? ben ben Leaves me with one - if specific engines have to be initialised "by ben hand", then we haven't got much in the way of abstraction, have we? Is ben there a better way? Config files? External programs? Depends on what you mean. Each engine will probably come with it's own set of programs to handle it (for example, with CHIL, you have to create private keys in the box with the programs that come with said box, there's no way OpenSSL can support that operation), so external programs is one answer... No, I meant stuff that needs to be done within OpenSSL to make the engine go. Cheers, Ben. -- http://www.apache-ssl.org/ben.html "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
Libor Krystek wrote: Geoff Thorpe wrote: ENGINE_ctrl(); make your "init()" handler fail (setting an explanatory error) unless ENGINE_ctrl() has already been called to pass in everything you need. ENGINE_init() is used to get a "functional" reference, not just a "structural" one. Eg. ENGINE_by_id() returns only a structural reference - so using that you can call ENGINE_ctrl() to initialise whatever you need. It is only when a structural reference is required that the ENGINE's "init" handler will be called to try and make the ENGINE actually get ready to work. This typically happens by calling ENGINE_init() directly, or something implicit like "ENGINE_set_default()" that requires the ENGINE be "working". For structural references, the ENGINE's own functions are never called - that's so that you can still look around the various ENGINEs available even though many of them won't be supported on your host system (because of lack of drivers, vendor libraries, etc). Does that answer your question? Yes, your answer is satisfactory. If I understand then I can't use openssl.exe main application for testing my new engine (of course after compilation of openssl with new engine features). Exactly, and this is wrong and bad. We should fix it. Cheers, Ben. -- http://www.apache-ssl.org/ben.html "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: new engine
On Tue, 31 Oct 2000, Ben Laurie wrote: Yes, your answer is satisfactory. If I understand then I can't use openssl.exe main application for testing my new engine (of course after compilation of openssl with new engine features). Exactly, and this is wrong and bad. We should fix it. Enough of the one liners already - go ahead and fix it! Sheesh! This is not *that* big a deal anyhow. Anything that could be assumed as far as default values for initialising an ENGINE can be set by the author of the implementation. Anything that needs to be pulled out of some kind of environment specific to the ENGINE (eg. registry or configs relating to the engine-specific "thing") can also be pulled out by the implementation (and its author). Which leaves us with the only worthwhile point made in this little exchange so far, namely, that it should be possible to set various things inside OpenSSL config files that could be passed into the ENGINEs like a batch file. This *can* be done, but it depends how you choose syntatically to do it ... numerically isn't so hot except for "generic" parameter types that can be defined in OpenSSL once and stay fixed (which is not likely to be the case often IMHO). On the other hand, doing it string based might be ugly too. Anyway, some way of batching a series of ENGINE_ctrl() functions by way of config files could be done. The case that started this deals with some kind of engine implementation that requires a hostname, username, and password. That's hardly typical for engines (so far) - and if any such thing as "default values" for those settings exists on that system, it's very likely they're stored somewhere in the environment that controls (or administers) the stuff that engine deals with. So the "requirement" that they be specified by OpenSSL config files is at best a nice-to-have, but hardly a showstopper. Your comment that; Leaves me with one - if specific engines have to be initialised "by hand", then we haven't got much in the way of abstraction, have we? Is there a better way? Config files? External programs? is a little simplisitic, as have been most of the bite(yte)-sized comments thus far; "to make the engine go" (as you said in another post) requires whatever the engine's purpose dictates and whatever the environment it operates in dictates. If defaults can be assumed, then by definition they can be used inside the engine implementation to obviate the need for ENGINE_ctrl()ing. Likewise if the defaults *exist* somewhere, they can equally be pulled out automatically. The only issue is whether this can or should be driven by generic OpenSSL-based settings rather than ENGINE-specific settings. Other possible cases of engines requiring settings are equally likely to require smart cards, paths, one-time-pad keys, retina-scans, filenames, network addresses, challenge-response variants, hardware settings, system settings, system limitations (eg. RAM, disk caching, whatever), UIDs, PINs, IRQs, DMAs, and probably a tonne of other things. There should be some way to feed arbitrary name-value pairs into an ENGINE perhaps, maybe an ENGINE_ctrl_ext() or some such variant, but the lack of that feature thus far is hardly "not much in the way of abstraction", and likewise the addition of that feature is not going to make such a general problem go away. BTW: Right now, all the existing engine implementations typically work immediately without any "setup" beyond what they work out for themselves before, during, or after initialisation. Cheers, Geoff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]