failure building native win32 with mingw
(re-sent with attachments moved onto tarball on server -- apologies if you receive more than one copy of this mail.) Folks: I'm trying to build a native win32 openssl-0.9.8i using cygwin gcc (this is a kind of cross-compilation). The goal is to build a binary distribution of the pyOpenSSL package for Python. The INSTALL.W32 contains instructions, but they don't link shared libs correctly -- exhibit A "openssl-098i-pristine.make.log.txt" in details.tar.bz2. This problem is described on this web page -- http://vitus.wagner.pp.ru/articles/openssl-mingw.html . I tried the patch to Configure from that page (attached as "Configure.patch.txt" in details.tar.bz2) and it did indeed cause the shared libs to be linked correctly and then "make test" passes. However, then when I try to build pyOpenSSL using the openssl-0.9.8i header files I get an error -- "ossl_typ.h:178: error: parse error before numeric constant". Please see attached "gcc.cmdline.txt" and "gcc.log.txt" in details.tar.bz2. The relevant part of the pyOpenSSL "connection.c" is visible here: http://bazaar.launchpad.net/%7Eexarkun/pyopenssl/trunk/annotate/78? start_revid=78&file_id=connection.c-20080219014912-qyb7kjf196jhzlyv-149 It says "#include ". I investigated a bit but I couldn't figure out what is wrong. OCSP_RESPONSE is being defined with a typedef in ossl_typ.h line 178, and that gives this parse error, which presumably has something to do with "DECLARE_ASN1_FUNCTIONS(OCSP_RESPONSE)" over in asn1.h. Hopefully someone on this list can easily spot the problem. Thanks! Regards, Zooko http://launchpadlibrarian.net/20156403/details.tar.bz2 --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $10/month __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
openssl performance optimizations in multithreaded environment...
Hi, It was noticed that openssl-0.9.8 had performance improvement changes in RSA blinding code. This improves the handshake performance quite decently, as per a quick and dirty whitebox test run against that code. I have 2 questions arising out of this work: (a) Have additional performance improvements been made in recent openssl code in general and not just in the above area? (b) Is there some (multithreaded) scalability benchmark available using which I could test performance improvements amongst 2 openssl version? Thanks in advance, Vineet
RE: [PATCH] ts verify for expired certificate patch
I agree with pretty much all of this. As far as the verification process goes in openssl, the certificate is verified before the token is I think which means you will need the date/time at which the digest was signed prior to "validating" the token. Brad -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kyle Hamilton Sent: Wednesday, 3 December 2008 7:04 AM To: openssl-dev@openssl.org Subject: Re: [PATCH] ts verify for expired certificate patch On Mon, Dec 1, 2008 at 9:13 PM, Brad Mitchell <[EMAIL PROTECTED]> wrote: > I don't think there is anything in the openssl (ts) functions to accept > revocation to make this decision anyway. External daemons do exist, such as (e.g.) http://www.carillon.ca/tools/pathfinder.php > At the end of the day, time-stamping is only going to be as secure as the > Time Stamping Authority is. The security of the certificates or rather, the > private key is paramount and if this is a third party performing the time > stamping, then this shouldn't be an issue. The party performing the > verification should only have the X.509 cert for the TSA and the X.509 for > the CA. Verisign is considered The Standard (as far as Microsoft is concerned). In fact: http://support.microsoft.com/kb/293781 lists all the certificates that are essential for Windows 2000, XP, Server 2003, Vista, and Server 2008. I'm not sure we should be considering Microsoft a standards-making body, but it's definitely a large consumer of the standards.) > From a point of view of gathering evidence and time stamping it, if you were > to present data many years later and not be able to verify the data due to > certificates being expired (CA or TSA or otherwise) then the standard would > be pretty useless as well as the concept/idea. I would think that so long > as you can take the revocation into account, a timestamp should be able to > be verified if any and all certificates used in the process are currently > expired, i.e. can't be used to sign at this point in time, but were once > valid and were valid at the time of signing. > > Look forward to other comments and suggestions about this. > > Brad The point of the signature protocol is to effectively mirror the common-law requirements for signature verification: *At the time of the signature*, was the person identified as the signer intending to affix his mark as a part of a ceremony to agree to bind him-or-herself to the performance of a task, the non-performance of a task, or to limit the exercise of his or her rights in any way? (This is essentially part of the 'final proof' of a contract -- contract law, at least in the US, requires a clear understanding of the terms of the agreement, competent parties who could be legally said to be able to agree, and the agreement must include an exchange of valuable consideration. But, if the signature isn't valid, there's no need to determine if the other parts of a contract exist, much less what they entail.) The 'at the time of the signature' is the important part. This is why the time-stamp validation protocol is much more complex than most other X.509 signature validation, it's intended to provide evidence for a court that cannot be obtained any other way. -Kyle H __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] ts verify for expired certificate patch
On Mon, Dec 1, 2008 at 9:13 PM, Brad Mitchell <[EMAIL PROTECTED]> wrote: > I don't think there is anything in the openssl (ts) functions to accept > revocation to make this decision anyway. External daemons do exist, such as (e.g.) http://www.carillon.ca/tools/pathfinder.php > At the end of the day, time-stamping is only going to be as secure as the > Time Stamping Authority is. The security of the certificates or rather, the > private key is paramount and if this is a third party performing the time > stamping, then this shouldn't be an issue. The party performing the > verification should only have the X.509 cert for the TSA and the X.509 for > the CA. Verisign is considered The Standard (as far as Microsoft is concerned). In fact: http://support.microsoft.com/kb/293781 lists all the certificates that are essential for Windows 2000, XP, Server 2003, Vista, and Server 2008. I'm not sure we should be considering Microsoft a standards-making body, but it's definitely a large consumer of the standards.) > From a point of view of gathering evidence and time stamping it, if you were > to present data many years later and not be able to verify the data due to > certificates being expired (CA or TSA or otherwise) then the standard would > be pretty useless as well as the concept/idea. I would think that so long > as you can take the revocation into account, a timestamp should be able to > be verified if any and all certificates used in the process are currently > expired, i.e. can't be used to sign at this point in time, but were once > valid and were valid at the time of signing. > > Look forward to other comments and suggestions about this. > > Brad The point of the signature protocol is to effectively mirror the common-law requirements for signature verification: *At the time of the signature*, was the person identified as the signer intending to affix his mark as a part of a ceremony to agree to bind him-or-herself to the performance of a task, the non-performance of a task, or to limit the exercise of his or her rights in any way? (This is essentially part of the 'final proof' of a contract -- contract law, at least in the US, requires a clear understanding of the terms of the agreement, competent parties who could be legally said to be able to agree, and the agreement must include an exchange of valuable consideration. But, if the signature isn't valid, there's no need to determine if the other parts of a contract exist, much less what they entail.) The 'at the time of the signature' is the important part. This is why the time-stamp validation protocol is much more complex than most other X.509 signature validation, it's intended to provide evidence for a court that cannot be obtained any other way. -Kyle H __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [PATCH] ts verify for expired certificate patch
On Mon, Dec 1, 2008 at 8:53 PM, David Schwartz <[EMAIL PROTECTED]> wrote: > >> Problem Description: >> When a digest has been signed and a response is produced, >> the current version of openssl will not verify the contents >> correctly if the certificate used to sign the digest has expired. >> Solution: >> When verifying the response/token, the time at which the digest was >> signed is used by setting the time in the X509 store by using >> X509_STORE_CTX_set_time. >> I have attached a patch file to fix this. >> Brad > > I don't know enough about the specifics of this use in this case to comment > intelligently on whether this is good or bad. However, I do know enough to > urge caution before this patch is accepted. There are a list of possible > issues. Again, I don't know whether each of these issues actually applies to > this situation, but they must be all ruled out by someone competent before > this patch is accepted: > > 1) The certificate may have been revoked before this signature was made. Yet > the certificate may not be on the current revocation list because expired > certificates may be dropped from that list. This is a failure of the underlying communications protocol (X.509 and the PKIX implementation thereof), there is no means to query a server or OCSP responder to determine if the validity were changed before expiration. > 2) Are we trusting the expired certificate to tell us when the signature was > made? What if the certificate is not very secure and so was only made valid > for, say, 6 months because it was expected to take more than 6 months to > break it. Now that somoene has compromised it 2 years later, we don't want > them to create a signature that claims it was made during the validity > interval. http://en.wikipedia.org/wiki/Time_Stamp_Protocol has reference both to the specification (RFC 3161) (which, among other things, specifies that a hash is sent to a timestamping service, and the timestamping service sends back that hash, along with an X.509 structure which states when the timestamping service committed to that particular hash. It also has reference to a project to add TSP to OpenSSL. The way the TSP works, btw, is this: The hash must be generated first. So, the hash could not have been generated after the TSP responder said it committed to it. The timestamp must be generated last. So, the hash could not have been submitted before the TSP said it committed to it. The TSP in no circumstances ever receives the thing that the hash was made over. This is also, by the way, how MS and Mozilla implement code-signing signature validity after the signing certificate expires. > I believe current practice is only to accept a signature with an expired > certificate if the signature is timestamped by a trusted certificate that > has not itself expired. That way, we know by a certificate that is still > valid that the signature was in fact made during the validity interval for > that signing certificate. I think the rules are relaxed very slightly here, but I'm not entirely certain (I haven't examined current practice this deeply). The design I would make were it up to me (since 'timestamping' is important for legal issues such as you describe): The timestamping root signs timestamping authority certificates, which in turn have a validity period. If the timestamp is within the time period that the timestamp authority is valid, then the signature on the certificate issued by the timestamping root is checked, and the revocation logs of the timestamping root verified that the timestamp authority was not revoked before it issued the timestamp. Of course, this design relies on cryptographic knowledge being maintained and managed inside the court system, which may or may not be a valid design assumption. > Again, this is outside my field of expertise. I just want to make sure this > isn't adopted (by the distribution or by readers of this list) before it's > security is evaluated. > > DS In the OP's case, I'd copy the X509_STORE_CTX_set_time data from the timestamp, after verifying (to best efforts for the cost of the information being protected) that the timestamp is valid. But, this is part of why there's a mailing list here: people who are likely to have expertise in a field can give the necessary warnings. :) -Kyle H __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #1795] [ bug report + tentative patch ] SSL_CTX
Hi, I'm not very familiar with OpenSSL internals, so I'll just state the case as I see it and refrain from copying the public forum in my responses. I accept that your suggestion could be correct with respect to the usage model of "safestack" in the OpenSSL source, and that my view of it is from a more distant POV. The bug as things stand is that the sk_find() operation mutates an unsorted stack, and this non-obvious consequence of calling it violates the re-entrancy guarantee that some safestack consumers make to their own eventual consumers. I think your approach to fixing the problem is to address the lack of re-entrancy of sk_find(), while I'm suggesting callers just be cognizant of the fact that it may mutate non-sorted stacks. What I gather is that your suggestion is to make sk_find() work by having it lazily sort the stack before doing its binary search - as happens at present - but in such a way that parallel calls to _find() will be safe, and that you can pragmatically avoid most of the lock contention overhead with the "if (unlikely(test) { lock(); if (test)" trick. My problem with this approach is that you've traded one non-obvious consequence (_find can modify things) with another (if sk_find is called from multiple threads, then it is the _only_ safe operation to call on the stack object - even other ostensibly read-only operations (i.e., sk_value(), sk_dup()) can't be allowed. I think that the semantics as they stand are probably less confusing than what they'd be after your proposed modification: you haven't completely hidden the mutative nature of the sk_find call, so the API user still needs to be aware of it, but its now somewhat more difficult to define. Ger Hobbelt wrote: On Fri, Nov 28, 2008 at 7:10 PM, Peter Edwards <[EMAIL PROTECTED]> wrote: [...] a) either forego on the side effect and resort to a linear (slow) [...] This isn't ideal, obviously Yup. I know. b) provide a threadsafe environment by surrounding a least each sort [...] You're walking a well worn path WRT race conditions here: if you allow arbitrary calls to the STACK interface to add and remove items, then all A.. the 'arbitrary' in there... n-o-o-o-o-ooo, didn't say /that/! (more precisely: didn't mention it at all; another of those rounds where I forget to list /all/ caveats. So: 'yup!' again. I was well aware of that, but I didn't list that particular caveat of STACK: using insert/delete on a STACK-based object is definitely thread UNsafe. And verily. It was before and it will be after the suggested fix to find(), no matter what one does. Should've mentioned it, mea culpa, but my take here is that this caveat has nothing to do with the STACK implementation per se; the sort of screw-up you describe happens to anyone 'intelligent' enough (koff koff) to mutate collections in a multithreading scenario where such collections present pointers/references to the collected instances to the outside world. So, in short, I wasn't planning on fixing /that/, because nobody can. Or should. What I /did/ plan though, is fix a bug particular to the current STACK implementation and the _fixable_ thread-unsafety threat (with 't') it contains. Take this scenario, which _should_ be safe when regarded from a black box perspective: [single thread running:] // // INIT phase of code // new STACK(); STACK.push(obj1); STACK.push(obj2); // insert/delete/push/pop are okay here! kickstart_multiple_threads(); [--> multiple threads running in parallel, all doing this:] //-- // OPERATIONAL phase of the code //-- ... const type *objref = STACK.find(key); // insert/delete/push/pop are prohibited here! // // yes, that means STACK, and the objects it points at, are // immutable once those multiple threads started. // That's implicit to the reference-instead-of-copy STACK // storage approach. ... - Look at the above: this can work, no locking required /anywhere/. UNLESS you start to play smart and modify find() in such a way that, upon hitting an UNsorted collection, it first sorts the collection, then performs a binary search. (As OpenSSL sk_*() does). The sort() in the find() violates the 'immutable' requirement, so you end up with a headache. :-)) You got me thinking again this morning with your remark (much appreciated!) but after reconsidering my suggestion from yesterday night, it still stands. Here's why: 1) I did /not/ try to fix 'arbitrary' so I'll consider that part settled. See above. 2) Why would a mutating sort() work - including scenarios such as the one you show? Because (a) the sort is now made to happen only once. Never again. No matter how many times you call find() once you've entered the 'STACK is now assumed immutable as we're in multithreaded operational run mode'. (b) thanks to the (now atomic) {is_sorted() +
[openssl.org #1798] patch requested for safestack
I have had problems with the C++ compiler for solaris forte developer 6 U2. Some macros in safestack.h pass in void* and do not cast the values first. Patch details: --- ../../../../openssl-orig/crypto/stack/safestack.h Tue Oct 14 20:00:13 2008 +++ safestack.h Tue Dec 2 15:01:48 2008 @@ -132,9 +132,9 @@ #define SKM_sk_free(type, st) \ sk_free(CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_num(type, st) \ - sk_num(CHECKED_PTR_OF(STACK_OF(type), st)) + sk_num((const _STACK *)CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_value(type, st,i) \ - ((type *)sk_value(CHECKED_PTR_OF(STACK_OF(type), st), i)) + ((type *)sk_value((const _STACK *)CHECKED_PTR_OF(STACK_OF(type), st), i)) #define SKM_sk_set(type, st,i,val) \ sk_set(CHECKED_PTR_OF(STACK_OF(type), st), i, CHECKED_PTR_OF(type, val)) #define SKM_sk_zero(type, st) \ @@ -160,7 +160,7 @@ #define SKM_sk_dup(type, st) \ (STACK_OF(type) *)sk_dup(CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_pop_free(type, st, free_func) \ - sk_pop_free(CHECKED_PTR_OF(STACK_OF(type), st), CHECKED_SK_FREE_FUNC(type, free_func)) + sk_pop_free((_STACK *)CHECKED_PTR_OF(STACK_OF(type), st), CHECKED_SK_FREE_FUNC(type, free_func)) #define SKM_sk_shift(type, st) \ (type *)sk_shift(CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_pop(type, st) \ I have had problems with the C++ compiler for solaris forte developer 6 U2. Some macros in safestack.h pass in void* and do not cast the values first. Patch details: --- ../../../../openssl-orig/crypto/stack/safestack.h Tue Oct 14 20:00:13 2008 +++ safestack.h Tue Dec 2 15:01:48 2008 @@ -132,9 +132,9 @@ #define SKM_sk_free(type, st) \ sk_free(CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_num(type, st) \ - sk_num(CHECKED_PTR_OF(STACK_OF(type), st)) + sk_num((const _STACK *)CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_value(type, st,i) \ - ((type *)sk_value(CHECKED_PTR_OF(STACK_OF(type), st), i)) + ((type *)sk_value((const _STACK *)CHECKED_PTR_OF(STACK_OF(type), st), i)) #define SKM_sk_set(type, st,i,val) \ sk_set(CHECKED_PTR_OF(STACK_OF(type), st), i, CHECKED_PTR_OF(type, val)) #define SKM_sk_zero(type, st) \ @@ -160,7 +160,7 @@ #define SKM_sk_dup(type, st) \ (STACK_OF(type) *)sk_dup(CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_pop_free(type, st, free_func) \ - sk_pop_free(CHECKED_PTR_OF(STACK_OF(type), st), CHECKED_SK_FREE_FUNC(type, free_func)) + sk_pop_free((_STACK *)CHECKED_PTR_OF(STACK_OF(type), st), CHECKED_SK_FREE_FUNC(type, free_func)) #define SKM_sk_shift(type, st) \ (type *)sk_shift(CHECKED_PTR_OF(STACK_OF(type), st)) #define SKM_sk_pop(type, st) \
[openssl.org #1797] re: unable to verify timestamps sgined with a certificate that is now expired
I've made a PATCH to address this issue but may or may not be the correct way to go about it. If you sign a digest with a certificate and want to verify this later on, the current library does not support being able to set the time for verification in the X509 cert store when verifying the certificate. I've changed the TS_RESP_verify to extract the time from the TST and specify the time. Patch details: diff -ur openssl-orig/crypto/asn1/a_gentm.c openssl-work/crypto/asn1/a_gentm.c --- openssl-orig/crypto/asn1/a_gentm.cWed Oct 8 10:00:10 2008 +++ openssl-work/crypto/asn1/a_gentm.cTue Dec 2 14:38:03 2008 @@ -208,6 +208,24 @@ return(0); } +time_t ASN1_GENERALIZEDTIME_get(const ASN1_GENERALIZEDTIME *tm) +{ + char strtime[30]; + time_t returnTime = (time_t) 0; + int i ; + if (tm->length != 15) + return (time_t)-1; + + for (i = ( sizeof(time_t)*8-2) ; i>=0; i--) + { + returnTime += (time_t)(1 << i) ; + strftime(strtime, 16, "%Y%m%d%H%M%SZ",gmtime(&returnTime)); + if (strncmp((char*)tm->data,strtime,15) < 0) + returnTime -= (time_t)(1 << i) ; + } + return returnTime; +} + ASN1_GENERALIZEDTIME *ASN1_GENERALIZEDTIME_set(ASN1_GENERALIZEDTIME *s, time_t t) { diff -ur openssl-orig/crypto/asn1/asn1.h openssl-work/crypto/asn1/asn1.h --- openssl-orig/crypto/asn1/asn1.h Wed Nov 12 15:00:10 2008 +++ openssl-work/crypto/asn1/asn1.h Tue Dec 2 14:21:14 2008 @@ -850,6 +850,7 @@ ASN1_GENERALIZEDTIME *ASN1_GENERALIZEDTIME_adj(ASN1_GENERALIZEDTIME *s, time_t t, int offset_day, long offset_sec); int ASN1_GENERALIZEDTIME_set_string(ASN1_GENERALIZEDTIME *s, const char *str); +time_t ASN1_GENERALIZEDTIME_get(const ASN1_GENERALIZEDTIME *tm); DECLARE_ASN1_FUNCTIONS(ASN1_OCTET_STRING) ASN1_OCTET_STRING *ASN1_OCTET_STRING_dup(const ASN1_OCTET_STRING *a); diff -ur openssl-orig/crypto/ts/ts.h openssl-work/crypto/ts/ts.h --- openssl-orig/crypto/ts/ts.h Wed Nov 12 15:00:22 2008 +++ openssl-work/crypto/ts/ts.h Tue Dec 2 14:44:19 2008 @@ -612,7 +612,7 @@ */ int TS_RESP_verify_signature(PKCS7 *token, STACK_OF(X509) *certs, - X509_STORE *store, X509 **signer_out); + X509_STORE *store, X509 **signer_out, TS_TST_INFO *tst_info); /* Context structure for the generic verify method. */ diff -ur openssl-orig/crypto/ts/ts_rsp_verify.c openssl-work/crypto/ts/ts_rsp_verify.c --- openssl-orig/crypto/ts/ts_rsp_verify.c Tue Apr 24 12:01:28 2007 +++ openssl-work/crypto/ts/ts_rsp_verify.c Tue Dec 2 14:46:52 2008 @@ -65,7 +65,7 @@ /* Private function declarations. */ static int TS_verify_cert(X509_STORE *store, STACK_OF(X509) *untrusted, - X509 *signer, STACK_OF(X509) **chain); + X509 *signer, STACK_OF(X509) **chain, time_t certTime); static int TS_check_signing_certs(PKCS7_SIGNER_INFO *si, STACK_OF(X509) *chain); static ESS_SIGNING_CERT *ESS_get_signing_cert(PKCS7_SIGNER_INFO *si); static int TS_find_cert(STACK_OF(ESS_CERT_ID) *cert_ids, X509 *cert); @@ -138,7 +138,7 @@ * - Returns the signer certificate in 'signer', if 'signer' is not NULL. */ int TS_RESP_verify_signature(PKCS7 *token, STACK_OF(X509) *certs, - X509_STORE *store, X509 **signer_out) + X509_STORE *store, X509 **signer_out, TS_TST_INFO *tst_info) { STACK_OF(PKCS7_SIGNER_INFO) *sinfos = NULL; PKCS7_SIGNER_INFO *si; @@ -186,8 +186,16 @@ if (!signers || sk_X509_num(signers) != 1) goto err; signer = sk_X509_value(signers, 0); + /* Get the time that the token was signed at */ + const ASN1_GENERALIZEDTIME *theTime = TS_TST_INFO_get_time(tst_info); + time_t certTime = time(NULL); + if (theTime) + { + certTime = ASN1_GENERALIZEDTIME_get(theTime); + } + /* Now verify the certificate. */ - if (!TS_verify_cert(store, certs, signer, &chain)) goto err; + if (!TS_verify_cert(store, certs, signer, &chain, certTime)) goto err; /* Check if the signer certificate is consistent with the ESS extension. */ @@ -229,7 +237,7 @@ * freeing the vector. */ static int TS_verify_cert(X509_STORE *store, STACK_OF(X509) *untrusted, - X509 *signer, STACK_OF(X509) **chain) + X509 *signer, STACK_OF(X509) **chain, time_t certTime) { X509_STORE_CTXcert_ctx; int i; @@ -239,6 +247,10 @@ *chain = NULL; X509_STORE_CTX_init(&cert_ctx, store, signer, untrusted); X509_STORE_CTX_set_purpose(&cert_ctx, X509_PURPOSE_TIMESTAMP_SIGN); + + /* set the time in the certificate store for verification */ + X509_STORE_CTX_set_time(&cert_ctx, 0, certTime); + i = X509_verify_cert(&cert_ctx); if (i <= 0) { @@ -418,7 +430,7 @@ /* Verif