Re: [openssl-users] Build problems on Windows

[jeff saremi]

thanks a lot. I opened a "VS2015 x64 Native Tools" window as opposed to a 
"VS2015 x64 x86 Cross Tools" and everything worked amazingly with no issues.

The names are super confusing.

but i'm ok now. thanks


From: openssl-users  on behalf of Jeffrey 
Walton 
Sent: Tuesday, January 10, 2017 11:52 AM
To: OpenSSL Users
Subject: Re: [openssl-users] Build problems on Windows


IF EXIST libcrypto-1_1-x64.dll.manifest DEL /F /Q 
libcrypto-1_1-x64.dll.manifest
link /nologo /debug /dll  /implib:libcrypto.lib 
/out:libcrypto-1_1-x64.dll /def:libcrypto-1_1-x64.def 
@C:\Users\jesaremi\AppData\Local\Temp\nm8557.tmp || (DEL /Q libcrypto.* 
libcrypto-1_1-x64.* && EXIT 1)
crypto\aes\aes_cfb.obj : fatal error LNK1112: module machine type 'X86' 
conflicts with target machine type 'x64'
NMAKE : fatal error U1077: 'link' : return code '0x1'
Stop.

It sounds like the wrong Developer Tools Command Prompt was opened. You can 
find them through Start -> Programs -> Visual Studio  -> Developer Tools. 
Also see https://msdn.microsoft.com/en-us/library/ms229859(v=vs.110).aspx .
Developer Command Prompt for Visual 
Studio
msdn.microsoft.com
The Developer Command Prompt for Visual Studio automatically sets the 
environment variables that enable you to easily use .NET Framework tools.



If you plan on building for x86 and you configure for VC-WIN32, then be sure 
you open the x86 command prompt, and not the x64 one.

If you want to build for x64, then be sure to configure with VC-WIN64A, and be 
sure to open a x64 developer command prompt.

If you have the correct command prompt open, then perform a clean or distclean. 
You may have old artifacts lying around.

Jeff
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] Build problems on Windows

[jeff saremi]

I installed ActivePerl and got a lot further

I now get link errors. Please see below. The commands are the same: perl 
Configure VS-WIN64A and nmake:


"C:\Perl64\bin\perl.exe" "util\mkdef.pl" "crypto" 32 > 
libcrypto-1_1-x64.def
"C:\Perl64\bin\perl.exe" -i.tmp -pe "s|^LIBRARY\s+crypto32|LIBRARY 
libcrypto-1_1-x64|;" libcrypto-1_1-x64.def
DEL libcrypto-1_1-x64.def.tmp
"C:\Perl64\bin\perl.exe" "util\mkrc.pl" libcrypto-1_1-x64.dll > 
libcrypto-1_1-x64.rc
rc /folibcrypto-1_1-x64.res libcrypto-1_1-x64.rc
Microsoft (R) Windows (R) Resource Compiler Version 6.3.9600.17336
Copyright (C) Microsoft Corporation.  All rights reserved.

IF EXIST libcrypto-1_1-x64.dll.manifest DEL /F /Q 
libcrypto-1_1-x64.dll.manifest
link /nologo /debug /dll  /implib:libcrypto.lib 
/out:libcrypto-1_1-x64.dll /def:libcrypto-1_1-x64.def 
@C:\Users\jesaremi\AppData\Local\Temp\nm8557.tmp || (DEL /Q libcrypto.* 
libcrypto-1_1-x64.* && EXIT 1)
crypto\aes\aes_cfb.obj : fatal error LNK1112: module machine type 'X86' 
conflicts with target machine type 'x64'
NMAKE : fatal error U1077: 'link' : return code '0x1'
Stop.




From: openssl-users  on behalf of Jakob Bohm 

Sent: Tuesday, January 10, 2017 11:20 AM
To: openssl-users@openssl.org
Subject: Re: [openssl-users] Build problems on Windows

On 10/01/2017 19:43, Matt Caswell wrote:
>   Notes on Perl on Windows
>   
>
>   There are a number of build targets that can be viewed as "Windows".
>   Indeed, there are VC-* configs targeting VisualStudio C, as well as
>   MinGW and Cygwin. The key recommendation is to use "matching" Perl,
>   one that matches build environment. For example, if you will build
>   on Cygwin be sure to use the Cygwin package manager to install Perl.
>   For MSYS builds use the MSYS provided Perl. For VC-* builds we
>   recommend ActiveState Perl, available from
>   http://www.activestate.com/ActivePerl.
ActivePerl | ActiveState
www.activestate.com
ActivePerl Business and Enterprise Editions feature our precompiled, supported, 
quality-assured Perl distribution used by millions of developers around the 
world for ...


>
Really?, I thought ActiveState ActivePerl was pretty much dead/historic.

While I have not bothered with OpenSSL 1.1.x builds yet, I usually use
Strawberry Perl for VC-related work, and it seems to work fine with the
1.0.2 sources.  Since I have not tested with 1.1.x sources, this is
obviously not intended as advice to people trying to build, more as
something you might consider for an updated version of NOTES.PERL
(after testing it of cause).

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
[https://www.wisemo.com/wp-content/uploads/WMO_03.jpg]

WiseMo A/S · Remote desktop control of Smartphones ...
www.wisemo.com
WiseMo provides remote desktop access from anywhere. Secure, fast and stable 
remote control software for Tablet, Smartphone and PC


Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] Build problems on Windows

[jeff saremi]

i was not aware of that. thanks so much. I'll go back and install a proper Perl



From: openssl-users  on behalf of Matt 
Caswell 
Sent: Tuesday, January 10, 2017 10:43 AM
To: openssl-users@openssl.org
Subject: Re: [openssl-users] Build problems on Windows



On 10/01/17 18:34, jeff saremi wrote:
> D:\repos\openssl2\openssl-1.1.0c>perl -v
>
> This is perl 5, version 22, subversion 1 (v5.22.1) built for
> x86_64-msys-thread-multi
> Copyright 1987-2015, Larry Wall

You are using msys perl but doing a VC build.

See this extract from NOTES.PERL in the distribution:

 Notes on Perl on Windows
 

 There are a number of build targets that can be viewed as "Windows".
 Indeed, there are VC-* configs targeting VisualStudio C, as well as
 MinGW and Cygwin. The key recommendation is to use "matching" Perl,
 one that matches build environment. For example, if you will build
 on Cygwin be sure to use the Cygwin package manager to install Perl.
 For MSYS builds use the MSYS provided Perl. For VC-* builds we
 recommend ActiveState Perl, available from
 http://www.activestate.com/ActivePerl.
ActivePerl | ActiveState<http://www.activestate.com/ActivePerl>
www.activestate.com
ActivePerl Business and Enterprise Editions feature our precompiled, supported, 
quality-assured Perl distribution used by millions of developers around the 
world for ...




Matt


>
>
>
> 
> *From:* openssl-users  on behalf of
> Jakob Bohm 
> *Sent:* Monday, January 9, 2017 9:46 PM
> *To:* openssl-users@openssl.org
> *Subject:* Re: [openssl-users] Build problems on Windows
>
> On 10/01/2017 05:04, jeff saremi wrote:
>>
>> Hello
>>
>> I downloaded openssl-1.1.0c and i'm trying to build this on Windows 10
>> using Visual Studio 2015. I'm following the INSTALL and NOTES.WIN
>> instructions however I get stopped rather quickly with file not found
>> issues..
>>
>> I have also installed nasm. The build fails for 32 or 64 with slightly
>> different paths in the error. Here's the sequence of commands:
>> 1.perl Configure VC-WIN32
>> 2.nmake
>>
>>
>> output:
>>
>> D:\repos\openssl-1.1.0c>perl Configure VC-WIN64A
>> Configuring OpenSSL version 1.1.0c (0x1010003fL)
>> no-asan [default]  OPENSSL_NO_ASAN
>> no-crypto-mdebug [default]  OPENSSL_NO_CRYPTO_MDEBUG
>> no-crypto-mdebug-backtrace [default]
>> OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE
>> no-ec_nistp_64_gcc_128 [default] OPENSSL_NO_EC_NISTP_64_GCC_128
>> no-egd  [default]  OPENSSL_NO_EGD
>> no-fuzz-afl [default]  OPENSSL_NO_FUZZ_AFL
>> no-fuzz-libfuzzer [default]  OPENSSL_NO_FUZZ_LIBFUZZER
>> no-heartbeats   [default]  OPENSSL_NO_HEARTBEATS
>> no-md2  [default]  OPENSSL_NO_MD2 (skip dir)
>> no-msan [default]  OPENSSL_NO_MSAN
>> no-rc5  [default]  OPENSSL_NO_RC5 (skip dir)
>> no-sctp [default]  OPENSSL_NO_SCTP
>> no-ssl-trace[default]  OPENSSL_NO_SSL_TRACE
>> no-ssl3 [default]  OPENSSL_NO_SSL3
>> no-ssl3-method  [default]  OPENSSL_NO_SSL3_METHOD
>> no-ubsan[default]  OPENSSL_NO_UBSAN
>> no-unit-test[default]  OPENSSL_NO_UNIT_TEST
>> no-weak-ssl-ciphers [default]  OPENSSL_NO_WEAK_SSL_CIPHERS
>> no-zlib [default]
>> no-zlib-dynamic [default]
>> Configuring for VC-WIN64A
>> CC=cl
>> CFLAG =-W3 -wd4090 -Gs0 -GF -Gy -nologo -DOPENSSL_SYS_WIN32
>> -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE -DUNICODE
>> -D_UNICODE /MD /O2
>> SHARED_CFLAG  =
>> DEFINES   =OPENSSL_USE_APPLINK DSO_WIN32 NDEBUG OPENSSL_THREADS
>> OPENSSL_NO_STATIC_ENGINE OPENSSL_PIC OPENSSL_IA32_SSE2
>> OPENSSL_BN_ASM_MONT OPENSSL_BN_ASM_MONT5 OPENSSL_BN_ASM_GF2m SHA1_ASM
>> SHA256_ASM SHA512_ASM RC4_ASM MD5_ASM AES_ASM VPAES_ASM BSAES_ASM
>> GHASH_ASM ECP_NISTZ256_ASM POLY1305_ASM
>> LFLAG =/nologo /debug
>> PLIB_LFLAG=
>> EX_LIBS   =ws2_32.lib gdi32.lib advapi32.lib crypt32.lib user32.lib
>> APPS_OBJ  =win32_init.o ../ms/applink.o
>> CPUID_OBJ =x86_64cpuid.o
>> UPLINK_OBJ=../ms/uplink.o uplink-x86_64.o
>> BN_ASM=bn_asm.o x86_64-mont.o x86_64-mont5.o x86_64-gf2m.o
>> rsaz_exp.o rsaz-x86_64.o rsaz-avx2.o
>> EC_ASM=ecp_nistz256.o ecp_nistz256-x86_64.o
>> DES_ENC   =des_enc.o fcrypt_b.o
>> AES_ENC   =aes-x86_64.o vpaes-x86_64.o bsaes-x86_64.o
>> aesni-x86_64.o aesni-sha1-x86_64.o aesni-sha256-x86_64.o aesni-mb-x

Re: [openssl-users] Build problems on Windows

[jeff saremi]

D:\repos\openssl2\openssl-1.1.0c>perl -v

This is perl 5, version 22, subversion 1 (v5.22.1) built for 
x86_64-msys-thread-multi
Copyright 1987-2015, Larry Wall




From: openssl-users  on behalf of Jakob Bohm 

Sent: Monday, January 9, 2017 9:46 PM
To: openssl-users@openssl.org
Subject: Re: [openssl-users] Build problems on Windows

On 10/01/2017 05:04, jeff saremi wrote:
>
> Hello
>
> I downloaded openssl-1.1.0c and i'm trying to build this on Windows 10
> using Visual Studio 2015. I'm following the INSTALL and NOTES.WIN
> instructions however I get stopped rather quickly with file not found
> issues..
>
> I have also installed nasm. The build fails for 32 or 64 with slightly
> different paths in the error. Here's the sequence of commands:
> 1.perl Configure VC-WIN32
> 2.nmake
>
>
> output:
>
> D:\repos\openssl-1.1.0c>perl Configure VC-WIN64A
> Configuring OpenSSL version 1.1.0c (0x1010003fL)
> no-asan [default]  OPENSSL_NO_ASAN
> no-crypto-mdebug [default]  OPENSSL_NO_CRYPTO_MDEBUG
> no-crypto-mdebug-backtrace [default]
> OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE
> no-ec_nistp_64_gcc_128 [default] OPENSSL_NO_EC_NISTP_64_GCC_128
> no-egd  [default]  OPENSSL_NO_EGD
> no-fuzz-afl [default]  OPENSSL_NO_FUZZ_AFL
> no-fuzz-libfuzzer [default]  OPENSSL_NO_FUZZ_LIBFUZZER
> no-heartbeats   [default]  OPENSSL_NO_HEARTBEATS
> no-md2  [default]  OPENSSL_NO_MD2 (skip dir)
> no-msan [default]  OPENSSL_NO_MSAN
> no-rc5  [default]  OPENSSL_NO_RC5 (skip dir)
> no-sctp [default]  OPENSSL_NO_SCTP
> no-ssl-trace[default]  OPENSSL_NO_SSL_TRACE
> no-ssl3 [default]  OPENSSL_NO_SSL3
> no-ssl3-method  [default]  OPENSSL_NO_SSL3_METHOD
> no-ubsan[default]  OPENSSL_NO_UBSAN
> no-unit-test[default]  OPENSSL_NO_UNIT_TEST
> no-weak-ssl-ciphers [default]  OPENSSL_NO_WEAK_SSL_CIPHERS
> no-zlib [default]
> no-zlib-dynamic [default]
> Configuring for VC-WIN64A
> CC=cl
> CFLAG =-W3 -wd4090 -Gs0 -GF -Gy -nologo -DOPENSSL_SYS_WIN32
> -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE -DUNICODE
> -D_UNICODE /MD /O2
> SHARED_CFLAG  =
> DEFINES   =OPENSSL_USE_APPLINK DSO_WIN32 NDEBUG OPENSSL_THREADS
> OPENSSL_NO_STATIC_ENGINE OPENSSL_PIC OPENSSL_IA32_SSE2
> OPENSSL_BN_ASM_MONT OPENSSL_BN_ASM_MONT5 OPENSSL_BN_ASM_GF2m SHA1_ASM
> SHA256_ASM SHA512_ASM RC4_ASM MD5_ASM AES_ASM VPAES_ASM BSAES_ASM
> GHASH_ASM ECP_NISTZ256_ASM POLY1305_ASM
> LFLAG =/nologo /debug
> PLIB_LFLAG=
> EX_LIBS   =ws2_32.lib gdi32.lib advapi32.lib crypt32.lib user32.lib
> APPS_OBJ  =win32_init.o ../ms/applink.o
> CPUID_OBJ =x86_64cpuid.o
> UPLINK_OBJ=../ms/uplink.o uplink-x86_64.o
> BN_ASM=bn_asm.o x86_64-mont.o x86_64-mont5.o x86_64-gf2m.o
> rsaz_exp.o rsaz-x86_64.o rsaz-avx2.o
> EC_ASM=ecp_nistz256.o ecp_nistz256-x86_64.o
> DES_ENC   =des_enc.o fcrypt_b.o
> AES_ENC   =aes-x86_64.o vpaes-x86_64.o bsaes-x86_64.o
> aesni-x86_64.o aesni-sha1-x86_64.o aesni-sha256-x86_64.o aesni-mb-x86_64.o
> BF_ENC=bf_enc.o
> CAST_ENC  =c_enc.o
> RC4_ENC   =rc4-x86_64.o rc4-md5-x86_64.o
> RC5_ENC   =rc5_enc.o
> MD5_OBJ_ASM   =md5-x86_64.o
> SHA1_OBJ_ASM  =sha1-x86_64.o sha256-x86_64.o sha512-x86_64.o
> sha1-mb-x86_64.o sha256-mb-x86_64.o
> RMD160_OBJ_ASM=
> CMLL_ENC  =cmll-x86_64.o cmll_misc.o
> MODES_OBJ =ghash-x86_64.o aesni-gcm-x86_64.o
> PADLOCK_OBJ   =e_padlock-x86_64.o
> CHACHA_ENC=chacha-x86_64.o
> POLY1305_OBJ  =poly1305-x86_64.o
> BLAKE2_OBJ=
> PROCESSOR =
> RANLIB=true
> ARFLAGS   =/nologo
> PERL  =perl
>
> SIXTY_FOUR_BIT mode
>
> Configured for VC-WIN64A.
>
> D:\repos\openssl-1.1.0c>nmake
>
> Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
> Copyright (C) Microsoft Corporation.  All rights reserved.
>
> "perl" "-I." -Mconfigdata "util/dofile.pl" "-omakefile"
> "crypto/include/internal/bn_conf.h.in" > crypto/include/internal/bn_conf.h
> "perl" "-I." -Mconfigdata "util/dofile.pl" "-omakefile"
> "crypto/include/internal/dso_conf.h.in" >
> crypto/include/internal/dso_conf.h
> "perl" "-I." -Mconfigdata "util/dofile.pl" "-omakefile"
> "include/openssl/opensslconf.h.in" > include/openssl/opensslconf.h
> set ASM=nasm
> "perl" "crypto/aes/asm/aes-x86_64.pl" "auto"
> crypto/aes/aes-x86_64.asm
>

[openssl-users] Build problems on Windows

[jeff saremi]

Hello

I downloaded openssl-1.1.0c and i'm trying to build this on Windows 10 using 
Visual Studio 2015. I'm following the INSTALL and NOTES.WIN instructions 
however I get stopped rather quickly with file not found issues..

I have also installed nasm. The build fails for 32 or 64 with slightly 
different paths in the error. Here's the sequence of commands:
1.perl Configure VC-WIN32
2.nmake


output:

D:\repos\openssl-1.1.0c>perl Configure VC-WIN64A
Configuring OpenSSL version 1.1.0c (0x1010003fL)
no-asan [default]  OPENSSL_NO_ASAN
no-crypto-mdebug [default]  OPENSSL_NO_CRYPTO_MDEBUG
no-crypto-mdebug-backtrace [default]  OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE
no-ec_nistp_64_gcc_128 [default]  OPENSSL_NO_EC_NISTP_64_GCC_128
no-egd  [default]  OPENSSL_NO_EGD
no-fuzz-afl [default]  OPENSSL_NO_FUZZ_AFL
no-fuzz-libfuzzer [default]  OPENSSL_NO_FUZZ_LIBFUZZER
no-heartbeats   [default]  OPENSSL_NO_HEARTBEATS
no-md2  [default]  OPENSSL_NO_MD2 (skip dir)
no-msan [default]  OPENSSL_NO_MSAN
no-rc5  [default]  OPENSSL_NO_RC5 (skip dir)
no-sctp [default]  OPENSSL_NO_SCTP
no-ssl-trace[default]  OPENSSL_NO_SSL_TRACE
no-ssl3 [default]  OPENSSL_NO_SSL3
no-ssl3-method  [default]  OPENSSL_NO_SSL3_METHOD
no-ubsan[default]  OPENSSL_NO_UBSAN
no-unit-test[default]  OPENSSL_NO_UNIT_TEST
no-weak-ssl-ciphers [default]  OPENSSL_NO_WEAK_SSL_CIPHERS
no-zlib [default]
no-zlib-dynamic [default]
Configuring for VC-WIN64A
CC=cl
CFLAG =-W3 -wd4090 -Gs0 -GF -Gy -nologo -DOPENSSL_SYS_WIN32 
-DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE -DUNICODE 
-D_UNICODE /MD /O2
SHARED_CFLAG  =
DEFINES   =OPENSSL_USE_APPLINK DSO_WIN32 NDEBUG OPENSSL_THREADS 
OPENSSL_NO_STATIC_ENGINE OPENSSL_PIC OPENSSL_IA32_SSE2 OPENSSL_BN_ASM_MONT 
OPENSSL_BN_ASM_MONT5 OPENSSL_BN_ASM_GF2m SHA1_ASM SHA256_ASM SHA512_ASM RC4_ASM 
MD5_ASM AES_ASM VPAES_ASM BSAES_ASM GHASH_ASM ECP_NISTZ256_ASM POLY1305_ASM
LFLAG =/nologo /debug
PLIB_LFLAG=
EX_LIBS   =ws2_32.lib gdi32.lib advapi32.lib crypt32.lib user32.lib
APPS_OBJ  =win32_init.o ../ms/applink.o
CPUID_OBJ =x86_64cpuid.o
UPLINK_OBJ=../ms/uplink.o uplink-x86_64.o
BN_ASM=bn_asm.o x86_64-mont.o x86_64-mont5.o x86_64-gf2m.o rsaz_exp.o 
rsaz-x86_64.o rsaz-avx2.o
EC_ASM=ecp_nistz256.o ecp_nistz256-x86_64.o
DES_ENC   =des_enc.o fcrypt_b.o
AES_ENC   =aes-x86_64.o vpaes-x86_64.o bsaes-x86_64.o aesni-x86_64.o 
aesni-sha1-x86_64.o aesni-sha256-x86_64.o aesni-mb-x86_64.o
BF_ENC=bf_enc.o
CAST_ENC  =c_enc.o
RC4_ENC   =rc4-x86_64.o rc4-md5-x86_64.o
RC5_ENC   =rc5_enc.o
MD5_OBJ_ASM   =md5-x86_64.o
SHA1_OBJ_ASM  =sha1-x86_64.o sha256-x86_64.o sha512-x86_64.o sha1-mb-x86_64.o 
sha256-mb-x86_64.o
RMD160_OBJ_ASM=
CMLL_ENC  =cmll-x86_64.o cmll_misc.o
MODES_OBJ =ghash-x86_64.o aesni-gcm-x86_64.o
PADLOCK_OBJ   =e_padlock-x86_64.o
CHACHA_ENC=chacha-x86_64.o
POLY1305_OBJ  =poly1305-x86_64.o
BLAKE2_OBJ=
PROCESSOR =
RANLIB=true
ARFLAGS   =/nologo
PERL  =perl

SIXTY_FOUR_BIT mode

Configured for VC-WIN64A.

D:\repos\openssl-1.1.0c>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

"perl" "-I." -Mconfigdata "util/dofile.pl"  "-omakefile" 
"crypto/include/internal/bn_conf.h.in" > crypto/include/internal/bn_conf.h
"perl" "-I." -Mconfigdata "util/dofile.pl"  "-omakefile" 
"crypto/include/internal/dso_conf.h.in" > crypto/include/internal/dso_conf.h
"perl" "-I." -Mconfigdata "util/dofile.pl"  "-omakefile" 
"include/openssl/opensslconf.h.in" > include/openssl/opensslconf.h
set ASM=nasm
"perl" "crypto/aes/asm/aes-x86_64.pl" "auto" crypto/aes/aes-x86_64.asm

nasm -f win64 -DNEAR -Ox -g -ocrypto/aes/aes-x86_64.obj 
"crypto/aes/aes-x86_64.asm"
nasm: fatal: unable to open input file `crypto/aes/aes-x86_64.asm'
NMAKE : fatal error U1077: 'C:\nasm-2.12.02\nasm.EXE' : return code '0x1'
Stop.



-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: Help Needed: SSL Connect starting from a weird state

[Jeff Saremi]

My initial analysis of this was very misleading. I have to apologize for
that.
The problem was that during the first part of the handshake
(clienthello), the call failed without anything being written out.
Tracing ssl23_client_hello() in s23_clnt.c showed that the following
statement returned false and as a result -1 was returned as the error.
if (RAND_pseudo_bytes(...) <=0)
  return -1;

And for any instances of error for which an internal OpenSSL ERR is not
set, SSL_ERROR_SYSCALL is used, which is further misleading.

I did a cursory search of anywhere that a call to RAND_pseudo_bytes can
fail and there are tens of such instances for which OpenSSL ERR is not
set. In fact, there's only one instance of a call to RANDerr which is
inside md_rand.c. I guess this would be something for OpenSSL guys to
ponder.

Another strange thing is no matter how many times we ran the
application, the call always failed on the same spot; the same call to
RAND_pseudo_byes each time, not before or after. This was regardless of
how many successful calls were made prior to.

Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Help Needed: SSL Connect starting from a weird state

[Jeff Saremi]

We've been running our SSL code for a while now with no issues. But
recently one of our developers started encountering this problem.
We did the best we could to troubleshoot to no avail. I know the 
problem is not OpenSSL and it's something we're doing incorrectly,
probably at the start up.

The problem:
SSL completed without having done a single send or receive during the
handshake.

What we get in the print out, after issuing SSL_connect() is this:

Printout:
18:13:56.925 [4228] connect
18:13:56.927 [4228] SSL nonblock rc:-1 shutdown:0 state:23WCHA
(from:UNKWN )
18:13:56.928 [4228] ssl_err:5 SSL_ERROR_SYSCALL

The rough version of the code printing the above is this:
printf("connect\n");
const char *fromState = SSL_state_string(mSsl);
rc = SSL_connect(mSsl);
printf("SSL nonblock rc:%d shutdown:%d state:%s (from:%s)\n",
rc,
SSL_get_shutdown(mSsl),
SSL_state_string(mSsl),
fromState);
int ssl_error = SSL_get_error(mSsl, rc);
switch(ssl_error)
{
case SSL_ERROR_SYSCALL:
  printf("%d SSL_ERROR_SYSCALL\n", SSL_ERROR_SYSCALL);
...


What I would expect to see would be something along the lines of the
following:

SSL nonblock rc:1 shutdown:0 state:SSLOK (from:UNKWN )

or
SSL nonblock rc:-1 shutdown:0 state:SSLOK (from:SSLOK )


For additional debugging I have enabled callbacks using the following
too:
SSL_set_msg_callback

And I see a lot of that happening but not in this case.
In this particular case, after switching the destination IP and port all
we get is what I showed you. Not even one single byte is exchanged
anywhere.

Looking inside ssl_stat.c I see the following:
case SSL23_ST_CW_CLNT_HELLO_A:  str="23WCHA"; break;

Looking inside s23_clnt.c I see these lines near the beginning of
ssl23_client_hello():

buf=(unsigned char *)s->init_buf->data;
if (s->state == SSL23_ST_CW_CLNT_HELLO_A)

How can my code start in this state?

Any hints would be appreciated.
thanks
jeff

Re: Running SSL on own socket code

[Jeff Saremi]

Yes it was as straightforward as you mentioned.
Thanks to those who responded.
jeff

On 11-06-02 05:47 AM, Neo Liu wrote:
> 
> 
> On Wed, Jun 1, 2011 at 10:22 PM, Victor Duchovni
>  <mailto:victor.ducho...@morganstanley.com>> wrote:
> 
> On Tue, May 31, 2011 at 09:05:29AM -0400, Jeff Saremi wrote:
> 
> > I'd like to know the feasibility or complexity around using my own
> > socket code with OpenSSL's ssl code. If I provide OpenSSL with a
> pair of
> > BIOs to read and write would that be sufficient? How tightly
> integrated
> > the code is with bio_connect and bio_socket? thanks
> > jeff
> 
> man BIO_new_bio_pair
> 
> Look at the example.
> 
> 
> As Victor pointed that you can use BIO pair to handle data transmission
> and separate data transmission procedure from SSL handshake and
> encrypt/decrypt procedure.
> Using BIO pair, the SSL structure only handle the data in SSL Record
> layer but not any lower layer.
> 
> --
>Viktor.
> __
> OpenSSL Project
> http://www.openssl.org
> 
> <http://console.mxlogic.com/redir/?nS6mkQm4PrapEVssoo76M0llLxix_00jt-psLuZXTLsTsTkN3UBWHv2B3OXtfzg5lm-5a7Y01dKc9zzqbdNRXL8Kf6Sm3obZ8Qg0JK49g8Cy0o60Gq8aMd45j43h1iJyLPd45GEurDUvf5zZB0SyrhdIFCzBNNdNwSXjFrmHp>
> User Support Mailing List  
>  openssl-users@openssl.org <mailto:openssl-users@openssl.org>
> Automated List Manager  
> majord...@openssl.org <mailto:majord...@openssl.org>
> 
> 
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Running SSL on own socket code

[Jeff Saremi]

I'd like to know the feasibility or complexity around using my own
socket code with OpenSSL's ssl code. If I provide OpenSSL with a pair of
BIOs to read and write would that be sufficient? How tightly integrated
the code is with bio_connect and bio_socket? thanks
jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Valgrind Suppressed Leak on dlfcn_globallookup

[Jeff Saremi]

I'd like to eliminate this suppressed error in valgrind.
Does anyone have any idea? I have my cleanup code also pasted father below

==3317== 20 bytes in 1 blocks are still reachable in loss record 1 of 1
==3317==at 0x402425F: calloc (vg_replace_malloc.c:467)
==3317==by 0x4063105: _dlerror_run (dlerror.c:142)
==3317==by 0x4062B40: dlopen@@GLIBC_2.1 (dlopen.c:88)
==3317==by 0x82A1A99: dlfcn_globallookup (dso_dlfcn.c:472)


ENGINE_cleanup();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_free_strings();
OBJ_NAME_cleanup(-1);
CRYPTO_cleanup();

ERR_remove_thread_state(NULL);

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Possibility to create CRL without the CA key

[Jeff Saremi]

read my post:
http://www.mail-archive.com/openssl-users@openssl.org/msg63740.html


On 11-05-02 06:50 AM, Viliam Ďurina wrote:
> Hello,
>
> I'm doing my own CA with openssl and want to regularly generate CRLs.
> We plan limited use of the CA (say 1-2 certificates per year), so the
> CA private key is stored in a safe on a USB stick until it is used
> next time. But, as far as I know, we will need it to generate CRL
> quite often. I see two possible solutions:
>
> 1. be able to sign the CRL with another key, signed with that CA: is
> this possible?
>
> 2. generate the CRL with very long validity (say 1 year) and
> regenerate a new one when needed: isn't this breaking some PKI rules
> or common practices?
>
> Thanks,
> Viliam
>
> __
> OpenSSL Project http://www.openssl.org
> User Support Mailing Listopenssl-users@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: CRL validation must be skipped for certs with no CRLDPs

[Jeff Saremi]

If I just try to describe the problem in a different way it would be:

According to the RFC, is it an error for a certificate and its chain not
to have any CRLs and CRL distribtuion points?

If the answer to the above is yes, then what OpenSSL does is OK because
the programmer would have to explicitly by way of writing callbacks or
by way of disabling CRLs altogether handle the situation.

If the answer is no, then the body of code that is operating today on
the Internet and is using SSL (think of it as browsers) should operate
with no errors. In this sense, OpenSSL would be an exception because its
default and natural implementation causes an error.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: CRL validation must be skipped for certs with no CRLDPs

[Jeff Saremi]

I'm not sure how you read this. I read it like a programmer.
In programming primitives, the spec would be coded like this:

Here's the spec:
"This algorithm begins by assuming that the certificate is not revoked
For each distribution point (DP) in the certificate's CRL distribution
points extension, for each corresponding CRL "

And here's the C code:

cert.status = UNREVOKED;
for (i = 0; i < cert.crldp.size; ++i)
{
  /* more processing loops */
}

Right there, the main loop should not even execute because
cert.crldp.size is zero.
But let's play along and say that for some unknown reason we get to the
section of the spec you're referring to:

Here's the spec:

> I think you should read on.  Specifically, the last paragraph seems to
> me to indicate different behaviour is intended:
> 
>If the revocation status has not been determined, repeat the process
>above with any available CRLs not specified in a distribution point
>but issued by the certificate issuer.  

Here's the code:

cert.status = NOT_REVOKED;
for (i = 0; i < cert.crldp.size; ++i)
{
  /* some processing loops */
  if(cert.status == UNDETERMINED)
  {
/* do what Bruce Stephens suggested */
  }
}

As you can see again there's another stop and that stop is that our
certificate's status is not "UNDETERMINED" but rather "UNREVOKED".

But lets view this whole thing from another perspective:
I have a limited certificate authority in my organization. have a root
cert which does not get revoked. I distribute that to the ones who want
to deal with me. I also generate a multitude of endpoint certificates. I
do not intend for any of these to become revoked, ever. Hence, I do not
bother with the process of certificate revocation and CRL issuance. Now
as a result of OpenSSL's imlementation, none of my clients who use
OpenSSL will be able to deal with me online. Because my certificates do
not have a CRL and CRLDP period. Should I stop doing business
altogether? Should I contact my business partners and tell them that
they should be applying a patch to their code, overriding verification
failure in their certVerifyCallbacks? Let's just think that through a
little.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

CRL validation must be skipped for certs with no CRLDPs

[Jeff Saremi]

This was mentioned briefly in another thread by myself. I thought it
deserved its own discussion. The copies of how it went on are pasted below.


Section 6.3.3. of RFC 5280 - CRL Processing
"This algorithm begins by assuming that the certificate is not revoked
For each distribution point (DP) in the certificate's CRL distribution
points extension, for each corresponding CRL "

So my expectation is that after I instruct OpenSSL to perform CRL
checking -- whether I set or not set any CRLs -- no checking must be
done on any certificates which don't have CRLDP in them.

The spec ofcourse mentions that implementations are not required to
follow this algorithm, however, it also mentions that the end result
must be the same as when they did.

Now as an average user of OpenSSL API, I expect not to have to write any
piece of code (such as over-ruling the validation failure in validation
callback) for the normal process of certificate/CRL validation to take
its course. Is this a reasonable expectation?

thanks
Jeff


* Original Problem Statement  
Re: Need Help with Programmatic Downloading+Checking of CRLs
...

> So as per previous posts, I implemented lookup_crl().
> Now one of the major problems is what do I return from this method, if
> the certificate has no CRL distribution points!
> Returning an empty stack causes get_crl_delta() to fail.
> Is there a flag that I can setup to let this cert be excluded from CRL
> checking?
> Is that something I should be doing in lookup_crl? Or should the
> framework be smart enough not to even ask me for a CRL in this case?
> 
> thanks
> jeff


> There are other "out of band" mechanisms where a CRL might be available but
> not mentioned in a CRLDP. OpenSSL has no way of telling what those might be
> and if the absence is really an error or not.
> 
> The best you can do is trap the issuer error in the verify callback and ignore
> it if appropriate.
> 
> Steve.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Handling Indirect CRL Issuer

[Jeff Saremi]

With great many thanks to Dr. Henson for not only responding to every
post I have had so far but also for providing solid guidance on how to
address the problem leading to the heading of this thread, I am adding
some extra material and some verbatim quotes from Dr. Henson here so
that they might be of some benefit to some one.

> What you do is to generate a CRL issuing certificate which has exactly the
> same subject, issuer name as the CA. In that certificate include a keyUsage
> extension with only the crl signing bit asserted. If you had the option the CA
> certificate should have keyUsage and certsign but NOT crl sign set.
> 
> You then issue CRLs from this second certificate which will have a different
> AKID. Then OpenSSL will use that certificate (if it can find it) instead of
> the CA certificate.

To create a second certificate duplicating an existing one completely
you may face some challenges, but the following two commands should
help. Also you may want to possibly use "-preserveDN" command line
option as well as setting "preserve=yes" in your config file:

> openssl x509 -in cert.pem -signkey newkey.pem -out newcert.pem
> 
> This should convert an existing certificate into a self signed version with a
> new key. From there you can convert it into a certificate request with:
> 
> openssl x509 -in newcert.pem -x509toreq -signkey newkey.pem

Also make sure you create those AuthorityKeyIdentifiers in your
certs/crls by having lines like the following in appropriate places in
your config:
authorityKeyIdentifier=keyid,issuer:always

Finally, let's assume you are in possession of your 2nd certificate (or
the Indirect CRL Issuer's certificate). For this to be processed
properly you would need to add it to X509_STORE_CTX as an "untrusted"
cert. Setting it along with your trust chain certs won't work.

To do that and since I had to do this in the context of an SSL
connection, I decided to use a callback like the following:

a) create an "app verify cert" callback:
int appVerifyCallback(X509_STORE_CTX *ctx,void *arg)
{
STACK_OF(X509*) untrustedStack = sk_X509_new_null();
// add your "untrusted" certs such as the 2nd CA cert
// or your Indirect CRL Issuer to the stack
X509 *cert = ...
sk_X509_push(untrustedStack, cert);

// this call sets the ctx->untrusted
X509_STORE_CTX_set_chain(ctx, untrustedStack);
return 1;
}

b) add this to your SSL context:
SSL_CTX_set_cert_verify_callback(mCtx,
&appVerifyCallback,
(void*)untrustedCerts);

And you should be all set to validate those certs and CRLs.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Need Help with Programmatic Downloading+Checking of CRLs

[Jeff Saremi]

> There are other "out of band" mechanisms where a CRL might be available but
> not mentioned in a CRLDP. OpenSSL has no way of telling what those might be
> and if the absence is really an error or not.
> 
> The best you can do is trap the issuer error in the verify callback and ignore
> it if appropriate.
> 
> Steve.
> --
> Dr Stephen N. Henson. OpenSSL project core developer.
> Commercial tech support now available see: http://www.openssl.org

Is introducing a mechanism to lookup_crl to allow this bypass unreasonable?

Or introducing an override method (like the pointers in X509_STORE) to
check if the code should ask for a CRL or not? Such as the following:

static int check_revocation(X509_STORE_CTX *ctx)
{
int i, last, ok;
if (!(ctx->param->flags & X509_V_FLAG_CRL_CHECK))
return 1;
if (ctx->param->flags & X509_V_FLAG_CRL_CHECK_ALL)
last = sk_X509_num(ctx->chain) - 1;
...

for(i = 0; i <= last; i++)
{
ctx->error_depth = i;
if(ctx->crl_check_required(i))
{
ok = check_cert(ctx);
if (!ok) return ok;
}
}
return 1;
}

crl_check_required(int i) in the default (most simplistic case) could
just do a check on CRLDP in the certificate. And if as you said there
are other CRLs that could have been provided, then one could overwrite it.


Here's another suggestion:

There's a lot of logic and functionality in the following methods:
check_cert()
and
get_crl_delta()

If these could be overwrite-able in X509_STORE would be great.
Especially get_crl_delta.
If get_crl defaulted to get_crl_delta that solution would still be
workable as well..
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

How can I make CertificateIssuer extension show up in CRL Entries?

[Jeff Saremi]

I'm generating my CRLs using "openssl ca" command.  In the CRL entry
extension list, I can see "X509v3 CRL Reason Code" but I'd like to also
include the certificate issuer. (I think this is needed if an issuer
does the issuing indirectly for another CA).

Is there a command line option of config entry for that?
thanks
jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Handling Indirect CRL Issuer

[Jeff Saremi]

It looks like we need to support indirect CRL Issuers at least for CRL's
issued for ourselves.

I have done most of the work. It looks I don't quite know how to
generate CRLs with the indirect CRL issuer or I don't know how to
generate the CRL issuer's certificate using the root certificate.

So I have added the CRL issuer's cert to the trusted ones. But when I'm
trying to use the CRL i get stopped here:


crl_akid_check()
{
...
  if(X509_check_akid()
// this is where if fails


and inside X509_check_akid()

...
/* Check key ids (if present) */
if(akid->keyid && issuer->skid &&
 ASN1_OCTET_STRING_cmp(akid->keyid, issuer->skid) )
return X509_V_ERR_AKID_SKID_MISMATCH;


There's definitely something I don't know about AKID's and how to set
them properly.

To help you out here are the certificates and CRLs (i have masked some
fields):

*** Our ROOT cert *
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 0 (0x0)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=TestMoregaRootCA, C=CA, O=TestMorega
Validity
Not Before: Jun  8 00:29:30 2010 GMT
Not After : Jun  3 00:29:30 2030 GMT
Subject: CN=TestMoregaRootCA, C=CA, O=TestMorega
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
...
6c:68:70:a5:c1:7e:5e:b8:e4:82:ff:6d:c6:3
X509v3 extensions:
X509v3 Subject Key Identifier:
70:8F:22:BC:D7:55:20:6E:00:D7:3A:D3:70:40:F5:49:91:20:90:60
X509v3 Authority Key Identifier:

keyid:70:8F:22:BC:D7:55:20:6E:00:D7:3A:D3:70:40:F5:49:91:20:90:60
DirName:/CN=TestMoregaRootCA/C=CA/O=TestMorega
serial:00
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Basic Constraints: critical
CA:TRUE



 CRL Issuer Cert issued by the root 
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 20 (0x14)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=TestMoregaRootCA, C=CA, O=TestMorega
Validity
Not Before: Mar 16 18:31:26 2011 GMT
Not After : Mar 11 18:31:26 2031 GMT
Subject: C=CA, O=TestMorega, CN=TestMoregaCRLIssuer
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (1024 bit)
Modulus:
...
1c:52:ce:81:2c:50:52:30:43
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
7F:AC:68:90:EE:3C:8B:7B:6D:0E:A0:71:68:BE:57:D0:45:42:E9:C6
X509v3 Authority Key Identifier:

keyid:70:8F:22:BC:D7:55:20:6E:00:D7:3A:D3:70:40:F5:49:91:20:90:60
DirName:/CN=TestMoregaRootCA/C=CA/O=TestMorega
serial:00
X509v3 Key Usage: critical
Digital Signature, CRL Sign


 A sample CRL issued by the Indirect CRL Issuer 
Certificate Revocation List (CRL):
Version 2 (0x1)
Signature Algorithm: sha256WithRSAEncryption
Issuer: /C=CA/O=TestMorega/CN=TestMoregaCRLIssuer
Last Update: Mar 17 12:56:55 2011 GMT
Next Update: Apr 16 12:56:55 2011 GMT
CRL extensions:
X509v3 Authority Key Identifier:

keyid:7F:AC:68:90:EE:3C:8B:7B:6D:0E:A0:71:68:BE:57:D0:45:42:E9:C6
DirName:/CN=TestMoregaRootCA/C=CA/O=TestMorega
serial:14
X509v3 Issuing Distrubution Point: critical
Full Name:
  URI:http://localhost/
Indirect CRL
Authority Information Access:
CA Issuers - URI:http://localhost/crlissuer.cer
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Handling Indirect CRL Issuer

[Jeff Saremi]

Does anyone have an example of how an indirect CRL issuer is handled?
This is my understanding of needs to be done.
If at least someone could verify that, I'd be really appreciative:

1. download the CRL
2. If not indirect, handle as usual (let's pretend for now that we know
how to handle these in OpenSSL)
3. If Indirect flag is set, check Authority Information Access.
(possibly using something like:
AUTHORITY_INFO_ACCESS *info = (AUTHORITY_INFO_ACCESS*)
X509_CRL_get_ext_d2i(crl, NID_info_access, NULL, NULL);)
4. Download the issuer's certificate using the URL above.
5. Add the cert to the store? (using X509_STORE_add_cert()?)

Any other steps?

thanks
jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Need Help with Programmatic Downloading+Checking of CRLs

[Jeff Saremi]

So as per previous posts, I implemented lookup_crl().
Now one of the major problems is what do I return from this method, if
the certificate has no CRL distribution points!
Returning an empty stack causes get_crl_delta() to fail.
Is there a flag that I can setup to let this cert be excluded from CRL
checking?
Is that something I should be doing in lookup_crl? Or should the
framework be smart enough not to even ask me for a CRL in this case?

thanks
jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

what does X509_STORE_get1_crls() return and how?

[Jeff Saremi]

If I call X509_STORE_get1_crls(ctx, nm) with nm being the issuer name,
the method is supposed to return a list of CRL's with that issuer name.
How does it do that when it comes to CRLs issued by a CRL issuer
authorized by the original issuer?
Does it use Authority Key Identifier?

thanks,
jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Need Help with Programmatic Downloading+Checking of CRLs

[Jeff Saremi]

> Try supplying your own lookup_crls() implementation instead. This can be much
> simpler and just needs to return any CRLs which match the supplied X509_NAME
> value. If there are multiple CRLs it will pick the most appopriate.
> 
> Steve.
> --
> Dr Stephen N. Henson. OpenSSL project core developer.
> Commercial tech support now available see: http://www.openssl.org

That's a lot better. Thanks

I'd like to describe the steps I'm taking in my lookup function. If
someone could verify them I would be grateful:

1. try to get crls by calling X509_STORE_get1_crls with passed arguments.
2. If the stack is not empty then return it to the caller
3. Otherwise, get the current certificate's CRL distribution point by
using X509_STORE_CTX_get_current_cert() and a bunch of other lines I
pasted in my other email
4. Get the CRL by fetching the URL
5. Now do another call to the store based on the name of the CRL issuer:
X509_STORE_get1_crls(ctx, X509_CRL_get_issuer(crl))
6. If this call return any CRLs that means the CRL existed in the store
but the CRL Issuer was different than the certificate Issuer. (if I
understood the spec correctly, this is quite possible)
7. If no CRLs were found, then add the downloaded CRL: X509_STORE_add_crl()
8. Do another call to X509_STORE_get1_crls() and return the results
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Need Help with Programmatic Downloading+Checking of CRLs

[Jeff Saremi]

Thanks Patrick. Unfortunately this has be a part of our code to run on
various platforms. If you know of any openssl-based implementation that
does this and is opensource please let me know so at least I could use
that as an example.
> 
> Patrick Patterson
> Tue, 15 Mar 2011 13:11:11 -0700
> 
> Hi Jeff:
> 
> If you are looking for a solution that not only handles CRL but OCSP as well, 
> you might want to check out Pathfinder:
> 
> http://www.carillon.ca/tools/pathfinder.php
> 
> It allows you to easily add a custom callback to the _verify() routines that 
> will enable all of this. It also does caching and complex path resolution, in 
> case that is interesting to you.
> 
> Best Regards,
> 
> Patrick.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Need Help with Programmatic Downloading+Checking of CRLs

[Jeff Saremi]

Here are some more interesting points that I'm banging my head against
the wall until I find the answer:

- Overwriting get_crl forces you to provide your own CRL checking logic
as well. Specifically two things need to be set during this check which
are required witin check_cert():
a) score of this CRL
b) current_reason

For now -- just to get going to the next step -- I had to come up with
the following kludge in my get_crl():
#define CRL_SCORE_VALID (/*CRL_SCORE_NOCRITICAL*/
0x100|/*CRL_SCORE_TIME*/
0x040|/*CRL_SCORE_SCOPE*/0x080|/*CRL_SCORE_SAME_PATH*/0x008)

  ctx->current_crl_score |= CRL_SCORE_VALID;
  ctx->current_reasons = 0x807f; // CRLDP_ALL_REASONS;

- It isn't clear if I need to add my CRL -- after being downloaded -- to
the X509_STORE within get_crl(). For now I have added it similar to the
following:

  STACK_OF(X509_CRL)* crl_sk = X509_STORE_get1_crls(ctx,
  X509_CRL_get_issuer(*crl));
  if(!sk_X509_CRL_num(crl_sk))
  {
if ((rc = X509_STORE_add_crl(ctx->ctx, *crl)) <= 0)
break;
  }
  sk_X509_CRL_free(crl_sk);


- It seems that at the end of the chain, when usually there is a
self-signing certificate, one will end up with two of the same calls to
get_crl() with the same issuer. There is a possibility that one could
check X509_STORE and fill the passed parameter X509_CRL** with that.

- For certificates which do not have a CRL distribution point, what is
one supposed to do? Return 0 or 1? What should the X509_CRL** be filled
with?

thanks
Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Need Help with Programmatic Downloading+Checking of CRLs

[Jeff Saremi]

I seriously need help with this piece. I searched the forum and I could
not find what i was looking for.
During an SSL handshake, I need to be able to examine the CRL
distribution points on a certificate (chain), download them, and pass
them along to OpenSSL for further revocation checks.
I thought I understood that the problem would be solved by just
overwriting "get_crl" method of X509_STORE. But it looks like there's a
lot more going to that just to read a URL and download the target.
Here's what I did and after all it didn't work. Because I got the error
"Different CRL scope" after return from my "get_crl"

 set up X509_STORE just before SSL connection:
// do trusted certs
if (crlDownloader) {
  x509Store->get_crl = &getCrl;
  CRYPTO_set_ex_data(&x509Store->ex_data,
  CRL_GET_EXDATA_INDEX, (void *)crlDownloader);
  X509_STORE_set_flags(x509Store,
  X509_V_FLAG_CRL_CHECK |  X509_V_FLAG_CRL_CHECK_ALL);
}
SSL_CTX_set_cert_store(ctx, x509Store);


 My get_crl:
int getCrl(X509_STORE_CTX *ctx, X509_CRL **crl, X509 *cert)
{
  CrlDownloader *crlDownloader = (CrlDownloader*)CRYPTO_get_ex_data(
&ctx->ctx->ex_data,
CRL_GET_EXDATA_INDEX);
  try
  {
// read the distribution points from the certificate (see below)
for(/* each url in distribution URLs */)
{
  X509_CRL* downloadedCrl = crlDownloader->download(url);
  *crl = downloadedCrl;
  X509_STORE_add_crl(ctx->ctx, *crl));
  break;
}
return 1;
  }
  catch(std::exception &e)
  {
// print error
  }
  return 0;
}

* util code to return a distribution URl minimally
for (i = 0; i < sk_DIST_POINT_num(cert->crldp); i++)
{
  DIST_POINT *dp = sk_DIST_POINT_value(cert->crldp, i);
  for (i = 0; i < sk_GENERAL_NAME_num(dp->distpoint->name.fullname); i++)
  {
GENERAL_NAME *gen = sk_GENERAL_NAME_value(
dp->distpoint->name.fullname, i);
if (gen->type == GEN_URI)
{
  list.push_back(std::string((char*)gen->d.ia5->data));
}
  }
}

Note that if I modify my code, download the CRL at the beginning, add it
to X509_STORE and continue, then everything works as expected. But
that's not acceptable because I would have to know apriori what those
CRL distribution points would be.

So as i mentioned the above still does not work.
Looking at get_crl_delta() shows that there's a lot more logic involved
than just downloading the CRL. Unfortunately, most if not all of the
methods called inside that module are static so they're not available to
my get_crl.

I'd appreciate your feedback and guidance.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Cert Verification based on CRL when least expected

[Jeff Saremi]

We're trying to enable CRL checking on our client-side application. When
doing limited unit tests, everything is OK. But when we extend the tests
to real server we see errors from places where least expected.

The error is "unable to get certificate CRL depth:0". The scenario is a
mutually authenticated SSL connection with the presence of 2 CRL records
on the client side.

None of the CRL records on the client side, have anything at all to do
with the server we're trying to handshake with. And that is the puzzling
fact. The server's certificate is pure and is genuinely signed by
VeriSign. The CRLs are related to two internal certs we issues by our
self-signed root. (which are at the time of test completely out of the
picture).

The SSL error is always this:
SSL error 'certificate verify failed' (file:.\ssl\s3_clnt.c line:1056
data:'')
That's the one stop that I think all errors are caught.

Now i need guidance on the best way to troubleshoot this. Is there a
debug flag or print flag I can turn on during the certificate validation
to see all details? I cannot use the "verify" tool of course.

thanks
Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

RE: SSL_VERIFY_PEER and the presence of client's X509 certificate after the handshake

[Jeff Saremi]

So this is some minor debugging I did to get to this problem.
Modified the following methods to add two printf lines:

ssl_lib.c:
X509 *SSL_get_peer_certificate(const SSL *s)
{
X509 *r;

if ((s == NULL) || (s->session == NULL))
{
printf("SSL_get_peer_certificate: s or session was null. returning
null x509\n");
r=NULL;
}
else
{
printf("SSL_get_peer_certificate: returning session->peer:
%p\n", s->session->peer);
r=s->session->peer;
}
...

d1-srvr.c:
int dtls1_accept(SSL *s)
{
...
case SSL3_ST_SW_CERT_REQ_A:
case SSL3_ST_SW_CERT_REQ_B:
if (/* don't request cert unless asked for it: */
...
else
{
s->s3->tmp.cert_request=1;
dtls1_start_timer(s);
ret=dtls1_send_certificate_request(s);
printf("ssl_accept: sent cert request; rc=%d\n", ret);
...

Recompiled and re-linked. When a call comes from the client I see the
following two lines:

ssl_accept: sent cert request; rc=1
...
SSL_get_peer_certificate: returning session->peer: (nil)


The same client and the same server, moments later, have the following
printouts:

...
ssl_accept: sent cert request; rc=1
...
SSL_get_peer_certificate: returning session->peer: 0xd74258

I cannot be 100% sure what changes on the client or on the server in
between. But the low-level client socket and ssl connections are exactly
the same in both scenarios. Also both use OpenSSL 1.0a
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

SSL_VERIFY_PEER and the presence of client's X509 certificate after the handshake

[Jeff Saremi]

We have some intermittent problems which seem to go away after
restarting our server.
The problem is that the client's certificate disappears on the server,
even though SSL_VERIFY_PEER is set in the context using
SSL_CTX_set_verify().

So under situations that are not entirely clear, a call to
SSL_get_peer_certificate() returns null after a successful SSL accept is
done on the server.

My question is if there are conditions under which one cannot rely on
the presence of the peer certificate even if SSL_VERIFY_PEER is set?

thanks
Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Enhancement Request: 64bit BIO API

[Jeff Saremi]

In another hi-jacked thread I think it was determined that the current
OpenSSL BIO API does not support 64bit operations. This post is to
request that such support be added to OpenSSL.

With the 32bit operating systems disappearing fast on desktops and
servers, with the ever-increasing involvement of cryptography in media
technologies (DRM), and insatiable appetite to stream larger and
higher-def media files, the need for 64bit BIO is felt ever than before.

>From the perspective of a user of such API, I don't have a preference on
how it's done:

- as a set of separate API above and beyond what is there now -acts like
32 under 32bits (add 'l' or '64' prefixes or suffixes at will)
- as a transparent approach: in 64bit compilations you get 64bit
versions otherwise 32
- as a complementary set always available regardless of the underlying
addressing capability

Thanks,
Jeff Saremi
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

RE: Getting detailed ssl-handshake debug output

[Jeff Saremi]

 I received the answer in a direct email and just wanted to share it
here too:

- command line:

openssl s_client -state -debug -connect host:port

- programmatic:

One can look in the callback mechanism used in the above, specifically in 
apps/s_cb.c
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Getting detailed ssl-handshake debug output

[Jeff Saremi]

 I'd like to know if there's a way -- programmatic, config, environment
-- that I can get detailed print of what goes on during a handshake at
the client or the server? Below is the output from Apache Tomcat as an
example of the level of details i'm looking for:

http-442-1, READ: TLSv1 Handshake, length = 73
*** ClientHello, TLSv1
RandomCookie:  GMT: 1269551866 bytes = { 178, 23, 135, 211, 154, 110,
144, 59, 9
9, 139, 224, 45, 156, 231, 232, 123, 36, 95, 187, 165, 56, 121, 211, 63,
117, 43
, 7, 82 }
Session ID:  {}
Cipher Suites: [TLS_DHE_RSA_WITH_AES_256_CBC_SHA,
TLS_DHE_DSS_WITH_AES_256_CBC_S
HA, TLS_RSA_WITH_AES_256_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA,
SSL_DHE_DSS
_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA,
TLS_DHE_RSA_WITH_AES_128_
CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA,
SSL_RSA
_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, Unknown 0x0:0xff]
Compression Methods:  { 0 }
Unsupported extension type_35, data:
***
%% Created:  [Session-1, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA]
*** ServerHello, TLSv1
RandomCookie:  GMT: 1269551766 bytes = { 32, 121, 10, 209, 123, 137,
160, 183, 1
86, 107, 255, 108, 79, 16, 190, 91, 180, 86, 18, 136, 232, 108, 249,
191, 90, 17
6, 87, 231 }
Session ID:  {76, 172, 211, 150, 251, 114, 230, 220, 75, 218, 174, 105,
134, 185
, 144, 119, 92, 182, 1, 58, 247, 172, 121, 90, 212, 100, 58, 220, 93,
76, 97, 11
1}
Cipher Suite: SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
Compression Method: 0
***
Cipher suite:  SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
*** Certificate chain
chain [0] = [
[
  Version: V1
  Subject: OU=Tomcat, O=ACME, emailaddress=tom...@acme.com, C=CA,
CN=localhost
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5
 
  Key:  Sun RSA public key, 512 bits
  modulus:
973285574783538290665814718553460486271776249697428968977460338357983
...
  public exponent: 65537
  Validity: [From: Mon Jun 21 14:33:25 EDT 2010,
   To: Tue Jun 21 14:33:25 EDT 2011]
  Issuer: OU=Root CA, O=ACME Systems Inc., L=TEST PURPOSES ONLY, C=CA,
CN=ACME Systems Root CA
  SerialNumber: [02]
 
]
  Algorithm: [SHA1withRSA]
  Signature:
: A5 A9 E6 5F BE 51 75 E5   E3 25 9D 92 AB 45 FA 1E  ..._.Qu..%...E..
...
 
]
***
*** Diffie-Hellman ServerKeyExchange
DH Modulus:  { 233, 230, 66, 89, 157, 53, 95, 55, 201, 127, 253, 53,
103, 18, 11
... }
DH Base:  { 48, 71, 10, 213, 160, 5, 251, 20, 206, 45, 157, 205, 135,
227, 139,
... }
Server DH Public Key:  { 159, 193, 69, 114, 138, 167, 128, 50, 5, 51,
77, 127, 2
...}
Signed with a DSA or RSA public key
*** CertificateRequest
Cert Types: RSA, DSS
Cert Authorities:



*** ServerHelloDone
http-442-1, WRITE: TLSv1 Handshake, length = 1544
http-442-1, READ: TLSv1 Handshake, length = 3309
*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=fd0172c2-3f02-432e-8317-097b8fabff7d, OU=Windows/1.00,
O=instance
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
 
  Key:  Sun RSA public key, 1024 bits
  modulus:
128531339772544414974300233324968135333513753311766363920169114394683
...
 
  public exponent: 65537
  Validity: [From: Tue Oct 05 17:49:02 EDT 2010,
   To: Wed Oct 05 17:49:02 EDT 2011]
  Issuer: CN=TESTActivationCA, OU=Activation CA, O=TEST ACTIVATION
  SerialNumber: [012b7e5e 79df]

[2]: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  CN=guest, OU=ACME PC Client, O=instance
]
 
[3]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
  Key_Agreement
]
 
[4]: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:false
  PathLen: undefined
]
 
]
  Algorithm: [SHA256withRSA]
  Signature:
: 97 32 64 63 D4 DA ED AF   CD 7F EC 77 A6 7C 72 85  .2dc...w..r.
...
 
]
chain [1] = [
[
  Version: V3
  Subject: CN=TESTActivationCA, OU=Activation CA, O=TEST ACTIVATION
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
 
  Key:  Sun RSA public key, 2048 bits
  modulus:
241401315179803415263681113133745704037912047640810783616090692543408
...
  public exponent: 65537
  Validity: [From: Wed Jun 09 14:04:45 EDT 2010,
   To: Thu Jun 09 14:04:45 EDT 2011]
  Issuer: OU=Root CA, O=ACME Systems Inc., L=TEST PURPOSES ONLY, C=CA,
CN=ACME Systems Root CA
  SerialNumber: [01]
 
Certificate Extensions: 6
[1]: ObjectId: 2.5.29.30 Criticality=true
NameConstraints: [
Permitted:   GeneralSubtrees:
[
   GeneralSubtree: [
GeneralName: O=instance
Minimum: 0  Maximum: undefined]
]
   ]
 
[2]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
: 7C FB 2B 96 C9 0D 37 89   01 83 D9 5A 67 41 3B 3C  ..+...7ZgA;<
0010: E7 45 81 43.E.C
]
]
 
[3]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
: 70 8F 22 BC D7 55 20 6E   00 D7 3A D3 70 40 F5 49  p."..U n..:@.i
0010: 91 20 90 60. .`
]
 
[OU=Root CA, O=ACME Systems Inc., L=TEST PURPOSES ONLY, C=C

Re: 64bit BIOs and support in OpenSSL

[Jeff Saremi]

Are there any plans to change this? Getting streams larger than 4GB is not very 
unusual these days anymore.


> On Mon, Oct 04, 2010 at 10:37:55AM -0400, Jeff Saremi wrote:
>
> >  Does BIO support 64 bit IO (large files)? If so would the rest of
> > OpenSSL (such as the ssl itself) support those BIOs?
> > I configured the build with 64bit support and didn't see any noticeable
> > changes.
> > Specifically, I'd like to know if BIO_tell() is able to return a 64bit
> > value?
>
> No, BIO_tell() is a macro:
>
> include/openssl/bio.h:
>   #define BIO_tell(b)   (int)BIO_ctrl(b,BIO_C_FILE_TELL,0,NULL)
>
> The (int) cast means that the result is never larger than INT_MAX.
>
> If the library is compiled on a machine with 64-bit longs, then
> the underlying BIO_ctrl() will return the value from lseek() or
> ftell() cast to a (long). This does not imply that 64-bit files
> will not break in some other way.
>
> -- 
>   Viktor.
> __
> OpenSSL Project http://www.openssl.org
> User Support Mailing Listopenssl-users@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

64bit BIOs and support in OpenSSL

[Jeff Saremi]

 Does BIO support 64 bit IO (large files)? If so would the rest of
OpenSSL (such as the ssl itself) support those BIOs?
I configured the build with 64bit support and didn't see any noticeable
changes.
Specifically, I'd like to know if BIO_tell() is able to return a 64bit
value?
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Is there a function to invoke ad-hoc to report an error without having to load/unload strings?

[Jeff Saremi]

 I'm looking for something like:

ERR_report_oneoff_error(const char *func, const char *reason, int line,
const char *file)

that I could use without having to define structures like the following
or calling Err_load_strings and Err_unload_strings:
static ERR_STRING_DATA BIO_str_functs[]=
{
{ERR_FUNC(BIO_F_ACPT_STATE),"ACPT_STATE"},
{ERR_FUNC(BIO_F_BIO_ACCEPT),"BIO_accept"},
...
}


static ERR_STRING_DATA BIO_str_reasons[]=
{
{ERR_REASON(BIO_R_ACCEPT_ERROR)  ,"accept error"},
{ERR_REASON(BIO_R_BAD_FOPEN_MODE),"bad fopen mode"},
...
}

thanks
jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: dynamic locks don't get cleaned up

[Jeff Saremi]

 It's saddening for the problem to exist in the first place but it's
more when no body gives it the time of the day.
But that's life.
I went and fixed the problem on my own:

in crypto.h i declared:

void CRYPTO_cleanup();


and in cryptlib.c i wrote:

void CRYPTO_cleanup()
{
if(dyn_locks)
sk_CRYPTO_dynlock_free(dyn_locks);
if(app_locks)
sk_OPENSSL_STRING_free(app_locks);

}


and i my own application I added a call to the newly added method:
ENGINE_cleanup();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_free_strings();
OBJ_NAME_cleanup(-1);
>>> CRYPTO_cleanup();


Running my application with Valgrind produces the following output.
That's what we'd like to see:
==3959==
==3959== HEAP SUMMARY:
==3959== in use at exit: 0 bytes in 0 blocks
==3959==   total heap usage: 4,496 allocs, 4,496 frees, 137,484 bytes
allocated
==3959==
>>> ==3959== All heap blocks were freed -- no leaks are possible


On 10-08-18 04:28 PM, Jeff Saremi wrote:
>  any takers from the openssl team? true, false? known issue, user error?
> anything?
>
> On 10-08-17 04:23 PM, Jeff Saremi wrote:
>> I apologize if this shows up more than once. I've been having problems
>> sending emails out, all day.
>>
>>  First I encountered this with valgrind but then I decided to have
>> openssl print the leaks and it was also confirmed.
>> I have reduced my code to the following two lines. Prior to this if
>> course initilization of openssl and then the cleanup. Either there's a
>> call that I'm missing or the the dynamic locks don't get cleaned up upon
>> exit. Looking at the code in cryptlib.c, i don't see anywhere freeing up
>> the memory allocated to the following stacks:
>>
>> static STACK_OF(OPENSSL_STRING) *app_locks=NULL;
>> static STACK_OF(CRYPTO_dynlock) *dyn_locks=NULL;
>>
>> And here's my sample code:
>> int l = CRYPTO_get_new_dynlockid();
>> CRYPTO_destroy_dynlockid(l);
>>
>> Before this I have a bunch of lines like:
>> CRYPTO_malloc_debug_init();
>> CRYPTO_set_mem_debug_options(V_CRYPTO_MDEBUG_ALL);
>> CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON);
>>
>> SSL_library_init();
>> OpenSSL_add_all_digests();
>> CRYPTO_set_id_callback(..);
>> CRYPTO_set_locking_callback(...);
>> CRYPTO_set_dynlock_create_callback(...);
>> CRYPTO_set_dynlock_lock_callback(...);
>> CRYPTO_set_dynlock_destroy_callback(...);
>>
>> And at the end of the code I have something like:
>> CRYPTO_set_id_callback(NULL);
>> CRYPTO_set_locking_callback(NULL);
>> CRYPTO_set_dynlock_create_callback(NULL);
>> CRYPTO_set_dynlock_lock_callback(NULL);
>> CRYPTO_set_dynlock_destroy_callback(NULL);
>>
>> ENGINE_cleanup();
>> EVP_cleanup();
>> CRYPTO_cleanup_all_ex_data();
>> ERR_free_strings();
>> OBJ_NAME_cleanup(-1);
>> ERR_remove_thread_state(NULL);
>> SSL_free_comp_methods();
>> CRYPTO_mem_leaks_fp(stderr);
>>
>> As you can see I have included every cleanup call I could find.
>> Running the code produces the following output:
>> [19:49:10]   188 file=stack.c, line=125, thread=19596, number=20,
>> address=08DF0E50
>> [19:49:10]   189 file=stack.c, line=127, thread=19596, number=16,
>> address=08DF0F78
>> 36 bytes leaked in 2 chunks
>>
>>
>>
>>
>> __
>> OpenSSL Project http://www.openssl.org
>> User Support Mailing Listopenssl-users@openssl.org
>> Automated List Manager   majord...@openssl.org
> __
> OpenSSL Project http://www.openssl.org
> User Support Mailing Listopenssl-users@openssl.org
> Automated List Manager   majord...@openssl.org

Re: dynamic locks don't get cleaned up

[Jeff Saremi]

 any takers from the openssl team? true, false? known issue, user error?
anything?

On 10-08-17 04:23 PM, Jeff Saremi wrote:
> I apologize if this shows up more than once. I've been having problems
> sending emails out, all day.
>
>  First I encountered this with valgrind but then I decided to have
> openssl print the leaks and it was also confirmed.
> I have reduced my code to the following two lines. Prior to this if
> course initilization of openssl and then the cleanup. Either there's a
> call that I'm missing or the the dynamic locks don't get cleaned up upon
> exit. Looking at the code in cryptlib.c, i don't see anywhere freeing up
> the memory allocated to the following stacks:
>
> static STACK_OF(OPENSSL_STRING) *app_locks=NULL;
> static STACK_OF(CRYPTO_dynlock) *dyn_locks=NULL;
>
> And here's my sample code:
> int l = CRYPTO_get_new_dynlockid();
> CRYPTO_destroy_dynlockid(l);
>
> Before this I have a bunch of lines like:
> CRYPTO_malloc_debug_init();
> CRYPTO_set_mem_debug_options(V_CRYPTO_MDEBUG_ALL);
> CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON);
>
> SSL_library_init();
> OpenSSL_add_all_digests();
> CRYPTO_set_id_callback(..);
> CRYPTO_set_locking_callback(...);
> CRYPTO_set_dynlock_create_callback(...);
> CRYPTO_set_dynlock_lock_callback(...);
> CRYPTO_set_dynlock_destroy_callback(...);
>
> And at the end of the code I have something like:
> CRYPTO_set_id_callback(NULL);
> CRYPTO_set_locking_callback(NULL);
> CRYPTO_set_dynlock_create_callback(NULL);
> CRYPTO_set_dynlock_lock_callback(NULL);
> CRYPTO_set_dynlock_destroy_callback(NULL);
>
> ENGINE_cleanup();
> EVP_cleanup();
> CRYPTO_cleanup_all_ex_data();
> ERR_free_strings();
> OBJ_NAME_cleanup(-1);
> ERR_remove_thread_state(NULL);
> SSL_free_comp_methods();
> CRYPTO_mem_leaks_fp(stderr);
>
> As you can see I have included every cleanup call I could find.
> Running the code produces the following output:
> [19:49:10]   188 file=stack.c, line=125, thread=19596, number=20,
> address=08DF0E50
> [19:49:10]   189 file=stack.c, line=127, thread=19596, number=16,
> address=08DF0F78
> 36 bytes leaked in 2 chunks
>
>
>
>
> __
> OpenSSL Project http://www.openssl.org
> User Support Mailing Listopenssl-users@openssl.org
> Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

dynamic locks don't get cleaned up

[Jeff Saremi]

I apologize if this shows up more than once. I've been having problems
sending emails out, all day.

 First I encountered this with valgrind but then I decided to have
openssl print the leaks and it was also confirmed.
I have reduced my code to the following two lines. Prior to this if
course initilization of openssl and then the cleanup. Either there's a
call that I'm missing or the the dynamic locks don't get cleaned up upon
exit. Looking at the code in cryptlib.c, i don't see anywhere freeing up
the memory allocated to the following stacks:

static STACK_OF(OPENSSL_STRING) *app_locks=NULL;
static STACK_OF(CRYPTO_dynlock) *dyn_locks=NULL;

And here's my sample code:
int l = CRYPTO_get_new_dynlockid();
CRYPTO_destroy_dynlockid(l);

Before this I have a bunch of lines like:
CRYPTO_malloc_debug_init();
CRYPTO_set_mem_debug_options(V_CRYPTO_MDEBUG_ALL);
CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON);

SSL_library_init();
OpenSSL_add_all_digests();
CRYPTO_set_id_callback(..);
CRYPTO_set_locking_callback(...);
CRYPTO_set_dynlock_create_callback(...);
CRYPTO_set_dynlock_lock_callback(...);
CRYPTO_set_dynlock_destroy_callback(...);

And at the end of the code I have something like:
CRYPTO_set_id_callback(NULL);
CRYPTO_set_locking_callback(NULL);
CRYPTO_set_dynlock_create_callback(NULL);
CRYPTO_set_dynlock_lock_callback(NULL);
CRYPTO_set_dynlock_destroy_callback(NULL);

ENGINE_cleanup();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_free_strings();
OBJ_NAME_cleanup(-1);
ERR_remove_thread_state(NULL);
SSL_free_comp_methods();
CRYPTO_mem_leaks_fp(stderr);

As you can see I have included every cleanup call I could find.
Running the code produces the following output:
[19:49:10]   188 file=stack.c, line=125, thread=19596, number=20,
address=08DF0E50
[19:49:10]   189 file=stack.c, line=127, thread=19596, number=16,
address=08DF0F78
36 bytes leaked in 2 chunks




__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: My custom engine_finish method does not get called through ENGINE_finish

[Jeff Saremi]

 I fixed the problem. I'm listing it here in case someone else runs into
this:

In the loadEngine() I call ENGINE_set_default(). I have to the opposite
when unloading my engine.
Unfortunately, I could not find any "ENGINE_unset_default()" or
"ENGINE_unregister_all()" so I had to explicitly unregister my method
pointers. So my new unloadEngine looks like the following:

void unloadEngine(ENGINE *e)
{
ENGINE_unregister_pkey_asn1_meths(e);
ENGINE_unregister_pkey_meths(e);
ENGINE_unregister_RSA(e);

ENGINE_remove(e);
/* Release the functional reference from ENGINE_init() */
   ENGINE_finish(e);
/* Release the structural reference from ENGINE_by_id() */
ENGINE_free(e);
}


On 10-08-13 09:23 AM, Jeff Saremi wrote:
>  I'm trying to use my custom engine however I cannot get it to clean up
> nicely.
> For the initialization i used the sample in openssl ENGINE(3) documentation.
>
> Here's how it goes:
>
> ENGINE *loadEngine()
> {
> ENGINE *e;
> e = ENGINE_by_id(MY_ENGINE_ID);
> if(!e)
> ENGINE_load_my();
> e = ENGINE_by_id(MY_ENGINE_ID); /* try again */
> if(!e)
> /* the engine isn't available */
> return NULL;
> if(!ENGINE_init(e)) {
> /* the engine couldn't initialise, release 'e' */
> ENGINE_free(e);
> return NULL;
> }
> ENGINE_set_default(e, ENGINE_METHOD_ALL);
> return e;
> }
> void unloadEngine(ENGINE *e)
> {
> /* Release the functional reference from ENGINE_init() */
> ENGINE_finish(e);
> /* Release the structural reference from ENGINE_by_id() */
> ENGINE_free(e);
> /* ENGINE_cleanup();  my engine_finish method does not get
> called unless I include this line */
> }
> void testKeyGen()
> {
>/* some openssl initialization code such as loading ciphers,
> algorithms, existing engines, establishing dynamic locks and so on */
>
> ENGINE *e = loadEngine();
> ASSERT(e, "could not create the engine\n");
> EVP_PKEY *pkey = NULL;
> genPKey(e, &pkey); /* some calls to EVP_PKEY_CTX_new_id,
> EVP_PKEY_keygen and so on */
> ASSERT(pkey, "could not generate PKEY\n");
>
> /* so far so good */
> if(pkey)
> EVP_PKEY_free(pkey);
>
> unloadEngine(e);
>/* at the end of this I have unfreed memory (the dynamic lock I
> created in my engine because my_finish was not called */
> }
>
> And here's my engine initialization and deinit routines. I stepped
> through the code in ENGINE_finish() and realized that the following line
> (to_return = e->finish(e); ) does not get executed because apparently
> the ref count is not zero:
>
> * file openssl/crypto/engine/eng_init.c  **
> int engine_unlocked_finish(ENGINE *e, int unlock_for_handlers)
> {
> if((e->funct_ref == 0) && e->finish)
> {
> if(unlock_for_handlers)
> CRYPTO_w_unlock(CRYPTO_LOCK_ENGINE);
> to_return = e->finish(e);
> if(unlock_for_handlers)
> CRYPTO_w_lock(CRYPTO_LOCK_ENGINE);
> if(!to_return)
> return 0;
> ...
> }
>
> * continuation of my code -- my engine init routines  *
>
> static int my_init(ENGINE *e)
> {
> ERR_load_MY_strings();
> my_lock_id = CRYPTO_get_new_dynlockid();
> ...
> return 1;
> }
> static int my_finish(ENGINE *e)
> {
> ERR_unload_MY_strings();
> CRYPTO_destroy_dynlockid(my_lock_id);
> return 1;
> }
> static int bind_helper(ENGINE *e)
> {
> ...
> if (!ENGINE_set_id(e, MY_ENGINE_ID)
> || !ENGINE_set_name(e, MY_ENGINE_NAME)
> || !ENGINE_set_destroy_function(e, my_destroy)
> || !ENGINE_set_init_function(e, my_init)
> || !ENGINE_set_finish_function(e, my_finish)
> ...
> return 0;
> return 1;
> }
> static ENGINE *engine_my(void)
> {
> ENGINE *ret = ENGINE_new();
> if (!ret)
> return NULL;
> if (!bind_helper(ret))
> {
> ENGINE_free(ret);
> return NULL;
> }
> return ret;
> }
> void ENGINE_load_my(void)
> {
> ENGINE *toadd = engine_my();
> if (!toadd)
> return;
> ENGINE_add(toadd);
> ENGINE_free(toadd);
> ERR_clear_error();
> }
>
>
>
>
>
> This email contains Morega Systems Inc. Privileged and Confidential 
> information.



This email contains Morega Systems Inc. Privileged and Confidential information.

My custom engine_finish method does not get called through ENGINE_finish

[Jeff Saremi]

 I'm trying to use my custom engine however I cannot get it to clean up
nicely.
For the initialization i used the sample in openssl ENGINE(3) documentation.

Here's how it goes:

ENGINE *loadEngine()
{
ENGINE *e;
e = ENGINE_by_id(MY_ENGINE_ID);
if(!e)
ENGINE_load_my();
e = ENGINE_by_id(MY_ENGINE_ID); /* try again */
if(!e)
/* the engine isn't available */
return NULL;
if(!ENGINE_init(e)) {
/* the engine couldn't initialise, release 'e' */
ENGINE_free(e);
return NULL;
}
ENGINE_set_default(e, ENGINE_METHOD_ALL);
return e;
}
void unloadEngine(ENGINE *e)
{
/* Release the functional reference from ENGINE_init() */
ENGINE_finish(e);
/* Release the structural reference from ENGINE_by_id() */
ENGINE_free(e);
/* ENGINE_cleanup();  my engine_finish method does not get
called unless I include this line */
}
void testKeyGen()
{
   /* some openssl initialization code such as loading ciphers,
algorithms, existing engines, establishing dynamic locks and so on */

ENGINE *e = loadEngine();
ASSERT(e, "could not create the engine\n");
EVP_PKEY *pkey = NULL;
genPKey(e, &pkey); /* some calls to EVP_PKEY_CTX_new_id,
EVP_PKEY_keygen and so on */
ASSERT(pkey, "could not generate PKEY\n");

/* so far so good */
if(pkey)
EVP_PKEY_free(pkey);

unloadEngine(e);
   /* at the end of this I have unfreed memory (the dynamic lock I
created in my engine because my_finish was not called */
}

And here's my engine initialization and deinit routines. I stepped
through the code in ENGINE_finish() and realized that the following line
(to_return = e->finish(e); ) does not get executed because apparently
the ref count is not zero:

* file openssl/crypto/engine/eng_init.c  **
int engine_unlocked_finish(ENGINE *e, int unlock_for_handlers)
{
if((e->funct_ref == 0) && e->finish)
{
if(unlock_for_handlers)
CRYPTO_w_unlock(CRYPTO_LOCK_ENGINE);
to_return = e->finish(e);
if(unlock_for_handlers)
CRYPTO_w_lock(CRYPTO_LOCK_ENGINE);
if(!to_return)
return 0;
...
}

* continuation of my code -- my engine init routines  *

static int my_init(ENGINE *e)
{
ERR_load_MY_strings();
my_lock_id = CRYPTO_get_new_dynlockid();
...
return 1;
}
static int my_finish(ENGINE *e)
{
ERR_unload_MY_strings();
CRYPTO_destroy_dynlockid(my_lock_id);
return 1;
}
static int bind_helper(ENGINE *e)
{
...
if (!ENGINE_set_id(e, MY_ENGINE_ID)
|| !ENGINE_set_name(e, MY_ENGINE_NAME)
|| !ENGINE_set_destroy_function(e, my_destroy)
|| !ENGINE_set_init_function(e, my_init)
|| !ENGINE_set_finish_function(e, my_finish)
...
return 0;
return 1;
}
static ENGINE *engine_my(void)
{
ENGINE *ret = ENGINE_new();
if (!ret)
return NULL;
if (!bind_helper(ret))
{
ENGINE_free(ret);
return NULL;
}
return ret;
}
void ENGINE_load_my(void)
{
ENGINE *toadd = engine_my();
if (!toadd)
return;
ENGINE_add(toadd);
ENGINE_free(toadd);
ERR_clear_error();
}





This email contains Morega Systems Inc. Privileged and Confidential information.

Re: Procedure to define and add new lock types?

[Jeff Saremi]

 I think i found the answer to this. The responsibility is lied within
the application.
The app should setup its own list of locks, possibly above and beyond
the CRYPTO_NUM_LOCKS.
And then provide appropriate routines to pass to the following:
CRYPTO_set_locking_callback
CRYPTO_set_dynlock_create_callback
CRYPTO_set_dynlock_lock_callback
CRYPTO_set_dynlock_destroy_callback

The desired lock type is passed in the struct CRYPTO_dynlock_value *
parameter to the cusom dynlock_lock_callback.


On 10-08-09 09:18 AM, Jeff Saremi wrote:
>  In my engine, i'd like to use a new lock for a set of operations. This
> lock is not CRYPTO_LOCK_ENGINE or any other defined ones.
> Is there any way in OpenSSL to define a say user lock (type) and then
> use CRYPTO_Lock function on it?
> What would happen to CRYPTO_NUM_LOCKS? would that increase by way of
> defining and adding this lock?
>
> thanks
> jeff
>
>
>
> This email contains Morega Systems Inc. Privileged and Confidential 
> information.
> __
> OpenSSL Project http://www.openssl.org
> User Support Mailing Listopenssl-users@openssl.org
> Automated List Manager   majord...@openssl.org



This email contains Morega Systems Inc. Privileged and Confidential information.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Procedure to define and add new lock types?

[Jeff Saremi]

 In my engine, i'd like to use a new lock for a set of operations. This
lock is not CRYPTO_LOCK_ENGINE or any other defined ones.
Is there any way in OpenSSL to define a say user lock (type) and then
use CRYPTO_Lock function on it?
What would happen to CRYPTO_NUM_LOCKS? would that increase by way of
defining and adding this lock?

thanks
jeff



This email contains Morega Systems Inc. Privileged and Confidential information.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Re: Frustration with method based on EVP_PKEY_METHOD

[Jeff Saremi]

OK. Thanks for replying.

On Thu, 2010-07-08 at 18:59 +0200, Dr. Stephen Henson wrote:
> The only current example of an external EVP_PKEY_METHOD (in the gost ENGINE)
> implements everything from scratch. 
> 
> It is perfectly reasonable for another implementation to copy existing methods
> or intercept them. The fact you can't do that with the current API is an
> omission, sorry about that. I'll look into fixing that.
> 
> Steve.
> --
> Dr Stephen N. Henson. OpenSSL project core developer.
> Commercial tech support now available see: http://www.openssl.org




This email contains Morega Systems Inc. Privileged and Confidential information.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Frustration with method based on EVP_PKEY_METHOD

[Jeff Saremi]

The structure itself is only forward-declared meaning that a programmer
has no way of getting into the actual evp_pkey_method_st and its fields.
Now that would be fine if there were a complete set of methods
EVP_PKEY_meth_* to manipulate the pointers to EVP_PKEY_METHOD. But alas
there's only a few.
For instance I cannot copy one struct to another meaning
EVP_PKEY_meth_copy() is missing. Note that we do have
EVP_PKEY_asn1_copy() which in many ways behaves the same way as its
counter part.
We can't also get a pointer to an existing function i.e. by calling
EVP_PKEY_meth_get_ctrl(). There are no "get" methods.

If there's any rationale behind this, I'd love to know that.
thanks
jeff



This email contains Morega Systems Inc. Privileged and Confidential information.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

ENGINE: RSA_METHOD->rsa_keygen does not behave like other function pointers

[Jeff Saremi]

To avoid having to override every RSA function, I could fill my engine's
custom RSA_METHOD with methods from RSA_PKCS1_SSLeay. Alternatively I
could have my functions wrap the internal OpenSSL functions. For
instance:
int myEngineRsaModExp(BIGNUM *r0,const BIGNUM *I,RSA *rsa,BN_CTX *ctx)
{
  int rc = 0;
  ... 
  // do some custom work here
  // now call internal method
  rc = RSA_PKCS1_SSLeay()->rsa_mod_exp(r0, I, rsa, cts);

  // do some more customized work
  return rc;

}

However, I can't do that with "rsa_keygen". For some reason this pointer
is null (actually there could be more of these -- so far I have come
across this one only).

The source code for RSA_generate_key_ex which calls the builtin
"rsa_builtin_kegen" has a comment including the following:

/* this wrapper would normally be placed in ... the ...
 * implementation would probably be in rsa_eay.c.
 * Nonetheless, is kept here so
 * that we don't introduce a new linker dependency. 

Could this be fixed with so that RSA_PKCS1_SSLeay() has a pointer to the
builtin implementation. Since i don't have access to the original,
declared-as-static method I have to come up unsafe hacks like this (a
very crude version):

int rc = 0;
RSA_METHOD *myEnginesMethod = rsa->meth;
rsa->meth = RSA_null_method();
rc = RSA_generate_key_ex(rsa, bits, e, cb);
rsa->meth = myEnginesMethod;
return rc;



This email contains Morega Systems Inc. Privileged and Confidential information.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

ENGINE: where are corresponding 'save' methods for ENGINE_set_load_pub/privatekey_function?

[Jeff Saremi]

How does one provide own engine functions for saving public and private
keys?
thanks
jeff



This email contains Morega Systems Inc. Privileged and Confidential information.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

Q: Ramifications of creating new X509-Store and setting it in SSL_CTX while SSL_accept's being done

[Jeff Saremi]

In order to update the CRLs in our server socket and due to seemingly
lack of any CRL update methods, I decided to create new X509_STOREs and
set them in the SSL_CTX every time there's a change in the CRL list.

I'd like to know the effects of that if at the same time new SSLs being
created, setup for accept(), or  accepting.

Are there any specific synchronization primitives i should be aware of
when doing this.

Thanks

Je

 




This email contains Morega Systems Inc. Privileged and Confidential information.