Oliver Graute wrote: > On 25/02/16, Oliver Graute wrote: >> On 24/02/16, Rob Crittenden wrote: >>> Oliver Graute wrote: >>>> On 23/02/16, Oliver Graute wrote: >>>>> On Tue, Feb 23, 2016 at 5:10 PM, Oliver Graute <[email protected]> >>>>> wrote: >>>>>> On 23/02/16, Rob Crittenden wrote: >>>>>>> Oliver Graute wrote: >>>>>>>> On 22/02/16, Rob Crittenden wrote: >>>>>>>>> Oliver Graute wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I installed the mod_nss plugin in version 1.0.12 on my apache >>>>>>>>>> webserver, >>>>>>>>>> TLS on Port 443 is working fine until I enable the new NSSSession >>>>>>>>>> ticket >>>>>>>>>> feature in my nss.conf with: >>>>>>>>>> >>>>>>>>>> #RFC 5077 >>>>>>>>>> NSSSessionTickets on >>>>>>>>>> >>>>>>>>>> then something is broken, I see segfaults in my apache error log: >>>>>>>>>> >>>>>>>>>> [Fri Feb 19 10:12:15.338660 2016] [mpm_prefork:notice] [pid 413] >>>>>>>>>> AH00163: Apache/2.4.16 (Unix) mod_nss/1.0.12 NSS/3.19.2 Basic ECC >>>>>>>>>> PHP/5.5.10 configured -- resuming normal operations >>>>>>>>>> [Fri Feb 19 10:12:15.338843 2016] [mpm_prefork:info] [pid 413] >>>>>>>>>> AH00164: Server built: Feb 22 2016 12:44:38 >>>>>>>>>> [Fri Feb 19 10:12:15.339046 2016] [core:notice] [pid 413] AH00094: >>>>>>>>>> Command line: '/usr/sbin/httpd -D FOREGROUND -D SSL -D PHP5' >>>>>>>>>> [Fri Feb 19 10:12:15.339160 2016] [mpm_prefork:debug] [pid 413] >>>>>>>>>> prefork.c(995): AH00165: Accept mutex: sysvsem (default: sysvsem) >>>>>>>>>> [Fri Feb 19 10:12:15.386483 2016] [:debug] [pid 416] >>>>>>>>>> nss_engine_init.c(286): SNI is enabled >>>>>>>>>> [Fri Feb 19 10:12:15.386853 2016] [:info] [pid 416] Init: Seeding >>>>>>>>>> PRNG with 136 bytes of entropy >>>>>>>>>> [Fri Feb 19 10:12:40.374175 2016] [core:notice] [pid 413] AH00052: >>>>>>>>>> child pid 416 exit signal Segmentation fault (11) >>>>>>>>>> [Fri Feb 19 10:12:41.496820 2016] [:debug] [pid 423] >>>>>>>>>> nss_engine_init.c(286): SNI is enabled >>>>>>>>>> [Fri Feb 19 10:12:41.497224 2016] [:info] [pid 423] Init: Seeding >>>>>>>>>> PRNG with 136 bytes of entropy >>>>>>>>>> [Fri Feb 19 10:12:42.388948 2016] [core:notice] [pid 413] AH00052: >>>>>>>>>> child pid 423 exit signal Segmentation fault (11) >>>>>>>>>> [Fri Feb 19 10:12:43.508779 2016] [:debug] [pid 424] >>>>>>>>>> nss_engine_init.c(286): SNI is enabled >>>>>>>>>> [Fri Feb 19 10:12:43.509217 2016] [:info] [pid 424] Init: Seeding >>>>>>>>>> PRNG with 136 bytes of entropy >>>>>>>>>> [Fri Feb 19 10:12:44.404130 2016] [core:notice] [pid 413] AH00052: >>>>>>>>>> child pid 424 exit signal Segmentation fault (11) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> and in Chrome Browser I got: >>>>>>>>>> >>>>>>>>>> ERR_SSL_VERSION_OR_CIPHER_MISMATCH >>>>>>>>>> >>>>>>>>>> I tested also a basic ssl client connection with openssl: >>>>>>>>>> >>>>>>>>>> openssl s_client -connect 192.168.1.229:443 -state -debug >>>>>>>>>> >>>>>>>>>> SSL_connect:SSLv3 read server certificate A >>>>>>>>>> SSL_connect:SSLv3 read server key exchange A >>>>>>>>>> SSL_connect:SSLv3 read server done A >>>>>>>>>> write to 0x205dec0 [0x206dd50] (75 bytes => 75 (0x4B)) >>>>>>>>>> 0000 - 16 03 03 00 46 10 00 00-42 41 04 3d c7 93 63 45 >>>>>>>>>> ....F...BA.=..cE >>>>>>>>>> 0010 - 79 41 11 bc 06 c0 b7 c6-d1 b5 33 d9 86 a6 d5 e9 >>>>>>>>>> yA........3..... >>>>>>>>>> 0020 - 36 e4 2b ac 0e bc 70 d6-d6 8c a7 a9 3c dd 1b 0c >>>>>>>>>> 6.+...p.....<... >>>>>>>>>> 0030 - 77 48 20 38 dd 1e c9 a1-05 6c 5c b6 c9 f4 99 f2 wH >>>>>>>>>> 8.....l\..... >>>>>>>>>> 0040 - 1a 18 ae 81 63 71 65 90-e8 a5 b6 ....cqe.... >>>>>>>>>> SSL_connect:SSLv3 write client key exchange A >>>>>>>>>> write to 0x205dec0 [0x206dd50] (6 bytes => 6 (0x6)) >>>>>>>>>> 0000 - 14 03 03 00 01 01 ...... >>>>>>>>>> SSL_connect:SSLv3 write change cipher spec A >>>>>>>>>> write to 0x205dec0 [0x206dd50] (45 bytes => 45 (0x2D)) >>>>>>>>>> 0000 - 16 03 03 00 28 b1 e0 60-8a 2c 97 cf a0 4f 97 ee >>>>>>>>>> ....(..`.,...O.. >>>>>>>>>> 0010 - cd 8f 05 41 aa 50 a6 73-a3 4c 86 1e 5f 3c 7b 2b >>>>>>>>>> ...A.P.s.L.._<{+ >>>>>>>>>> 0020 - 2d 7e 6a 68 dc 97 94 9d-91 15 c0 0e 27 >>>>>>>>>> -~jh........' >>>>>>>>>> SSL_connect:SSLv3 write finished A >>>>>>>>>> SSL_connect:SSLv3 flush data >>>>>>>>>> read from 0x205dec0 [0x2063f83] (5 bytes => 0 (0x0)) >>>>>>>>>> SSL_connect:failed in SSLv3 read server session ticket A >>>>>>>>>> 140123095688864:error:140790E5:SSL routines:SSL23_WRITE:ssl >>>>>>>>>> handshake failure:s23_lib.c:177: >>>>>>>>>> >>>>>>>>>> apache and mod_nss are build from the sources for an embedded yocto >>>>>>>>>> environment. >>>>>>>>>> >>>>>>>>>> some ideas, whats going on here? >>>>>>>>> >>>>>>>>> Can you get a stack trace from the core? >>>>>>>> >>>>>>>> I can give you an strace, see below. Other stack tools are currently >>>>>>>> not >>>>>>>> available, because I need to compile them first for my yocto >>>>>>>> environment. If you need something special please tell me. >>>>>>>> >>>>>>>>> This is Apache 2.4.x? >>>>>>>> >>>>>>>> yes it is Apache 2.4.16 >>>>>>>> >>>>>>>>> Is it failing on a request or on startup? >>>>>>>> >>>>>>>> its failing on every https request. >>>>>>> >>>>>>> strace in this case isn't particularly helpful as it doesn't show where >>>>>>> it is crashing. >>>>>>> >>>>>>> Can I see your nss.conf? >>>>>>> >>>>>>> What version of NSS are you using? >>>>>> >>>>>> I'am using nss in version 3.19.2 >>>>>> >>>>>> here my nss.conf >>>>>> >>>>>> # >>>>>> # This is the Apache server configuration file providing SSL support >>>>>> using. >>>>>> # the mod_nss plugin. It contains the configuration directives to >>>>>> instruct >>>>>> # the server how to serve pages over an https connection. >>>>>> # >>>>>> # Do NOT simply read the instructions in here without understanding >>>>>> # what they do. They're here only as hints or reminders. If you are >>>>>> unsure >>>>>> # consult the online docs. You have been warned. >>>>>> # >>>>>> >>>>>> # >>>>>> # When we also provide SSL we have to listen to the >>>>>> # standard HTTP port (see above) and to the HTTPS port >>>>>> # >>>>>> # Note: Configurations that use IPv6 but not IPv4-mapped addresses need >>>>>> two >>>>>> # Listen directives: "Listen [::]:443" and "Listen 0.0.0.0:443" >>>>>> # >>>>>> Listen 443 >>>>>> >>>>>> ## >>>>>> ## SSL Global Context >>>>>> ## >>>>>> ## All SSL configuration in this context applies both to >>>>>> ## the main server and all SSL-enabled virtual hosts. >>>>>> ## >>>>>> >>>>>> # >>>>>> # Some MIME-types for downloading Certificates and CRLs >>>>>> # >>>>>> AddType application/x-x509-ca-cert .crt >>>>>> AddType application/x-pkcs7-crl .crl >>>>>> >>>>>> >>>>>> # Pass Phrase Helper: >>>>>> # This helper program stores the token password pins between >>>>>> # restarts of Apache. >>>>>> # Unfortunately the directive is required even if we use no password >>>>>> # (though in such case the program should never be used) >>>>>> NSSPassPhraseHelper /usr/lib/apache2/bin/nss_pcache >>>>>> >>>>>> # Pass Phrase Dialog: >>>>>> # Configure the pass phrase gathering process. >>>>>> # The filtering dialog program (`builtin' is a internal >>>>>> # terminal dialog) has to provide the pass phrase on stdout. >>>>>> #NSSPassPhraseDialog builtin >>>>>> NSSPassPhraseDialog file:/etc/apache2/password.conf >>>>>> >>>>>> # Configure the SSL Session Cache. >>>>>> # NSSSessionCacheSize is the number of entries in the cache. >>>>>> # NSSSessionCacheTimeout is the SSL2 session timeout (in seconds). >>>>>> # NSSSession3CacheTimeout is the SSL3/TLS session timeout (in seconds). >>>>>> NSSSessionCacheSize 10000 >>>>>> NSSSessionCacheTimeout 100 >>>>>> NSSSession3CacheTimeout 86400 >>>>>> >>>>>> #RFC 5077 >>>>>> NSSSessionTickets off >>>>> >>>>> SessionTickets are on not off >>>>> >>>>> NSSSessionTickets on >>>>> >>>> >>>> in my /etc/apache2/extra/httpd-vhosts.conf I also added some mod_nss >>>> parameters for every virtual host. It seems that the problem is here. >>>> Because >>>> if I remove the include of httpd-vhosts.conf no segfaults occur. >>>> >>>> here my httpd-vhosts.conf with nss related parameters. Are these >>>> parameters capable for a virtual host configuration? >>> >>> You've included some directives that are expected to be global: >>> >>> NSSPassPhraseHelper >>> NSSPassPhraseDialog >>> NSSSession*Cache* >>> NSSCertificateDatabase >>> >>> I doubt they hurt anything. >> >> ok thx, I removed these directives from the httpd-vhosts.conf >> >>> >>> I still can't duplicate a crash. I ran it through valgrind to see if >>> there was a memory issue and came up with nothing too. >>> >>> This could be an architecture problem that I just can't see on x86 >>> hardware I suppose. >> >> I'am using a arm hardware (imx6) >> >>> This particular crash seems rather strange too. Within mod_nss enabling >>> this just calls: >>> >>> SSL_OptionSet(mctx->model, SSL_ENABLE_SESSION_TICKETS, >>> mctx->sc->session_tickets); >>> >>> Basically enabling it in the model server socket. I would imagine the >>> crash is probably deeper in NSS. >> >> >>> To narrow things down you might try the NSS tool selfserv. It has an >>> option, -u, to enable session tickets. It might eliminate mod_nss as the >>> crash source (or it might implicate it too). >> >> >> thx for the hint to the NSS selfserv tool. >> >> I'am using it with DSA Keys instead of Eliptic Curve Keys because >> selfserf can't handle ECC. (selfserv -Y didn't show me ECC ciphers) >> >> First I generate a new nss db. >> >> certutil -N -d /etc/apache2/nss-conf -f /etc/apache2/password2.conf >> certutil -G -k dsa -n localhost -t "TC,," -d /etc/apache2/nss-conf -f >> /etc/apache2/password2.conf >> certutil -K -d /etc/apache2/nss-conf -f /etc/apache2/password2.conf >> certutil -L -d /etc/apache2/nss-conf -f /etc/apache2/password2.conf >> certutil -S -s "CN=localhost" -x -n localhost -t "TC,," -d >> /etc/apache2/nss-conf/ -f /etc/apache2/password2.conf >> >> Then starting selfserv with params: >> >> selfserv -n "localhost" -V tls1.2: -d /etc/apache2/nss-conf/ -f >> /etc/apache2/password2.conf -p 443 -v -u >> >> selfserv: About to call accept. >> >> Now I perform a https request with the Chrome Browser >> >> GET /test.php HTTP/1.1 >> Host: 192.168.1.229 >> Connection: keep-alive >> Cache-Control: max-age=0 >> Accept: >> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >> Upgrade-Insecure-Requests: 1 >> User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like >> Gecko) Chrome/46.0.2490.80 Safari/537.36 >> DNT: 1 >> Accept-Encoding: gzip, deflate, sdch >> Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4 >> >> EOF >> >> it looks for me that it works with selfserv. But I'm not sure if the Session >> Ticket works and if it makes a difference not using ECC Keys. > > I checked with wireshark, and I saw the TLS Session Ticket Extension > in the hello packet there. > > I'also switched back to my old ciphers in my apache setup nss.conf > > NSSCipherSuite > +rsa_rc4_128_md5,+rsa_rc4_128_sha,+rsa_3des_sha,-rsa_des_sha,-rsa_rc4_40_md5,-rsa_rc2_40_md5,-rsa_null_md5,-rsa_null_sha,+fips_3des_sha,-fips_des_sha,-rsa_des_56_sha,-rsa_rc4_56_sha,+rsa_aes_128_sha,+rsa_aes_256_sha > > and this is working together with NSSSessionTickets on. > > So it breaks if i'am using ECC Ciphersuite together with Session Tickets! > > NSSCipherSuite +ecdhe_ecdsa_aes_128_gcm_sha256,+ecdhe_ecdsa_aes_128_sha256 > NSSSessionTickets on > > is this something that can be explained?
Hmm, interesting. It gives me something to look at. I was in the middle of responding to your previous message when this one came in. I had been testing with an RSA key. Let me try an ECC key to see if I can reproduce the crash (no success yet). Here is the previous bit I was going to send: This actually generated an RSA key for the cert you used for selfserv. certutil will generate a certificate if the -k <id> option isn't provided. You can do the same with ECC by adding to the last command -k ec -q <curve> rob _______________________________________________ Mod_nss-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/mod_nss-list
