Some of Comcast's MX servers (mx1.comcast.net, mx2.comcast.net are the
ones I've verified so far) appear to be handing out test SSL
certificates, at least that's the best guess I can make from the
research I've done so far. I reserve the right to be absolutely wrong.
I've been battling over this for 2 days now. I've seen other references
to this problem and even some (one from this list back in January)
suggesting that comcast says they're going to fix the problem, but it
doesn't seem that they have. None of the references to the bad SSL
certificate that I've seen have indicated delivery failures such as the
ones I'm about to describe.
The problem this is causing us are connections dropping and connections
timing out while trying to deliver mail to comcast (I don't understand
why just yet). It doesn't happen all the time, but it happens enough
that we're getting complaints about it. For now I've disabled smtp_tls_*
which gets the comcast destined mail out of the queue and on to their
servers. I'm not really happy about these changes and I don't understand
how the bad SSL certs are related to the connection drops and timeouts,
but to the best I can tell they are related because turning off TLS in
the smtp client pushes the mail out just fine. Maybe it's just
coincidence but every single time this has happened, turning off TLS in
the smtp client was the only way I could get the mail to change hands
with comcast without a connection drop or timeout in the middle.
I've included logs, postconf -n as well as openssl s_client tests at the
end of this email in that order.
J.P.
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net: num=19:self signed certificate in
certificate chain
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net: num=24:invalid CA certificate
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net: num=26:unsupported certificate purpose
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net: num=10:certificate has expired
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net:certificate has expired
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net:certificate has expired
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net: num=10:certificate has expired
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net:certificate has expired
Feb 12 10:37:45 mail1 postfix/smtp[15974]: certificate verification
failed for mx1.comcast.net:certificate has expired
Feb 12 10:38:21 mail1 postfix/smtp[15974]: 1D48E6A011B: lost connection
with mx1.comcast.net[76.96.62.116] while sending message body
Feb 12 10:39:15 mail1 postfix/smtp[15974]: 1D48E6A011B:
to=<omittedforpriva...@comcast.net>,
relay=mx2.comcast.net[76.96.30.116]:25, delay=108,
delays=3.5/0.01/64/41, dsn=4.4.2, status=deferred (lost connection with
mx2.comcast.net[76.96.30.116] while sending message body)
Feb 12 10:41:39 mail1 postfix/qmgr[30562]: 1D48E6A011B:
from=<omittedforpriv...@judelawfirm.com>, size=5299467, nrcpt=2 (queue
active)
Feb 12 10:50:41 mail1 postfix/error[16632]: 1D48E6A011B:
to=<omittedforpriva...@comcast.net>, relay=none, delay=793,
delays=252/542/0/0, dsn=4.4.2, status=deferred (delivery temporarily
suspended: conversation with mx2.comcast.net[76.96.30.116] timed out
while sending message body)
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
content_filter = smtp-amavis:[127.0.0.1]:10024
daemon_directory = /usr/libexec/postfix
debug_peer_level = 2
disable_vrfy_command = yes
html_directory = no
inet_interfaces = all
mail_owner = postfix
mailbox_size_limit = 0
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
maximal_backoff_time = 300s
message_size_limit = 0
minimal_backoff_time = 120s
mydestination = judelawfirm.com, mail1.judelawfirm.com, mail1.jude,
localhost, localhost.localdomain, localhost.judelawfirm.com
mydomain = judelawfirm.com
myhostname = mail1.judelawfirm.com
mynetworks = 127.0.0.0/8, 192.168.1.0/24
myorigin = mail1.judelawfirm.com
newaliases_path = /usr/bin/newaliases.postfix
queue_directory = /var/spool/postfix
queue_run_delay = 120s
readme_directory = /usr/share/doc/postfix-2.4.5/README_FILES
sample_directory = /usr/share/doc/postfix-2.4.5/samples
sender_bcc_maps = hash:/etc/aliases_bcc
sender_canonical_classes = header_sender
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtpd_banner = $myhostname ESMTP
smtpd_data_restrictions = reject_unauth_pipelining
permit_mynetworks permit_sasl_authenticated
smtpd_helo_required = yes
smtpd_helo_restrictions = reject_invalid_hostname
smtpd_recipient_restrictions = permit_sasl_authenticated
permit_mynetworks reject_unauth_destination
reject_unlisted_recipient check_sender_access
hash:/etc/postfix/sender_access reject_non_fqdn_recipient
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain =
smtpd_sasl_security_options = noanonymous
smtpd_sender_restrictions = reject_non_fqdn_sender
reject_unknown_sender_domain
smtpd_tls_CAfile = /etc/postfix/ssl/cacert.pem
smtpd_tls_auth_only = no
smtpd_tls_cert_file = /etc/postfix/ssl/smtpd.crt
smtpd_tls_key_file = /etc/postfix/ssl/smtpd.key
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
strict_rfc821_envelopes = yes
tls_random_source = dev:/dev/urandom
transport_maps = hash:/etc/postfix/transport
unknown_local_recipient_reject_code = 550
# openssl s_client -connect mx1.comcast.net:25 -CAfile ./ca-bundle.crt
-crlf -starttls smtp
CONNECTED(00000003)
depth=1 /C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=localhost
i:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
1 s:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
i:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICGDCCAYECAgEBMA0GCSqGSIb3DQEBBAUAMFcxCzAJBgNVBAYTAlVTMRMwEQYD
VQQKEwpSVEZNLCBJbmMuMRkwFwYDVQQLExBXaWRnZXRzIERpdmlzaW9uMRgwFgYD
VQQDEw9UZXN0IENBMjAwMTA1MTcwHhcNMDEwNTE3MTYxMDU5WhcNMDQwMzA2MTYx
MDU5WjBRMQswCQYDVQQGEwJVUzETMBEGA1UEChMKUlRGTSwgSW5jLjEZMBcGA1UE
CxMQV2lkZ2V0cyBEaXZpc2lvbjESMBAGA1UEAxMJbG9jYWxob3N0MIGfMA0GCSqG
SIb3DQEBAQUAA4GNADCBiQKBgQCiWhMjNOPlPLNW4DJFBiL2fFEIkHuRor0pKw25
J0ZYHW93lHQ4yxA6afQr99ayRjMY0D26pH41f0qjDgO4OXskBsaYOFzapSZtQMbT
97OCZ7aHtK8z0ZGNW/cslu+1oOLomgRxJomIFgW1RyUUkQP1n0hemtUdCLOLlO7Q
CPqZLQIDAQABMA0GCSqGSIb3DQEBBAUAA4GBAIumUwl1OoWuyN2xfoBHYAs+lRLY
KmFLoI5+iMcGxWIsksmA+b0FLRAN43wmhPnums8eXgYbDCrKLv2xWcvKDP3mps7m
AMivwtu/eFpYz6J8Mo1fsV4Ys08A/uPXkT23jyKo2hMu8mywkqXCXYF2e+7pEeBr
dsbmkWK5NgoMl8eM
-----END CERTIFICATE-----
subject=/C=US/O=RTFM, Inc./OU=Widgets Division/CN=localhost
issuer=/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
---
No client certificate CA names sent
---
SSL handshake has read 1758 bytes and written 341 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : DHE-RSA-AES256-SHA
Session-ID:
4DD332EA82819930BF3CBD36E71BDE0250D71C59311F0A5C0599C0A32152ABEF
Session-ID-ctx:
Master-Key:
1705F794600CFB4C57B7130941DB407BCA58DEE04A815D17CDECF3983BFDBB76D88363EA21C11D571019EDA1BD1425F8
Key-Arg : None
Krb5 Principal: None
Start Time: 1234554304
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
---
220 IMTA03.westchester.pa.mail.comcast.net comcast ESMTP server ready
# openssl s_client -connect mx2.comcast.net:25 -CAfile ./ca-bundle.crt
-crlf -starttls smtp
CONNECTED(00000003)
depth=1 /C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=localhost
i:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
1 s:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
i:/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICGDCCAYECAgEBMA0GCSqGSIb3DQEBBAUAMFcxCzAJBgNVBAYTAlVTMRMwEQYD
VQQKEwpSVEZNLCBJbmMuMRkwFwYDVQQLExBXaWRnZXRzIERpdmlzaW9uMRgwFgYD
VQQDEw9UZXN0IENBMjAwMTA1MTcwHhcNMDEwNTE3MTYxMDU5WhcNMDQwMzA2MTYx
MDU5WjBRMQswCQYDVQQGEwJVUzETMBEGA1UEChMKUlRGTSwgSW5jLjEZMBcGA1UE
CxMQV2lkZ2V0cyBEaXZpc2lvbjESMBAGA1UEAxMJbG9jYWxob3N0MIGfMA0GCSqG
SIb3DQEBAQUAA4GNADCBiQKBgQCiWhMjNOPlPLNW4DJFBiL2fFEIkHuRor0pKw25
J0ZYHW93lHQ4yxA6afQr99ayRjMY0D26pH41f0qjDgO4OXskBsaYOFzapSZtQMbT
97OCZ7aHtK8z0ZGNW/cslu+1oOLomgRxJomIFgW1RyUUkQP1n0hemtUdCLOLlO7Q
CPqZLQIDAQABMA0GCSqGSIb3DQEBBAUAA4GBAIumUwl1OoWuyN2xfoBHYAs+lRLY
KmFLoI5+iMcGxWIsksmA+b0FLRAN43wmhPnums8eXgYbDCrKLv2xWcvKDP3mps7m
AMivwtu/eFpYz6J8Mo1fsV4Ys08A/uPXkT23jyKo2hMu8mywkqXCXYF2e+7pEeBr
dsbmkWK5NgoMl8eM
-----END CERTIFICATE-----
subject=/C=US/O=RTFM, Inc./OU=Widgets Division/CN=localhost
issuer=/C=US/O=RTFM, Inc./OU=Widgets Division/CN=Test CA20010517
---
No client certificate CA names sent
---
SSL handshake has read 1757 bytes and written 341 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : DHE-RSA-AES256-SHA
Session-ID:
F9365A40A54CDC1591244CF614C42F6121CAD4ACEA5759BDC3D3D1F070E01889
Session-ID-ctx:
Master-Key:
7A79C457DD16E4AFB37912FC4C1A9E60D3A5CA6A40AE2FC34D4784E67C90D72783E1265D38BF167AD5271078F0CF8014
Key-Arg : None
Krb5 Principal: None
Start Time: 1234554337
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
---
220 IMTA16.emeryville.ca.mail.comcast.net comcast ESMTP server ready
