Hi William,
I am attaching a patch for squid-3.5 in the case you want to test it.
On 01/15/2016 06:17 PM, William Lima wrote:
Hi Christos,
Great news! I can also test your patch. About the other (Notes) leaks, they
still occur, although they are smaller if compared to the Cert Validation.
William
----- Mensagem original -----
De: "Christos Tsantilas" <[email protected]>
Para: "William Lima" <[email protected]>
Cc: [email protected]
Enviadas: Quinta-feira, 14 de Janeiro de 2016 17:33:26
Assunto: Re: [squid-dev] NotePairs, SSL and Cert Validation memory leaks
Hi William,
We had already recognise the reason of this leek and we are testing
localy a patch.
I am not able to reproduce Notes related memory leaks.
Are the other Notes related memory leaks disappeared for you using the
r13974?
On 01/14/2016 09:24 PM, William Lima wrote:
Christos,
The problem persists with the r13974.
15,620 (192 direct, 15,428 indirect) bytes in 4 blocks are definitely lost
in loss record 2,921 of 2,998
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642FD6: xcalloc (xalloc.cc:83)
by 0x63D582: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x29ACB5: cbdataInternalAlloc(int) (cbdata.cc:281)
by 0x4EA600:
Ssl::PeerConnector::sslCrtvdCheckForErrors(Ssl::CertValidationResponse const&,
Ssl::ErrorDetail*&) (CbDataList.h:37)
by 0x4EC2AC:
Ssl::PeerConnector::sslCrtvdHandleReply(RefCount<Ssl::CertValidationResponse>)
(PeerConnector.cc:489)
by 0x4EEEDE: UnaryMemFunT<Ssl::PeerConnector,
RefCount<Ssl::CertValidationResponse>, RefCount<Ssl::CertValidationResponse>
>::doDial() (AsyncJobCalls.h:121)
by 0x4EE59A: JobDialer<Ssl::PeerConnector>::dial(AsyncCall&)
(AsyncJobCalls.h:174)
by 0x4B479F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7E62: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B81EF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CE9B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31D067: EventLoop::run() (EventLoop.cc:82)
by 0x390A1D: SquidMain(int, char**) (main.cc:1539)
by 0x3917D7: main (main.cc:1263)
William
----- Mensagem original -----
De: "Christos Tsantilas" <[email protected]>
Para: [email protected]
Enviadas: Quarta-feira, 13 de Janeiro de 2016 14:24:11
Assunto: Re: [squid-dev] NotePairs, SSL and Cert Validation memory leaks
On 01/13/2016 05:47 PM, Alex Rousskov wrote:
On 01/13/2016 07:50 AM, William Lima wrote:
I'm using r13967.
If Christos is right, you should be using r13974.
Unfortunately the patch-r13967 is fixing some similar problems. This is
the last with similar fixes.
Alex.
----- Mensagem original -----
De: "Christos Tsantilas" <[email protected]>
Para: "William Lima" <[email protected]>,
[email protected]
Enviadas: Quarta-feira, 13 de Janeiro de 2016 12:28:57
Assunto: Re: [squid-dev] NotePairs, SSL and Cert Validation memory leaks
The latest squid-3.5.13 release should include fixes for these leaks.
Which squid version are you using?
On 01/11/2016 07:04 PM, William Lima wrote:
Hi all,
I have identified those memory leaks in the latest version of Squid 3.5:
128 (48 direct, 80 indirect) bytes in 1 blocks are definitely lost in loss
record 1,875 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A59D1: NotePairs::add(char const*, char const*) (Notes.h:211)
by 0x563C63: Helper::Reply::parse(char*, unsigned long) (Reply.cc:106)
by 0x563D6E: Helper::Reply::Reply(char*, unsigned long) (Reply.cc:23)
by 0x33DC72: helperStatefulHandleRead(RefCount<Comm::Connection>
const&, char*, unsigned long, Comm::Flag, int, void*) (helper.cc:1000)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
128 (48 direct, 80 indirect) bytes in 1 blocks are definitely lost in loss
record 1,876 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A59D1: NotePairs::add(char const*, char const*) (Notes.h:211)
by 0x562F01: Helper::Reply::parseResponseKeys() (Reply.cc:182)
by 0x5635AA: Helper::Reply::parse(char*, unsigned long) (Reply.cc:127)
by 0x563D6E: Helper::Reply::Reply(char*, unsigned long) (Reply.cc:23)
by 0x33F178: helperHandleRead(RefCount<Comm::Connection> const&, char*,
unsigned long, Comm::Flag, int, void*) (helper.cc:817)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
144 (96 direct, 48 indirect) bytes in 2 blocks are definitely lost in loss
record 1,917 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x29AB25: cbdataInternalAlloc(int) (cbdata.cc:281)
by 0x4F9220: ssl_verify_cb(int, x509_store_ctx_st*) (CbDataList.h:37)
by 0x5B85EB1: X509_verify_cert (x509_vfy.c:349)
by 0x5843087: ssl_verify_cert_chain (ssl_cert.c:554)
by 0x58221C2: ssl3_get_server_certificate (s3_clnt.c:1161)
by 0x5824831: ssl3_connect (s3_clnt.c:334)
by 0x582D676: ssl23_connect (s23_clnt.c:776)
by 0x4ED7DB: Ssl::PeerConnector::negotiateSsl() (PeerConnector.cc:248)
by 0x4EDF9A: JobDialer<Ssl::PeerConnector>::dial(AsyncCall&)
(AsyncJobCalls.h:174)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
192 (144 direct, 48 indirect) bytes in 3 blocks are definitely lost in
loss record 2,046 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x29AB25: cbdataInternalAlloc(int) (cbdata.cc:281)
by 0x4F902A: ssl_verify_cb(int, x509_store_ctx_st*) (CbDataList.h:37)
by 0x5B865CD: X509_verify_cert (x509_vfy.c:679)
by 0x5843087: ssl_verify_cert_chain (ssl_cert.c:554)
by 0x58221C2: ssl3_get_server_certificate (s3_clnt.c:1161)
by 0x5824831: ssl3_connect (s3_clnt.c:334)
by 0x582D676: ssl23_connect (s23_clnt.c:776)
by 0x4ED7DB: Ssl::PeerConnector::negotiateSsl() (PeerConnector.cc:248)
by 0x4EDF9A: JobDialer<Ssl::PeerConnector>::dial(AsyncCall&)
(AsyncJobCalls.h:174)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
5,000 (1,776 direct, 3,224 indirect) bytes in 37 blocks are definitely
lost in loss record 3,060 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A5AF7: NotePairs::append(NotePairs const*) (Notes.h:211)
by 0x327345: ExternalACLEntry::update(ExternalACLEntryData const&)
(ExternalACLEntry.cc:41)
by 0x322A3D: external_acl_cache_add(external_acl*, char const*,
ExternalACLEntryData const&) (external_acl.cc:1264)
by 0x3230E4: externalAclHandleReply(void*, Helper::Reply const&)
(external_acl.cc:1376)
by 0x33F189: helperHandleRead(RefCount<Comm::Connection> const&, char*,
unsigned long, Comm::Flag, int, void*) (helper.cc:818)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
6,368 (2,256 direct, 4,112 indirect) bytes in 47 blocks are definitely
lost in loss record 3,077 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A5AF7: NotePairs::append(NotePairs const*) (Notes.h:211)
by 0x322C2B: externalAclHandleReply(void*, Helper::Reply const&)
(external_acl.cc:1346)
by 0x33F189: helperHandleRead(RefCount<Comm::Connection> const&, char*,
unsigned long, Comm::Flag, int, void*) (helper.cc:818)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
8,088 (2,736 direct, 5,352 indirect) bytes in 57 blocks are definitely
lost in loss record 3,090 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A5EE9: NotePairs::appendNewOnly(NotePairs const*) (Notes.h:211)
by 0x48AD6C: Auth::Ntlm::UserRequest::HandleReply(void*, Helper::Reply
const&) (UserRequest.cc:274)
by 0x33DCB1: helperStatefulHandleRead(RefCount<Comm::Connection>
const&, char*, unsigned long, Comm::Flag, int, void*) (helper.cc:1002)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
15,551 (96 direct, 15,455 indirect) bytes in 2 blocks are definitely lost
in loss record 3,137 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x29AB25: cbdataInternalAlloc(int) (cbdata.cc:281)
by 0x4F902A: ssl_verify_cb(int, x509_store_ctx_st*) (CbDataList.h:37)
by 0x5B8643C: X509_verify_cert (x509_vfy.c:382)
by 0x5843087: ssl_verify_cert_chain (ssl_cert.c:554)
by 0x58221C2: ssl3_get_server_certificate (s3_clnt.c:1161)
by 0x5824831: ssl3_connect (s3_clnt.c:334)
by 0x582D676: ssl23_connect (s23_clnt.c:776)
by 0x4ED7DB: Ssl::PeerConnector::negotiateSsl() (PeerConnector.cc:248)
by 0x4EDF9A: JobDialer<Ssl::PeerConnector>::dial(AsyncCall&)
(AsyncJobCalls.h:174)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
15,810 (192 direct, 15,618 indirect) bytes in 4 blocks are definitely lost
in loss record 3,140 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x29AB25: cbdataInternalAlloc(int) (cbdata.cc:281)
by 0x4EA000:
Ssl::PeerConnector::sslCrtvdCheckForErrors(Ssl::CertValidationResponse const&,
Ssl::ErrorDetail*&) (CbDataList.h:37)
by 0x4EBCAC:
Ssl::PeerConnector::sslCrtvdHandleReply(RefCount<Ssl::CertValidationResponse>)
(PeerConnector.cc:489)
by 0x4EE8DE: UnaryMemFunT<Ssl::PeerConnector,
RefCount<Ssl::CertValidationResponse>, RefCount<Ssl::CertValidationResponse>
>::doDial() (AsyncJobCalls.h:121)
by 0x4EDF9A: JobDialer<Ssl::PeerConnector>::dial(AsyncCall&)
(AsyncJobCalls.h:174)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
17,432 (5,712 direct, 11,720 indirect) bytes in 119 blocks are definitely
lost in loss record 3,151 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A5AF7: NotePairs::append(NotePairs const*) (Notes.h:211)
by 0x327345: ExternalACLEntry::update(ExternalACLEntryData const&)
(ExternalACLEntry.cc:41)
by 0x3224FA: external_acl_cache_add(external_acl*, char const*,
ExternalACLEntryData const&) (external_acl.cc:1257)
by 0x3230E4: externalAclHandleReply(void*, Helper::Reply const&)
(external_acl.cc:1376)
by 0x33F189: helperHandleRead(RefCount<Comm::Connection> const&, char*,
unsigned long, Comm::Flag, int, void*) (helper.cc:818)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
36,456 (12,912 direct, 23,544 indirect) bytes in 269 blocks are definitely
lost in loss record 3,188 of 3,225
at 0x4C267BB: calloc (vg_replace_malloc.c:593)
by 0x642906: xcalloc (xalloc.cc:83)
by 0x63CEB2: MemPoolMalloc::allocate() (MemPoolMalloc.cc:39)
by 0x3A5AF7: NotePairs::append(NotePairs const*) (Notes.h:211)
by 0x3A5C91: UpdateRequestNotes(ConnStateData*, HttpRequest&, NotePairs
const&) (Notes.cc:275)
by 0x324708: ACLExternal::match(ACLChecklist*) (external_acl.cc:716)
by 0x4ABE07: ACL::matches(ACLChecklist*) const (Acl.cc:154)
by 0x4AF1DA: ACLChecklist::matchChild(Acl::InnerNode const*,
__gnu_cxx::__normal_iterator<ACL* const*, std::vector<ACL*, std::allocator<ACL*> >
>, ACL const*) (Checklist.cc:94)
by 0x4AE372: Acl::AndNode::doMatch(ACLChecklist*, __gnu_cxx::__normal_iterator<ACL*
const*, std::vector<ACL*, std::allocator<ACL*> > >) const (BoolOps.cc:89)
by 0x4B1953: Acl::InnerNode::resumeMatchingAt(ACLChecklist*,
__gnu_cxx::__normal_iterator<ACL* const*, std::vector<ACL*, std::allocator<ACL*> >
>) const (InnerNode.cc:95)
by 0x4AF08B: ACLChecklist::matchChild(Acl::InnerNode const*,
__gnu_cxx::__normal_iterator<ACL* const*, std::vector<ACL*, std::allocator<ACL*> >
>, ACL const*) (Checklist.cc:99)
by 0x4AE2EE: Acl::OrNode::doMatch(ACLChecklist*, __gnu_cxx::__normal_iterator<ACL*
const*, std::vector<ACL*, std::allocator<ACL*> > >) const (BoolOps.cc:133)
by 0x4B1953: Acl::InnerNode::resumeMatchingAt(ACLChecklist*,
__gnu_cxx::__normal_iterator<ACL* const*, std::vector<ACL*, std::allocator<ACL*> >
>) const (InnerNode.cc:95)
by 0x4AF7DC: ACLChecklist::matchAndFinish() (Checklist.cc:300)
by 0x4B0E27:
ACLChecklist::resumeNonBlockingCheck(ACLChecklist::AsyncState*)
(Checklist.cc:282)
by 0x322E0E: externalAclHandleReply(void*, Helper::Reply const&)
(external_acl.cc:1390)
by 0x33F189: helperHandleRead(RefCount<Comm::Connection> const&, char*,
unsigned long, Comm::Flag, int, void*) (helper.cc:818)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
98,125 (5,360 direct, 92,765 indirect) bytes in 670 blocks are definitely
lost in loss record 3,215 of 3,225
at 0x4C27A2E: malloc (vg_replace_malloc.c:270)
by 0x642864: xmalloc (xalloc.cc:114)
by 0x4FCE69: sslCrtvdHandleReplyWrapper(void*, Helper::Reply const&)
(SquidNew.h:23)
by 0x33F189: helperHandleRead(RefCount<Comm::Connection> const&, char*,
unsigned long, Comm::Flag, int, void*) (helper.cc:818)
by 0x4B419F: AsyncCall::make() (AsyncCall.cc:40)
by 0x4B7862: AsyncCallQueue::fireNext() (AsyncCallQueue.cc:56)
by 0x4B7BEF: AsyncCallQueue::fire() (AsyncCallQueue.cc:42)
by 0x31CD0B: EventLoop::runOnce() (EventLoop.cc:120)
by 0x31CED7: EventLoop::run() (EventLoop.cc:82)
by 0x39088D: SquidMain(int, char**) (main.cc:1539)
by 0x391647: main (main.cc:1263)
Does anyone have a clue about the NotePairs leaks?
Thanks in advance,
William Lima
--
Tsantilas Christos
Network and Systems Engineer
email:[email protected]
web:http://www.chtsanti.net
Phone:+30 6977678842
Cert Validation memory leaks
In the case SSL errors detected by certificate validator helper the objects
stored in Ssl::ServerBump::sslErrors member and will never released.
This member normally points to an Ssl::CertErrors list attached to the related
SSL object which is responsible to release this list.
When the cert validator detects errors a new errors list created and attached
to the related Ssl::ServerBump::sslErrors member but the SSL objects still hold
the old one. The old list released but not the new one.
This patch also fixes the case the cbdata protected Ssl::CertErrors list,
still is used through the related Ssl::ServerBump object but it is not valid
any more, because the SSL object which hold it gone.
This patch instead of storing the Ssl::CertErrors list to Ssl::ServerBump
object stores the SSL object and increases its reference to avoid be released
This is a Measurement Factory project
=== modified file 'src/acl/FilledChecklist.h'
--- src/acl/FilledChecklist.h 2016-01-01 00:14:27 +0000
+++ src/acl/FilledChecklist.h 2016-01-18 17:01:12 +0000
@@ -62,41 +62,41 @@
public:
Ip::Address src_addr;
Ip::Address dst_addr;
Ip::Address my_addr;
CachePeer *dst_peer;
char *dst_rdns;
HttpRequest *request;
HttpReply *reply;
char rfc931[USER_IDENT_SZ];
#if USE_AUTH
Auth::UserRequest::Pointer auth_user_request;
#endif
#if SQUID_SNMP
char *snmp_community;
#endif
#if USE_OPENSSL
/// SSL [certificate validation] errors, in undefined order
- Ssl::CertErrors *sslErrors;
+ const Ssl::CertErrors *sslErrors;
/// The peer certificate
Ssl::X509_Pointer serverCert;
#endif
AccessLogEntry::Pointer al; ///< info for the future access.log entry
ExternalACLEntryPointer extacl_entry;
private:
ConnStateData * conn_; /**< hack for ident and NTLM */
int fd_; /**< may be available when conn_ is not */
bool destinationDomainChecked_;
bool sourceDomainChecked_;
/// not implemented; will cause link failures if used
ACLFilledChecklist(const ACLFilledChecklist &);
/// not implemented; will cause link failures if used
ACLFilledChecklist &operator=(const ACLFilledChecklist &);
CBDATA_CLASS2(ACLFilledChecklist);
};
=== modified file 'src/client_side.cc'
--- src/client_side.cc 2016-01-01 00:14:27 +0000
+++ src/client_side.cc 2016-01-18 17:01:12 +0000
@@ -4015,41 +4015,41 @@
if (!sslServerBump) {
assert(port->signingCert.get());
certProperties.signWithX509.resetAndLock(port->signingCert.get());
if (port->signPkey.get())
certProperties.signWithPkey.resetAndLock(port->signPkey.get());
certProperties.signAlgorithm = Ssl::algSignTrusted;
return;
}
// In case of an error while connecting to the secure server, use a fake
// trusted certificate, with no mimicked fields and no adaptation
// algorithms. There is nothing we can mimic so we want to minimize the
// number of warnings the user will have to see to get to the error page.
assert(sslServerBump->entry);
if (sslServerBump->entry->isEmpty()) {
if (X509 *mimicCert = sslServerBump->serverCert.get())
certProperties.mimicCert.resetAndLock(mimicCert);
ACLFilledChecklist checklist(NULL, sslServerBump->request.getRaw(),
clientConnection != NULL ? clientConnection->rfc931 : dash_str);
- checklist.sslErrors = cbdataReference(sslServerBump->sslErrors);
+ checklist.sslErrors = cbdataReference(sslServerBump->sslErrors());
for (sslproxy_cert_adapt *ca = Config.ssl_client.cert_adapt; ca != NULL; ca = ca->next) {
// If the algorithm already set, then ignore it.
if ((ca->alg == Ssl::algSetCommonName && certProperties.setCommonName) ||
(ca->alg == Ssl::algSetValidAfter && certProperties.setValidAfter) ||
(ca->alg == Ssl::algSetValidBefore && certProperties.setValidBefore) )
continue;
if (ca->aclList && checklist.fastCheck(ca->aclList) == ACCESS_ALLOWED) {
const char *alg = Ssl::CertAdaptAlgorithmStr[ca->alg];
const char *param = ca->param;
// For parameterless CN adaptation, use hostname from the
// CONNECT request.
if (ca->alg == Ssl::algSetCommonName) {
if (!param)
param = sslConnectHostOrIp.termedBuf();
certProperties.commonName = param;
certProperties.setCommonName = true;
} else if (ca->alg == Ssl::algSetValidAfter)
=== modified file 'src/ssl/PeerConnector.cc'
--- src/ssl/PeerConnector.cc 2016-01-01 00:14:27 +0000
+++ src/ssl/PeerConnector.cc 2016-01-18 17:44:44 +0000
@@ -182,40 +182,43 @@
srvBio->recordInput(true);
srvBio->mode(csd->sslBumpMode);
}
}
} else {
// Set client SSL options
SSL_set_options(ssl, ::Config.ssl_client.parsedOptions);
// Use SNI TLS extension only when we connect directly
// to the origin server and we know the server host name.
const char *sniServer = NULL;
const bool redirected = request->flags.redirected && ::Config.onoff.redir_rewrites_host;
if (!hostName || redirected)
sniServer = !request->GetHostIsNumeric() ? request->GetHost() : NULL;
else
sniServer = hostName->c_str();
if (sniServer)
Ssl::setClientSNI(ssl, sniServer);
}
+
+ if (Ssl::ServerBump *serverBump = csd->serverBump())
+ serverBump->attachServerSSL(ssl);
}
// If CertValidation Helper used do not lookup checklist for errors,
// but keep a list of errors to send it to CertValidator
if (!Ssl::TheConfig.ssl_crt_validator) {
// Create the ACL check list now, while we have access to more info.
// The list is used in ssl_verify_cb() and is freed in ssl_free().
if (acl_access *acl = ::Config.ssl_client.cert_error) {
ACLFilledChecklist *check = new ACLFilledChecklist(acl, request.getRaw(), dash_str);
// check->fd(fd); XXX: need client FD here
SSL_set_ex_data(ssl, ssl_ex_index_cert_error_check, check);
}
}
// store peeked cert to check SQUID_X509_V_ERR_CERT_CHANGE
X509 *peeked_cert;
if (request->clientConnectionManager.valid() &&
request->clientConnectionManager->serverBump() &&
(peeked_cert = request->clientConnectionManager->serverBump()->serverCert.get())) {
CRYPTO_add(&(peeked_cert->references),1,CRYPTO_LOCK_X509);
@@ -302,48 +305,40 @@
csd->resetSslCommonName(Ssl::CommonHostName(serverCert.get()));
debugs(83, 5, "HTTPS server CN: " << csd->sslCommonName() <<
" bumped: " << *serverConnection());
}
}
}
bool
Ssl::PeerConnector::sslFinalized()
{
const int fd = serverConnection()->fd;
SSL *ssl = fd_table[fd].ssl;
// In the case the session is resuming, the certificates does not exist and
// we did not do any cert validation
if (resumingSession)
return true;
handleServerCertificate();
- if (ConnStateData *csd = request->clientConnectionManager.valid()) {
- if (Ssl::ServerBump *serverBump = csd->serverBump()) {
- // remember validation errors, if any
- if (Ssl::CertErrors *errs = static_cast<Ssl::CertErrors *>(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors)))
- serverBump->sslErrors = cbdataReference(errs);
- }
- }
-
if (Ssl::TheConfig.ssl_crt_validator) {
Ssl::CertValidationRequest validationRequest;
// WARNING: Currently we do not use any locking for any of the
// members of the Ssl::CertValidationRequest class. In this code the
// Ssl::CertValidationRequest object used only to pass data to
// Ssl::CertValidationHelper::submit method.
validationRequest.ssl = ssl;
validationRequest.domainName = request->GetHost();
if (Ssl::CertErrors *errs = static_cast<Ssl::CertErrors *>(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors)))
// validationRequest disappears on return so no need to cbdataReference
validationRequest.errors = errs;
else
validationRequest.errors = NULL;
try {
debugs(83, 5, "Sending SSL certificate for validation to ssl_crtvd.");
AsyncCall::Pointer call = asyncCall(83,5, "Ssl::PeerConnector::sslCrtvdHandleReply", Ssl::CertValidationHelper::CbDialer(this, &Ssl::PeerConnector::sslCrtvdHandleReply, NULL));
Ssl::CertValidationHelper::GetInstance()->sslSubmit(validationRequest, call);
return false;
} catch (const std::exception &e) {
debugs(83, DBG_IMPORTANT, "ERROR: Failed to compose ssl_crtvd " <<
@@ -453,81 +448,71 @@
{
if (const ConnStateData *csd = request->clientConnectionManager.valid()) {
const Ssl::BumpMode currentMode = csd->sslBumpMode;
if (currentMode == Ssl::bumpStare) {
debugs(83,5, "default to bumping after staring");
return Ssl::bumpBump;
}
debugs(83,5, "default to splicing after " << currentMode);
} else {
debugs(83,3, "default to splicing due to missing info");
}
return Ssl::bumpSplice;
}
void
Ssl::PeerConnector::sslCrtvdHandleReply(Ssl::CertValidationResponse::Pointer validationResponse)
{
Must(validationResponse != NULL);
- Ssl::CertErrors *errs = NULL;
Ssl::ErrorDetail *errDetails = NULL;
bool validatorFailed = false;
if (!Comm::IsConnOpen(serverConnection())) {
return;
}
debugs(83,5, request->GetHost() << " cert validation result: " << validationResponse->resultCode);
- if (validationResponse->resultCode == ::Helper::Error)
- errs = sslCrtvdCheckForErrors(*validationResponse, errDetails);
- else if (validationResponse->resultCode != ::Helper::Okay)
+ if (validationResponse->resultCode == ::Helper::Error) {
+ if (Ssl::CertErrors *errs = sslCrtvdCheckForErrors(*validationResponse, errDetails)) {
+ SSL *ssl = fd_table[serverConnection()->fd].ssl;
+ Ssl::CertErrors *oldErrs = static_cast<Ssl::CertErrors*>(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors));
+ SSL_set_ex_data(ssl, ssl_ex_index_ssl_errors, (void *)errs);
+ delete oldErrs;
+ }
+ } else if (validationResponse->resultCode != ::Helper::Okay)
validatorFailed = true;
if (!errDetails && !validatorFailed) {
serverCertificateVerified();
if (splice)
switchToTunnel(request.getRaw(), clientConn, serverConn);
else
callBack();
return;
}
ErrorState *anErr = NULL;
if (validatorFailed) {
anErr = new ErrorState(ERR_GATEWAY_FAILURE, Http::scInternalServerError, request.getRaw());
} else {
-
- // Check the list error with
- if (errDetails && request->clientConnectionManager.valid()) {
- // remember the server certificate from the ErrorDetail object
- if (Ssl::ServerBump *serverBump = request->clientConnectionManager->serverBump()) {
- // remember validation errors, if any
- if (errs) {
- if (serverBump->sslErrors)
- cbdataReferenceDone(serverBump->sslErrors);
- serverBump->sslErrors = cbdataReference(errs);
- }
- }
- }
-
anErr = new ErrorState(ERR_SECURE_CONNECT_FAIL, Http::scServiceUnavailable, request.getRaw());
anErr->detail = errDetails;
/*anErr->xerrno= Should preserved*/
}
bail(anErr);
if (serverConnection()->getPeer()) {
peerConnectFailed(serverConnection()->getPeer());
}
serverConn->close();
return;
}
/// Checks errors in the cert. validator response against sslproxy_cert_error.
/// The first honored error, if any, is returned via errDetails parameter.
/// The method returns all seen errors except SSL_ERROR_NONE as Ssl::CertErrors.
Ssl::CertErrors *
Ssl::PeerConnector::sslCrtvdCheckForErrors(Ssl::CertValidationResponse const &resp, Ssl::ErrorDetail *& errDetails)
{
Ssl::CertErrors *errs = NULL;
@@ -676,48 +661,43 @@
anErr->xerrno = sysErrNo;
Ssl::ErrorDetail *errFromFailure = (Ssl::ErrorDetail *)SSL_get_ex_data(ssl, ssl_ex_index_ssl_error_detail);
if (errFromFailure != NULL) {
// The errFromFailure is attached to the ssl object
// and will be released when ssl object destroyed.
// Copy errFromFailure to a new Ssl::ErrorDetail object
anErr->detail = new Ssl::ErrorDetail(*errFromFailure);
} else {
// server_cert can be NULL here
X509 *server_cert = SSL_get_peer_certificate(ssl);
anErr->detail = new Ssl::ErrorDetail(SQUID_ERR_SSL_HANDSHAKE, server_cert, NULL);
X509_free(server_cert);
}
if (ssl_lib_error != SSL_ERROR_NONE)
anErr->detail->setLibError(ssl_lib_error);
if (request->clientConnectionManager.valid()) {
// remember the server certificate from the ErrorDetail object
- if (Ssl::ServerBump *serverBump = request->clientConnectionManager->serverBump()) {
+ if (Ssl::ServerBump *serverBump = request->clientConnectionManager->serverBump())
serverBump->serverCert.resetAndLock(anErr->detail->peerCert());
- // remember validation errors, if any
- if (Ssl::CertErrors *errs = static_cast<Ssl::CertErrors*>(SSL_get_ex_data(ssl, ssl_ex_index_ssl_errors)))
- serverBump->sslErrors = cbdataReference(errs);
- }
-
// For intercepted connections, set the host name to the server
// certificate CN. Otherwise, we just hope that CONNECT is using
// a user-entered address (a host name or a user-entered IP).
const bool isConnectRequest = !request->clientConnectionManager->port->flags.isIntercepted();
if (request->flags.sslPeek && !isConnectRequest) {
if (X509 *srvX509 = anErr->detail->peerCert()) {
if (const char *name = Ssl::CommonHostName(srvX509)) {
request->SetHost(name);
debugs(83, 3, HERE << "reset request host: " << name);
}
}
}
}
bail(anErr);
}
void
Ssl::PeerConnector::bail(ErrorState *error)
{
=== modified file 'src/ssl/ServerBump.cc'
--- src/ssl/ServerBump.cc 2016-01-01 00:14:27 +0000
+++ src/ssl/ServerBump.cc 2016-01-18 17:48:25 +0000
@@ -1,52 +1,69 @@
/*
* Copyright (C) 1996-2016 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 33 Client-side Routines */
#include "squid.h"
#include "client_side.h"
+#include "globals.h"
#include "FwdState.h"
#include "ssl/ServerBump.h"
#include "Store.h"
#include "StoreClient.h"
#include "URL.h"
CBDATA_NAMESPACED_CLASS_INIT(Ssl, ServerBump);
Ssl::ServerBump::ServerBump(HttpRequest *fakeRequest, StoreEntry *e, Ssl::BumpMode md):
request(fakeRequest),
- sslErrors(NULL),
step(bumpStep1)
{
debugs(33, 4, HERE << "will peek at " << request->GetHost() << ':' << request->port);
act.step1 = md;
act.step2 = act.step3 = Ssl::bumpNone;
const char *uri = urlCanonical(request.getRaw());
if (e) {
entry = e;
entry->lock("Ssl::ServerBump");
} else
entry = storeCreateEntry(uri, uri, request->flags, request->method);
// We do not need to be a client because the error contents will be used
// later, but an entry without any client will trim all its contents away.
sc = storeClientListAdd(entry, this);
}
Ssl::ServerBump::~ServerBump()
{
debugs(33, 4, HERE << "destroying");
if (entry) {
debugs(33, 4, HERE << *entry);
storeUnregister(sc, entry, this);
entry->unlock("Ssl::ServerBump");
}
- cbdataReferenceDone(sslErrors);
}
+void
+Ssl::ServerBump::attachServerSSL(SSL *ssl)
+{
+ if (serverSSL.get())
+ return;
+
+ serverSSL.resetAndLock(ssl);
+}
+
+const Ssl::CertErrors *
+Ssl::ServerBump::sslErrors() const
+{
+ if (!serverSSL.get())
+ return NULL;
+
+ const Ssl::CertErrors *errs = static_cast<const Ssl::CertErrors*>(SSL_get_ex_data(serverSSL.get(), ssl_ex_index_ssl_errors));
+ return errs;
+}
=== modified file 'src/ssl/ServerBump.h'
--- src/ssl/ServerBump.h 2016-01-01 00:14:27 +0000
+++ src/ssl/ServerBump.h 2016-01-18 17:23:39 +0000
@@ -13,44 +13,48 @@
#include "base/CbcPointer.h"
#include "comm/forward.h"
#include "HttpRequest.h"
#include "ip/Address.h"
class ConnStateData;
class store_client;
namespace Ssl
{
/**
\ingroup ServerProtocolSSLAPI
* Maintains bump-server-first related information.
*/
class ServerBump
{
public:
explicit ServerBump(HttpRequest *fakeRequest, StoreEntry *e = NULL, Ssl::BumpMode mode = Ssl::bumpServerFirst);
~ServerBump();
+ void attachServerSSL(SSL *); ///< Sets the server SSL object
+ const Ssl::CertErrors *sslErrors() const; ///< SSL [certificate validation] errors
/// faked, minimal request; required by Client API
HttpRequest::Pointer request;
StoreEntry *entry; ///< for receiving Squid-generated error messages
- Ssl::X509_Pointer serverCert; ///< HTTPS server certificate
- Ssl::CertErrors *sslErrors; ///< SSL [certificate validation] errors
+ /// HTTPS server certificate. Maybe it is different than the one
+ /// it is stored in serverSSL object (error SQUID_X509_V_ERR_CERT_CHANGE)
+ Ssl::X509_Pointer serverCert;
struct {
Ssl::BumpMode step1; ///< The SSL bump mode at step1
Ssl::BumpMode step2; ///< The SSL bump mode at step2
Ssl::BumpMode step3; ///< The SSL bump mode at step3
} act; ///< bumping actions at various bumping steps
Ssl::BumpStep step; ///< The SSL bumping step
SBuf clientSni; ///< the SSL client SNI name
+ Ssl::SSL_Pointer serverSSL; ///< The SSL object on server side.
private:
store_client *sc; ///< dummy client to prevent entry trimming
CBDATA_CLASS2(ServerBump);
};
} // namespace Ssl
#endif
=== modified file 'src/ssl/gadgets.h'
--- src/ssl/gadgets.h 2016-01-01 00:14:27 +0000
+++ src/ssl/gadgets.h 2016-01-18 17:29:18 +0000
@@ -96,41 +96,41 @@
CtoCpp1(ASN1_INTEGER_free, ASN1_INTEGER *)
typedef TidyPointer<ASN1_INTEGER, ASN1_INTEGER_free_cpp> ASN1_INT_Pointer;
CtoCpp1(TXT_DB_free, TXT_DB *)
typedef TidyPointer<TXT_DB, TXT_DB_free_cpp> TXT_DB_Pointer;
CtoCpp1(X509_NAME_free, X509_NAME *)
typedef TidyPointer<X509_NAME, X509_NAME_free_cpp> X509_NAME_Pointer;
CtoCpp1(RSA_free, RSA *)
typedef TidyPointer<RSA, RSA_free_cpp> RSA_Pointer;
CtoCpp1(X509_REQ_free, X509_REQ *)
typedef TidyPointer<X509_REQ, X509_REQ_free_cpp> X509_REQ_Pointer;
CtoCpp1(SSL_CTX_free, SSL_CTX *)
typedef TidyPointer<SSL_CTX, SSL_CTX_free_cpp> SSL_CTX_Pointer;
CtoCpp1(SSL_free, SSL *)
-typedef TidyPointer<SSL, SSL_free_cpp> SSL_Pointer;
+typedef LockingPointer<SSL, SSL_free_cpp, CRYPTO_LOCK_SSL> SSL_Pointer;
CtoCpp1(DH_free, DH *);
typedef TidyPointer<DH, DH_free_cpp> DH_Pointer;
sk_free_wrapper(sk_X509_CRL, STACK_OF(X509_CRL) *, X509_CRL_free)
typedef TidyPointer<STACK_OF(X509_CRL), sk_X509_CRL_free_wrapper> X509_CRL_STACK_Pointer;
sk_free_wrapper(sk_X509_NAME, STACK_OF(X509_NAME) *, X509_NAME_free)
typedef TidyPointer<STACK_OF(X509_NAME), sk_X509_NAME_free_wrapper> X509_NAME_STACK_Pointer;
/**
\ingroup SslCrtdSslAPI
* Create 1024 bits rsa key.
*/
EVP_PKEY * createSslPrivateKey();
/**
\ingroup SslCrtdSslAPI
* Write private key and SSL certificate to memory.
*/
_______________________________________________
squid-dev mailing list
[email protected]
http://lists.squid-cache.org/listinfo/squid-dev
