[trafficserver] branch master updated (97489e6 -> 0b760e7)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 97489e6 Cleanup strategy debug logs (#8656) add 0b760e7 Make SSL writes more efficient when using dynamic record sizing (#8670) No new revisions were added by this update. Summary of changes: iocore/net/SSLNetVConnection.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[trafficserver] 01/01: Make SSL writes more efficient when using dynamic record sizing
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch hrtime in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit f7b8f63428e075a316d1cef9e4bd2f3dbdcb86f1 Author: Sudheer Vinukonda AuthorDate: Mon Feb 14 13:22:34 2022 -0800 Make SSL writes more efficient when using dynamic record sizing Avoid calling clock time during subsequent SSL writes since the clock does get updated in the event loop already. --- iocore/net/SSLNetVConnection.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index 63b5c33..2523895 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -773,7 +773,7 @@ SSLNetVConnection::load_buffer_and_write(int64_t towrite, MIOBufferAccessor &buf // Dynamic TLS record sizing ink_hrtime now = 0; if (SSLConfigParams::ssl_maxrecord == -1) { -now = Thread::get_hrtime_updated(); +now = Thread::get_hrtime(); int msec_since_last_write = ink_hrtime_diff_msec(now, sslLastWriteTime); if (msec_since_last_write > SSL_DEF_TLS_RECORD_MSEC_THRESHOLD) {
[trafficserver] branch hrtime created (now f7b8f63)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch hrtime in repository https://gitbox.apache.org/repos/asf/trafficserver.git. at f7b8f63 Make SSL writes more efficient when using dynamic record sizing This branch includes the following new commits: new f7b8f63 Make SSL writes more efficient when using dynamic record sizing The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[trafficserver] branch master updated (a20db3c -> 1177cc7)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from a20db3c Remove scrap log object dead code (#7935) add 1177cc7 Add support for Remap rule hit stats (#7936) No new revisions were added by this update. Summary of changes: proxy/http/HttpProxyServerMain.cc | 2 + proxy/http/remap/Makefile.am | 2 + .../{NextHopRoundRobin.h => RemapHitCount.cc} | 37 ++ .../http/remap/RemapHitCount.h | 8 ++-- proxy/http/remap/UrlMapping.cc | 7 proxy/http/remap/UrlMapping.h | 22 +++ proxy/http/remap/UrlMappingPathIndex.cc| 13 +++ proxy/http/remap/UrlMappingPathIndex.h | 1 + proxy/http/remap/UrlRewrite.cc | 45 ++ proxy/http/remap/UrlRewrite.h | 2 + 10 files changed, 119 insertions(+), 20 deletions(-) copy proxy/http/remap/{NextHopRoundRobin.h => RemapHitCount.cc} (56%) copy plugins/esi/test/print_funcs.h => proxy/http/remap/RemapHitCount.h (83%)
[trafficserver] branch master updated (f80ed73 -> f36cf6a)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from f80ed73 Do not delete the continuation twice (#7862) add f36cf6a Enforce HTTP parsing restrictions on HTTP versions supported (#7875) No new revisions were added by this update. Summary of changes: proxy/hdrs/HTTP.cc | 37 + proxy/hdrs/HTTP.h | 4 +++- proxy/http/HttpSM.cc | 4 proxy/http/HttpTransact.cc | 4 4 files changed, 40 insertions(+), 9 deletions(-)
[trafficserver] branch master updated: SSL Cert lookup using PP dest ip when ProxyProtocol is enabled (#7802)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 31a580d SSL Cert lookup using PP dest ip when ProxyProtocol is enabled (#7802) 31a580d is described below commit 31a580d0d89b2c141655ea167e4ad1b6b4e4973c Author: Sudheer Vinukonda AuthorDate: Tue May 11 21:30:55 2021 -0700 SSL Cert lookup using PP dest ip when ProxyProtocol is enabled (#7802) --- iocore/net/SSLNetVConnection.cc | 7 +++ iocore/net/SSLUtils.cc | 20 +++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index 18d0637..b6be2c6 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -446,6 +446,13 @@ SSLNetVConnection::read_raw_data() if (this->has_proxy_protocol(buffer, &r)) { Debug("proxyprotocol", "ssl has proxy protocol header"); set_remote_addr(get_proxy_protocol_src_addr()); + if (is_debug_tag_set("proxyprotocol")) { +IpEndpoint dst; +dst.sa = *(this->get_proxy_protocol_dst_addr()); +ip_port_text_buffer ipb1; +ats_ip_nptop(&dst, ipb1, sizeof(ipb1)); +Debug("proxyprotocol", "ssl_has_proxy_v1, dest IP received [%s]", ipb1); + } } else { Debug("proxyprotocol", "proxy protocol was enabled, but required header was not present in the " "transaction - closing connection"); diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc index 1d57f0b..508d11f 100644 --- a/iocore/net/SSLUtils.cc +++ b/iocore/net/SSLUtils.cc @@ -38,6 +38,7 @@ #include "P_OCSPStapling.h" #include "P_SSLSNI.h" #include "P_SSLConfig.h" +#include "ProxyProtocol.h" #include "SSLSessionCache.h" #include "SSLSessionTicket.h" #include "SSLDynlock.h" @@ -299,7 +300,24 @@ set_context_cert(SSL *ssl) IpEndpoint ip; int namelen = sizeof(ip); -if (0 == safe_getsockname(netvc->get_socket(), &ip.sa, &namelen)) { +if (netvc->get_is_proxy_protocol() && netvc->get_proxy_protocol_version() != ProxyProtocolVersion::UNDEFINED) { + ip.sa = *(netvc->get_proxy_protocol_dst_addr()); + ip_port_text_buffer ipb1; + ats_ip_nptop(&ip, ipb1, sizeof(ipb1)); + cc = lookup->find(ip); + if (is_debug_tag_set("proxyprotocol")) { +IpEndpoint src; +ip_port_text_buffer ipb2; +int ip_len = sizeof(src); + +if (0 != safe_getpeername(netvc->get_socket(), &src.sa, &ip_len)) { + Debug("proxyprotocol", "Failed to get src ip, errno = [%d]", errno); + return EVENT_ERROR; +} +ats_ip_nptop(&src, ipb2, sizeof(ipb2)); +Debug("proxyprotocol", "IP context is %p for [%s] -> [%s], default context %p", cc, ipb2, ipb1, lookup->defaultContext()); + } +} else if (0 == safe_getsockname(netvc->get_socket(), &ip.sa, &namelen)) { cc = lookup->find(ip); } if (cc) {
[trafficserver] branch master updated: fix DNS spike issue for TCP_RETRY mode (#7307)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new c1ed4a4 fix DNS spike issue for TCP_RETRY mode (#7307) c1ed4a4 is described below commit c1ed4a4a50ba4b9d32933c58c2c0eb53f3636076 Author: Xin Li <33378623+whut...@users.noreply.github.com> AuthorDate: Mon May 10 11:04:06 2021 -0700 fix DNS spike issue for TCP_RETRY mode (#7307) fix compiling Error rename metrics update the docs for the new config and metric Co-authored-by: xinli1 --- doc/admin-guide/files/records.config.en.rst| 7 +++ .../monitoring/statistics/core/dns.en.rst | 12 + iocore/dns/DNS.cc | 63 +- iocore/dns/P_DNSProcessor.h| 11 +++- mgmt/RecordsConfig.cc | 2 + 5 files changed, 92 insertions(+), 3 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 7ab3992..ac443fe 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -2602,6 +2602,13 @@ DNS ``2`` TCP_ONLY: |TS| always talks to nameservers over TCP. = == +.. ts:cv:: CONFIG proxy.config.dns.max_tcp_continuous_failures INT 10 + + If DNS connection mode is TCP_RETRY, set the threshold of the continuous TCP + query failures count for the TCP connection, reset the TCP connection immediately + if the continuous TCP query failures conut over the threshold. If the threshold + is 0 (or less than 0) we close this feature. + .. ts:cv:: CONFIG proxy.config.dns.max_dns_in_flight INT 2048 Maximum inflight DNS queries made by |TS| at any given instant diff --git a/doc/admin-guide/monitoring/statistics/core/dns.en.rst b/doc/admin-guide/monitoring/statistics/core/dns.en.rst index 0e0f93e..fed97d0 100644 --- a/doc/admin-guide/monitoring/statistics/core/dns.en.rst +++ b/doc/admin-guide/monitoring/statistics/core/dns.en.rst @@ -35,6 +35,18 @@ DNS The number of DNS lookups currently in progress. +.. ts:stat:: global proxy.process.dns.tcp_retries integer + :type: gauge + :ungathered: + + The number of DNS query over TCP in TCP_RETRY connection mode. + +.. ts:stat:: global proxy.process.dns.tcp_reset integer + :type: gauge + :ungathered: + + The number of resetting TCP connection in TCP_RETRY connection mode. + .. ts:stat:: global proxy.process.dns.lookup_avg_time integer :type: derivative :units: milliseconds diff --git a/iocore/dns/DNS.cc b/iocore/dns/DNS.cc index 74e247e..e7e9ebd 100644 --- a/iocore/dns/DNS.cc +++ b/iocore/dns/DNS.cc @@ -44,6 +44,7 @@ int dns_failover_number = DEFAULT_FAILOVER_NUMBER; int dns_failover_period = DEFAULT_FAILOVER_PERIOD; int dns_failover_try_period = DEFAULT_FAILOVER_TRY_PERIOD; int dns_max_dns_in_flight= MAX_DNS_IN_FLIGHT; +int dns_max_tcp_continuous_failures = MAX_DNS_TCP_CONTINUOUS_FAILURES; int dns_validate_qname = 0; unsigned int dns_handler_initialized = 0; int dns_ns_rr= 0; @@ -217,6 +218,7 @@ DNSProcessor::start(int, size_t stacksize) REC_EstablishStaticConfigInt32(dns_max_dns_in_flight, "proxy.config.dns.max_dns_in_flight"); REC_EstablishStaticConfigInt32(dns_validate_qname, "proxy.config.dns.validate_query_name"); REC_EstablishStaticConfigInt32(dns_ns_rr, "proxy.config.dns.round_robin_nameservers"); + REC_EstablishStaticConfigInt32(dns_max_tcp_continuous_failures, "proxy.config.dns.max_tcp_continuous_failures"); REC_ReadConfigStringAlloc(dns_ns_list, "proxy.config.dns.nameservers"); REC_ReadConfigStringAlloc(dns_local_ipv4, "proxy.config.dns.local_ipv4"); REC_ReadConfigStringAlloc(dns_local_ipv6, "proxy.config.dns.local_ipv6"); @@ -459,6 +461,17 @@ DNSHandler::open_cons(sockaddr const *target, bool failed, int icon) } /** + Close the old TCP connection and open a new one + */ +bool +DNSHandler::reset_tcp_conn(int ndx) +{ + DNS_INCREMENT_DYN_STAT(dns_tcp_reset_stat); + tcpcon[ndx].close(); + return open_con(&m_res->nsaddr_list[ndx].sa, true, ndx, true); +} + +/** Open (and close) connections as necessary and also assures that the epoll fd struct is properly updated. @@ -472,11 +485,12 @@ DNSHandler::open_cons(sockaddr const *target, bool failed, int icon) target != nullptr and icon > 0 : open connection to target. */ -void +bool DNSHandler::open_con(sockaddr const *target, bool failed, int icon, bool over_tcp) { ip_port_text_buffer ip_text; PollDescriptor *pd = get_PollDescriptor(dnsProcessor.thread)
[trafficserver] branch master updated (6009f46 -> d7847f2)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6009f46 DNS: Clean up argument passing to DNS queries. (#7778) add d7847f2 Short circuit remap reload when a valid remap file is not specified (#7782) No new revisions were added by this update. Summary of changes: proxy/ReverseProxy.cc | 5 +++-- proxy/http/remap/RemapConfig.cc | 12 +++- proxy/http/remap/UrlRewrite.cc | 6 -- 3 files changed, 10 insertions(+), 13 deletions(-)
[trafficserver] branch master updated: Elevate privileges for traffic_manager during SSL cert reload (#7770)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new ee11108 Elevate privileges for traffic_manager during SSL cert reload (#7770) ee11108 is described below commit ee11108158ee3d8aea4532ae18a5b4405a73b092 Author: Sudheer Vinukonda AuthorDate: Mon May 3 15:14:45 2021 -0700 Elevate privileges for traffic_manager during SSL cert reload (#7770) traffic_manager now watches the raw SSL certs as well and needs the privs to stat those files. --- src/traffic_manager/AddConfigFilesHere.cc | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/traffic_manager/AddConfigFilesHere.cc b/src/traffic_manager/AddConfigFilesHere.cc index 3fdf572..4e49476 100644 --- a/src/traffic_manager/AddConfigFilesHere.cc +++ b/src/traffic_manager/AddConfigFilesHere.cc @@ -45,14 +45,14 @@ testcall(char *foo, char * /*configName */) } void -registerFile(const char *configName, const char *defaultName, bool isRequired) +registerFile(const char *configName, const char *defaultName, bool isRequired, bool isElevateNeeded = false) { bool found= false; const char *fname = REC_readString(configName, &found); if (!found) { fname = defaultName; } - configFiles->addFile(fname, configName, false, isRequired); + configFiles->addFile(fname, configName, isElevateNeeded, isRequired); } // @@ -87,7 +87,9 @@ initializeRegistry() registerFile("proxy.config.cache.hosting_filename", ts::filename::HOSTING, NOT_REQUIRED); registerFile("", ts::filename::PLUGIN, NOT_REQUIRED); registerFile("proxy.config.dns.splitdns.filename", ts::filename::SPLITDNS, NOT_REQUIRED); - registerFile("proxy.config.ssl.server.multicert.filename", ts::filename::SSL_MULTICERT, NOT_REQUIRED); + uint32_t elevate_setting = 0; + REC_ReadConfigInteger(elevate_setting, "proxy.config.ssl.cert.load_elevated"); + registerFile("proxy.config.ssl.server.multicert.filename", ts::filename::SSL_MULTICERT, NOT_REQUIRED, elevate_setting); registerFile("proxy.config.ssl.servername.filename", ts::filename::SNI, NOT_REQUIRED); configFiles->registerCallback(testcall);
[trafficserver] branch master updated: 7096: Synchronize Server Session Management and Network I/O (#7278)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new a2d1515 7096: Synchronize Server Session Management and Network I/O (#7278) a2d1515 is described below commit a2d15151a5c1a69a3826e086ac497a94a4bfa7ea Author: Sudheer Vinukonda AuthorDate: Thu Oct 15 17:03:36 2020 -0700 7096: Synchronize Server Session Management and Network I/O (#7278) 1. Session Acquisition with global session pools - Ensure that the continuation and mutex is preserved through the entire session context migration mechanism until the SM is attached to the new VC 2. Session Release - Delay clearing of read and write io buffers to protect against race between do_io_close and net_read_io 3. Fix dangling ssl->vc reference for TLS Session Resumption --- iocore/net/I_NetVConnection.h | 11 +++ iocore/net/P_UnixNetVConnection.h | 3 ++ iocore/net/SSLNetVConnection.cc | 2 -- iocore/net/SSLUtils.cc| 2 ++ iocore/net/TLSSessionResumptionSupport.cc | 6 iocore/net/TLSSessionResumptionSupport.h | 1 + iocore/net/UnixNetVConnection.cc | 54 --- proxy/http/Http1ServerSession.cc | 7 ++-- proxy/http/HttpSM.cc | 11 +-- proxy/http/HttpSessionManager.cc | 27 10 files changed, 92 insertions(+), 32 deletions(-) diff --git a/iocore/net/I_NetVConnection.h b/iocore/net/I_NetVConnection.h index 0609e18..fac5762 100644 --- a/iocore/net/I_NetVConnection.h +++ b/iocore/net/I_NetVConnection.h @@ -387,6 +387,12 @@ public: */ VIO *do_io_read(Continuation *c, int64_t nbytes, MIOBuffer *buf) override = 0; + virtual Continuation * + read_vio_cont() + { +return nullptr; + } + /** Initiates write. Thread-safe, may be called when not handling an event from the NetVConnection, or the NetVConnection creation @@ -423,6 +429,11 @@ public: */ VIO *do_io_write(Continuation *c, int64_t nbytes, IOBufferReader *buf, bool owner = false) override = 0; + virtual Continuation * + write_vio_cont() + { +return nullptr; + } /** Closes the vconnection. A state machine MUST call do_io_close() when it has finished with a VConnection. do_io_close() indicates diff --git a/iocore/net/P_UnixNetVConnection.h b/iocore/net/P_UnixNetVConnection.h index cd94178..1d61200 100644 --- a/iocore/net/P_UnixNetVConnection.h +++ b/iocore/net/P_UnixNetVConnection.h @@ -111,6 +111,9 @@ public: VIO *do_io_read(Continuation *c, int64_t nbytes, MIOBuffer *buf) override; VIO *do_io_write(Continuation *c, int64_t nbytes, IOBufferReader *buf, bool owner = false) override; + Continuation *read_vio_cont() override; + Continuation *write_vio_cont() override; + bool get_data(int id, void *data) override; Action *send_OOB(Continuation *cont, char *buf, int len) override; diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index f4f656d..ff39b45 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -212,7 +212,6 @@ make_ssl_connection(SSL_CTX *ctx, SSLNetVConnection *netvc) } SSLNetVCAttach(ssl, netvc); -TLSSessionResumptionSupport::bind(ssl, netvc); } return ssl; @@ -1821,7 +1820,6 @@ SSLNetVConnection::populate(Connection &con, Continuation *c, void *arg) sslHandshakeStatus = SSL_HANDSHAKE_DONE; SSLNetVCAttach(this->ssl, this); - TLSSessionResumptionSupport::bind(this->ssl, this); return EVENT_DONE; } diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc index 442aec3..79be614 100644 --- a/iocore/net/SSLUtils.cc +++ b/iocore/net/SSLUtils.cc @@ -1700,12 +1700,14 @@ void SSLNetVCAttach(SSL *ssl, SSLNetVConnection *vc) { SSL_set_ex_data(ssl, ssl_vc_index, vc); + TLSSessionResumptionSupport::bind(ssl, vc); } void SSLNetVCDetach(SSL *ssl) { SSL_set_ex_data(ssl, ssl_vc_index, nullptr); + TLSSessionResumptionSupport::unbind(ssl); } SSLNetVConnection * diff --git a/iocore/net/TLSSessionResumptionSupport.cc b/iocore/net/TLSSessionResumptionSupport.cc index 8935c99..ab6f38e 100644 --- a/iocore/net/TLSSessionResumptionSupport.cc +++ b/iocore/net/TLSSessionResumptionSupport.cc @@ -75,6 +75,12 @@ TLSSessionResumptionSupport::bind(SSL *ssl, TLSSessionResumptionSupport *srs) SSL_set_ex_data(ssl, _ex_data_index, srs); } +void +TLSSessionResumptionSupport::unbind(SSL *ssl) +{ + SSL_set_ex_data(ssl, _ex_data_index, nullptr); +} + int TLSSessionResumptionSupport::processSessionTicket(SSL *ssl, unsigned char *keyname, unsigned char *iv, EVP_CIPHER_CTX *cipher_ctx, HMAC_CTX *hctx, int enc) diff --git a/iocore/net/TLSSessionResumptionSupport.h
[trafficserver] branch master updated (3b6cccf -> d945b42)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 3b6cccf add a null check to avoid crashing (#7035) add d945b42 Fix code to eliminate warning and enable feature (#7031) No new revisions were added by this update. Summary of changes: mgmt/RecordsConfig.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[trafficserver] branch master updated: Remove incorrect assert in inactivity timeout handling (#7012)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 4137029 Remove incorrect assert in inactivity timeout handling (#7012) 4137029 is described below commit 41370296d76ba13ef5de65eb4ed9d0fbeeb6f673 Author: Sudheer Vinukonda AuthorDate: Fri Jul 17 07:05:18 2020 -0700 Remove incorrect assert in inactivity timeout handling (#7012) Also fix duplicate decrement of current client connection metric --- proxy/http/Http1ClientSession.cc | 16 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/proxy/http/Http1ClientSession.cc b/proxy/http/Http1ClientSession.cc index 9ddd708..2005be4 100644 --- a/proxy/http/Http1ClientSession.cc +++ b/proxy/http/Http1ClientSession.cc @@ -88,8 +88,13 @@ Http1ClientSession::release_transaction() released_transactions++; if (transact_count == released_transactions) { // Make sure we previously called release() or do_io_close() on the session -ink_release_assert(read_state != HCS_ACTIVE_READER && read_state != HCS_INIT); -destroy(); +ink_release_assert(read_state != HCS_INIT); +if (read_state == HCS_ACTIVE_READER) { + // (in)active timeout + do_io_close(HTTP_ERRNO); +} else { + destroy(); +} } } @@ -254,10 +259,13 @@ Http1ClientSession::do_io_close(int alerrno) // READ_READY event. _reader->consume(_reader->read_avail()); } else { -read_state = HCS_CLOSED; HttpSsnDebug("[%" PRId64 "] session closed", con_id); HTTP_SUM_DYN_STAT(http_transactions_per_client_con, transact_count); -HTTP_DECREMENT_DYN_STAT(http_current_client_connections_stat); +if (read_state != HCS_ACTIVE_READER) { + // donot double decrement + HTTP_DECREMENT_DYN_STAT(http_current_client_connections_stat); +} +read_state= HCS_CLOSED; conn_decrease = false; // Can go ahead and close the netvc now, but keeping around the session object // until all the transactions are closed
[trafficserver] branch master updated: RateLimiting and Connection Config changes (#6968)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new e2a0d8c RateLimiting and Connection Config changes (#6968) e2a0d8c is described below commit e2a0d8cf580b0e954848fbc645a77040ebcbd1f3 Author: Sudheer Vinukonda AuthorDate: Thu Jul 2 09:29:01 2020 -0700 RateLimiting and Connection Config changes (#6968) Conn config renaming to support a protocol agnostic rate limiter (using request concurrency as opposed to active connections) --- doc/admin-guide/files/records.config.en.rst| 27 .../monitoring/statistics/core/network-io.en.rst | 2 +- iocore/net/Net.cc | 4 +-- iocore/net/P_Net.h | 2 +- iocore/net/P_UnixNet.h | 6 ++-- iocore/net/UnixNet.cc | 36 ++ mgmt/RecordsConfig.cc | 2 +- 7 files changed, 39 insertions(+), 40 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 999f21a..9c24300 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -414,19 +414,20 @@ Network .. ts:cv:: CONFIG proxy.config.net.max_connections_in INT 3 - The total number of client connections that the :program:`traffic_server` - can handle simultaneously. This should be tuned according to your memory size, - and expected work load (network, cpu etc). This limit includes both keepalive - and active client connections that :program:`traffic_server` can handle at - any given instant. - -.. ts:cv:: CONFIG proxy.config.net.max_active_connections_in INT 1 - - The total number of active client connections that the |TS| can handle - simultaneously. This should be tuned according to your memory size, - and expected work load (network, cpu etc). If this is set to 0, active - connection tracking is disabled and active connections have no separate - limit and the total connections follow `proxy.config.net.connections_throttle` + The total number of client requests that |TS| can handle simultaneously. + This should be tuned according to your memory size, and expected work load + (network, cpu etc). This limit includes both idle (keep alive) connections + and active requests that |TS| can handle at any given instant. The delta + between `proxy.config.net.max_connections_in` and `proxy.config.net.max_requests_in` + is the amount of maximum idle (keepalive) connections |TS| will maintain. + +.. ts:cv:: CONFIG proxy.config.net.max_requests_in INT 0 + + The total number of concurrent requests or active client connections + that the |TS| can handle simultaneously. This should be tuned according + to your memory size, and expected work load (network, cpu etc). When + set to 0, active request tracking is disabled and max requests has no + separate limit and the total connections follow `proxy.config.net.connections_throttle` .. ts:cv:: CONFIG proxy.config.net.default_inactivity_timeout INT 86400 :reloadable: diff --git a/doc/admin-guide/monitoring/statistics/core/network-io.en.rst b/doc/admin-guide/monitoring/statistics/core/network-io.en.rst index 3777359..5ea7687 100644 --- a/doc/admin-guide/monitoring/statistics/core/network-io.en.rst +++ b/doc/admin-guide/monitoring/statistics/core/network-io.en.rst @@ -66,7 +66,7 @@ Network I/O .. ts:stat:: global proxy.process.net.connections_throttled_out integer :type: counter -.. ts:stat:: global proxy.process.net.max.active.connections_throttled_in integer +.. ts:stat:: global proxy.process.net.max.requests_throttled_in integer :type: counter .. ts:stat:: global proxy.process.net.default_inactivity_timeout_applied integer diff --git a/iocore/net/Net.cc b/iocore/net/Net.cc index 2ee21f9..f0cac5b 100644 --- a/iocore/net/Net.cc +++ b/iocore/net/Net.cc @@ -142,8 +142,8 @@ register_net_stats() (int)net_connections_throttled_in_stat, RecRawStatSyncSum); RecRegisterRawStat(net_rsb, RECT_PROCESS, "proxy.process.net.connections_throttled_out", RECD_INT, RECP_PERSISTENT, (int)net_connections_throttled_out_stat, RecRawStatSyncSum); - RecRegisterRawStat(net_rsb, RECT_PROCESS, "proxy.process.net.max.active.connections_throttled_in", RECD_INT, RECP_PERSISTENT, - (int)net_connections_max_active_throttled_in_stat, RecRawStatSyncSum); + RecRegisterRawStat(net_rsb, RECT_PROCESS, "proxy.process.net.max.requests_throttled_in", RECD_INT, RECP_PERSISTENT, + (int)net_requests_max_throttled_in_stat, RecRawStatSyncSum); } void diff --git a/iocore/net/P_Net.h b/iocore/net/
[trafficserver] branch master updated: Update docs for some DNS config settings (#6969)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new e5fcd1d Update docs for some DNS config settings (#6969) e5fcd1d is described below commit e5fcd1d415a96627917d03da47b86118751df7a5 Author: Sudheer Vinukonda AuthorDate: Wed Jul 1 16:17:55 2020 -0700 Update docs for some DNS config settings (#6969) --- doc/admin-guide/files/records.config.en.rst | 12 1 file changed, 12 insertions(+) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index a32fc7c..999f21a 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -2554,6 +2554,18 @@ DNS ``2`` TCP_ONLY: |TS| always talks to nameservers over TCP. = == +.. ts:cv:: CONFIG proxy.config.dns.max_dns_in_flight INT 2048 + + Maximum inflight DNS queries made by |TS| at any given instant + +.. ts:cv:: CONFIG proxy.config.dns.lookup_timeout INT 20 + + Time to wait for a DNS response in seconds. + +.. ts:cv:: CONFIG proxy.config.dns.retries INT 5 + + Maximum number of retries made by |TS| on a given DNS query + HostDB ==
[trafficserver] branch master updated (c81215c -> 524d2e4)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from c81215c Fix format string for int64_t (#6963) add 524d2e4 Assert on valid boundaries for UserArgTable access (#6953) No new revisions were added by this update. Summary of changes: include/tscore/PluginUserArgs.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
[trafficserver] branch master updated (6ff0f48 -> ad49416)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6ff0f48 Make HostDBInfo class safer to use. (#6858) add ad49416 Prevent buffer overflow during log filter actions (#6950) No new revisions were added by this update. Summary of changes: proxy/logging/LogAccess.cc | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-)
[trafficserver] branch master updated (6d9d1ba -> d0adba9)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6d9d1ba Update buffer-writer.en.rst (wrong header link) (#6934) add d0adba9 Prevent use-after-free of TransactionPlugin (#6937) No new revisions were added by this update. Summary of changes: include/tscpp/api/TransactionPlugin.h | 2 ++ src/tscpp/api/TransactionPlugin.cc| 6 ++ src/tscpp/api/utils_internal.cc | 16 ++-- src/tscpp/api/utils_internal.h| 2 +- 4 files changed, 19 insertions(+), 7 deletions(-)
[trafficserver] branch master updated (6d9d1ba -> d0adba9)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6d9d1ba Update buffer-writer.en.rst (wrong header link) (#6934) add d0adba9 Prevent use-after-free of TransactionPlugin (#6937) No new revisions were added by this update. Summary of changes: include/tscpp/api/TransactionPlugin.h | 2 ++ src/tscpp/api/TransactionPlugin.cc| 6 ++ src/tscpp/api/utils_internal.cc | 16 ++-- src/tscpp/api/utils_internal.h| 2 +- 4 files changed, 19 insertions(+), 7 deletions(-)
[trafficserver] branch master updated (6d9d1ba -> d0adba9)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6d9d1ba Update buffer-writer.en.rst (wrong header link) (#6934) add d0adba9 Prevent use-after-free of TransactionPlugin (#6937) No new revisions were added by this update. Summary of changes: include/tscpp/api/TransactionPlugin.h | 2 ++ src/tscpp/api/TransactionPlugin.cc| 6 ++ src/tscpp/api/utils_internal.cc | 16 ++-- src/tscpp/api/utils_internal.h| 2 +- 4 files changed, 19 insertions(+), 7 deletions(-)
[trafficserver] branch master updated: Prevent stale netvc access on SSL Callbacks (#6925)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new c7e8054 Prevent stale netvc access on SSL Callbacks (#6925) c7e8054 is described below commit c7e80542aa1a5323399226f636ef196955f60791 Author: Sudheer Vinukonda AuthorDate: Mon Jun 22 05:44:47 2020 -0700 Prevent stale netvc access on SSL Callbacks (#6925) Since SSL Callbacks are asynchronous in nature, it's possible the associated NetVC is already freed causing a potential use-after-free problem. --- iocore/net/SSLNetVConnection.cc | 4 ++-- iocore/net/SSLUtils.cc | 36 +++- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index 8d19880..5209a93 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -1505,7 +1505,7 @@ SSLNetVConnection::advertise_next_protocol(SSL *ssl, const unsigned char **out, { SSLNetVConnection *netvc = SSLNetVCAccess(ssl); - ink_release_assert(netvc != nullptr); + ink_release_assert(netvc && netvc->ssl == ssl); if (netvc->getNPN(out, outlen)) { // Successful return tells OpenSSL to advertise. @@ -1522,7 +1522,7 @@ SSLNetVConnection::select_next_protocol(SSL *ssl, const unsigned char **out, uns { SSLNetVConnection *netvc = SSLNetVCAccess(ssl); - ink_release_assert(netvc != nullptr); + ink_release_assert(netvc && netvc->ssl == ssl); const unsigned char *npnptr = nullptr; unsigned int npnsize= 0; if (netvc->getNPN(&npnptr, &npnsize)) { diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc index 9387a65..56fae1d 100644 --- a/iocore/net/SSLUtils.cc +++ b/iocore/net/SSLUtils.cc @@ -250,6 +250,12 @@ set_context_cert(SSL *ssl) bool found = true; int retval = 1; + if (!netvc || netvc->ssl != ssl) { +Debug("ssl.error", "set_context_cert call back on stale netvc"); +retval = 0; // Error +goto done; + } + Debug("ssl", "set_context_cert ssl=%p server=%s handshake_complete=%d", ssl, servername, netvc->getSSLHandShakeComplete()); // catch the client renegotiation early on @@ -317,6 +323,11 @@ ssl_verify_client_callback(int preverify_ok, X509_STORE_CTX *ctx) auto *ssl= static_cast(X509_STORE_CTX_get_ex_data(ctx, SSL_get_ex_data_X509_STORE_CTX_idx())); SSLNetVConnection *netvc = SSLNetVCAccess(ssl); + if (!netvc || netvc->ssl != ssl) { +Debug("ssl.error", "ssl_verify_client_callback call back on stale netvc"); +return false; + } + netvc->set_verify_cert(ctx); netvc->callHooks(TS_EVENT_SSL_VERIFY_CLIENT); netvc->set_verify_cert(nullptr); @@ -355,6 +366,12 @@ ssl_client_hello_callback(SSL *s, int *al, void *arg) const char *servername = nullptr; const unsigned char *p; size_t remaining, len; + + if (!netvc || netvc->ssl != s) { +Debug("ssl.error", "ssl_client_hello_callback call back on stale netvc"); +return SSL_CLIENT_HELLO_ERROR; + } + // Parse the server name if the get extension call succeeds and there are more than 2 bytes to parse if (SSL_client_hello_get0_ext(s, TLSEXT_TYPE_server_name, &p, &remaining) && remaining > 2) { // Parse to get to the name, originally from test/handshake_helper.c in openssl tree @@ -414,6 +431,11 @@ ssl_cert_callback(SSL *ssl, void * /*arg*/) bool reenabled; int retval = 1; + if (!netvc || netvc->ssl != ssl) { +Debug("ssl.error", "ssl_cert_callback call back on stale netvc"); +return 0; + } + // If we are in tunnel mode, don't select a cert. Pause! if (HttpProxyPort::TRANSPORT_BLIND_TUNNEL == netvc->attributes) { return -1; // Pause @@ -447,6 +469,12 @@ static int ssl_servername_callback(SSL *ssl, int * /* ad */, void * /*arg*/) { SSLNetVConnection *netvc = SSLNetVCAccess(ssl); + + if (!netvc || netvc->ssl != ssl) { +Debug("ssl.error", "ssl_servername_callback call back on stale netvc"); +return SSL_TLSEXT_ERR_ALERT_FATAL; + } + netvc->callHooks(TS_EVENT_SSL_SERVERNAME); const char *name = SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name); @@ -1019,7 +1047,12 @@ ssl_callback_info(const SSL *ssl, int where, int ret) SSLNetVConnection *netvc = SSLNetVCAccess(ssl); - if (netvc && (where & SSL_CB_ACCEPT_LOOP) && netvc->getSSLHandShakeComplete() == true && + if (!netvc || netvc->ssl != ssl) { +Debug("ssl.error", "ssl_callback_info call back on stale netvc"); +return; + } + + if ((where & SSL_CB_ACCEPT_LOO
[trafficserver] branch master updated: Customize Max IOBuffer Size (#6869)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 6239297 Customize Max IOBuffer Size (#6869) 6239297 is described below commit 6239297fe7cf448915d41f2298f402e24520d896 Author: Sudheer Vinukonda AuthorDate: Thu Jun 18 18:23:35 2020 -0700 Customize Max IOBuffer Size (#6869) * Customize Max IOBuffer Size. Allow callers explicitly configure and pass in the max buffer sizes when asking for unknown sized memory chunks depending on the usecase instead of relying on a common/global default config. - Remove proxy.config.io.max_buffer_size - Add docs for the new configs --- doc/admin-guide/files/records.config.en.rst| 25 iocore/cache/CachePages.cc | 2 +- iocore/cache/CacheTest.cc | 4 +- iocore/eventsystem/EventSystem.cc | 13 +- iocore/eventsystem/I_IOBuffer.h| 23 +-- iocore/eventsystem/P_IOBuffer.h| 8 +--- iocore/eventsystem/unit_tests/test_IOBuffer.cc | 22 -- iocore/hostdb/HostDB.cc| 3 ++ iocore/hostdb/I_HostDBProcessor.h | 4 +- iocore/net/NetVCTest.cc| 4 +- iocore/net/P_SSLConfig.h | 1 + iocore/net/P_SSLNetVConnection.h | 3 +- iocore/net/QUICNetVConnection.cc | 4 +- iocore/net/QUICPacketHandler.cc| 2 +- iocore/net/SSLConfig.cc| 3 ++ iocore/net/SSLNextProtocolAccept.cc| 5 ++- iocore/net/Socks.cc| 2 +- iocore/net/quic/QUICFrame.cc | 47 +++--- iocore/net/quic/QUICPacketPayloadProtector.cc | 4 +- iocore/net/quic/test/test_QUICFrame.cc | 18 - iocore/net/quic/test/test_QUICFrameDispatcher.cc | 2 +- .../net/quic/test/test_QUICFrameRetransmitter.cc | 6 +-- .../net/quic/test/test_QUICIncomingFrameBuffer.cc | 12 +++--- iocore/net/quic/test/test_QUICStreamManager.cc | 10 ++--- iocore/utils/OneWayMultiTunnel.cc | 4 +- iocore/utils/OneWayTunnel.cc | 4 +- mgmt/RecordsConfig.cc | 10 - plugins/experimental/memcache/tsmemcache.cc| 6 +-- proxy/Transform.cc | 8 ++-- proxy/http/HttpConfig.cc | 4 ++ proxy/http/HttpConfig.h| 3 ++ proxy/http/HttpSM.cc | 20 - proxy/http/HttpTransact.cc | 5 ++- proxy/http2/unit_tests/test_Http2Frame.cc | 2 +- proxy/http3/Http3HeaderFramer.cc | 2 +- proxy/http3/test/test_QPACK.cc | 2 +- proxy/logging/LogBuffer.cc | 4 +- proxy/logging/LogConfig.cc | 12 +- proxy/logging/LogConfig.h | 1 + src/traffic_quic/quic_client.cc| 4 +- src/traffic_server/InkAPITest.cc | 2 +- src/traffic_server/InkIOCoreAPI.cc | 2 +- src/traffic_server/SocksProxy.cc | 2 +- 43 files changed, 180 insertions(+), 144 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 2a24465..f24c359 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -4407,6 +4407,31 @@ Sockets platforms. (Currently only linux). IO buffers are allocated with the MADV_DONTDUMP with madvise() on linux platforms that support MADV_DONTDUMP. Enabled by default. +.. ts:cv:: CONFIG proxy.config.ssl.misc.io.max_buffer_index INT 8 + + Configures the max IOBuffer Block index used for various SSL Operations + such as Handshake or Protocol Probe. Default value is 8 which maps to a 32K buffer + +.. ts:cv:: CONFIG proxy.config.hostdb.io.max_buffer_index INT 8 + + Configures the max IOBuffer Block index used for storing HostDB records. + Default value is 8 which maps to a 32K buffer + +.. ts:cv:: CONFIG proxy.config.payload.io.max_buffer_index INT 8 + + Configures the max IOBuffer Block index used for storing request payload buffer + for a POST request. Default value is 8 which maps to a 32K buffer + +.. ts:cv:: CONFIG proxy.config.msg.io.max_buffer_index INT 8 + + Configures the max IOBuffer Block index used for storing miscellaneous transactional + buffers such as error response body. Default value is 8 which maps to a 32K buffer + +.. ts:cv:: CONFIG proxy.config.log.io.max_buffer_index INT 8 + + Configures the max IOBuffer Block
[trafficserver] branch master updated (fcbcd73 -> f214fcf)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from fcbcd73 Removes SSLNetVConnection::sslContextSet add f214fcf Ensure read_avail is set for the first non-empty block (#6916) No new revisions were added by this update. Summary of changes: src/traffic_server/FetchSM.cc | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-)
[trafficserver] branch master updated (fcbcd73 -> f214fcf)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from fcbcd73 Removes SSLNetVConnection::sslContextSet add f214fcf Ensure read_avail is set for the first non-empty block (#6916) No new revisions were added by this update. Summary of changes: src/traffic_server/FetchSM.cc | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-)
[trafficserver] branch master updated: set sni_name with remapped origin name if sni_policy is not the default value (#6898)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 6f564de set sni_name with remapped origin name if sni_policy is not the default value (#6898) 6f564de is described below commit 6f564de71cda287b5b79b89d1c8c327a24ba5472 Author: Xin Li <33378623+whut...@users.noreply.github.com> AuthorDate: Mon Jun 15 18:59:49 2020 -0700 set sni_name with remapped origin name if sni_policy is not the default value (#6898) Co-authored-by: xinli1 --- proxy/http/HttpSM.cc | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proxy/http/HttpSM.cc b/proxy/http/HttpSM.cc index 4c308b6..96c6b32 100644 --- a/proxy/http/HttpSM.cc +++ b/proxy/http/HttpSM.cc @@ -4800,11 +4800,13 @@ HttpSM::get_outbound_sni() const { const char *sni_name = nullptr; size_t len = 0; - if (t_state.txn_conf->ssl_client_sni_policy != nullptr && !strcmp(t_state.txn_conf->ssl_client_sni_policy, "remap")) { + if (t_state.txn_conf->ssl_client_sni_policy == nullptr || !strcmp(t_state.txn_conf->ssl_client_sni_policy, "host")) { +// By default the host header field value is used for the SNI. +sni_name = t_state.hdr_info.server_request.host_get(reinterpret_cast(&len)); + } else { +// If other is specified, like "remap" and "verify_with_name_source", the remapped origin name is used for the SNI value len = strlen(t_state.server_info.name); sni_name = t_state.server_info.name; - } else { // Do the default of host header for SNI -sni_name = t_state.hdr_info.server_request.host_get(reinterpret_cast(&len)); } return std::string_view(sni_name, len); }
[trafficserver] branch master updated: Track thread changes during origin connect and cache open write (#6872)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 59c5e25 Track thread changes during origin connect and cache open write (#6872) 59c5e25 is described below commit 59c5e25846b0dfe8720b5027fecac72af2da1545 Author: Sudheer Vinukonda AuthorDate: Thu Jun 11 22:29:57 2020 -0700 Track thread changes during origin connect and cache open write (#6872) Also make adjust_thread() inline as an optimization --- proxy/ProxyTransaction.cc | 17 - proxy/ProxyTransaction.h | 17 + proxy/http/HttpConfig.cc | 6 ++ proxy/http/HttpConfig.h | 3 +++ proxy/http/HttpSM.cc | 2 ++ 5 files changed, 28 insertions(+), 17 deletions(-) diff --git a/proxy/ProxyTransaction.cc b/proxy/ProxyTransaction.cc index 3f824e0..9be892f 100644 --- a/proxy/ProxyTransaction.cc +++ b/proxy/ProxyTransaction.cc @@ -83,23 +83,6 @@ ProxyTransaction::destroy() this->mutex.clear(); } -// See if we need to schedule on the primary thread for the transaction or change the thread that is associated with the VC. -// If we reschedule, the scheduled action is returned. Otherwise, NULL is returned -Action * -ProxyTransaction::adjust_thread(Continuation *cont, int event, void *data) -{ - NetVConnection *vc = this->get_netvc(); - EThread *this_thread = this_ethread(); - if (vc && vc->thread != this_thread) { -if (vc->thread->is_event_type(ET_NET)) { - return vc->thread->schedule_imm(cont, event, data); -} else { // Not a net thread, take over this thread - vc->thread = this_thread; -} - } - return nullptr; -} - void ProxyTransaction::set_rx_error_code(ProxyError e) { diff --git a/proxy/ProxyTransaction.h b/proxy/ProxyTransaction.h index 51f35db..83a2111 100644 --- a/proxy/ProxyTransaction.h +++ b/proxy/ProxyTransaction.h @@ -221,3 +221,20 @@ ProxyTransaction::support_sni() const { return _proxy_ssn ? _proxy_ssn->support_sni() : false; } + +// See if we need to schedule on the primary thread for the transaction or change the thread that is associated with the VC. +// If we reschedule, the scheduled action is returned. Otherwise, NULL is returned +inline Action * +ProxyTransaction::adjust_thread(Continuation *cont, int event, void *data) +{ + NetVConnection *vc = this->get_netvc(); + EThread *this_thread = this_ethread(); + if (vc && vc->thread != this_thread) { +if (vc->thread->is_event_type(ET_NET)) { + return vc->thread->schedule_imm(cont, event, data); +} else { // Not a net thread, take over this thread + vc->thread = this_thread; +} + } + return nullptr; +} diff --git a/proxy/http/HttpConfig.cc b/proxy/http/HttpConfig.cc index 4f00b34..0c51d04 100644 --- a/proxy/http/HttpConfig.cc +++ b/proxy/http/HttpConfig.cc @@ -942,6 +942,12 @@ register_stat_callbacks() (int)http_origin_connections_throttled_stat, RecRawStatSyncCount); RecRegisterRawStat(http_rsb, RECT_PROCESS, "proxy.process.http.post_body_too_large", RECD_COUNTER, RECP_PERSISTENT, (int)http_post_body_too_large, RecRawStatSyncCount); + RecRegisterRawStat(http_rsb, RECT_PROCESS, "proxy.process.http.origin.connect.adjust_thread", RECD_COUNTER, RECP_NON_PERSISTENT, + (int)http_origin_connect_adjust_thread_stat, RecRawStatSyncCount); + HTTP_CLEAR_DYN_STAT(http_origin_connect_adjust_thread_stat); + RecRegisterRawStat(http_rsb, RECT_PROCESS, "proxy.process.http.cache.open_write.adjust_thread", RECD_COUNTER, RECP_NON_PERSISTENT, + (int)http_cache_open_write_adjust_thread_stat, RecRawStatSyncCount); + HTTP_CLEAR_DYN_STAT(http_cache_open_write_adjust_thread_stat); // milestones RecRegisterRawStat(http_rsb, RECT_PROCESS, "proxy.process.http.milestone.ua_begin", RECD_COUNTER, RECP_PERSISTENT, (int)http_ua_begin_time_stat, RecRawStatSyncSum); diff --git a/proxy/http/HttpConfig.h b/proxy/http/HttpConfig.h index ed0fcce..980c629 100644 --- a/proxy/http/HttpConfig.h +++ b/proxy/http/HttpConfig.h @@ -328,6 +328,9 @@ enum { http_origin_connections_throttled_stat, + http_origin_connect_adjust_thread_stat, + http_cache_open_write_adjust_thread_stat, + http_stat_count }; diff --git a/proxy/http/HttpSM.cc b/proxy/http/HttpSM.cc index 6dfbe23..323564f 100644 --- a/proxy/http/HttpSM.cc +++ b/proxy/http/HttpSM.cc @@ -2451,6 +2451,7 @@ HttpSM::state_cache_open_write(int event, void *data) pending_action->cancel(); } if ((pending_action = ua_txn->adjust_thread(this, event, data))) { + HTTP_INCREMENT_DYN_STAT(http_cache_open_write_adjust_thread_stat); return 0; // Go away if we reschedule } }
[trafficserver] branch master updated (fd4e818 -> 079ed98)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from fd4e818 avoid dynamic_cast to get Pi-tag for non_internal requests (#6868) add 079ed98 Protect against nullptr access during SSL Callback (#6866) No new revisions were added by this update. Summary of changes: iocore/net/SSLClientUtils.cc | 2 +- iocore/net/SSLUtils.cc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
[trafficserver] branch master updated (e9da3e0 -> fd4e818)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from e9da3e0 Adding HTTP status 451 in apidefs as well (See PR#6789) (#6797) add fd4e818 avoid dynamic_cast to get Pi-tag for non_internal requests (#6868) No new revisions were added by this update. Summary of changes: proxy/ProxyTransaction.cc | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-)
[trafficserver] branch master updated: Schedule Transform on the same thread as the continuation (#6843)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 23ba370 Schedule Transform on the same thread as the continuation (#6843) 23ba370 is described below commit 23ba370aac41a1ca79a4c59ef6a7df593cd502f7 Author: Sudheer Vinukonda AuthorDate: Tue Jun 2 12:31:30 2020 -0700 Schedule Transform on the same thread as the continuation (#6843) --- proxy/Transform.cc | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/proxy/Transform.cc b/proxy/Transform.cc index e51b26c..9024f11 100644 --- a/proxy/Transform.cc +++ b/proxy/Transform.cc @@ -125,11 +125,11 @@ TransformTerminus::TransformTerminus(TransformVConnection *tvc) SET_HANDLER(&TransformTerminus::handle_event); } -#define RETRY() \ - if (ink_atomic_increment((int *)&m_event_count, 1) < 0) { \ -ink_assert(!"not reached"); \ - } \ - eventProcessor.schedule_in(this, HRTIME_MSECONDS(10), ET_NET); \ +#define RETRY() \ + if (ink_atomic_increment((int *)&m_event_count, 1) < 0) { \ +ink_assert(!"not reached"); \ + } \ + this_ethread()->schedule_in(this, HRTIME_MSECONDS(10)); \ return 0; int @@ -280,7 +280,7 @@ TransformTerminus::do_io_read(Continuation *c, int64_t nbytes, MIOBuffer *buf) } Debug("transform", "[TransformTerminus::do_io_read] event_count %d", m_event_count); - eventProcessor.schedule_imm(this, ET_NET); + this_ethread()->schedule_imm_local(this); return &m_read_vio; } @@ -305,7 +305,7 @@ TransformTerminus::do_io_write(Continuation *c, int64_t nbytes, IOBufferReader * } Debug("transform", "[TransformTerminus::do_io_write] event_count %d", m_event_count); - eventProcessor.schedule_imm(this, ET_NET); + this_ethread()->schedule_imm_local(this); return &m_write_vio; } @@ -335,7 +335,7 @@ TransformTerminus::do_io_close(int error) m_write_vio.op = VIO::NONE; m_write_vio.buffer.clear(); - eventProcessor.schedule_imm(this, ET_NET); + this_ethread()->schedule_imm_local(this); } /*- @@ -368,7 +368,7 @@ TransformTerminus::reenable(VIO *vio) ink_assert(!"not reached"); } Debug("transform", "[TransformTerminus::reenable] event_count %d", m_event_count); -eventProcessor.schedule_imm(this, ET_NET); +this_ethread()->schedule_imm_local(this); } else { Debug("transform", "[TransformTerminus::reenable] skipping due to " "pending events"); @@ -731,7 +731,7 @@ void TransformTest::run() { if (is_action_tag_set("transform_test")) { -eventProcessor.schedule_imm(new TransformControl(), ET_NET); +this_ethread()->schedule_imm_local(new TransformControl()); } } #endif
[trafficserver] branch master updated: Issue 6838 Fixing the comparison in waited_enough (drain functionality) (#6839)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 92a1f87 Issue 6838 Fixing the comparison in waited_enough (drain functionality) (#6839) 92a1f87 is described below commit 92a1f87d14c7ab6444131dc30bf2b06af36000da Author: VijayAnand Subramanian AuthorDate: Tue Jun 2 04:08:14 2020 +0530 Issue 6838 Fixing the comparison in waited_enough (drain functionality) (#6839) Co-authored-by: Vijayanand Subramanian --- doc/admin-guide/files/records.config.en.rst | 7 +-- src/traffic_manager/traffic_manager.cc | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 87c0caa..2a24465 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -381,8 +381,11 @@ Thread Variables :reloadable: The shutdown timeout(in seconds) to apply when stopping Traffic - Server, in which ATS can initiate graceful shutdowns. It only supports - HTTP/2 graceful shutdown for now. Stopping |TS| here means sending + Server, in which ATS can initiate graceful shutdowns. In order + to effect graceful shutdown, the value specified should be greater + than 0. Value of 0 will not effect an abrupt shutdown. Abrupt + shutdowns can be achieved with out specifying --drain; + (traffic_ctl server stop /restart). Stopping |TS| here means sending `traffic_server` a signal either by `bin/trafficserver stop` or `kill`. .. ts:cv:: CONFIG proxy.config.thread.max_heartbeat_mseconds INT 60 diff --git a/src/traffic_manager/traffic_manager.cc b/src/traffic_manager/traffic_manager.cc index 0a410e2..0ddb653 100644 --- a/src/traffic_manager/traffic_manager.cc +++ b/src/traffic_manager/traffic_manager.cc @@ -198,7 +198,7 @@ waited_enough() return false; } - return (lmgmt->mgmt_shutdown_triggered_at + timeout >= time(nullptr)); + return (timeout ? (lmgmt->mgmt_shutdown_triggered_at + timeout <= time(nullptr)) : false); } static void
[trafficserver] branch master updated: Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 34b57fc Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800) 34b57fc is described below commit 34b57fccb40ef711ce2e6b31042c96efc74c0ecc Author: Sudheer Vinukonda AuthorDate: Wed May 20 09:13:58 2020 -0700 Add TXN_CLOSE hook to CPPAPI TransactionPlugin (#6800) * Add TXN_CLOSE hook to CPPAPI TransactionPlugin * Clean up TransactionPlugin object and associated Continuation in txn_close * Address review comments * More review comments --- include/tscpp/api/Plugin.h| 12 ++- include/tscpp/api/TransactionPlugin.h | 4 src/tscpp/api/GlobalPlugin.cc | 1 + src/tscpp/api/utils_internal.cc | 39 --- 4 files changed, 48 insertions(+), 8 deletions(-) diff --git a/include/tscpp/api/Plugin.h b/include/tscpp/api/Plugin.h index 2f57352..f37b805 100644 --- a/include/tscpp/api/Plugin.h +++ b/include/tscpp/api/Plugin.h @@ -58,7 +58,8 @@ public: HOOK_READ_REQUEST_HEADERS, /**< This hook will be fired after the request is read. */ HOOK_READ_CACHE_HEADERS,/**< This hook will be fired after the CACHE hdrs. */ HOOK_CACHE_LOOKUP_COMPLETE, /**< This hook will be fired after cache lookup complete. */ -HOOK_SELECT_ALT /**< This hook will be fired after select alt. */ +HOOK_TXN_CLOSE, /**< This hook will be fired after send response headers, only for TransactionPlugins::registerHook()!. */ +HOOK_SELECT_ALT /**< This hook will be fired after select alt. */ }; /** @@ -143,6 +144,15 @@ public: }; /** + * This method must be implemented when you hook HOOK_TXN_CLOSE + */ + virtual void + handleTxnClose(Transaction &transaction) + { +transaction.resume(); + }; + + /** * This method must be implemented when you hook HOOK_SELECT_ALT */ virtual void handleSelectAlt(const Request &clientReq, const Request &cachedReq, const Response &cachedResp){}; diff --git a/include/tscpp/api/TransactionPlugin.h b/include/tscpp/api/TransactionPlugin.h index b34fba0..ce3f1ca 100644 --- a/include/tscpp/api/TransactionPlugin.h +++ b/include/tscpp/api/TransactionPlugin.h @@ -93,6 +93,10 @@ public: * see HookType and Plugin for the correspond HookTypes and callback methods. If you fail to implement the * callback, a default implementation will be used that will only resume the Transaction. * + * \note For automatic destruction, you must either register dynamically allocated instances of + * classes derived from this class with the the corresponding Transaction object (using + * Transaction::addPlugin()), or register HOOK_TXN_CLOSE (but not both). + * * @param HookType the type of hook you wish to register * @see HookType * @see Plugin diff --git a/src/tscpp/api/GlobalPlugin.cc b/src/tscpp/api/GlobalPlugin.cc index b1be230..8e5f05c 100644 --- a/src/tscpp/api/GlobalPlugin.cc +++ b/src/tscpp/api/GlobalPlugin.cc @@ -87,6 +87,7 @@ GlobalPlugin::~GlobalPlugin() void GlobalPlugin::registerHook(Plugin::HookType hook_type) { + assert(hook_type != Plugin::HOOK_TXN_CLOSE); TSHttpHookID hook_id = utils::internal::convertInternalHookToTsHook(hook_type); TSHttpHookAdd(hook_id, state_->cont_); LOG_DEBUG("Registered global plugin %p for hook %s", this, HOOK_TYPE_STRINGS[hook_type].c_str()); diff --git a/src/tscpp/api/utils_internal.cc b/src/tscpp/api/utils_internal.cc index 7cb86e0..61f9044 100644 --- a/src/tscpp/api/utils_internal.cc +++ b/src/tscpp/api/utils_internal.cc @@ -49,6 +49,25 @@ resetTransactionHandles(Transaction &transaction, TSEvent event) return; } +void +cleanupTransaction(Transaction &transaction, TSHttpTxn ats_txn_handle) +{ + delete &transaction; + // reset the txn arg to prevent use-after-free + TSUserArgSet(ats_txn_handle, TRANSACTION_STORAGE_INDEX, nullptr); +} + +void +cleanupTransactionPlugin(Plugin *plugin) +{ + TransactionPlugin *transaction_plugin = static_cast(plugin); + std::shared_ptr trans_mutex= utils::internal::getTransactionPluginMutex(*transaction_plugin); + LOG_DEBUG("Locking TransactionPlugin mutex to delete transaction plugin at %p", transaction_plugin); + trans_mutex->lock(); + delete transaction_plugin; + trans_mutex->unlock(); +} + int handleTransactionEvents(TSCont cont, TSEvent event, void *edata) { @@ -77,14 +96,9 @@ handleTransactionEvents(TSCont cont, TSEvent event, void *edata) resetTransactionHandles(transaction, event); const std::list &plugins = utils::internal::getTransactionPlugins(transaction); for (auto plugin : plugins) { - std::shared_ptr trans_mutex = utils::internal::getTransa
[trafficserver] branch master updated: Adding HTTP Status code 451 for Unavailable For Legal Reasons (RFC 7725) (#6789)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 550a2da Adding HTTP Status code 451 for Unavailable For Legal Reasons (RFC 7725) (#6789) 550a2da is described below commit 550a2daea047f2f0d06b66eb10d8e35edbc3fb0e Author: Saurav Kumar <2020sau...@gmail.com> AuthorDate: Fri May 15 08:59:34 2020 -0700 Adding HTTP Status code 451 for Unavailable For Legal Reasons (RFC 7725) (#6789) --- include/tscpp/api/HttpStatus.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/tscpp/api/HttpStatus.h b/include/tscpp/api/HttpStatus.h index 1fb3e15..89f6dd8 100644 --- a/include/tscpp/api/HttpStatus.h +++ b/include/tscpp/api/HttpStatus.h @@ -82,6 +82,7 @@ enum HttpStatus { HTTP_STATUS_PRECONDITION_REQUIRED = 428, HTTP_STATUS_TOO_MANY_REQUESTS = 429, HTTP_STATUS_REQUEST_HEADER_FIELDS_TOO_LARGE = 431, + HTTP_STATUS_UNAVAILABLE_FOR_LEGAL_REASONS = 451, HTTP_STATUS_INTERNAL_SERVER_ERROR = 500, HTTP_STATUS_NOT_IMPLEMENTED = 501,
[trafficserver] branch master updated (7229939 -> 2188810)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 7229939 Update existingh formatting with new clang-format package add 2188810 1. Set a non-zero default value for TLS Client Handshake Timeout (#6781) No new revisions were added by this update. Summary of changes: doc/admin-guide/files/records.config.en.rst | 8 mgmt/RecordsConfig.cc | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-)
[trafficserver] branch master updated: Ensure inactivity timeout is not set when passed in timeout value is 0 (#6772)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 0ab955a Ensure inactivity timeout is not set when passed in timeout value is 0 (#6772) 0ab955a is described below commit 0ab955afa3767e05f0d9393eb91071a3d79c6fa6 Author: Sudheer Vinukonda AuthorDate: Wed May 13 13:39:11 2020 -0700 Ensure inactivity timeout is not set when passed in timeout value is 0 (#6772) --- iocore/net/UnixNetVConnection.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/iocore/net/UnixNetVConnection.cc b/iocore/net/UnixNetVConnection.cc index ca47deb..181420e 100644 --- a/iocore/net/UnixNetVConnection.cc +++ b/iocore/net/UnixNetVConnection.cc @@ -1353,7 +1353,7 @@ UnixNetVConnection::set_inactivity_timeout(ink_hrtime timeout_in) { Debug("socket", "Set inactive timeout=%" PRId64 ", for NetVC=%p", timeout_in, this); inactivity_timeout_in = timeout_in; - next_inactivity_timeout_at = Thread::get_hrtime() + inactivity_timeout_in; + next_inactivity_timeout_at = (timeout_in > 0) ? Thread::get_hrtime() + inactivity_timeout_in : 0; } TS_INLINE void
[trafficserver] branch master updated (42323fa -> 66f2306)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 42323fa Script to find mutexes/futexes that are contending add 66f2306 Add metrics to track default inactivity timed out connections (#6755) No new revisions were added by this update. Summary of changes: .../monitoring/statistics/core/network-io.en.rst | 7 +++ iocore/net/I_NetVConnection.h | 4 +++- iocore/net/Net.cc | 6 -- iocore/net/NetEvent.h | 6 +- iocore/net/P_Net.h| 3 ++- iocore/net/P_UnixNetVConnection.h | 2 ++ iocore/net/UnixNet.cc | 12 iocore/net/UnixNetVConnection.cc | 19 +++ proxy/PluginVC.cc | 12 proxy/PluginVC.h | 2 ++ 10 files changed, 64 insertions(+), 9 deletions(-)
[trafficserver] branch master updated: Enforce Active Connection limits (#6754)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new deee3ef Enforce Active Connection limits (#6754) deee3ef is described below commit deee3efbbe64e19d31a953e222edbffb8ab06954 Author: Sudheer Vinukonda AuthorDate: Tue May 12 11:13:00 2020 -0700 Enforce Active Connection limits (#6754) * Enforce Active Connection limits 1. Throttle connections when there's no room in active conn queue 2. Adjust manage_active_queue() to not fail when the conn is already in active queue 3. Return true for PluginVC (dummy connection) add_to_active_queue 4. Metrics for throttling 5. Allow to disable active connection tracking 6. Doc updates --- doc/admin-guide/files/records.config.en.rst | 16 .../monitoring/statistics/core/network-io.en.rst | 11 ++- iocore/net/Net.cc| 2 ++ iocore/net/P_Net.h | 1 + iocore/net/UnixNet.cc| 15 +-- proxy/PluginVC.cc| 2 +- proxy/http/Http1ClientSession.cc | 7 ++- proxy/http2/Http2ConnectionState.cc | 8 +++- 8 files changed, 56 insertions(+), 6 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 53bd09c..899f7a5 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -409,6 +409,22 @@ Network handled. This should be tuned according to your memory size, and expected work load. If this is set to 0, the throttling logic is disabled. +.. ts:cv:: CONFIG proxy.config.net.max_connections_in INT 3 + + The total number of client connections that the :program:`traffic_server` + can handle simultaneously. This should be tuned according to your memory size, + and expected work load (network, cpu etc). This limit includes both keepalive + and active client connections that :program:`traffic_server` can handle at + any given instant. + +.. ts:cv:: CONFIG proxy.config.net.max_active_connections_in INT 1 + + The total number of active client connections that the |TS| can handle + simultaneously. This should be tuned according to your memory size, + and expected work load (network, cpu etc). If this is set to 0, active + connection tracking is disabled and active connections have no separate + limit and the total connections follow `proxy.config.net.connections_throttle` + .. ts:cv:: CONFIG proxy.config.net.default_inactivity_timeout INT 86400 :reloadable: diff --git a/doc/admin-guide/monitoring/statistics/core/network-io.en.rst b/doc/admin-guide/monitoring/statistics/core/network-io.en.rst index 5393086..8b83253 100644 --- a/doc/admin-guide/monitoring/statistics/core/network-io.en.rst +++ b/doc/admin-guide/monitoring/statistics/core/network-io.en.rst @@ -60,8 +60,17 @@ Network I/O .. ts:stat:: global proxy.process.net.connections_currently_open integer :type: counter +.. ts:stat:: global proxy.process.net.connections_throttled_in integer + :type: counter + +.. ts:stat:: global proxy.process.net.connections_throttled_out integer + :type: counter + +.. ts:stat:: global proxy.process.net.max.active.connections_throttled_in integer + :type: counter + .. ts:stat:: global proxy.process.net.default_inactivity_timeout_applied integer -.. ts:stat:: global proxy.process.net.dynamic_keep_alive_timeout_in_count integer +.. ts:stat:: global proxy.process.net.default_inactivity_timeout_count integer .. ts:stat:: global proxy.process.net.dynamic_keep_alive_timeout_in_total integer .. ts:stat:: global proxy.process.net.inactivity_cop_lock_acquire_failure integer .. ts:stat:: global proxy.process.net.net_handler_run integer diff --git a/iocore/net/Net.cc b/iocore/net/Net.cc index f5bbb88..1813c0d 100644 --- a/iocore/net/Net.cc +++ b/iocore/net/Net.cc @@ -140,6 +140,8 @@ register_net_stats() (int)net_connections_throttled_in_stat, RecRawStatSyncSum); RecRegisterRawStat(net_rsb, RECT_PROCESS, "proxy.process.net.connections_throttled_out", RECD_INT, RECP_PERSISTENT, (int)net_connections_throttled_out_stat, RecRawStatSyncSum); + RecRegisterRawStat(net_rsb, RECT_PROCESS, "proxy.process.net.max.active.connections_throttled_in", RECD_INT, RECP_PERSISTENT, + (int)net_connections_max_active_throttled_in_stat, RecRawStatSyncSum); } void diff --git a/iocore/net/P_Net.h b/iocore/net/P_Net.h index 15a55bb..9a749f9 100644 --- a/iocore/net/P_Net.h +++ b/iocore/net/P_Net.h @@ -57,6 +57,7 @@ enum Net_Stats
[trafficserver] branch master updated: Add Access log fields for ProxyProtocol Context
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 583e5ed Add Access log fields for ProxyProtocol Context 583e5ed is described below commit 583e5ed5f3db1e33075b8e5b880d6f6bf1e730f3 Author: Sudheer Vinukonda AuthorDate: Tue Apr 28 08:56:20 2020 -0700 Add Access log fields for ProxyProtocol Context --- doc/admin-guide/logging/formatting.en.rst | 7 proxy/logging/Log.cc | 15 + proxy/logging/LogAccess.cc| 55 +++ proxy/logging/LogAccess.h | 3 ++ 4 files changed, 80 insertions(+) diff --git a/doc/admin-guide/logging/formatting.en.rst b/doc/admin-guide/logging/formatting.en.rst index c81f435..242afe2 100644 --- a/doc/admin-guide/logging/formatting.en.rst +++ b/doc/admin-guide/logging/formatting.en.rst @@ -505,6 +505,13 @@ shi Origin Server IP address resolved via DNS by |TS| for the origin server. shn Origin Server Host name of the origin server. nhi Origin Server Destination IP address of next hop nhp Origin Server Destination port of next hop +ppv Proxy Protocol Proxy Protocol Version used (if any) between the Loadbalancer + Versionand |TS| +pps Proxy Protocol Source IP received via Proxy Protocol context from the LB to + Source IP the |TS| +ppd Proxy Protocol Destination IP received via Proxy Protocol context from the LB + Dest IPto the |TS| + = == == .. note:: diff --git a/proxy/logging/Log.cc b/proxy/logging/Log.cc index 8be6bd7..4d46f6d 100644 --- a/proxy/logging/Log.cc +++ b/proxy/logging/Log.cc @@ -917,6 +917,21 @@ Log::init_fields() global_field_list.add(field, false); field_symbol_hash.emplace("ctpd", field); + field = new LogField("proxy_protocol_version", "ppv", LogField::dINT, &LogAccess::marshal_proxy_protocol_version, + reinterpret_cast(&LogAccess::unmarshal_str)); + global_field_list.add(field, false); + field_symbol_hash.emplace("ppv", field); + + field = new LogField("proxy_protocol_src_ip", "pps", LogField::IP, &LogAccess::marshal_proxy_protocol_src_ip, + &LogAccess::unmarshal_ip_to_str); + global_field_list.add(field, false); + field_symbol_hash.emplace("ppsip", field); + + field = new LogField("proxy_protocol_dst_ip", "ppd", LogField::IP, &LogAccess::marshal_proxy_protocol_dst_ip, + &LogAccess::unmarshal_ip_to_str); + global_field_list.add(field, false); + field_symbol_hash.emplace("ppdip", field); + field = new LogField("version_build_number", "vbn", LogField::STRING, &LogAccess::marshal_version_build_number, (LogField::UnmarshalFunc)&LogAccess::unmarshal_str); global_field_list.add(field, false); diff --git a/proxy/logging/LogAccess.cc b/proxy/logging/LogAccess.cc index 08f5b0a..6d8d9e1 100644 --- a/proxy/logging/LogAccess.cc +++ b/proxy/logging/LogAccess.cc @@ -1315,6 +1315,61 @@ LogAccess::marshal_version_build_number(char *buf) -*/ int +LogAccess::marshal_proxy_protocol_version(char *buf) +{ + const char *version_str = nullptr; + int len = INK_MIN_ALIGN; + + if (m_http_sm) { +NetVConnection::ProxyProtocolVersion ver = m_http_sm->t_state.pp_info.proxy_protocol_version; +switch (ver) { +case NetVConnection::ProxyProtocolVersion::V1: + version_str = "V1"; + break; +case NetVConnection::ProxyProtocolVersion::V2: + version_str = "V2"; + break; +case NetVConnection::ProxyProtocolVersion::UNDEFINED: +default: + version_str = "-"; + break; +} +len = LogAccess::strlen(version_str); + } + + if (buf) { +marshal_str(buf, version_str, len); + } + return len; +} + +/*- + -*/ +int +LogAccess::marshal_proxy_protocol_src_ip(char *buf) +{ + sockaddr const *ip = nullptr; + if (m_http_sm && m_http_sm->t_state.pp_info.proxy_protocol_version != NetVConnection::ProxyProtocolVersion::UNDEFINED) { +ip = &m_http_sm->t_state.pp_info.src_addr.sa; + } + return marshal_ip(buf, ip); +} + +/*- + -*/ +int +LogAccess::marshal_proxy_protocol_dst_ip(char *buf) +{ + sockaddr const *ip = nullptr; +
[trafficserver] branch master updated (7f0c8ff -> 0b31904)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 7f0c8ff Fixes memory leak during log configuration add 0b31904 Update TSStatFindName to check that sync callback is set on the stat No new revisions were added by this update. Summary of changes: lib/records/I_RecCore.h | 2 +- lib/records/RecCore.cc | 20 +++- src/traffic_server/InkAPI.cc | 2 +- 3 files changed, 13 insertions(+), 11 deletions(-)
[trafficserver] branch master updated: Add docs for memory leak detection configs
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 70b1bf6 Add docs for memory leak detection configs 70b1bf6 is described below commit 70b1bf69c06cd3d967a8bdf9b351e4ec97bf01f9 Author: Sudheer Vinukonda AuthorDate: Fri Apr 10 09:43:58 2020 -0700 Add docs for memory leak detection configs --- doc/admin-guide/files/records.config.en.rst | 27 --- doc/developer-guide/debugging/memory-leaks.en.rst | 16 ++ 2 files changed, 40 insertions(+), 3 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 2ddd655..127de69 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -4329,11 +4329,32 @@ Sockets For more information on the implications of enabling huge pages, see `Wikipedia <http://en.wikipedia.org/wiki/Page_%28computer_memory%29#Page_size_trade-off>_`. +.. ts:cv:: CONFIG proxy.config.dump_mem_info_frequency INT 0 + :reloadable: + + Enable . When enabled makes Traffic Server dump IO Buffer memory information + to ``traffic.out`` at (intervals are in seconds). A zero value implies it is + disabled + +.. ts:cv:: CONFIG proxy.config.res_track_memory INT 0 + + When enabled makes Traffic Server track memory usage (allocations and releases). This + information is dumped to ``traffic.out`` when the user sends a SIGUSR1 signal or + periodically when :ts:cv:`proxy.config.dump_mem_info_frequency` is enabled. + + = == + Value Description + = == + ``0`` Memory tracking Disabled + ``1`` Tracks IO Buffer Memory allocations and releases + ``2`` Tracks IO Buffer Memory and OpenSSL Memory allocations and releases + = == + .. ts:cv:: CONFIG proxy.config.allocator.dontdump_iobuffers INT 1 - Enable (1) the exclusion of IO buffers from core files when ATS crashes on supported - platforms. (Currently only linux). IO buffers are allocated with the MADV_DONTDUMP - with madvise() on linux platforms that support MADV_DONTDUMP. Enabled by default. + Enable (1) the exclusion of IO buffers from core files when ATS crashes on supported + platforms. (Currently only linux). IO buffers are allocated with the MADV_DONTDUMP + with madvise() on linux platforms that support MADV_DONTDUMP. Enabled by default. .. ts:cv:: CONFIG proxy.config.http.enabled INT 1 diff --git a/doc/developer-guide/debugging/memory-leaks.en.rst b/doc/developer-guide/debugging/memory-leaks.en.rst index 64a8b78..23d4516 100644 --- a/doc/developer-guide/debugging/memory-leaks.en.rst +++ b/doc/developer-guide/debugging/memory-leaks.en.rst @@ -33,3 +33,19 @@ related to memory - you can use memory dump information. Enable This causes Traffic Server to dump memory information to ``traffic.out`` at (intervals are in seconds). A zero value means that it is disabled. + +:: + + CONFIG proxy.config.res_track_memory INT + + When enabled makes Traffic Server track memory usage (allocations and releases). This + information is dumped to ``traffic.out`` when the user sends a SIGUSR1 signal or + periodically when :ts:cv:`proxy.config.dump_mem_info_frequency` is enabled. + + = == + Value Description + = == + ``0`` Memory tracking Disabled + ``1`` Tracks IO Buffer Memory allocations and releases + ``2`` Tracks IO Buffer Memory and OpenSSL Memory allocations and releases + = ==
[trafficserver] branch master updated (f8a8b2f -> ba98187)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from f8a8b2f Add extension ip.test.ext to Au Test, with Test method to allocate extra TCP ports. add ba98187 Remove configure option --max-api-stats which does not do anything. Should have been removed as part of commit ea1fb0c87261b1fbf375fa6ade26deda1d01995b No new revisions were added by this update. Summary of changes: configure.ac | 10 -- 1 file changed, 10 deletions(-)
[trafficserver] branch master updated: Return TSFetchSM from TSFetchUrl so TSFetchFlagSet can set fetch flags
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 3b246a9 Return TSFetchSM from TSFetchUrl so TSFetchFlagSet can set fetch flags 3b246a9 is described below commit 3b246a9ad0d1508725bc4f270baa24e41d65e2fb Author: Sudheer Vinukonda AuthorDate: Wed Apr 1 19:14:58 2020 -0700 Return TSFetchSM from TSFetchUrl so TSFetchFlagSet can set fetch flags This is a miss in the commit 7675d0b8082247de35146cedbce6875cb2d39f03 --- doc/developer-guide/api/functions/TSFetchCreate.en.rst | 4 ++-- include/ts/apidefs.h.in| 2 ++ include/ts/experimental.h | 2 -- include/ts/ts.h| 4 ++-- src/traffic_server/InkAPI.cc | 4 +++- 5 files changed, 9 insertions(+), 7 deletions(-) diff --git a/doc/developer-guide/api/functions/TSFetchCreate.en.rst b/doc/developer-guide/api/functions/TSFetchCreate.en.rst index 9fe8ed5..4dc4924 100644 --- a/doc/developer-guide/api/functions/TSFetchCreate.en.rst +++ b/doc/developer-guide/api/functions/TSFetchCreate.en.rst @@ -32,9 +32,9 @@ Synopsis #include .. function:: void TSFetchPages(TSFetchUrlParams_t *) -.. function:: void TSFetchUrl(const char *, int, sockaddr const *, TSCont, TSFetchWakeUpOptions, TSFetchEvent) +.. function:: TSFetchSM TSFetchUrl(const char *, int, sockaddr const *, TSCont, TSFetchWakeUpOptions, TSFetchEvent) .. function:: void TSFetchFlagSet(TSFetchSM, int) -.. function:: TSFetch TSFetchCreate(TSCont, const char *, const char *, const char *, struct sockaddr const *, int) +.. function:: TSFetchSM TSFetchCreate(TSCont, const char *, const char *, const char *, struct sockaddr const *, int) .. function:: void TSFetchHeaderAdd(TSFetchSM, const char *, int, const char *, int) .. function:: void TSFetchWriteData(TSFetchSM, const void *, size_t) .. function:: ssize_t TSFetchReadData(TSFetchSM, void *, size_t) diff --git a/include/ts/apidefs.h.in b/include/ts/apidefs.h.in index c728b01..c874121 100644 --- a/include/ts/apidefs.h.in +++ b/include/ts/apidefs.h.in @@ -922,6 +922,8 @@ typedef struct tsapi_hostlookupresult *TSHostLookupResult; typedef struct tsapi_aiocallback *TSAIOCallback; typedef struct tsapi_net_accept *TSAcceptor; +typedef struct tsapi_fetchsm *TSFetchSM; + typedef void *(*TSThreadFunc)(void *data); typedef int (*TSEventFunc)(TSCont contp, TSEvent event, void *edata); typedef void (*TSConfigDestroyFunc)(void *data); diff --git a/include/ts/experimental.h b/include/ts/experimental.h index b731127..d2ebfd0 100644 --- a/include/ts/experimental.h +++ b/include/ts/experimental.h @@ -50,8 +50,6 @@ typedef enum { TS_FETCH_FLAGS_NOT_INTERNAL_REQUEST = 1 << 4 // Allow this fetch to be created as a non-internal request. } TSFetchFlags; -typedef struct tsapi_fetchsm *TSFetchSM; - /* Forward declaration of in_addr, any user of these APIs should probably include net/netinet.h or whatever is appropriate on the platform. */ struct in_addr; diff --git a/include/ts/ts.h b/include/ts/ts.h index b18ce03..1e82590 100644 --- a/include/ts/ts.h +++ b/include/ts/ts.h @@ -1722,8 +1722,8 @@ tsapi TSVConn TSHttpConnect(struct sockaddr const *addr); */ tsapi TSVConn TSHttpConnectTransparent(struct sockaddr const *client_addr, struct sockaddr const *server_addr); -tsapi void TSFetchUrl(const char *request, int request_len, struct sockaddr const *addr, TSCont contp, - TSFetchWakeUpOptions callback_options, TSFetchEvent event); +tsapi TSFetchSM TSFetchUrl(const char *request, int request_len, struct sockaddr const *addr, TSCont contp, + TSFetchWakeUpOptions callback_options, TSFetchEvent event); tsapi void TSFetchPages(TSFetchUrlParams_t *params); /* Check if HTTP State machine is internal or not */ diff --git a/src/traffic_server/InkAPI.cc b/src/traffic_server/InkAPI.cc index 0824666..d9fef70 100644 --- a/src/traffic_server/InkAPI.cc +++ b/src/traffic_server/InkAPI.cc @@ -7916,7 +7916,7 @@ TSFetchPages(TSFetchUrlParams_t *params) } } -void +TSFetchSM TSFetchUrl(const char *headers, int request_len, sockaddr const *ip, TSCont contp, TSFetchWakeUpOptions callback_options, TSFetchEvent events) { @@ -7928,6 +7928,8 @@ TSFetchUrl(const char *headers, int request_len, sockaddr const *ip, TSCont cont fetch_sm->init((Continuation *)contp, callback_options, events, headers, request_len, ip); fetch_sm->httpConnect(); + + return reinterpret_cast(fetch_sm); } void
[trafficserver] branch master updated: Doc updates to TSContSchedule* API
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 2f2ccfd Doc updates to TSContSchedule* API 2f2ccfd is described below commit 2f2ccfd3a4e98353c4f0a9a9d8e6a496ce4397e3 Author: Sudheer Vinukonda AuthorDate: Fri Mar 27 09:11:19 2020 -0700 Doc updates to TSContSchedule* API --- .../api/functions/TSContSchedule.en.rst| 4 ...SContSchedule.en.rst => TSContScheduleEvery.en.rst} | 18 +++--- .../api/functions/TSContScheduleOnPool.en.rst | 1 + .../api/functions/TSContScheduleOnThread.en.rst| 1 + .../api/functions/TSContThreadAffinitySet.en.rst | 3 +++ 5 files changed, 20 insertions(+), 7 deletions(-) diff --git a/doc/developer-guide/api/functions/TSContSchedule.en.rst b/doc/developer-guide/api/functions/TSContSchedule.en.rst index 26d8228..4dbb9f6 100644 --- a/doc/developer-guide/api/functions/TSContSchedule.en.rst +++ b/doc/developer-guide/api/functions/TSContSchedule.en.rst @@ -44,11 +44,15 @@ another thread this can be problematic to be correctly timed. The return value c :func:`TSActionDone` to see if the continuation ran before the return, which is possible if :arg:`timeout` is `0`. Returns ``nullptr`` if thread affinity was cleared. +TSContSchedule() or TSContScheduleEvery() will default to set the thread affinity to the calling thread +when no affinity is already set for example, using :func:`TSContThreadAffinitySet` + Note that the TSContSchedule() family of API shall only be called from an ATS EThread. Calling it from raw non-EThreads can result in unpredictable behavior. See Also +:doc:`TSContScheduleEvery.en` :doc:`TSContScheduleOnPool.en` :doc:`TSContScheduleOnThread.en` diff --git a/doc/developer-guide/api/functions/TSContSchedule.en.rst b/doc/developer-guide/api/functions/TSContScheduleEvery.en.rst similarity index 71% copy from doc/developer-guide/api/functions/TSContSchedule.en.rst copy to doc/developer-guide/api/functions/TSContScheduleEvery.en.rst index 26d8228..bc61392 100644 --- a/doc/developer-guide/api/functions/TSContSchedule.en.rst +++ b/doc/developer-guide/api/functions/TSContScheduleEvery.en.rst @@ -18,8 +18,8 @@ .. default-domain:: c -TSContSchedule -** +TSContScheduleEvery +*** Synopsis @@ -28,15 +28,15 @@ Synopsis #include -.. function:: TSAction TSContSchedule(TSCont contp, TSHRTime timeout) +.. function:: TSAction TSContScheduleEvery(TSCont contp, TSHRTime every) Description === -Schedules :arg:`contp` to run :arg:`delay` milliseconds in the future. This is approximate. The delay -will be at least :arg:`delay` but possibly more. Resolutions finer than roughly 5 milliseconds will -not be effective. :arg:`contp` is required to have a mutex, which is provided to -:func:`TSContCreate`. +Schedules :arg:`contp` to periodically run every :arg:`delay` milliseconds in the future. +This is approximate. The delay will be at least :arg:`delay` but possibly more. +Resolutions finer than roughly 5 milliseconds will not be effective. :arg:`contp` is +required to have a mutex, which is provided to :func:`TSContCreate`. The return value can be used to cancel the scheduled event via :func:`TSActionCancel`. This is effective until the continuation :arg:`contp` is being dispatched. However, if it is scheduled on @@ -44,11 +44,15 @@ another thread this can be problematic to be correctly timed. The return value c :func:`TSActionDone` to see if the continuation ran before the return, which is possible if :arg:`timeout` is `0`. Returns ``nullptr`` if thread affinity was cleared. +TSContSchedule() or TSContScheduleEvery() will default to set the thread affinity to the calling thread +when no affinity is already set for example, using :func:`TSContThreadAffinitySet` + Note that the TSContSchedule() family of API shall only be called from an ATS EThread. Calling it from raw non-EThreads can result in unpredictable behavior. See Also +:doc:`TSContSchedule.en` :doc:`TSContScheduleOnPool.en` :doc:`TSContScheduleOnThread.en` diff --git a/doc/developer-guide/api/functions/TSContScheduleOnPool.en.rst b/doc/developer-guide/api/functions/TSContScheduleOnPool.en.rst index d14b57f..e1b3dbf 100644 --- a/doc/developer-guide/api/functions/TSContScheduleOnPool.en.rst +++ b/doc/developer-guide/api/functions/TSContScheduleOnPool.en.rst @@ -125,4 +125,5 @@ See Also :doc:`TSContSchedule.en` +:doc:`TSContScheduleEvery.en` :doc:`TSContScheduleOnThread.en` diff --git a/doc/developer-guide/api/functions/TSContScheduleOnThread.en.rst b/doc/developer-guide/api/functions/TSContScheduleOnThread.en.rst index 27c48bf..0f91166 100644 --- a/doc/developer-guide/api/functions/TSContScheduleOnThread.en.rst ++
[trafficserver] branch master updated: [Doc][DevGuide][TSStatSync] Fixing documentation for SUM and COUNT types
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new af6645a [Doc][DevGuide][TSStatSync] Fixing documentation for SUM and COUNT types af6645a is described below commit af6645a85b346eff465eb187645476e8efdb8f1b Author: Saurav Kumar <2020sau...@gmail.com> AuthorDate: Fri Mar 27 13:32:16 2020 -0700 [Doc][DevGuide][TSStatSync] Fixing documentation for SUM and COUNT types --- doc/developer-guide/api/types/TSStatSync.en.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/developer-guide/api/types/TSStatSync.en.rst b/doc/developer-guide/api/types/TSStatSync.en.rst index f7867ae..0289443 100644 --- a/doc/developer-guide/api/types/TSStatSync.en.rst +++ b/doc/developer-guide/api/types/TSStatSync.en.rst @@ -35,11 +35,11 @@ Enumeration Members .. c:member:: TSStatSync TS_STAT_SYNC_SUM - Values should add be summed. + This stat sync type should be used for gauge metrics (i.e can increase or decrease with time). It may be manipulated using TSStatIntIncrement, TSStatIntDecrement, TSStatIntSet. E.g for counting number of available origin-servers or number of active threads. .. c:member:: TSStatSync TS_STAT_SYNC_COUNT - Values should be added together. + This stat sync type should be used for counter metrics (i.e it should only increase with time). It should only be manipulated using TSStatIntIncrement. E.g for tracking call counts or uptime. .. c:member:: TSStatSync TS_STAT_SYNC_AVG
[trafficserver] branch master updated: Support body factory template suppression for internal requests
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 3a0cdb2 Support body factory template suppression for internal requests 3a0cdb2 is described below commit 3a0cdb202c9f680416c194cadecd95c0d90b8cb2 Author: Sudheer Vinukonda AuthorDate: Tue Mar 24 15:27:26 2020 -0700 Support body factory template suppression for internal requests Repurpose unused config proxy.config.body_factory.response_suppression_mode (2) --- doc/admin-guide/files/records.config.en.rst | 2 +- mgmt/RecordsConfig.cc | 2 +- proxy/http/HttpBodyFactory.cc | 26 +++--- 3 files changed, 13 insertions(+), 17 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index fe163d3..f5b11e4 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -2395,7 +2395,7 @@ Customizable User Response Pages = == ``0`` Never suppress generated response pages. ``1`` Always suppress generated response pages. - ``2`` Suppress response pages only for intercepted traffic. + ``2`` Suppress response pages only for internal traffic. = == .. ts:cv:: CONFIG proxy.config.http_ui_enabled INT 0 diff --git a/mgmt/RecordsConfig.cc b/mgmt/RecordsConfig.cc index 24cbc3c..a7f7cb2 100644 --- a/mgmt/RecordsConfig.cc +++ b/mgmt/RecordsConfig.cc @@ -674,7 +674,7 @@ static const RecordElement RecordsConfig[] = , //# 0 - never suppress generated responses //# 1 - always suppress generated responses - //# 2 - suppress responses for intercepted traffic + //# 2 - suppress responses for internal traffic {RECT_CONFIG, "proxy.config.body_factory.response_suppression_mode", RECD_INT, "0", RECU_RESTART_TS, RR_NULL, RECC_INT, "[0-2]", RECA_NULL} , {RECT_CONFIG, "proxy.config.body_factory.template_base", RECD_STRING, "NONE", RECU_DYNAMIC, RR_NULL, RECC_STR, ".*", RECA_NULL} diff --git a/proxy/http/HttpBodyFactory.cc b/proxy/http/HttpBodyFactory.cc index a13d06d..4622cda 100644 --- a/proxy/http/HttpBodyFactory.cc +++ b/proxy/http/HttpBodyFactory.cc @@ -73,6 +73,16 @@ HttpBodyFactory::fabricate_with_old_api(const char *type, HttpTransact::State *c bool found_requested_template = false; bool plain_flag = false; + /// + // if suppressing this response, return NULL // + /// + if (is_response_suppressed(context)) { +if (enable_logging) { + Log::error("BODY_FACTORY: suppressing '%s' response for url '%s'", type, url); +} +return nullptr; + } + lock(); *resulting_buffer_length = 0; @@ -99,16 +109,6 @@ HttpBodyFactory::fabricate_with_old_api(const char *type, HttpTransact::State *c } } } - /// - // if suppressing this response, return NULL // - /// - if (is_response_suppressed(context)) { -if (enable_logging) { - Log::error("BODY_FACTORY: suppressing '%s' response for url '%s'", type, url); -} -unlock(); -return nullptr; - } // // if language-targeting activated, get client Accept-Language & Accept-Charset // // @@ -673,11 +673,7 @@ HttpBodyFactory::is_response_suppressed(HttpTransact::State *context) } else if (response_suppression_mode == 1) { return true; } else if (response_suppression_mode == 2) { -if (context->req_flavor == HttpTransact::REQ_FLAVOR_INTERCEPTED) { - return true; -} else { - return false; -} +return context->request_data.internal_txn; } else { return false; }
[trafficserver] branch master updated: When using TSContSchedule() and TSContScheduleAPI() set the calling thread as the thread affinity when not already set
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 318728c When using TSContSchedule() and TSContScheduleAPI() set the calling thread as the thread affinity when not already set 318728c is described below commit 318728c62ea6dd766e7cd189753db2ca1508bbeb Author: Sudheer Vinukonda AuthorDate: Thu Mar 26 13:03:47 2020 -0700 When using TSContSchedule() and TSContScheduleAPI() set the calling thread as the thread affinity when not already set --- src/traffic_server/InkAPI.cc | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/traffic_server/InkAPI.cc b/src/traffic_server/InkAPI.cc index f21c855..0824666 100644 --- a/src/traffic_server/InkAPI.cc +++ b/src/traffic_server/InkAPI.cc @@ -4530,7 +4530,8 @@ TSContSchedule(TSCont contp, TSHRTime timeout) EThread *eth = i->getThreadAffinity(); if (eth == nullptr) { -return nullptr; +eth = this_ethread(); +i->setThreadAffinity(eth); } TSAction action; @@ -4643,7 +4644,8 @@ TSContScheduleEvery(TSCont contp, TSHRTime every /* millisecs */) EThread *eth = i->getThreadAffinity(); if (eth == nullptr) { -return nullptr; +eth = this_ethread(); +i->setThreadAffinity(eth); } TSAction action = reinterpret_cast(eth->schedule_every(i, HRTIME_MSECONDS(every)));
[trafficserver] branch master updated (793d95b -> 7ff627f)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 793d95b Make request/response body as an option for AuTest microserver add 7ff627f Ensure TSContSchedule API family are called from an EThread. No new revisions were added by this update. Summary of changes: doc/developer-guide/api/functions/TSContSchedule.en.rst | 3 +++ .../api/functions/TSContScheduleOnPool.en.rst| 3 +++ .../api/functions/TSContScheduleOnThread.en.rst | 3 +++ src/traffic_server/InkAPI.cc | 12 4 files changed, 21 insertions(+)
[trafficserver] branch master updated: Add FetchSM support to dechunking in non-streaming mode. Add new TS API TSFetchFlagSet()
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 7675d0b Add FetchSM support to dechunking in non-streaming mode. Add new TS API TSFetchFlagSet() 7675d0b is described below commit 7675d0b8082247de35146cedbce6875cb2d39f03 Author: Sudheer Vinukonda AuthorDate: Wed Mar 18 16:17:27 2020 -0700 Add FetchSM support to dechunking in non-streaming mode. Add new TS API TSFetchFlagSet() --- .../api/functions/TSFetchCreate.en.rst | 55 ++ include/ts/experimental.h | 10 src/traffic_server/FetchSM.cc | 34 +++-- src/traffic_server/FetchSM.h | 7 ++- src/traffic_server/InkAPI.cc | 7 +++ 5 files changed, 109 insertions(+), 4 deletions(-) diff --git a/doc/developer-guide/api/functions/TSFetchCreate.en.rst b/doc/developer-guide/api/functions/TSFetchCreate.en.rst new file mode 100644 index 000..9fe8ed5 --- /dev/null +++ b/doc/developer-guide/api/functions/TSFetchCreate.en.rst @@ -0,0 +1,55 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. include:: ../../../common.defs + +.. default-domain:: c + +TSFetchCreate +* + +Traffic Server asynchronous Fetch API. + +Synopsis + + +.. code-block:: cpp + +#include + +.. function:: void TSFetchPages(TSFetchUrlParams_t *) +.. function:: void TSFetchUrl(const char *, int, sockaddr const *, TSCont, TSFetchWakeUpOptions, TSFetchEvent) +.. function:: void TSFetchFlagSet(TSFetchSM, int) +.. function:: TSFetch TSFetchCreate(TSCont, const char *, const char *, const char *, struct sockaddr const *, int) +.. function:: void TSFetchHeaderAdd(TSFetchSM, const char *, int, const char *, int) +.. function:: void TSFetchWriteData(TSFetchSM, const void *, size_t) +.. function:: ssize_t TSFetchReadData(TSFetchSM, void *, size_t) +.. function:: void TSFetchLaunch(TSFetchSM) +.. function:: void TSFetchDestroy(TSFetchSM) +.. function:: void TSFetchUserDataSet(TSFetchSM, void *) +.. function:: void* TSFetchUserDataGet(TSFetchSM) +.. function:: TSMBuffer TSFetchRespHdrMBufGet(TSFetchSM) +.. function:: TSMLoc TSFetchRespHdrMLocGet(TSFetchSM) + +Description +=== + +Traffic Server provides a number of routines for fetching resources asynchronously. +These API are useful to support a number of use cases that may involve sideways +calls, while handling the client request. Some typical examples include centralized +rate limiting framework, database lookups for login/authentication, refreshing configs +in the background asynchronously, ESI etc. diff --git a/include/ts/experimental.h b/include/ts/experimental.h index 00b9ae7..b731127 100644 --- a/include/ts/experimental.h +++ b/include/ts/experimental.h @@ -380,6 +380,16 @@ tsapi TSFetchSM TSFetchCreate(TSCont contp, const char *method, const char *url, struct sockaddr const *client_addr, int flags); /* + * Set fetch flags to FetchSM Context + * + * @param fetch_sm: returned value of TSFetchCreate(). + * @param flags: can be bitwise OR of several TSFetchFlags. + * + * return void + */ +tsapi void TSFetchFlagSet(TSFetchSM fetch_sm, int flags); + +/* * Create FetchSM, this API will enable stream IO automatically. * * @param fetch_sm: returned value of TSFetchCreate(). diff --git a/src/traffic_server/FetchSM.cc b/src/traffic_server/FetchSM.cc index 9a01e18..f682311 100644 --- a/src/traffic_server/FetchSM.cc +++ b/src/traffic_server/FetchSM.cc @@ -359,6 +359,7 @@ void FetchSM::get_info_from_buffer(IOBufferReader *reader) { char *buf, *info; + IOBufferBlock *blk; int64_t read_avail, read_done; if (!reader) { @@ -376,15 +377,42 @@ FetchSM::get_info_from_buffer(IOBufferReader *reader) info= (char *)ats_malloc(sizeof(char) * (read_avail + 1)); client_response = info; - // To maintain backwards compatibility we don't allow chunking when it's not streaming. - if (!(fetch_flags & TS_FETCH_FLAGS_STREAM) ||
[trafficserver] branch master updated (6e955a8 -> 889e0fa)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 6e955a8 cache_range_requests plugin: detect and handle TSCacheUrlSet failures which poison the cache (#6464) add 889e0fa Issue #6400 - Adds config option to enable/disable dynamic reload feature for plugins => proxy.config.plugin.dynamic_reload_mode - 1 (default) enables the dynamic reload feature, 0 disables it => Adds to and refactors unit-tests for the dynamic plugin reload feature No new revisions were added by this update. Summary of changes: doc/admin-guide/files/records.config.en.rst| 5 + .../design-documents/reloading-plugins.en.rst | 6 + mgmt/RecordsConfig.cc | 2 + proxy/Plugin.cc| 27 + proxy/Plugin.h | 12 + proxy/http/remap/PluginDso.cc | 38 +- proxy/http/remap/PluginDso.h | 6 +- proxy/http/remap/PluginFactory.cc | 32 +- proxy/http/remap/PluginFactory.h | 4 +- proxy/http/remap/RemapConfig.cc| 3 +- .../http/remap/unit-tests/plugin_testing_common.cc | 21 + .../http/remap/unit-tests/plugin_testing_common.h | 4 + proxy/http/remap/unit-tests/test_PluginFactory.cc | 583 ++--- src/traffic_server/traffic_server.cc | 2 + 14 files changed, 644 insertions(+), 101 deletions(-)
[trafficserver] branch master updated (aa903d9 -> 0f1f028)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from aa903d9 Fixed encoding test to work with OpenSSL 1.0.2 add 0f1f028 Reset captive_action.cancelled during open read retry to prevent assert No new revisions were added by this update. Summary of changes: proxy/http/HttpCacheSM.cc | 2 ++ 1 file changed, 2 insertions(+)
[trafficserver] branch 8.0.x updated: Enhance Connection Collapse in ATS core
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch 8.0.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/8.0.x by this push: new f76c4c7 Enhance Connection Collapse in ATS core f76c4c7 is described below commit f76c4c77b09ca0675a82c2a0b50b0699bad8a43f Author: Sudheer Vinukonda AuthorDate: Tue Oct 22 19:16:21 2019 -0700 Enhance Connection Collapse in ATS core Add an option to support cache open read retry on a write lock failure. With this option, as long as read-while-writer is set up following the guidelines in the docs, there should be no need for any plugins to augment the core. Eventual plan is to deprecate collapsed_forwarding plugin with this new support. For more context on this, see https://cwiki.apache.org/confluence/display/TS/Presentations+-+2019?preview=/112821251/132320653/Collapsed%20Forwarding%20.pdf --- doc/admin-guide/files/records.config.en.rst | 9 + proxy/http/HttpCacheSM.cc | 55 + proxy/http/HttpSM.cc| 9 + proxy/http/HttpTransact.cc | 22 proxy/http/HttpTransact.h | 1 + 5 files changed, 76 insertions(+), 20 deletions(-) diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 3dc98a7..0213466 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -2181,6 +2181,9 @@ all the different user-agent versions of documents it encounters. The number of times to attempt a cache open write upon failure to get a write lock. +This config is ignored when :ts:cv:`proxy.config.http.cache.open_write_fail_action` is +set to ``5``. + .. ts:cv:: CONFIG proxy.config.http.cache.open_write_fail_action INT 0 :reloadable: :overridable: @@ -2203,6 +2206,12 @@ all the different user-agent versions of documents it encounters. :ts:cv:`proxy.config.http.cache.max_stale_age`. Otherwise, go to origin server. ``4`` Return a ``502`` error on either a cache miss or on a revalidation. + ``5`` Retry Cache Read on a Cache Write Lock failure. This option together + with `proxy.config.cache.enable_read_while_writer` configuration + allows to collapse concurrent requests without a need for any plugin. + Make sure to configure Read While Writer feature correctly following + the docs in Cache Basics section. Note that this option may result in + CACHE_LOOKUP_COMPLETE HOOK being called back more than once. = == Customizable User Response Pages diff --git a/proxy/http/HttpCacheSM.cc b/proxy/http/HttpCacheSM.cc index 52190bc..daf39c1 100644 --- a/proxy/http/HttpCacheSM.cc +++ b/proxy/http/HttpCacheSM.cc @@ -175,7 +175,8 @@ HttpCacheSM::state_cache_open_write(int event, void *data) { STATE_ENTER(&HttpCacheSM::state_cache_open_write, event); ink_assert(captive_action.cancelled == 0); - pending_action = nullptr; + pending_action= nullptr; + bool read_retry_on_write_fail = false; switch (event) { case CACHE_EVENT_OPEN_WRITE: @@ -187,9 +188,26 @@ HttpCacheSM::state_cache_open_write(int event, void *data) break; case CACHE_EVENT_OPEN_WRITE_FAILED: -if (open_write_tries <= master_sm->t_state.txn_conf->max_cache_open_write_retries) { +if (master_sm->t_state.txn_conf->cache_open_write_fail_action == HttpTransact::CACHE_WL_FAIL_ACTION_READ_RETRY) { + // fall back to open_read_tries + // Note that when CACHE_WL_FAIL_ACTION_READ_RETRY is configured, max_cache_open_write_retries + // is automatically ignored. Make sure to not disable max_cache_open_read_retries + // with CACHE_WL_FAIL_ACTION_READ_RETRY as this results in proxy'ing to origin + // without write retries in both a cache miss or a cache refresh scenario. + if (open_write_tries <= master_sm->t_state.txn_conf->max_cache_open_write_retries) { +Debug("http_cache", "[%" PRId64 "] [state_cache_open_write] cache open write failure %d. read retry triggered", + master_sm->sm_id, open_write_tries); +open_read_tries = 0; +read_retry_on_write_fail = true; +// make sure it doesn't loop indefinitely +open_write_tries = master_sm->t_state.txn_conf->max_cache_open_write_retries + 1; + } +} +if (read_retry_on_write_fail || open_write_tries <= master_sm->t_state.txn_conf->max_cache_open_write_retries) { // Retry open write; open_write_cb = false; + // reset captive_action since HttpSM cancelled it + captive_action.cancelled = 0; do_sche
[trafficserver] branch master updated (9836b0b -> 1f20e80)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 9836b0b Remove trailing white space from json formatter add 1f20e80 Skip unnecessary HostDB update on a fall back to serving stale record No new revisions were added by this update. Summary of changes: iocore/hostdb/HostDB.cc | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-)
[trafficserver] 02/02: Make sure shutdown_cont_event isn't holding any garbage references.
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit b0685dbd4d00f303ded4727a74f06d0a7cddcd93 Author: Sudheer Vinukonda AuthorDate: Fri Sep 27 10:15:41 2019 -0700 Make sure shutdown_cont_event isn't holding any garbage references. As it seems like there could be a race condition because the handling of HTTP2_SESSION_EVENT_SHUTDOWN_INIT is going through the event loop again and therefore may come in after the Session and its associated contexts are already cleaned up thus causing an use-after-free possibility --- proxy/http2/Http2ConnectionState.h | 1 + 1 file changed, 1 insertion(+) diff --git a/proxy/http2/Http2ConnectionState.h b/proxy/http2/Http2ConnectionState.h index 85245c4..f68db97 100644 --- a/proxy/http2/Http2ConnectionState.h +++ b/proxy/http2/Http2ConnectionState.h @@ -147,6 +147,7 @@ public: in_destroy = true; if (shutdown_cont_event) { shutdown_cont_event->cancel(); + shutdown_cont_event = nullptr; } cleanup_streams();
[trafficserver] 01/02: Add in_destroy to Http2ConnectionState to prevent double delete (similar to Http2ClientSession)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit e894d18608ec371d2b8976ad131b9f2cd0b2e832 Author: Sudheer Vinukonda AuthorDate: Mon Aug 26 20:21:20 2019 -0700 Add in_destroy to Http2ConnectionState to prevent double delete (similar to Http2ClientSession) Add zombie_event when destroy() is called more than once. --- proxy/http2/Http2ConnectionState.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/proxy/http2/Http2ConnectionState.h b/proxy/http2/Http2ConnectionState.h index afd4385..85245c4 100644 --- a/proxy/http2/Http2ConnectionState.h +++ b/proxy/http2/Http2ConnectionState.h @@ -140,6 +140,11 @@ public: void destroy() { +if (in_destroy) { + schedule_zombie_event(); + return; +} +in_destroy = true; if (shutdown_cont_event) { shutdown_cont_event->cancel(); } @@ -374,6 +379,7 @@ private: Http2StreamId continued_stream_id = 0; bool _scheduled = false; bool fini_received= false; + bool in_destroy = false; int recursion = 0; Http2ShutdownState shutdown_state = HTTP2_SHUTDOWN_NONE; Http2ErrorCode shutdown_reason= Http2ErrorCode::HTTP2_ERROR_MAX;
[trafficserver] branch master updated (62e4cb1 -> b0685db)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 62e4cb1 Fixes a corner case where the NextHop consistent hash ring may not be searched in it's entirety for an available host due to a premature wrapped ring indication. new e894d18 Add in_destroy to Http2ConnectionState to prevent double delete (similar to Http2ClientSession) new b0685db Make sure shutdown_cont_event isn't holding any garbage references. The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: proxy/http2/Http2ConnectionState.h | 7 +++ 1 file changed, 7 insertions(+)
[trafficserver] branch master updated: Add invalid config warning when cache open write fail and read retry are inconsistent
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 757ac78 Add invalid config warning when cache open write fail and read retry are inconsistent 757ac78 is described below commit 757ac78146dfa4aca18a4508ced2b9544d070f64 Author: Sudheer Vinukonda AuthorDate: Mon Dec 23 19:05:40 2019 -0800 Add invalid config warning when cache open write fail and read retry are inconsistent --- proxy/http/HttpCacheSM.cc | 10 -- proxy/http/HttpConfig.cc | 8 proxy/http/HttpConfig.h| 10 ++ proxy/http/HttpSM.cc | 8 proxy/http/HttpTransact.cc | 2 +- proxy/http/HttpTransact.h | 10 -- 6 files changed, 31 insertions(+), 17 deletions(-) diff --git a/proxy/http/HttpCacheSM.cc b/proxy/http/HttpCacheSM.cc index bea5264..8fbbb60 100644 --- a/proxy/http/HttpCacheSM.cc +++ b/proxy/http/HttpCacheSM.cc @@ -172,7 +172,7 @@ HttpCacheSM::state_cache_open_write(int event, void *data) break; case CACHE_EVENT_OPEN_WRITE_FAILED: -if (master_sm->t_state.txn_conf->cache_open_write_fail_action == HttpTransact::CACHE_WL_FAIL_ACTION_READ_RETRY) { +if (master_sm->t_state.txn_conf->cache_open_write_fail_action == CACHE_WL_FAIL_ACTION_READ_RETRY) { // fall back to open_read_tries // Note that when CACHE_WL_FAIL_ACTION_READ_RETRY is configured, max_cache_open_write_retries // is automatically ignored. Make sure to not disable max_cache_open_read_retries @@ -181,6 +181,12 @@ HttpCacheSM::state_cache_open_write(int event, void *data) if (open_write_tries <= master_sm->t_state.txn_conf->max_cache_open_write_retries) { Debug("http_cache", "[%" PRId64 "] [state_cache_open_write] cache open write failure %d. read retry triggered", master_sm->sm_id, open_write_tries); +if (master_sm->t_state.txn_conf->max_cache_open_read_retries <= 0) { + Debug("http_cache", +"[%" PRId64 "] [state_cache_open_write] invalid config, cache write fail set to" +" read retry, but, max_cache_open_read_retries is not enabled", +master_sm->sm_id); +} open_read_tries = 0; read_retry_on_write_fail = true; // make sure it doesn't loop indefinitely @@ -206,7 +212,7 @@ HttpCacheSM::state_cache_open_write(int event, void *data) break; case EVENT_INTERVAL: -if (master_sm->t_state.txn_conf->cache_open_write_fail_action == HttpTransact::CACHE_WL_FAIL_ACTION_READ_RETRY) { +if (master_sm->t_state.txn_conf->cache_open_write_fail_action == CACHE_WL_FAIL_ACTION_READ_RETRY) { Debug("http_cache", "[%" PRId64 "] [state_cache_open_write] cache open write failure %d. " "falling back to read retry...", diff --git a/proxy/http/HttpConfig.cc b/proxy/http/HttpConfig.cc index 6ee3c46..1284be7 100644 --- a/proxy/http/HttpConfig.cc +++ b/proxy/http/HttpConfig.cc @@ -1468,6 +1468,14 @@ HttpConfig::reconfigure() params->keepalive_internal_vc = INT_TO_BOOL(m_master.keepalive_internal_vc); params->oride.cache_open_write_fail_action = m_master.oride.cache_open_write_fail_action; + if (params->oride.cache_open_write_fail_action == CACHE_WL_FAIL_ACTION_READ_RETRY) { +if (params->oride.max_cache_open_read_retries <= 0 || params->oride.max_cache_open_write_retries <= 0) { + Warning("Invalid config, cache_open_write_fail_action (%d), max_cache_open_read_retries (%" PRIu64 "), " + "max_cache_open_write_retries (%" PRIu64 ")", + params->oride.cache_open_write_fail_action, params->oride.max_cache_open_read_retries, + params->oride.max_cache_open_write_retries); +} + } params->oride.cache_when_to_revalidate = m_master.oride.cache_when_to_revalidate; params->max_post_size = m_master.max_post_size; diff --git a/proxy/http/HttpConfig.h b/proxy/http/HttpConfig.h index 153..11a815c 100644 --- a/proxy/http/HttpConfig.h +++ b/proxy/http/HttpConfig.h @@ -330,6 +330,16 @@ enum { http_stat_count }; +enum CacheOpenWriteFailAction_t { + CACHE_WL_FAIL_ACTION_DEFAULT = 0x00, + CACHE_WL_FAIL_ACTION_ERROR_ON_MISS = 0x01, + CACHE_WL_FAIL_ACTION_STALE_ON_REVALIDATE = 0x02, + CACHE_WL_FAIL_ACTION_ERROR_ON_MISS_STALE_ON_REVALIDATE = 0x03, + CACHE_WL_FAIL_ACTION_ERROR_ON_MISS_OR_REVALIDATE = 0x04, + CACHE_WL_FAIL_ACTION_READ_RETRY= 0x05, + TOTAL_CACHE_WL_FAIL_ACTION_TYPES +}; + extern RecRawS
[trafficserver] branch master updated (cd2afa6 -> 994a2f0)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from cd2afa6 Lua plugin fix: Account for null in output from TSUrlPercentDecode. add 994a2f0 Run dos2unix on all files in tree No new revisions were added by this update. Summary of changes: .../data/www.customplugin204.test_get.txt | 4 +-- .../data/www.customtemplate204.test_get.txt| 4 +-- .../body_factory/data/www.default204.test_get.txt | 4 +-- .../body_factory/data/www.default304.test_get.txt | 4 +-- .../body_factory/data/www.example.test_get_200.txt | 6 ++--- .../body_factory/data/www.example.test_get_304.txt | 8 +++--- .../body_factory/data/www.example.test_head.txt| 6 ++--- .../data/www.example.test_head_200.txt | 6 ++--- .../headers/data/www.passthrough.test_get.txt | 4 +-- .../headers/data/www.redirect0.test_get.txt| 4 +-- .../headers/data/www.redirect301.test_get.txt | 4 +-- .../headers/data/www.redirect302.test_get.txt | 4 +-- .../headers/data/www.redirect307.test_get.txt | 4 +-- .../headers/data/www.redirect308.test_get.txt | 4 +-- .../headers/general-connection-failure-502.gold| 14 +- .../regex_remap/gold/regex_remap_crash.gold| 8 +++--- .../regex_remap/gold/regex_remap_smoke.gold| 8 +++--- .../regex_revalidate/gold/regex_reval-hit.gold | 20 +++ .../regex_revalidate/gold/regex_reval-miss.gold| 20 +++ .../regex_revalidate/gold/regex_reval-stale.gold | 20 +++ .../pluginTest/slice/gold/slice_200.stdout.gold| 16 ++-- .../pluginTest/slice/gold/slice_206.stdout.gold| 18 ++--- .../pluginTest/slice/gold/slice_first.stdout.gold | 18 ++--- .../pluginTest/slice/gold/slice_last.stderr.gold | 2 +- .../pluginTest/slice/gold/slice_last.stdout.gold | 18 ++--- .../pluginTest/slice/gold/slice_mid.stderr.gold| 2 +- .../pluginTest/slice/gold/slice_mid.stdout.gold| 18 ++--- .../pluginTest/slice/gold_error/crr.stdout.gold| 18 ++--- .../pluginTest/slice/gold_error/etag.stdout.gold | 18 ++--- .../pluginTest/slice/gold_error/lm.stdout.gold | 18 ++--- .../pluginTest/slice/gold_error/non206.stdout.gold | 20 +++ tests/gold_tests/pluginTest/url_sig/url_sig.gold | 30 +++--- tests/gold_tests/redirect/gold/redirect.gold | 6 ++--- 33 files changed, 179 insertions(+), 179 deletions(-)
[trafficserver] branch master updated: Update docs for SSL Handshake stats
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 032cedb Update docs for SSL Handshake stats 032cedb is described below commit 032cedbc752dfdec238b3b48f18f05e79b592057 Author: Sudheer Vinukonda AuthorDate: Tue Nov 5 08:13:41 2019 -0800 Update docs for SSL Handshake stats --- .../monitoring/statistics/core/ssl.en.rst | 22 -- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/doc/admin-guide/monitoring/statistics/core/ssl.en.rst b/doc/admin-guide/monitoring/statistics/core/ssl.en.rst index 34e35a9..3b6ee48 100644 --- a/doc/admin-guide/monitoring/statistics/core/ssl.en.rst +++ b/doc/admin-guide/monitoring/statistics/core/ssl.en.rst @@ -108,10 +108,28 @@ SSL/TLS The total amount of time spent performing SSL/TLS handshakes for new sessions since statistics collection began. -.. ts:stat:: global proxy.process.ssl.total_success_handshake_count integer +.. ts:stat:: global proxy.process.ssl.total_attempts_handshake_count_in integer :type: counter - The total number of SSL/TLS handshakes successfully performed since + The total number of inbound SSL/TLS handshake attempts received since + statistics collection began. + +.. ts:stat:: global proxy.process.ssl.total_success_handshake_count_in integer + :type: counter + + The total number of inbound SSL/TLS handshakes successfully performed since + statistics collection began. + +.. ts:stat:: global proxy.process.ssl.total_attempts_handshake_count_out integer + :type: counter + + The total number of outbound SSL/TLS handshake attempts made since + statistics collection began. + +.. ts:stat:: global proxy.process.ssl.total_success_handshake_count_out integer + :type: counter + + The total number of outbound SSL/TLS handshakes successfully performed since statistics collection began. .. ts:stat:: global proxy.process.ssl.total_ticket_keys_renewed integer
[trafficserver] branch master updated (bbcffa0 -> d22de24)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from bbcffa0 Removes the records.config.shadow alternative config file add d22de24 Add Metrics to track SSL Handshake attempts No new revisions were added by this update. Summary of changes: iocore/net/SSLNetVConnection.cc | 2 ++ iocore/net/SSLStats.cc | 4 iocore/net/SSLStats.h | 2 ++ 3 files changed, 8 insertions(+)
[trafficserver] branch master updated (35f31c6 -> 4877105)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 35f31c6 Minor fixes. add 4877105 set host name in TLS extension for SNI Host check in service side with sni policy verify_with_name_source. No new revisions were added by this update. Summary of changes: doc/admin-guide/files/records.config.en.rst | 16 +++- iocore/net/I_NetVConnection.h | 23 +++ iocore/net/P_UnixNetVConnection.h | 1 + iocore/net/SSLNetVConnection.cc | 9 + iocore/net/quic/QUICTLS_openssl.cc | 3 ++- proxy/http/HttpSM.cc| 11 +++ 6 files changed, 57 insertions(+), 6 deletions(-)
[trafficserver] branch master updated (2d0233f -> 180f723)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 2d0233f Doc: improve documentation for event loop statistics. add 180f723 Enhance Connection Collapse in ATS core No new revisions were added by this update. Summary of changes: doc/admin-guide/files/records.config.en.rst | 9 + proxy/http/HttpCacheSM.cc | 55 + proxy/http/HttpSM.cc| 9 + proxy/http/HttpTransact.cc | 22 proxy/http/HttpTransact.h | 1 + 5 files changed, 76 insertions(+), 20 deletions(-)
[trafficserver] branch master updated: Fix use-after-free problem related to logging headers
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 702c6ac Fix use-after-free problem related to logging headers 702c6ac is described below commit 702c6ac75cc416b988a1f8407baebeb1986e6bb3 Author: Sudheer Vinukonda AuthorDate: Wed Oct 2 09:27:22 2019 -0700 Fix use-after-free problem related to logging headers mime_header_value_set() has a coalesce logic to coalesce dead allocations in the header heap and in the process can free up previously allocated strings that Logging references (cquuc etc). When logging tries to access those fields subsequently it results in a use-after-free (caught using ASAN build). In the wipe field logging use case, there isn't a need to recreate new header heaps, just need to replace the fields with wiped values. So, added a new utility function to simply overwrite the fields. The fix has been retested with ASAN and looks good on prod host. --- proxy/hdrs/HdrTest.cc | 27 +++ proxy/hdrs/MIME.h | 25 ++--- proxy/logging/LogAccess.cc | 5 - 3 files changed, 53 insertions(+), 4 deletions(-) diff --git a/proxy/hdrs/HdrTest.cc b/proxy/hdrs/HdrTest.cc index 9d8e30f..b0f84d3 100644 --- a/proxy/hdrs/HdrTest.cc +++ b/proxy/hdrs/HdrTest.cc @@ -495,6 +495,33 @@ HdrTest::test_mime() obj_describe((HdrHeapObjImpl *)(hdr.m_mime), true); + const char *field_name = "Test_heap_reuse"; + + MIMEField *f = hdr.field_create(field_name, static_cast(strlen(field_name))); + ink_release_assert(f->m_ptr_value == nullptr); + + hdr.field_attach(f); + ink_release_assert(f->m_ptr_value == nullptr); + + const char *test_value = "mytest"; + + printf("Testing Heap Reuse..\n"); + hdr.field_value_set(f, "orig_value", strlen("orig_value")); + const char *m_ptr_value_orig = f->m_ptr_value; + hdr.field_value_set(f, test_value, strlen(test_value), true); + ink_release_assert(f->m_ptr_value != test_value); // should be copied + ink_release_assert(f->m_ptr_value == m_ptr_value_orig); // heap doesn't change + ink_release_assert(f->m_len_value == strlen(test_value)); + ink_release_assert(memcmp(f->m_ptr_value, test_value, f->m_len_value) == 0); + + m_ptr_value_orig = f->m_ptr_value; + const char *new_test_value = "myTest"; + hdr.field_value_set(f, new_test_value, strlen(new_test_value), false); + ink_release_assert(f->m_ptr_value != new_test_value); // should be copied + ink_release_assert(f->m_ptr_value != m_ptr_value_orig); // new heap + ink_release_assert(f->m_len_value == strlen(new_test_value)); + ink_release_assert(memcmp(f->m_ptr_value, new_test_value, f->m_len_value) == 0); + hdr.fields_clear(); hdr.destroy(); diff --git a/proxy/hdrs/MIME.h b/proxy/hdrs/MIME.h index ebf1677..b42f286 100644 --- a/proxy/hdrs/MIME.h +++ b/proxy/hdrs/MIME.h @@ -1036,7 +1036,7 @@ public: void value_append(const char *name, int name_length, const char *value, int value_length, bool prepend_comma = false, const char separator = ','); - void field_value_set(MIMEField *field, const char *value, int value_length); + void field_value_set(MIMEField *field, const char *value, int value_length, bool reuse_heaps = false); void field_value_set_int(MIMEField *field, int32_t value); void field_value_set_uint(MIMEField *field, uint32_t value); void field_value_set_int64(MIMEField *field, int64_t value); @@ -1092,6 +1092,12 @@ public: // No gratuitous copies & refcounts! MIMEHdr(const MIMEHdr &m) = delete; MIMEHdr &operator=(const MIMEHdr &m) = delete; + +private: + // Interface to replace (overwrite) field value without + // changing the heap as long as the new value is not longer + // than the current value + bool field_value_replace(MIMEField *field, const char *value, int value_length); }; /*- @@ -1397,10 +1403,23 @@ MIMEHdr::value_get_comma_list(const char *name, int name_length, StrList *list) /*- -*/ +inline bool +MIMEHdr::field_value_replace(MIMEField *field, const char *value, int value_length) +{ + if (field->m_len_value >= value_length) { +memcpy((char *)field->m_ptr_value, value, value_length); +field->m_len_value = value_length; +return true; + } + return false; +} + inline void -MIMEHdr::field_value_set(MIMEField *field, const char *value, int value_length) +MIMEHdr::field_value_set(MIMEField *field, const char *value, int
[trafficserver] branch master updated: Update Server IP in Transaction when attaching a session from the pool
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 9de4f24 Update Server IP in Transaction when attaching a session from the pool 9de4f24 is described below commit 9de4f2481556b3d7eca9c63813c5d4a8be23cbf1 Author: Sudheer Vinukonda AuthorDate: Tue Sep 3 14:05:44 2019 -0700 Update Server IP in Transaction when attaching a session from the pool IP may be mismatched when using Host based matching as the DNS IP is ignored which may cause txn logging (nhi) to be inaccurate Fix build --- proxy/http/HttpSM.cc | 11 +++ 1 file changed, 11 insertions(+) diff --git a/proxy/http/HttpSM.cc b/proxy/http/HttpSM.cc index a7c9a3d..793b82f 100644 --- a/proxy/http/HttpSM.cc +++ b/proxy/http/HttpSM.cc @@ -5888,6 +5888,17 @@ HttpSM::attach_server_session(Http1ServerSession *s) hsm_release_assert(s->state == HSS_ACTIVE); server_session= s; server_transact_count = server_session->transact_count++; + + // update the dst_addr when using an existing session + // for e.g using Host based session pools may ignore the DNS IP + if (!ats_ip_addr_eq(&t_state.current.server->dst_addr, &server_session->get_server_ip())) { +ip_port_text_buffer ipb1, ipb2; +Debug("http_ss", "updating ip when attaching server session from %s to %s", + ats_ip_ntop(&t_state.current.server->dst_addr.sa, ipb1, sizeof(ipb1)), + ats_ip_ntop(&server_session->get_server_ip(), ipb2, sizeof(ipb2))); +ats_ip_copy(&t_state.current.server->dst_addr, &server_session->get_server_ip()); + } + // Propagate the per client IP debugging if (ua_txn) { s->get_netvc()->control_flags.set_flags(get_cont_flags().get_flags());
[trafficserver] branch master updated (dc09101 -> 7e5093e)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from dc09101 Make TS_NULL_MLOC a valid C compile-time constant. add 7e5093e Make proxy.config.http.per_server.min_keep_alive_connections overridable No new revisions were added by this update. Summary of changes: doc/admin-guide/files/records.config.en.rst | 3 ++- include/ts/apidefs.h.in | 1 + mgmt/RecordsConfig.cc | 2 +- plugins/lua/ts_lua_http_config.c| 2 ++ proxy/http/HttpConfig.cc| 17 - proxy/http/HttpConfig.h | 6 +++--- proxy/http/HttpConnectionCount.cc | 19 ++- proxy/http/HttpConnectionCount.h| 20 proxy/http/HttpSM.cc| 2 +- proxy/http/HttpSessionManager.cc| 3 ++- src/traffic_server/InkAPI.cc| 5 + src/traffic_server/InkAPITest.cc| 1 + 12 files changed, 56 insertions(+), 25 deletions(-)
[trafficserver] branch master updated: Add protection against null pointer access
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 3db3932 Add protection against null pointer access 3db3932 is described below commit 3db39328b277c9cb7e21103bdcfdae4b22715f83 Author: Sudheer Vinukonda AuthorDate: Wed Aug 7 10:11:15 2019 -0700 Add protection against null pointer access Unmapped URLs are not initialized for short-circuited requests before remap as pristine url is not set until remap. Add protection against null pointer access. --- proxy/logging/LogAccess.cc | 8 proxy/logging/LogFilter.cc | 4 +++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/proxy/logging/LogAccess.cc b/proxy/logging/LogAccess.cc index ec8d781..d364e30 100644 --- a/proxy/logging/LogAccess.cc +++ b/proxy/logging/LogAccess.cc @@ -1159,7 +1159,7 @@ LogAccess::set_client_req_url_canon(char *buf, int len) void LogAccess::set_client_req_unmapped_url_canon(char *buf, int len) { - if (buf) { + if (buf && m_client_req_unmapped_url_canon_str) { m_client_req_unmapped_url_canon_len = len; ink_strlcpy(m_client_req_unmapped_url_canon_str, buf, m_client_req_unmapped_url_canon_len + 1); } @@ -1168,7 +1168,7 @@ LogAccess::set_client_req_unmapped_url_canon(char *buf, int len) void LogAccess::set_client_req_unmapped_url_path(char *buf, int len) { - if (buf) { + if (buf && m_client_req_unmapped_url_path_str) { m_client_req_unmapped_url_path_len = len; ink_strlcpy(m_client_req_unmapped_url_path_str, buf, m_client_req_unmapped_url_path_len + 1); } @@ -1177,7 +1177,7 @@ LogAccess::set_client_req_unmapped_url_path(char *buf, int len) void LogAccess::set_client_req_unmapped_url_host(char *buf, int len) { - if (buf) { + if (buf && m_client_req_unmapped_url_host_str) { m_client_req_unmapped_url_host_len = len; ink_strlcpy(m_client_req_unmapped_url_host_str, buf, m_client_req_unmapped_url_host_len + 1); } @@ -1187,7 +1187,7 @@ void LogAccess::set_client_req_url_path(char *buf, int len) { //?? use m_client_req_unmapped_url_path_str for now..may need to enhance later.. - if (buf) { + if (buf && m_client_req_unmapped_url_path_str) { m_client_req_url_path_len = len; ink_strlcpy(m_client_req_unmapped_url_path_str, buf, m_client_req_url_path_len + 1); } diff --git a/proxy/logging/LogFilter.cc b/proxy/logging/LogFilter.cc index 4bdec48..6091bd5 100644 --- a/proxy/logging/LogFilter.cc +++ b/proxy/logging/LogFilter.cc @@ -366,7 +366,9 @@ LogFilterString::wipe_this_entry(LogAccess *lad) ink_assert(!"INVALID FILTER OPERATOR"); } - m_field->updateField(lad, buf, strlen(buf)); + if (cond_satisfied) { +m_field->updateField(lad, buf, strlen(buf)); + } ats_free(big_buf); ats_free(big_buf_upper);
[trafficserver] 01/02: Preserve the raw log fields when wiping using case insensitive contains
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 1df04706d5f28ec11fdebdfc1d03b6df7e46673b Author: Sudheer Vinukonda AuthorDate: Thu Jul 25 16:58:25 2019 -0700 Preserve the raw log fields when wiping using case insensitive contains Fix clang error --- proxy/logging/LogFilter.cc | 26 ++ proxy/logging/LogFilter.h | 40 ++-- 2 files changed, 40 insertions(+), 26 deletions(-) diff --git a/proxy/logging/LogFilter.cc b/proxy/logging/LogFilter.cc index b80887d..bebab28 100644 --- a/proxy/logging/LogFilter.cc +++ b/proxy/logging/LogFilter.cc @@ -313,9 +313,12 @@ LogFilterString::wipe_this_entry(LogAccess *lad) static const unsigned BUFSIZE = 1024; char small_buf[BUFSIZE]; - char *big_buf= nullptr; - char *buf= small_buf; - size_t marsh_len = m_field->marshal_len(lad); // includes null termination + char small_buf_upper[BUFSIZE]; + char *big_buf = nullptr; + char *big_buf_upper = nullptr; + char *buf = small_buf; + char *buf_upper = small_buf_upper; + size_t marsh_len= m_field->marshal_len(lad); // includes null termination if (marsh_len > BUFSIZE) { big_buf = (char *)ats_malloc(marsh_len); @@ -339,19 +342,25 @@ LogFilterString::wipe_this_entry(LogAccess *lad) // actual length, so we just use the fact that a MATCH is not possible // when marsh_len <= (length of the filter string) // -cond_satisfied = _checkConditionAndWipe(&strcmp, &buf, marsh_len, m_value, DATA_LENGTH_LARGER); +cond_satisfied = _checkConditionAndWipe(&strcmp, &buf, marsh_len, m_value, nullptr, DATA_LENGTH_LARGER); break; case CASE_INSENSITIVE_MATCH: -cond_satisfied = _checkConditionAndWipe(&strcasecmp, &buf, marsh_len, m_value, DATA_LENGTH_LARGER); +cond_satisfied = _checkConditionAndWipe(&strcasecmp, &buf, marsh_len, m_value, nullptr, DATA_LENGTH_LARGER); break; case CONTAIN: -cond_satisfied = _checkConditionAndWipe(&_isSubstring, &buf, marsh_len, m_value, DATA_LENGTH_LARGER); +cond_satisfied = _checkConditionAndWipe(&_isSubstring, &buf, marsh_len, m_value, nullptr, DATA_LENGTH_LARGER); break; case CASE_INSENSITIVE_CONTAIN: +if (big_buf) { + big_buf_upper = (char *)ats_malloc((unsigned int)marsh_len); + buf_upper = big_buf_upper; +} else { + buf = small_buf; // make clang happy +} for (size_t i = 0; i < marsh_len; i++) { - buf[i] = ParseRules::ink_toupper(buf[i]); + buf_upper[i] = ParseRules::ink_toupper(buf[i]); } -cond_satisfied = _checkConditionAndWipe(&_isSubstring, &buf, marsh_len, m_value_uppercase, DATA_LENGTH_LARGER); +cond_satisfied = _checkConditionAndWipe(&_isSubstring, &buf_upper, marsh_len, m_value_uppercase, &buf, DATA_LENGTH_LARGER); break; default: ink_assert(!"INVALID FILTER OPERATOR"); @@ -360,6 +369,7 @@ LogFilterString::wipe_this_entry(LogAccess *lad) m_field->updateField(lad, buf, strlen(buf)); ats_free(big_buf); + ats_free(big_buf_upper); return cond_satisfied; } diff --git a/proxy/logging/LogFilter.h b/proxy/logging/LogFilter.h index 4fe7c8e..45a2440 100644 --- a/proxy/logging/LogFilter.h +++ b/proxy/logging/LogFilter.h @@ -171,7 +171,7 @@ private: LengthCondition lc); inline bool _checkConditionAndWipe(OperatorFunction f, char **field_value, size_t field_value_length, char **val, - LengthCondition lc); + char **orig_field_value, LengthCondition lc); // -- member functions that are not allowed -- LogFilterString(); @@ -388,24 +388,28 @@ LogFilterString::_checkCondition(OperatorFunction f, const char *field_value, si --*/ static void -wipeField(char **dest, char *field) +wipeField(char **dest, char *field, char **orig_dest) { - char *buf_dest = *dest; + char *buf_dest = *dest; + char *buf_orig_dest = orig_dest ? *orig_dest : *dest; if (buf_dest) { -char *query_param = strstr(buf_dest, "?"); +char *query_param = strstr(buf_dest, "?"); +char *orig_query_param = strstr(buf_orig_dest, "?"); -if (!query_param) { +if (!query_param || !orig_query_param) { return; } -char *p1 = strstr(query_param, field); +char *p1 = strstr(query_param, field); +int field_pos = p1 - query_param; +p1= orig_query_param + field_pos; if (p1) { - char tmp_text[strlen(buf_dest) + 10]; + char tmp_text[strlen(buf_orig_dest) + 10]; char *temp_text = tmp_text; - memcpy(temp_text, buf_dest, (p1 - bu
[trafficserver] branch master updated (9772eb6 -> e068b76)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 9772eb6 Add soft limit for HTTP Request URI and Header field length. Add a default body_factory template when rejecting a request that's too long new 1df0470 Preserve the raw log fields when wiping using case insensitive contains new e068b76 Add support for updating Container fields as well The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: proxy/logging/LogAccess.cc | 49 ++ proxy/logging/LogAccess.h | 1 + proxy/logging/LogField.cc | 37 -- proxy/logging/LogField.h | 2 ++ proxy/logging/LogFilter.cc | 26 proxy/logging/LogFilter.h | 42 +-- 6 files changed, 128 insertions(+), 29 deletions(-)
[trafficserver] 02/02: Add support for updating Container fields as well
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit e068b7685f01594b8faa48792459f875b5e5c3ad Author: Sudheer Vinukonda AuthorDate: Mon Jul 29 14:41:11 2019 -0700 Add support for updating Container fields as well Minor optimization Refactor argument names and signature per review comments Fix compile error after running clang-format Fix argument names to better clarify what they represent --- proxy/logging/LogAccess.cc | 49 +++ proxy/logging/LogAccess.h | 1 + proxy/logging/LogField.cc | 37 +++-- proxy/logging/LogField.h | 2 ++ proxy/logging/LogFilter.cc | 2 +- proxy/logging/LogFilter.h | 58 +++--- 6 files changed, 117 insertions(+), 32 deletions(-) diff --git a/proxy/logging/LogAccess.cc b/proxy/logging/LogAccess.cc index 91f6c7f..ec8d781 100644 --- a/proxy/logging/LogAccess.cc +++ b/proxy/logging/LogAccess.cc @@ -2796,3 +2796,52 @@ LogAccess::marshal_milestone_diff(TSMilestonesType ms1, TSMilestonesType ms2, ch } return INK_MIN_ALIGN; } + +void +LogAccess::set_http_header_field(LogField::Container container, char *field, char *buf, int len) +{ + HTTPHdr *header; + + switch (container) { + case LogField::CQH: + case LogField::ECQH: +header = m_client_request; +break; + + case LogField::PSH: + case LogField::EPSH: +header = m_proxy_response; +break; + + case LogField::PQH: + case LogField::EPQH: +header = m_proxy_request; +break; + + case LogField::SSH: + case LogField::ESSH: +header = m_server_response; +break; + + case LogField::CSSH: + case LogField::ECSSH: +header = m_cache_response; +break; + + default: +header = nullptr; +break; + } + + if (header && buf) { +MIMEField *fld = header->field_find(field, (int)::strlen(field)); +if (fld) { + // Loop over dups, update each of them + // + while (fld) { +header->value_set((const char *)field, (int)::strlen(field), buf, len); +fld = fld->m_next_dup; + } +} + } +} diff --git a/proxy/logging/LogAccess.h b/proxy/logging/LogAccess.h index 264ccea..1a8a2ef 100644 --- a/proxy/logging/LogAccess.h +++ b/proxy/logging/LogAccess.h @@ -280,6 +280,7 @@ public: inkcoreapi int marshal_milestone_fmt_time(TSMilestonesType ms, char *buf); inkcoreapi int marshal_milestone_fmt_ms(TSMilestonesType ms, char *buf); inkcoreapi int marshal_milestone_diff(TSMilestonesType ms1, TSMilestonesType ms2, char *buf); + inkcoreapi void set_http_header_field(LogField::Container container, char *field, char *buf, int len); // // unmarshalling routines // diff --git a/proxy/logging/LogField.cc b/proxy/logging/LogField.cc index 563638d..fab35da 100644 --- a/proxy/logging/LogField.cc +++ b/proxy/logging/LogField.cc @@ -29,6 +29,7 @@ ***/ #include "tscore/ink_platform.h" +#include "MIME.h" #include "LogUtils.h" #include "LogField.h" #include "LogBuffer.h" @@ -326,7 +327,7 @@ LogField::LogField(const char *field, Container container, SetFunc _setfunc) m_milestone2(TS_MILESTONE_LAST_ENTRY), m_time_field(false), m_alias_map(nullptr), -m_set_func(_setfunc) +m_set_func(nullptr) { ink_assert(m_name != nullptr); ink_assert(m_symbol != nullptr); @@ -462,13 +463,39 @@ LogField::marshal_len(LogAccess *lad) } } +bool +LogField::isContainerUpdateFieldSupported(Container container) +{ + switch (container) { + case CQH: + case PSH: + case PQH: + case SSH: + case CSSH: + case ECQH: + case EPSH: + case EPQH: + case ESSH: + case ECSSH: + case SCFG: +return true; + default: +return false; + } +} + void LogField::updateField(LogAccess *lad, char *buf, int len) { if (m_container == NO_CONTAINER) { return (lad->*m_set_func)(buf, len); + } else { +if (isContainerUpdateFieldSupported(m_container)) { + return set_http_header_field(lad, m_container, this->m_name, buf, len); +} else { + // no set function defined for the container +} } - // else...// future enhancement } /*- @@ -694,6 +721,12 @@ LogField::fieldlist_contains_aggregates(const char *fieldlist) return false; } +void +LogField::set_http_header_field(LogAccess *lad, LogField::Container container, char *field, char *buf, int len) +{ + return lad->set_http_header_field(container, field, buf, len); +} + /*- LogFieldList diff --git a/proxy/logging/LogField.h b/proxy/logging/LogField.h index e02dba3..5b19aac 100644 --- a/proxy/
[trafficserver] branch master updated: Add soft limit for HTTP Request URI and Header field length. Add a default body_factory template when rejecting a request that's too long
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 9772eb6 Add soft limit for HTTP Request URI and Header field length. Add a default body_factory template when rejecting a request that's too long 9772eb6 is described below commit 9772eb68b143ed74d1604ec90fa3714a283c4148 Author: Sudheer Vinukonda AuthorDate: Fri Jul 26 14:43:43 2019 -0700 Add soft limit for HTTP Request URI and Header field length. Add a default body_factory template when rejecting a request that's too long Add the soft limit check for MIME header field name as well And some code cleanup Fix the default error response body Fix build failure Add unit tests Add docs for the new settings Adjust the header field size control to include header name and value, to better align with the RFC definition Fix build error Simplify the header field size check inline with the config setting Fix the condition to match the docs accurately Fix the condition to match the docs accurately add new body_factory template to Makefile.am Fix format typo in Makefile dummy commit to trigger build which still seems to pick old file Yet another dummy commit to trigger build Address more review comments --- configs/body_factory/default/Makefile.am | 3 +- .../body_factory/default/request#uri_len_too_long | 15 ++ doc/admin-guide/files/records.config.en.rst| 14 ++ doc/admin-guide/monitoring/error-messages.en.rst | 7 +++ mgmt/RecordsConfig.cc | 4 ++ proxy/hdrs/HTTP.cc | 19 +--- proxy/hdrs/HTTP.h | 15 -- proxy/hdrs/HdrTSOnly.cc| 6 ++- proxy/hdrs/HdrTest.cc | 54 ++ proxy/hdrs/HdrTest.h | 1 + proxy/hdrs/MIME.cc | 6 +-- proxy/hdrs/MIME.h | 9 ++-- proxy/http/HttpConfig.cc | 6 +++ proxy/http/HttpConfig.h| 3 ++ proxy/http/HttpSM.cc | 9 +++- proxy/http/HttpTransact.cc | 17 ++- 16 files changed, 164 insertions(+), 24 deletions(-) diff --git a/configs/body_factory/default/Makefile.am b/configs/body_factory/default/Makefile.am index 307284b..953037c 100644 --- a/configs/body_factory/default/Makefile.am +++ b/configs/body_factory/default/Makefile.am @@ -44,5 +44,6 @@ dist_bodyfactory_DATA = \ timeout\#activity \ timeout\#inactivity \ transcoding\#unsupported \ - urlrouting\#no_mapping + urlrouting\#no_mapping \ +request\#uri_len_too_long diff --git a/configs/body_factory/default/request#uri_len_too_long b/configs/body_factory/default/request#uri_len_too_long new file mode 100644 index 000..1b83c42 --- /dev/null +++ b/configs/body_factory/default/request#uri_len_too_long @@ -0,0 +1,15 @@ + + +URI Too Long + + + +URI Too Long + + + +Description: Could not process this request because +the request uri was longer than proxy.config.http.request_line_max_size + + + diff --git a/doc/admin-guide/files/records.config.en.rst b/doc/admin-guide/files/records.config.en.rst index 394d733..243d03a 100644 --- a/doc/admin-guide/files/records.config.en.rst +++ b/doc/admin-guide/files/records.config.en.rst @@ -1097,6 +1097,20 @@ mptcp This enables buffering the content for incoming ``POST`` requests. If enabled no outbound connection is made until the entire ``POST`` request has been buffered. +.. ts:cv:: CONFIG proxy.config.http.request_line_max_size INT 65535 + + Controls the maximum size, in bytes, of an HTTP Request Line in requests. Requests + with a request line exceeding this size will be treated as invalid and + rejected by the proxy. Note that the HTTP request line typically includes HTTP method, + request target and HTTP version string except when the request is made using absolute + URI in which case the request line may also include the request scheme and domain name. + +.. ts:cv:: CONFIG proxy.config.http.header_field_max_size INT 131070 + + Controls the maximum size, in bytes, of an HTTP header field in requests. Headers + in a request with the sum of their name and value that exceed this size will cause the + entire request to be treated as invalid and rejected by the proxy. + .. ts:cv:: CONFIG proxy.config.http.request_header_max_size INT 131072 Controls the maximum size, in bytes, of an HTTP header in requests. Headers diff --git a/doc/admin-guide/monitoring/error-messages.en.rst b/doc/a
[trafficserver] 01/02: Update docs to document wipe_field_action that we use in production
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit dd3d50055c75b5247662ce267c943ed40ceaf796 Author: Sudheer Vinukonda AuthorDate: Mon Jul 15 15:19:58 2019 -0700 Update docs to document wipe_field_action that we use in production --- doc/admin-guide/files/logging.yaml.en.rst | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/admin-guide/files/logging.yaml.en.rst b/doc/admin-guide/files/logging.yaml.en.rst index 96a6fe3..a53d053 100644 --- a/doc/admin-guide/files/logging.yaml.en.rst +++ b/doc/admin-guide/files/logging.yaml.en.rst @@ -159,14 +159,14 @@ given one. Filters --- -Two different type of filters are available: ``accept`` and ``reject``. They -may be used, optionally, to accept or reject logging for matching events. +Trafficserver supports different type of filters : ``accept``, ``reject`` and ``wipe_field_value``. +They may be used, optionally, to accept, reject logging or mask query param values for matching events. Filter objects are created by assigning them a ``name`` to be used later to -refer to the filter, as well as an ``action`` (either ``accept`` or -``reject``). ``Accept`` and ``reject`` filters require a ``condition`` against -which to match all events. The ``condition`` fields must be in the following -format:: +refer to the filter, as well as an ``action`` (either ``accept``, ``reject`` or +``wipe_field_value``). ``Accept``, ``reject`` or ``wipe_field_value` filters require +a ``condition`` against which to match all events. The ``condition`` fields must +be in the following format::
[trafficserver] branch master updated (fd5c84a -> 37bf053)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from fd5c84a Convert regression tests for XPACK into Catch based unit tests new dd3d500 Update docs to document wipe_field_action that we use in production new 37bf053 Fix formatting The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: doc/admin-guide/files/logging.yaml.en.rst | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-)
[trafficserver] 02/02: Fix formatting
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 37bf05307ed53e4f5cd65601aa67cb5e0d412d16 Author: Sudheer Vinukonda AuthorDate: Mon Jul 15 15:23:01 2019 -0700 Fix formatting --- doc/admin-guide/files/logging.yaml.en.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/admin-guide/files/logging.yaml.en.rst b/doc/admin-guide/files/logging.yaml.en.rst index a53d053..f4d6937 100644 --- a/doc/admin-guide/files/logging.yaml.en.rst +++ b/doc/admin-guide/files/logging.yaml.en.rst @@ -164,7 +164,7 @@ They may be used, optionally, to accept, reject logging or mask query param valu Filter objects are created by assigning them a ``name`` to be used later to refer to the filter, as well as an ``action`` (either ``accept``, ``reject`` or -``wipe_field_value``). ``Accept``, ``reject`` or ``wipe_field_value` filters require +``wipe_field_value``). ``Accept``, ``reject`` or ``wipe_field_value`` filters require a ``condition`` against which to match all events. The ``condition`` fields must be in the following format::
[trafficserver] 03/03: Fix build error for pre openssl-1.1.1
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 42b7694c418cd965252e8def0352deb010dc4ec2 Author: Sudheer Vinukonda AuthorDate: Thu Jun 20 14:51:54 2019 -0700 Fix build error for pre openssl-1.1.1 --- iocore/net/SSLNetVConnection.cc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index 7a460ca..b4ef157 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -1829,9 +1829,11 @@ SSLNetVConnection::increment_ssl_version_metric(int version) const case TLS1_2_VERSION: SSL_INCREMENT_DYN_STAT(ssl_total_tlsv12); break; +#ifdef TLS1_3_VERSION case TLS1_3_VERSION: SSL_INCREMENT_DYN_STAT(ssl_total_tlsv13); break; +#endif default: Debug("ssl", "Unrecognized SSL version %d", version); break;
[trafficserver] branch master updated (130dcd0 -> 42b7694)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git. from 130dcd0 Auto port selection for more autests new 59b02ba Add metrics to track SSLv3 and TLS versions new cac7766 Use SSL_version() directly instead of SSL_get_version() which returns a string (Thanks @maskit for the pointer). new 42b7694 Fix build error for pre openssl-1.1.1 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: iocore/net/P_SSLNetVConnection.h | 1 + iocore/net/SSLNetVConnection.cc | 30 +- iocore/net/SSLStats.cc | 12 iocore/net/SSLStats.h| 7 +++ 4 files changed, 49 insertions(+), 1 deletion(-)
[trafficserver] 02/03: Use SSL_version() directly instead of SSL_get_version() which returns a string (Thanks @maskit for the pointer).
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit cac7766f7483e70040ec81164d2a8cde5a9c4233 Author: Sudheer Vinukonda AuthorDate: Thu Jun 20 12:42:20 2019 -0700 Use SSL_version() directly instead of SSL_get_version() which returns a string (Thanks @maskit for the pointer). --- iocore/net/P_SSLNetVConnection.h | 3 +-- iocore/net/SSLNetVConnection.cc | 45 ++-- 2 files changed, 21 insertions(+), 27 deletions(-) diff --git a/iocore/net/P_SSLNetVConnection.h b/iocore/net/P_SSLNetVConnection.h index d093e73..bea84aa 100644 --- a/iocore/net/P_SSLNetVConnection.h +++ b/iocore/net/P_SSLNetVConnection.h @@ -355,8 +355,6 @@ public: int populate_protocol(std::string_view *results, int n) const override; const char *protocol_contains(std::string_view tag) const override; - void increment_ssl_version_metric(const char *version) const; - /** * Populate the current object based on the socket information in in the * con parameter and the ssl object in the arg parameter @@ -403,6 +401,7 @@ public: private: std::string_view map_tls_protocol_to_tag(const char *proto_string) const; bool update_rbio(bool move_to_socket); + void increment_ssl_version_metric(int version) const; enum SSLHandshakeStatus sslHandshakeStatus = SSL_HANDSHAKE_ONGOING; bool sslClientRenegotiationAbort = false; diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index 4d9444b..7a460ca 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -1271,7 +1271,7 @@ SSLNetVConnection::sslServerHandShakeEvent(int &err) const unsigned char *proto = nullptr; unsigned len = 0; - increment_ssl_version_metric(getSSLProtocol()); + increment_ssl_version_metric(SSL_version(ssl)); // If it's possible to negotiate both NPN and ALPN, then ALPN // is preferred since it is the server's preference. The server @@ -1814,32 +1814,27 @@ SSLNetVConnection::populate(Connection &con, Continuation *c, void *arg) } void -SSLNetVConnection::increment_ssl_version_metric(const char *version) const +SSLNetVConnection::increment_ssl_version_metric(int version) const { - if (version) { -// openSSL guarantees the case of the protocol string. -if (version[0] == 'T' && version[1] == 'L' && version[2] == 'S' && version[3] == 'v' && version[4] == '1') { - if (version[5] == 0) { -SSL_INCREMENT_DYN_STAT(ssl_total_tlsv1); - } else if (version[5] == '.' && version[7] == 0) { -switch (version[6]) { -case '1': - SSL_INCREMENT_DYN_STAT(ssl_total_tlsv11); - break; -case '2': - SSL_INCREMENT_DYN_STAT(ssl_total_tlsv12); - break; -case '3': - SSL_INCREMENT_DYN_STAT(ssl_total_tlsv13); - break; -default: - break; -} - } -} - } else if (version[0] == 'S' && version[1] == 'S' && version[2] == 'L' && version[3] == 'v' && version[4] == '3' && - version[5] == 0) { + switch (version) { + case SSL3_VERSION: SSL_INCREMENT_DYN_STAT(ssl_total_sslv3); +break; + case TLS1_VERSION: +SSL_INCREMENT_DYN_STAT(ssl_total_tlsv1); +break; + case TLS1_1_VERSION: +SSL_INCREMENT_DYN_STAT(ssl_total_tlsv11); +break; + case TLS1_2_VERSION: +SSL_INCREMENT_DYN_STAT(ssl_total_tlsv12); +break; + case TLS1_3_VERSION: +SSL_INCREMENT_DYN_STAT(ssl_total_tlsv13); +break; + default: +Debug("ssl", "Unrecognized SSL version %d", version); +break; } }
[trafficserver] 01/03: Add metrics to track SSLv3 and TLS versions
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 59b02bae506e2521894ccd071856317dba9344a6 Author: Sudheer Vinukonda AuthorDate: Tue Jun 18 10:01:08 2019 -0700 Add metrics to track SSLv3 and TLS versions --- iocore/net/P_SSLNetVConnection.h | 2 ++ iocore/net/SSLNetVConnection.cc | 33 - iocore/net/SSLStats.cc | 12 iocore/net/SSLStats.h| 7 +++ 4 files changed, 53 insertions(+), 1 deletion(-) diff --git a/iocore/net/P_SSLNetVConnection.h b/iocore/net/P_SSLNetVConnection.h index 23733a2..d093e73 100644 --- a/iocore/net/P_SSLNetVConnection.h +++ b/iocore/net/P_SSLNetVConnection.h @@ -355,6 +355,8 @@ public: int populate_protocol(std::string_view *results, int n) const override; const char *protocol_contains(std::string_view tag) const override; + void increment_ssl_version_metric(const char *version) const; + /** * Populate the current object based on the socket information in in the * con parameter and the ssl object in the arg parameter diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc index 16d8e1b..4d9444b 100644 --- a/iocore/net/SSLNetVConnection.cc +++ b/iocore/net/SSLNetVConnection.cc @@ -1267,11 +1267,12 @@ SSLNetVConnection::sslServerHandShakeEvent(int &err) SSL_INCREMENT_DYN_STAT_EX(ssl_total_handshake_time_stat, ssl_handshake_time); SSL_INCREMENT_DYN_STAT(ssl_total_success_handshake_count_in_stat); } - { const unsigned char *proto = nullptr; unsigned len = 0; + increment_ssl_version_metric(getSSLProtocol()); + // If it's possible to negotiate both NPN and ALPN, then ALPN // is preferred since it is the server's preference. The server // preference would not be meaningful if we let the client @@ -1812,6 +1813,36 @@ SSLNetVConnection::populate(Connection &con, Continuation *c, void *arg) return EVENT_DONE; } +void +SSLNetVConnection::increment_ssl_version_metric(const char *version) const +{ + if (version) { +// openSSL guarantees the case of the protocol string. +if (version[0] == 'T' && version[1] == 'L' && version[2] == 'S' && version[3] == 'v' && version[4] == '1') { + if (version[5] == 0) { +SSL_INCREMENT_DYN_STAT(ssl_total_tlsv1); + } else if (version[5] == '.' && version[7] == 0) { +switch (version[6]) { +case '1': + SSL_INCREMENT_DYN_STAT(ssl_total_tlsv11); + break; +case '2': + SSL_INCREMENT_DYN_STAT(ssl_total_tlsv12); + break; +case '3': + SSL_INCREMENT_DYN_STAT(ssl_total_tlsv13); + break; +default: + break; +} + } +} + } else if (version[0] == 'S' && version[1] == 'S' && version[2] == 'L' && version[3] == 'v' && version[4] == '3' && + version[5] == 0) { +SSL_INCREMENT_DYN_STAT(ssl_total_sslv3); + } +} + std::string_view SSLNetVConnection::map_tls_protocol_to_tag(const char *proto_string) const { diff --git a/iocore/net/SSLStats.cc b/iocore/net/SSLStats.cc index b15f5d6..5b466c5 100644 --- a/iocore/net/SSLStats.cc +++ b/iocore/net/SSLStats.cc @@ -205,6 +205,18 @@ SSLInitializeStatistics() RecRegisterRawStat(ssl_rsb, RECT_PROCESS, "proxy.process.ssl.ssl_ocsp_refresh_cert_failure", RECD_INT, RECP_PERSISTENT, (int)ssl_ocsp_refresh_cert_failure_stat, RecRawStatSyncCount); + /* SSL Version stats */ + RecRegisterRawStat(ssl_rsb, RECT_PROCESS, "proxy.process.ssl.ssl_total_sslv3", RECD_COUNTER, RECP_PERSISTENT, + (int)ssl_total_sslv3, RecRawStatSyncCount); + RecRegisterRawStat(ssl_rsb, RECT_PROCESS, "proxy.process.ssl.ssl_total_tlsv1", RECD_COUNTER, RECP_PERSISTENT, + (int)ssl_total_tlsv1, RecRawStatSyncCount); + RecRegisterRawStat(ssl_rsb, RECT_PROCESS, "proxy.process.ssl.ssl_total_tlsv11", RECD_COUNTER, RECP_PERSISTENT, + (int)ssl_total_tlsv11, RecRawStatSyncCount); + RecRegisterRawStat(ssl_rsb, RECT_PROCESS, "proxy.process.ssl.ssl_total_tlsv12", RECD_COUNTER, RECP_PERSISTENT, + (int)ssl_total_tlsv12, RecRawStatSyncCount); + RecRegisterRawStat(ssl_rsb, RECT_PROCESS, "proxy.process.ssl.ssl_total_tlsv13", RECD_COUNTER, RECP_PERSISTENT, + (int)ssl_total_tlsv13, RecRawStatSyncCount); + // Get and register the SSL cipher stats. Note that we are using the default SSL context to obtain // the cipher list. This means that the set of cipher
[trafficserver] branch master updated: Elevate privs to load TLS Session Ticket Key file
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/trafficserver.git The following commit(s) were added to refs/heads/master by this push: new 4eadecd Elevate privs to load TLS Session Ticket Key file 4eadecd is described below commit 4eadecdd2611c805b7f66406b10caa14d141d1b1 Author: Sudheer Vinukonda AuthorDate: Wed Jun 12 17:55:45 2019 -0700 Elevate privs to load TLS Session Ticket Key file --- iocore/net/SSLConfig.cc | 5 + 1 file changed, 5 insertions(+) diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc index f854883..bf933bc 100644 --- a/iocore/net/SSLConfig.cc +++ b/iocore/net/SSLConfig.cc @@ -568,6 +568,11 @@ SSLTicketParams::LoadTicket(bool &nochange) no_default_keyblock = ticket_params->default_global_keyblock == nullptr; } + // elevate/allow file access to root read only files/certs + uint32_t elevate_setting = 0; + REC_ReadConfigInteger(elevate_setting, "proxy.config.ssl.cert.load_elevated"); + ElevateAccess elevate_access(elevate_setting ? ElevateAccess::FILE_PRIVILEGE : 0); // destructor will demote for us + if (REC_ReadConfigStringAlloc(ticket_key_filename, "proxy.config.ssl.server.ticket_key.filename") == REC_ERR_OKAY && ticket_key_filename != nullptr) { ats_scoped_str ticket_key_path(Layout::relative_to(params->serverCertPathOnly, ticket_key_filename));
[trafficserver] branch svinukon_elevate_privs deleted (was 6a2db44)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch svinukon_elevate_privs in repository https://gitbox.apache.org/repos/asf/trafficserver.git. was 6a2db44 Elevate privileges when loading SSL Session Ticket key file The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[trafficserver] 01/01: Elevate privileges when loading SSL Session Ticket key file
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch svinukon_elevate_privs in repository https://gitbox.apache.org/repos/asf/trafficserver.git commit 6a2db443fca9a4d04f52c450f51c8b1e50aaac2a Author: Sudheer Vinukonda AuthorDate: Mon Jun 10 15:02:11 2019 -0700 Elevate privileges when loading SSL Session Ticket key file --- iocore/net/SSLConfig.cc | 5 + 1 file changed, 5 insertions(+) diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc index 0183800..36bd751 100644 --- a/iocore/net/SSLConfig.cc +++ b/iocore/net/SSLConfig.cc @@ -574,6 +574,11 @@ SSLTicketParams::LoadTicket(bool &nochange) no_default_keyblock = ticket_params->default_global_keyblock == nullptr; } + // elevate/allow file access to root read only files/certs + uint32_t elevate_setting = 0; + REC_ReadConfigInteger(elevate_setting, "proxy.config.ssl.cert.load_elevated"); + ElevateAccess elevate_access(elevate_setting ? ElevateAccess::FILE_PRIVILEGE : 0); // destructor will demote for us + if (REC_ReadConfigStringAlloc(ticket_key_filename, "proxy.config.ssl.server.ticket_key.filename") == REC_ERR_OKAY && ticket_key_filename != nullptr) { ats_scoped_str ticket_key_path(Layout::relative_to(params->serverCertPathOnly, ticket_key_filename));
[trafficserver] branch svinukon_elevate_privs created (now 6a2db44)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch svinukon_elevate_privs in repository https://gitbox.apache.org/repos/asf/trafficserver.git. at 6a2db44 Elevate privileges when loading SSL Session Ticket key file This branch includes the following new commits: new 6a2db44 Elevate privileges when loading SSL Session Ticket key file The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[trafficserver] 01/01: Merge pull request #645 from sudheerv/ts4452
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://git-dual.apache.org/repos/asf/trafficserver.git commit 286e660ac7e68c0a478f522b911ee931917c0a70 Merge: f320d1c 2063780 Author: Sudheer Vinukonda AuthorDate: Tue May 17 15:35:08 2016 -0700 Merge pull request #645 from sudheerv/ts4452 [TS-4452] Fix the type for open_write_fail_action to MgmtByte. proxy/http/HttpConfig.cc | 2 +- proxy/http/HttpConfig.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact "commits@trafficserver.apache.org" .
[trafficserver] branch master updated (f320d1c -> 286e660)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://git-dual.apache.org/repos/asf/trafficserver.git. from f320d1c Merge pull request #638 from calavera/fix_null_reference_in_tunnel adds 2063780 [TS-4452] Fix the type for open_write_fail_action to MgmtByte. new 286e660 Merge pull request #645 from sudheerv/ts4452 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: proxy/http/HttpConfig.cc | 2 +- proxy/http/HttpConfig.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact ['"commits@trafficserver.apache.org" '].
[trafficserver] 01/01: Merge pull request #573 from sudheerv/dummy
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://git-dual.apache.org/repos/asf/trafficserver.git commit 6517827b2417d198c1e8cd4e2745e1183c799fb3 Merge: 400742a ecd317f Author: Sudheer Vinukonda AuthorDate: Fri Apr 15 09:03:57 2016 -0700 Merge pull request #573 from sudheerv/dummy Dummy commit to test github email. doc/admin-guide/storage/index.en.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact "commits@trafficserver.apache.org" .
[trafficserver] branch master updated (400742a -> 6517827)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://git-dual.apache.org/repos/asf/trafficserver.git. from 400742a TS-4250 fix (#563) adds ecd317f Dummy commit to test github email. new 6517827 Merge pull request #573 from sudheerv/dummy The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: doc/admin-guide/storage/index.en.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- To stop receiving notification emails like this one, please contact ['"commits@trafficserver.apache.org" '].
[trafficserver] branch master updated (8d4c256 -> acb3343)
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a change to branch master in repository https://git-dual.apache.org/repos/asf/trafficserver.git. from 8d4c256 TS-4340: fix small issue related to match typ e NONE adds ca28e79 [TS-3857] Change ERROR default to not log into syslogs. adds 26b4b43 [TS-3857] defer the default setting for 7.0 new acb3343 Merge pull request #567 from sudheerv/ts-3857 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "adds" were already present in the repository and have only been added to this reference. Summary of changes: proxy/logging/LogFile.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact ['"commits@trafficserver.apache.org" '].
[trafficserver] 01/01: Merge pull request #567 from sudheerv/ts-3857
This is an automated email from the ASF dual-hosted git repository. sudheerv pushed a commit to branch master in repository https://git-dual.apache.org/repos/asf/trafficserver.git commit acb3343a568981cb4fa9c9052d01bfbb648a9980 Merge: 8d4c256 26b4b43 Author: sudheerv AuthorDate: Thu Apr 14 15:06:39 2016 -0700 Merge pull request #567 from sudheerv/ts-3857 [TS-3857] Change ERROR default to not log into syslogs. proxy/logging/LogFile.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- To stop receiving notification emails like this one, please contact "commits@trafficserver.apache.org" .
[2/2] trafficserver git commit: update docs.
update docs. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/6991aecd Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/6991aecd Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/6991aecd Branch: refs/heads/master Commit: 6991aecd9cc0385cf5f76e0e0e444e8eb220f176 Parents: 2378e8a Author: Sudheer Vinukonda Authored: Sat Mar 5 03:28:03 2016 + Committer: Sudheer Vinukonda Committed: Sat Mar 5 03:28:03 2016 + -- plugins/experimental/collapsed_forwarding/README | 8 .../collapsed_forwarding/collapsed_forwarding.cc | 8 2 files changed, 16 insertions(+) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/6991aecd/plugins/experimental/collapsed_forwarding/README -- diff --git a/plugins/experimental/collapsed_forwarding/README b/plugins/experimental/collapsed_forwarding/README index 4aa2686..6cdd5c5 100644 --- a/plugins/experimental/collapsed_forwarding/README +++ b/plugins/experimental/collapsed_forwarding/README @@ -18,6 +18,14 @@ // proxy.config.http.background_fill_active_timeout 0 / // proxy.config.http.background_fill_completed_threshold 0 / +// Additionally, given that collapsed forwarding works based on cache write +// lock failure detection, the plugin requires cache to be enabled and ready. +// On a restart, Traffic Server typically takes a few seconds to initialize +// the cache depending on the cache size and number of dirents. While the +// cache is not ready yet, collapsed forwarding can not detect the write lock +// contention and so can not work. The setting proxy.config.http.wait_for_cache +// may be enabled which allows blocking incoming connections from being +// accepted until cache is ready. // This plugin currently supports only per-remap mode activation. http://git-wip-us.apache.org/repos/asf/trafficserver/blob/6991aecd/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc -- diff --git a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc index 18861c7..3a40e4c 100644 --- a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc +++ b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc @@ -39,6 +39,14 @@ // proxy.config.http.background_fill_active_timeout 0 / // proxy.config.http.background_fill_completed_threshold 0 / +// Additionally, given that collapsed forwarding works based on cache write +// lock failure detection, the plugin requires cache to be enabled and ready. +// On a restart, Traffic Server typically takes a few seconds to initialize +// the cache depending on the cache size and number of dirents. While the +// cache is not ready yet, collapsed forwarding can not detect the write lock +// contention and so can not work. The setting proxy.config.http.wait_for_cache +// may be enabled which allows blocking incoming connections from being +// accepted until cache is ready. // This plugin currently supports only per-remap mode activation.
[1/2] trafficserver git commit: update docs.
Repository: trafficserver Updated Branches: refs/heads/master 95b6f4750 -> 6991aecd9 update docs. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/2378e8a9 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/2378e8a9 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/2378e8a9 Branch: refs/heads/master Commit: 2378e8a92eef12144b3a30200d5fe6108e607058 Parents: 95b6f47 Author: Sudheer Vinukonda Authored: Sat Mar 5 03:26:37 2016 + Committer: Sudheer Vinukonda Committed: Sat Mar 5 03:26:37 2016 + -- doc/admin-guide/plugins/collapsed_forwarding.en.rst | 16 1 file changed, 12 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2378e8a9/doc/admin-guide/plugins/collapsed_forwarding.en.rst -- diff --git a/doc/admin-guide/plugins/collapsed_forwarding.en.rst b/doc/admin-guide/plugins/collapsed_forwarding.en.rst index f6c1607..65bd960 100644 --- a/doc/admin-guide/plugins/collapsed_forwarding.en.rst +++ b/doc/admin-guide/plugins/collapsed_forwarding.en.rst @@ -82,6 +82,14 @@ plugin to work:: :ts:cv:`proxy.config.http.background_fill_active_timeout` 0 :ts:cv:`proxy.config.http.background_fill_completed_threshold` 0 +Additionally, given that collapsed forwarding works based on cache write +lock failure detection, the plugin requires cache to be enabled and ready. +On a restart, Traffic Server typically takes a few seconds to initialize +the cache depending on the cache size and number of dirents. While the +cache is not ready yet, collapsed forwarding can not detect the write lock +contention and so can not work. The setting :ts:cv:`proxy.config.http.wait_for_cache` +may be enabled which allows blocking incoming connections from being +accepted until cache is ready. Description --- @@ -96,7 +104,7 @@ process intensive file at the Origin layer. This ultimately can cause stability problems on the origin layer disrupting the overall network performance. -ATS supports several kind of connection collapse mechanisms including +Traffic Server supports several kind of connection collapse mechanisms including Read-While-Writer (RWW), Stale-While-Revalidate (SWR) etc each very effective dealing with a majority of the use cases that can result in the Thundering herd problem. @@ -157,9 +165,9 @@ from the cache. However, the Open Read Retry can not help with the concurrent requests that hit (1.1) above, jumping to (3) directly. Only one such request will be able to obtain the exclusive write lock and all other requests are -leaked upstream. This is where, the recently developed ATS feature -Open Write Fail Action will help. The feature detects the write lock -failure and can return a stale copy for a Cache Revalidation or a +leaked upstream. This is where, the recently developed Traffic Server +feature Open Write Fail Action will help. The feature detects the write +lock failure and can return a stale copy for a Cache Revalidation or a 5xx status code for a Cache Miss with a special internal header <@Ats-Internal> that allows a TS plugin to take other special actions depending on the use-case.
trafficserver git commit: fix docs..
Repository: trafficserver Updated Branches: refs/heads/master 6bd9d1569 -> 62bbecfcc fix docs.. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/62bbecfc Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/62bbecfc Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/62bbecfc Branch: refs/heads/master Commit: 62bbecfcc7aa6d4409dfbf95fae04896ba56d950 Parents: 6bd9d15 Author: Sudheer Vinukonda Authored: Wed Mar 2 07:09:44 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 07:09:44 2016 + -- doc/admin-guide/plugins/collapsed_forwarding.en.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/62bbecfc/doc/admin-guide/plugins/collapsed_forwarding.en.rst -- diff --git a/doc/admin-guide/plugins/collapsed_forwarding.en.rst b/doc/admin-guide/plugins/collapsed_forwarding.en.rst index 7d36b04..f6c1607 100644 --- a/doc/admin-guide/plugins/collapsed_forwarding.en.rst +++ b/doc/admin-guide/plugins/collapsed_forwarding.en.rst @@ -21,7 +21,7 @@ Collapsed Forwarding Plugin under the License. -This is a plugin for Apache Traffic Server that allows you to achieve +This is a plugin for Apache Traffic Server that allows to achieve effective connection collapse by blocking all but one of the multiple concurrent requests for the same object from going to the Origin.
trafficserver git commit: fix docs..
Repository: trafficserver Updated Branches: refs/heads/master a73169751 -> 6bd9d1569 fix docs.. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/6bd9d156 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/6bd9d156 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/6bd9d156 Branch: refs/heads/master Commit: 6bd9d1569d0824e454789442b49130db3100a66e Parents: a731697 Author: Sudheer Vinukonda Authored: Wed Mar 2 07:08:20 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 07:08:20 2016 + -- .../plugins/collapsed_forwarding.en.rst | 25 ++-- 1 file changed, 12 insertions(+), 13 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/6bd9d156/doc/admin-guide/plugins/collapsed_forwarding.en.rst -- diff --git a/doc/admin-guide/plugins/collapsed_forwarding.en.rst b/doc/admin-guide/plugins/collapsed_forwarding.en.rst index f6fab48..7d36b04 100644 --- a/doc/admin-guide/plugins/collapsed_forwarding.en.rst +++ b/doc/admin-guide/plugins/collapsed_forwarding.en.rst @@ -21,11 +21,9 @@ Collapsed Forwarding Plugin under the License. -This is a plugin for Apache Traffic Server that allows you to proactively -fetch content from Origin in a way that it will fill the object into -cache. This is particularly useful when all (or most) of your client requests -are of the byte-Range type. The underlying problem being that Traffic Server -is not able to cache request / responses with byte ranges. +This is a plugin for Apache Traffic Server that allows you to achieve +effective connection collapse by blocking all but one of the multiple +concurrent requests for the same object from going to the Origin. Installation @@ -66,14 +64,15 @@ below to the specific remap line:: Functionality - -ATS plugin to allow collapsed forwarding of concurrent requests for the same -object. This plugin is based on open_write_fail_action feature, which detects -cache open write failure on a cache miss and returns a 502 error along with a -special @-header indicating the reason for 502 error. The plugin acts on the -error by using an internal redirect follow back to itself, essentially blocking -the request until a response arrives, at which point, relies on read-while-writer -feature to start downloading the object to all waiting clients. The following -config parameters are assumed to be set for this plugin to work:: +Traffic Server plugin to allow collapsed forwarding of concurrent requests for +the same object. This plugin is based on open_write_fail_action feature, which +detects cache open write failure on a cache miss and returns a 502 error along +with a special @-header indicating the reason for 502 error. The plugin acts +on the error by using an internal redirect follow back to itself, essentially +blocking the request until a response arrives, at which point, relies on +read-while-writer feature to start downloading the object to all waiting +clients. The following config parameters are assumed to be set for this +plugin to work:: :ts:cv:`proxy.config.http.cache.open_write_fail_action`1 :ts:cv:`proxy.config.cache.enable_read_while_writer` 1
trafficserver git commit: doc formatting..
Repository: trafficserver Updated Branches: refs/heads/master 587e2184f -> 2797931ab doc formatting.. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/2797931a Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/2797931a Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/2797931a Branch: refs/heads/master Commit: 2797931ab087b38aad9ed27ac277aaae54fb0ee9 Parents: 587e218 Author: Sudheer Vinukonda Authored: Wed Mar 2 02:31:19 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 02:31:19 2016 + -- doc/admin-guide/plugins/collapsed_forwarding.en.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/2797931a/doc/admin-guide/plugins/collapsed_forwarding.en.rst -- diff --git a/doc/admin-guide/plugins/collapsed_forwarding.en.rst b/doc/admin-guide/plugins/collapsed_forwarding.en.rst index f9adfd6..f6fab48 100644 --- a/doc/admin-guide/plugins/collapsed_forwarding.en.rst +++ b/doc/admin-guide/plugins/collapsed_forwarding.en.rst @@ -73,7 +73,7 @@ special @-header indicating the reason for 502 error. The plugin acts on the error by using an internal redirect follow back to itself, essentially blocking the request until a response arrives, at which point, relies on read-while-writer feature to start downloading the object to all waiting clients. The following -config parameters are assumed to be set for this plugin to work: +config parameters are assumed to be set for this plugin to work:: :ts:cv:`proxy.config.http.cache.open_write_fail_action`1 :ts:cv:`proxy.config.cache.enable_read_while_writer` 1
trafficserver git commit: more updates to index..
Repository: trafficserver Updated Branches: refs/heads/master 905608c44 -> 587e2184f more updates to index.. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/587e2184 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/587e2184 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/587e2184 Branch: refs/heads/master Commit: 587e2184f26a2836fad3ab956149bcfd44933ef5 Parents: 905608c Author: Sudheer Vinukonda Authored: Wed Mar 2 02:29:06 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 02:29:06 2016 + -- doc/admin-guide/plugins/index.en.rst | 4 1 file changed, 4 insertions(+) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/587e2184/doc/admin-guide/plugins/index.en.rst -- diff --git a/doc/admin-guide/plugins/index.en.rst b/doc/admin-guide/plugins/index.en.rst index 18354e4..de4084f 100644 --- a/doc/admin-guide/plugins/index.en.rst +++ b/doc/admin-guide/plugins/index.en.rst @@ -189,3 +189,7 @@ directory of the |TS| source tree. Experimental plugins can be compiled by passi :doc:`X-Debug ` Allows HTTP clients to debug the operation of the Traffic Server cache using the X-Debug header. +:doc:`Collapsed-Forwarding ` + Allows to Collapse multiple Concurrent requests by downloading once from the Origin and serving + all clients in parallel. +
trafficserver git commit: some doc indendation updates..
Repository: trafficserver Updated Branches: refs/heads/master fe6f0349a -> 905608c44 some doc indendation updates.. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/905608c4 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/905608c4 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/905608c4 Branch: refs/heads/master Commit: 905608c449da38e964256e182edf68f6435257e0 Parents: fe6f034 Author: Sudheer Vinukonda Authored: Wed Mar 2 02:03:50 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 02:03:50 2016 + -- .../plugins/collapsed_forwarding.en.rst | 110 +++ 1 file changed, 66 insertions(+), 44 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/905608c4/doc/admin-guide/plugins/collapsed_forwarding.en.rst -- diff --git a/doc/admin-guide/plugins/collapsed_forwarding.en.rst b/doc/admin-guide/plugins/collapsed_forwarding.en.rst index 2f75577..f9adfd6 100644 --- a/doc/admin-guide/plugins/collapsed_forwarding.en.rst +++ b/doc/admin-guide/plugins/collapsed_forwarding.en.rst @@ -86,41 +86,57 @@ config parameters are assumed to be set for this plugin to work: Description --- -Traffic Server has been affected severely by the Thundering Herd problem caused by its inability -to do effective connection collapse of multiple concurrent requests for the same segment. This is -especially critical when Traffic Server is used as a solution to use cases such as delivering a -large scale video live streaming. This problem results in a specific behavior where multiple number -of requests for the same file are leaked upstream to the Origin layer choking the upstream bandwidth -due to the duplicated large file downloads or process intensive file at the Origin layer. This -ultimately can cause stability problems on the origin layer disrupting the overall network performance. - -ATS supports several kind of connection collapse mechanisms including Read-While-Writer (RWW), -stale-while-revalidate (SWR) etc each very effective dealing with a majority of the use cases -that can result in the Thundering herd problem. +Traffic Server has been affected severely by the Thundering Herd problem caused +by its inability to do effective connection collapse of multiple concurrent +requests for the same segment. This is especially critical when Traffic Server +is used as a solution to use cases such as delivering a large scale video +live streaming. This problem results in a specific behavior where multiple +number of requests for the same file are leaked upstream to the Origin layer +choking the upstream bandwidth due to the duplicated large file downloads or +process intensive file at the Origin layer. This ultimately can cause +stability problems on the origin layer disrupting the overall network +performance. + +ATS supports several kind of connection collapse mechanisms including +Read-While-Writer (RWW), Stale-While-Revalidate (SWR) etc each very effective +dealing with a majority of the use cases that can result in the +Thundering herd problem. -For a large scale video streaming scenario, thereâs a combination of a large number of revalidations -(e.g. media playlists) and cache misses (e.g. media segments) that occur for the same file. Traffic Serverâs -RWW works great in collapsing the concurrent requests in such a scenario, however, as described in -``_admin-configuration-reducing-origin-requests``, Traffic Serverâs implementation of RWW has a significant -limitation, which restricts its ability to invoke RWW only when the response headers are already received. -This means that any number of concurrent requests for the same file that are received before the response -headers arrive are leaked upstream, which can result in a severe Thundering herd problem, depending on -the network latencies (which impact the TTFB for the response headers) at a given instant of time. +For a large scale Video Streaming scenario, thereâs a combination of a +large number of revalidations (e.g. media playlists) and cache misses +(e.g. media segments) that occur for the same file. Traffic Serverâs +RWW works great in collapsing the concurrent requests in such a scenario, +however, as described in ``_admin-configuration-reducing-origin-requests``, +Traffic Serverâs implementation of RWW has a significant limitation, which +restricts its ability to invoke RWW only when the response headers are +already received. This means that any number of concurrent requests for +the same file that are received before the response headers arrive are +leaked upstream, which can result in a severe Thundering herd problem, +depending on the network lat
trafficserver git commit: update index.
Repository: trafficserver Updated Branches: refs/heads/master 5a0db7c2c -> fe6f0349a update index. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/fe6f0349 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/fe6f0349 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/fe6f0349 Branch: refs/heads/master Commit: fe6f0349a1b6c831ce8001fd07791b22c68b9bf8 Parents: 5a0db7c Author: Sudheer Vinukonda Authored: Wed Mar 2 01:55:15 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 01:55:15 2016 + -- doc/admin-guide/plugins/index.en.rst | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/fe6f0349/doc/admin-guide/plugins/index.en.rst -- diff --git a/doc/admin-guide/plugins/index.en.rst b/doc/admin-guide/plugins/index.en.rst index f237749..18354e4 100644 --- a/doc/admin-guide/plugins/index.en.rst +++ b/doc/admin-guide/plugins/index.en.rst @@ -118,6 +118,7 @@ directory of the |TS| source tree. Experimental plugins can be compiled by passi TS Lua WebP Transform X-Debug + Collapsed-Forwarding :doc:`AuthProxy ` Delegates the authorization decision of a request to an external HTTP service.
trafficserver git commit: Docs for collapsed_forwarding plugin.
Repository: trafficserver Updated Branches: refs/heads/master 75d182c44 -> 5a0db7c2c Docs for collapsed_forwarding plugin. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/5a0db7c2 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/5a0db7c2 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/5a0db7c2 Branch: refs/heads/master Commit: 5a0db7c2c3731e5567026ff0e58ab501028ddac4 Parents: 75d182c Author: Sudheer Vinukonda Authored: Wed Mar 2 01:53:54 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 01:53:54 2016 + -- .../plugins/collapsed_forwarding.en.rst | 155 +++ 1 file changed, 155 insertions(+) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/5a0db7c2/doc/admin-guide/plugins/collapsed_forwarding.en.rst -- diff --git a/doc/admin-guide/plugins/collapsed_forwarding.en.rst b/doc/admin-guide/plugins/collapsed_forwarding.en.rst new file mode 100644 index 000..2f75577 --- /dev/null +++ b/doc/admin-guide/plugins/collapsed_forwarding.en.rst @@ -0,0 +1,155 @@ +.. _admin-plugins-collapsed-forwarding: + +Collapsed Forwarding Plugin +*** + +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + + +This is a plugin for Apache Traffic Server that allows you to proactively +fetch content from Origin in a way that it will fill the object into +cache. This is particularly useful when all (or most) of your client requests +are of the byte-Range type. The underlying problem being that Traffic Server +is not able to cache request / responses with byte ranges. + +Installation + + +To make this plugin available, you must either enable experimental plugins +when building |TS|:: + +./configure --enable-experimental-plugins + +Or use :program:`tsxs` to compile the plugin against your current |TS| build. +To do this, you must ensure that: + +#. Development packages for |TS| are installed. + +#. The :program:`tsxs` binary is in your path. + +#. The version of this plugin you are building, and the version of |TS| against + which you are building it are compatible. + +Once those conditions are satisfied, enter the source directory for the plugin +and perform the following:: + +make -f Makefile.tsxs +make -f Makefile.tsxs install + +Using the plugin + + +This plugin functions as a per remap plugin, and it takes two optional +arguments for specifying the delay between successive retries and a max +number of retries. + +To activate the plugin in per remap mode, in :file:`remap.config`, simply append the +below to the specific remap line:: + + @plugin=collapsed_forwarding.so @pparam=--delay= @pparam=--retries= + +Functionality +- + +ATS plugin to allow collapsed forwarding of concurrent requests for the same +object. This plugin is based on open_write_fail_action feature, which detects +cache open write failure on a cache miss and returns a 502 error along with a +special @-header indicating the reason for 502 error. The plugin acts on the +error by using an internal redirect follow back to itself, essentially blocking +the request until a response arrives, at which point, relies on read-while-writer +feature to start downloading the object to all waiting clients. The following +config parameters are assumed to be set for this plugin to work: + +:ts:cv:`proxy.config.http.cache.open_write_fail_action`1 +:ts:cv:`proxy.config.cache.enable_read_while_writer` 1 +:ts:cv:`proxy.config.http.redirection_enabled` 1 +:ts:cv:`proxy.config.http.number_of_redirections` 10 +:ts:cv:`proxy.config.http.redirect_use_orig_cache_key` 1 +:ts:cv:`proxy.config.http.background_fill_active_timeout` 0 +:ts:cv:`proxy.config.http.background_fill_completed_threshold` 0 + + +Description +--- +Traffic Server has been affected severely by the Thundering Herd problem caused by its inability +to do effective connecti
trafficserver git commit: Remove incorrect description.
Repository: trafficserver Updated Branches: refs/heads/master 0861ec445 -> 27cd0674e Remove incorrect description. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/27cd0674 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/27cd0674 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/27cd0674 Branch: refs/heads/master Commit: 27cd0674e171b2144210543eccb25ebeda864245 Parents: 0861ec4 Author: Sudheer Vinukonda Authored: Wed Mar 2 01:13:35 2016 + Committer: Sudheer Vinukonda Committed: Wed Mar 2 01:13:35 2016 + -- plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc | 3 --- 1 file changed, 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/27cd0674/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc -- diff --git a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc index 52f3077..bafe396 100644 --- a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc +++ b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc @@ -1,8 +1,5 @@ /** @file -Plugin to perform background fetches of certain content that would -otherwise not be cached. For example, Range: requests / responses. - @section license License Licensed to the Apache Software Foundation (ASF) under one
[trafficserver] Git Push Summary
Repository: trafficserver Updated Branches: refs/heads/ts4222 [deleted] 69381bb0a
[trafficserver] Git Push Summary
Repository: trafficserver Updated Branches: refs/heads/fcollapse [deleted] 0861ec445
[2/3] trafficserver git commit: Remove unnecessary rcsId definition.
Remove unnecessary rcsId definition. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/7cb5d963 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/7cb5d963 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/7cb5d963 Branch: refs/heads/master Commit: 7cb5d9632ed7cd4ed671469f58a654a1ff50ca93 Parents: 9e2c463 Author: Sudheer Vinukonda Authored: Mon Feb 29 18:45:41 2016 + Committer: Sudheer Vinukonda Committed: Mon Feb 29 18:45:41 2016 + -- .../experimental/collapsed_forwarding/collapsed_forwarding.cc | 5 - 1 file changed, 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/7cb5d963/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc -- diff --git a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc index 5bd4e7b..52f3077 100644 --- a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc +++ b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc @@ -46,11 +46,6 @@ // This plugin currently supports only per-remap mode activation. -#define UNUSED __attribute__((unused)) -static char UNUSED rcsId__collapsed_forwarding_cc[] = - "@(#) $Id: collapsed_forwarding.cc 218 2016-02-26 01:29:16Z sudheerv $ built on " __DATE__ " " __TIME__; - - #include #include #include
[1/3] trafficserver git commit: [TS-4243] Collapsed Forwarding Plugin based on Open Write Fail Action feature.
.redirect_use_orig_cache_key 1 / +// proxy.config.http.background_fill_active_timeout 0 / +// proxy.config.http.background_fill_completed_threshold 0 / + + +// This plugin currently supports only per-remap mode activation. + + +More details are available at + +https://docs.trafficserver.apache.org/en/6.0.x/admin/http-proxy-caching.en.html#reducing-origin-server-requests-avoiding-the-thundering-herd + +Installation: + +make +sudo make install + +If you don't have the traffic server binaries in your path, then you will need +to specify the path to tsxs manually: + +make TSXS=/opt/trafficserver/bin/tsxs +sudo make TSXS=/opt/trafficserver/bin/tsxs install + +Configuration: + +Add @plugin=cache_range_requests.so to your remap.config rules. + +Or for a global plugin where all range requests are processed, +Add cache_range_requests.so to the plugin.config + http://git-wip-us.apache.org/repos/asf/trafficserver/blob/9e2c463a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc -- diff --git a/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc new file mode 100644 index 000..5bd4e7b --- /dev/null +++ b/plugins/experimental/collapsed_forwarding/collapsed_forwarding.cc @@ -0,0 +1,314 @@ +/** @file + +Plugin to perform background fetches of certain content that would +otherwise not be cached. For example, Range: requests / responses. + +@section license License + +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + + +// collapsed_forwarding:: +// +// ATS plugin to allow collapsed forwarding of concurrent requests for the same +// object. This plugin is based on open_write_fail_action feature, which detects +// cache open write failure on a cache miss and returns a 502 error along with a +// special @-header indicating the reason for 502 error. The plugin acts on the +// error by using an internal redirect follow back to itself, essentially blocking +// the request until a response arrives, at which point, relies on read-while-writer +// feature to start downloading the object to all waiting clients. The following +// config parameters are assumed to be set for this plugin to work: + +// proxy.config.http.cache.open_write_fail_action1 / +// proxy.config.cache.enable_read_while_writer 1 / +// proxy.config.http.redirection_enabled 1 / +// proxy.config.http.number_of_redirections 10 / +// proxy.config.http.redirect_use_orig_cache_key 1 / +// proxy.config.http.background_fill_active_timeout 0 / +// proxy.config.http.background_fill_completed_threshold 0 / + + +// This plugin currently supports only per-remap mode activation. + + +#define UNUSED __attribute__((unused)) +static char UNUSED rcsId__collapsed_forwarding_cc[] = + "@(#) $Id: collapsed_forwarding.cc 218 2016-02-26 01:29:16Z sudheerv $ built on " __DATE__ " " __TIME__; + + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static const char *DEBUG_TAG = (char *)"collapsed_forwarding"; + +stat
[3/3] trafficserver git commit: update README.
update README. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/0861ec44 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/0861ec44 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/0861ec44 Branch: refs/heads/master Commit: 0861ec4457156030392d79ea5efb8ed11d20c4d0 Parents: 7cb5d96 Author: Sudheer Vinukonda Authored: Mon Feb 29 20:40:55 2016 + Committer: Sudheer Vinukonda Committed: Mon Feb 29 20:40:55 2016 + -- plugins/experimental/collapsed_forwarding/README | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0861ec44/plugins/experimental/collapsed_forwarding/README -- diff --git a/plugins/experimental/collapsed_forwarding/README b/plugins/experimental/collapsed_forwarding/README index 72fe545..4aa2686 100644 --- a/plugins/experimental/collapsed_forwarding/README +++ b/plugins/experimental/collapsed_forwarding/README @@ -39,8 +39,6 @@ to specify the path to tsxs manually: Configuration: -Add @plugin=cache_range_requests.so to your remap.config rules. - -Or for a global plugin where all range requests are processed, -Add cache_range_requests.so to the plugin.config - +Add @plugin=collapsed_forwarding.so to your remap.config rules. The plugin optionally +accepts a --delay= parameter and a --retries= parameter that can help tune +the total number of retries and the delay between consecutive retries.
trafficserver git commit: update README.
Repository: trafficserver Updated Branches: refs/heads/fcollapse 7cb5d9632 -> 0861ec445 update README. Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/0861ec44 Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/0861ec44 Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/0861ec44 Branch: refs/heads/fcollapse Commit: 0861ec4457156030392d79ea5efb8ed11d20c4d0 Parents: 7cb5d96 Author: Sudheer Vinukonda Authored: Mon Feb 29 20:40:55 2016 + Committer: Sudheer Vinukonda Committed: Mon Feb 29 20:40:55 2016 + -- plugins/experimental/collapsed_forwarding/README | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0861ec44/plugins/experimental/collapsed_forwarding/README -- diff --git a/plugins/experimental/collapsed_forwarding/README b/plugins/experimental/collapsed_forwarding/README index 72fe545..4aa2686 100644 --- a/plugins/experimental/collapsed_forwarding/README +++ b/plugins/experimental/collapsed_forwarding/README @@ -39,8 +39,6 @@ to specify the path to tsxs manually: Configuration: -Add @plugin=cache_range_requests.so to your remap.config rules. - -Or for a global plugin where all range requests are processed, -Add cache_range_requests.so to the plugin.config - +Add @plugin=collapsed_forwarding.so to your remap.config rules. The plugin optionally +accepts a --delay= parameter and a --retries= parameter that can help tune +the total number of retries and the delay between consecutive retries.