Krinkle has uploaded a new change for review.
https://gerrit.wikimedia.org/r/239826
Change subject: varnish: Don't disable Cache-Control for all mobile traffic
......................................................................
varnish: Don't disable Cache-Control for all mobile traffic
Mobile-frontend traffic is currently overwriting Cache-Control on all requests.
One branch preserves some small amount of s-maxage (for third-party proxies, no
idea which or why). But both branches of this code disabled "max-age" which is
what browsers use.
The end result is that it forbid browsers to ever use local cache for load.php
requests. Since there is no "no-store" pragma here, browsers do still store it
for a short period of time. But much shorter than the otherwise-allowed 30 days
and since it has max-age=0 and must-revalidate the browser is forced to
rerequest all resources on every page view (usually getting "304 Not Modified").
This code appears related to the s-maxage stripping we have in text-frontend
(which exists to counter-act $wgUseSquid, which is only desired within the data
centre, not outside). However I'm not sure it really is that since it didn't
set it to 0 but to 300.
This code was introduced in 2011 with e1e13ed3d6. It wasn't a problem then
because we had load.php on bits.wikimedia.org. In May 2015 we migrated bits
to wiki domains. But mobile pages accidentally got configured to use load.php
from the desktop domain. Last month we fixed that (T106966) and the inevitable
finally happened. That switch increased load.php traffic by 30% from ~ 1.1M
to 1.4M reqs/minute. And it raised HTTP 304 ratio from under 1% to over 60%.
Bug: T113007
Change-Id: Icbcc04efaff41134bd406cf498e062c7edf41956
---
M templates/varnish/mobile-frontend.inc.vcl.erb
M templates/varnish/text-frontend.inc.vcl.erb
2 files changed, 13 insertions(+), 10 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/26/239826/1
diff --git a/templates/varnish/mobile-frontend.inc.vcl.erb
b/templates/varnish/mobile-frontend.inc.vcl.erb
index 5c73b50..88a0822 100644
--- a/templates/varnish/mobile-frontend.inc.vcl.erb
+++ b/templates/varnish/mobile-frontend.inc.vcl.erb
@@ -134,6 +134,13 @@
}
sub vcl_deliver {
+ // Strip s-maxage Cache-Control of wiki pages. The s-maxage still
applies to Varnish (sent
+ // by MediaWiki $wgUseSquid, sends purges internally). But pages musn't
be cached elsewhere.
+ // NOTE: Only apply to pages. Don't steal cachability of api.php,
load.php, etc. (T102898, T113007)
+ if ((req.url ~ "^/wiki/" || req.url ~ "^/w/index\.php") &&
resp.http.Cache-Control ~ "s-maxage=[1-9]") {
+ set resp.http.Cache-Control = "private, s-maxage=0, max-age=0,
must-revalidate";
+ }
+
<% if @vcl_config.fetch("enable_geoiplookup", false) -%>
// Perform GeoIP look-up and send the result as a session cookie
if (req.http.X-Orig-Cookie !~ "(^|;\s*)GeoIP=[^;]"
@@ -162,10 +169,4 @@
}
call analytics_last_access_deliver;
-
- if (resp.http.Cache-Control ~ "s-maxage=[1-9]") {
- set resp.http.Cache-Control = "s-maxage=300, must-revalidate,
max-age=0";
- } else {
- set resp.http.Cache-Control = "private, s-maxage=0, max-age=0,
must-revalidate";
- }
}
diff --git a/templates/varnish/text-frontend.inc.vcl.erb
b/templates/varnish/text-frontend.inc.vcl.erb
index 66e9e43..161cc54 100644
--- a/templates/varnish/text-frontend.inc.vcl.erb
+++ b/templates/varnish/text-frontend.inc.vcl.erb
@@ -149,10 +149,12 @@
}
sub vcl_deliver {
- // Override the Cache-Control header for wiki content pages
- // NOTE: Language variants URLs are not currently covered by these
- // regexps. Instead of writing regexps for every edge-case, we should
- // impose some order and coherence on our URL routing schemes.
+ // Strip s-maxage Cache-Control of wiki pages. The s-maxage still
applies to Varnish (sent
+ // by MediaWiki $wgUseSquid, sends purges internally). But pages musn't
be cached elsewhere.
+ // NOTE: Language variants URLs are not currently covered by these
regexps.
+ // Instead of writing regexps for every edge-case, we should impose
some order and coherence
+ // on our URL routing schemes.
+ // NOTE: Only apply to pages. Don't steal cachability of api.php,
load.php, etc. (T102898, T113007)
if (req.url ~ "^/wiki/" || req.url ~ "^/w/index\.php" || req.url ~
"^/\?title=") {
// ...but exempt CentralNotice banner special pages
if (req.url !~ "^/wiki/Special\:Banner") {
--
To view, visit https://gerrit.wikimedia.org/r/239826
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icbcc04efaff41134bd406cf498e062c7edf41956
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Krinkle <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
