[MediaWiki-commits] [Gerrit] operations/puppet[production]: varnishrls4: use VSL query and proper tags
Ema has submitted this change and it was merged. Change subject: varnishrls4: use VSL query and proper tags .. varnishrls4: use VSL query and proper tags Extract transactions from the VSM log with a VSL query matching ResourceLoader URLs. Further, use the right tags in process_transaction. Bug: T131353 Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc --- M modules/varnish/files/varnishrls4 1 file changed, 14 insertions(+), 15 deletions(-) Approvals: Elukey: Looks good to me, but someone else must approve Ema: Verified; Looks good to me, approved BBlack: Looks good to me, but someone else must approve diff --git a/modules/varnish/files/varnishrls4 b/modules/varnish/files/varnishrls4 index dd03f29..6949546 100755 --- a/modules/varnish/files/varnishrls4 +++ b/modules/varnish/files/varnishrls4 @@ -35,6 +35,7 @@ import varnishlog import varnishprocessor + class ResourceLoaderVarnishLogProcessor(varnishprocessor.VarnishLogProcessor): description = 'ResourceLoader Browser Cache Hit Ratio StatsD Reporter' key_prefix = 'ResourceLoader' @@ -44,10 +45,10 @@ status_code = transaction['RespStatus'] metric_keys = ['reqs.all', 'resps.' + status_code] -if 'BerespHeader' in transaction: +if 'ReqHeader' in transaction: metric_keys.append('reqs.if_none_match') -cache_control_header = transaction.get('BereqHeader') +cache_control_header = transaction.get('RespHeader') cache_control = 'no' if cache_control_header: match = re.search(r'(?<=max-age=)\d+', cache_control_header) @@ -63,20 +64,18 @@ self.flush_stats() def start(self): +# VSL query matching ResourceLoader ReqURLs +query = 'ReqURL ~ "^/w/load.php"' + varnishlog.varnishlog(( -('n', 'frontend'), # Consider the frontend Varnish instance -('c', None), # Only consider interactions with the client -('i', 'RespStatus'), # Get RespStatus for the HTTP status code -('i', 'BerespHeader'), # Get BerespHeader for If-None-Match header -('i', 'BereqHeader'), # Get BereqHeader for Cache-control header -('i', 'ReqURL'), # Get ReqURL to match /w/load.php -('i', 'Timestamp'),# Get Timestamp to delimit requests -('C', ''), # Use case-insensitive matching -('I', '^(/w/load\.php' # ...to match ResourceLoader ReqURLs - '|if-none-match:' # ...or If-None-Match BerespHeaders - '|cache-control:' # ...or Cache-control BereqHeaders - '|([\d.]+ ?){6}$' # ...or Timestamp lines - '|[1-5]\d{2}$)'), # ...or RxStatus codes +('q', query), # VSL query +('n', 'frontend'),# Consider the frontend Varnish instance +('i', 'RespStatus'), # Get RespStatus for the HTTP status code +('i', 'ReqURL'), # Get ReqURL to match /w/load.php +('i', 'Timestamp'), # Timestamp lines +('C', ''),# Use case-insensitive matching +('I', 'ReqHeader:if-none-match'), +('I', 'RespHeader:cache-control'), ), self.handle_log_record) lp = ResourceLoaderVarnishLogProcessor() -- To view, visit https://gerrit.wikimedia.org/r/316966 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc Gerrit-PatchSet: 3 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ema Gerrit-Reviewer: BBlack Gerrit-Reviewer: Elukey Gerrit-Reviewer: Ema Gerrit-Reviewer: jenkins-bot <> ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] operations/puppet[production]: varnishrls4: use VSL query and proper tags
Ema has uploaded a new change for review. https://gerrit.wikimedia.org/r/316966 Change subject: varnishrls4: use VSL query and proper tags .. varnishrls4: use VSL query and proper tags Extract transactions from the VSM log with a VSL query matching: - ResourceLoader URLs - requests with If-None-Match - responses with Cache-Control Further, use the right tags in process_transaction. Bug: T131353 Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc --- M modules/varnish/files/varnishrls4 1 file changed, 16 insertions(+), 15 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/66/316966/1 diff --git a/modules/varnish/files/varnishrls4 b/modules/varnish/files/varnishrls4 index dd03f29..15657c1 100755 --- a/modules/varnish/files/varnishrls4 +++ b/modules/varnish/files/varnishrls4 @@ -35,6 +35,7 @@ import varnishlog import varnishprocessor + class ResourceLoaderVarnishLogProcessor(varnishprocessor.VarnishLogProcessor): description = 'ResourceLoader Browser Cache Hit Ratio StatsD Reporter' key_prefix = 'ResourceLoader' @@ -44,10 +45,10 @@ status_code = transaction['RespStatus'] metric_keys = ['reqs.all', 'resps.' + status_code] -if 'BerespHeader' in transaction: +if 'ReqHeader' in transaction: metric_keys.append('reqs.if_none_match') -cache_control_header = transaction.get('BereqHeader') +cache_control_header = transaction.get('RespHeader') cache_control = 'no' if cache_control_header: match = re.search(r'(?<=max-age=)\d+', cache_control_header) @@ -63,20 +64,20 @@ self.flush_stats() def start(self): +query = ( +'ReqURL ~ "/w/load.php" or '# to match ResourceLoader ReqURLs +'ReqHeader: if-none-match or ' # or If-None-Match ReqHeaders +'RespHeader: cache-control')# or Cache-Control RespHeaders + varnishlog.varnishlog(( -('n', 'frontend'), # Consider the frontend Varnish instance -('c', None), # Only consider interactions with the client -('i', 'RespStatus'), # Get RespStatus for the HTTP status code -('i', 'BerespHeader'), # Get BerespHeader for If-None-Match header -('i', 'BereqHeader'), # Get BereqHeader for Cache-control header -('i', 'ReqURL'), # Get ReqURL to match /w/load.php -('i', 'Timestamp'),# Get Timestamp to delimit requests -('C', ''), # Use case-insensitive matching -('I', '^(/w/load\.php' # ...to match ResourceLoader ReqURLs - '|if-none-match:' # ...or If-None-Match BerespHeaders - '|cache-control:' # ...or Cache-control BereqHeaders - '|([\d.]+ ?){6}$' # ...or Timestamp lines - '|[1-5]\d{2}$)'), # ...or RxStatus codes +('q', query), # VSL query +('n', 'frontend'),# Consider the frontend Varnish instance +('i', 'RespStatus'), # Get RespStatus for the HTTP status code +('i', 'ReqURL'), # Get ReqURL to match /w/load.php +('i', 'Timestamp'), # Timestamp lines +('C', ''),# Use case-insensitive matching +('I', 'ReqHeader:if-none-match'), +('I', 'RespHeader:cache-control'), ), self.handle_log_record) lp = ResourceLoaderVarnishLogProcessor() -- To view, visit https://gerrit.wikimedia.org/r/316966 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ema ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits