[MediaWiki-commits] [Gerrit] operations/puppet[production]: varnishrls4: use VSL query and proper tags

2016-10-21 Thread Ema (Code Review) 
Ema has submitted this change and it was merged.

Change subject: varnishrls4: use VSL query and proper tags
..


varnishrls4: use VSL query and proper tags

Extract transactions from the VSM log with a VSL query matching
ResourceLoader URLs. Further, use the right tags in process_transaction.

Bug: T131353
Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc
---
M modules/varnish/files/varnishrls4
1 file changed, 14 insertions(+), 15 deletions(-)

Approvals:
  Elukey: Looks good to me, but someone else must approve
  Ema: Verified; Looks good to me, approved
  BBlack: Looks good to me, but someone else must approve



diff --git a/modules/varnish/files/varnishrls4 
b/modules/varnish/files/varnishrls4
index dd03f29..6949546 100755
--- a/modules/varnish/files/varnishrls4
+++ b/modules/varnish/files/varnishrls4
@@ -35,6 +35,7 @@
 import varnishlog
 import varnishprocessor
 
+
 class ResourceLoaderVarnishLogProcessor(varnishprocessor.VarnishLogProcessor):
 description = 'ResourceLoader Browser Cache Hit Ratio StatsD Reporter'
 key_prefix = 'ResourceLoader'
@@ -44,10 +45,10 @@
 status_code = transaction['RespStatus']
 metric_keys = ['reqs.all', 'resps.' + status_code]
 
-if 'BerespHeader' in transaction:
+if 'ReqHeader' in transaction:
 metric_keys.append('reqs.if_none_match')
 
-cache_control_header = transaction.get('BereqHeader')
+cache_control_header = transaction.get('RespHeader')
 cache_control = 'no'
 if cache_control_header:
 match = re.search(r'(?<=max-age=)\d+', cache_control_header)
@@ -63,20 +64,18 @@
 self.flush_stats()
 
 def start(self):
+# VSL query matching ResourceLoader ReqURLs
+query = 'ReqURL ~ "^/w/load.php"'
+
 varnishlog.varnishlog((
-('n', 'frontend'), # Consider the frontend Varnish instance
-('c', None),   # Only consider interactions with the client
-('i', 'RespStatus'),   # Get RespStatus for the HTTP status code
-('i', 'BerespHeader'), # Get BerespHeader for If-None-Match header
-('i', 'BereqHeader'),  # Get BereqHeader for Cache-control header
-('i', 'ReqURL'),   # Get ReqURL to match /w/load.php
-('i', 'Timestamp'),# Get Timestamp to delimit requests
-('C', ''), # Use case-insensitive matching
-('I', '^(/w/load\.php'  # ...to match ResourceLoader ReqURLs
-  '|if-none-match:' # ...or If-None-Match BerespHeaders
-  '|cache-control:' # ...or Cache-control BereqHeaders
-  '|([\d.]+ ?){6}$' # ...or Timestamp lines
-  '|[1-5]\d{2}$)'), # ...or RxStatus codes
+('q', query), # VSL query
+('n', 'frontend'),# Consider the frontend Varnish instance
+('i', 'RespStatus'),  # Get RespStatus for the HTTP status code
+('i', 'ReqURL'),  # Get ReqURL to match /w/load.php
+('i', 'Timestamp'),   # Timestamp lines
+('C', ''),# Use case-insensitive matching
+('I', 'ReqHeader:if-none-match'),
+('I', 'RespHeader:cache-control'),
 ), self.handle_log_record)
 
 lp = ResourceLoaderVarnishLogProcessor()

-- 
To view, visit https://gerrit.wikimedia.org/r/316966
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ema 
Gerrit-Reviewer: BBlack 
Gerrit-Reviewer: Elukey 
Gerrit-Reviewer: Ema 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: varnishrls4: use VSL query and proper tags

2016-10-20 Thread Ema (Code Review) 
Ema has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/316966

Change subject: varnishrls4: use VSL query and proper tags
..

varnishrls4: use VSL query and proper tags

Extract transactions from the VSM log with a VSL query matching:

- ResourceLoader URLs
- requests with If-None-Match
- responses with Cache-Control

Further, use the right tags in process_transaction.

Bug: T131353
Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc
---
M modules/varnish/files/varnishrls4
1 file changed, 16 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/66/316966/1

diff --git a/modules/varnish/files/varnishrls4 
b/modules/varnish/files/varnishrls4
index dd03f29..15657c1 100755
--- a/modules/varnish/files/varnishrls4
+++ b/modules/varnish/files/varnishrls4
@@ -35,6 +35,7 @@
 import varnishlog
 import varnishprocessor
 
+
 class ResourceLoaderVarnishLogProcessor(varnishprocessor.VarnishLogProcessor):
 description = 'ResourceLoader Browser Cache Hit Ratio StatsD Reporter'
 key_prefix = 'ResourceLoader'
@@ -44,10 +45,10 @@
 status_code = transaction['RespStatus']
 metric_keys = ['reqs.all', 'resps.' + status_code]
 
-if 'BerespHeader' in transaction:
+if 'ReqHeader' in transaction:
 metric_keys.append('reqs.if_none_match')
 
-cache_control_header = transaction.get('BereqHeader')
+cache_control_header = transaction.get('RespHeader')
 cache_control = 'no'
 if cache_control_header:
 match = re.search(r'(?<=max-age=)\d+', cache_control_header)
@@ -63,20 +64,20 @@
 self.flush_stats()
 
 def start(self):
+query = (
+'ReqURL ~ "/w/load.php" or '# to match ResourceLoader ReqURLs
+'ReqHeader: if-none-match or '  # or If-None-Match ReqHeaders
+'RespHeader: cache-control')# or Cache-Control RespHeaders
+
 varnishlog.varnishlog((
-('n', 'frontend'), # Consider the frontend Varnish instance
-('c', None),   # Only consider interactions with the client
-('i', 'RespStatus'),   # Get RespStatus for the HTTP status code
-('i', 'BerespHeader'), # Get BerespHeader for If-None-Match header
-('i', 'BereqHeader'),  # Get BereqHeader for Cache-control header
-('i', 'ReqURL'),   # Get ReqURL to match /w/load.php
-('i', 'Timestamp'),# Get Timestamp to delimit requests
-('C', ''), # Use case-insensitive matching
-('I', '^(/w/load\.php'  # ...to match ResourceLoader ReqURLs
-  '|if-none-match:' # ...or If-None-Match BerespHeaders
-  '|cache-control:' # ...or Cache-control BereqHeaders
-  '|([\d.]+ ?){6}$' # ...or Timestamp lines
-  '|[1-5]\d{2}$)'), # ...or RxStatus codes
+('q', query), # VSL query
+('n', 'frontend'),# Consider the frontend Varnish instance
+('i', 'RespStatus'),  # Get RespStatus for the HTTP status code
+('i', 'ReqURL'),  # Get ReqURL to match /w/load.php
+('i', 'Timestamp'),   # Timestamp lines
+('C', ''),# Use case-insensitive matching
+('I', 'ReqHeader:if-none-match'),
+('I', 'RespHeader:cache-control'),
 ), self.handle_log_record)
 
 lp = ResourceLoaderVarnishLogProcessor()

-- 
To view, visit https://gerrit.wikimedia.org/r/316966
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4b3a5ccc8dedb5ff8b51c3a22ca9cef8efb803dc
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ema 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits