Ottomata has submitted this change and it was merged.
Change subject: Fix for mobile-100 kafkatee output filter
......................................................................
Fix for mobile-100 kafkatee output filter
Since August 23, kafkatee has been consuming more than just the
mobile webrequest topic. We didn't have any filtering in place
for the mobile output, so it was saving much more than mobile
requests.
Change-Id: I6339e98e199ce633d71f8007c59ebab617291b89
---
M manifests/role/analytics/kafkatee.pp
1 file changed, 9 insertions(+), 2 deletions(-)
Approvals:
Ottomata: Verified; Looks good to me, approved
jenkins-bot: Verified
diff --git a/manifests/role/analytics/kafkatee.pp
b/manifests/role/analytics/kafkatee.pp
index a7eb4f3..62059f9 100644
--- a/manifests/role/analytics/kafkatee.pp
+++ b/manifests/role/analytics/kafkatee.pp
@@ -75,11 +75,18 @@
class role::analytics::kafkatee::webrequest::mobile inherits
role::analytics::kafkatee::webrequest {
include role::analytics::kafkatee::input::webrequest::mobile
- # 1/100 sampling of traffic to mobile varnishes
+ # Include this to infer mobile varnish frontend hostnames on which to
filter.
+ include role::cache::configuration
+ $cache_configuration =
$role::cache::configuration::active_nodes['production']['mobile']
+ $mobile_hosts_regex = inline_template('(<%=
@cache_configuration.values.flatten.sort.join(\'|\') %>)')
+
+ # 1/100 sampling of traffic from mobile varnishes
::kafkatee::output { 'mobile-sampled-100':
- destination =>
"${webrequest_log_directory}/mobile-sampled-100.tsv.log",
+ destination => "/bin/grep -P '${mobile_hosts_regex}' >>
${webrequest_log_directory}/mobile-sampled-100.tsv.log",
sample => 100,
+ type => 'pipe',
}
+
# Capture all logs with 'zero=' set. The X-Analytics header is set with
this
# by mobile varnish frontends upon getting a Wikipedia Zero request.
::kafkatee::output { 'zero':
--
To view, visit https://gerrit.wikimedia.org/r/159381
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I6339e98e199ce633d71f8007c59ebab617291b89
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <[email protected]>
Gerrit-Reviewer: Ottomata <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
