Filippo Giunchedi has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/338119 )
Change subject: udp2log: mirror traffic via udpmirror.py
......................................................................
udp2log: mirror traffic via udpmirror.py
udp2log requires line-oriented input, thus ditch socat and introduce a
simple python script to read lines from stdin and relay them via udp.
Note udpmirror is mostly intended to ease the fluorine -> mwlog1001
transition, it isn't intended to be used permanently.
Bug: T123728
Change-Id: I8f08411b187cd82ccaa781adefee6666421c9f2d
---
M modules/role/manifests/logging/mediawiki/udp2log.pp
A modules/udp2log/files/udpmirror.py
M modules/udp2log/manifests/instance.pp
M modules/udp2log/templates/filters.mw.erb
4 files changed, 55 insertions(+), 5 deletions(-)
Approvals:
Filippo Giunchedi: Verified; Looks good to me, approved
Volans: Looks good to me, but someone else must approve
diff --git a/modules/role/manifests/logging/mediawiki/udp2log.pp
b/modules/role/manifests/logging/mediawiki/udp2log.pp
index 41dc34d..b678d80 100644
--- a/modules/role/manifests/logging/mediawiki/udp2log.pp
+++ b/modules/role/manifests/logging/mediawiki/udp2log.pp
@@ -50,6 +50,13 @@
source => 'puppet:///modules/udp2log/demux.py',
}
+ file { '/usr/local/bin/udpmirror.py':
+ mode => '0544',
+ owner => 'root',
+ group => 'root',
+ source => 'puppet:///modules/udp2log/udpmirror.py',
+ }
+
$logstash_host = $::realm ? {
# TODO: Find a way to use multicast that doesn't cause duplicate
# messages to be stored in logstash. This is a SPOF.
diff --git a/modules/udp2log/files/udpmirror.py
b/modules/udp2log/files/udpmirror.py
new file mode 100755
index 0000000..2b700e2
--- /dev/null
+++ b/modules/udp2log/files/udpmirror.py
@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+
+#####################################################################
+# THIS FILE IS MANAGED BY PUPPET
+# puppet:///modules/udp2log/udpmirror.py
+#####################################################################
+
+# Simple python script to send stdin to another host line by line
+
+
+import argparse
+import logging
+import socket
+import sys
+
+
+DESCRIPTION = '''Relay stdin via UDP to the specified host/port, line by line.
+When destination host resolves to multiple addresses (v4 only), lines will be
+relayed to each.'''
+
+
+def main():
+ parser = argparse.ArgumentParser(description=DESCRIPTION)
+ parser.add_argument('host', help='Destination host')
+ parser.add_argument('port', help='Destination port')
+ args = parser.parse_args()
+
+ logging.basicConfig(level=logging.INFO)
+ log = logging.getLogger(__name__)
+ sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+ gai = socket.getaddrinfo(args.host, args.port, socket.AF_INET)
+
+ while True:
+ line = sys.stdin.readline()
+ if line == '':
+ break
+
+ for addrinfo in gai:
+ try:
+ sock.sendto(line, addrinfo[4])
+ except (socket.gaierror, socket.error):
+ log.exception('Error sending to %e:', addrinfo[4])
+ continue
+
+
+if __name__ == '__main__':
+ sys.exit(main())
diff --git a/modules/udp2log/manifests/instance.pp
b/modules/udp2log/manifests/instance.pp
index 9a72a1c..7bb5922 100644
--- a/modules/udp2log/manifests/instance.pp
+++ b/modules/udp2log/manifests/instance.pp
@@ -53,10 +53,6 @@
require_package('udplog')
- if $mirror_destinations {
- require_package('socat')
- }
-
base::service_unit { "udp2log-${name}":
ensure => $ensure,
sysvinit => true,
diff --git a/modules/udp2log/templates/filters.mw.erb
b/modules/udp2log/templates/filters.mw.erb
index 1ce1156..98525ac 100644
--- a/modules/udp2log/templates/filters.mw.erb
+++ b/modules/udp2log/templates/filters.mw.erb
@@ -17,7 +17,7 @@
<% if @mirror_destinations -%>
<% @mirror_destinations.each do |destination| -%>
-pipe 1 /usr/bin/socat - UDP-SENDTO:<%= destination %>:<%= @port %>
+pipe 1 /usr/local/bin/udpmirror.py <%= destination %> <%= @port %>
<% end -%>
<% end -%>
--
To view, visit https://gerrit.wikimedia.org/r/338119
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I8f08411b187cd82ccaa781adefee6666421c9f2d
Gerrit-PatchSet: 6
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Filippo Giunchedi <[email protected]>
Gerrit-Reviewer: BryanDavis <[email protected]>
Gerrit-Reviewer: Dzahn <[email protected]>
Gerrit-Reviewer: Filippo Giunchedi <[email protected]>
Gerrit-Reviewer: Volans <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
