Filippo Giunchedi has submitted this change and it was merged. Change subject: txstatsd: add support for graphite line-protocol ......................................................................
txstatsd: add support for graphite line-protocol txstatsd uses graphite's native carbon client, which works well but only supports pickle format. However I couldn't find any non-graphite tools available that can route pickle protocol, we are going to use carbon-c-relay in the first iteration which is plaintext-only. See also http://graphite.readthedocs.org/en/latest/feeding-carbon.html#the-plaintext-protocol Changing carbon client code completely isn't an appealing option to me and we already ship txstatsd via custom debian packages anyway. So the ugly approach in this case is to monkey-patch the functions used to serialize metrics and restrict plaintext protocol support to txstatsd only :( Change-Id: I3b4230f8a358adaee7134bd586b503774574a31e --- M txstatsd.conf-example M txstatsd/service.py 2 files changed, 30 insertions(+), 1 deletion(-) Approvals: Filippo Giunchedi: Verified; Looks good to me, approved Faidon Liambotis: Looks good to me, but someone else must approve diff --git a/txstatsd.conf-example b/txstatsd.conf-example index 1119fa9..53eef1d 100644 --- a/txstatsd.conf-example +++ b/txstatsd.conf-example @@ -5,6 +5,8 @@ carbon-cache-port: 2003 # The UDP port where we will listen. listen-port: 8125 +# Write metrics using graphite's line-oriented protocol instead of pickle +use-line-protocol: 0 # The number of milliseconds between each flush. flush-interval: 60000 @@ -23,4 +25,4 @@ monitor-response: txstatsd pong [plugin_sample] -sample-key: sample-value \ No newline at end of file +sample-key: sample-value diff --git a/txstatsd/service.py b/txstatsd/service.py index f8721d7..bc77694 100644 --- a/txstatsd/service.py +++ b/txstatsd/service.py @@ -156,6 +156,8 @@ "The port where carbon cache is listening.", int], ["carbon-cache-name", "n", None, "An identifier for the carbon-cache instance."], + ["use-line-protocol", "P", 0, + "Use graphite line protocol when sending metrics."], ["listen-port", "l", 8125, "The UDP port where we will listen.", int], ["flush-interval", "i", 60000, @@ -252,6 +254,23 @@ return current_stats +def _CarbonClientLineProtocol_sendDatapoints(self, datapoints): + """Serialize data in line (or plaintext) format, one metric per line.""" + def _serialize(data): + return "".join("%s %s %s\n" % (x, y[1], y[0]) for x, y in data) + + from carbon import instrumentation + self.sendString(_serialize(datapoints)) + instrumentation.increment(self.sent, len(datapoints)) + self.factory.checkQueue() + + +def _Int32StringReceiver_sendString(self, string): + """Write data verbatim to the transport, StringReceiver standard + implementation would length-prefix the string instead.""" + self.transport.write(string) + + def createService(options): """Create a txStatsD service.""" from carbon.routers import ConsistentHashingRouter @@ -331,6 +350,14 @@ report_name.upper(), ()): reporting.schedule(reporter, 60, metrics.gauge) + # monkey-patch line protocol sending function :( + if options["use-line-protocol"]: + import carbon.client + carbon.client.CarbonClientProtocol._sendDatapoints = \ + _CarbonClientLineProtocol_sendDatapoints + carbon.client.CarbonClientProtocol.sendString = \ + _Int32StringReceiver_sendString + # XXX Make this configurable. router = ConsistentHashingRouter() carbon_client = CarbonClientManager(router) -- To view, visit https://gerrit.wikimedia.org/r/180786 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3b4230f8a358adaee7134bd586b503774574a31e Gerrit-PatchSet: 4 Gerrit-Project: operations/debs/txstatsd Gerrit-Branch: master Gerrit-Owner: Filippo Giunchedi <fgiunch...@wikimedia.org> Gerrit-Reviewer: Alexandros Kosiaris <akosia...@wikimedia.org> Gerrit-Reviewer: Chasemp <chas...@gmail.com> Gerrit-Reviewer: Faidon Liambotis <fai...@wikimedia.org> Gerrit-Reviewer: Filippo Giunchedi <fgiunch...@wikimedia.org> Gerrit-Reviewer: Yuvipanda <yuvipa...@gmail.com> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
