Elukey has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/370631 )
Change subject: Force Kafka protocol version ...................................................................... Force Kafka protocol version This change is the suggested workaround from the librdkafka authors to limit the effect of https://issues.apache.org/jira/browse/KAFKA-3547. More info: https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility Bug: T172681 Change-Id: I7a643fe17f9d70718b885e1de9b85494c70cc041 --- M manifests/instance.pp M templates/varnishkafka_v4.conf.erb 2 files changed, 17 insertions(+), 0 deletions(-) Approvals: Elukey: Looks good to me, approved jenkins-bot: Verified diff --git a/manifests/instance.pp b/manifests/instance.pp index 0b25c14..bd078eb 100644 --- a/manifests/instance.pp +++ b/manifests/instance.pp @@ -72,6 +72,10 @@ # changes. Default: true. # $conf_template # +# $force_protocol_version - The Kafka protocol version used to produce events. +# Suggested workaround for https://issues.apache.org/jira/browse/KAFKA-3547 +# (Kafka 0.9.0.[0,1] protocol versions affected) +# define varnishkafka::instance( $brokers = ['localhost:9092'], $topic = 'varnish', @@ -111,6 +115,7 @@ $should_subscribe = true, $conf_template = 'varnishkafka/varnishkafka.conf.erb', + $force_protocol_version = undef, ) { require ::varnishkafka diff --git a/templates/varnishkafka_v4.conf.erb b/templates/varnishkafka_v4.conf.erb index e0a9d52..d1ae795 100644 --- a/templates/varnishkafka_v4.conf.erb +++ b/templates/varnishkafka_v4.conf.erb @@ -236,6 +236,18 @@ # Valid values are 'none', 'gzip', and 'snappy'. kafka.compression.codec = <%= @compression_codec %> +<% if @force_protocol_version -%> +# +# Kafka 0.9.0.[0,1] protocol versions may cause +# https://issues.apache.org/jira/browse/KAFKA-3547. This workaround is suggested +# by the author of librdkafka and forces the protocol used to produce events +# to a specific version. The goal is to avoid any issue like the one described +# in https://phabricator.wikimedia.org/T172681 +# More info https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility +kafka.api.version.request=false +kafka.broker.version.fallback=<%= @force_protocol_version %> + +<% end -%> # # Topic configuration # -- To view, visit https://gerrit.wikimedia.org/r/370631 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7a643fe17f9d70718b885e1de9b85494c70cc041 Gerrit-PatchSet: 2 Gerrit-Project: operations/puppet/varnishkafka Gerrit-Branch: master Gerrit-Owner: Elukey <ltosc...@wikimedia.org> Gerrit-Reviewer: Elukey <ltosc...@wikimedia.org> Gerrit-Reviewer: Ottomata <ao...@wikimedia.org> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
