This has been covered in passing in the preceding threads but I'd like to make
it explicit, can we have a command line option/script for getting the Kafka
version (probably with Scala version too)?
I ran into this recently, where we wanted to verify that the right version of
Kafka had been installed on a number of machines. I ended up getting the
version number from the bundled jars but it wasn't ideal. Being able to run
kafka --version would have been great.
---
Daniel
> On 15/11/2014, at 8:33 am, Magnus Edenhill <mag...@edenhill.se> wrote:
>
> Hi Gwen,
>
> the protocol version in an InfoResponse message would be the maximum
> supported mainline protocol version, each change to the
> effective protocol specification bumps the version by one.
>
>
> The current per-request-type version ID is unfortunately useless to the
> client, it only allows the broker to act differently
> on different versions of a known request type.
>
> Since the broker currently does not handle unknown request types gracefully
> - it simply disconnects the client, which from the client's perspective
> could mean anything and nothing - the addition of an InfoRequest/Respones
> would allow the client to know if a desired broker-feature is available or
> not and thus allows the client library to provide the application (and in
> the end the user) with usable error message (compare "Broker disconnected"
> to "offsetCommit not supported by 0.8.0 broker").
> Having the broker return a fabricated response with the header.err field
> set to UNKNOWN_REQUEST_TYPE would be very useful as well of course.
>
>
> Regards,
> Magnus
>
>
>
> 2014-11-14 18:17 GMT+01:00 Gwen Shapira <gshap...@cloudera.com>:
>
>> I'm not sure there's a single "protocol version". Each
>> request/response has its own version ID embedded in the request
>> itself.
>>
>> Broker version, broker ID and (as Joel suggested) git hash are all
>> reasonable. And +1 for adding this to API to support non-JVM clients.
>>
>> Gwen
>>
>> On Fri, Nov 14, 2014 at 8:46 AM, Magnus Edenhill <mag...@edenhill.se>
>> wrote:
>>> Please let's not add dependencies on more third party protocols/software,
>>> the move away from Zookeeper dependence on the clients is very much
>> welcomed
>>> and it'd be a pity to see a new foreign dependency added for such a
>> trivial
>>> thing
>>> as propagating the version of a broker to its client.
>>>
>>> My suggestion is to add a new protocol request which returns:
>>> - broker version
>>> - protocol version
>>> - broker id
>>>
>>> A generic future-proof solution would be the use of tags (or named TLVs):
>>> InfoResponse: [InfoTag]
>>> InfoTag:
>>> intX tag ( KAFKA_TAG_BROKER_VERSION, KAFKA_TAG_PROTO_VERSION,
>>> KAFKA_TAG_SSL, ... )
>>> intX type ( KAFKA_TYPE_STR, KAFKA_TYPE_INT32, KAFKA_TYPE_INT64,
>> ...)
>>> intX len
>>> bytes payload
>>>
>>> Local site/vendor tags could be defined in the configuration file,
>>>
>>>
>>> This would also allow clients to enable/disable features based on
>> protocol
>>> version,
>>> e.g., there is no point in trying offsetCommit on a 0.8.0 broker and the
>>> client library
>>> can inform the application about this early, rather than having its
>>> offsetCommit requests
>>> fail by connection teardown (which is not much of an error propagation).
>>>
>>>
>>> My two cents,
>>> Magnus
>>>
>>>
>>> 2014-11-12 20:11 GMT+01:00 Mark Roberts <wiz...@gmail.com>:
>>>
>>>> I haven't worked much with JMX before, but some quick googling (10-20
>>>> minutes) is very inconclusive as to how I would go about getting the
>> server
>>>> version I'm connecting to from a Python client. Can someone please
>>>> reassure me that it's relatively trivial for non Java clients to query
>> JMX
>>>> for the server version of every server in the cluster? Is there a reason
>>>> not to include this in the API itself?
>>>>
>>>> -Mark
>>>>
>>>> On Wed, Nov 12, 2014 at 9:50 AM, Joel Koshy <jjkosh...@gmail.com>
>> wrote:
>>>>
>>>>> +1 on the JMX + gradle properties. Is there any (seamless) way of
>>>>> including the exact git hash? That would be extremely useful if users
>>>>> need help debugging and happen to be on an unreleased build (say, off
>>>>> trunk)
>>>>>
>>>>>> On Wed, Nov 12, 2014 at 09:34:35AM -0800, Gwen Shapira wrote:
>>>>>> Actually, Jun suggested exposing this via JMX.
>>>>>>
>>>>>> On Wed, Nov 12, 2014 at 9:31 AM, Gwen Shapira <
>> gshap...@cloudera.com>
>>>>> wrote:
>>>>>>
>>>>>>> Good question.
>>>>>>>
>>>>>>> The server will need to expose this in the protocol, so Kafka
>> clients
>>>>> will
>>>>>>> know what they are talking to.
>>>>>>>
>>>>>>> We may also want to expose this in the producer and consumer, so
>>>> people
>>>>>>> who use Kafka's built-in clients will know which version they
>> have in
>>>>> the
>>>>>>> environment.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Nov 12, 2014 at 9:09 AM, Mark Roberts <wiz...@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> Just to be clear: this is going to be exposed via some Api the
>>>> clients
>>>>>>>> can call at startup?
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 12, 2014, at 08:59, Guozhang Wang <wangg...@gmail.com>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Sounds great, +1 on this.
>>>>>>>>>
>>>>>>>>>> On Tue, Nov 11, 2014 at 1:36 PM, Gwen Shapira <
>>>>> gshap...@cloudera.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> So it looks like we can use Gradle to add properties to
>> manifest
>>>>> file
>>>>>>>> and
>>>>>>>>>> then use getResourceAsStream to read the file and parse it.
>>>>>>>>>>
>>>>>>>>>> The Gradle part would be something like:
>>>>>>>>>> jar.manifest {
>>>>>>>>>> attributes('Implementation-Title': project.name,
>>>>>>>>>> 'Implementation-Version': project.version,
>>>>>>>>>> 'Built-By': System.getProperty('user.name'),
>>>>>>>>>> 'Built-JDK': System.getProperty('java.version'),
>>>>>>>>>> 'Built-Host': getHostname(),
>>>>>>>>>> 'Source-Compatibility':
>> project.sourceCompatibility,
>>>>>>>>>> 'Target-Compatibility': project.targetCompatibility
>>>>>>>>>> )
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> The code part would be:
>> this.getClass().getClassLoader().getResourceAsStream("/META-INF/MANIFEST.MF")
>>>>>>>>>>
>>>>>>>>>> Does that look like the right approach?
>>>>>>>>>>
>>>>>>>>>> Gwen
>>>>>>>>>>
>>>>>>>>>> On Tue, Nov 11, 2014 at 10:43 AM, Bhavesh Mistry <
>>>>>>>>>> mistry.p.bhav...@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> If is maven artifact then you will get following pre-build
>>>>> property
>>>>>>>> file
>>>>>>>>>>> from maven build called pom.properties under
>>>>>>>>>>> /META-INF/maven/groupid/artifactId/pom.properties folder.
>>>>>>>>>>>
>>>>>>>>>>> Here is sample:
>>>>>>>>>>> #Generated by Maven
>>>>>>>>>>> #Mon Oct 10 10:44:31 EDT 2011
>>>>>>>>>>> version=10.0.1
>>>>>>>>>>> groupId=com.google.guava
>>>>>>>>>>> artifactId=guava
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Bhavesh
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Nov 11, 2014 at 10:34 AM, Gwen Shapira <
>>>>> gshap...@cloudera.com
>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> In Sqoop we do the following:
>>>>>>>>>>>>
>>>>>>>>>>>> Maven runs a shell script, passing the version as a
>> parameter.
>>>>>>>>>>>> The shell-script generates a small java class, which is then
>>>>> built
>>>>>>>>>> with a
>>>>>>>>>>>> Maven plugin.
>>>>>>>>>>>> Our code references this generated class when we expose
>>>>>>>> "getVersion()".
>>>>>>>>>>>>
>>>>>>>>>>>> Its complex and ugly, so I'm kind of hoping that there's a
>>>>> better way
>>>>>>>>>> to
>>>>>>>>>>> do
>>>>>>>>>>>> it :)
>>>>>>>>>>>>
>>>>>>>>>>>> Gwen
>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 9:42 AM, Jun Rao <jun...@gmail.com
>>>
>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Currently, the version number is only stored in our build
>>>> config
>>>>>>>>>> file,
>>>>>>>>>>>>> gradle.properties. Not sure how we can automatically
>> extract
>>>> it
>>>>> and
>>>>>>>>>>>> expose
>>>>>>>>>>>>> it in an mbean. How do other projects do this?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 7:05 AM, Otis Gospodnetic <
>>>>>>>>>>>>> otis.gospodne...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Jun,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sounds good. But is the version number stored anywhere
>> from
>>>>> where
>>>>>>>>>> it
>>>>>>>>>>>>> could
>>>>>>>>>>>>>> be gotten?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Otis
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized
>> Log
>>>>>>>>>>> Management
>>>>>>>>>>>>>> Solr & Elasticsearch Support * http://sematext.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Nov 11, 2014 at 12:45 AM, Jun Rao <
>> jun...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Otis,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We don't have an api for that now. We can probably expose
>>>> this
>>>>>>>>>> as a
>>>>>>>>>>>> JMX
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> part of kafka-1481.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Nov 10, 2014 at 7:17 PM, Otis Gospodnetic <
>>>>>>>>>>>>>>> otis.gospodne...@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Is there a way to detect which version of Kafka one is
>>>>> running?
>>>>>>>>>>>>>>>> Is there an API for that, or a constant with this
>> value, or
>>>>>>>>>> maybe
>>>>>>>>>>>> an
>>>>>>>>>>>>>>> MBean
>>>>>>>>>>>>>>>> or some other way to get to this info?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Otis
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized
>> Log
>>>>>>>>>>>>> Management
>>>>>>>>>>>>>>>> Solr & Elasticsearch Support * http://sematext.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> -- Guozhang
>>
