Re: Which is True? Kafka site vs Confluent 3.2 site upgrade doc details contradiction regarding 0.10.2 clients backward compatible to resp. 0.10.0 vs 0.10.1?

2017-04-01 Thread Matthias J. Sax 
The reason Streams API 0.10.2 is not backward compatible to 0.10.0
broker is not related to Producer/Consumer API. Streams API (as of
0.10.2) uses a new AdminClient (cf. KIP-4) for topic management that is
not supported by 0.10.0 brokers. 0.10.0 broker topics are managed via ZK
client only -- and ZK client got remove form Streams API in 0.10.2.

-Matthias

On 4/1/17 12:27 AM, Hans Jespersen wrote:
> They are both true. The Apache text is talking about the compatibility of the 
> Producer/Consumer API and the Confluent text is talking about the Streams API.
> 
> -hans
> 
>> On Mar 31, 2017, at 11:46 PM, Roger Vandusen 
>>  wrote:
>>
>> Read below and answer: So which is the source of truth ?
>> Is 0.10.2.0 compatible to 0.10.0 or 0.10.1?
>>
>> Which site needs correction?
>>
>>
>> From current Kafka docs.
>> Statement from Kafka site:
>> https://kafka.apache.org/documentation/#upgrade
>>
>> Starting with version 0.10.2, Java clients (producer and consumer) have 
>> acquired the ability to communicate with older brokers.
>> Version 0.10.2 clients can talk to version 0.10.0 or newer brokers.
>> However, if your brokers are older than 0.10.0, you must upgrade all the 
>> brokers in the Kafka cluster before upgrading your clients.
>> Version 0.10.2 brokers support 0.8.x and newer clients. Before 0.10.2, Kafka 
>> is backward compatible,
>> which means that clients from Kafka 0.8.x releases (CP 1.0.x) will work with 
>> brokers from Kafka 0.9.x, 0.10.0, 0.10.1 and 0.10.2
>> releases (CP 2.0.x, 3.0.x, 3.1.x and 3.2.x), but not vice-versa.
>> This means you always need to plan upgrades such that all brokers are 
>> upgraded before clients.
>>
>>
>> Hmm...do some more reading and research on Confluent site found this which 
>> seems to contradict the above statement:
>> http://docs.confluent.io/3.2.0/streams/upgrade-guide.html
>>
>>
>> Upgrading from CP 3.1.x (Kafka 0.10.1.x-cp2) to CP 3.2.0 (Kafka 0.10.2.0-cp1)
>> Compatibility
>> Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.0-cp1) are 
>> forward and backward compatible with certain Kafka clusters.
>>
>> Compatibility Matrix:
>>
>>
>> Kafka Broker (columns)
>>
>> Streams API (rows)
>>
>> 3.0.x / 0.10.0.x
>>
>> 3.1.x / 0.10.1.x
>>
>> 3.2.0 / 0.10.2.0
>>
>> 3.0.x / 0.10.0.x
>>
>> compatible
>>
>> compatible
>>
>> compatible
>>
>> 3.1.x / 0.10.1.x
>>
>>
>>
>> compatible
>>
>> compatible
>>
>> 3.2.0 / 0.10.2.0
>>
>>
>>
>> compatible
>>
>> compatible
>>
>>
>> EMPHASIS ON CONTRADICTION BELOW from site desciption:
>>
>> Backward-compatible to CP 3.1.x clusters (Kafka 0.10.1.x-cp2):
>> This is the first release allowing to upgrade your Kafka Streams application 
>> without a broker upgrade.
>> New Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.x-cp1) will 
>> work with older Kafka clusters running CP 3.1.x (Kafka 0.10.1.x-cp2).
>> Kafka clusters running CP 3.0.x (Kafka 0.10.0.x-cp1) are not compatible with 
>> new CP 3.2.0 Kafka Streams applications though.
>>
>> -Roger
>>



signature.asc
Description: OpenPGP digital signature

Re: Which is True? Kafka site vs Confluent 3.2 site upgrade doc details contradiction regarding 0.10.2 clients backward compatible to resp. 0.10.0 vs 0.10.1?

2017-04-01 Thread Hans Jespersen 
They are both true. The Apache text is talking about the compatibility of the 
Producer/Consumer API and the Confluent text is talking about the Streams API.

-hans

> On Mar 31, 2017, at 11:46 PM, Roger Vandusen 
>  wrote:
> 
> Read below and answer: So which is the source of truth ?
> Is 0.10.2.0 compatible to 0.10.0 or 0.10.1?
> 
> Which site needs correction?
> 
> 
> From current Kafka docs.
> Statement from Kafka site:
> https://kafka.apache.org/documentation/#upgrade
> 
> Starting with version 0.10.2, Java clients (producer and consumer) have 
> acquired the ability to communicate with older brokers.
> Version 0.10.2 clients can talk to version 0.10.0 or newer brokers.
> However, if your brokers are older than 0.10.0, you must upgrade all the 
> brokers in the Kafka cluster before upgrading your clients.
> Version 0.10.2 brokers support 0.8.x and newer clients. Before 0.10.2, Kafka 
> is backward compatible,
> which means that clients from Kafka 0.8.x releases (CP 1.0.x) will work with 
> brokers from Kafka 0.9.x, 0.10.0, 0.10.1 and 0.10.2
> releases (CP 2.0.x, 3.0.x, 3.1.x and 3.2.x), but not vice-versa.
> This means you always need to plan upgrades such that all brokers are 
> upgraded before clients.
> 
> 
> Hmm...do some more reading and research on Confluent site found this which 
> seems to contradict the above statement:
> http://docs.confluent.io/3.2.0/streams/upgrade-guide.html
> 
> 
> Upgrading from CP 3.1.x (Kafka 0.10.1.x-cp2) to CP 3.2.0 (Kafka 0.10.2.0-cp1)
> Compatibility
> Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.0-cp1) are 
> forward and backward compatible with certain Kafka clusters.
> 
> Compatibility Matrix:
> 
> 
> Kafka Broker (columns)
> 
> Streams API (rows)
> 
> 3.0.x / 0.10.0.x
> 
> 3.1.x / 0.10.1.x
> 
> 3.2.0 / 0.10.2.0
> 
> 3.0.x / 0.10.0.x
> 
> compatible
> 
> compatible
> 
> compatible
> 
> 3.1.x / 0.10.1.x
> 
> 
> 
> compatible
> 
> compatible
> 
> 3.2.0 / 0.10.2.0
> 
> 
> 
> compatible
> 
> compatible
> 
> 
> EMPHASIS ON CONTRADICTION BELOW from site desciption:
> 
> Backward-compatible to CP 3.1.x clusters (Kafka 0.10.1.x-cp2):
> This is the first release allowing to upgrade your Kafka Streams application 
> without a broker upgrade.
> New Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.x-cp1) will 
> work with older Kafka clusters running CP 3.1.x (Kafka 0.10.1.x-cp2).
> Kafka clusters running CP 3.0.x (Kafka 0.10.0.x-cp1) are not compatible with 
> new CP 3.2.0 Kafka Streams applications though.
> 
> -Roger
>

Which is True? Kafka site vs Confluent 3.2 site upgrade doc details contradiction regarding 0.10.2 clients backward compatible to resp. 0.10.0 vs 0.10.1?

2017-04-01 Thread Roger Vandusen 
Read below and answer: So which is the source of truth ?
Is 0.10.2.0 compatible to 0.10.0 or 0.10.1?

Which site needs correction?


From current Kafka docs.
Statement from Kafka site:
https://kafka.apache.org/documentation/#upgrade

Starting with version 0.10.2, Java clients (producer and consumer) have 
acquired the ability to communicate with older brokers.
Version 0.10.2 clients can talk to version 0.10.0 or newer brokers.
However, if your brokers are older than 0.10.0, you must upgrade all the 
brokers in the Kafka cluster before upgrading your clients.
Version 0.10.2 brokers support 0.8.x and newer clients. Before 0.10.2, Kafka is 
backward compatible,
which means that clients from Kafka 0.8.x releases (CP 1.0.x) will work with 
brokers from Kafka 0.9.x, 0.10.0, 0.10.1 and 0.10.2
releases (CP 2.0.x, 3.0.x, 3.1.x and 3.2.x), but not vice-versa.
This means you always need to plan upgrades such that all brokers are upgraded 
before clients.


Hmm...do some more reading and research on Confluent site found this which 
seems to contradict the above statement:
http://docs.confluent.io/3.2.0/streams/upgrade-guide.html


Upgrading from CP 3.1.x (Kafka 0.10.1.x-cp2) to CP 3.2.0 (Kafka 0.10.2.0-cp1)
Compatibility
Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.0-cp1) are forward 
and backward compatible with certain Kafka clusters.

Compatibility Matrix:


Kafka Broker (columns)

Streams API (rows)

3.0.x / 0.10.0.x

3.1.x / 0.10.1.x

3.2.0 / 0.10.2.0

3.0.x / 0.10.0.x

compatible

compatible

compatible

3.1.x / 0.10.1.x



compatible

compatible

3.2.0 / 0.10.2.0



compatible

compatible


EMPHASIS ON CONTRADICTION BELOW from site desciption:

Backward-compatible to CP 3.1.x clusters (Kafka 0.10.1.x-cp2):
This is the first release allowing to upgrade your Kafka Streams application 
without a broker upgrade.
New Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.x-cp1) will 
work with older Kafka clusters running CP 3.1.x (Kafka 0.10.1.x-cp2).
Kafka clusters running CP 3.0.x (Kafka 0.10.0.x-cp1) are not compatible with 
new CP 3.2.0 Kafka Streams applications though.

-Roger