Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

[hans]

This is a recognized area for improvement and better version compatibility is 
something that is being actively worked on. librdkafka clients already allow 
for both forward and backward compatibility. Soon the java clients will be able 
to do so as well.

-hans


> On Dec 24, 2016, at 12:26 AM, Sachin Mittal  wrote:
> 
> What I understand is the Kafka has following apis
> 1. Core broker APIs which runs the server. (Heart of kafka)
> 2. Producer API
> 3. Consumer API
> 4. Connect API
> 5. Streams API
> 
> All four use broker for different purpose.
> There should be some sort of compatibility matrix.
> 
> Kafka server is something central to all applications and one where
> frequent upgrades are not possible in a production environment.
> 
> So every release of other future APIs should have some backward
> compatibility with Kafka server.
> 
> Like x.y.z.w server should atleast be compatible with
> x.y.**.** APIs.
> 
> So any new release x.y.**.** should make sure that they are compatible
> oldest x.y.**.** release atleast.
> 
> And perhaps some sort of compatibility matrix page should be made available.
> 
> Just my 2 cents.
> 
> Thanks
> Sachin
> 
> 
> 
> 
>> On Sat, Dec 24, 2016 at 12:05 AM, Hans Jespersen  wrote:
>> 
>> No. All Java clients (including Streams) need to be the same version (or
>> lower) as the brokers they connect to.
>> 
>> -hans
>> 
>> 
>> 
>> 
>>> On Dec 23, 2016, at 1:03 AM, Sachin Mittal  wrote:
>>> 
>>> Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker.
>>> I was facing broker connect issue and it just did not work.
>>> 
>>> Thanks
>>> Sachin
>>> 
>>> 
 On Fri, Dec 23, 2016 at 4:25 AM,  wrote:
 
 Kafka clients (currently) do not work against older Kafka
>> brokers/servers
 so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka
 broker.
 
 -hans
 
> On Dec 22, 2016, at 2:25 PM, Joanne Contact 
 wrote:
> 
> Hello I have a program which requires 0.10.1.0 streams API. The jar is
> packaged by maven with all dependencies. I tried to consume a Kafka
> topic spit from a Kafka 9 cluster.
> 
> It has such error:
> org.apache.kafka.common.protocol.types.SchemaException: Error reading
> field 'topic_metadata': Error reading array of size 1768180577, only
> 167 bytes available
> 
> I wonder if there is any work around?
> 
> Thanks,
> 
> J
 
>> 
>> 

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

[Sachin Mittal]

What I understand is the Kafka has following apis
1. Core broker APIs which runs the server. (Heart of kafka)
2. Producer API
3. Consumer API
4. Connect API
5. Streams API

All four use broker for different purpose.
There should be some sort of compatibility matrix.

Kafka server is something central to all applications and one where
frequent upgrades are not possible in a production environment.

So every release of other future APIs should have some backward
compatibility with Kafka server.

Like x.y.z.w server should atleast be compatible with
x.y.**.** APIs.

So any new release x.y.**.** should make sure that they are compatible
oldest x.y.**.** release atleast.

And perhaps some sort of compatibility matrix page should be made available.

Just my 2 cents.

Thanks
Sachin




On Sat, Dec 24, 2016 at 12:05 AM, Hans Jespersen  wrote:

> No. All Java clients (including Streams) need to be the same version (or
> lower) as the brokers they connect to.
>
> -hans
>
>
>
>
> > On Dec 23, 2016, at 1:03 AM, Sachin Mittal  wrote:
> >
> > Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker.
> > I was facing broker connect issue and it just did not work.
> >
> > Thanks
> > Sachin
> >
> >
> > On Fri, Dec 23, 2016 at 4:25 AM,  wrote:
> >
> >> Kafka clients (currently) do not work against older Kafka
> brokers/servers
> >> so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka
> >> broker.
> >>
> >> -hans
> >>
> >>> On Dec 22, 2016, at 2:25 PM, Joanne Contact 
> >> wrote:
> >>>
> >>> Hello I have a program which requires 0.10.1.0 streams API. The jar is
> >>> packaged by maven with all dependencies. I tried to consume a Kafka
> >>> topic spit from a Kafka 9 cluster.
> >>>
> >>> It has such error:
> >>> org.apache.kafka.common.protocol.types.SchemaException: Error reading
> >>> field 'topic_metadata': Error reading array of size 1768180577, only
> >>> 167 bytes available
> >>>
> >>> I wonder if there is any work around?
> >>>
> >>> Thanks,
> >>>
> >>> J
> >>
>
>

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

[Hans Jespersen]

No. All Java clients (including Streams) need to be the same version (or lower) 
as the brokers they connect to.

-hans




> On Dec 23, 2016, at 1:03 AM, Sachin Mittal  wrote:
> 
> Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker.
> I was facing broker connect issue and it just did not work.
> 
> Thanks
> Sachin
> 
> 
> On Fri, Dec 23, 2016 at 4:25 AM,  wrote:
> 
>> Kafka clients (currently) do not work against older Kafka brokers/servers
>> so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka
>> broker.
>> 
>> -hans
>> 
>>> On Dec 22, 2016, at 2:25 PM, Joanne Contact 
>> wrote:
>>> 
>>> Hello I have a program which requires 0.10.1.0 streams API. The jar is
>>> packaged by maven with all dependencies. I tried to consume a Kafka
>>> topic spit from a Kafka 9 cluster.
>>> 
>>> It has such error:
>>> org.apache.kafka.common.protocol.types.SchemaException: Error reading
>>> field 'topic_metadata': Error reading array of size 1768180577, only
>>> 167 bytes available
>>> 
>>> I wonder if there is any work around?
>>> 
>>> Thanks,
>>> 
>>> J
>> 

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

[Sachin Mittal]

Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker.
I was facing broker connect issue and it just did not work.

Thanks
Sachin


On Fri, Dec 23, 2016 at 4:25 AM,  wrote:

> Kafka clients (currently) do not work against older Kafka brokers/servers
> so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka
> broker.
>
> -hans
>
> > On Dec 22, 2016, at 2:25 PM, Joanne Contact 
> wrote:
> >
> > Hello I have a program which requires 0.10.1.0 streams API. The jar is
> > packaged by maven with all dependencies. I tried to consume a Kafka
> > topic spit from a Kafka 9 cluster.
> >
> > It has such error:
> > org.apache.kafka.common.protocol.types.SchemaException: Error reading
> > field 'topic_metadata': Error reading array of size 1768180577, only
> > 167 bytes available
> >
> > I wonder if there is any work around?
> >
> > Thanks,
> >
> > J
>

can kafka 10 stream API read the topic from a Kafka 9 cluster?

[Joanne Contact]

Hello I have a program which requires 0.10.1.0 streams API. The jar is
packaged by maven with all dependencies. I tried to consume a Kafka
topic spit from a Kafka 9 cluster.

It has such error:
 org.apache.kafka.common.protocol.types.SchemaException: Error reading
field 'topic_metadata': Error reading array of size 1768180577, only
167 bytes available

I wonder if there is any work around?

Thanks,

J