date:20161224

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

2016-12-24 Thread Sachin Mittal

What I understand is the Kafka has following apis
1. Core broker APIs which runs the server. (Heart of kafka)
2. Producer API
3. Consumer API
4. Connect API
5. Streams API

All four use broker for different purpose.
There should be some sort of compatibility matrix.

Kafka server is something central to all applications and one where
frequent upgrades are not possible in a production environment.

So every release of other future APIs should have some backward
compatibility with Kafka server.

Like x.y.z.w server should atleast be compatible with
x.y.**.** APIs.

So any new release x.y.**.** should make sure that they are compatible
oldest x.y.**.** release atleast.

And perhaps some sort of compatibility matrix page should be made available.

Just my 2 cents.

Thanks
Sachin

On Sat, Dec 24, 2016 at 12:05 AM, Hans Jespersen  wrote:

> No. All Java clients (including Streams) need to be the same version (or
> lower) as the brokers they connect to.
>
> -hans
>
>
>
>
> > On Dec 23, 2016, at 1:03 AM, Sachin Mittal  wrote:
> >
> > Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker.
> > I was facing broker connect issue and it just did not work.
> >
> > Thanks
> > Sachin
> >
> >
> > On Fri, Dec 23, 2016 at 4:25 AM,  wrote:
> >
> >> Kafka clients (currently) do not work against older Kafka
> brokers/servers
> >> so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka
> >> broker.
> >>
> >> -hans
> >>
> >>> On Dec 22, 2016, at 2:25 PM, Joanne Contact 
> >> wrote:
> >>>
> >>> Hello I have a program which requires 0.10.1.0 streams API. The jar is
> >>> packaged by maven with all dependencies. I tried to consume a Kafka
> >>> topic spit from a Kafka 9 cluster.
> >>>
> >>> It has such error:
> >>> org.apache.kafka.common.protocol.types.SchemaException: Error reading
> >>> field 'topic_metadata': Error reading array of size 1768180577, only
> >>> 167 bytes available
> >>>
> >>> I wonder if there is any work around?
> >>>
> >>> Thanks,
> >>>
> >>> J
> >>
>
>

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

2016-12-24 Thread hans

This is a recognized area for improvement and better version compatibility is 
something that is being actively worked on. librdkafka clients already allow 
for both forward and backward compatibility. Soon the java clients will be able 
to do so as well.

-hans


> On Dec 24, 2016, at 12:26 AM, Sachin Mittal  wrote:
> 
> What I understand is the Kafka has following apis
> 1. Core broker APIs which runs the server. (Heart of kafka)
> 2. Producer API
> 3. Consumer API
> 4. Connect API
> 5. Streams API
> 
> All four use broker for different purpose.
> There should be some sort of compatibility matrix.
> 
> Kafka server is something central to all applications and one where
> frequent upgrades are not possible in a production environment.
> 
> So every release of other future APIs should have some backward
> compatibility with Kafka server.
> 
> Like x.y.z.w server should atleast be compatible with
> x.y.**.** APIs.
> 
> So any new release x.y.**.** should make sure that they are compatible
> oldest x.y.**.** release atleast.
> 
> And perhaps some sort of compatibility matrix page should be made available.
> 
> Just my 2 cents.
> 
> Thanks
> Sachin
> 
> 
> 
> 
>> On Sat, Dec 24, 2016 at 12:05 AM, Hans Jespersen  wrote:
>> 
>> No. All Java clients (including Streams) need to be the same version (or
>> lower) as the brokers they connect to.
>> 
>> -hans
>> 
>> 
>> 
>> 
>>> On Dec 23, 2016, at 1:03 AM, Sachin Mittal  wrote:
>>> 
>>> Is Kafka streams 0.10.2.0-SNAPSHOT compatible with 0.10.0.1 broker.
>>> I was facing broker connect issue and it just did not work.
>>> 
>>> Thanks
>>> Sachin
>>> 
>>> 
 On Fri, Dec 23, 2016 at 4:25 AM,  wrote:
 
 Kafka clients (currently) do not work against older Kafka
>> brokers/servers
 so you have no other option but to upgrade to a 0.10.1.0 or higher Kafka
 broker.
 
 -hans
 
> On Dec 22, 2016, at 2:25 PM, Joanne Contact 
 wrote:
> 
> Hello I have a program which requires 0.10.1.0 streams API. The jar is
> packaged by maven with all dependencies. I tried to consume a Kafka
> topic spit from a Kafka 9 cluster.
> 
> It has such error:
> org.apache.kafka.common.protocol.types.SchemaException: Error reading
> field 'topic_metadata': Error reading array of size 1768180577, only
> 167 bytes available
> 
> I wonder if there is any work around?
> 
> Thanks,
> 
> J
 
>> 
>>

Re: [ANNOUCE] Apache Kafka 0.10.1.1 Released

2016-12-24 Thread Ismael Juma

Hi Guozhang and Allen,

I filed an INFRA ticket about this:

https://issues.apache.org/jira/browse/INFRA-13172

This has happened to me before and it typically requires human intervention
if the mirroring doesn't happen in a few minutes.

Ismael

On Fri, Dec 23, 2016 at 8:27 PM, Guozhang Wang  wrote:

> Allen,
>
> Please see my previous email. The asf-site repo has been updated, but we
> cannot control when it will be reload and reflected in the web site yet.
> Usually it takes 5 min to a couple hours, but this time it somehow has not
> refreshed yet.
>
>
> Guozhang
>
>
> On Fri, Dec 23, 2016 at 12:07 PM, allen chan  >
> wrote:
>
> > From what i can tell, it looks like the main kafka website is not updated
> > with this release. Download page shows 0.10.1.0 as latest release.
> > The above link for release notes does not work either.
> >
> > Not Found
> >
> > The requested URL /dist/kafka/0.10.1.1/RELEASE_NOTES.html was not found
> on
> > this server.
> >
> > On Wed, Dec 21, 2016 at 7:50 PM, Guozhang Wang 
> wrote:
> >
> > > The Apache Kafka community is pleased to announce the release for
> Apache
> > > Kafka 0.10.1.1. This is a bug fix release that fixes 30 issues in
> > 0.10.1.0.
> > >
> > > All of the changes in this release can be found in the release notes:
> > > *https://archive.apache.org/dist/kafka/0.10.1.1/RELEASE_NOTES.html
> > > 
> > >
> > > Apache Kafka is a distributed streaming platform with four four core
> > APIs:
> > >
> > > ** The Producer API allows an application to publish a stream records
> to
> > > one or more Kafka topics.
> > >
> > > ** The Consumer API allows an application to subscribe to one or more
> > > topics and process the stream of records produced to them.
> > >
> > > ** The Streams API allows an application to act as a stream processor,
> > > consuming an input stream from one or more topics and producing an
> output
> > > stream to one or more output topics, effectively transforming the input
> > > streams to output streams.
> > >
> > > ** The Connector API allows building and running reusable producers or
> > > consumers that connect Kafka topics to existing applications or data
> > > systems. For example, a connector to a relational database might
> capture
> > > every change to a table.three key capabilities:
> > >
> > >
> > > With these APIs, Kafka can be used for two broad classes of
> application:
> > >
> > > ** Building real-time streaming data pipelines that reliably get data
> > > between systems or applications.
> > >
> > > ** Building real-time streaming applications that transform or react to
> > the
> > > streams of data.
> > >
> > >
> > > You can download the source release from
> > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > 1/kafka-0.10.1.1-src.tgz
> > >
> > > and binary releases from
> > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > 1/kafka_2.10-0.10.1.1.tgz
> > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > 1/kafka_2.11-0.10.1.1.tgz
> > >  > > 1/kafka_2.11-0.10.1.1.tgz>
> > >
> > > A big thank you for the following 21 contributors to this release!
> > >
> > > Alexey Ozeritsky, Anton Karamanov, Ben Stopford, Bernard Leach, Bill
> > > Bejeck, Damian Guy, Dan Norwood, Eno Thereska, Ewen Cheslack-Postava,
> > > Guozhang Wang, Jason Gustafson, Jiangjie Qin, Jun He, Jun Rao, Kim
> > > Christensen, Manikumar Reddy O, Matthias J. Sax, Mayuresh Gharat,
> Rajini
> > > Sivaram, Sumant Tambe, Vahid Hashemian
> > >
> > > We welcome your help and feedback. For more information on how to
> > > report problems, and to get involved, visit the project website at
> > > http://kafka.apache.org/
> > >
> > >
> > > Thanks,
> > > -- Guozhang
> > >
> >
> >
> >
> > --
> > Allen Michael Chan
> >
>
>
>
> --
> -- Guozhang
>

Re: [ANNOUCE] Apache Kafka 0.10.1.1 Released

2016-12-24 Thread Abel .

I think there might be a mistake in the name of the release notes file name
as there is a dot at the end of the name. I don't know if this can be
associated to the issue:

http://archive.apache.org/dist/kafka/0.10.1.1/RELEASE_NOTES.html.

Abel.

On Sat, Dec 24, 2016 at 4:29 AM, Ismael Juma  wrote:

> Hi Guozhang and Allen,
>
> I filed an INFRA ticket about this:
>
> https://issues.apache.org/jira/browse/INFRA-13172
>
> This has happened to me before and it typically requires human intervention
> if the mirroring doesn't happen in a few minutes.
>
> Ismael
>
> On Fri, Dec 23, 2016 at 8:27 PM, Guozhang Wang  wrote:
>
> > Allen,
> >
> > Please see my previous email. The asf-site repo has been updated, but we
> > cannot control when it will be reload and reflected in the web site yet.
> > Usually it takes 5 min to a couple hours, but this time it somehow has
> not
> > refreshed yet.
> >
> >
> > Guozhang
> >
> >
> > On Fri, Dec 23, 2016 at 12:07 PM, allen chan <
> allen.michael.c...@gmail.com
> > >
> > wrote:
> >
> > > From what i can tell, it looks like the main kafka website is not
> updated
> > > with this release. Download page shows 0.10.1.0 as latest release.
> > > The above link for release notes does not work either.
> > >
> > > Not Found
> > >
> > > The requested URL /dist/kafka/0.10.1.1/RELEASE_NOTES.html was not
> found
> > on
> > > this server.
> > >
> > > On Wed, Dec 21, 2016 at 7:50 PM, Guozhang Wang 
> > wrote:
> > >
> > > > The Apache Kafka community is pleased to announce the release for
> > Apache
> > > > Kafka 0.10.1.1. This is a bug fix release that fixes 30 issues in
> > > 0.10.1.0.
> > > >
> > > > All of the changes in this release can be found in the release notes:
> > > > *https://archive.apache.org/dist/kafka/0.10.1.1/RELEASE_NOTES.html
> > > > 
> > > >
> > > > Apache Kafka is a distributed streaming platform with four four core
> > > APIs:
> > > >
> > > > ** The Producer API allows an application to publish a stream records
> > to
> > > > one or more Kafka topics.
> > > >
> > > > ** The Consumer API allows an application to subscribe to one or more
> > > > topics and process the stream of records produced to them.
> > > >
> > > > ** The Streams API allows an application to act as a stream
> processor,
> > > > consuming an input stream from one or more topics and producing an
> > output
> > > > stream to one or more output topics, effectively transforming the
> input
> > > > streams to output streams.
> > > >
> > > > ** The Connector API allows building and running reusable producers
> or
> > > > consumers that connect Kafka topics to existing applications or data
> > > > systems. For example, a connector to a relational database might
> > capture
> > > > every change to a table.three key capabilities:
> > > >
> > > >
> > > > With these APIs, Kafka can be used for two broad classes of
> > application:
> > > >
> > > > ** Building real-time streaming data pipelines that reliably get data
> > > > between systems or applications.
> > > >
> > > > ** Building real-time streaming applications that transform or react
> to
> > > the
> > > > streams of data.
> > > >
> > > >
> > > > You can download the source release from
> > > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > > 1/kafka-0.10.1.1-src.tgz
> > > >
> > > > and binary releases from
> > > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > > 1/kafka_2.10-0.10.1.1.tgz
> > > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > > 1/kafka_2.11-0.10.1.1.tgz
> > > >  > > > 1/kafka_2.11-0.10.1.1.tgz>
> > > >
> > > > A big thank you for the following 21 contributors to this release!
> > > >
> > > > Alexey Ozeritsky, Anton Karamanov, Ben Stopford, Bernard Leach, Bill
> > > > Bejeck, Damian Guy, Dan Norwood, Eno Thereska, Ewen Cheslack-Postava,
> > > > Guozhang Wang, Jason Gustafson, Jiangjie Qin, Jun He, Jun Rao, Kim
> > > > Christensen, Manikumar Reddy O, Matthias J. Sax, Mayuresh Gharat,
> > Rajini
> > > > Sivaram, Sumant Tambe, Vahid Hashemian
> > > >
> > > > We welcome your help and feedback. For more information on how to
> > > > report problems, and to get involved, visit the project website at
> > > > http://kafka.apache.org/
> > > >
> > > >
> > > > Thanks,
> > > > -- Guozhang
> > > >
> > >
> > >
> > >
> > > --
> > > Allen Michael Chan
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>

Re: [ANNOUCE] Apache Kafka 0.10.1.1 Released

2016-12-24 Thread Sachin Mittal

The downloads page is still point to 0.10.1.0
http://kafka.apache.org/downloads

Docs is also still pointing 0.10.1.0
http://kafka.apache.org/0101/javadoc/index.html

The documentation page seems to be pointing to 0.10.1
Note sure if it is 0.10.1.0 or 0.10.1.1.
http://kafka.apache.org/documentation

Thanks
Sachin



On Sun, Dec 25, 2016 at 8:06 AM, Abel .  wrote:

> I think there might be a mistake in the name of the release notes file name
> as there is a dot at the end of the name. I don't know if this can be
> associated to the issue:
>
> http://archive.apache.org/dist/kafka/0.10.1.1/RELEASE_NOTES.html.
>
> Abel.
>
> On Sat, Dec 24, 2016 at 4:29 AM, Ismael Juma  wrote:
>
> > Hi Guozhang and Allen,
> >
> > I filed an INFRA ticket about this:
> >
> > https://issues.apache.org/jira/browse/INFRA-13172
> >
> > This has happened to me before and it typically requires human
> intervention
> > if the mirroring doesn't happen in a few minutes.
> >
> > Ismael
> >
> > On Fri, Dec 23, 2016 at 8:27 PM, Guozhang Wang 
> wrote:
> >
> > > Allen,
> > >
> > > Please see my previous email. The asf-site repo has been updated, but
> we
> > > cannot control when it will be reload and reflected in the web site
> yet.
> > > Usually it takes 5 min to a couple hours, but this time it somehow has
> > not
> > > refreshed yet.
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Fri, Dec 23, 2016 at 12:07 PM, allen chan <
> > allen.michael.c...@gmail.com
> > > >
> > > wrote:
> > >
> > > > From what i can tell, it looks like the main kafka website is not
> > updated
> > > > with this release. Download page shows 0.10.1.0 as latest release.
> > > > The above link for release notes does not work either.
> > > >
> > > > Not Found
> > > >
> > > > The requested URL /dist/kafka/0.10.1.1/RELEASE_NOTES.html was not
> > found
> > > on
> > > > this server.
> > > >
> > > > On Wed, Dec 21, 2016 at 7:50 PM, Guozhang Wang 
> > > wrote:
> > > >
> > > > > The Apache Kafka community is pleased to announce the release for
> > > Apache
> > > > > Kafka 0.10.1.1. This is a bug fix release that fixes 30 issues in
> > > > 0.10.1.0.
> > > > >
> > > > > All of the changes in this release can be found in the release
> notes:
> > > > > *https://archive.apache.org/dist/kafka/0.10.1.1/RELEASE_NOTES.html
> > > > >  .>
> > > > >
> > > > > Apache Kafka is a distributed streaming platform with four four
> core
> > > > APIs:
> > > > >
> > > > > ** The Producer API allows an application to publish a stream
> records
> > > to
> > > > > one or more Kafka topics.
> > > > >
> > > > > ** The Consumer API allows an application to subscribe to one or
> more
> > > > > topics and process the stream of records produced to them.
> > > > >
> > > > > ** The Streams API allows an application to act as a stream
> > processor,
> > > > > consuming an input stream from one or more topics and producing an
> > > output
> > > > > stream to one or more output topics, effectively transforming the
> > input
> > > > > streams to output streams.
> > > > >
> > > > > ** The Connector API allows building and running reusable producers
> > or
> > > > > consumers that connect Kafka topics to existing applications or
> data
> > > > > systems. For example, a connector to a relational database might
> > > capture
> > > > > every change to a table.three key capabilities:
> > > > >
> > > > >
> > > > > With these APIs, Kafka can be used for two broad classes of
> > > application:
> > > > >
> > > > > ** Building real-time streaming data pipelines that reliably get
> data
> > > > > between systems or applications.
> > > > >
> > > > > ** Building real-time streaming applications that transform or
> react
> > to
> > > > the
> > > > > streams of data.
> > > > >
> > > > >
> > > > > You can download the source release from
> > > > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > > > 1/kafka-0.10.1.1-src.tgz
> > > > >
> > > > > and binary releases from
> > > > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > > > 1/kafka_2.10-0.10.1.1.tgz
> > > > > https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.
> > > > > 1/kafka_2.11-0.10.1.1.tgz
> > > > >  > > > > 1/kafka_2.11-0.10.1.1.tgz>
> > > > >
> > > > > A big thank you for the following 21 contributors to this release!
> > > > >
> > > > > Alexey Ozeritsky, Anton Karamanov, Ben Stopford, Bernard Leach,
> Bill
> > > > > Bejeck, Damian Guy, Dan Norwood, Eno Thereska, Ewen
> Cheslack-Postava,
> > > > > Guozhang Wang, Jason Gustafson, Jiangjie Qin, Jun He, Jun Rao, Kim
> > > > > Christensen, Manikumar Reddy O, Matthias J. Sax, Mayuresh Gharat,
> > > Rajini
> > > > > Sivaram, Sumant Tambe, Vahid Hashemian
> > > > >
> > > > > We welcome your help and feedback. For more information on how to
> > > > > report problems, and to get involved, visit the project website at
> > > > > http://kafka.apache.org/
> > > > >
>

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

Re: can kafka 10 stream API read the topic from a Kafka 9 cluster?

Re: [ANNOUCE] Apache Kafka 0.10.1.1 Released

Re: [ANNOUCE] Apache Kafka 0.10.1.1 Released

Re: [ANNOUCE] Apache Kafka 0.10.1.1 Released

5 matches

Site Navigation

Mail list logo

Footer information