[GitHub] samza pull request #395: SAMZA-1530; Bump up Kafka dependency to 0.11

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/samza/pull/395


---

[GitHub] samza pull request #395: SAMZA-1530; Bump up Kafka dependency to 0.11

[lindong28]

GitHub user lindong28 reopened a pull request:

https://github.com/apache/samza/pull/395

SAMZA-1530; Bump up Kafka dependency to 0.11



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lindong28/samza SAMZA-1530

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/samza/pull/395.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #395


commit ff02efd07fcfbccde224b9be254621c88af26d0d
Author: Dong Lin 
Date:   2018-01-02T18:37:36Z

SAMZA-1530; Bump up Kafka dependency to 0.11

commit 0448e4af331406db8d74bb9a49c24c256858ee0e
Author: Dong Lin 
Date:   2018-01-03T23:13:42Z

Address comments




---

[GitHub] samza pull request #395: SAMZA-1530; Bump up Kafka dependency to 0.11

[lindong28]

Github user lindong28 closed the pull request at:

https://github.com/apache/samza/pull/395


---

[GitHub] samza pull request #395: SAMZA-1530; Bump up Kafka dependency to 0.11

[lindong28]

GitHub user lindong28 opened a pull request:

https://github.com/apache/samza/pull/395

SAMZA-1530; Bump up Kafka dependency to 0.11



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lindong28/samza SAMZA-1530

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/samza/pull/395.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #395


commit f264d1dd45cbf6d5395964e3a42c0b629d4ed09b
Author: Dong Lin 
Date:   2018-01-02T18:37:36Z

SAMZA-1530; Bump up Kafka dependency to 0.11




---

Re: kafka dependency version

[Gaurav Agarwal]

thanks! will do.

On Mon, Aug 22, 2016 at 10:56 PM, Yi Pan  wrote:

> Hi, Gaurav,
>
> There is already an effort going on for this one: SAMZA-855
> . It would be good if you
> can try out the patch.
>
> Thanks!
>
> -Yi
>
> On Mon, Aug 22, 2016 at 1:11 AM, Gaurav Agarwal 
> wrote:
>
> > My initial attempt to build against kafka 0.9.0 or 0.10.0 stopped short
> due
> > to some non-trivial interface changes and refactorings in Kafka; and I
> did
> > not want to make those changes without having a correct understanding.
> >
> > Can someone guide me what would be the correct procedure to upgrade this
> > dependency? Should I attempt to make the build/tests pass against kafka
> > 0.10 and submit a PR? Or would someone from samza dev team prefer to do
> > this?
> >
> > On Mon, Aug 22, 2016 at 12:36 PM, Gaurav Agarwal <
> gauravagarw...@gmail.com
> > >
> > wrote:
> >
> > > Hi All,
> > >
> > > What kafka version is supported by Samza release 0.10.1?
> > >
> > > In the gradle build file (dependency-versions.gradle) I see it as
> > kafkaVersion
> > > = "0.8.2.1". This is very old kafka version and lot of very serious
> bugs
> > > have been fixed since then (including the one we are hitting where
> kafka
> > > client blocks indefeinitelly in face of betwork hiccups).
> > >
> > > Any clues on what version is safe enough to upgrade this dependency to
> > (if
> > > I am planning to self build and host the samza binaries privately)?
> > 0.9.0 /
> > > 0.10.0 ?
> > > I noticed that AdminUtils.CreateTopic() call breaks if I take kafka
> > > 0.10.0. There may be more errors (some may not manifest as directly as
> > this
> > > one).
> > >
> > > Any plans to upgrade this dependency for samza soon?
> > >
> > >  https://issues.apache.org/jira/browse/KAFKA-1733
> > > --
> > > --cheers, gaurav
> > >
> >
>

Re: kafka dependency version

[Yi Pan]

Hi, Gaurav,

There is already an effort going on for this one: SAMZA-855
. It would be good if you
can try out the patch.

Thanks!

-Yi

On Mon, Aug 22, 2016 at 1:11 AM, Gaurav Agarwal 
wrote:

> My initial attempt to build against kafka 0.9.0 or 0.10.0 stopped short due
> to some non-trivial interface changes and refactorings in Kafka; and I did
> not want to make those changes without having a correct understanding.
>
> Can someone guide me what would be the correct procedure to upgrade this
> dependency? Should I attempt to make the build/tests pass against kafka
> 0.10 and submit a PR? Or would someone from samza dev team prefer to do
> this?
>
> On Mon, Aug 22, 2016 at 12:36 PM, Gaurav Agarwal  >
> wrote:
>
> > Hi All,
> >
> > What kafka version is supported by Samza release 0.10.1?
> >
> > In the gradle build file (dependency-versions.gradle) I see it as
> kafkaVersion
> > = "0.8.2.1". This is very old kafka version and lot of very serious bugs
> > have been fixed since then (including the one we are hitting where kafka
> > client blocks indefeinitelly in face of betwork hiccups).
> >
> > Any clues on what version is safe enough to upgrade this dependency to
> (if
> > I am planning to self build and host the samza binaries privately)?
> 0.9.0 /
> > 0.10.0 ?
> > I noticed that AdminUtils.CreateTopic() call breaks if I take kafka
> > 0.10.0. There may be more errors (some may not manifest as directly as
> this
> > one).
> >
> > Any plans to upgrade this dependency for samza soon?
> >
> >  https://issues.apache.org/jira/browse/KAFKA-1733
> > --
> > --cheers, gaurav
> >
>

Re: kafka dependency version

[Gaurav Agarwal]

My initial attempt to build against kafka 0.9.0 or 0.10.0 stopped short due
to some non-trivial interface changes and refactorings in Kafka; and I did
not want to make those changes without having a correct understanding.

Can someone guide me what would be the correct procedure to upgrade this
dependency? Should I attempt to make the build/tests pass against kafka
0.10 and submit a PR? Or would someone from samza dev team prefer to do
this?

On Mon, Aug 22, 2016 at 12:36 PM, Gaurav Agarwal 
wrote:

> Hi All,
>
> What kafka version is supported by Samza release 0.10.1?
>
> In the gradle build file (dependency-versions.gradle) I see it as kafkaVersion
> = "0.8.2.1". This is very old kafka version and lot of very serious bugs
> have been fixed since then (including the one we are hitting where kafka
> client blocks indefeinitelly in face of betwork hiccups).
>
> Any clues on what version is safe enough to upgrade this dependency to (if
> I am planning to self build and host the samza binaries privately)? 0.9.0 /
> 0.10.0 ?
> I noticed that AdminUtils.CreateTopic() call breaks if I take kafka
> 0.10.0. There may be more errors (some may not manifest as directly as this
> one).
>
> Any plans to upgrade this dependency for samza soon?
>
>  https://issues.apache.org/jira/browse/KAFKA-1733
> --
> --cheers, gaurav
>

kafka dependency version

[Gaurav Agarwal]

Hi All,

What kafka version is supported by Samza release 0.10.1?

In the gradle build file (dependency-versions.gradle) I see it as kafkaVersion
= "0.8.2.1". This is very old kafka version and lot of very serious bugs
have been fixed since then (including the one we are hitting where kafka
client blocks indefeinitelly in face of betwork hiccups).

Any clues on what version is safe enough to upgrade this dependency to (if
I am planning to self build and host the samza binaries privately)? 0.9.0 /
0.10.0 ?
I noticed that AdminUtils.CreateTopic() call breaks if I take kafka 0.10.0.
There may be more errors (some may not manifest as directly as this one).

Any plans to upgrade this dependency for samza soon?

 https://issues.apache.org/jira/browse/KAFKA-1733
-- 
--cheers, gaurav

Re: Kafka dependency

[Yi Pan]

Hi, Nick,

Good point. Sorry I forgot to push the latest tag out to the remote repo.
Here is the tag that used to publish all binary tar balls:
release-0.10.0-rc2

Thanks!

-Yi

On Tue, May 10, 2016 at 11:24 AM, Nick Quinn  wrote:

> Thanks Yi!
>
> One more question, if I may:  I saw that you dropped a 0.10.0 in December
> of 2015, but the version in Github is still 0.10.0-rc. Do you guys plan on
> an official release of 0.10.0 or are you settling for a simple rc release?
> The reason I ask is that it is hard to match the git commits to the release
> when there is no official 0.10.0 release.
>
> Thanks!
> Nick
>
>
> -Original Message-
> From: Yi Pan [mailto:nickpa...@gmail.com]
> Sent: Tuesday, May 10, 2016 11:22 AM
> To: dev@samza.apache.org
> Subject: Re: Kafka dependency
>
> Hi, Nick,
>
> We do have plan to update the Kafka dependency in Samza. However, Samza
> only uses Kafka client library. We have confirmed that any Kafka 0.8.2
> clients should be supported by Kafka 0.9 brokers. Hence, it should not
> block you if you are thinking of upgrading Kafka broker versions (e.g.
> LinkedIn has been running with this combination for a long time). The main
> reason that we are taking a bit careful path on the upgrading of Kafka
> client version is that there are Samza users who are still running Kafka
> 0.8.2 broker version and if we force upgrade the client version to 0.9, it
> would likely not be supported by a lower version of Kafka broker.
>
> If you have any specific use case that requires Kafka client version 0.9
> and above, please speak up. We will put into consideration in our upgrade
> plan and timeline.
>
> Thanks a lot!
>
> -Yi
>
> On Tue, May 10, 2016 at 10:28 AM, Nick Quinn 
> wrote:
>
> > Hi guys-
> >
> >
> >
> > I was wondering why Samza still has a dependency on Kafka 0.8.2. Does
> > your development team have any plans to update the Kafka dependency
> > version that Samza is using?
> >
> > Best,
> >
> > Nick
> >
> >
>

Re: Kafka dependency

[Robert Crim]

Just want to chime in to add we'd like to use some of the new security
features in Kafka 0.9, ACLs and TLS in particular.

On Tue, May 10, 2016 at 11:24 AM, Nick Quinn  wrote:

> Thanks Yi!
>
> One more question, if I may:  I saw that you dropped a 0.10.0 in December
> of 2015, but the version in Github is still 0.10.0-rc. Do you guys plan on
> an official release of 0.10.0 or are you settling for a simple rc release?
> The reason I ask is that it is hard to match the git commits to the release
> when there is no official 0.10.0 release.
>
> Thanks!
> Nick
>
>
> -Original Message-
> From: Yi Pan [mailto:nickpa...@gmail.com]
> Sent: Tuesday, May 10, 2016 11:22 AM
> To: dev@samza.apache.org
> Subject: Re: Kafka dependency
>
> Hi, Nick,
>
> We do have plan to update the Kafka dependency in Samza. However, Samza
> only uses Kafka client library. We have confirmed that any Kafka 0.8.2
> clients should be supported by Kafka 0.9 brokers. Hence, it should not
> block you if you are thinking of upgrading Kafka broker versions (e.g.
> LinkedIn has been running with this combination for a long time). The main
> reason that we are taking a bit careful path on the upgrading of Kafka
> client version is that there are Samza users who are still running Kafka
> 0.8.2 broker version and if we force upgrade the client version to 0.9, it
> would likely not be supported by a lower version of Kafka broker.
>
> If you have any specific use case that requires Kafka client version 0.9
> and above, please speak up. We will put into consideration in our upgrade
> plan and timeline.
>
> Thanks a lot!
>
> -Yi
>
> On Tue, May 10, 2016 at 10:28 AM, Nick Quinn 
> wrote:
>
> > Hi guys-
> >
> >
> >
> > I was wondering why Samza still has a dependency on Kafka 0.8.2. Does
> > your development team have any plans to update the Kafka dependency
> > version that Samza is using?
> >
> > Best,
> >
> > Nick
> >
> >
>

RE: Kafka dependency

[Nick Quinn]

Thanks Yi! 

One more question, if I may:  I saw that you dropped a 0.10.0 in December of 
2015, but the version in Github is still 0.10.0-rc. Do you guys plan on an 
official release of 0.10.0 or are you settling for a simple rc release? The 
reason I ask is that it is hard to match the git commits to the release when 
there is no official 0.10.0 release.

Thanks!
Nick


-Original Message-
From: Yi Pan [mailto:nickpa...@gmail.com] 
Sent: Tuesday, May 10, 2016 11:22 AM
To: dev@samza.apache.org
Subject: Re: Kafka dependency

Hi, Nick,

We do have plan to update the Kafka dependency in Samza. However, Samza only 
uses Kafka client library. We have confirmed that any Kafka 0.8.2 clients 
should be supported by Kafka 0.9 brokers. Hence, it should not block you if you 
are thinking of upgrading Kafka broker versions (e.g.
LinkedIn has been running with this combination for a long time). The main 
reason that we are taking a bit careful path on the upgrading of Kafka client 
version is that there are Samza users who are still running Kafka
0.8.2 broker version and if we force upgrade the client version to 0.9, it 
would likely not be supported by a lower version of Kafka broker.

If you have any specific use case that requires Kafka client version 0.9 and 
above, please speak up. We will put into consideration in our upgrade plan and 
timeline.

Thanks a lot!

-Yi

On Tue, May 10, 2016 at 10:28 AM, Nick Quinn  wrote:

> Hi guys-
>
>
>
> I was wondering why Samza still has a dependency on Kafka 0.8.2. Does 
> your development team have any plans to update the Kafka dependency 
> version that Samza is using?
>
> Best,
>
> Nick
>
>

Re: Kafka dependency

[Yi Pan]

Hi, Nick,

We do have plan to update the Kafka dependency in Samza. However, Samza
only uses Kafka client library. We have confirmed that any Kafka 0.8.2
clients should be supported by Kafka 0.9 brokers. Hence, it should not
block you if you are thinking of upgrading Kafka broker versions (e.g.
LinkedIn has been running with this combination for a long time). The main
reason that we are taking a bit careful path on the upgrading of Kafka
client version is that there are Samza users who are still running Kafka
0.8.2 broker version and if we force upgrade the client version to 0.9, it
would likely not be supported by a lower version of Kafka broker.

If you have any specific use case that requires Kafka client version 0.9
and above, please speak up. We will put into consideration in our upgrade
plan and timeline.

Thanks a lot!

-Yi

On Tue, May 10, 2016 at 10:28 AM, Nick Quinn  wrote:

> Hi guys-
>
>
>
> I was wondering why Samza still has a dependency on Kafka 0.8.2. Does your
> development team have any plans to update the Kafka dependency version that
> Samza is using?
>
> Best,
>
> Nick
>
>

Kafka dependency

[Nick Quinn]

Hi guys-



I was wondering why Samza still has a dependency on Kafka 0.8.2. Does your 
development team have any plans to update the Kafka dependency version that 
Samza is using?

Best,

Nick