Re: suggestion for 1.1.0 release notes

[Ismael Juma]

I updated the JIRA to make this clearer.

Ismael

On Fri, Mar 30, 2018 at 3:38 PM, Gibson, Bill (ArcSight) <
bill.gib...@microfocus.com> wrote:

> The release notes include this resolved issue:
>
> [KAFKA-6390] - Update ZooKeeper to 3.4.11, Gradle and other minor updates
>
> I saw 3.4.10 in the downloaded bits, not 3.4.11. It looks like the update
> to ZooKeeper 3.4.11 was done, but then reverted (commit 825bfe5) because of
> ZOOKEEPER-2960. The reversion is not visible in KAFKA-6390, nor elsewhere
> in release notes.
>
> Since the 1.1.0 release is not affected by the ZK 3.4.11 bug, it might be
> a good idea to say so.  I had to search a bit to find this result.
>
> https://issues.apache.org/jira/browse/KAFKA-6390
> https://github.com/apache/kafka/commit/825bfe5adefe4b5f967068b332cb0f
> eedd7dc4fc
> https://issues.apache.org/jira/browse/ZOOKEEPER-2960
>
>

Re: Kafka Mirrormaker issue

[Andrew Otto]

I’m currently stuck on MirrorMaker version 0.9, and I’m not sure when the
new consumer client became the default.  Does your 0.10 version have a
—new.consumer option listed in the help message?  If so, then the new
consumer client is not the default.  I haven’t seen the problem you are
describing (I’m still having plenty of others though) since I’ve switched
to using the new consumer.

Another thought, what is the value of your partition.assignment.strategy?
I’ve found round robin (default in later versions of MirrorMaker) to be a
lot more consistent than whatever the default is in 0.9.  Not sure what the
default in 0.10 is.



On Fri, Mar 30, 2018 at 11:40 AM, Siva A  wrote:

> Any other update on this?
>
> On Mon, Mar 26, 2018, 7:42 PM Andrew Otto  wrote:
>
> > I’ve had similar problems, but I don’t have an explanation for ya :/
> >
> > On Sun, Mar 25, 2018 at 12:19 PM, Siva A 
> wrote:
> >
> > > Hi,
> > >
> > > We have 3 nodes Kafka cluster(0.10.0.1) and its mirroring the data from
> > > another 3 node cluster of same Kafka version.
> > > Both the clusters are Kerberized and we are running the Mirrormaker on
> > the
> > > target cluster using the single principal/keytab with the one way trust
> > on
> > > the KDC.
> > >
> > > At times, the mirrormaker stops functioning(Doesn't mirror the data)
> but
> > > the process is still running. If we restart the service then it works
> > fine
> > > for a day or so.
> > >
> > > I don't see any error on the Kafka logs as well.
> > > Is there anyone seen this kind of issue?
> > >
> > > Thanks
> > > Siva
> > >
> >
>

Re: suggestion for 1.1.0 release notes

[Ted Yu]

Nice reminder.

zookeeper community seems to be close to releasing 3.4.12 :

http://search-hadoop.com/m/ZooKeeper/SvKKH1jwLHJq2tr22?subj=Re+VOTE+Apache+ZooKeeper+release+3+4+12+candidate+1

FYI

On Fri, Mar 30, 2018 at 3:38 PM, Gibson, Bill (ArcSight) <
bill.gib...@microfocus.com> wrote:

> The release notes include this resolved issue:
>
> [KAFKA-6390] - Update ZooKeeper to 3.4.11, Gradle and other minor updates
>
> I saw 3.4.10 in the downloaded bits, not 3.4.11. It looks like the update
> to ZooKeeper 3.4.11 was done, but then reverted (commit 825bfe5) because of
> ZOOKEEPER-2960. The reversion is not visible in KAFKA-6390, nor elsewhere
> in release notes.
>
> Since the 1.1.0 release is not affected by the ZK 3.4.11 bug, it might be
> a good idea to say so.  I had to search a bit to find this result.
>
> https://issues.apache.org/jira/browse/KAFKA-6390
> https://github.com/apache/kafka/commit/825bfe5adefe4b5f967068b332cb0f
> eedd7dc4fc
> https://issues.apache.org/jira/browse/ZOOKEEPER-2960
>
>

suggestion for 1.1.0 release notes

[Gibson, Bill (ArcSight)]

The release notes include this resolved issue:

[KAFKA-6390] - Update ZooKeeper to 3.4.11, Gradle and other minor updates

I saw 3.4.10 in the downloaded bits, not 3.4.11. It looks like the update to 
ZooKeeper 3.4.11 was done, but then reverted (commit 825bfe5) because of 
ZOOKEEPER-2960. The reversion is not visible in KAFKA-6390, nor elsewhere in 
release notes.

Since the 1.1.0 release is not affected by the ZK 3.4.11 bug, it might be a 
good idea to say so.  I had to search a bit to find this result.

https://issues.apache.org/jira/browse/KAFKA-6390
https://github.com/apache/kafka/commit/825bfe5adefe4b5f967068b332cb0feedd7dc4fc
https://issues.apache.org/jira/browse/ZOOKEEPER-2960

Re: Is Restart needed after change in trust store for Kafka 1.1 ?

[Hans Jespersen]

This is the KIP-226 for Dynamic Broker Configuration

https://cwiki.apache.org/confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration
 


and this is the JIRA for what is released in Apache Kafka 1.1.0

https://issues.apache.org/jira/browse/KAFKA-6240 


which seems in include dynamic reconfiguration of SSL keystores

https://issues.apache.org/jira/browse/KAFKA-6241 



-- 
/**
 * Hans Jespersen, Director Systems Engineering, Confluent Inc.
 * h...@confluent.io (650)924-2670
 */









> On Mar 30, 2018, at 2:27 PM, Raghav  wrote:
> 
> Anyone ?
> 
> On Thu, Mar 29, 2018 at 6:11 PM, Raghav  wrote:
> 
>> Hi
>> 
>> We have a 3 node Kafka cluster running. Time to time, we have some changes
>> in trust store and we restart Kafka to take new changes into account. We
>> are on Kafka 0.10.x.
>> 
>> If we move to 1.1, would we still need to restart Kafka upon trust store
>> changes ?
>> 
>> Thanks.
>> 
>> --
>> Raghav
>> 
> 
> 
> 
> -- 
> Raghav

Re: Is Restart needed after change in trust store for Kafka 1.1 ?

[Raghav]

Anyone ?

On Thu, Mar 29, 2018 at 6:11 PM, Raghav  wrote:

> Hi
>
> We have a 3 node Kafka cluster running. Time to time, we have some changes
> in trust store and we restart Kafka to take new changes into account. We
> are on Kafka 0.10.x.
>
> If we move to 1.1, would we still need to restart Kafka upon trust store
> changes ?
>
> Thanks.
>
> --
> Raghav
>



-- 
Raghav

Re: Kafka Mirrormaker issue

[Siva A]

Any other update on this?

On Mon, Mar 26, 2018, 7:42 PM Andrew Otto  wrote:

> I’ve had similar problems, but I don’t have an explanation for ya :/
>
> On Sun, Mar 25, 2018 at 12:19 PM, Siva A  wrote:
>
> > Hi,
> >
> > We have 3 nodes Kafka cluster(0.10.0.1) and its mirroring the data from
> > another 3 node cluster of same Kafka version.
> > Both the clusters are Kerberized and we are running the Mirrormaker on
> the
> > target cluster using the single principal/keytab with the one way trust
> on
> > the KDC.
> >
> > At times, the mirrormaker stops functioning(Doesn't mirror the data) but
> > the process is still running. If we restart the service then it works
> fine
> > for a day or so.
> >
> > I don't see any error on the Kafka logs as well.
> > Is there anyone seen this kind of issue?
> >
> > Thanks
> > Siva
> >
>

delivery semantics?

[Victor L]

Can someone clarify it:
- "At most once": messages may be lost but are never redelivered
Does it mean: "commit, when message is received by leader partition,
retries disabled for producer no matter of ISR status"?
- "At least once": commit only when message is ackd by all partitions in
replication set, if not - keep retrying by producer"?

- "Exactly once - each message is delivered only once"... What does that
mean? Delivered to leader partition and never replicated?

Thank you,
V

kafka - authenticate producers only

[Vasily Ivanov]

Hello,

Is it possible to allow anyone to read from any topic over PLAINTEXT,
but force SASL auth and ACLs for producers? 


I am using kafka 1.0.0