Re: [ANNOUNCE] New Committer: Randall Hauch

2019-02-14 Thread James Cheng
Congrats, Randall! Well deserved! -James Sent from my iPhone > On Feb 14, 2019, at 6:16 PM, Guozhang Wang wrote: > > Hello all, > > The PMC of Apache Kafka is happy to announce another new committer joining > the project today: we have invited Randall Hauch as a project committer and > he has

Re: [kafka-clients] [VOTE] 2.1.1 RC2

2019-02-14 Thread Jason Gustafson
Ran the quickstart against the 2.11 artifact and checked the release notes. For some reason, KAFKA-7897 is not included in the notes, though I definitely see it in the tagged version. The RC was probably created before the JIRA was resolved. I think we can regenerate without another RC, so +1 from

Re: [ANNOUNCE] New Committer: Randall Hauch

2019-02-14 Thread Vahid Hashemian
Congrats Randall! On Thu, Feb 14, 2019, 19:44 Ismael Juma wrote: > Congratulations Randall! > > On Thu, Feb 14, 2019, 6:16 PM Guozhang Wang > > Hello all, > > > > The PMC of Apache Kafka is happy to announce another new committer > joining > > the project today: we have invited Randall Hauch as

Accessing Kafka stream's KTable underlying RocksDB memory usage

2019-02-14 Thread P. won
Hi, I have a kafka stream app that currently takes 3 topics and aggregates them into a KTable. This app resides inside a microservice which has been allocated 512 MB memory to work with. After implementing this, I've noticed that the docker container running the microservice eventually runs out of

Re: [ANNOUNCE] New Committer: Randall Hauch

2019-02-14 Thread Ismael Juma
Congratulations Randall! On Thu, Feb 14, 2019, 6:16 PM Guozhang Wang Hello all, > > The PMC of Apache Kafka is happy to announce another new committer joining > the project today: we have invited Randall Hauch as a project committer and > he has accepted. > > Randall has been participating in the

Re: [ANNOUNCE] New Committer: Randall Hauch

2019-02-14 Thread hacker win7
Congrats! > On Feb 15, 2019, at 10:16, Guozhang Wang wrote: > > Hello all, > > The PMC of Apache Kafka is happy to announce another new committer joining > the project today: we have invited Randall Hauch as a project committer and > he has accepted. > > Randall has been participating in the K

Re: [ANNOUNCE] New Committer: Randall Hauch

2019-02-14 Thread Bill Bejeck
Congrats Randall! -Bill On Thu, Feb 14, 2019 at 9:16 PM Guozhang Wang wrote: > Hello all, > > The PMC of Apache Kafka is happy to announce another new committer joining > the project today: we have invited Randall Hauch as a project committer and > he has accepted. > > Randall has been particip

[ANNOUNCE] New Committer: Randall Hauch

2019-02-14 Thread Guozhang Wang
Hello all, The PMC of Apache Kafka is happy to announce another new committer joining the project today: we have invited Randall Hauch as a project committer and he has accepted. Randall has been participating in the Kafka community for the past 3 years, and is well known as the founder of the De

Re: [ANNOUNCE] New Committer: Bill B

2019-02-14 Thread Martin Gainty
and a hearty congratulations on your recent win welcome bill! Martin From: Ivan Ponomarev Sent: Thursday, February 14, 2019 5:25 AM To: d...@kafka.apache.org; users@kafka.apache.org Subject: Re: [ANNOUNCE] New Committer: Bill Bejeck Congratulations, Bill! Your

Re: [kafka-clients] [VOTE] 2.1.1 RC2

2019-02-14 Thread Jun Rao
Hi, Colin, Thanks for running the release. Verified the quickstart for 2.12 binary. +1 from me. Jun On Fri, Feb 8, 2019 at 12:02 PM Colin McCabe wrote: > Hi all, > > This is the third candidate for release of Apache Kafka 2.1.1. This > release includes many bug fixes for Apache Kafka 2.1. > >

Re: Kafka CDC from Databases to S3

2019-02-14 Thread Alan Salewski
> On Fri, 15 Feb. 2019, 7:03 am John Ziebro > > We are planning on implementing Kafka to stream changes from a MariaDB as > > well as a MS SQL Server database into an S3 bucket and [...] > > What are your suggestions for handling this scenario? > > > > Thank you, > > John Z On 2019-02-15 07:26:

Re: Kafka CDC from Databases to S3

2019-02-14 Thread Liam Clarke
Kafka Connect JDBC source can pick up changes based on primary keys and mod time columns. Debezium consumes the DBs change log directly. Both would work for your purpose. Have a look at the documentation for the Kafka Connect S3 sink to see if it fits your needs on the other end. Cheers, Liam Cl

Re: Kafka CDC from Databases to S3

2019-02-14 Thread Randall Hauch
Kafka Connect was designed explicitly to support this kind of use case. Debezium's connectors implement the Kafka Connect API, and they just released a MS SQL Server source connector that captures the changes from SQL Server and writes them to Kafka. Debezium's MySQL source connector may work with

Kafka CDC from Databases to S3

2019-02-14 Thread John Ziebro
Hi, We are planning on implementing Kafka to stream changes from a MariaDB as well as a MS SQL Server database into an S3 bucket and would like to stay with opensource solutions as much as possible. To simplify this process it looks like we might need to use something that leverages Kafka like

Kafka Migration from 0.10.2.1 to 2.1.0

2019-02-14 Thread Fernando Palero Molina
Hi all, I'm trying to migrate an old Kafka version (0.10.2.1) to the new Kafka version (2.1.0). The company implemented a class in java for creating Connectors, for that purpose the class ConnectionFactory is used to create connectors (ConnectorFactory.newConnector(*String connectorClassOrAlias* )

Re: [ANNOUNCE] New Committer: Bill Bejeck

2019-02-14 Thread Ivan Ponomarev
Congratulations, Bill! Your 'Kafka Streams in Action' is a great book. These months it is always travelling with me in my backpack with my laptop )) Regards, Ivan 14.02.2019 3:56, Guozhang Wang пишет: Hello all, The PMC of Apache Kafka is happy to announce that we've added Bill Bejeck as o

Re: Kafka SSL and multiple domain names

2019-02-14 Thread Martin Gainty
if you require security over multiple subject alt names then you will need a UCC certificate from cert-provider (godaddy/thawte/verisign are some of the providers) https://www.techopedia.com/definition/29764/unified-communications-certificate-ucc [https://cdn.techopedia.com/resource/images/favico

Kafka SSL and multiple domain names

2019-02-14 Thread Gérald Quintana
Hello, We need to have the same Kafka cluster bound to multiple DNS aliases/domain names. However, for some poor reason, we can't have a single SSL certificate with subject alt names matching all DNS aliases. Is it possible to use different SSL certs depending on the hostname used by the client?

Fwd: Kafka connect tasks rebalancing

2019-02-14 Thread Федор Чернилин
Hello. I'm concerned about the following question. The documentation of Kafka Connect states that "When a worker fails, tasks are rebalanced across the active workers. *When a task fails, no rebalance is triggered as a task failure is considered an exceptional case. As such, failed tasks are not au

Re: Kafka Streams & Distributed state question

2019-02-14 Thread Andrian Jardan
This is exactly what I was looking for, many thanks for the suggestion !! If there are any other solutions, would be happy to hear though. Retrying and querying other nodes sounds like an approach that would work, but a much better option would be to re-use the hashing algo kafka itself uses t