[jira] [Created] (KAFKA-7411) Change system to future and change back will make replication not working

Pengwei created KAFKA-7411: -- Summary: Change system to future and change back will make replication not working Key: KAFKA-7411 URL: https://issues.apache.org/jira/browse/KAFKA-7411 Project: Kafka

Re: [VOTE] KIP-366 - Make FunctionConversations private

Matt, This is now all updated. On Fri, 7 Sep 2018 at 03:34 Matthias J. Sax wrote: > Can you please update the KIP accordingly? > > It still says "make private" instead of "deprecating" > > -Matthias > > On 9/6/18 10:07 AM, Attila Sasvári wrote: > > +1 (non-binding) > > > > On Thu, Sep 6, 2018 at

Re: [VOTE] KIP-366 - Make FunctionConversations private

Ok so we have only one binding vote up to now. I guess we need at least 3 binding votes. Thanks On Fri, 14 Sep 2018 at 09:59 Joan Goyeau wrote: > Matt, This is now all updated. > > On Fri, 7 Sep 2018 at 03:34 Matthias J. Sax wrote: > >> Can you please update the KIP accordingly? >> >> It still

[jira] [Created] (KAFKA-7412) Bug prone response from producer.send(ProducerRecord, Callback) if Kafka broker is not running

Michal Turek created KAFKA-7412: --- Summary: Bug prone response from producer.send(ProducerRecord, Callback) if Kafka broker is not running Key: KAFKA-7412 URL: https://issues.apache.org/jira/browse/KAFKA-7412

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Ron, 1) Yes, we should update the version of `SaslHandshakeRequest`. Clients would attempt to re-authenticate only if broker's SaslHandshakeRequest version >=2. 2) I think we should enable broker's re-authentication and connection termination code regardless of client version. Re-authentication

[jira] [Created] (KAFKA-7413) Replace slave terminology with follower in website

Sayat Satybaldiyev created KAFKA-7413: - Summary: Replace slave terminology with follower in website Key: KAFKA-7413 URL: https://issues.apache.org/jira/browse/KAFKA-7413 Project: Kafka Is

Re: [DISCUSS] KIP-371: Add a configuration to build custom SSL principal name

Hi All, We'd appreciate any thoughts / comments on the proposed options for customizing SSL principal names. We are happy to discuss any alternative approaches/suggestions. Thanks, On Sat, Sep 8, 2018 at 12:45 AM Manikumar wrote: > Hi all, > > I have created a KIP that proposes couple of optio

Re: Review and merge - KAFKA-6764

Can you pls check this? On Wed, Sep 12, 2018 at 3:12 PM Suman B N wrote: > Team, > > Review and merge below pull request. > Merge Request: https://github.com/apache/kafka/pull/5637 > Jira: https://issues.apache.org/jira/browse/KAFKA-6764 > > -- > *Suman* > *OlaCabs* > -- *Suman* *OlaCabs*

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Rajini. Agreed, we will bump the SaslHandshakeRequest API number (no wire format change, of course). Agreed, responding to re-authentication will always be enabled on the broker since it is initiated by the client. Agreed, connection termination by the broker will be opt-in. I'm thinking *sa

Re: [DISCUSS] KIP-371: Add a configuration to build custom SSL principal name

Manikumar, thanks. If I understand the KIP motivation right, this is currently already possible, but you want to have an easier way of doing it, right? The motivation would be stronger if we had 2-3 common cases (from experience) where the suggested pattern would solve them, and perhaps 1-2 cases w

Re: [VOTE] KIP-366 - Make FunctionConversations private

+1 (binding) Thanks On Fri, 14 Sep 2018 at 10:00 Joan Goyeau wrote: > Ok so we have only one binding vote up to now. I guess we need at least 3 > binding votes. > > Thanks > > On Fri, 14 Sep 2018 at 09:59 Joan Goyeau wrote: > > > Matt, This is now all updated. > > > > On Fri, 7 Sep 2018 at 03:

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

It will be good to shorten the config name. We have a config named ` connections.max.idle.ms`. We could add something like ` connections.max.expired.credentials.ms`. As you said, it would be prefixed with listener and SASL mechanism name. We should be able to support connection termination in futur

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

On 2018/05/18 13:00:52, Molnár Bálint wrote: > Hi Jakub, > > Yes I tried it and as I saw in various forums it works well for Kafka as > well, but because it mimics the Zookeeper API it creates quite many > metadata related value. > Kafka uses only a subset from the Zookeeper API, so those val

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

On 2018/05/18 13:00:52, Molnár Bálint wrote: > Hi Jakub, > > Yes I tried it and as I saw in various forums it works well for Kafka as > well, but because it mimics the Zookeeper API it creates quite many > metadata related value. > Kafka uses only a subset from the Zookeeper API, so those val

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

On 2018/05/09 14:29:17, Molnár Bálint wrote: > Hi, > I just rebased the Etcd implementation proposal on trunk. Pinging to see if > anyone has feedback on my questions from my previous email. > > Molnár Bálint ezt írta (időpont: 2018. ápr. 4., > Sze, 10:08): > > > Hi, > > Thanks again for th

[jira] [Created] (KAFKA-7414) Do not fail broker on out of range offsets in replica fetcher

Jason Gustafson created KAFKA-7414: -- Summary: Do not fail broker on out of range offsets in replica fetcher Key: KAFKA-7414 URL: https://issues.apache.org/jira/browse/KAFKA-7414 Project: Kafka

Re: [DISCUSS] KIP-372: Naming Joins and Grouping

Hello John, Not sure if I completely understand your email above. Are you suggesting to still use the proposed Joined / Grouped object to indicate the underlying processor names *in addition to* the repartition topic names? My reasoning is that, if we do not want to use the proposed names to indi

Re: [DISCUSS] KIP-371: Add a configuration to build custom SSL principal name

Hi Eno, Thanks for the review. Most often users want to extract one of the field (eg. CN). CN is the commonly used field. For this simple change, users need to build and maintain the custom principal builder class and also package and deploy to the all brokers. Having configurable rules will be u

Re: [DISCUSS] KIP-371: Add a configuration to build custom SSL principal name

Definitely a helpful change. +1 for Option 2. On 9/14/18, 10:52 AM, "Manikumar" wrote: Hi Eno, Thanks for the review. Most often users want to extract one of the field (eg. CN). CN is the commonly used field. For this simple change, users need to build and maintain

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Hello Adam, Thanks for the explanation. Regarding the final step (i.e. the high watermark store, now altered to be replaced with a window store), I think another current on-going KIP may actually help: https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+i

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

Correction on my previous email: KAFKA-5533 is the wrong link, as it is for corresponding changelog mechanisms. But as part of KIP-258 we do want to have "handling out-of-order data for source KTable" such that instead of blindly apply the updates to the materialized store, i.e. following offset or

[jira] [Created] (KAFKA-7415) OffsetsForLeaderEpoch may incorrectly respond with undefined epoch causing truncation to HW

Anna Povzner created KAFKA-7415: --- Summary: OffsetsForLeaderEpoch may incorrectly respond with undefined epoch causing truncation to HW Key: KAFKA-7415 URL: https://issues.apache.org/jira/browse/KAFKA-7415

Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task

Hi, we recently had a discussion on a different ticket to reduce the size of the metadata we need to send: https://issues.apache.org/jira/browse/KAFKA-7149 It seems, that we actually don't need to include the number of stores in the metadata, but that we can compute the number of stores locally o

[jira] [Resolved] (KAFKA-7280) ConcurrentModificationException in FetchSessionHandler in heartbeat thread

[ https://issues.apache.org/jira/browse/KAFKA-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-7280. Resolution: Fixed Fix Version/s: (was: 1.1.2) > ConcurrentModificationException

Re: [VOTE] KIP-110: Add Codec for ZStandard Compression

+1(non-binding) On 9/13/18, 1:44 AM, "Mickael Maison" wrote: +1 (non binding) Thanks for your perseverance, it's great to see this KIP finally reaching completion! On Thu, Sep 13, 2018 at 4:58 AM Harsha wrote: > > +1 (binding). > > Thanks, > Harsha >

Build failed in Jenkins: kafka-0.11.0-jdk7 #399

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H27 (ubuntu xenial) in workspace

Build failed in Jenkins: kafka-2.0-jdk8 #142

See -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H27 (ubuntu xenial) in workspace Al

Access for a KIP creation

Hi, I would like to request an access right for a KIP creation. Reference: https://github.com/apache/kafka/pull/5344#issuecomment-419942283 My cwiki ID is "wlsc" (on https://cwiki.apache.org/confluence/). Thanks, Wladimir Schmidt

Re: [VOTE] KIP-366 - Make FunctionConversations private

+1 (binding) -Matthias On 9/14/18 7:55 AM, Damian Guy wrote: > +1 (binding) > > Thanks > > On Fri, 14 Sep 2018 at 10:00 Joan Goyeau wrote: > >> Ok so we have only one binding vote up to now. I guess we need at least 3 >> binding votes. >> >> Thanks >> >> On Fri, 14 Sep 2018 at 09:59 Joan Goye

Re: [DISCUSS] KIP-372: Naming Joins and Grouping

All, Thanks for the comments, I'll respond to all of your comments in the recieved order. Guozhang, > And if [join-name] not specified, stay the same, which is: > * [previous-processor-name]-repartition for both Stream-Stream (S-S) join and S-T join I believe the current approach is [appId]

Re: [DISCUSS] KIP-273 Kafka to support using ETCD beside Zookeeper

On Fri, Sep 14, 2018, at 00:46, lbari...@gmail.com wrote: > Hello, Colin! > > I have read the discussion on KIP-273 and want to raise this question. > Deploying Kafka to Kubernetes is a crucial problem in my current work. > And usage of ZooKeeper makes troubles, because it works unstable in > c

[jira] [Created] (KAFKA-7416) kerberos credentials not being refreshed

Tyler Monahan created KAFKA-7416: Summary: kerberos credentials not being refreshed Key: KAFKA-7416 URL: https://issues.apache.org/jira/browse/KAFKA-7416 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

Hi Chia-Ping, I like the idea of having a close(Duration) call. I would rather keep the existing close(long, TimeUnit) call to avoid breaking compatibility, however. Does the KIP specify keeping the old overload? I can't seem to access the wiki now. best, Colin On Sat, Sep 8, 2018, at 11:

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

Yeah, but it suggests deprecating the old method. Ismael On Fri, Sep 14, 2018 at 2:06 PM Colin McCabe wrote: > Hi Chia-Ping, > > I like the idea of having a close(Duration) call. I would rather keep the > existing close(long, TimeUnit) call to avoid breaking compatibility, > however. Does the

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

Hi All, After looking into AdminClient.java and ConsumerClient.java, following the original idea, I think some type specific codes are unavoidable (we can have a enum class that contain a list of batch-enabled APIs). As the compatibility codes that breaks down the batch, we need to either map one

Build failed in Jenkins: kafka-0.11.0-jdk7 #400

See Changes: [github] KAFKA-7192: Wipe out state store if EOS is turned on and checkpoint file -- [...truncated 1.56 MB...] org.apache.kafka.streams.integration.KStreamRe

Re: [DISCUSS] KIP-372: Naming Joins and Grouping

Hey Bill (and Guozhang), Apologies. I misunderstood what Guozhang was getting at in his earlier remark. I'm a +1 on the current proposal. Thanks, -John On Fri, Sep 14, 2018 at 3:10 PM Bill Bejeck wrote: > All, > > Thanks for the comments, I'll respond to all of your comments in the > recieved

Build failed in Jenkins: kafka-trunk-jdk8 #2967

See Changes: [jason] KAFKA-7280; Synchronize consumer fetch request/response handling (#5495) -- [...truncated 2.68 MB...] org.apache.kafka.streams.test.OutputVerifierTest

[VOTE] KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

Hi all, I would like to start a vote on KIP-361. Link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-361%3A+Add+Consumer+Configuration+to+Disable+Auto+Topic+Creation Thanks, Dhruvil

Re: [VOTE] KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

Thanks for the KIP, +1 (binding). Ismael On Fri, Sep 14, 2018 at 4:56 PM Dhruvil Shah wrote: > Hi all, > > I would like to start a vote on KIP-361. > > Link to the KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-361%3A+Add+Consumer+Configuration+to+Disable+Auto+Topic+Creation > >

Re: Access for a KIP creation

Hello Wladimir, Thanks for your interests. Your ID has been granted permissions. Guozhang On Fri, Sep 14, 2018 at 11:43 AM, Wladimir Schmidt wrote: > Hi, > > I would like to request an access right for a KIP creation. > Reference: https://github.com/apache/kafka/pull/5344#issuecomment-41994

Re: [VOTE] KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

+1 (non-binding) Brandon K. > On Sep 14, 2018, at 6:57 PM, Ismael Juma wrote: > > Thanks for the KIP, +1 (binding). > > Ismael > >> On Fri, Sep 14, 2018 at 4:56 PM Dhruvil Shah wrote: >> >> Hi all, >> >> I would like to start a vote on KIP-361. >> >> Link to the KIP: >> >> https://cwiki.

Re: [DISCUSS] KIP-372: Naming Joins and Grouping

Hello Bill, I've made another pass over the wiki page and it lgtm now. Thanks! Guozhang On Fri, Sep 14, 2018 at 3:05 PM, John Roesler wrote: > Hey Bill (and Guozhang), > > Apologies. I misunderstood what Guozhang was getting at in his earlier > remark. > > I'm a +1 on the current proposal. >

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi Rajini. I'm going to choose *connections.max.expired.credentials.ms * as the option name because it is consistent with the comment I made in the section about how to get the client and server to agree on credential lifetime: "We could add a new AP

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Minor correction: I'm proposing "connections.max.reauth.ms" as the broker-side configuration property, not " connections.max.expired.credentials.ms". Ron On Fri, Sep 14, 2018 at 8:40 PM Ron Dagostino wrote: > Hi Rajini. I'm going to choose *connections.max.expired.credentials.ms >

Jenkins build is back to normal : kafka-2.0-jdk8 #143

See

Re: [DISCUSS] KIP 368: Allow SASL Connections to Periodically Re-Authenticate

Hi everyone. I've updated the KIP to reflect all discussion to date, including the decision to go with the low-level approach. This latest version of the KIP includes the above "connections.max.reauth.ms" proposal, which I know has not been discussed yet. It mentions new metrics, some of which m

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

hi Colin > I like the idea of having a close(Duration) call. I would rather keep the > existing close(long, TimeUnit) call to avoid breaking compatibility, however. > Does the KIP specify keeping the old overload? I can't seem to access the > wiki now. As Ismael said. All existing close(lon

Re: [VOTE] KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

+1 (non-binding). Thanks, Chia-Ping. best, On Fri, Sep 14, 2018, at 21:01, Chia-Ping Tsai wrote: > hi Colin > > > I like the idea of having a close(Duration) call. I would rather keep the > > existing close(long, TimeUnit) call to avoid breaking compatibility, > > however. Does the KIP spe

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

Congratulations, Dong Lin! best, Colin On Wed, Aug 22, 2018, at 05:26, Satish Duggana wrote: > Congrats Dong Lin! > > On Wed, Aug 22, 2018 at 10:08 AM, Abhimanyu Nagrath < > abhimanyunagr...@gmail.com> wrote: > > > Congratulations, Dong! > > > > On Wed, Aug 22, 2018 at 6:20 AM Dhruvil Shah wro

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

On Wed, Aug 15, 2018, at 05:04, Viktor Somogyi wrote: > Hi, > > To weigh-in, I agree with Colin on the API naming, overloads shouldn't > change behavior. I think all of the Java APIs I've used so far followed > this principle and I think we shouldn't diverge. > > Also I think I have an entry abou