Re: [VOTE] 1.1.1 RC3

+1 (non binding) rolling upgrade of shared staging multitenacy (200+ consumer groups) cluster from 1.1.0 to 1.1.1-rc3 using the kafka_2.11-1.1.1.tgz artifact. cluster looks healthy after upgrade. Lack of burrow lag suggests consumers are still happy, and incoming messages remains the same. On

Re: [DISCUSS]: KIP-339: Create a new ModifyConfigs API

bq. Map changes, Set removals, Is it possible to combine the two parameters into one Map where null Config value signifies removal of config ? This way, the following wouldn't occur (reducing un-intended config removal): bq. If a configuration key is specified in both *changes* and

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

wiki page is currently read-only and it is unavailable for write operation. Will update it later. On Wed, Jul 11, 2018 at 8:48 PM, Dong Lin wrote: > Ah I see. Thanks for the suggestion. It is updated now. > > On Wed, Jul 11, 2018 at 8:13 PM, Ted Yu wrote: > >> bq. the same approach used by

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Ah I see. Thanks for the suggestion. It is updated now. On Wed, Jul 11, 2018 at 8:13 PM, Ted Yu wrote: > bq. the same approach used by "--config-file" in ConfigCommand. > > I should have copied more from the KIP. > What I meant was that ConfigCommand doesn't use "--config-file" option. So >

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

bq. the same approach used by "--config-file" in ConfigCommand. I should have copied more from the KIP. What I meant was that ConfigCommand doesn't use "--config-file" option. So 'same approach' implies StreamsResetter class, not ConfigCommand. I didn't mean to change ConfigCommand w.r.t. name

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Do you mean we should replace "--command-config" with "--config-file" in ConfigCommand? There is backward compatibility concern with the change. I am not sure the benefit of this change is worth the effort to deprecate the old config. Maybe we should do it separately if more people thing it is

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

bq. "--config-file" in ConfigCommand. Please update the above - it should be StreamsResetter On Wed, Jul 11, 2018 at 7:59 PM Dong Lin wrote: > Hey Ted, > > Thanks much for the suggestion. Yeah "config-file" looks better than > "command-config". I have updated the KIP as suggested. > > Thanks,

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Hey Ted, Thanks much for the suggestion. Yeah "config-file" looks better than "command-config". I have updated the KIP as suggested. Thanks, Dong On Wed, Jul 11, 2018 at 5:57 PM, Ted Yu wrote: > Looking at StreamsResetter.java : > >commandConfigOption =

Re: [DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Looking at StreamsResetter.java : commandConfigOption = optionParser.accepts("config-file", "Property file containing configs to be passed to admin cl Not sure you have considered naming the option in the above fashion. Probably add the above to Alternative section. Cheers On Wed, Jul

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

Hi all, I have updated KIP-328 with all the feedback I've gotten so far. Please take another look and let me know what you think! Thanks, -John On Wed, Jul 11, 2018 at 12:28 AM Guozhang Wang wrote: > That is a good point.. > > I cannot think of a better option than documentation and warning,

Build failed in Jenkins: kafka-trunk-jdk8 #2797

See Changes: [github] MINOR: Tighten FileRecords size checks to prevent overflow (#5332) -- [...truncated 869.89 KB...]

Jenkins build is back to normal : kafka-trunk-jdk10 #282

See

[DISCUSS] KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Hi all, I have created KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file. See https://cwiki.apache.org/confluence/display/KAFKA/KIP-340%3A+Allow+kafka-reassign-partitions.sh+and+kafka-log-dirs.sh+to+take+admin+client+property+file . This KIP

Re: [VOTE] 1.1.1 RC3

+1 (non-binbding) ... built from source, run tests and used it with several of my applications without any problems. Thanks & Regards Jakub On Mon, Jul 9, 2018 at 12:36 AM Dong Lin wrote: > Hello Kafka users, developers and client-developers, > > This is the fourth candidate for release of

Re: [VOTE] 2.0.0 RC2

+1 (non-binbding) ... I built the RC2 from source, run tests and used it with several of my applications without any problems. Thanks & Regards Jakub On Tue, Jul 10, 2018 at 7:17 PM Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers, > > > This is the third candidate

[jira] [Created] (KAFKA-7152) replica should be in-sync if its LEO equals leader's LEO

Dong Lin created KAFKA-7152: --- Summary: replica should be in-sync if its LEO equals leader's LEO Key: KAFKA-7152 URL: https://issues.apache.org/jira/browse/KAFKA-7152 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-7151) Broker running out of disk space may result in state where unclean leader election is required

Anna Povzner created KAFKA-7151: --- Summary: Broker running out of disk space may result in state where unclean leader election is required Key: KAFKA-7151 URL: https://issues.apache.org/jira/browse/KAFKA-7151

Re: [VOTE] 2.0.0 RC2

+1 (non-binding) Built executables from source and ran quickstart (Ubuntu / Java 8) Thanks! --Vahid From: Brett Rann To: dev@kafka.apache.org Cc: Users , kafka-clients Date: 07/10/2018 09:53 PM Subject:Re: [VOTE] 2.0.0 RC2 +1 (non binding) rolling upgrade of tiny

[jira] [Created] (KAFKA-7150) Error in processing fetched data for one partition may stop follower fetching other partitions

Anna Povzner created KAFKA-7150: --- Summary: Error in processing fetched data for one partition may stop follower fetching other partitions Key: KAFKA-7150 URL: https://issues.apache.org/jira/browse/KAFKA-7150

Re: [VOTE] KIP-232: Detect outdated metadata using leaderEpoch and partitionEpoch

Hey Jun, Certainly. We can discuss later after KIP-320 settles. Thanks! Dong On Wed, Jul 11, 2018 at 8:54 AM, Jun Rao wrote: > Hi, Dong, > > Sorry for the late response. Since KIP-320 is covering some of the similar > problems described in this KIP, perhaps we can wait until KIP-320 settles

[DISCUSS]: KIP-339: Create a new ModifyConfigs API

Hi all, Previously, we discussed some issues with alterConfigs here on the mailing list, and eventually came to the conclusion that the RPC as implemented can't be used for a shell command modifying configurations. I wrote up a small KIP to fix the issues with the RPC. Please take a look at

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hey Anna, Thanks much for the explanation. Approach 1 also sounds good to me. I think findOffsets() is useful for users who don't use automatic offset reset policy. Just one more question. Since users who store offsets externally need to provide leaderEpoch to findOffsets(...), do we need an

Re: [DISCUSS] KIP-320: Allow fetchers to detect and handle log truncation

Hi Dong, What I called “not covering all use cases” is what you call best-effort (not guaranteeing some corner cases). I think we are on the same page here. I wanted to be clear in the API whether the consumer seeks to a position (offset) or to a record (offset, leader epoch). The only

Re: [VOTE] KIP-232: Detect outdated metadata using leaderEpoch and partitionEpoch

Hi, Dong, Sorry for the late response. Since KIP-320 is covering some of the similar problems described in this KIP, perhaps we can wait until KIP-320 settles and see what's still left uncovered in this KIP. Thanks, Jun On Mon, Jun 4, 2018 at 7:03 PM, Dong Lin wrote: > Hey Jun, > > It seems

Re: [VOTE] KIP-291: Have separate queues for control requests and data requests

Hi, Lucas, 2. Good point about not knowing the request type in memory pool. Looking at the implementation. It seems that queued.max.request.bytes is orthogonal to queued.max.requests. So, this seems fine. 3. The implementation that you suggested sounds good. It would be useful not to

Re: [DISCUSS] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

LGTM. Thanks for the KIP. Thanks. On Wed, Jul 11, 2018 at 3:31 PM Chia-Ping Tsai wrote: > hi Kafka, > > KIP-338 is trying to make us exclude internal topics easily when using the > kafka-topics.sh. An new option will be added to kafka-topics.sh > (TopicCommand) and user can use the option to

[DISCUSS] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

hi Kafka, KIP-338 is trying to make us exclude internal topics easily when using the kafka-topics.sh. An new option will be added to kafka-topics.sh (TopicCommand) and user can use the option to exclude all internal topics when running the list or describe command.

Re: Kafka Streams new release

>From the release plan: " While the target release date is fixed at ~2w after code freeze, RCs will roll out as needed until the release vote passes" https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=80448820 You can follow the voting threads on this mailing list - the current

Re: Requesting Permission To Create KIP And Assign JIRAs

Hi, I received access to assign JIRAs now, but I still cannot create KIPs. When I hover over the "Create KIP" button, it says I do not have permission to create content. When I click on the button, it goes a 404 Not Found page. Can someone add me? Thanks! Regards, Kevin On Tue, Jun 26, 2018

Kafka Streams new release

When is the next Kafka Streams release (Kafka Streams 2.0.0) which was tentatively on 26June?

[jira] [Reopened] (KAFKA-7141) kafka-consumer-group doesn't describe existing group

[ https://issues.apache.org/jira/browse/KAFKA-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxihx reopened KAFKA-7141: --- Assignee: huxihx > kafka-consumer-group doesn't describe existing group >