[jira] [Updated] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

[ https://issues.apache.org/jira/browse/KAFKA-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Cheng updated KAFKA-5191: --- Attachment: generated_fetcher_docs.png > Autogenerate Consumer Fetcher metrics >

[jira] [Updated] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

[ https://issues.apache.org/jira/browse/KAFKA-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Cheng updated KAFKA-5191: --- Reviewer: Ismael Juma Status: Patch Available (was: In Progress) > Autogenerate Consumer

[jira] [Commented] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

[ https://issues.apache.org/jira/browse/KAFKA-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000279#comment-16000279 ] ASF GitHub Bot commented on KAFKA-5191: --- GitHub user wushujames opened a pull request:

[GitHub] kafka pull request #2993: KAFKA-5191: Autogenerate Consumer Fetcher metrics

GitHub user wushujames opened a pull request: https://github.com/apache/kafka/pull/2993 KAFKA-5191: Autogenerate Consumer Fetcher metrics Autogenerate docs for the Consumer Fetcher's metrics. This is a smaller subset of the original PR https://github.com/apache/kafka/pull/1202.

[jira] [Commented] (KAFKA-5172) CachingSessionStore doesn't fetchPrevious correctly.

[ https://issues.apache.org/jira/browse/KAFKA-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000278#comment-16000278 ] ASF GitHub Bot commented on KAFKA-5172: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #2972: KAFKA-5172: Fix fetchPrevious to find the correct ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/2972 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5172) CachingSessionStore doesn't fetchPrevious correctly.

[ https://issues.apache.org/jira/browse/KAFKA-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-5172. -- Resolution: Fixed Fix Version/s: 0.11.0.0 Issue resolved by pull request 2972

[jira] [Commented] (KAFKA-5099) Replica Deletion Regression from KIP-101

[ https://issues.apache.org/jira/browse/KAFKA-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000261#comment-16000261 ] Onur Karaman commented on KAFKA-5099: - By the way, I reran the file descriptor experiment against both

[jira] [Work started] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

[ https://issues.apache.org/jira/browse/KAFKA-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-5191 started by James Cheng. -- > Autogenerate Consumer Fetcher metrics > - > >

[jira] [Assigned] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

[ https://issues.apache.org/jira/browse/KAFKA-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Cheng reassigned KAFKA-5191: -- Assignee: James Cheng > Autogenerate Consumer Fetcher metrics >

[jira] [Created] (KAFKA-5191) Autogenerate Consumer Fetcher metrics

James Cheng created KAFKA-5191: -- Summary: Autogenerate Consumer Fetcher metrics Key: KAFKA-5191 URL: https://issues.apache.org/jira/browse/KAFKA-5191 Project: Kafka Issue Type: Sub-task

[GitHub] kafka pull request #1202: KAFKA-3480: Autogenerate metrics documentation

Github user wushujames closed the pull request at: https://github.com/apache/kafka/pull/1202 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3480) Autogenerate metrics documentation

[ https://issues.apache.org/jira/browse/KAFKA-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000256#comment-16000256 ] ASF GitHub Bot commented on KAFKA-3480: --- Github user wushujames closed the pull request at:

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

The KIP talks about allowing the user to provide a AlterConfigsPolicy. I assume that even if the user does not provide a custom policy, that there are some basic validation that the broker will do by default? The reason I'm asking is I'm thinking ahead to

Re: [VOTE] KIP-133: List and Alter Configs Admin APIs (second attempt)

+1 > On May 7, 2017, at 9:01 PM, Ismael Juma wrote: > > [Seems like the original message ended up in the discuss thread in GMail, > so trying again] > > Hi everyone, > > I believe I addressed the comments in the discussion thread and given the > impending KIP freeze, I

Re: [VOTE] KIP-153 (separating replication traffic from BytesOutPerSec metric)

+1 > On May 7, 2017, at 7:40 PM, Jun Rao wrote: > > Hi, Everyone, > > Since this is a relatively simple change, I would like to start the voting > process for KIP-153 : Include only client traffic in BytesOutPerSec metric. > >

[jira] [Created] (KAFKA-5190) builds with low parallelism exhaust system open files and crash

radai rosenblatt created KAFKA-5190: --- Summary: builds with low parallelism exhaust system open files and crash Key: KAFKA-5190 URL: https://issues.apache.org/jira/browse/KAFKA-5190 Project: Kafka

[VOTE] KIP-133: List and Alter Configs Admin APIs (second attempt)

[Seems like the original message ended up in the discuss thread in GMail, so trying again] Hi everyone, I believe I addressed the comments in the discussion thread and given the impending KIP freeze, I would like to start the voting process for KIP-133: List and Alter Configs Admin APIs:

[VOTE] KIP-133: List and Alter Configs Admin APIs

Hi everyone, I believe I addressed the comments in the discussion thread and given the impending KIP freeze, I would like to start the voting process for KIP-133: List and Alter Configs Admin APIs: https://cwiki.apache.org/confluence/display/KAFKA/KIP- 133%3A+List+and+Alter+Configs+Admin+APIs

[jira] [Commented] (KAFKA-4982) Add listener tag to socket-server-metrics.connection-... metrics

[ https://issues.apache.org/jira/browse/KAFKA-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000223#comment-16000223 ] Ismael Juma commented on KAFKA-4982: [~ecomar], are you planning to submit a PR for this soon? The KIP

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

Thanks for the feedback Colin. Comments inline. On Sun, May 7, 2017 at 9:29 PM, Colin McCabe wrote: > > Hmm. What's the behavior if I try to list the configuration for a topic > that doesn't exist? It seems like in this case, the whole request has > to return an error, and

Re: [VOTE] KIP-153 (separating replication traffic from BytesOutPerSec metric)

Thanks for the KIP, Jun. +1 from me. Ismael On Mon, May 8, 2017 at 3:40 AM, Jun Rao wrote: > Hi, Everyone, > > Since this is a relatively simple change, I would like to start the voting > process for KIP-153 : Include only client traffic in BytesOutPerSec metric. > >

[jira] [Commented] (KAFKA-5189) trunk unstable - DescribeConsumerGroupTest.testDescribeGroupWithNewConsumerWithShortInitializationTimeout fails 90% of times

[ https://issues.apache.org/jira/browse/KAFKA-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000197#comment-16000197 ] Ismael Juma commented on KAFKA-5189: Possibly related to KAFKA-4948. > trunk unstable - >

[VOTE] KIP-153 (separating replication traffic from BytesOutPerSec metric)

Hi, Everyone, Since this is a relatively simple change, I would like to start the voting process for KIP-153 : Include only client traffic in BytesOutPerSec metric. https://cwiki.apache.org/confluence/display/KAFKA/KIP-153+% 3A+Include+only+client+traffic+in+BytesOutPerSec+metric The vote will

[jira] [Created] (KAFKA-5189) trunk unstable - DescribeConsumerGroupTest.testDescribeGroupWithNewConsumerWithShortInitializationTimeout fails 90% of times

radai rosenblatt created KAFKA-5189: --- Summary: trunk unstable - DescribeConsumerGroupTest.testDescribeGroupWithNewConsumerWithShortInitializationTimeout fails 90% of times Key: KAFKA-5189 URL:

[VOTE] KIP-153 : Include only client traffic in BytesOutPerSec metric

Hi, Everyone, I would like to start the voting process for KIP-153 : Include only client traffic in BytesOutPerSec metric. https://cwiki.apache.org/confluence/display/KAFKA/KIP- 153+%3A+Include+only+client+traffic+in+BytesOutPerSec+metric The vote will run for a minimum of 72 hours. Thanks,

Re: [DISCUSS] KIP-153 : Include only client traffic in BytesOutPerSec metric

Hi, Xavier, I think the per topic metric is more useful for measuring the traffic from the clients. For internal replication, perhaps the aggregate value is enough. Thanks, Jun On Fri, May 5, 2017 at 6:11 PM, Xavier Léauté wrote: > Any reason we are not keeping the

[jira] [Commented] (KAFKA-3806) Adjust default values of log.retention.hours and offsets.retention.minutes

[ https://issues.apache.org/jira/browse/KAFKA-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000185#comment-16000185 ] Di Shang commented on KAFKA-3806: - This is really problematic for mirrormaker since it manages offset

KIP-Template access

The KIP-Template doesn’t show up when I click to create a new KIP page My username is GeoSmith Thanks, George

[jira] [Commented] (KAFKA-4795) Confusion around topic deletion

[ https://issues.apache.org/jira/browse/KAFKA-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000140#comment-16000140 ] Ismael Juma commented on KAFKA-4795: [~vahid], supposedly you may still see this if the Controller is

[jira] [Updated] (KAFKA-4291) TopicCommand --describe shows topics marked for deletion as under-replicated and unavailable

[ https://issues.apache.org/jira/browse/KAFKA-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4291: --- Status: Patch Available (was: Open) > TopicCommand --describe shows topics marked for deletion as

[jira] [Updated] (KAFKA-4291) TopicCommand --describe shows topics marked for deletion as under-replicated and unavailable

[ https://issues.apache.org/jira/browse/KAFKA-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4291: --- Fix Version/s: 0.11.0.0 > TopicCommand --describe shows topics marked for deletion as

Re: [DISCUSS] KIP 151 - Expose Connector type in REST API

Nice. Thanks! -Konstantine On Sat, May 6, 2017 at 10:43 PM, dan wrote: > thanks for the feedback, it all sounds good. i have made the changes to the > pr and the kip. > > dan > > On Fri, May 5, 2017 at 9:29 AM, Konstantine Karantasis < > konstant...@confluent.io> wrote:

[jira] [Created] (KAFKA-5188) Add Integration tests for transactional producer

Jason Gustafson created KAFKA-5188: -- Summary: Add Integration tests for transactional producer Key: KAFKA-5188 URL: https://issues.apache.org/jira/browse/KAFKA-5188 Project: Kafka Issue

[jira] [Commented] (KAFKA-5185) Adding the RecordMetadata that is returned by the producer to the commitRecord method for SourceTask

[ https://issues.apache.org/jira/browse/KAFKA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000103#comment-16000103 ] George Smith commented on KAFKA-5185: - [~kkonstantine] Thanks, will do. > Adding the RecordMetadata

[jira] [Created] (KAFKA-5187) producer.close() times out at 30 seconds

Somnath Choudhuri created KAFKA-5187: Summary: producer.close() times out at 30 seconds Key: KAFKA-5187 URL: https://issues.apache.org/jira/browse/KAFKA-5187 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-5186) Avoid expensive initialization of producer state when upgrading

Jason Gustafson created KAFKA-5186: -- Summary: Avoid expensive initialization of producer state when upgrading Key: KAFKA-5186 URL: https://issues.apache.org/jira/browse/KAFKA-5186 Project: Kafka

[jira] [Commented] (KAFKA-5185) Adding the RecordMetadata that is returned by the producer to the commitRecord method for SourceTask

[ https://issues.apache.org/jira/browse/KAFKA-5185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1688#comment-1688 ] Konstantine Karantasis commented on KAFKA-5185: --- Thank you [~GeoSmith] for the suggested

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Michal, thanks a lot for this comment. I did not consider lambdas when proposing RichFunctions. I thinks it very important to preserve the ability to use lambdas! @Jeyhun: I did not put any thought into this, but can we have a design that allows for both? Also, with regard to lambdas, it might

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

On Fri, May 5, 2017, at 19:24, Ismael Juma wrote: > Hi Colin, > > Thanks for the review. Comments inline. > > On Fri, May 5, 2017 at 9:45 PM, Colin McCabe wrote: > > > As Jun commented, it's probably more consistent to use > > Collection, unless this really needs to be an

Re: [DISCUSS] KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods

To visualise this better, I've created a WIP PR: https://github.com/apache/kafka/pull/2992 tentatively having named the new Stores class TypedStores. Thanks, Michał On 07/05/17 10:16, Eno Thereska wrote: I like this KIP in general and I agree it’s needed. Perhaps Damian can comment on the

[GitHub] kafka pull request #2992: KIP-147 WIP for discussion

GitHub user mihbor opened a pull request: https://github.com/apache/kafka/pull/2992 KIP-147 WIP for discussion This is WIP to help discussion, do not merge. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mihbor/kafka KIP-147-wip

[jira] [Created] (KAFKA-5185) Adding the RecordMetadata that is returned by the producer to the commitRecord method for SourceTask

George Smith created KAFKA-5185: --- Summary: Adding the RecordMetadata that is returned by the producer to the commitRecord method for SourceTask Key: KAFKA-5185 URL: https://issues.apache.org/jira/browse/KAFKA-5185

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

Hi James, On Sun, May 7, 2017 at 8:36 AM, James Cheng wrote: > Makes sense, and I don't have any concerns about it. Actually, I'm excited > for the capability. That means you can completely tell how a topic is > configured, without needing to have any knowledge of how the

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Hi Jeyhun, Thanks for your quick reply. Indeed, I understand the existing ValueMapper/Joiner etc. have to remain as-is for backwards compatibility. I was just expressing my surprise that their proposed richer equivalents weren't functional interfaces too. Thanks, Michał On 07/05/17

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Hi Michal, Thanks for your comments. We proposed the similar solution to yours in KIP (please look at rejected alternatives). However, after the discussion in mailing list I extended it to rich functions. Maybe we should keep them both: simple and rich versions. Cheers, Jeyhun On Sun, May 7,

Re: [DISCUSS]: KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Do I understanding correctly, that with the proposed pattern one could not pass a lambda expression and access the context from within it? TBH, I was hoping that for simple things this would be possible: myStream.map( (key, value, ctx) -> new KeyValue<>(ctx.partition(), value) ) or (more to

Re: [DISCUSS] KIP-150 - Kafka-Streams Cogroup

Hi Kyle, Thanks for the KIP again. A couple of comments: - minor: could you add an exact example (similar to what Jay’s example is, or like your Spark/Pig pointers had) to make this super concrete? - my main concern is that we’re exposing this optimization to the DSL. In an ideal world, an

Kafka Connect Parquet Support?

With the Kafka Connect S3 sink, I can choose Avro or JSON output format. Is there any chance that Parquet will be supported? For record at a time processing, Parquet isn't a good fit. But for reading/writing batches of records, which is what the Kafka Connect Sink is writing, Parquet is generally

Re: [DISCUSS] KIP-147: Add missing type parameters to StateStoreSupplier factories and KGroupedStream/Table methods

I like this KIP in general and I agree it’s needed. Perhaps Damian can comment on the session store issue? Thanks Eno > On May 6, 2017, at 10:32 PM, Michal Borowiecki > wrote: > > Hi Matthias, > > Agreed. I tried your proposal and indeed it would work. > >

Re: [VOTE] KIP-138: Change punctuate semantics

+1 (non binding) Thanks Eno > On May 6, 2017, at 11:01 PM, Bill Bejeck wrote: > > +1 > > Thanks, > Bill > > On Sat, May 6, 2017 at 5:58 PM, Matthias J. Sax > wrote: > >> +1 >> >> Thanks a lot for this KIP! >> >> -Matthias >> >> On 5/6/17 10:18

[jira] [Commented] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999734#comment-15999734 ] ASF GitHub Bot commented on KAFKA-5096: --- GitHub user johnma14 opened a pull request:

[GitHub] kafka pull request #2990: KAFKA-5096: Log invalid user configs and use defau...

GitHub user johnma14 opened a pull request: https://github.com/apache/kafka/pull/2990 KAFKA-5096: Log invalid user configs and use defaults Kafka Streams does not allow users to modify some consumer configurations. Currently, it does not allow modifying the value of

[jira] [Commented] (KAFKA-5096) Only log invalid user configs and overwrite with correct one

[ https://issues.apache.org/jira/browse/KAFKA-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999731#comment-15999731 ] ASF GitHub Bot commented on KAFKA-5096: --- Github user johnma14 closed the pull request at:

[GitHub] kafka pull request #2938: KAFKA-5096: Log invalid user configs and use defau...

Github user johnma14 closed the pull request at: https://github.com/apache/kafka/pull/2938 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-133: List and Alter Configs Admin APIs

> On May 6, 2017, at 11:27 AM, Ismael Juma wrote: > > Hi James, > > Yes, that's right, it will return all config values. For topic configs, > that means falling back to the respective broker config value, which could > also be a default. If we fallback to the broker config

How do I generate/view the site docs, when developing?

Hi, I'm working on some site docs changes, as part of https://issues.apache.org/jira/browse/KAFKA-3480 How do I build the docs, so that I can view them locally and make sure they are right? It used to be that I could "./gradlew siteDocsTar" and then view the created website. But it appears

[jira] [Commented] (KAFKA-4477) Node reduces its ISR to itself, and doesn't recover. Other nodes do not take leadership, cluster remains sick until node is restarted.

[ https://issues.apache.org/jira/browse/KAFKA-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999690#comment-15999690 ] Arpan commented on KAFKA-4477: -- I have opened separate issue KAFKA-5153 . > Node reduces its ISR to itself,