[jira] [Commented] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354183#comment-15354183 ] Jun Rao commented on KAFKA-3915: The writeBuffer is set to maxMessageSize. We can't really exceed that.

Re: Kafka Connect Transformers

Need permissions to edit the wiki. Username is ‘snisarg’. Thanks, Nisarg. > On Jun 28, 2016, at 09:08, Nisarg Shah wrote: > > Hello, > > I need to create a page so that I can write a Kafka Improvement Proposal for > the below. My username is ‘snisarg’. > > Thanks, >

[jira] [Commented] (KAFKA-3252) compression type for a topic should be used during log compaction

[ https://issues.apache.org/jira/browse/KAFKA-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354083#comment-15354083 ] Jun Rao commented on KAFKA-3252: [~omkreddy], sorry for creating the duplicated jira. I took a quick look

[jira] [Commented] (KAFKA-3810) replication of internal topics should not be limited by replica.fetch.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15354011#comment-15354011 ] Jun Rao commented on KAFKA-3810: [~onurkaraman], [~becket_qin], thanks for the patch. I have a couple of

[jira] [Commented] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353972#comment-15353972 ] ASF GitHub Bot commented on KAFKA-3905: --- Github user iBuddha closed the pull request at:

[GitHub] kafka pull request #1564: KAFKA-3905:check collection elements in KafkaConsu...

Github user iBuddha closed the pull request at: https://github.com/apache/kafka/pull/1564 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-3917) Some __consumer_offsets replicas grow way too big

Maxim Vladimirskiy created KAFKA-3917: - Summary: Some __consumer_offsets replicas grow way too big Key: KAFKA-3917 URL: https://issues.apache.org/jira/browse/KAFKA-3917 Project: Kafka

Re: [DISCUSS] KIP-48 Support for delegation tokens as an authentication mechanism

Hi, I am suggesting that we will only allow the renewal by users that authenticated using *non* delegation token mechanism. For example, If user Alice authenticated using kerberos and requested delegation tokens, only user Alice authenticated via non delegation token mechanism can renew. Clients

[jira] [Updated] (KAFKA-3916) Connection from controller to broker disconnects

[ https://issues.apache.org/jira/browse/KAFKA-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Powell updated KAFKA-3916: --- Description: We recently upgraded from 0.8.2.1 to 0.9.0.1. Since then, several times per day, the

[jira] [Updated] (KAFKA-3916) Connection from controller to broker disconnects

[ https://issues.apache.org/jira/browse/KAFKA-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Powell updated KAFKA-3916: --- Description: We recently upgraded from 0.8.2.1 to 0.9.0.1. Since then, several times per day, the

[jira] [Created] (KAFKA-3916) Connection from controller to broker disconnects

Dave Powell created KAFKA-3916: -- Summary: Connection from controller to broker disconnects Key: KAFKA-3916 URL: https://issues.apache.org/jira/browse/KAFKA-3916 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-3809) Auto-generate documentation for topic-level configuration

[ https://issues.apache.org/jira/browse/KAFKA-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353716#comment-15353716 ] James Cheng commented on KAFKA-3809: [~gwenshap] or [~hachikuji], do either of you have time to review

Re: [DISCUSS] Client Side Auto Topic Creation

Hey Grant, I'm also inclined to the latter two options you've suggested (and probably more to the one which uses AdminClient since it encourages separation of privileges). Raising an exception gives the user the most control over how to handle unknown topics. One of the problems that we currently

[GitHub] kafka pull request #1568: MINOR: Fix consumer constructor doc string

GitHub user granthenke opened a pull request: https://github.com/apache/kafka/pull/1568 MINOR: Fix consumer constructor doc string You can merge this pull request into a Git repository by running: $ git pull https://github.com/granthenke/kafka consumer-doc Alternatively you

Re: [VOTE] KIP-65 Expose timestamps to Connect

Thanks all, looks like the vote passed with +5 binding and +1 non-binding. I'll update the KIP page. On Tue, Jun 28, 2016 at 12:54 PM Ewen Cheslack-Postava wrote: > +1 (binding) > > -Ewen > > On Fri, Jun 24, 2016 at 4:58 PM, Ismael Juma wrote: > > > +1

Re: [VOTE] KIP-65 Expose timestamps to Connect

+1 (binding) -Ewen On Fri, Jun 24, 2016 at 4:58 PM, Ismael Juma wrote: > +1 (binding), provided that we make the usage of `Long/null` versus > `long/-1` consistent. > > Ismael > > On Sat, Jun 25, 2016 at 12:42 AM, Gwen Shapira wrote: > > > +1 > > > > On

Issue? With java Producer (v9)

Hello. I've posted a similar email to the users group but haven't had much luck determining if what I'm seeing is a bug or simply misconfiguration. This issue I'm seeing is with the v9 producer. Here is some test code: @Test public void test1() throws InterruptedException { Producer

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353508#comment-15353508 ] ASF GitHub Bot commented on KAFKA-3761: --- Github user theduderog closed the pull request at:

[jira] [Commented] (KAFKA-3761) Controller has RunningAsBroker instead of RunningAsController state

[ https://issues.apache.org/jira/browse/KAFKA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353509#comment-15353509 ] ASF GitHub Bot commented on KAFKA-3761: --- GitHub user theduderog reopened a pull request:

[GitHub] kafka pull request #1437: KAFKA-3761: Remove BrokerState "RunningAsControlle...

GitHub user theduderog reopened a pull request: https://github.com/apache/kafka/pull/1437 KAFKA-3761: Remove BrokerState "RunningAsController" The reasons to remove it are: 1. It's currently broken. The purpose of the [JIRA](https://issues.apache.org/jira/browse/KAFKA-3761)

[GitHub] kafka pull request #1437: KAFKA-3761: Remove BrokerState "RunningAsControlle...

Github user theduderog closed the pull request at: https://github.com/apache/kafka/pull/1437 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3822) Kafka Consumer close() hangs indefinitely if Kafka Broker shutdown while connected

[ https://issues.apache.org/jira/browse/KAFKA-3822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353497#comment-15353497 ] Ashish K Singh commented on KAFKA-3822: --- [~hachikuji] while working out a POC, I thought more about

[GitHub] kafka pull request #1567: Minor: fix Bash shebang on vagrant/ scripts

GitHub user shikhar opened a pull request: https://github.com/apache/kafka/pull/1567 Minor: fix Bash shebang on vagrant/ scripts You can merge this pull request into a Git repository by running: $ git pull https://github.com/shikhar/kafka vagrant-scripts-shebang

[jira] [Commented] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353422#comment-15353422 ] Ismael Juma commented on KAFKA-3915: "Another point is that in the future, when copying the messages

[jira] [Updated] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3915: --- Priority: Blocker (was: Critical) > LogCleaner IO buffers do not account for potential size

[jira] [Commented] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15353396#comment-15353396 ] Jun Rao commented on KAFKA-3915: [~twbecker], thanks for reporting this. Keeping the old message format

Re: [DISCUSS] KIP-54 Sticky Partition Assignment Strategy

Hi Gouzhang, Thanks for the reference. A similar question was asked earlier about whether, with sticky assignor, consumers stick to their previous partitions if they die and come back later. Currently the sticky assignor does not support that because it only preserves only the last assignment

Re: KTable.filter usage, memory consumption and materialized view semantics

Is this point of view consistent with new ticket 3911 (Enforce KTable materialisation ) just submitted by Eno. T? Should the two tickets be linked somehow if they are related? My concern is that, the overhead of requesting the source KTable to be materialized (i.e. creating a state store, and

[GitHub] kafka pull request #1566: MINOR: bug fixes to ducktape services

GitHub user alexlod opened a pull request: https://github.com/apache/kafka/pull/1566 MINOR: bug fixes to ducktape services These are required for Schema Registry muckrake tests. Here's a (mostly successful) run with these changes:

[jira] [Work started] (KAFKA-3911) Enforce KTable materialization

[ https://issues.apache.org/jira/browse/KAFKA-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3911 started by Eno Thereska. --- > Enforce KTable materialization > -- > >

[DISCUSS] Client Side Auto Topic Creation

With the KIP-4 create topic schema voted and passed and a PR available upstream. I wanted to discuss moving the auto topic creation from the broker side to the client side (KAFKA-2410 ). This change has many benefits - Remove the need for

Re: Kafka Connect Transformers

Hello, I need to create a page so that I can write a Kafka Improvement Proposal for the below. My username is ‘snisarg’. Thanks, Nisarg > On Jun 19, 2016, at 10:43 PM, Nisarg Shah wrote: > > Hello, > > I am looking to do https://issues.apache.org/jira/browse/KAFKA-3209

[jira] [Updated] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3915: --- Priority: Critical (was: Major) > LogCleaner IO buffers do not account for potential size difference

[jira] [Updated] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

[ https://issues.apache.org/jira/browse/KAFKA-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3915: --- Fix Version/s: 0.10.0.1 > LogCleaner IO buffers do not account for potential size difference due to

Re: [VOTE] KIP-4 Delete Topics Schema

Thanks to all who voted. The KIP-4 Delete Topics changes passed with +4 (binding), and +3 (non-binding). There is a branch based off the create topics PR available here: https://github.com/granthenke/kafka/tree/delete-wire-new I will open a PR once the create topics patch is in:

[jira] [Created] (KAFKA-3915) LogCleaner IO buffers do not account for potential size difference due to message format change

Tommy Becker created KAFKA-3915: --- Summary: LogCleaner IO buffers do not account for potential size difference due to message format change Key: KAFKA-3915 URL: https://issues.apache.org/jira/browse/KAFKA-3915

Re: How to create a KIP?

Hi David, If you provide your wiki id, one of us can give you permission. Ismael On Tue, Jun 28, 2016 at 7:24 AM, Pengwei (L) wrote: > Hi ALL, >I want to submit a KIP, but according to the website: >

How to create a KIP?

Hi ALL, I want to submit a KIP, but according to the website: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals: This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka. To create your own KIP, click on

Re: [VOTE] KIP-55: Secure quotas for authenticated users

Jun, Thank you for the review. I have changed all default property configs to be stored with the node name . So the defaults are /config/clients/ for default client-id quota, /config/users/ for default user quota and /config/users/ for default quota. Hope that makes sense. On

[jira] [Created] (KAFKA-3914) Global discovery of state stores

Damian Guy created KAFKA-3914: - Summary: Global discovery of state stores Key: KAFKA-3914 URL: https://issues.apache.org/jira/browse/KAFKA-3914 Project: Kafka Issue Type: Sub-task

[jira] [Work started] (KAFKA-3912) Query local state stores

[ https://issues.apache.org/jira/browse/KAFKA-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3912 started by Damian Guy. - > Query local state stores > > > Key:

[jira] [Updated] (KAFKA-3913) Old consumer's metrics error when using IPv6

[ https://issues.apache.org/jira/browse/KAFKA-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3913: --- Priority: Major (was: Minor) > Old consumer's metrics error when using IPv6 >

[jira] [Commented] (KAFKA-3912) Query local state stores

[ https://issues.apache.org/jira/browse/KAFKA-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352784#comment-15352784 ] ASF GitHub Bot commented on KAFKA-3912: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #1565: KAFKA-3912: Query local state stores

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1565 KAFKA-3912: Query local state stores @guozhangwang @enothereska please review You can merge this pull request into a Git repository by running: $ git pull https://github.com/dguy/kafka kafka-3912

[jira] [Created] (KAFKA-3913) Old consumer's metrics error when using IPv6

Pengwei created KAFKA-3913: -- Summary: Old consumer's metrics error when using IPv6 Key: KAFKA-3913 URL: https://issues.apache.org/jira/browse/KAFKA-3913 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-3910) Cyclic schema support in ConnectSchema and SchemaBuilder

[ https://issues.apache.org/jira/browse/KAFKA-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Hofman updated KAFKA-3910: --- Description: Cyclic schema's are not supported by ConnectSchema or SchemaBuilder. Subsequently the

[jira] [Comment Edited] (KAFKA-3705) Support non-key joining in KTable

[ https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352103#comment-15352103 ] Jan Filipiak edited comment on KAFKA-3705 at 6/28/16 9:41 AM: -- I will just

[jira] [Updated] (KAFKA-3912) Query local state stores

[ https://issues.apache.org/jira/browse/KAFKA-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3912: -- Description: Implement read-only interface and QueryableStoreType for key-value state store Implement

[jira] [Created] (KAFKA-3912) Query local state stores

Damian Guy created KAFKA-3912: - Summary: Query local state stores Key: KAFKA-3912 URL: https://issues.apache.org/jira/browse/KAFKA-3912 Project: Kafka Issue Type: Sub-task Components:

[jira] [Created] (KAFKA-3911) Enforce KTable materialization

Eno Thereska created KAFKA-3911: --- Summary: Enforce KTable materialization Key: KAFKA-3911 URL: https://issues.apache.org/jira/browse/KAFKA-3911 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-3910) Cyclic schema support in ConnectSchema and SchemaBuilder

John Hofman created KAFKA-3910: -- Summary: Cyclic schema support in ConnectSchema and SchemaBuilder Key: KAFKA-3910 URL: https://issues.apache.org/jira/browse/KAFKA-3910 Project: Kafka Issue

[jira] [Updated] (KAFKA-3870) Expose state store names to DSL

[ https://issues.apache.org/jira/browse/KAFKA-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3870: Description: Currently in Kafka Streams the state store names are hidden in the DSL. This JIRA

[jira] [Updated] (KAFKA-3870) Expose state store names to DSL

[ https://issues.apache.org/jira/browse/KAFKA-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3870: Summary: Expose state store names to DSL (was: Remove the window name from Windows class) >

[jira] [Updated] (KAFKA-3870) Remove the window name from Windows class

[ https://issues.apache.org/jira/browse/KAFKA-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-3870: Issue Type: Sub-task (was: Improvement) Parent: KAFKA-3909 > Remove the window name from

[jira] [Created] (KAFKA-3909) Queryable state for Kafka Streams

Eno Thereska created KAFKA-3909: --- Summary: Queryable state for Kafka Streams Key: KAFKA-3909 URL: https://issues.apache.org/jira/browse/KAFKA-3909 Project: Kafka Issue Type: New Feature

[DISCUSS] KIP-67: Queryable state for Kafka Stream

Hi, We have created KIP 67: Queryable state for Kafka Streams` https://cwiki.apache.org/confluence/display/KAFKA/KIP-67%3A+Queryable+state+for+Kafka+Streams Please take a look. Feedback is appreciated. Thank you

[jira] [Commented] (KAFKA-3904) File descriptor leaking (Too many open files) for long running stream process

[ https://issues.apache.org/jira/browse/KAFKA-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352555#comment-15352555 ] Ismael Juma commented on KAFKA-3904: [~h...@pinterest.com], I didn't mean to suggest that

[jira] [Commented] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe

[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352551#comment-15352551 ] ASF GitHub Bot commented on KAFKA-3905: --- GitHub user iBuddha opened a pull request:

[GitHub] kafka pull request #1564: KAFKA-3905:check collection elements in KafkaConsu...

GitHub user iBuddha opened a pull request: https://github.com/apache/kafka/pull/1564 KAFKA-3905:check collection elements in KafkaConsumer#subscribe. If topic collection has null or "" as its element, throw an IllegalArgumentException. You can merge this pull request into a Git