[jira] [Commented] (KAFKA-2394) Use RollingFileAppender by default in log4j.properties

[ https://issues.apache.org/jira/browse/KAFKA-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339041#comment-15339041 ] Ewen Cheslack-Postava commented on KAFKA-2394: -- [~cotedm] I think that makes sense. My

[jira] [Issue Comment Deleted] (KAFKA-3827) log.message.format.version should default to inter.broker.protocol.version

[ https://issues.apache.org/jira/browse/KAFKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3827: --- Comment: was deleted (was: [~junrao] Is that the behavior we really want? I don't think that turning

Re: [DISCUSS] KIP-4 Create Topic Schema

For what is worth, I also agree that this is better. It is awkward to implement and it is not done by any other request and that is why I didn't push for it, but happy that we are going in that direction. It's worth noting that even with this change, it seems to me that looking at the logs may

[jira] [Updated] (KAFKA-3829) Warn that kafka-connect group.id must not conflict with connector names

[ https://issues.apache.org/jira/browse/KAFKA-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3829: - Priority: Critical (was: Minor) Fix Version/s: 0.10.1.0

[jira] [Commented] (KAFKA-3827) log.message.format.version should default to inter.broker.protocol.version

[ https://issues.apache.org/jira/browse/KAFKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339031#comment-15339031 ] Ewen Cheslack-Postava commented on KAFKA-3827: -- [~junrao] Is that the behavior we really

[jira] [Commented] (KAFKA-3827) log.message.format.version should default to inter.broker.protocol.version

[ https://issues.apache.org/jira/browse/KAFKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339030#comment-15339030 ] Ewen Cheslack-Postava commented on KAFKA-3827: -- [~junrao] Is that the behavior we really

[jira] [Updated] (KAFKA-3824) Docs indicate auto.commit breaks at least once delivery but that is incorrect

[ https://issues.apache.org/jira/browse/KAFKA-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3824: - Assignee: Jason Gustafson (was: kambiz shahri) Remaining Estimate:

[jira] [Commented] (KAFKA-3487) Support per-connector/per-task classloaders in Connect

[ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15339018#comment-15339018 ] Ewen Cheslack-Postava commented on KAFKA-3487: -- [~rhauch] Absolutely, did not mean to imply

Re: [DISCUSS] KIP-4 Create Topic Schema

Thanks Grant. Sorry to make the handling on the broker more complicated :( but it seems like the right way to handle this. -Ewen On Sun, Jun 19, 2016 at 8:57 PM, Grant Henke wrote: > Apologies for the delay in response here. > > It will take a bit of tracking inside the

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

I'm in favor of a global config that is then evenly divided amongst the threads of a Kafka Streams instance. On Mon, Jun 13, 2016 at 6:23 PM, Guozhang Wang wrote: > Although this KIP is not mainly for memory management of Kafka Streams, > since it touches on quite some part

Re: [DISCUSS] KIP-64 -Allow underlying distributed filesystem to take over replication depending on configuration

Hi Arvind, Thank you for proposing this KIP. I am not sure how much experience you have in modifying Kafka's core module, so I don't know if you are aware of how deeply the storage and replication layer are integrated within Kafka. There is no clean API to rip out, this KIP will essentially

Re: [DISCUSS] KIP-4 Create Topic Schema

Apologies for the delay in response here. It will take a bit of tracking inside the request object to track this error and then handle it in KafkaApis, but it is possible. I am happy to take that preferred approach. I will update the wiki & patch to handle this scenario and re-initiate the vote

[jira] [Updated] (KAFKA-3868) New producer metric record-size-avg does not provide average record size as advertised

[ https://issues.apache.org/jira/browse/KAFKA-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3868: --- Assignee: (was: Jun Rao) > New producer metric record-size-avg does not provide average record size as >

[jira] [Commented] (KAFKA-3487) Support per-connector/per-task classloaders in Connect

[ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338920#comment-15338920 ] Randall Hauch commented on KAFKA-3487: -- [~ewencp] wrote: {quote} There's lots of other stuff you can

[jira] [Updated] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh

[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh Thakrar updated KAFKA-3663: -- Description: This is a proposal for an admin tool - say, kafka-brokers.sh to provide broker

Re: [VOTE] KIP-62: Allow consumer to send heartbeats from a background thread

+1 on the KIP. On Sat, Jun 18, 2016 at 11:52 AM, Ismael Juma wrote: > If we do that, shouldn't `max.poll.records` remain with the current default > of `Integer.MAX_VALUE`? > > On Sat, Jun 18, 2016 at 5:18 PM, Jay Kreps wrote: > > > +1 > > > > One small but

Re: [DISCUSS] KIP-4 Create Topic Schema

I'm on the same page as Jun & Dana wrt disconnecting. Closing a connection should really be a last resort because we can no longer trust correct behavior in this session. In this case, we detect a bad request, but there's no reason to believe it will affect subsequent requests. There are

Re: [VOTE] KIP-4 Create Topics Schema

Don't necessarily want to add noise here, but I'm -1 based on the disconnect part. See discussion in other thread. (I'm +1 otherwise, and happy to have my vote applied assuming we clean up that one issue.) -Ewen On Thu, Jun 16, 2016 at 6:05 PM, Harsha wrote: > +1 (binding) >

Jenkins build is back to normal : kafka-0.10.0-jdk7 #130

See

Build failed in Jenkins: kafka-trunk-jdk8 #709

See Changes: [me] KAFKA-3850: WorkerSinkTask commit prior to rebalance should be retried [me] MINOR: Mention `log.message.format.version=0.10.0` in rolling upgrade -- [...truncated 8122

[jira] [Commented] (KAFKA-3487) Support per-connector/per-task classloaders in Connect

[ https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338861#comment-15338861 ] Ewen Cheslack-Postava commented on KAFKA-3487: -- [~rhauch] Definitely agree that we want to

[jira] [Created] (KAFKA-3868) New producer metric record-size-avg does not provide average record size as advertised

Ewen Cheslack-Postava created KAFKA-3868: Summary: New producer metric record-size-avg does not provide average record size as advertised Key: KAFKA-3868 URL:

Jenkins build is back to normal : kafka-trunk-jdk7 #1377

See

[GitHub] kafka pull request #1518: KAFKA-3632; remove fetcher metrics on shutdown and...

Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/1518 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: Operator order

Hello Jeyhun, Another way to do this "dynamic routing" is to specify your topology using the lower-level processor API: http://docs.confluent.io/3.0.0/streams/developer-guide.html#processor-api More specifically, you can for example specify both A and D as parents of E when adding processor E,

[jira] [Assigned] (KAFKA-3825) Allow users to specify different types of state stores in Streams DSL

[ https://issues.apache.org/jira/browse/KAFKA-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeyhun Karimov reassigned KAFKA-3825: - Assignee: Jeyhun Karimov > Allow users to specify different types of state stores in

[GitHub] kafka pull request #1514: MINOR: Mention `log.message.format.version=0.10.0`...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1514 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3850) WorkerSinkTask should retry commits if woken up during rebalance or shutdown

[ https://issues.apache.org/jira/browse/KAFKA-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3850: - Resolution: Fixed Fix Version/s: 0.10.1.0 Status: Resolved

[jira] [Commented] (KAFKA-3850) WorkerSinkTask should retry commits if woken up during rebalance or shutdown

[ https://issues.apache.org/jira/browse/KAFKA-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338822#comment-15338822 ] ASF GitHub Bot commented on KAFKA-3850: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request #1511: KAFKA-3850: WorkerSinkTask commit prior to rebalan...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1511 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Commented] (KAFKA-3861) Shrunk ISR before leader crash makes the partition unavailable

[ https://issues.apache.org/jira/browse/KAFKA-3861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338804#comment-15338804 ] Maysam Yabandeh commented on KAFKA-3861: Thanks [~wushujames]. I left a comment on KAFKA-3410. >

Build failed in Jenkins: kafka-trunk-jdk7 #1376

See Changes: [me] KAFKA-3810: replication of internal topics should not be limited by -- [...truncated 12746 lines...] org.apache.kafka.connect.runtime.distributed.WorkerCoordinatorTest >

Build failed in Jenkins: kafka-trunk-jdk8 #708

See Changes: [me] KAFKA-3810: replication of internal topics should not be limited by -- [...truncated 10574 lines...] org.apache.kafka.clients.consumer.internals.SubscriptionStateTest >

[jira] [Resolved] (KAFKA-3810) replication of internal topics should not be limited by replica.fetch.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3810. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request

[jira] [Commented] (KAFKA-3810) replication of internal topics should not be limited by replica.fetch.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338679#comment-15338679 ] ASF GitHub Bot commented on KAFKA-3810: --- Github user asfgit closed the pull request at:

Re: [DISCUSS] KIP-53 Add custom policies for reconnect attempts to NetworkdClient

I took a stab at implementing a simplified exponential + randomized backoff policy here: https://github.com/apache/kafka/pull/1523 Rather than changing public interfaces / using plugins, it just adds a new client configuration "reconnect.backoff.max" that can be used to enable increasing backoff

[GitHub] kafka pull request #1484: KAFKA-3810: replication of internal topics should ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1484 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1523: Support exponential backoff policy via reconnect.b...

GitHub user dpkp opened a pull request: https://github.com/apache/kafka/pull/1523 Support exponential backoff policy via reconnect.backoff.max This PR is an alternate / simplified approach to alternate backoff policies than KIP-53 and KAFKA-3496. Summary: * add

[jira] [Updated] (KAFKA-3864) Kafka Connect Struct.get returning defaultValue from Struct not the field schema

[ https://issues.apache.org/jira/browse/KAFKA-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3864: - Fix Version/s: 0.10.0.1 0.10.1.0 > Kafka Connect Struct.get

[jira] [Updated] (KAFKA-3864) Kafka Connect Struct.get returning defaultValue from Struct not the field schema

[ https://issues.apache.org/jira/browse/KAFKA-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3864: - Priority: Blocker (was: Major) > Kafka Connect Struct.get returning defaultValue

Consumer autocommit interference via unintentional/internal poll() calls

I searched through jira and the mailing list for prior discussion of this and could not find any. Forgive me if I missed it, and if so please send a link! It was raised in the kafka-python issue list by an astute reader that the KafkaConsumer autocommit semantics can be accidentally broken by

Re: Reduce function Null checks

Hi Jeyhun, thanks for reporting! It got already fixed in trunk version. https://issues.apache.org/jira/browse/KAFKA-3589 https://github.com/apache/kafka/pull/1246/files -Matthias On 06/19/2016 02:46 PM, Jeyhun Karimov wrote: > Hi community, > > When using, reduce(Reducer,Reducer,

Re: Operator order

Thanks for clarification. Still don't have an better answer as before. How much overhead my suggestion gives is hard to predict. However, the filter operators will run in the same thread (it's more or less just another chained method call), thus, it should not be too large. Furthermore, it should

Reduce function Null checks

Hi community, When using, reduce(Reducer,Reducer, KeyValueMapper,String) function in KTable, the NullPointerExeption is thrown. In specified function, below call is made: return reduce(adder, subtractor, selector, null, null, name); and afterwards, in reduce(Reducer,Reducer,

Jenkins build is back to normal : kafka-trunk-jdk8 #707

See

[GitHub] kafka pull request #1500: MINOR: Fix javadoc typos in ConsumerRebalanceListe...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka pull request #1500: MINOR: Fix javadoc typos in ConsumerRebalanceListe...

Github user vahidhashemian closed the pull request at: https://github.com/apache/kafka/pull/1500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] kafka pull request #1500: MINOR: Fix javadoc typos in ConsumerRebalanceListe...

GitHub user vahidhashemian reopened a pull request: https://github.com/apache/kafka/pull/1500 MINOR: Fix javadoc typos in ConsumerRebalanceListener You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka