[jira] [Updated] (KAFKA-3486) Autocommit does not execute periodically when partitions are manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-3486: --- Status: Patch Available (was: In Progress) > Autocommit does not execute periodically when

[jira] [Commented] (KAFKA-3486) Autocommit does not execute periodically when partitions are manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219334#comment-15219334 ] ASF GitHub Bot commented on KAFKA-3486: --- GitHub user hachikuji opened a pull request:

[GitHub] kafka pull request: KAFKA-3486: fix autocommit when partitions ass...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/1169 KAFKA-3486: fix autocommit when partitions assigned manually You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-3486

Build failed in Jenkins: kafka-trunk-jdk7 #1161

See Changes: [cshapi] KAFKA-3475; Introduce our own `MiniKdc` [cshapi] KAFKA-3381: Add system test for SimpleConsumerShell -- [...truncated 682 lines...] kafka.server.LogOffsetTest >

[jira] [Updated] (KAFKA-3470) Consumer group coordinator should take commit requests as effective as heartbeats

[ https://issues.apache.org/jira/browse/KAFKA-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zaiming Shi updated KAFKA-3470: --- Description: Group coordinator does not reset heartbeat timer when commit request is received. This

[jira] [Updated] (KAFKA-3470) Consumer group coordinator should take commit requests as effective as heartbeats

[ https://issues.apache.org/jira/browse/KAFKA-3470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zaiming Shi updated KAFKA-3470: --- Description: Group coordinator does not reset heartbeat timer when commit request is received. This

Jenkins build is back to normal : kafka-trunk-jdk8 #493

See

Build failed in Jenkins: kafka-0.10.0-jdk7 #24

See Changes: [cshapi] KAFKA-3475; Introduce our own `MiniKdc` [cshapi] KAFKA-3381: Add system test for SimpleConsumerShell -- [...truncated 1619 lines...] kafka.log.BrokerCompressionTest >

Data lose prevention configuration no longer works with Kafka 9.x

Hi List, Our requirement is to have the producers wait (indefinitely) until kafka broker comes up (in case of maintenance or network interruptions). We have disk caching in front of producer, when the producer waits and it drains when the producer eventually reconnects. This functionality

[jira] [Created] (KAFKA-3487) Support per-connector/per-task classloaders in Connect

Ewen Cheslack-Postava created KAFKA-3487: Summary: Support per-connector/per-task classloaders in Connect Key: KAFKA-3487 URL: https://issues.apache.org/jira/browse/KAFKA-3487 Project: Kafka

[jira] [Updated] (KAFKA-3486) Autocommit does not execute periodically when partitions are manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3486: - Fix Version/s: 0.10.0.0 > Autocommit does not execute periodically when

[jira] [Commented] (KAFKA-3471) min.insync.replicas isn't respected when there's a delaying follower who still in ISR

[ https://issues.apache.org/jira/browse/KAFKA-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219271#comment-15219271 ] ASF GitHub Bot commented on KAFKA-3471: --- Github user kawamuray closed the pull request at:

[GitHub] kafka pull request: KAFKA-3471: min.insync.replicas isn't respecte...

Github user kawamuray closed the pull request at: https://github.com/apache/kafka/pull/1146 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-3471) min.insync.replicas isn't respected when there's a delaying follower who still in ISR

[ https://issues.apache.org/jira/browse/KAFKA-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura resolved KAFKA-3471. -- Resolution: Not A Bug > min.insync.replicas isn't respected when there's a delaying follower

[jira] [Closed] (KAFKA-3471) min.insync.replicas isn't respected when there's a delaying follower who still in ISR

[ https://issues.apache.org/jira/browse/KAFKA-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura closed KAFKA-3471. > min.insync.replicas isn't respected when there's a delaying follower who > still in ISR >

[jira] [Updated] (KAFKA-3471) min.insync.replicas isn't respected when there's a delaying follower who still in ISR

[ https://issues.apache.org/jira/browse/KAFKA-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuto Kawamura updated KAFKA-3471: - Status: Open (was: Patch Available) > min.insync.replicas isn't respected when there's a

[jira] [Commented] (KAFKA-3471) min.insync.replicas isn't respected when there's a delaying follower who still in ISR

[ https://issues.apache.org/jira/browse/KAFKA-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219267#comment-15219267 ] Yuto Kawamura commented on KAFKA-3471: -- Thank you very much for your advice. It was very helpful for

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

+1 -Ewen On Wed, Mar 30, 2016 at 7:20 PM, Gwen Shapira wrote: > So my +1 is back :) > > On Wed, Mar 30, 2016 at 4:51 PM, Ashish Singh wrote: > > > My bad, I moved the config option to rejected alternatives. The new > method > >

[jira] [Updated] (KAFKA-3381) Add system test for SimpleConsumerShell

[ https://issues.apache.org/jira/browse/KAFKA-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-3381: Resolution: Fixed Fix Version/s: 0.10.0.0 0.10.1.0 Status:

[jira] [Commented] (KAFKA-3381) Add system test for SimpleConsumerShell

[ https://issues.apache.org/jira/browse/KAFKA-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219226#comment-15219226 ] ASF GitHub Bot commented on KAFKA-3381: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-3381: Add system test for SimpleConsumer...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1053 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3475) Introduce our own `MiniKdc`

[ https://issues.apache.org/jira/browse/KAFKA-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-3475: Resolution: Fixed Fix Version/s: 0.10.0.0 Status: Resolved (was: Patch Available)

[jira] [Commented] (KAFKA-3475) Introduce our own `MiniKdc`

[ https://issues.apache.org/jira/browse/KAFKA-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219220#comment-15219220 ] ASF GitHub Bot commented on KAFKA-3475: --- Github user asfgit closed the pull request at:

[GitHub] kafka pull request: KAFKA-3475; Introduce our own `MiniKdc`

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1155 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

So my +1 is back :) On Wed, Mar 30, 2016 at 4:51 PM, Ashish Singh wrote: > My bad, I moved the config option to rejected alternatives. The new method > getSupportedPrincipalTypes is what the KIP is proposing. > ​ > > On Wed, Mar 30, 2016 at 4:43 PM, Gwen Shapira

Re: KIP-4 Wiki Update

If the meta data change for KIP-4 happens maybe we just update the KIP-4 page for the release, it would be great to have this available in 0.10 On Wed, Mar 30, 2016 at 3:36 PM, Grant Henke wrote: > Additionally feel free to review the PR (#1095 >

[jira] [Commented] (KAFKA-1981) Make log compaction point configurable

[ https://issues.apache.org/jira/browse/KAFKA-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219089#comment-15219089 ] ASF GitHub Bot commented on KAFKA-1981: --- GitHub user ewasserman opened a pull request:

[GitHub] kafka pull request: KAFKA-1981 Make log compaction point configura...

GitHub user ewasserman opened a pull request: https://github.com/apache/kafka/pull/1168 KAFKA-1981 Make log compaction point configurable @jkreps Implements control over the portion of the head of the log that will not be compacted (i.e. preserved in detail).

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

My bad, I moved the config option to rejected alternatives. The new method getSupportedPrincipalTypes is what the KIP is proposing. ​ On Wed, Mar 30, 2016 at 4:43 PM, Gwen Shapira wrote: > My appologies, but I need to retract my vote. > > The KIP currently specifies two

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

My appologies, but I need to retract my vote. The KIP currently specifies two alternatives (none rejected). So I'm not sure what I'm voting on... For reference, my original +1 was for the new method: getSupportedPrincipalTypes() I'm not even sure how would a config work in that case. Anyway, we

[GitHub] kafka pull request: MINOR: Add check for empty topics iterator in ...

GitHub user SinghAsDev opened a pull request: https://github.com/apache/kafka/pull/1167 MINOR: Add check for empty topics iterator in ReplicaVerificationTool. You can merge this pull request into a Git repository by running: $ git pull https://github.com/SinghAsDev/kafka

Re: [VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

+1. It's a small change and I can see how this will help improve usability for some of the authorizers. Gwen On Wed, Mar 30, 2016 at 11:02 AM, Ashish Singh wrote: > Hi Guys, > > I would like to open the vote for KIP-50. > > KIP: > >

[jira] [Work started] (KAFKA-3486) Autocommit does not execute periodically when partitions are manually assigned

[ https://issues.apache.org/jira/browse/KAFKA-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3486 started by Jason Gustafson. -- > Autocommit does not execute periodically when partitions are manually

Build failed in Jenkins: kafka-trunk-jdk8 #492

See Changes: [wangguoz] MINOR: a simple benchmark for Streams -- [...truncated 1577 lines...] kafka.integration.AutoOffsetResetTest > testResetToLatestWhenOffsetTooLow PASSED

Build failed in Jenkins: kafka-trunk-jdk7 #1160

See Changes: [wangguoz] MINOR: a simple benchmark for Streams -- [...truncated 5395 lines...] kafka.log.BrokerCompressionTest > testBrokerSideCompression[3] PASSED

Re: [VOTE] 0.10.0.0 RC1

I think we are in agreement here. Note that KIP-47 is already voted and merged AFAIK, so it will be in 0.10.0 either way (actually, it will be nice if someone tries using this feature on a the RC...) Gwen On Wed, Mar 30, 2016 at 11:53 AM, Grant Henke wrote: > Below are my

[GitHub] kafka pull request: MINOR: a simple benchmark for Streams

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1164 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: KIP-4 Wiki Update

Additionally feel free to review the PR (#1095 ) to see what the current proposal/implementation looks like. Right now its using null to signify no topics. That can be changed based on the discussion here. On Wed, Mar 30, 2016 at 2:34 PM, Grant Henke

Re: [VOTE] 0.10.0.0 RC1

Below are my thoughts on the release features. I don't feel too strongly about it, but I figured I would lay out my current perspective on the 0.10 release. When considering the optimal time to do this major release the considerations I am weighing are: - Compatibility - Are more

Re: KIP-4 Wiki Update

We can add an internal class until then (it's pretty trivial) since the request classes are internal. Ismael On Wed, Mar 30, 2016 at 7:00 PM, Gwen Shapira wrote: > I like it, but we are not on Java8 yet, and I don't think we want to block > on that :) > > On Wed, Mar 30,

[VOTE] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

Hi Guys, I would like to open the vote for KIP-50. KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-50+-+Enhance+Authorizer+interface+to+be+aware+of+supported+Principal+Types Discuss thread: here

Re: KIP-4 Wiki Update

I like it, but we are not on Java8 yet, and I don't think we want to block on that :) On Wed, Mar 30, 2016 at 10:53 AM, Ismael Juma wrote: > On Wed, Mar 30, 2016 at 6:21 PM, Gwen Shapira wrote: > > > Ismael, can you detail how the Optional approach would

Re: [VOTE] 0.10.0.0 RC1

If there is time, we will be very keen on including KIP-43 in the 0.10.0 release. Thanks, Rajini On Wed, Mar 30, 2016 at 6:37 PM, Ashish Singh wrote: > If it is possible, I am also in favor of having some time to include a few > more KIPs in 0.10. > > On Wed, Mar 30, 2016

Re: KIP-4 Wiki Update

On Wed, Mar 30, 2016 at 6:21 PM, Gwen Shapira wrote: > Ismael, can you detail how the Optional approach would work in the wire > protocol? It sounds good, but I'm unclear on what this would look like on > the wire. > In the wire protocol, we would still use a size of -1 like

Re: [DISCUSS] KIP-50 - Enhance Authorizer interface to be aware of supported Principal Types

I am going to initiate a vote thread for this. On Mon, Mar 28, 2016 at 1:28 PM, Ashish Singh wrote: > Hello Harsha, > > Pinging again. This is a minor KIP and it has been lying around for quite > some time. If providing supported principal types via a config is what you >

Re: [VOTE] 0.10.0.0 RC1

I think it would be nice to get a resolution on KIP-35 before the release. We were reluctant to push it through when the timeline was tight (because of unclear implications), but if we have more time to consider it, it definitely would feel better not to have the issue hanging. -Jason On Wed,

Re: KIP-4 Wiki Update

Process comment: Since KIP-4 is voted and signed. Perhaps a small KIP detailing the suggested Metadata API changes so it will be easy to refer to what we are discussing? Gwen On Wed, Mar 30, 2016 at 10:21 AM, Gwen Shapira wrote: > I guess Scala corrupted my brain because I

Re: KIP-4 Wiki Update

I guess Scala corrupted my brain because I can't see why ()-> all and ([])-> none is intuitive :) I would much prefer: empty-> no topics, null -> exception and boolean for "all". Ismael, can you detail how the Optional approach would work in the wire protocol? It sounds good, but I'm unclear on

Re: [VOTE] 0.10.0.0 RC1

1. Thank you, Guozhang. We'll roll out a new RC early next week if this is ok? 2. I'm seeing very little community feedback (positive or negative) on the release vote. Would people feel better about the release if we delay it a bit to get some high-priority and really awesome KIPs in? (KIP-35,

Re: KIP-4 Wiki Update

My concern is that this approach is error-prone and a bit magical. It is possible to do that in the wire protocol while using a safer approach in the code if people prefer this option (by using a type like `Optional` in Java). Ismael On Wed, Mar 30, 2016 at 5:33 PM, Jason Gustafson

Re: KIP-4 Wiki Update

Yeah, that actually is intuitive when viewed that way. -Jason On Wed, Mar 30, 2016 at 9:25 AM, Dana Powers wrote: > Perhaps python has corrupted my brain, but a null arg seems quite clean to > me: > > getTopics() -> returns all > getTopics([]) -> returns none >

Re: KIP-4 Wiki Update

Perhaps python has corrupted my brain, but a null arg seems quite clean to me: getTopics() -> returns all getTopics([]) -> returns none getTopics([foo, bar]) -> returns foo and bar only -Dana On Wed, Mar 30, 2016 at 9:10 AM, Jason Gustafson wrote: > > > > Yes, I think

Data lose prevention configuration no longer works with Kafka 9.x

Hi List, Our requirement is to have the producers wait (indefinitely) until kafka broker comes up (in case of maintenance or network interruptions). We have disk caching in front of producer, when the producer waits and it drains when the producer eventually reconnects. This functionality

Re: KIP-4 Wiki Update

> > Yes, I think empty should be "no topics" too. However, I would suggest > using a boolean to indicate "all topics" and null should not be allowed (as > it is now). I think this is a clearer API and it's similar to > how org.apache.kafka.clients.Metadata works today. +1. Having null imply all

Re: UNKNOWN_MEMBER_ID issue

Hi there, Have you upgraded to 0.9.0.1? There were a couple bugs that may cause what you're seeing. After restarting, the maximum time to rejoin should be bounded by the session timeout (which defaults to 30 seconds). If it takes longer, you may have found a bug and we'd appreciate hearing the

Re: KIP-4 Wiki Update

I haven't had a chance to look at the KIP in detail, but a quick comment below. On Wed, Mar 30, 2016 at 4:49 PM, Dana Powers wrote: > > MetadataRequest v1: long-term / conceptually, I think a "null" topic list > aligns better with fetching all topics. Empty list aligns

Re: KIP-4 Wiki Update

Grant - sorry I was unable to attend. Getting API access to admin functionality has been a big ask for python client users. I like this KIP a lot. I reviewed the details quickly. Here are some comments: MetadataRequest v1: long-term / conceptually, I think a "null" topic list aligns better with

Re: KIP-4 Wiki Update

I didn't get anyone in attendance for this meeting. If you would like to discuss it please let me know. Thank you, Grant On Mon, Mar 28, 2016 at 9:18 AM, Grant Henke wrote: > I am hoping to get more discussion and feedback around the blocking vs > async discussion so I can

UNKNOWN_MEMBER_ID issue

This question is about UNKNOWN_MEMBER_ID issue In fact , we meet the block issue . But we don't have solution to resolve this problem .. For example : we have one job running on production environment, and consume data . But suddenly , the consumer can not consume message (maybe heartbeat or

Build failed in Jenkins: kafka-trunk-jdk7 #1159

See Changes: [me] MINOR: Advance system test ducktape dependency from 0.3.10 to 0.4.0 -- [...truncated 5999 lines...] org.apache.kafka.connect.runtime.AbstractHerderTest > taskStatus PASSED

Request for edit access to Kafka wiki

Hi All, We (GS Lab Pvt Ltd ) want to publish C++ Kafka client. We have git repository ready. Now we want add entry on Kafka “Clients” page (Confluence wiki page) for this new client. We did create login for the Confluence and login with that. But I am not able to edit the

Build failed in Jenkins: kafka-trunk-jdk8 #491

See Changes: [me] MINOR: Advance system test ducktape dependency from 0.3.10 to 0.4.0 -- [...truncated 869 lines...] kafka.server.ServerGenerateBrokerIdTest > testUserConfigAndGeneratedBrokerId

[GitHub] kafka pull request: MINOR: Advance system test ducktape dependency...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1165 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Updated] (KAFKA-3475) Introduce our own `MiniKdc`

[ https://issues.apache.org/jira/browse/KAFKA-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3475: --- Fix Version/s: 0.10.0.1 Status: Patch Available (was: Open) > Introduce our own `MiniKdc`