Build failed in Jenkins: kafka-trunk-jdk7 #2928

See Changes: [wangguoz] KAFKA-6119: Bump epoch when expiring transactions in the -- [...truncated 382.23 KB...] kafka.server.ServerGenerateBrokerIdTest > testDisableGener

RE: Use self contained tokens instead of ACL

Hi Sönke, Thanks for your feedback, sorry that I didn’t gave you the whole picture in first place: We are using an Architecture, which tries to avoid to fetch or pull anything from a 3rd party during runtime. Therefore we are using self-contained tokens and client side load balancing with a mi

Re: Use self contained tokens instead of ACL

Hi Peter, thanks for the explanation, it all makes sense now :) I can't say that I immediately see an easy way forward though to be honest. The big issue, I think, is getting the token to Kafka (and hopefully there is an easy way that I simply don't know of and someone will correct me) - implemen

Re: Use self contained tokens instead of ACL

Hi, We have a accepted KIP for adding delegation token support for Kafka. https://cwiki.apache.org/confluence/display/KAFKA/KIP-48+Delegation+token+support+for+Kafka currently the PR is under review process. Maybe this can used be as starting point for your requirement. https://github.com/apache

Re: Re: [DISCUSS] KIP-213 Support non-key joining in KTable

I think if you explain what A and B are in the beginning, it makes sense to use them since readers would know who they reference. Cheers On Thu, Oct 26, 2017 at 11:04 PM, Jan Filipiak wrote: > > > Thanks for the remarks. hope I didn't miss any. > Not even sure if it makes sense to introduce A a

[GitHub] kafka pull request #4143: MINOR: PartitionReassignmentHandler should generat...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/4143 MINOR: PartitionReassignmentHandler should generate event only when znode is created and eliminate unnecessary allocations As suggested by Jun in https://github.com/apache/kafka/pull/4141, we only

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

I am personally still not convinced, that we should add `commit()` at all. @Guozhang: you created the original Jira. Can you elaborate a little bit? Isn't requesting commits a low level API that should not be exposed in the DSL? Just want to understand the motivation better. Why would anybody that

Re: [DISCUSS] KIP-215: Add topic regex support for Connect sinks

I added a note in the KIP about ConfigException being thrown. I also changed the proposed default for the new config to empty string rather than null. Absent a clear definition of what "common" regex syntax is, it seems an undue burden to ask the user to guess at what Pattern features are safe. If

[jira] [Created] (KAFKA-6136) Transient test failure: SaslPlainSslEndToEndAuthorizationTest.testTwoConsumersWithDifferentSaslCredentials

Ismael Juma created KAFKA-6136: -- Summary: Transient test failure: SaslPlainSslEndToEndAuthorizationTest.testTwoConsumersWithDifferentSaslCredentials Key: KAFKA-6136 URL: https://issues.apache.org/jira/browse/KAFKA-61

[GitHub] kafka pull request #4144: MINOR: Ensure that the producer in testAlterReplic...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/4144 MINOR: Ensure that the producer in testAlterReplicaLogDirs is always closed Failure to close the producer could cause a transient failure, more details below. The request timeout was only

[GitHub] kafka pull request #4139: MINOR: Indentation Fix

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4139 ---

[GitHub] kafka pull request #4119: MINOR: added -1 value description as "high waterma...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4119 ---

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

Hi Matthias, Thanks for your comments. I agree that this is not the best way to do. A bit of history behind this design. Prior doing this, I tried to provide ProcessorContext itself as an argument in Rich interfaces. However, we dont want to give users that flexibility and “power”. Moreover, Proc

[GitHub] kafka pull request #3835: MINOR: update operations doc on topic deletion

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3835 ---

Build failed in Jenkins: kafka-trunk-jdk9 #156

See Changes: [ismael] MINOR: Fix indentation in KafkaApis.handleOffsetFetchRequest [ismael] MINOR: Document "high watermark" magic value for delete records request [ismael] MINOR: Update docs wrt topic deletion

Build failed in Jenkins: kafka-trunk-jdk8 #2174

See Changes: [ismael] MINOR: Fix indentation in KafkaApis.handleOffsetFetchRequest -- [...truncated 386.48 KB...] kafka.server.epoch.LeaderEpochFileCacheTest > shouldNot

[GitHub] kafka pull request #3951: MINOR: update listener.security.protocol.map confi...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3951 ---

Build failed in Jenkins: kafka-1.0-jdk7 #62

See Changes: [ismael] MINOR: Document "high watermark" magic value for delete records request -- [...truncated 1.82 MB...] org.apache.kafka.streams.integration.KTableKTableJo

[jira] [Resolved] (KAFKA-6133) NullPointerException in S3 Connector when using rotate.interval.ms

[ https://issues.apache.org/jira/browse/KAFKA-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-6133. -- Resolution: Invalid Closing this as INVALID, since this is not Apache Kafka code. These connect

Build failed in Jenkins: kafka-trunk-jdk8 #2175

See Changes: [ismael] MINOR: Document "high watermark" magic value for delete records request [ismael] MINOR: Update docs wrt topic deletion being enabled by default -- [

[GitHub] kafka pull request #4144: MINOR: Ensure that the producer in testAlterReplic...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4144 ---

Re: Use self contained tokens instead of ACL

Hi Manikumar, I've looked over the KIP and had a quick look at the code in the PR as well. In principle I think this would help Peter along depending on how plugable some of the components are. Since Peter wants to generate Tokens not in Kafka but in an external System the entire part in Kafka of

Re: [DISCUSS] KIP-214: Add zookeeper.max.in.flight.requests config to the broker

Seems sensible to me. Ismael On Thu, Oct 26, 2017 at 6:37 AM, Onur Karaman wrote: > Hey everyone. > > Giving this another shot since it looks like there was a KIP number > collision on the wiki page. > > I made a config kip, KIP-214: Add zookeeper.max.in.flight.requests config > to the broker:

[GitHub] kafka pull request #4145: KAFKA-4928: Add integration test for DumpLogSegmen...

GitHub user Sammers21 opened a pull request: https://github.com/apache/kafka/pull/4145 KAFKA-4928: Add integration test for DumpLogSegments Adding tests for `kafka.tools.DumpLogSegments` You can merge this pull request into a Git repository by running: $ git pull https://github

Re: [DISCUSS]: KIP-159: Introducing Rich functions to Streams

Thanks. I understand what you are saying, but I don't agree that > but also we need a commit() method I would just not provide `commit()` at DSL level and close the corresponding Jira as "not a problem" or similar. -Matthias On 10/27/17 3:42 PM, Jeyhun Karimov wrote: > Hi Matthias, > > Thanks

[jira] [Created] (KAFKA-6137) RestoreIntegrationTest sometimes fails with assertion error

Ted Yu created KAFKA-6137: - Summary: RestoreIntegrationTest sometimes fails with assertion error Key: KAFKA-6137 URL: https://issues.apache.org/jira/browse/KAFKA-6137 Project: Kafka Issue Type: Test

[jira] [Resolved] (KAFKA-6109) ResetIntegrationTest may fail due to IllegalArgumentException

[ https://issues.apache.org/jira/browse/KAFKA-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved KAFKA-6109. --- Resolution: Cannot Reproduce > ResetIntegrationTest may fail due to IllegalArgumentException > ---

Jenkins build is back to normal : kafka-1.0-jdk7 #63

See

Errors when running integration/kafka/tools/MirrorMakerIntegrationTest

Hi, I see some error logs when running integration/kafka/tools/MirrorMakerIntegrationTest. Log: [2017-10-27 18:28:02,557] ERROR ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes (org.apache.zookeeper.server.ZooKeeperServer:47

Jenkins build is back to normal : kafka-trunk-jdk9 #157

See

[jira] [Created] (KAFKA-6138) Simplify StreamsBuilder#addGlobalStore

Matthias J. Sax created KAFKA-6138: -- Summary: Simplify StreamsBuilder#addGlobalStore Key: KAFKA-6138 URL: https://issues.apache.org/jira/browse/KAFKA-6138 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #4034: MINOR: Remove TLS renegotiation code

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/4034 ---

Re: Errors when running integration/kafka/tools/MirrorMakerIntegrationTest

The error log appears in other tests as well. I ran MirrorMakerIntegrationTest on trunk just now - it passed. I didn't see the test failing in any recent Jenkins builds. There're flaky tests, such as KAFKA-6137 FYI On Fri, Oct 27, 2017 at 8:35 AM, Pavel Drankov wrote: > Hi, > > I see some er

Re: Errors when running integration/kafka/tools/MirrorMakerIntegrationTest

Of course it don't fail tests. I mean to fix log appearance. Best wishes, Pavel On 27 October 2017 at 18:48, Ted Yu wrote: > The error log appears in other tests as well. > > I ran MirrorMakerIntegrationTest on trunk just now - it passed. > I didn't see the test failing in any recent Jenkins bu

Re: Errors when running integration/kafka/tools/MirrorMakerIntegrationTest

Yes, I think it would be good to eliminate these lines from the output. A PR is welcome if you're interested. Ismael On Fri, Oct 27, 2017 at 5:03 PM, Pavel Drankov wrote: > Of course it don't fail tests. I mean to fix log appearance. > > Best wishes, > Pavel > > On 27 October 2017 at 18:48, Ted

Re: Errors when running integration/kafka/tools/MirrorMakerIntegrationTest

Yes, I m interested and would like to make a PR. Best wishes, Pavel On 27 October 2017 at 19:05, Ismael Juma wrote: > Yes, I think it would be good to eliminate these lines from the output. A > PR is welcome if you're interested. > > Ismael > > On Fri, Oct 27, 2017 at 5:03 PM, Pavel Drankov >

Jenkins build is back to normal : kafka-trunk-jdk7 #2930

See

[GitHub] kafka pull request #4146: MINOR: Tighten up locking when aborting expired tr...

GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/4146 MINOR: Tighten up locking when aborting expired transactions This is a followup to #4137 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apurvam

Jenkins build is back to normal : kafka-trunk-jdk8 #2176

See

[VOTE] KIP-214: Add zookeeper.max.in.flight.requests config to the broker

I'd like to start the vote for KIP-214: Add zookeeper.max.in.flight.requests config to the broker https://cwiki.apache.org/confluence/display/KAFKA/KIP-214%3A+Add+zookeeper.max.in.flight.requests+config+to+the+broker - Onur

Re: [VOTE] KIP-214: Add zookeeper.max.in.flight.requests config to the broker

+1 On Fri, Oct 27, 2017 at 10:15 AM, Onur Karaman wrote: > I'd like to start the vote for KIP-214: Add > zookeeper.max.in.flight.requests config to the broker > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 214%3A+Add+zookeeper.max.in.flight.requests+config+to+the+broker > > - Onur

Build failed in Jenkins: kafka-1.0-jdk7 #64

See Changes: [ismael] MINOR: Mention "per listener" security overrides in -- [...truncated 1.82 MB...] org.apache.kafka.streams.KafkaStreamsTest > testStateGlobalThreadClose P

[VOTE] 1.0.0 RC4

Hello Kafka users, developers and client-developers, This is the fifth candidate for release of Apache Kafka 1.0.0. The main PRs that gets merged in after RC3 are the following: *https://github.com/apache/kafka/commit/def1a768a6301c14ad6611358716ab03de04e76b

Re: [DISCUSS] KIP-209 Connection String Support

On Tue, Oct 24, 2017, at 22:51, Michael André Pearce wrote: > Fair enough on URL encoding but as mentioned it is important to be able > to escape, I agree with backslash option. > > I would still like some form of prefix to the string to denote it is for > kafka. I don't think a prefix is necessa

Re: [DISCUSS] KIP-212: Enforce set of legal characters for connector names

On Wed, Oct 25, 2017, at 01:07, Sönke Liebau wrote: > I've spent some time looking at this and testing various characters and > it > would appear that Randall's suspicion was spot on. I think we can support > a > fairly large set of characters with very minor changes. > > I was put of by the excep

Re: [VOTE] KIP-207:The Offsets which ListOffsetsResponse returns should monotonically increase even during a partition leader change

Thanks, everyone. I'm going to close the vote tomorrow if there are no more comments or votes. regards, Colin On Thu, Oct 26, 2017, at 08:09, Manikumar wrote: > Thanks for the KIP. > +1 (non-binding) > > > On Thu, Oct 26, 2017 at 5:58 AM, Jason Gustafson > wrote: > > > +1. Thanks for the KI

Build failed in Jenkins: kafka-trunk-jdk8 #2177

See Changes: [ismael] MINOR: Ensure that the producer in testAlterReplicaLogDirs is always [ismael] MINOR: Remove TLS renegotiation code -- [...truncated 1.88 MB...] org

[GitHub] kafka-site issue #104: Replace link

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/104 LGTM. Could you submit a PR for changes in ` 0110/streams` as well in `kafka` repo? ---

[GitHub] kafka-site issue #104: Replace link

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/104 Merged to asf-site. ---

[GitHub] kafka-site pull request #104: Replace link

Github user asfgit closed the pull request at: https://github.com/apache/kafka-site/pull/104 ---

Build failed in Jenkins: kafka-0.11.0-jdk7 #330

See Changes: [wangguoz] KAFKA-6119: Bump epoch when expiring transactions in the -- [...truncated 7.52 KB...] if (!Console.readLine().equalsIgnoreCase("y")) {

[GitHub] kafka pull request #4147: MINOR: Fix inconsistency in StopReplica/LeaderAndI...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/4147 MINOR: Fix inconsistency in StopReplica/LeaderAndIsr error counts You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka fix-error-in

[GitHub] kafka pull request #4148: KAFKA-6120: RecordCollector should not retry sendi...

GitHub user mjsax opened a pull request: https://github.com/apache/kafka/pull/4148 KAFKA-6120: RecordCollector should not retry sending You can merge this pull request into a Git repository by running: $ git pull https://github.com/mjsax/kafka kafka-6120-recordCollector Alter

Call for Paper: Kafka Summit London 2018

Hi Kafka Users and Contributors, We are having the first ever Kafka Summit that is really close to Europe even if not quite part of Europe. Call for papers is now open and you should all submit abstracts and tell your colleagues to submit abstracts too! Seriously, if you use Kafka, whether beginn

Producer performance is awful when acks=all

Hey all, I'm currently running Kafka 0.10.1 on Windows and am doing some perf testing. I tried out the perf test setup described in this blog post: https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines. When I try "single producer thread,

Re: [DISCUSS] KIP-212: Enforce set of legal characters for connector names

Yes, connector names should not be blank or contain just whitespace. In fact, I might recommend that we trim whitespace at the front and rear of new connector names and then disallowing any zero-length name. Existing connectors would remain valid, and this would not break backward compatibility. Th

[DISCUSS] KIP-217: Expose a timeout to allow an expired ZK session to be re-created

Hi, Everyone, We created "KIP-217: Expose a timeout to allow an expired ZK session to be re-created". https://cwiki.apache.org/confluence/display/KAFKA/KIP-217%3A+Expose+a+timeout+to+allow+an+expired+ZK+session+to+be+re-created Please take a look and provide your feedback. Thanks, Jun

Re: Producer performance is awful when acks=all

Hi Vijay, It is surprising to see such a large drop. The first thing that it suggests to me is that perhaps there is a network bottleneck that is being glossed over in the acks=1 case, but not in the acks=all case. In the acks=1 case, is the broker co-located on the same machine as the producer,

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

> Please correct me if I'm wrong, but my understanding is that the record > metadata is always null if an exception occurred while trying to produce. That is right. Thanks. I looked at the example code, and one thing I realized that since we are not passing the context in the handle function, we

[GitHub] kafka pull request #4149: KAFKA-6126: Reduce rebalance time by not checking ...

GitHub user jeyhunkarimov opened a pull request: https://github.com/apache/kafka/pull/4149 KAFKA-6126: Reduce rebalance time by not checking if created topics are available You can merge this pull request into a Git repository by running: $ git pull https://github.com/jeyhunk

Re: [DISCUSS] KIP-210: Provide for custom error handling when Kafka Streams fails to produce

I can’t think of a reason that would be problematic. Most of the time I would write a handler like this, I either want to ignore the error or fail and bring everything down so that I can spin it back up later and resume from earlier offsets. When we start up after crashing we’ll eventually try to

Jenkins build is back to normal : kafka-0.11.0-jdk7 #331

See

[GitHub] kafka pull request #4150: MINOR: Add valid values for message.timestamp.type

GitHub user makearl opened a pull request: https://github.com/apache/kafka/pull/4150 MINOR: Add valid values for message.timestamp.type The documentation for `message.timestamp.type` is missing valid values (https://kafka.apache.org/documentation/#topicconfigs). This change adds val

[GitHub] kafka pull request #4151: [WIP] KIP-215: Add topic regex support for Connect...

GitHub user jklukas opened a pull request: https://github.com/apache/kafka/pull/4151 [WIP] KIP-215: Add topic regex support for Connect sinks I'm starting this WIP PR to prove out the concept of the KIP and identify some additional public interface changes that I had not previously

Re: [DISCUSS] KIP-215: Add topic regex support for Connect sinks

I've updated the KIP to use the topics.regex name and opened a WIP PR with an implementation that shows some additional complexity in how the configuration option gets passed through, affecting various public function signatures. I would appreciate any eyes on that for feedback on whether more des

[jira] [Created] (KAFKA-6139) error when loading plugins

Eugene Burd created KAFKA-6139: -- Summary: error when loading plugins Key: KAFKA-6139 URL: https://issues.apache.org/jira/browse/KAFKA-6139 Project: Kafka Issue Type: Bug Components: Ka

[jira] [Resolved] (KAFKA-6112) SSL + ACL does not seem to work

[ https://issues.apache.org/jira/browse/KAFKA-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sönke Liebau resolved KAFKA-6112. - Resolution: Cannot Reproduce As stated in earlier comment this is most probably a configuration is