[jira] [Created] (KAFKA-8785) Flakey test LeaderElectionCommandTest#testPathToJsonFile

Boyang Chen created KAFKA-8785: -- Summary: Flakey test LeaderElectionCommandTest#testPathToJsonFile Key: KAFKA-8785 URL: https://issues.apache.org/jira/browse/KAFKA-8785 Project: Kafka Issue Type

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

Hi Rajini 3.2 - This makes sense. Thanks for clarifying. Rest looks fine. Once the implementations are done, it will be more clear on the different values RequestType and Mode. Thanks Bosco On 8/9/19, 5:08 AM, "Rajini Sivaram" wrote: Hi Don, Thanks for the suggestions. A few

Jenkins build is back to normal : kafka-trunk-jdk11 #744

See

Build failed in Jenkins: kafka-trunk-jdk8 #3841

See Changes: [github] MINOR: Bump system test version from 2.2.0 to 2.2.1 (#6873) -- [...truncated 2.59 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldN

[jira] [Created] (KAFKA-8784) Remove default implementation of RocksDBConfigSetter#close

Sophie Blee-Goldman created KAFKA-8784: -- Summary: Remove default implementation of RocksDBConfigSetter#close Key: KAFKA-8784 URL: https://issues.apache.org/jira/browse/KAFKA-8784 Project: Kafka

[jira] [Created] (KAFKA-8783) SaslOAuthBearerSslEndToEndAuthorizationTest.testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

Matthias J. Sax created KAFKA-8783: -- Summary: SaslOAuthBearerSslEndToEndAuthorizationTest.testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl Key: KAFKA-8783 URL: https://issues.apache.org/jira/browse/KAFKA-878

Build failed in Jenkins: kafka-trunk-jdk8 #3840

See Changes: [ismael] MINOR: Ignore dynamic log4j log level tests (#7183) -- [...truncated 2.62 MB...] org.apache.kafka.connect.json.JsonConverterTest > mapToJsonNonStrin

Re: [DISCUSS] KIP-482: The Kafka Protocol should Support Optional Fields

Hi all, I've made some updates to this KIP. Specifically, I wanted to avoid including escape bytes in the serialization format, since it was too complex. Also, I think this is a good opportunity to slim down our variable length fields. best, Colin On Thu, Jul 11, 2019, at 20:52, Colin McCab

[jira] [Created] (KAFKA-8782) ReplicationQuotaManagerTest and ClientQuotaManagerTest should close Metrics object

Anna Povzner created KAFKA-8782: --- Summary: ReplicationQuotaManagerTest and ClientQuotaManagerTest should close Metrics object Key: KAFKA-8782 URL: https://issues.apache.org/jira/browse/KAFKA-8782 Projec

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (binding) cheers, Colin On Fri, Aug 9, 2019, at 09:56, Ron Dagostino wrote: > +1 (non-binding) > > The simplest of KIPs, with perhaps the biggest impact. Like removing > the thorn from the soles of my feet. > > Thanks for doing it. > > > On Aug 9, 2019, at 12:50 PM, Dongjin Lee wrote: >

Re: Alternative of poll(0) without pulling records

Hi Gabor, What is it that you want to do here? If you just want to check that the partitions exist, but not fetch any data, you could use AdminClient#describeTopics for that. If you want to create the topics, you could use AdminClient#createTopics. best, Colin On Fri, Aug 9, 2019, at 11:23

Build failed in Jenkins: kafka-trunk-jdk11 #743

See Changes: [ismael] MINOR: Ignore dynamic log4j log level tests (#7183) -- [...truncated 2.59 MB...] org.apache.kafka.common.network.SelectorTest > testExistingConnecti

Build failed in Jenkins: kafka-trunk-jdk8 #3839

See Changes: [github] KAFKA-8748: Fix flaky testDescribeLogDirsRequest (#7182) -- [...truncated 2.60 MB...] kafka.zk.LiteralAclStoreTest > shouldThrowFromEncodeOnNoneLite

[jira] [Created] (KAFKA-8781) Flaky Test TopicCommandWithAdminClientTest#testDescribeReportOverriddenConfigs

Matthias J. Sax created KAFKA-8781: -- Summary: Flaky Test TopicCommandWithAdminClientTest#testDescribeReportOverriddenConfigs Key: KAFKA-8781 URL: https://issues.apache.org/jira/browse/KAFKA-8781 Proj

Jenkins build is back to normal : kafka-trunk-jdk11 #742

See

Re: Alternative of poll(0) without pulling records

> Each KafkaConsumer method that returns metadata will already block until such metadata is available The old API was waiting infinitely but the new has a timeout which has effect on the metadata fetch as well. Spark is interested in only the assigned partitions and/or latest/earliest/... offsets.

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

Hi all, A few points: 1. I think the way backwards compatibility is being used here is not correct. Any functionality that is only enabled if set via a config is backwards compatible. People may disagree with the functionality or the config, but it's not a backwards compatibility issue. 2. It's

Re: [DISCUSS] KIP-487: Automatic Topic Creation on Producer

On Thu, Aug 8, 2019, at 11:51, Jun Rao wrote: > Hi, Justine, > > Thanks for the KIP. Overall, it seems to be a good improvement. > > However, I think Harsha's point seems reasonable. We had > auto.create.topics.enable config on the broker to allow admins to disable > topic creation from the produ

Re: Alternative of poll(0) without pulling records

> pull some records even they're only interested in metadata. Jungtaek, what is the use-case here? Each KafkaConsumer method that returns metadata will already block until such metadata is available... so why would you need to apply this "hack" in the first place? Ryanne On Wed, Aug 7, 2019 at 2

Re: [DISCUSS]KIP-216: IQ should throw different exceptions for different errors

My bad. The short link `https://shorturl.at/CDNT9` seems incorrect. Please use the following instead: https://shorturl.at/bkKQU --- Vito On Fri, Aug 9, 2019 at 10:53 AM Vito Jeng wrote: > Thanks, Matthias! > > > About `StreamThreadNotStartedException`: > > Thank y

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Hi Mickael, Thanks for taking a look. I don't think we want to support that kind of multi-tenancy at the controller level. If the cluster is small enough that we want to pack the controller(s) with something else, we could run them alongside the brokers, or possibly inside three of the broker

Re: [DISCUSS] KIP-505 : Add new public method to only update assignment metadata in consumer

Thanks for the KIP. Can you elaborate a little bit more on the use case for this feature? Why would a consumer need to update it's metadata explicitly? -Matthias On 8/8/19 8:46 PM, Jungtaek Lim wrote: > Hi devs, > > I'd like to initiate discussion around KIP-505, exposing new public method > t

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding) The simplest of KIPs, with perhaps the biggest impact. Like removing the thorn from the soles of my feet. Thanks for doing it. > On Aug 9, 2019, at 12:50 PM, Dongjin Lee wrote: > > +1 (non-binding) > > Two binding +1 (Gwen, Harsha) with Six non-binding +1 until now. > We ne

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding) Two binding +1 (Gwen, Harsha) with Six non-binding +1 until now. We need one more binding +1. Thanks, Dongjin On Sat, 10 Aug 2019 at 12:59 AM Tom Bentley wrote: > +1 (non-binding). Thanks! > > On Fri, Aug 9, 2019 at 12:37 PM Satish Duggana > wrote: > > > +1 (non-binding) Than

Re: [DISCUSS] KIP-499 - Unify connection name flag for command line tool

@Jason I see. Thanks for the context. @Colin Sure. I also agree limiting the scope of the KIP. Thanks, Dongjin On Fri, 9 Aug 2019 at 6:45 AM Colin McCabe wrote: > I agree that limiting the scope of the KIP would be good. > > The configuration is actually bootstrap.servers with an S, though. >

Re: [VOTE] KIP-497: Add inter-broker API to alter ISR

Hey Jason, +1 from me too. One note though: since it's a new protocol we could perhaps rename CurrentZkVersion to something like "IsrEpoch" or "IsrVersion". I think that'd reflect its purpose better. Best, Viktor On Wed, Aug 7, 2019 at 8:37 PM Jason Gustafson wrote: > Hi All, > > I'd like to s

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding). Thanks! On Fri, Aug 9, 2019 at 12:37 PM Satish Duggana wrote: > +1 (non-binding) Thanks for the KIP, so useful. > > On Fri, Aug 9, 2019 at 4:42 PM Mickael Maison > wrote: > > > +1 (non binding) > > Thanks for the KIP! > > > > On Fri, Aug 9, 2019 at 9:36 AM Andrew Schofield > >

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

Good catches! Fixed :) On Thu, Aug 8, 2019 at 10:36 PM Arjun Satish wrote: > Cool! > > Couple of nits: > > - In public interfaces, typo: *json.decimal.serialization.fromat* > - In public interfaces, you use the term "HEX" instead of "BASE64". > > > > On Wed, Aug 7, 2019 at 9:51 AM Almog Gavra w

Re: [DISCUSS] KIP-435: Internal Partition Reassignment Batching

Hey Stanislav, I reiterated on the current algorithm and indeed it would change the order of replicas in ZK but wouldn't do a leader election, so one would need to run the preferred replica election tool. However still in the light of this I'm not sure I'd keep the existing behavior as users would

[jira] [Created] (KAFKA-8780) Set SCRAM passwords via the Admin interface

Tom Bentley created KAFKA-8780: -- Summary: Set SCRAM passwords via the Admin interface Key: KAFKA-8780 URL: https://issues.apache.org/jira/browse/KAFKA-8780 Project: Kafka Issue Type: New Feature

Re: [VOTE] KIP-396: Add Commit/List Offsets Operations to AdminClient

Hi Jason, 4. Yes, I've added that in the KIP 5. I've updated the KIP to use `alterConsumerGroupOffsets` so it's consistent with other APIs Thank you for the feedback On Wed, Aug 7, 2019 at 7:08 PM Jason Gustafson wrote: > > Thanks Mickael, +1 from me. > > Just a couple more comments: > > > Yes

Re: [DISCUSS] KIP-373: Allow users to create delegation tokens for other users

Hey Manikumar, Thanks for the feedback. I'm not sure I fully grasp the use-case. Would this be a quota? Do we say something like "there can be 10 active delegation tokens at a time that is created by superuserA for other users"? I think such a feature could be useful to limit the responsibility of

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

Hi Colin, Thanks for the reviewing the KIP and the suggestions to improve. *1. Why not just pass the cluster ID directly to Authorizer#start* I was thinking that we already have APIs that use ClusterResourceListener, so kept it consistent. But you are right, since we have a start method, we coul

Re: [DISCUSS] KIP-504 - Add new Java Authorizer Interface

Hi Don, Thanks for the suggestions. A few responses below: 3.1 Can rename and improve docs if we keep this. Let's finish the discussion on Colin's suggestions regarding this first. 3.2 No, I was thinking of some requests that have an old way of authorizing and a new way where we have retained the

Build failed in Jenkins: kafka-trunk-jdk8 #3838

See Changes: [manikumar] KAFKA-8598: Use automatic RPC generation in RenewDelegationToken -- [...truncated 2.58 MB...] org.apache.kafka.connect.data.DecimalTest > testToLo

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding) Thanks for the KIP, so useful. On Fri, Aug 9, 2019 at 4:42 PM Mickael Maison wrote: > +1 (non binding) > Thanks for the KIP! > > On Fri, Aug 9, 2019 at 9:36 AM Andrew Schofield > wrote: > > > > +1 (non-binding) > > > > On 09/08/2019, 08:39, "Sönke Liebau" > wrote: > > > >

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non binding) Thanks for the KIP! On Fri, Aug 9, 2019 at 9:36 AM Andrew Schofield wrote: > > +1 (non-binding) > > On 09/08/2019, 08:39, "Sönke Liebau" wrote: > > +1 (non-binding) > > > > On Fri, 9 Aug 2019 at 04:45, Harsha Chintalapani wrote: > > > +1 (binding). much needed!! >

Build failed in Jenkins: kafka-trunk-jdk11 #741

See Changes: [manikumar] KAFKA-8598: Use automatic RPC generation in RenewDelegationToken -- [...truncated 2.59 MB...] org.apache.kafka.connect.transforms.SetSchemaMetada

[jira] [Created] (KAFKA-8779) Fix flaky tests introduced by dynamic log levels

Stanislav Kozlovski created KAFKA-8779: -- Summary: Fix flaky tests introduced by dynamic log levels Key: KAFKA-8779 URL: https://issues.apache.org/jira/browse/KAFKA-8779 Project: Kafka Is

[jira] [Created] (KAFKA-8778) Trogdor - do not expose the full spec JSON in list endpoint (/coordinator/tasks)

Stanislav Kozlovski created KAFKA-8778: -- Summary: Trogdor - do not expose the full spec JSON in list endpoint (/coordinator/tasks) Key: KAFKA-8778 URL: https://issues.apache.org/jira/browse/KAFKA-8778

[jira] [Created] (KAFKA-8777) Trogdor - Add GET /coordinator/tasks/ids endpoint

Stanislav Kozlovski created KAFKA-8777: -- Summary: Trogdor - Add GET /coordinator/tasks/ids endpoint Key: KAFKA-8777 URL: https://issues.apache.org/jira/browse/KAFKA-8777 Project: Kafka I

Re: [DISCUSS] KIP-317: Transparent Data Encryption

Hi Andrew, thanks for your feedback! I am interested though, why are you doubtful about getting a committer to volunteer an opinion? Shouldn't this be in their interest as well? I'll just continue along for now and start building a very rough poc implementation based on what's in the KIP so far t

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding) On 09/08/2019, 08:39, "Sönke Liebau" wrote: +1 (non-binding) On Fri, 9 Aug 2019 at 04:45, Harsha Chintalapani wrote: > +1 (binding). much needed!! > > > On Thu, Aug 08, 2019 at 6:43 PM, Gwen Shapira wrote: > > > +1 (binding

Re: [VOTE] KIP-499 - Unify connection name flag for command line tool

+1 (non-binding) On Fri, 9 Aug 2019 at 04:45, Harsha Chintalapani wrote: > +1 (binding). much needed!! > > > On Thu, Aug 08, 2019 at 6:43 PM, Gwen Shapira wrote: > > > +1 (binding) THANK YOU. It would be +100 if I could. > > > > On Thu, Aug 8, 2019 at 6:37 PM Mitchell wrote: > > > > Hello D

[jira] [Resolved] (KAFKA-8598) Replace RenewDelegationToken request/response with automated protocol

[ https://issues.apache.org/jira/browse/KAFKA-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-8598. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 7038 [https://github.com/a