Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Instead of keep diverging in different directions, it would be helpful if you guys take my detailed posts with 1st to 4th points I mentioned and start referring/commenting on each of those if you agree with them or not. On Mon, Aug 19, 2019 at 10:45 PM Maulin Vasavada wrote: > Hi Colin > > When

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Hi Colin When I refer to "standard" or "custom" algorithms I am following Java security Provider Terminology. You can refer to https://docs.oracle.com/javase/7/docs/technotes/guides/security/StandardNames.html#TrustManagerFactory link I provided earlier in the emails. It says PKIX is the default

Build failed in Jenkins: kafka-trunk-jdk11 #766

See Changes: [me] MINOR: Upgrade ducktape to 0.7.6 -- [...truncated 2.61 MB...] org.apache.kafka.streams.MockProcessorContextTest > shouldStoreAndReturnStateStores

[jira] [Created] (KAFKA-8820) Use Admin API of Replica Reassignment in CLI tools

Gwen Shapira created KAFKA-8820: --- Summary: Use Admin API of Replica Reassignment in CLI tools Key: KAFKA-8820 URL: https://issues.apache.org/jira/browse/KAFKA-8820 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks, Konstantine. Updated the KIP with the restrictions around log4j and added references to similar KIPs. Best, On Mon, Aug 19, 2019 at 3:20 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > Thanks Arjun, the example is useful! > > My point when I mentioned the restrictions

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

Hi Bruno, Just realized that for `addRateSensor` and `addLatencyAndRateSensor` we've actually added the total invocation metric already. Guozhang On Mon, Aug 19, 2019 at 4:11 PM Guozhang Wang wrote: > Hi Bruno, > > > On Tue, Aug 6, 2019 at 1:51 AM Bruno Cadonna wrote: > >> Hi Guozhang, >>

Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams

Hi Bruno, On Tue, Aug 6, 2019 at 1:51 AM Bruno Cadonna wrote: > Hi Guozhang, > > I left my comments inline. > > On Thu, Jul 18, 2019 at 8:28 PM Guozhang Wang wrote: > > > > Hello Bruno, > > > > Thanks for the feedbacks, replied inline. > > > > On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Hi all, The KIP has been out for a while, so I'm thinking about calling a vote some time this week. best, Colin On Mon, Aug 19, 2019, at 15:52, Colin McCabe wrote: > On Mon, Aug 19, 2019, at 12:52, David Arthur wrote: > > Thanks for the KIP, Colin. This looks great! > > > > I really like the

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

On Mon, Aug 19, 2019, at 12:52, David Arthur wrote: > Thanks for the KIP, Colin. This looks great! > > I really like the idea of separating the Controller and Broker JVMs. > > As you alluded to above, it might be nice to have a separate > broker-registration API to avoid overloading the metadata

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

Thanks Arjun, the example is useful! My point when I mentioned the restrictions around log4j is that this is information is significant and IMO needs to be included in the KIP. Speaking of its relevance to KIP-412, I think a reference would be nice too. Konstantine On Thu, Aug 15, 2019 at

[jira] [Created] (KAFKA-8819) Plugin path for converters not working as intended

Magesh kumar Nandakumar created KAFKA-8819: -- Summary: Plugin path for converters not working as intended Key: KAFKA-8819 URL: https://issues.apache.org/jira/browse/KAFKA-8819 Project: Kafka

Build failed in Jenkins: kafka-trunk-jdk11 #765

See -- [...truncated 2.09 MB...] org.apache.kafka.streams.kstream.internals.KTableImplTest > shouldThrowNullPointerOnTransformValuesWithKeyWhenMaterializedIsNull PASSED

[DISCUSS] KIP-509: Rebalance and restart Producers

At the moment producers do not get any support when restarting: where should it start producing data from? what data had been committed successfully in the past? and also for load distribution there is no support in the Kafka client itself. how many producers for a single source are currently

Re: [DISCUSS] KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum

Thanks for the KIP, Colin. This looks great! I really like the idea of separating the Controller and Broker JVMs. As you alluded to above, it might be nice to have a separate broker-registration API to avoid overloading the metadata fetch API. When a broker gets a metadata delta, will it be a

[jira] [Created] (KAFKA-8818) CreatePartitions Request protocol documentation

Fábio Silva created KAFKA-8818: -- Summary: CreatePartitions Request protocol documentation Key: KAFKA-8818 URL: https://issues.apache.org/jira/browse/KAFKA-8818 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

Hi Ryanne, The reasoning for this is provided in the KIP: "There would be no clear way to achieve consensus amongst the workers in a cluster on whether to switch to this new behavior." To elaborate on this--it would be bad if a follower worker began writing task configurations to the topic while

ACL for group creation?

Hi All I am looking through the Confluent docs and core Kafka docs and don't see an ACL for group creation: https://docs.confluent.io/current/kafka/authorization.html#acl-format and https://kafka.apache.org/documentation/#security_authz My scenario is simple: We use the consumer group as the

Re: Fwd: Warning from dev@kafka.apache.org

It seem the ezmlm bot was not able to deliver some emails from the mailing list to you. It's unclear what the root cause is though. Ezmlm sends out this notification email from below to you to make you aware that you may have missed some emails. Check the archive to read those emails. Otherwise

[jira] [Created] (KAFKA-8817) Flaky Test KafkaProducerTest.testCloseIsForcedOnPendingAddOffsetRequest

Chris Pettitt created KAFKA-8817: Summary: Flaky Test KafkaProducerTest.testCloseIsForcedOnPendingAddOffsetRequest Key: KAFKA-8817 URL: https://issues.apache.org/jira/browse/KAFKA-8817 Project: Kafka

Re: [DISCUSS] KIP-482: The Kafka Protocol should Support Optional Fields

Hi Satish, I wasn't originally going to propose supporting the struct type, although perhaps we could consider it. In general, supporting a struct containing an array has the same serialization issues as just supporting the array. Probably, we should just have a two-pass serialization

Re: [VOTE] KIP-503: deleted topics metric

Hello everyone, I'm going to close out the voting on this KIP. The results follow: * 3 binding +1 votes from Harsha, Manikumar, and Gwen * 5 non-binding +1 votes from Stanislov, Mickael, Robert, David Jacot, and Satish * No -1 votes Which gives us a passing vote. Thanks, everyone! -David On

Re: [DISCUSS] KIP-482: The Kafka Protocol should Support Optional Fields

Hi David, Good point. I think it makes the most sense to put the tagged fields sections at the end of the header. For consistency's sake, we can also put them at the end of the structures / requests / responses as well. I've updated the KIP. best, Colin On Mon, Aug 19, 2019, at 02:23,

Re: [DISCUSS] KIP-486 Support for pluggable KeyStore and TrustStore

Hi Maulin, A lot of JSSE providers don't implement custom algorithms. Spire is a good example of a JSSE provider that doesn't, and yet is still useful to many people. Your JSSE provider can work fine even if it doesn't implement a custom algorithm. Maybe I'm missing something, but I don't

[jira] [Created] (KAFKA-8816) RecordCollector offsets updated indirectly by StreamTask

Chris Pettitt created KAFKA-8816: Summary: RecordCollector offsets updated indirectly by StreamTask Key: KAFKA-8816 URL: https://issues.apache.org/jira/browse/KAFKA-8816 Project: Kafka Issue

Re: all tests passed but still got ‘Some checks were not successful’

I think we've been having some test runs hit the 4-hour time limit recently. I saw a few of those as well. This is just a guess, but I bet that there is a test that is hanging, and has no time limit configured. We should probably have time limits on every test. The person writing the test

Re: [VOTE] KIP-504 - Add new Java Authorizer Interface

+1 (non-binding) Thanks for the KIP, Rajini. Best, David On Sun, Aug 18, 2019 at 9:42 PM Mickael Maison wrote: > +1 (non binding) > Thanks for the KIP! > > On Sun, Aug 18, 2019 at 3:05 PM Ron Dagostino wrote: > > > > +1 (non-binding) > > > > Thanks, Rajini. > > > > Ron > > > > On Sat, Aug

Re: [DISCUSS] KIP-482: The Kafka Protocol should Support Optional Fields

Hi Colin, Thank you for the updated KIP. wrt. Request and Response Headers v1, where do you plan to put the optional fields? You mention that all the Requests and Responses will start with a tagged field buffer. Is it for that purpose? Best, David On Mon, Aug 19, 2019 at 7:27 AM Satish Duggana

Permissions to create a KIP

Hi, I would like to create a KIP with my user werner.daehn. Thanks in advance Werner