[jira] [Created] (KAFKA-7274) Incorrect subject credential used in inter-broker communication

TAO XIAO created KAFKA-7274: --- Summary: Incorrect subject credential used in inter-broker communication Key: KAFKA-7274 URL: https://issues.apache.org/jira/browse/KAFKA-7274 Project: Kafka Issue

Build failed in Jenkins: kafka-trunk-jdk8 #2883

See Changes: [jason] KAFKA-6751; Support dynamic configuration of -- [...truncated 2.48 MB...] org.apache.kafka.streams.processor.internals.ProcessorStateManagerTest >

Re: [VOTE] KIP-320: Allow fetchers to detect and handle log truncation

Hi, Jason, Thanks for the KIP. +1 from me. Jun On Wed, Aug 8, 2018 at 1:04 PM, Jason Gustafson wrote: > Hi All, > > I'd like to start a vote for KIP-320: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 320%3A+Allow+fetchers+to+detect+and+handle+log+truncation. > Thanks to everyone

[DISCUSS] KIP-349 Priorities for Source Topics

Since there are questions I changed the heading from VOTE to DISCUSS > On Aug 8, 2018, at 9:09 PM, Matt Farmer wrote: > > s it worth spelling out explicitly what the behavior is when two topics > have the same priority? I'm a bit fuzzy on how we choose what topics to > consume from right now,

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey Stanislav, Sorry, I was probably looking at an older version (I had the tab open for so long!). I have been thinking about `max.uncleanable.partitions` and wondering if it's what we really want. The main risk if the cleaner cannot clean a partition is eventually running out of disk space.

[DISCUSS] KIP-354 Time-based log compaction policy

Hi Kafka, This KIP tries to address GDPR concern to fulfill deletion request on time through time-based log compaction on a compaction enabled topic: https://cwiki.apache.org/confluence/display/KAFKA/KIP-354%3A+Time-based+log+compaction+policy Any feedback will be appreciated. Xiongqi

Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests

@Becket, I've asked for review by Jun and Joel in the vote thread. Regarding the separate thread and port, I did talk about it in the rejected alternative design 1. Please let me know if you'd like more elaboration or moving it to the motivation, etc. Thanks, Lucas On Wed, Aug 8, 2018 at 3:59

Re: [VOTE] KIP-291: Have separate queues for control requests and data requests

Hi Jun and Joel, The KIP writeup has changed by quite a bit since your +1. Can you please take another review? Thanks a lot for your time! Lucas On Tue, Jul 17, 2018 at 10:33 AM, Joel Koshy wrote: > +1 on the KIP. > > (I'm not sure we actually necessary to introduce the condition variables >

[jira] [Resolved] (KAFKA-6751) Make max.connections.per.ip.overrides a dynamic config

[ https://issues.apache.org/jira/browse/KAFKA-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-6751. Resolution: Fixed > Make max.connections.per.ip.overrides a dynamic config >

Re: [DISCUSS] KIP-258: Allow to Store Record Timestamps in RocksDB

Thanks for your input Guozhang and John. I see your point, that the upgrade API is not simple. If you don't thinks it's valuable to make generic store upgrades possible (atm), we can make the API internal, too. The impact is, that we only support a predefined set up upgrades (ie, KV to KVwithTs,

[jira] [Created] (KAFKA-7273) Converters should have access to headers.

Jeremy Custenborder created KAFKA-7273: -- Summary: Converters should have access to headers. Key: KAFKA-7273 URL: https://issues.apache.org/jira/browse/KAFKA-7273 Project: Kafka Issue

Build failed in Jenkins: kafka-2.0-jdk8 #113

See Changes: [wangguoz] KAFKA-7250: switch scala transform to TransformSupplier (#5481) -- [...truncated 885.51 KB...] kafka.zookeeper.ZooKeeperClientTest >

[jira] [Created] (KAFKA-7272) Fix ignored test in streams_upgrade_test.py: test_simple_upgrade_downgrade

John Roesler created KAFKA-7272: --- Summary: Fix ignored test in streams_upgrade_test.py: test_simple_upgrade_downgrade Key: KAFKA-7272 URL: https://issues.apache.org/jira/browse/KAFKA-7272 Project:

[jira] [Created] (KAFKA-7271) Fix ignored test in streams_upgrade_test.py: test_upgrade_downgrade_brokers

John Roesler created KAFKA-7271: --- Summary: Fix ignored test in streams_upgrade_test.py: test_upgrade_downgrade_brokers Key: KAFKA-7271 URL: https://issues.apache.org/jira/browse/KAFKA-7271 Project:

[jira] [Created] (KAFKA-7270) Add latest releases to streams_upgrade_test.py

John Roesler created KAFKA-7270: --- Summary: Add latest releases to streams_upgrade_test.py Key: KAFKA-7270 URL: https://issues.apache.org/jira/browse/KAFKA-7270 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-trunk-jdk8 #2882

See Changes: [wangguoz] KAFKA-7250: switch scala transform to TransformSupplier (#5481) -- [...truncated 874.84 KB...] kafka.security.auth.ResourceTest >

[jira] [Created] (KAFKA-7269) KStream.merge is not documented

John Roesler created KAFKA-7269: --- Summary: KStream.merge is not documented Key: KAFKA-7269 URL: https://issues.apache.org/jira/browse/KAFKA-7269 Project: Kafka Issue Type: Improvement

Re: [VOTE] KIP-353: Allow Users to Configure Multi-Streams Timestamp Synchronization Behavior

+1 non-binding On Thu, Aug 9, 2018 at 3:14 PM Matthias J. Sax wrote: > +1 (binding) > > On 8/9/18 11:57 AM, Guozhang Wang wrote: > > Hello all, > > > > I would like to start the voting processing on the following KIP, to > allow > > users control when a task can be processed based on its

Re: [VOTE] KIP-353: Allow Users to Configure Multi-Streams Timestamp Synchronization Behavior

+1 (binding) On 8/9/18 11:57 AM, Guozhang Wang wrote: > Hello all, > > I would like to start the voting processing on the following KIP, to allow > users control when a task can be processed based on its buffered records, > and how the stream time of a task be advanced. > >

Re: [DISCUSS] KIP-258: Allow to Store Record Timestamps in RocksDB

Hi Matthias, I think this KIP is looking really good. I have a few thoughts to add to the others: 1. You mentioned at one point users needing to configure `upgrade.mode="null"`. I think this was a typo and you meant to say they should remove the config. If they really have to set it to a string

Re: Kafka stream - Internal topic name and schema avro compatibility

Hi all, It definitely would be nice to reuse whatever state is still valid after a topology update, and KIP-307 is indeed likely what we have to do to solve that problem. The discuss thread for the KIP hasn't gotten a lot of traffic recently, so it might be nice if you reply to it with your

[jira] [Resolved] (KAFKA-6211) PartitionAssignmentState is protected whereas KafkaConsumerGroupService.describeGroup() returns PartitionAssignmentState Object

[ https://issues.apache.org/jira/browse/KAFKA-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6211. -- Resolution: Won't Fix Closing this in favour of AdminClient API.   > PartitionAssignmentState is

[jira] [Resolved] (KAFKA-6095) Error when displaying LAG information consumer partitions that use zookeeper

[ https://issues.apache.org/jira/browse/KAFKA-6095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6095. -- Resolution: Auto Closed Closing as scala consumer related code/tools are removed from 2.0 release.

[jira] [Resolved] (KAFKA-6432) Lookup indices may cause unnecessary page fault

[ https://issues.apache.org/jira/browse/KAFKA-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-6432. -- Resolution: Fixed Fix Version/s: 2.1.0 > Lookup indices may cause unnecessary page fault >

[VOTE] KIP-353: Allow Users to Configure Multi-Streams Timestamp Synchronization Behavior

Hello all, I would like to start the voting processing on the following KIP, to allow users control when a task can be processed based on its buffered records, and how the stream time of a task be advanced. https://cwiki.apache.org/confluence/display/KAFKA/KIP-

Re: [DISCUSS] KIP-353: Allow Users to Configure Kafka Streams Timestamp Synchronization

Thanks Matthias, will update the KIP accordingly. On Thu, Aug 9, 2018 at 11:26 AM, Matthias J. Sax wrote: > @Guozhang, I think you can start the VOTE for this KIP? I don't have any > further comments. > > One more nit: we should explicitly state, that the new config is > wall-clock time based.

Re: [DISCUSS] KIP-353: Allow Users to Configure Kafka Streams Timestamp Synchronization

I also have no comments. The KIP looks good to me. -John On Thu, Aug 9, 2018 at 1:26 PM Matthias J. Sax wrote: > @Guozhang, I think you can start the VOTE for this KIP? I don't have any > further comments. > > One more nit: we should explicitly state, that the new config is > wall-clock time

Re: [DISCUSS] KIP-353: Allow Users to Configure Kafka Streams Timestamp Synchronization

@Guozhang, I've read the KIP and I don't have any further comments in addition to what's already been discussed. Thanks, Bill On Thu, Aug 9, 2018 at 2:26 PM Matthias J. Sax wrote: > @Guozhang, I think you can start the VOTE for this KIP? I don't have any > further comments. > > One more nit:

Re: [DISCUSS] KIP-353: Allow Users to Configure Kafka Streams Timestamp Synchronization

@Guozhang, I think you can start the VOTE for this KIP? I don't have any further comments. One more nit: we should explicitly state, that the new config is wall-clock time based. -Matthias On 8/7/18 12:59 PM, Matthias J. Sax wrote: > Correct. It's not about reordering. Records will still be

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

Hi, thanks for the detailed discussion. I learned a lot about internals again :) I like the idea or a user config `member.name` and to keep `member.id` internal. Also agree with Guozhang, that reusing `client.id` might not be a good idea. To clarify the algorithm, each time we generate a new

Re: [DISCUSS] KIP-347: Enable batching in FindCoordinatorRequest

To add more context for KIP-347: https://github.com/apache/kafka/pull/5353 On Wed, Aug 8, 2018 at 3:55 PM Yishun Guan wrote: > Hi all, > > I would like to start a discussion on: > > KIP-347: Enable batching in FindCoordinatorRequest > https://cwiki.apache.org/confluence/x/CgZPBQ > > Thanks

Re: [DISCUSS] KIP-258: Allow to Store Record Timestamps in RocksDB

Hello Matthias, Thanks for the updated KIP. Some more comments: 1. The current set of proposed API is a bit too complicated, which makes the upgrade flow from user's perspective also a bit complex. I'd like to check different APIs and discuss about their needs separately: 1.a. StoreProxy:

Re: [DISCUSS] KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it

Thanks all for the suggestions. I'll work to make changes Regards, Ratish On Sat, Aug 4, 2018 at 2:20 PM Attila Sasvári wrote: > Thanks for the KIP. > > Regarding the motivation: > - KIP-113 added a new command, kafka-log-dirs.sh, to get the partitions > associated with a specific broker

Re: [DISCUSS] KIP-346 - Limit blast radius of log compaction failure

Hey Jason, 1. *10* is the default value, it says so in the KIP 2. This is a good catch. As the current implementation stands, it's not a useful metric since the thread continues to run even if all log directories are offline (although I'm not sure what the broker's behavior is in that scenario).

[jira] [Created] (KAFKA-7268) Review Handling Crypto Rules and update ECCN page if needed

Henri Yandell created KAFKA-7268: Summary: Review Handling Crypto Rules and update ECCN page if needed Key: KAFKA-7268 URL: https://issues.apache.org/jira/browse/KAFKA-7268 Project: Kafka

[jira] [Created] (KAFKA-7267) KafkaStreams Scala DSL process method should accept a ProcessorSupplier

John Roesler created KAFKA-7267: --- Summary: KafkaStreams Scala DSL process method should accept a ProcessorSupplier Key: KAFKA-7267 URL: https://issues.apache.org/jira/browse/KAFKA-7267 Project: Kafka

Re: Kafka stream - Internal topic name and schema avro compatibility

Hi Cédric 1. You should be able to rebuild your internal state after doing a full reset (Deleting all internal topics). The application will just need to consume data from the beginning of the input streams and rebuild the state accordingly. If you don't want to lose the state, or if it is too

[jira] [Created] (KAFKA-7266) Fix MetricsTest test flakiness

Stanislav Kozlovski created KAFKA-7266: -- Summary: Fix MetricsTest test flakiness Key: KAFKA-7266 URL: https://issues.apache.org/jira/browse/KAFKA-7266 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-7265) Broker and consumer metrics topic names are reported differently

Mathias Kub created KAFKA-7265: -- Summary: Broker and consumer metrics topic names are reported differently Key: KAFKA-7265 URL: https://issues.apache.org/jira/browse/KAFKA-7265 Project: Kafka

Re: [VOTE] KIP-280: Enhanced log compaction

Hi, So, after a "short" break, I've finally managed to find time to resume this KIP. Sorry to all for the delay. Continuing the conversation of the configurations being global vs  topic, I've checked this and it seems that they are only available globally. This configuration is passed to the

Re: [DISCUSS] KIP-342 Add Customizable SASL extensions to OAuthBearer authentication

Hi Ron/Stansilav, OK, let's just go with 2. I think it would be better to add a OAuth-specific extensions handler OAuthBearerExtensionsValidatorCallback that provides OAuthBearerToken. To summarise, we chose option 2 out of these four options: 1. {OAuthBearerValidatorCallback,

Build failed in Jenkins: kafka-trunk-jdk8 #2881

See Changes: [mjsax] MINOR: Fixed log in Topology Builder. (#5477) -- [...truncated 876.86 KB...] kafka.zookeeper.ZooKeeperClientTest > testExistsNonExistentZNode PASSED

Re: Kafka stream - Internal topic name and schema avro compatibility

Thanks John and Adam for your answer, After investigation, I am exactly in the case you describe John. After a modification in my toplogy, a processor KEY-SELECT get the same number of an old processor KEY-SELECT with the associated repartition topic. We use the app reset tool to clean all

Jenkins build is back to normal : kafka-trunk-jdk10 #389

See