[jira] [Created] (KAFKA-13508) Introduction of "-head" flag for describe tool for broker

[Suyash Srivastava (Jira)]

Suyash Srivastava created KAFKA-13508:
-

 Summary: Introduction of "-head" flag for describe tool for broker
 Key: KAFKA-13508
 URL: https://issues.apache.org/jira/browse/KAFKA-13508
 Project: Kafka
  Issue Type: Improvement
  Components: KafkaConnect, tools
Reporter: Suyash Srivastava


{{kafka-log-dirs.sh --bootstrap-server : --describe 
--broker-list }}

{{This command prints all log directories that results in an output of 500+ 
lines, if we only need the broker id's of the brokers that responded to check 
connectivity, the command should have an optional flag that shortens the output 
to only print broker id's, which leads to decrease in latency.}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [VOTE] KIP-805: Add range and scan query support in IQ v2

[Luke Chen]

Hi Vasiliki,

Thanks for the KIP!
It makes sense to have the range and scan query in IQv2, as in IQv1.

+1 (non-binding)

Thank you.
Luke

On Thu, Dec 2, 2021 at 5:41 AM John Roesler  wrote:

> Thanks for the KIP, Vicky!
>
> I’m +1 (binding)
>
> -John
>
> On Tue, Nov 30, 2021, at 14:51, Vasiliki Papavasileiou wrote:
> > Hello everyone,
> >
> > I would like to start a vote for KIP-805 that adds range and scan
> KeyValue
> > queries in IQ2.
> >
> > The KIP can be found here:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-805%3A+Add+range+and+scan+query+support+in+IQ+v2
> >
> > Cheers!
> > Vicky
>

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #30

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 501885 lines...]
[2021-12-06T02:46:28.339Z] > Task :raft:testClasses UP-TO-DATE
[2021-12-06T02:46:28.339Z] > Task :connect:json:testJar
[2021-12-06T02:46:28.339Z] > Task :connect:json:testSrcJar
[2021-12-06T02:46:29.282Z] > Task :metadata:compileTestJava UP-TO-DATE
[2021-12-06T02:46:29.282Z] > Task :metadata:testClasses UP-TO-DATE
[2021-12-06T02:46:29.282Z] > Task 
:clients:generateMetadataFileForMavenJavaPublication
[2021-12-06T02:46:29.282Z] > Task 
:clients:generatePomFileForMavenJavaPublication
[2021-12-06T02:46:29.282Z] 
[2021-12-06T02:46:29.282Z] > Task :streams:processMessages
[2021-12-06T02:46:29.282Z] Execution optimizations have been disabled for task 
':streams:processMessages' to ensure correctness due to the following reasons:
[2021-12-06T02:46:29.282Z]   - Gradle detected a problem with the following 
location: 
'/home/jenkins/jenkins-agent/workspace/Kafka_kafka_3.1/streams/src/generated/java/org/apache/kafka/streams/internals/generated'.
 Reason: Task ':streams:srcJar' uses this output of task 
':streams:processMessages' without declaring an explicit or implicit 
dependency. This can lead to incorrect results being produced, depending on 
what order the tasks are executed. Please refer to 
https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency
 for more details about this problem.
[2021-12-06T02:46:29.282Z] MessageGenerator: processed 1 Kafka message JSON 
files(s).
[2021-12-06T02:46:29.282Z] 
[2021-12-06T02:46:29.282Z] > Task :streams:compileJava UP-TO-DATE
[2021-12-06T02:46:29.282Z] > Task :streams:classes UP-TO-DATE
[2021-12-06T02:46:29.282Z] > Task :streams:test-utils:compileJava UP-TO-DATE
[2021-12-06T02:46:30.230Z] > Task :streams:copyDependantLibs
[2021-12-06T02:46:30.230Z] > Task :streams:jar UP-TO-DATE
[2021-12-06T02:46:30.230Z] > Task 
:streams:generateMetadataFileForMavenJavaPublication
[2021-12-06T02:46:32.907Z] > Task :connect:api:javadoc
[2021-12-06T02:46:32.907Z] > Task :connect:api:copyDependantLibs UP-TO-DATE
[2021-12-06T02:46:32.907Z] > Task :connect:api:jar UP-TO-DATE
[2021-12-06T02:46:32.907Z] > Task 
:connect:api:generateMetadataFileForMavenJavaPublication
[2021-12-06T02:46:32.907Z] > Task :connect:json:copyDependantLibs UP-TO-DATE
[2021-12-06T02:46:32.907Z] > Task :connect:json:jar UP-TO-DATE
[2021-12-06T02:46:32.907Z] > Task 
:connect:json:generateMetadataFileForMavenJavaPublication
[2021-12-06T02:46:32.907Z] > Task 
:connect:json:publishMavenJavaPublicationToMavenLocal
[2021-12-06T02:46:32.907Z] > Task :connect:json:publishToMavenLocal
[2021-12-06T02:46:32.907Z] > Task :connect:api:javadocJar
[2021-12-06T02:46:32.907Z] > Task :connect:api:compileTestJava UP-TO-DATE
[2021-12-06T02:46:32.907Z] > Task :connect:api:testClasses UP-TO-DATE
[2021-12-06T02:46:32.907Z] > Task :connect:api:testJar
[2021-12-06T02:46:32.907Z] > Task :connect:api:testSrcJar
[2021-12-06T02:46:32.907Z] > Task 
:connect:api:publishMavenJavaPublicationToMavenLocal
[2021-12-06T02:46:32.907Z] > Task :connect:api:publishToMavenLocal
[2021-12-06T02:46:36.540Z] > Task :streams:javadoc
[2021-12-06T02:46:36.540Z] > Task :streams:javadocJar
[2021-12-06T02:46:38.301Z] > Task :clients:javadoc
[2021-12-06T02:46:38.301Z] > Task :clients:javadocJar
[2021-12-06T02:46:39.242Z] 
[2021-12-06T02:46:39.242Z] > Task :clients:srcJar
[2021-12-06T02:46:39.242Z] Execution optimizations have been disabled for task 
':clients:srcJar' to ensure correctness due to the following reasons:
[2021-12-06T02:46:39.242Z]   - Gradle detected a problem with the following 
location: 
'/home/jenkins/jenkins-agent/workspace/Kafka_kafka_3.1/clients/src/generated/java'.
 Reason: Task ':clients:srcJar' uses this output of task 
':clients:processMessages' without declaring an explicit or implicit 
dependency. This can lead to incorrect results being produced, depending on 
what order the tasks are executed. Please refer to 
https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency
 for more details about this problem.
[2021-12-06T02:46:39.242Z] 
[2021-12-06T02:46:39.242Z] > Task :clients:testJar
[2021-12-06T02:46:40.184Z] > Task :clients:testSrcJar
[2021-12-06T02:46:40.184Z] > Task 
:clients:publishMavenJavaPublicationToMavenLocal
[2021-12-06T02:46:40.184Z] > Task :clients:publishToMavenLocal
[2021-12-06T02:46:59.218Z] > Task :core:compileScala
[2021-12-06T02:48:06.410Z] > Task :core:classes
[2021-12-06T02:48:06.410Z] > Task :core:compileTestJava NO-SOURCE
[2021-12-06T02:48:32.378Z] > Task :core:compileTestScala
[2021-12-06T02:49:21.031Z] > Task :core:testClasses
[2021-12-06T02:49:31.104Z] > Task :streams:compileTestJava
[2021-12-06T02:49:31.104Z] > Task :streams:testClasses
[2021-12-06T02:49:31.104Z] > Task :streams:testJar
[2021-12-06T02:49:31.104Z] > Task :streams:testSrcJar
[2021-12-06T02:49:31.104Z] > Task 
:streams:publishMavenJavaPublic

Re: [VOTE] KIP-792: Add "generation" field into consumer protocol

[Colin McCabe]

Hi Luke,

Thanks for the explanation.

I don't see any description of how the broker decides to use the new version of 
ConsumerProtocolSubscription or not. This probably needs to be locked to a new 
IBP version.

One scenario that we need to consider is what happens during a rolling upgrade. 
If the coordinator moves back and forth between brokers with different IBPs, it 
seems that the same epoch numbers could be reused for a group, if things are 
done in the obvious manner (old IBP = don't read or write epoch, new IBP = do).

best,
Colin


On Fri, Dec 3, 2021, at 18:46, Luke Chen wrote:
> Hi Colin,
> Thanks for your comment.
>
>> How are we going to avoid the situation where the broker restarts, and
> the same generation number is reused?
>
> Actually, this KIP doesn't have anything to do with the brokers. The
> "generation" field I added, is in the subscription metadata, which will not
> be deserialized by brokers. The metadata is only deserialized by consumer
> lead. And for the consumer lead, the only thing the lead cared about, is
> the highest generation of the ownedPartitions among all the consumers. With
> the highest generation of the ownedPartitions, the consumer lead can
> distribute the partitions as sticky as possible, and most importantly,
> without errors.
>
> That is, after this KIP, if the broker restarts, and the same generation
> number is reused, it won't break current rebalance behavior. But it'll help
> the consumer lead do the sticky assignments correctly.
>
> Thank you.
> Luke
>
> On Fri, Dec 3, 2021 at 6:30 AM Colin McCabe  wrote:
>
>> How are we going to avoid the situation where the broker restarts, and the
>> same generation number is reused?
>>
>> best,
>> Colin
>>
>> On Tue, Nov 30, 2021, at 16:36, Luke Chen wrote:
>> > Hi all,
>> >
>> > I'd like to start the vote for KIP-792: Add "generation" field into
>> > consumer protocol.
>> >
>> > The goal of this KIP is to allow the assignor/consumer coordinator to
>> have
>> > a way to identify the out-of-date members/assignments, to avoid rebalance
>> > stuck issues in current protocol.
>> >
>> > Detailed description can be found here:
>> >
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336614
>> >
>> > Any feedback is welcome.
>> >
>> > Thank you.
>> > Luke
>>

[jira] [Resolved] (KAFKA-13490) Fix some problems with KRaft createTopics and incrementalAlterConfigs

[Colin McCabe (Jira)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-13490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colin McCabe resolved KAFKA-13490.
--
Resolution: Fixed

> Fix some problems with KRaft createTopics and incrementalAlterConfigs
> -
>
> Key: KAFKA-13490
> URL: https://issues.apache.org/jira/browse/KAFKA-13490
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Colin McCabe
>Assignee: Colin McCabe
>Priority: Blocker
>  Labels: kip-500
> Fix For: 3.1.0
>
>
> For CreateTopics, fix a bug where if one createTopics in a batch failed, they 
> would all fail with the same error code. Make the error message for 
> TOPIC_ALREADY_EXISTS consistent with the ZK-based code by including the topic 
> name.
> For IncrementalAlterConfigs, before we allow topic configurations to be set, 
> we should check that they are valid. (This also applies to newly created 
> topics.) IncrementalAlterConfigs should ignore non-null payloads for DELETE 
> operations. Previously we would return an error in these cases. However, this 
> is not compatible with the old ZK-based code, which ignores the payload in 
> these cases.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Contributor Sign Up

[Bruno Cadonna]

Hi Anshul,

I granted you the requested permissions in Jira and wiki.

Looking forward to your contributions!

Best,
Bruno

On 05.12.21 08:29, Anshul Gangwal wrote:

Hi Admins,
I would like to sign-up as a Kafka contributor.
Could you grant me contributor access in wiki and Jira?

ID: anshulsg

Thanks,
Anshul

Re: Request Contributor access

[Bruno Cadonna]

Hi Prem,

I added you to the contributor group in Jira. You should be able to 
assign tickets to yourself, now.


Looking forward to your contributions.

Best,
Bruno



On 04.12.21 19:31, Prem Kamal wrote:

Hey folks,

I am willing to contribute to Apache Kafka.
Please, can you grant me contributor access?
Jira id : Prem237

Thanks,
Prem