[jira] [Commented] (KAFKA-14145) Faster propagation of high-watermark in KRaft topic partitions
[
https://issues.apache.org/jira/browse/KAFKA-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856152#comment-17856152
]
Josep Prat commented on KAFKA-14145:
Changing target fix version to 3.9 since this is not a blocker and we are past
code freeze
> Faster propagation of high-watermark in KRaft topic partitions
> --
>
> Key: KAFKA-14145
> URL: https://issues.apache.org/jira/browse/KAFKA-14145
> Project: Kafka
> Issue Type: Task
> Components: kraft
>Reporter: Jose Armando Garcia Sancio
>Assignee: Jose Armando Garcia Sancio
>Priority: Critical
> Fix For: 3.8.0
>
>
> Typically, the HWM is increase after one round of Fetch requests from the
> majority of the replicas. The HWM is propagated after another round of Fetch
> requests. If the LEO doesn't change the propagation of the HWM can be delay
> by one Fetch wait timeout (500ms).
> Looking at the KafkaRaftClient implementation we would have to have an index
> for both the fetch offset and the last sent high-watermark for that replica.
> Another issue here is that we changed the KafkaRaftManager so that it doesn't
> set the replica id when it is an observer/broker. Since the HWM is not part
> of the Fetch request the leader would have to keep track of this in the
> LeaderState.
> {code:java}
> val nodeId = if (config.processRoles.contains(ControllerRole)) {
> OptionalInt.of(config.nodeId)
> } else {
> OptionalInt.empty()
> } {code}
> We would need to find a better solution for
> https://issues.apache.org/jira/browse/KAFKA-13168 or improve the FETCH
> request so that it includes the HWM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-14145) Faster propagation of high-watermark in KRaft topic partitions
[
https://issues.apache.org/jira/browse/KAFKA-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17800622#comment-17800622
]
Stanislav Kozlovski commented on KAFKA-14145:
-
Changing target fix version to 3.8 since this is not a blocker and we are
cutting a 3.7 RC
> Faster propagation of high-watermark in KRaft topic partitions
> --
>
> Key: KAFKA-14145
> URL: https://issues.apache.org/jira/browse/KAFKA-14145
> Project: Kafka
> Issue Type: Task
> Components: kraft
>Reporter: Jose Armando Garcia Sancio
>Assignee: Jose Armando Garcia Sancio
>Priority: Critical
> Fix For: 3.7.0
>
>
> Typically, the HWM is increase after one round of Fetch requests from the
> majority of the replicas. The HWM is propagated after another round of Fetch
> requests. If the LEO doesn't change the propagation of the HWM can be delay
> by one Fetch wait timeout (500ms).
> Looking at the KafkaRaftClient implementation we would have to have an index
> for both the fetch offset and the last sent high-watermark for that replica.
> Another issue here is that we changed the KafkaRaftManager so that it doesn't
> set the replica id when it is an observer/broker. Since the HWM is not part
> of the Fetch request the leader would have to keep track of this in the
> LeaderState.
> {code:java}
> val nodeId = if (config.processRoles.contains(ControllerRole)) {
> OptionalInt.of(config.nodeId)
> } else {
> OptionalInt.empty()
> } {code}
> We would need to find a better solution for
> https://issues.apache.org/jira/browse/KAFKA-13168 or improve the FETCH
> request so that it includes the HWM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-14145) Faster propagation of high-watermark in KRaft topic partitions
[
https://issues.apache.org/jira/browse/KAFKA-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712957#comment-17712957
]
Mickael Maison commented on KAFKA-14145:
We're past feature freeze for 3.5.0 so moving to 3.6.0.
> Faster propagation of high-watermark in KRaft topic partitions
> --
>
> Key: KAFKA-14145
> URL: https://issues.apache.org/jira/browse/KAFKA-14145
> Project: Kafka
> Issue Type: Task
> Components: kraft
>Reporter: Jose Armando Garcia Sancio
>Assignee: Jose Armando Garcia Sancio
>Priority: Critical
> Fix For: 3.5.0
>
>
> Typically, the HWM is increase after one round of Fetch requests from the
> majority of the replicas. The HWM is propagated after another round of Fetch
> requests. If the LEO doesn't change the propagation of the HWM can be delay
> by one Fetch wait timeout (500ms).
> Looking at the KafkaRaftClient implementation we would have to have an index
> for both the fetch offset and the last sent high-watermark for that replica.
> Another issue here is that we changed the KafkaRaftManager so that it doesn't
> set the replica id when it is an observer/broker. Since the HWM is not part
> of the Fetch request the leader would have to keep track of this in the
> LeaderState.
> {code:java}
> val nodeId = if (config.processRoles.contains(ControllerRole)) {
> OptionalInt.of(config.nodeId)
> } else {
> OptionalInt.empty()
> } {code}
> We would need to find a better solution for
> https://issues.apache.org/jira/browse/KAFKA-13168 or improve the FETCH
> request so that it includes the HWM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KAFKA-14145) Faster propagation of high-watermark in KRaft topic partitions
[
https://issues.apache.org/jira/browse/KAFKA-14145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17647815#comment-17647815
]
A. Sophie Blee-Goldman commented on KAFKA-14145:
[~jagsancio] moving this to 3.5.0 since we are past code freeze for 3.4
> Faster propagation of high-watermark in KRaft topic partitions
> --
>
> Key: KAFKA-14145
> URL: https://issues.apache.org/jira/browse/KAFKA-14145
> Project: Kafka
> Issue Type: Task
> Components: kraft
>Reporter: Jose Armando Garcia Sancio
>Assignee: Jose Armando Garcia Sancio
>Priority: Critical
> Fix For: 3.5.0
>
>
> Typically, the HWM is increase after one round of Fetch requests from the
> majority of the replicas. The HWM is propagated after another round of Fetch
> requests. If the LEO doesn't change the propagation of the HWM can be delay
> by one Fetch wait timeout (500ms).
> Looking at the KafkaRaftClient implementation we would have to have an index
> for both the fetch offset and the last sent high-watermark for that replica.
> Another issue here is that we changed the KafkaRaftManager so that it doesn't
> set the replica id when it is an observer/broker. Since the HWM is not part
> of the Fetch request the leader would have to keep track of this in the
> LeaderState.
> {code:java}
> val nodeId = if (config.processRoles.contains(ControllerRole)) {
> OptionalInt.of(config.nodeId)
> } else {
> OptionalInt.empty()
> } {code}
> We would need to find a better solution for
> https://issues.apache.org/jira/browse/KAFKA-13168 or improve the FETCH
> request so that it includes the HWM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
