[jira] [Commented] (KAFKA-1811) ensuring registered broker host:port is unique
[ https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14332167#comment-14332167 ] Dave Parfitt commented on KAFKA-1811: - Chao Chu - sorry, I'm not able to finish working on this. I just had an addition to my family thats taking up all my free time. Cheers - Dave ensuring registered broker host:port is unique -- Key: KAFKA-1811 URL: https://issues.apache.org/jira/browse/KAFKA-1811 Project: Kafka Issue Type: Improvement Reporter: Jun Rao Labels: newbie Attachments: KAFKA_1811.patch Currently, we expect each of the registered broker to have a unique host:port pair. However, we don't enforce that, which causes various weird problems. It would be useful to ensure this during broker registration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1811) ensuring registered broker host:port is unique
[ https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244175#comment-14244175 ] Dave Parfitt commented on KAFKA-1811: - Hi Gwen - Thanks so much for taking the time to review my patch and pointing me in the right direction. 2. We also like unit tests Absolutely. I wanted to make sure I was going down the correct path first. 3. I'd consider pushing this check down to registerBrokerInZk. It seems like a natural place to ensure uniqueness of registered brokers before registering. My thinking was that I wanted to detect non-unique host:port combinations before the SocketServer started, however it probably makes more sense in registerBrokerInZk as you suggest. 4. Another thing to consider is race conditions - what if new broker registers while we are checking? Perhaps we can even use ZK itself to enforce uniqueness? I should have thought of this, is there any code in Kafka that you recommend I model this after? Cheers - Dave ensuring registered broker host:port is unique -- Key: KAFKA-1811 URL: https://issues.apache.org/jira/browse/KAFKA-1811 Project: Kafka Issue Type: Improvement Reporter: Jun Rao Labels: newbie Attachments: KAFKA_1811.patch Currently, we expect each of the registered broker to have a unique host:port pair. However, we don't enforce that, which causes various weird problems. It would be useful to ensure this during broker registration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1811) ensuring registered broker host:port is unique
[ https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Parfitt updated KAFKA-1811: Attachment: KAFKA_1811.patch ensuring registered broker host:port is unique -- Key: KAFKA-1811 URL: https://issues.apache.org/jira/browse/KAFKA-1811 Project: Kafka Issue Type: Improvement Reporter: Jun Rao Labels: newbie Attachments: KAFKA_1811.patch Currently, we expect each of the registered broker to have a unique host:port pair. However, we don't enforce that, which causes various weird problems. It would be useful to ensure this during broker registration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1811) ensuring registered broker host:port is unique
[ https://issues.apache.org/jira/browse/KAFKA-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243286#comment-14243286 ] Dave Parfitt commented on KAFKA-1811: - Patch attached. ensuring registered broker host:port is unique -- Key: KAFKA-1811 URL: https://issues.apache.org/jira/browse/KAFKA-1811 Project: Kafka Issue Type: Improvement Reporter: Jun Rao Labels: newbie Attachments: KAFKA_1811.patch Currently, we expect each of the registered broker to have a unique host:port pair. However, we don't enforce that, which causes various weird problems. It would be useful to ensure this during broker registration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1796) Sanity check partition command line tools
[
https://issues.apache.org/jira/browse/KAFKA-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233478#comment-14233478
]
Dave Parfitt commented on KAFKA-1796:
-
seems like simple json schemas could catch json input errors.
Perhaps https://github.com/fge/json-schema-validator ?
Sanity check partition command line tools
-
Key: KAFKA-1796
URL: https://issues.apache.org/jira/browse/KAFKA-1796
Project: Kafka
Issue Type: Bug
Reporter: Guozhang Wang
Labels: newbie
Fix For: 0.8.3
We need to sanity check the input json has the valid values before triggering
the admin process. For example, we have seen a scenario where the json input
for partition reassignment tools have partition replica info as {broker-1,
broker-1, broker-2} and it is still accepted in ZK and eventually lead to
under replicated count, etc. This is partially because we use a Map rather
than a Set reading the json input for this case; but in general we need to
make sure the input parameters like Json needs to be valid before writing it
to ZK.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1796) Sanity check partition command line tools
[
https://issues.apache.org/jira/browse/KAFKA-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233508#comment-14233508
]
Dave Parfitt commented on KAFKA-1796:
-
Here's a quick schema, input, and verification output:
https://gist.github.com/metadave/09a69b94f0564d27ec10
I could add this support to Kafka if you think this is the right direction.
Sanity check partition command line tools
-
Key: KAFKA-1796
URL: https://issues.apache.org/jira/browse/KAFKA-1796
Project: Kafka
Issue Type: Bug
Reporter: Guozhang Wang
Labels: newbie
Fix For: 0.8.3
We need to sanity check the input json has the valid values before triggering
the admin process. For example, we have seen a scenario where the json input
for partition reassignment tools have partition replica info as {broker-1,
broker-1, broker-2} and it is still accepted in ZK and eventually lead to
under replicated count, etc. This is partially because we use a Map rather
than a Set reading the json input for this case; but in general we need to
make sure the input parameters like Json needs to be valid before writing it
to ZK.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1803) UncleanLeaderElectionEnableProp in LogConfig should be of boolean
[ https://issues.apache.org/jira/browse/KAFKA-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Parfitt updated KAFKA-1803: Attachment: 0001-fix-for-KAFKA-1803.patch UncleanLeaderElectionEnableProp in LogConfig should be of boolean - Key: KAFKA-1803 URL: https://issues.apache.org/jira/browse/KAFKA-1803 Project: Kafka Issue Type: Bug Components: core Reporter: Jun Rao Labels: newbie Fix For: 0.8.3 Attachments: 0001-fix-for-KAFKA-1803.patch, KAFKA1803.patch Now that KAFKA-1798 is fixed, we should define UncleanLeaderElectionEnableProp as a boolean, instead of String in LogConfig and get rid of the customized validation for boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1803) UncleanLeaderElectionEnableProp in LogConfig should be of boolean
[ https://issues.apache.org/jira/browse/KAFKA-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Parfitt updated KAFKA-1803: Attachment: (was: 0001-fix-for-KAFKA-1803.patch) UncleanLeaderElectionEnableProp in LogConfig should be of boolean - Key: KAFKA-1803 URL: https://issues.apache.org/jira/browse/KAFKA-1803 Project: Kafka Issue Type: Bug Components: core Reporter: Jun Rao Labels: newbie Fix For: 0.8.3 Attachments: KAFKA1803.patch Now that KAFKA-1798 is fixed, we should define UncleanLeaderElectionEnableProp as a boolean, instead of String in LogConfig and get rid of the customized validation for boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1803) UncleanLeaderElectionEnableProp in LogConfig should be of boolean
[ https://issues.apache.org/jira/browse/KAFKA-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Parfitt updated KAFKA-1803: Attachment: KAFKA1803.2.patch UncleanLeaderElectionEnableProp in LogConfig should be of boolean - Key: KAFKA-1803 URL: https://issues.apache.org/jira/browse/KAFKA-1803 Project: Kafka Issue Type: Bug Components: core Reporter: Jun Rao Labels: newbie Fix For: 0.8.3 Attachments: KAFKA1803.2.patch, KAFKA1803.patch Now that KAFKA-1798 is fixed, we should define UncleanLeaderElectionEnableProp as a boolean, instead of String in LogConfig and get rid of the customized validation for boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1803) UncleanLeaderElectionEnableProp in LogConfig should be of boolean
[ https://issues.apache.org/jira/browse/KAFKA-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232435#comment-14232435 ] Dave Parfitt commented on KAFKA-1803: - fixed in KAFKA1803.2.patch, thanks! UncleanLeaderElectionEnableProp in LogConfig should be of boolean - Key: KAFKA-1803 URL: https://issues.apache.org/jira/browse/KAFKA-1803 Project: Kafka Issue Type: Bug Components: core Reporter: Jun Rao Labels: newbie Fix For: 0.8.3 Attachments: KAFKA1803.2.patch, KAFKA1803.patch Now that KAFKA-1798 is fixed, we should define UncleanLeaderElectionEnableProp as a boolean, instead of String in LogConfig and get rid of the customized validation for boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1803) UncleanLeaderElectionEnableProp in LogConfig should be of boolean
[ https://issues.apache.org/jira/browse/KAFKA-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dave Parfitt updated KAFKA-1803: Attachment: KAFKA1803.patch UncleanLeaderElectionEnableProp in LogConfig should be of boolean - Key: KAFKA-1803 URL: https://issues.apache.org/jira/browse/KAFKA-1803 Project: Kafka Issue Type: Bug Components: core Reporter: Jun Rao Labels: newbie Fix For: 0.8.3 Attachments: KAFKA1803.patch Now that KAFKA-1798 is fixed, we should define UncleanLeaderElectionEnableProp as a boolean, instead of String in LogConfig and get rid of the customized validation for boolean. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
