599166320 commented on issue #12929: URL: https://github.com/apache/druid/issues/12929#issuecomment-1231923942
> thank you for your reply。 In fact, we have used this technology in large-scale clusters and have seen better query performance. Our implementation may be more complicated than I have described. In a table, we use not only hash partitions but also enumeration partitions, considering data skew and combining hash partitions. I also want to talk about my own views on some of the issues you mentioned above. 1. what happens if the producer doesn't correctly do the hash codes? I think we can implement something similar to the client SDK to constrain this behavior. The Kafka header carries some partition information, indicating that the producer writes data according to the specification. 2. What if we want to add partitions to the kafka topic? The Kafka producer can specify to write the existing partition. Even if the partition is added, the old partition can still be written. In most cases, the partition of Kafka topic only allows capacity expansion, not capacity reduction. Because it is very expensive to delete the topic, the volume will not be reduced by deleting the topic. 3. We would want to build something that the average user could use easily, and correctly. I think that average users prefer to configure their tables step by step through the browser interface in the existing way, so there will be no mental burden. If Druid can provide the implementation of Kafka partition type at the database level, many developers do not need to implement and maintain these nonstandard codes, and can unlock more performance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
