[ https://issues.apache.org/jira/browse/STORM-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338487#comment-14338487 ]
Rick Kellogg commented on STORM-650: ------------------------------------ I would also like to see the following changes: * Public versus internal classes should be isolated into another package, i.e. KafkaSpout versus PartitionManager. * Instance variables inside of configuration classes (KafkaConfig, SpoutConfig), should be made private to enforce immutability. * Logger instance variables should all be private NOT public. * Review use of throw RuntimeException. Error handling is quite poor in some circumstances. > Storm-Kafka Refactoring and Improvements > ---------------------------------------- > > Key: STORM-650 > URL: https://issues.apache.org/jira/browse/STORM-650 > Project: Apache Storm > Issue Type: Improvement > Components: storm-kafka > Reporter: P. Taylor Goetz > > This is intended to be a parent/umbrella JIRA covering a number of > efforts/suggestions aimed at improving the storm-kafka module. The goal is to > facilitate communication and collaboration by providing a central point for > discussion and coordination. > The first phase should be to identify and agree upon a list of high-level > points we would like to address. Once that is complete, we can move on to > implementation/design discussions, followed by an implementation plan, > division of labor, etc. > A non-exhaustive, initial list of items follows. New/additional thoughts can > be proposed in the comments. > * Improve API for Specifying the Kafka Starting point > Configuring the kafka spout's starting position (e.g. forceFromStart=true) is > a common source of confusion. This should be refactored to provide an easy to > understand, unambiguous API for configuring this property. > * Use Kafka APIs Instead of Internal ZK Metadata (STORM-590) > Currently the Kafka spout relies on reading Kafka's internal metadata from > zookeeper. This should be refactored to use the Kafka Consumer API to protect > against changes to the internal metadata format stored in ZK. > * Improve Error Handling > There are a number of failure scenarios with the kafka spout that users may > want to react to differently based on their use case. Add a failure handler > API that allows users to implement and/or plug in alternative failure > handling implementations. It is assumed that default (sane) implementations > would be included and configured by default. > * Configuration/General Refactoring (BrokerHosts, etc.) (STORM-631) > (need to flesh this out better) Reduce unnecessary marker > interfaces/"instance of" checks. Unify configuration of core storm/trident > spout implementations. > * Kafka Spout doesn't pick up from the beginning of the queue unless > forceFromStart specified (STORM-563) > Discussion Items: > * How important is backward compatibility? -- This message was sent by Atlassian JIRA (v6.3.4#6332)