Hi, We have an internal version of Open source Kafka consumer and producer that we use and are working on adding that as a source and sink for flink.
It seems like the easiest way to add the consumer as source would be to override the FlinkKafkaConsumer class's createFetcher <https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer.java#L239> method to provide our own derived class of KafkaFetcher <https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/internals/KafkaFetcher.java> class which can hookup its own version of the consumerThread. But the fetcher classes are annotated as Internal and seems like it is not meant to be used this way. (And the changes for Producer would be on similar lines). Is there a recommendation for how to add new flavors of Kafka Consumer/Producer from the community? Would it be recommended to maintain a copy of all the connector classes so we don't have to deal with changes to classes tagged as internal? -- Thanks & Regards
