[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481515#comment-16481515 ]
Fred Teunissen commented on FLINK-8500: --------------------------------------- I agree that 'separation of concerns' is a good principle, but in this situation I have a problem with it. The interface DeserializationSchema<T> is responsible for deserialisation of the message, but in our case the messages are encrypted and every topic has its own decryption key. So we need at least the topic name during deserialisation, so currently we are using the interface KeyedDeserializationSchema<T>. I don't think this is kafka specific, so it should be solved in the "common deserialization schema" interface. Another concern I have is that by having a two-step approach for creating the records from kafka messages, you get more memory allocations and thus more garbage collection cycles. Like [~StephanEwen] said, this issue can be fixed for now by introducing a new default method on the interface KeyedDeserializationSchema<T>, but for the long run a new `common connector framework` should be designed and implemented. This new `common connector framework` should also address some other issues like 'temporarily idleness of partitions', 'customisable partition assignments' . When you agree, I can make a new PR with the new default method on the interface KeyedDeserializationSchema<T>. > Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher) > --------------------------------------------------------------------------- > > Key: FLINK-8500 > URL: https://issues.apache.org/jira/browse/FLINK-8500 > Project: Flink > Issue Type: Improvement > Components: Kafka Connector > Affects Versions: 1.4.0 > Reporter: yanxiaobin > Priority: Major > Fix For: 1.6.0 > > Attachments: image-2018-01-30-14-58-58-167.png, > image-2018-01-31-10-48-59-633.png > > > The method deserialize of KeyedDeserializationSchema needs a parameter > 'kafka message timestamp' (from ConsumerRecord) .In some business scenarios, > this is useful! > -- This message was sent by Atlassian JIRA (v7.6.3#76005)