Hello Mohit, When we originally design the new producer API we removed the serializer / deserializer from the old producer and made it generic as accepting only message<byte[], byte[]>, but we later concluded it would still be more beneficial to add the serde back into the producer API. And as you observed one consequence is that if you have multiple data serialization formats then you have to use one producer for each.
The recommended way of using Kafka producer in production is to stick with one serialization format, e.g. Avro, with different topics to have different schemas that could still be ser-/de- by the single producer. Guozhang On Wed, May 13, 2015 at 10:03 AM, Mohit Gupta <[email protected] > wrote: > Hello, > > I've a question regarding the design of the new Producer API. > > As per the design (KafkaProducer<K,V>), it seems that a separate producer > is required for every combination of key and value type. Where as, in > documentation ( and elsewhere ) it's recommended to create a single > producer instance per application and share it among the all the threads > for best performance? > > One way to create only single producer would be to use byte[] as key/value > type and handle the serialization at the client itself, rather than the > producer, similar to the example in javadocs. But wouldn't this defeat the > purpose of using generics in the producer? > > Specific to our use case, we have multiple types of messages, where each > message type can have multiple custom serializers. And, a message can be > pushed into mulitple topics with different serialization. > > > -- > Best Regards, > > Mohit Gupta > -- -- Guozhang
