I think the goal here is to make the actual message stored on broker to be encrypted, because after we have SSL, the transmission would be encrypted.
In general there might be tow approaches: 1. Broker do the encryption/decryption 2. Client do the encryption/decryption >From performance point of view, I would prefer [2]. It is just in that case, maybe user does not necessarily need to use SSL anymore because the data would be encrypted anyway. If we let client do the encryption, there are also two ways to do so - either we let producer take an encryptor or users can do serialization/encryption outside the producer and send raw bytes. The only difference between the two might be flexibility. For example, if someone wants to know the actual bytes of a message that got sent over the wire, doing it outside the producer would probably more preferable. Jiangjie (Becket) Qin On Thu, Jul 30, 2015 at 12:16 PM, eugene miretsky <eugene.miret...@gmail.com > wrote: > Hi, > > Based on the security wiki page > <https://cwiki.apache.org/confluence/display/KAFKA/Security> encryption of > data at rest is out of scope for the time being. However, we are > implementing encryption in Kafka and would like to see if there is > interest in submitting a patch got it. > > I suppose that one way to implement encryption would be to add an > 'encrypted key' field to the Message/MessageSet structures in the > wire protocole - however, this is a very big and fundamental change. > > A simpler way to add encryption support would be: > 1) Custom Serializer, but it wouldn't be compatible with other custom > serializers (Avro, etc. ) > 2) Add a step in KafkaProducer after serialization to encrypt the data > before it's being submitted to the accumulator (encryption is done in the > submitting thread, not in the producer io thread) > > Is there interest in adding #2 to Kafka? > > Cheers, > Eugene >
