Raghav, *My guess about the problem is that I was generate a csr (certificate signing request), which is different from actually extracting certificate. Please correct me if I am wrong.*
Yes, that is correct. Use "keytool -exportcert" to extract the certificate. *To actually address our problem of minimizing key exchanges between our Kafka Clients (customers) and us (Kafka Brokers), I experimented that if we generate a keystone and trust store for them, and then ask them to use it in their client, it works fine. It reduces the number of round trips. Let me know if something like this is ok or can their be a security breach ?* The issue with this approach is that you also have access to the customer's private key. And you need a secure way to transferring this key to the customer. The standard way of customer generating the key-pair and giving you only the public certificate avoids these issues. On Fri, May 19, 2017 at 1:19 PM, Raghav <raghavas...@gmail.com> wrote: > Rajini > > I was generating a certificate (using key tool by first doing -genkey and > creating a keystore, and then subsequently extracting certificate using > -centreq) for Kafka client (Producer). Once this certificate was available, > I was trying to add this certificate to Kafka Broker's trust store. While > doing this, key tool would not allow to add this certificate to trust store > of Kafka broker. > > My guess about the problem is that I was generate a csr (certificate > signing request), which is different from actually extracting certificate. > Please correct me if I am wrong. > > To actually address our problem of minimizing key exchanges between our > Kafka Clients (customers) and us (Kafka Brokers), I experimented that if we > generate a keystone and trust store for them, and then ask them to use it > in their client, it works fine. It reduces the number of round trips. Let > me know if something like this is ok or can their be a security breach ? > > Thanks. > > Raghav > > > > On Thu, May 18, 2017 at 10:26 AM, Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > >> Raghav, >> >> If you send me the full command sequence, I can take a look. Also, which >> JRE are you using? >> >> Regards, >> >> Rajini >> >> On Thu, May 18, 2017 at 12:19 PM, Raghav <raghavas...@gmail.com> wrote: >> >>> Rajini >>> >>> I just tried this. It turns out that I can't import cert-file by itself >>> in trust store until it is signed by a CA. Could be because of the format ? >>> Any idea here ... >>> >>> In the above steps, if I sign the server-cert-file and client-cert-file >>> by a private CA then I can add them to trust store and key store. In this >>> test, I did not add the CA cert in either keystone or trust store. >>> >>> Thanks for all your help. >>> >>> >>> >>> >>> On Thu, May 18, 2017 at 8:26 AM, Rajini Sivaram <rajinisiva...@gmail.com >>> > wrote: >>> >>>> Raghav, >>>> >>>> Perhaps what you want to do is: >>>> >>>> *You do (for the brokers):* >>>> >>>> Generate key-pair for broker: >>>> >>>> keytool -keystore kafka.server.keystore.jks -alias localhost -validity >>>> 365 -genkey >>>> >>>> Export certificate to a file to send to your customers: >>>> >>>> keytool -exportcert -file server-cert-file -keystore >>>> kafka.server.keystore.jks -alias localhost >>>> >>>> >>>> And you send server-cert-file to your customers. >>>> >>>> Once you get your customer's client-cert-file, you do: >>>> >>>> keytool -importcert -file client-cert-file -keystore >>>> kafka.server.truststore.jks -alias customerA >>>> >>>> If you are using SSL for inter-broker communication, your broker >>>> certificate also needs to be in the server truststore: >>>> >>>> keytool -importcert -file server-cert-file -keystore >>>> kafka.client.truststore.jks -alias broker >>>> >>>> >>>> *Your customers do (for the clients):* >>>> >>>> Generate key-pair for client: >>>> >>>> keytool -keystore kafka.client.keystore.jks -alias localhost -validity >>>> 365 -genkey >>>> >>>> Export certificate to a file to send to to you: >>>> >>>> keytool -exportcert -file client-cert-file -keystore >>>> kafka.client.keystore.jks -alias localhost >>>> >>>> >>>> Your customers send you their client-cert-file >>>> >>>> Your customers create their truststore using the broker certificate >>>> server-cert-file that you send to them: >>>> >>>> keytool -importcert -file server-cert-file -keystore >>>> kafka.client.truststore.jks -alias broker >>>> >>>> >>>> >>>> You then configure your brokers with (kafka.server.keystore.jks, ka >>>> fka.server.truststore.jks).Your customers configure their clients with >>>> (kafka.client.keystore.jks, kafka.client.truststore.jks). >>>> >>>> >>>> Hope that helps. >>>> >>>> Regards, >>>> >>>> Rajini >>>> >>>> >>>> >>>> On Thu, May 18, 2017 at 10:33 AM, Raghav <raghavas...@gmail.com> wrote: >>>> >>>>> Rajini, >>>>> >>>>> Sure, will submit a PR shortly. >>>>> >>>>> Your answer is very helpful, but I think I did not put the question >>>>> correctly. Pardon my ignore but I am still trying to get my ways around >>>>> Kafka security. >>>>> >>>>> I was trying to understand, can we (Kafka Broker) just add the >>>>> certificate (unsigned or signed) from customer to our trust store without >>>>> adding the CA cert to trust store... could that work ? >>>>> >>>>> 1. Let's say Kafka broker (there is only 1 for simplicity) generates a >>>>> keystore and generates a key using the command below >>>>> >>>>> keytool -keystore kafka.server.keystore.jks -alias localhost -validity >>>>> *365* -genkey >>>>> >>>>> keytool -keystore kafka.server.keystore.jks -alias localhost -certreq >>>>> -file server-cert-file >>>>> >>>>> 2. Similarly, Kafka Client (Producer) does the same >>>>> >>>>> keytool -keystore kafka.client.keystore.jks -alias localhost -validity >>>>> *365* -genkey >>>>> >>>>> keytool -keystore kafka.client.keystore.jks -alias localhost -certreq >>>>> -file client-cert-file >>>>> >>>>> >>>>> 3. Now, we add *client-cert-file* into the trust store of server, and >>>>> *server-cert-file* into the trust store of client. Given that each >>>>> trust store has other party's certificate in their trust store, does CA >>>>> certificate come into the picture ? >>>>> >>>>> On Thu, May 18, 2017 at 6:26 AM, Rajini Sivaram < >>>>> rajinisiva...@gmail.com> wrote: >>>>> >>>>>> Raghav, >>>>>> >>>>>> Yes, you can create a truststore with your customers' certificates >>>>>> and vice-versa. It will be best to give your CA certificate to your >>>>>> customers and get the CA certificate from each of your customers and add >>>>>> them to your broker's truststore. You can both then create additional >>>>>> certificates if you need without any changes to your truststore as long >>>>>> as >>>>>> the CA certificates are valid. Unlike certificates signed by a trusted >>>>>> authority, you will need to add the CAs of every customer to your >>>>>> truststore. Kafka brokers don't reload certificates, so if you wanted to >>>>>> add another customer's certificate to your truststore, you will need to >>>>>> restart your broker. >>>>>> >>>>>> Would you like to submit a PR with the information that is missing in >>>>>> the Apache Kafka documentation that you think may be useful? >>>>>> >>>>>> Regards, >>>>>> >>>>>> Rajini >>>>>> >>>>>> On Wed, May 17, 2017 at 6:21 PM, Raghav <raghavas...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Another quick question: >>>>>>> >>>>>>> Say we chose to add our customer's certificates directly to our >>>>>>> brokers trust store and vice verse, could that work ? There is no >>>>>>> documentation on Kafka or Confluent site for this ? >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> On Wed, May 17, 2017 at 1:56 PM, Rajini Sivaram < >>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>> >>>>>>>> Raghav, >>>>>>>> >>>>>>>> 1. Yes, your customers can use certificates signed by a trusted >>>>>>>> authority. You can simply omit the truststore configuration for your >>>>>>>> broker >>>>>>>> in server.properties, and Kafka would use the default, which will >>>>>>>> trust the >>>>>>>> client certificates. If your brokers are using SSL for inter-broker >>>>>>>> communication and you are still using your private CA for broker's >>>>>>>> keystore, then you will need two separate endpoints in your listener >>>>>>>> configuration, one for your customer's clients and another for >>>>>>>> inter-broker >>>>>>>> communication so that you can specify a truststore with your private >>>>>>>> ca-cert for your broker connections. >>>>>>>> >>>>>>>> 2. Yes, all the commands can specify password on the command line, >>>>>>>> so you should be able to generate all the stores using a script >>>>>>>> without any >>>>>>>> interactions. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rajini >>>>>>>> >>>>>>>> >>>>>>>> On Wed, May 17, 2017 at 2:49 PM, Raghav <raghavas...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> One follow up questions Rajini: >>>>>>>>> >>>>>>>>> 1. Can we use some other mechanism like have our customer's use a >>>>>>>>> well known CA which JKS understands, and in that case we don't have >>>>>>>>> to ask >>>>>>>>> our customers to do this certificate-in and certificate-out thing ? I >>>>>>>>> am >>>>>>>>> just trying to understand if we can make our customer's workflow >>>>>>>>> easier. >>>>>>>>> Anything else that you can suggest here.... >>>>>>>>> >>>>>>>>> 2. Can we automate the key gen steps mentioned on apache website >>>>>>>>> and adding to keystone and trust store so that we don't have to >>>>>>>>> manually >>>>>>>>> supply password ? Currently, everytime I tried to do steps mentioned >>>>>>>>> in >>>>>>>>> https://kafka.apache.org/documentation/#security I have to >>>>>>>>> manually give password. It would be great if we can automate this >>>>>>>>> process >>>>>>>>> either through script or Java code. Any suggestions ... >>>>>>>>> >>>>>>>>> >>>>>>>>> Many thanks. >>>>>>>>> >>>>>>>>> On Tue, May 16, 2017 at 10:58 AM, Raghav <raghavas...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Many thanks, Rajini. >>>>>>>>>> >>>>>>>>>> On Tue, May 16, 2017 at 8:43 AM, Rajini Sivaram < >>>>>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Raghav, >>>>>>>>>>> >>>>>>>>>>> If your Kafka broker is configured with >>>>>>>>>>> *ssl.client.auth=required,* your customer's clients need to >>>>>>>>>>> provide a keystore. In any case, they need a truststore since your >>>>>>>>>>> broker >>>>>>>>>>> is using SSL. For the truststore, you can given them ca-cert, as >>>>>>>>>>> you mentioned. Client keystore contains a certificate and a private >>>>>>>>>>> key. >>>>>>>>>>> >>>>>>>>>>> In the round-trip you described, customers generate the keys and >>>>>>>>>>> give you the certificate signing request, keeping their private key >>>>>>>>>>> private. You then send them back a signed certificate that goes >>>>>>>>>>> into their >>>>>>>>>>> keystore. This is the standard way of signing and is secure. >>>>>>>>>>> >>>>>>>>>>> In the single step scenario that you described, you generate the >>>>>>>>>>> customer's key-pair consisting of certificate and private key. You >>>>>>>>>>> then >>>>>>>>>>> need to send them both the signed certificate and the private key. >>>>>>>>>>> This is >>>>>>>>>>> less secure. Unlike the round-trip, you now have the private key of >>>>>>>>>>> the >>>>>>>>>>> customer. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Rajini >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, May 16, 2017 at 10:47 AM, Raghav <raghavas...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Rajini >>>>>>>>>>>> >>>>>>>>>>>> This was very helpful. I have another questions on similar >>>>>>>>>>>> lines. >>>>>>>>>>>> >>>>>>>>>>>> We host Kafka Broker, and we also have our own private CA. We >>>>>>>>>>>> want our customers to setup their Kafka Clients (Producer and >>>>>>>>>>>> Consumer) >>>>>>>>>>>> using SSL using *ssl.client.auth=required*. >>>>>>>>>>>> >>>>>>>>>>>> Is there a way, we can generate certificate for our clients, >>>>>>>>>>>> sign it using our private CA, and then hand over our customers >>>>>>>>>>>> these two >>>>>>>>>>>> certificates (1. ca-cert 2. cert-signed), which if they add to >>>>>>>>>>>> their >>>>>>>>>>>> keystroke and truststore, they can send message to our Kafka >>>>>>>>>>>> brokers while >>>>>>>>>>>> keeping *ssl.client.auth=required*. >>>>>>>>>>>> >>>>>>>>>>>> We are looking to minimize our customer's pre-setup steps. For >>>>>>>>>>>> example in normal scenario, customers will need to generate >>>>>>>>>>>> certificate, >>>>>>>>>>>> and hand over their certificate request to our private CA, which >>>>>>>>>>>> we then >>>>>>>>>>>> sign it, and send them signed certificate and private CA's >>>>>>>>>>>> certificate. So >>>>>>>>>>>> there is one round trip. Just wondering if we can reduce this 2 >>>>>>>>>>>> step into 1 >>>>>>>>>>>> step. >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, May 12, 2017 at 8:53 AM, Rajini Sivaram < >>>>>>>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Raqhav, >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Clients need a keystore if you are using TLS client >>>>>>>>>>>>> authentication. To >>>>>>>>>>>>> enable client authentication, you need to configure >>>>>>>>>>>>> ssl.client.auth in >>>>>>>>>>>>> server.properties. This can be set to required|requested|none. >>>>>>>>>>>>> If you don't >>>>>>>>>>>>> enable client authentication, any client will be able to >>>>>>>>>>>>> connect to your >>>>>>>>>>>>> broker. You could alternatively use SASL for client >>>>>>>>>>>>> authentication. >>>>>>>>>>>>> . >>>>>>>>>>>>> 2. Client keystore is mandatory if ssl.client.auth=required, >>>>>>>>>>>>> optional for >>>>>>>>>>>>> requested and not used for none. The truststore configured on >>>>>>>>>>>>> the client is >>>>>>>>>>>>> used to authenticate the server. So you have to provide it >>>>>>>>>>>>> unless your >>>>>>>>>>>>> broker is using certificates signed by a trusted authority. >>>>>>>>>>>>> >>>>>>>>>>>>> Hope that helps. >>>>>>>>>>>>> >>>>>>>>>>>>> Rajini >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, May 12, 2017 at 11:35 AM, Raghav < >>>>>>>>>>>>> raghavas...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> > Hi >>>>>>>>>>>>> > >>>>>>>>>>>>> > I read the documentation here: >>>>>>>>>>>>> > https://kafka.apache.org/documentation/#security_ssl >>>>>>>>>>>>> > >>>>>>>>>>>>> > I have few questions about trust store and keystore based on >>>>>>>>>>>>> this scenario: >>>>>>>>>>>>> > >>>>>>>>>>>>> > We have 5 Kafka Brokers in our cluster. We want our clients >>>>>>>>>>>>> to write to our >>>>>>>>>>>>> > Kafka brokers in a secure way. Suppose, we also host a >>>>>>>>>>>>> private CA as >>>>>>>>>>>>> > mentioned in the documentation above, and provide our >>>>>>>>>>>>> clients the *ca-cert* >>>>>>>>>>>>> > file, which they add it to their trust store. >>>>>>>>>>>>> > >>>>>>>>>>>>> > 1. Do we require our clients to generate their certificate >>>>>>>>>>>>> and have it >>>>>>>>>>>>> > signed by our private CA, and add it to their keystore? >>>>>>>>>>>>> > >>>>>>>>>>>>> > 2. When is keystore used by clients, and when is truststore >>>>>>>>>>>>> used by clients >>>>>>>>>>>>> > ? >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > Thanks. >>>>>>>>>>>>> > >>>>>>>>>>>>> > -- >>>>>>>>>>>>> > R >>>>>>>>>>>>> > >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Raghav >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Raghav >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Raghav >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Raghav >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Raghav >>>>> >>>> >>>> >>> >>> >>> -- >>> Raghav >>> >> >> > > > -- > Raghav >
