Hi, Hans. I was able to get the command line producer / consumer working with SSL but I'm not sure how to measure millisecond resolution latency with them. I thought maybe the '--property print.timestamp=true' argument would help, but only has second resolution. Do you know of any way to get the consumer to print out a receipt time-stamp with millisecond resolution? Or of any extended documentation on the command line tools in general?
Oh also, a couple other tidbits that may help: Ubuntu 16.04 Kafka 10.1.0 openjdk version "1.8.0_111" TLS 1.2 I was wondering if maybe this could be my problem: http://stackoverflow.com/questions/25992131/slow-aes-gcm-encryption-and-decryption-with-java-8u20 I didn't specify any cipher suites in either the broker or the client config which I gather leaves it up to the broker/client to decide during TLS handshaking. I'm not sure if there is an easy way to figure out which one they ended up with... I'll work on specifying which cipher suite I want and try to pick something with which java is simpatico. On Thu, Nov 17, 2016 at 4:04 PM, Hans Jespersen <h...@confluent.io> wrote: > What is the difference using the bin/kafka-console-producer and > kafka-console-consumer as pub/sub clients? > > see http://docs.confluent.io/3.1.0/kafka/ssl.html > > -hans > > /** > * Hans Jespersen, Principal Systems Engineer, Confluent Inc. > * h...@confluent.io (650)924-2670 > */ > > On Thu, Nov 17, 2016 at 11:56 PM, Aaron Wilkinson <aa...@modopayments.com> > wrote: > > > Pardon if this is a oft repeated issue, but all the information I could > > find said I should expect a 20-50% performance hit when using SSL with > > kafka, and I am seeing closer to 2000-3000% > > > > I'm trying to get kafka to behave like a fast, secured message bus. So I > > am sending small messages, one at a time. I have set up a simple, 2 > > machine experiment in AWS with 1 client machine and 1 zookeeper/broker > > machine and I'm an running a very linear test. > > > > There are 2 topics: "request" and "response" and 2 threads on the client > > machine each of which connects to those 2 topics. Thread 1 produces a > > "request", thread 2 consumes it and then produces a "response" which > thread > > 1 then consumes. At that point thread 1 proceeds to send the next > > "request" and the process repeats. > > > > So there are a total of 4 connections to the broker. > > > > I can run a sustained test without SSL and see 1 to 1.5 ms per message > hop > > (where a "hop" means the message has traveled across 1 of the 4 > > connections- either a production or a consumption of either the request > or > > the response). > > > > Each connection for which I turn on SSL increases the hop time 35 to 45 > ms. > > > > Now, the problem could be with the stack I'm using (PHP 7 talking to the > > broker via the librdkafka C library). But before I go about trying to > > reproduce this with a java client (which is not my forte) I was wondering > > if anyone else has run into a similar issue either with PHP or any other > > language / library. Or does anyone know a direct way to figure out > whether > > this slow down is at the broker or at the client? > > > > Thanks in advance for your help! > > Aaron > > >
