Re: SSL between Kafka and Spark Streaming API

2015-08-28 Thread Cassa L
Just to confirm, is this what you are mentioning about? Is there any example on how to set it? I believe it is for 0.8.3 version? https://cwiki.apache.org/confluence/display/KAFKA/Multiple+Listeners+for+Kafka+Brokers On Fri, Aug 28, 2015 at 12:52 PM, Sriharsha Chintalapani wrote: > You can con

Re: SSL between Kafka and Spark Streaming API

2015-08-28 Thread Cody Koeninger
Yeah, the direct api uses the simple consumer On Fri, Aug 28, 2015 at 1:32 PM, Cassa L wrote: > Hi I am using below Spark jars with Direct Stream API. > spark-streaming-kafka_2.10 > > When I look at its pom.xml, Kafka libraries that its pulling in is >org.apache.kafka >kafka_${scal

Re: SSL between Kafka and Spark Streaming API

2015-08-28 Thread Sourabh Chandak
Can we use the existing kafka spark streaming jar to connect to a kafka server running in SSL mode? We are fine with non SSL consumer as our kafka cluster and spark cluster are in the same network Thanks, Sourabh On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira wrote: > I can't speak for the Sp

Re: SSL between Kafka and Spark Streaming API

2015-08-28 Thread Cassa L
Hi I am using below Spark jars with Direct Stream API. spark-streaming-kafka_2.10 When I look at its pom.xml, Kafka libraries that its pulling in is org.apache.kafka kafka_${scala.binary.version} 0.8.2.1 I believe this DirectStream API uses SimpleConsumer API. Can someone from

SSL between Kafka and Spark Streaming API

2015-08-28 Thread Cassa L
Hi, I was going through SSL setup of Kafka. https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka However, I am also using Spark-Kafka streaming to read data from Kafka. Is there a way to activate SSL for spark streaming API or not possible at all? Thanks, LCassa