Re: Getting Message From Structured Streaming Format Kafka

2017-12-01 Thread Daniel de Oliveira Mantovani
Hello Burak, Sorry to the delayed answer, you were right. 1) - I change the sql-kafka connector version and fixed. 2) - The propose was just test, and I was using normal streaming also for other thing. I'm was wondering how did you know was the sql-kafka connector version reading the logs. I

Re: Getting Message From Structured Streaming Format Kafka

2017-11-02 Thread Burak Yavuz
Hi Daniel, Several things: 1) Your error seems to suggest you're using a different version of Spark and a different version of the sql-kafka connector. Could you make sure they are on the same Spark version? 2) With Structured Streaming, you may remove everything related to a StreamingContext.

Getting Message From Structured Streaming Format Kafka

2017-11-02 Thread Daniel de Oliveira Mantovani
Hello, I'm trying to run the following code, var newContextCreated = false // Flag to detect whether new context was created or not val kafkaBrokers = "localhost:9092" // comma separated list of broker:host private val batchDuration: Duration = Seconds(3) private val master: String = "local[2]"