onsumer strategy for creating the stream below. Any suggestions
>>>> on how
>>>> >> > to
>>>> >> > run this with better performance would be of great help.
>>>> >> >
>>>> >> > java.lang.AssertionError: assertion fai
324027964 after polling for 12
>>> >> >
>>> >> > val kafkaParams = Map[String, Object](
>>> >> > "bootstrap.servers" -> kafkaBrokers,
>>> >> > "key.deserializer" ->
t;> > "heartbeat.interval.ms" -> Integer.valueOf(2),
>> >> > "session.timeout.ms" -> Integer.valueOf(6),
>> >> > "request.timeout.ms" -> Integer.valueOf(9),
>> >> > "enab
ueOf(6),
> >> > "request.timeout.ms" -> Integer.valueOf(9),
> >> > "enable.auto.commit" -> (false: java.lang.Boolean),
> >> >
g.Boolean),
>> > "spark.streaming.kafka.consumer.cache.enabled" -> "false",
>> > "group.id" -> "test1"
>> > )
>> >
>> > val hubbleStream = KafkaUtils.createDirectStream[String, String](
>>
),
>> > "session.timeout.ms" -> Integer.valueOf(6),
>> > "request.timeout.ms" -> Integer.valueOf(9),
>> > "enable.auto.commit" -> (false: java.lang.Boolean),
>> > "spark.streaming.kafka.consumer.cache.enabled" -> &q
k.streaming.kafka.consumer.cache.enabled" -> "false",
> > "group.id" -> "test1"
> > )
> >
> > val hubbleStream = KafkaUtils.createDirectStream[String, String](
> > ssc,
> > LocationStrategies.Pref
-> "test1"
> )
>
> val hubbleStream = KafkaUtils.createDirectStream[String, String](
> ssc,
> LocationStrategies.PreferConsistent,
> ConsumerStrategies.Subscribe[String, String](topicsSet, kafkaParams)
> )
>
>
>
&
d" -> "false",
"group.id" -> "test1"
)
val hubbleStream = KafkaUtils.createDirectStream[String, String](
ssc,
LocationStrategies.PreferConsistent,
ConsumerStrategies.Subscribe[String, String](topicsSet, kafkaParams)
-to-make-sure-that-Spark-Kafka-Direct-Streaming-job-maintains-the-state-upon-code-deployment-tp28799.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe e-mail: user-unsubscr
checkpointing is required. Can I
achieve this, i.e. enabling meta checkpointing and not the data
checkpointing?
Thanks
Abhishek Patel
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Kafka-Direct-Streaming-tp23685.html
Sent from the Apache
:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Kafka-Direct-Streaming-tp23685.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
12 matches
Mail list logo