How to use Window Operations with kafka Direct-API?

2015-06-12 Thread ZIGEN
ssage in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-Window-Operations-with-kafka-Direct-API-tp23293.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-

Re: How to use Window Operations with kafka Direct-API?

2015-06-12 Thread Saisai Shao
dPartition(offsets.topic(), offsets.partition()); > > HashMap map = new > HashMap Object>(); > map.put(topicAndPartition, offsets.untilOffset()); >

Re: How to use Window Operations with kafka Direct-API?

2015-06-12 Thread zigen
t;> // ClassCastException occurred >> >> KafkaCluster kc = new KafkaCluster(toScalaMap(kafkaParams)); >> for (OffsetRange offsets : offsetsList) { >> >> TopicAndPartition topicAndPartition = new >&

Re: How to use Window Operations with kafka Direct-API?

2015-06-12 Thread Cody Koeninger
@Override >>> public List call(List list1, List list2) >>> throws >>> Exception { >>> ... >>> } >>> }, windowDuration, slideDuration); >>> >>> windowDs.foreachRDD(new Function>,