[ https://issues.apache.org/jira/browse/SPARK-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588439#comment-14588439 ]
Amit Ramesh commented on SPARK-8337: ------------------------------------ [~juanrh] this looks pretty good to me. And from what I can see shouldn't add much overhead compared to the existing logic. It is perfect in terms of what are in need of :). One stylistic suggestion is that you could return (key, value, kafka_offsets) where kafka_offsets is a dict of topic, parition and offset. This would keep things a little more consistent with what is returned when meta info is False. Thanks! Amit > KafkaUtils.createDirectStream for python is lacking API/feature parity with > the Scala/Java version > -------------------------------------------------------------------------------------------------- > > Key: SPARK-8337 > URL: https://issues.apache.org/jira/browse/SPARK-8337 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 1.4.0 > Reporter: Amit Ramesh > Priority: Critical > > See the following thread for context. > http://apache-spark-developers-list.1001551.n3.nabble.com/Re-Spark-1-4-Python-API-for-getting-Kafka-offsets-in-direct-mode-tt12714.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org