[ https://issues.apache.org/jira/browse/SPARK-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600010#comment-14600010 ]
Juan RodrĂguez Hortalá commented on SPARK-8337: ----------------------------------------------- Hi, As I said above, I don't know much about the internals of pyspark, and currently the original RDD from Scala is wrapped by several wrappers for the communication with python, and so the RDD implementing HasOffsetRanges is hidden by those layers. However, after its merge with SPARK-8389, it looks like this issue has got the attention of several Spark committers, and I'm sure they will be able to come up with a solution that makes OffsetRanges accessible from pyspark. Greetings, Juan > KafkaUtils.createDirectStream for python is lacking API/feature parity with > the Scala/Java version > -------------------------------------------------------------------------------------------------- > > Key: SPARK-8337 > URL: https://issues.apache.org/jira/browse/SPARK-8337 > Project: Spark > Issue Type: Bug > Components: PySpark, Streaming > Affects Versions: 1.4.0 > Reporter: Amit Ramesh > Priority: Critical > > See the following thread for context. > http://apache-spark-developers-list.1001551.n3.nabble.com/Re-Spark-1-4-Python-API-for-getting-Kafka-offsets-in-direct-mode-tt12714.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org