Francesco Beneventi created BAHIR-104:
-----------------------------------------
Summary: Dstream returned by the new multi topic support API is
not a pairRDD
Key: BAHIR-104
URL: https://issues.apache.org/jira/browse/BAHIR-104
Project: Bahir
Issue Type: Bug
Components: Spark Streaming Connectors
Affects Versions: Spark-2.1.0
Reporter: Francesco Beneventi
The new multi topic support API added with [BAHIR-89], when used in pyspark,
does not return a Dstream of <topic,message> tuples.
Example:
In pyspark, when creating a Dstream using the new API ( mqttstream =
MQTTUtils.createPairedStream(ssc, brokerUrl, topics) ) the expected contents of
mqttstream should be a collections of tuples:
(topic,message) , (topic,message) , (topic,message) , ...
Instead, the current content is a flattened list:
topic, message, topic, message, topic, message, ...
that is hard to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
