[
https://issues.apache.org/jira/browse/BAHIR-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959299#comment-15959299
]
Francesco Beneventi commented on BAHIR-104:
-------------------------------------------
[~anntinutj], I fixed the wrong method call before running my tests. You can
check it in my test repo
[Github|https://github.com/fbeneventi/bahir/commit/3755ecc56e19ac58fe20089a475253cc0bd19486#diff-f63d51c23efe7ce897bcc35fc52039f7]
Even with the correct API (helper.createPairedStream) I have the same issue.
I'm not really a java/scala programmer but can you help me figure out how to
address this problem?
After solving this problem I can provide a simple test case to use this API in
python.
> MQTT Dstream returned by the new multi topic support API is not a pairRDD
> -------------------------------------------------------------------------
>
> Key: BAHIR-104
> URL: https://issues.apache.org/jira/browse/BAHIR-104
> Project: Bahir
> Issue Type: Bug
> Components: Spark Streaming Connectors
> Affects Versions: Spark-2.1.0
> Reporter: Francesco Beneventi
> Labels: MQTT, SPARK
>
> The new multi topic support API added with [BAHIR-89], when used in pyspark,
> does not return a Dstream of <topic,message> tuples.
> Example:
> In pyspark, when creating a Dstream using the new API ( mqttstream =
> MQTTUtils.createPairedStream(ssc, brokerUrl, topics) ) the expected contents
> of mqttstream should be a collections of tuples:
> (topic,message) , (topic,message) , (topic,message) , ...
> Instead, the current content is a flattened list:
> topic, message, topic, message, topic, message, ...
> that is hard to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
