[
https://issues.apache.org/jira/browse/BAHIR-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270200#comment-16270200
]
ASF GitHub Bot commented on BAHIR-104:
--------------------------------------
GitHub user zubairnabi-intech opened a pull request:
https://github.com/apache/bahir/pull/55
[BAHIR-104] Multi-topic MQTT DStream in Python is now a PairRDD.
Fixing [BAHIR-104].
List of topics passed in Python converted to a Java String array before
passing it to Py4J.
Test added for this change.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/intech-iiot/bahir mqtt-python-paired-stream
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bahir/pull/55.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #55
----
commit 98d52c70db95e455b677a888aba3f5c369cab2a1
Author: Zubair Nabi <[email protected]>
Date: 2017-11-29T05:45:28Z
[BAHIR-104] Multi-topic MQTT DStream in Python is now a PairRDD.
----
> MQTT Dstream returned by the new multi topic support API is not a pairRDD
> -------------------------------------------------------------------------
>
> Key: BAHIR-104
> URL: https://issues.apache.org/jira/browse/BAHIR-104
> Project: Bahir
> Issue Type: Bug
> Components: Spark Streaming Connectors
> Affects Versions: Spark-2.1.0
> Reporter: Francesco Beneventi
> Labels: MQTT, SPARK
>
> The new multi topic support API added with [BAHIR-89], when used in pyspark,
> does not return a Dstream of <topic,message> tuples.
> Example:
> In pyspark, when creating a Dstream using the new API ( mqttstream =
> MQTTUtils.createPairedStream(ssc, brokerUrl, topics) ) the expected contents
> of mqttstream should be a collections of tuples:
> (topic,message) , (topic,message) , (topic,message) , ...
> Instead, the current content is a flattened list:
> topic, message, topic, message, topic, message, ...
> that is hard to use.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
