[
https://issues.apache.org/jira/browse/SPARK-8331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-8331.
------------------------------
Resolution: Invalid
I think this should start as a question on user@ since there are really a bunch
of questions here, not an issue report. There is no real info here about how
you are running this or basic things like whether you're using more than 1
executor.
> DirectStream wont work
> ----------------------
>
> Key: SPARK-8331
> URL: https://issues.apache.org/jira/browse/SPARK-8331
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.4.0
> Environment: all, Kafka 0.8.2.1
> Reporter: sid
>
> I created a topic and pumped in messages using
> kafka-console-producer < input file
> I use receiver based stream; and it works perfect
> JavaPairReceiverInputDStream<String, String> dstream =
> KafkaUtils.createStream(ssc, props.getProperty("zookeeper.connect"),
> props.getProperty("group.id"), topics);
> But when i use Direct Stream; it just sits there hanging
> JavaPairInputDStream<String, String> dstream =
> KafkaUtils.createDirectStream(ssc, String.class, String.class,
> StringDecoder.class, StringDecoder.class, params, topics);
>
> my params like group.id are simply ignored
> Property auto.offset.reset is overridden to smallest
> Property client.id is overridden to sparkTestLocal
> Property group.id is overridden to
> Property zookeeper.connect is overridden to
> Looking into code both KafkaCluster() using String encode/decode.
> Also since we are NOT passing number of partitions; i am guess all the 10
> partitions i have should be receiving in 10 threads i created
> I even manually gen kafka-console-producer; while the DirectStream is
> running.
> Just lies dead!!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
