[ https://issues.apache.org/jira/browse/SPARK-15039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268675#comment-15268675 ]
Tsai Li Ming edited comment on SPARK-15039 at 5/3/16 1:16 PM: -------------------------------------------------------------- [~zsxwing] Nothing suspicious in the logs. The streaming tab has 1 receiver but has 0 events/sec [~jerryshao] Have not tested standalone mode. But `--master loca[*]` works. I will test in Standalone mode. was (Author: ltsai): [~zsxwing] Nothing suspcisious in the logs. The streaming tab has 1 receiver but has 0 events/sec [~jerryshao] Have not tested standalone mode. But `--master loca[*]` works. > Kinesis reciever does not work in Yarn > -------------------------------------- > > Key: SPARK-15039 > URL: https://issues.apache.org/jira/browse/SPARK-15039 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.6.0 > Environment: YARN > HDP 2.4.0 > Reporter: Tsai Li Ming > > Hi, > Using the pyspark kinesis example, it does not receive any messages from > Kinesis when submitting to a YARN cluster, though it is working fine when > using local mode. > {code} > spark-submit \ > --executor-cores 4 \ > --num-executors 4 \ > --packages > com.databricks:spark-redshift_2.10:0.6.0,com.databricks:spark-csv_2.10:1.4.0,org.apache.spark:spark-streaming-kinesis-asl_2.10:1.5.1 > > {code} > I had to downgrade the package to 1.5.1. 1.6.1 does not work too. > Not sure whether this is related to SPARK-12453 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org