Hi Guys, I am doing some tests with JavaKafkaWordCount, my cluster is composed by 8 workers and 1 driver con spark-1.1.0, I am using Kafka too and I have some questions about.
1 - When I launch the command: bin/spark-submit --class org.apache.spark.examples.streaming.JavaKafkaWordCount —master spark://computer8:7077 --driver-memory 1g --executor-memory 2g --executor-cores 2 examples/target/scala-2.10/spark-examples-1.1.0-hadoop1.0.4.jar computer49:2181 test-consumer-group test 2 I see in the Spark WebAdmin that only 1 worker work. Why? 2 - In Kafka I can see the same thing: Group Topic Pid Offset logSize Lag Owner test-consumer-group test 0 147092 147092 0 test-consumer-group_computer1-1416319543858-817b566f-0 test-consumer-group test 1 232183 232183 0 test-consumer-group_computer1-1416319543858-817b566f-0 test-consumer-group test 2 186805 186805 0 test-consumer-group_computer1-1416319543858-817b566f-0 test-consumer-group test 3 0 0 0 test-consumer-group_computer1-1416319543858-817b566f-1 test-consumer-group test 4 0 0 0 test-consumer-group_computer1-1416319543858-817b566f-1 test-consumer-group test 5 0 0 0 test-consumer-group_computer1-1416319543858-817b566f-1 I would like to understand this behavior, Is it normal? Am I doing something wrong? Thanks Guys -- Informativa sulla Privacy: http://www.unibs.it/node/8155
