On my local system (8 core MBP) the Kinesis ASL example isn't working out of the box on a fresh build (Spark 1.5.2). I can see records going into the kinesis stream but the receiver is returning empty DStreams. The behavior is similar to an issue that's been discussed previously:
http://stackoverflow.com/questions/26941844/apache-spark-kinesis-sample-not-working http://apache-spark-user-list.1001560.n3.nabble.com/Having-problem-with-Spark-streaming-with-Kinesis-td19863.html#a19929 In those the feedback was that it is an issue related to there being sufficient cores allocated to both receive and process the incoming data. However, in my case I have attempted running with the default (local[*]) as well as local[2], local[4], and local[8] all with the same results. Is it possible that the actual number of worker threads is different from what's requested? Is there a way to check how many threads were actually allocated?
