- [Kinesis stream name]: The Kinesis stream that this streaming application receives from - The application name used in the streaming context becomes the Kinesis application name - The application name must be unique for a given account and region. - The Kinesis backend automatically associates the application name to the Kinesis stream using a DynamoDB table (always in the us-east-1 region) created during Kinesis Client Library initialization. - *Changing the application name or stream name can lead to Kinesis errors in some cases. If you see errors, you may need to manually delete the DynamoDB table.*
On Fri, May 8, 2015 at 2:06 PM, Mike Trienis <mike.trie...@orcsol.com> wrote: > Hi All, > > I am submitting the assembled fat jar file by the command: > > bin/spark-submit --jars /spark-streaming-kinesis-asl_2.10-1.3.0.jar > --class com.xxx.Consumer xxxx-0.1-SNAPSHOT.jar > > It reads the data file from kinesis using the stream name defined in a > configuration file. It turns out that it reads the data perfectly fine for > one stream name (i.e. the default), however, if I switch the stream name > and re-submit the jar, it no longer reads the data. From CloudWatch, I can > see that there is data put into the stream and spark is actually sending > get requests as well. However, it doesn't seem to be outputting the data. > > Has anyone else encountered a similar issue? Does spark cache the stream > name somewhere? I also have checkpointing enabled as well. > > Thanks, Mike. > > > > > >