I've noticed this happening when there was some dependency conflicts, and it is super hard to debug. It seems that the KinesisClientLibrary version in Spark 1.5.2 is 1.3.0, but it is 1.2.1 in Spark 1.5.1. I feel like that seems to be the problem...
Brian, did you verify that it works with the 1.6.0 branch? Thanks, Burak On Thu, Dec 10, 2015 at 11:45 AM, Brian London <brianmlon...@gmail.com> wrote: > Nick's symptoms sound identical to mine. I should mention that I just > pulled the latest version from github and it seems to be working there. To > reproduce: > > > 1. Download spark 1.5.2 from http://spark.apache.org/downloads.html > 2. build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests > clean package > 3. build/mvn -Pkinesis-asl -DskipTests clean package > 4. Then run simultaneously: > 1. bin/run-example streaming.KinesisWordCountASL [Kinesis app name] > [Kinesis stream name] [endpoint URL] > 2. bin/run-example streaming.KinesisWordProducerASL [Kinesis > stream name] [endpoint URL] 100 10 > > > On Thu, Dec 10, 2015 at 2:05 PM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> Hi Nick, >> >> Just to be sure: don't you see some ClassCastException in the log ? >> >> Thanks, >> Regards >> JB >> >> On 12/10/2015 07:56 PM, Nick Pentreath wrote: >> > Could you provide an example / test case and more detail on what issue >> > you're facing? >> > >> > I've just tested a simple program reading from a dev Kinesis stream and >> > using stream.print() to show the records, and it works under 1.5.1 but >> > doesn't appear to be working under 1.5.2. >> > >> > UI for 1.5.2: >> > >> > Inline image 1 >> > >> > UI for 1.5.1: >> > >> > Inline image 2 >> > >> > On Thu, Dec 10, 2015 at 5:50 PM, Brian London <brianmlon...@gmail.com >> > <mailto:brianmlon...@gmail.com>> wrote: >> > >> > Has anyone managed to run the Kinesis demo in Spark 1.5.2? The >> > Kinesis ASL that ships with 1.5.2 appears to not work for me >> > although 1.5.1 is fine. I spent some time with Amazon earlier in the >> > week and the only thing we could do to make it work is to change the >> > version to 1.5.1. Can someone please attempt to reproduce before I >> > open a JIRA issue for it? >> > >> > >> >> -- >> Jean-Baptiste Onofré >> jbono...@apache.org >> http://blog.nanthrax.net >> Talend - http://www.talend.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >>
