RE: Running Spark/YARN on AWS EMR - Issues finding file on hdfs?

Any resolution to this? Im having the same problem. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-Spark-YARN-on-AWS-EMR-Issues-finding-file-on-hdfs-tp10214p22918.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: zip files submitted with --py-files disappear from hdfs after a while on EMR

Any resolution to this? I am having the same problem. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/zip-files-submitted-with-py-files-disappear-from-hdfs-after-a-while-on-EMR-tp22342p22919.html Sent from the Apache Spark User List mailing list archive at

Re: Difference among batchDuration, windowDuration, slideDuration

I think hsy541 is still confused by what is still confusing to me. Namely, what is the value that sentence Each RDD in a DStream contains data from a certain interval is speaking of? This is from the Discretized Streams

spark flume tryOrIOException NoSuchMethodError

I am trying to process events from a flume avro sink, but i keep getting this same error. I am just running it locally using flumes avro-client. With the following commands to start the job and client. It seems like it should be a configuration problems since its a NoSuchMethodError, but