Re: java.lang.ClassCastException: java.lang.Long cannot be cast to scala.Tuple2
@ankur - I have also seen this recently. Is there a patch available for this issue? (in my recent experience on non-graphx apps, sort based shuffle looks better while dealing with memory pressure...) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-ClassCastException-java-lang-Long-cannot-be-cast-to-scala-Tuple2-tp13926p14501.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Compiling Spark master (284771ef) with sbt/sbt assembly fails on EC2
I also ran into same issue. What is the solution? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Compiling-Spark-master-284771ef-with-sbt-sbt-assembly-fails-on-EC2-tp11155p11189.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Readin from Amazon S3 behaves inconsistently: return different number of lines...
@sean - I am using latest code from master branch, up to commit# a7d145e98c55fa66a541293930f25d9cdc25f3b4 . In my case I have multiple directories with 1024 files(in that sizes of files may be different). For some directories I always get consistent result... and for others I can reproduce the inconsistent behavior. I am not much familiar with S3 protocol or s3 driver in spark. I am wondering, how does s3 driver verifies that all files(and their content) under a directory were correctly? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Reading-from-Amazon-S3-directory-via-textFile-api-behaves-inconsistently-tp11092p11170.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Spark job finishes then command shell is blocked/hangs?
which version of spark are you running? have you tried sc.stop as as last line of your program? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-job-finishes-then-command-shell-is-blocked-hangs-tp11095p11097.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Installing Spark 0.9.1 on EMR Cluster
Have you tried flag " --spark-version" of spark-ec2 ? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Installing-Spark-0-9-1-on-EMR-Cluster-tp11084p11096.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Readin from Amazon S3 behaves inconsistently: return different number of lines...
*First Question:*
On Amazon S3 I have a directory with 1024 files, where each file size is
~9Mb; and each line in a file has two entries separated by '\t'.
Here is my program, which is calculating total number of entries in the
dataset
--
val inputId = sc.textFile(inputhPath, noParts).flatMap {line=>
val lineArray = line.split("\\t")
Iterator(lineArray(0).toLong, lineArray(1).toLong)
}.distinct(noParts)
println("##input-cnt = %s; ".
format(inputId.count))
--
Where inputpath =
"s3n://my-AWS_ACCESS_KEY_ID:myAWS_ACCESS_KEY_SECRET@bucket-id/directory"
When I run this program multiple times on EC2, "input-cnt" across
iterations is not consistent. FYI, I uploaded the data to S3 two days back;
so I assume by now data is properly replicated/(eventually-concistency).
*
Is this a known issue with S3? What it the solution?
*
Note: When I ran same experiment on my yarn cluster; where inputhPath is
hdfs-path, I got the results as expected.
*Second Question:*
How can I copy data from s3 to ephemeral-hdfs? I tried distcp, but I got
this error:
--
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.net.ConnectException: Call to
ec2-54-227-208-124.compute-1.amazonaws.com/10.187.1y.120:9001 failed on
connection exception: java.net.ConnectException: Connection refused
--
(ideally I can use sc.textfile to read and write to hdfs; but I am not sure
if I will get all the data with sc.textFile due to the issue I mentioned
above)
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Readin-from-Amazon-S3-behaves-inconsistently-return-different-number-of-lines-tp11092.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
spark-ec2 script with Tachyon
Hi, It seems that spark-ec2 script deploys Tachyon module along with other setup. I am trying to use .persist(OFF_HEAP) for RDD persistence, but on worker I see this error -- Failed to connect (2) to master localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused -- >From netstat I see that worker is connected to master node on port 19998 -- Proto Recv-Q Send-Q Local Address Foreign Address State tcp0 0 ip-10-16-132-190.ec2.:49239 ip-10-158-45-248.ec2.:19998 ESTABLISHED -- Does Tachyon on EC work out of the box? or does it requite further configuration ? Am I supposed to set "spark.tachyonStore.url" to Masters IP ? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-ec2-script-with-Tachyon-tp9996.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Yay for 1.0.0! EC2 Still has problems.
I am also running into "modules/mod_authn_alias.so" issue on r3.8xlarge when launched cluster with ./spark-ec2; so ganglia is not accessible. From the posts it seems that Patrick suggested using Ubuntu 12.04. Can you please provide name of AMI that can be used with -a flag that will not have this issue? - I am running script with "--spark-git-repo=https://github.com/apache/spark";, which I assume should deploy the latest code. - I have been able to launch cluster on m2.4xlarge, where ganglia works. - From what I understand we are not supposed to use any random AMI??; it will be helpful to publish list of AMIS that people use with different instances. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Yay-for-1-0-0-EC2-Still-has-problems-tp6578p9307.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: Yay for 1.0.0! EC2 Still has problems.
I am also running into "modules/mod_authn_alias.so" issue on r3.8xlarge when launched cluster with ./spark-ec2; so ganglia is not accessible. From the posts it seems that Patrick suggested using Ubuntu 12.04. Can you please provide name of AMI that can be used with -a flag that will not have this issue? - I am running script with "--spark-git-repo=https://github.com/apache/spark";, which I assume should deploy the latest code. - I have been able to launch cluster on m2.4xlarge, where ganglia works. - From what I understand we are not supposed to use any random AMI??; it will be helpful to publish list of AMIS that people use with different instances. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Yay-for-1-0-0-EC2-Still-has-problems-tp6578p9306.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
java.io.FileNotFoundException: shuffle
Hi, I am running my spark job on Yarn; using latest code from master branch..synced few days back. I see this IO Exception during shuffle(in resource manager logs). What could be wrong and how to debug it? I have seen this few times before; I was suspecting that this could side effect of memory pressure..but I could never figure out the root cause. -- 14/07/02 07:34:45 WARN TaskSetManager: Loss was due to java.io.FileNotFoundException java.io.FileNotFoundException: /var/storage/sda3/nm-local/usercache/nit/appcache/application_1403208801430_0183/spark-local-20140702065054-388d/0e/shuffle_3_193_787 (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.(FileOutputStream.java:221) at org.apache.spark.storage.DiskBlockObjectWriter.open(BlockObjectWriter.scala:116) at org.apache.spark.storage.DiskBlockObjectWriter.write(BlockObjectWriter.scala:177) at org.apache.spark.scheduler.ShuffleMapTask$$anonfun$runTask$1.apply(ShuffleMapTask.scala:161) at org.apache.spark.scheduler.ShuffleMapTask$$anonfun$runTask$1.apply(ShuffleMapTask.scala:158) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:158) at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) at org.apache.spark.scheduler.Task.run(Task.scala:51) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:187) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) -- -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-io-FileNotFoundException-shuffle-tp8644.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
