Re: Spark task hangs infinitely when accessing S3 from AWS

2016-01-27 Thread Erisa Dervishi
Hi, I think I have the same issue mentioned here: https://issues.apache.org/jira/browse/SPARK-8898 I tried to run the job with 1 core and it didn't hang anymore. I can live with that for now, but any suggestions are welcome. Erisa On Tue, Jan 26, 2016 at 4:51 PM, Erisa Dervishi <er

Re: Spark task hangs infinitely when accessing S3 from AWS

2016-01-26 Thread Erisa Dervishi
.runWorker(ThreadPoolExecutor.java:1142) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) java.lang.Thread.run(Thread.java:745) On Tue, Jan 26, 2016 at 4:26 PM, Erisa Dervishi <erisa...@gmail.com> wrote: > I have quite a different situation though. > My job w

Re: Spark task hangs infinitely when accessing S3 from AWS

2016-01-26 Thread Erisa Dervishi
Hi, I kind am in your situation now while trying to read from S3. Where you able to find a workaround in the end? Thnx, Erisa On Thu, Nov 12, 2015 at 12:00 PM, aecc <alessandroa...@gmail.com> wrote: > Some other stats: > > The number of files I have in the folder is 4

Re: Efficient way to get top K values per key in (key, value) RDD?

2015-06-10 Thread erisa
, Erisa -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Efficient-way-to-get-top-K-values-per-key-in-key-value-RDD-tp20370p23263.html Sent from the Apache Spark User List mailing list archive at Nabble.com