Re: EC2 clusters ready in launch time + 30 seconds

2014-10-06 Thread David Rowe
to dev list once we get on top of our own product release and the bigtop work Nate -Original Message- From: David Rowe [mailto:davidr...@gmail.com] Sent: Thursday, October 02, 2014 4:44 PM To: Nicholas Chammas Cc: dev; Shivaram Venkataraman Subject

Re: BlockManager issues

2014-09-22 Thread David Rowe
I've run into this with large shuffles - I assumed that there was contention between the shuffle output files and the JVM for memory. Whenever we start getting these fetch failures, it corresponds with high load on the machines the blocks are being fetched from, and in some cases complete