Using the spark-ec2 script with m3.2xlarge instances seems to not have /mnt
and /mnt2 pointing to the 80gb SSDs that come with that instance. Does
anybody know whether extra steps are required when using this instance type?
Thanks,
Chris
I pulled the latest last night. I'm on commit 4da01e3.
On Sun, Jul 20, 2014 at 2:08 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Is this with the 1.0.0 scripts? I believe it's fixed in 1.0.1.
Matei
On Jul 20, 2014, at 1:22 AM, Chris DuBois chris.dub...@gmail.com wrote:
Using
and
using the default settings (don't set anything in SparkConf). Submit
the application with --driver-memory 20g.
The running times are slower than what I remember, but it depends on
the instance type.
Best,
Xiangrui
On Wed, Jul 16, 2014 at 10:18 PM, Chris DuBois chris.dub...@gmail.com
wrote
Hi all,
I am encountering the following error:
INFO scheduler.TaskSetManager: Loss was due to java.io.IOException: No
space left on device [duplicate 4]
For each slave, df -h looks roughtly like this, which makes the above error
surprising.
FilesystemSize Used Avail Use% Mounted
(df -i). The assembly build may create many
small files. -Xiangrui
On Tue, Jul 15, 2014 at 11:35 PM, Chris DuBois chris.dub...@gmail.com
wrote:
Hi all,
I am encountering the following error:
INFO scheduler.TaskSetManager: Loss was due to java.io.IOException: No
space
left on device
, Chris DuBois chris.dub...@gmail.com
wrote:
Hi all,
I am encountering the following error:
INFO scheduler.TaskSetManager: Loss was due to java.io.IOException: No
space
left on device [duplicate 4]
For each slave, df -h looks roughtly like this, which makes the above
error
this setting as the default value.
Best,
Xiangrui
On Wed, Jul 16, 2014 at 12:02 AM, Chris DuBois chris.dub...@gmail.com
wrote:
Thanks for the quick responses!
I used your final -Dspark.local.dir suggestion, but I see this during the
initialization of the application:
14/07/16 06:56
you
recommend?
Chris
On Wed, Jul 16, 2014 at 1:09 AM, Chris DuBois chris.dub...@gmail.com
wrote:
Hi Xiangrui,
Here is the result on the master node:
$ df -i
FilesystemInodes IUsed IFree IUse% Mounted on
/dev/xvda1524288 273997 250291 53% /
tmpfs
Hi Ben,
It worked for me, but only when using the default region. Using
--region=us-west-2 resulted in errors about security groups.
Chris
On Wed, Jul 16, 2014 at 8:53 AM, Ben Horner ben.hor...@atigeo.com wrote:
please add
From: Ben Horner [via Apache Spark User List] [hidden email]
/spark and /mnt2/spark as the
default local directory because they are local hard drives. Did your
last run of ALS on MovieLens 10M-100K with the default settings
succeed? -Xiangrui
On Wed, Jul 16, 2014 at 8:00 AM, Chris DuBois chris.dub...@gmail.com
wrote:
Hi Xiangrui,
I accidentally did
10 matches
Mail list logo