Hi,
I’m trying to get coarse mode to work under mesos(0.21.0), I thought this would
be a trivial change as Mesos was working well in fine-grained mode.
However the mesos tasks fail, I can’t pinpoint where things go wrong.
This is a mesos stderr log from a slave:
Fetching URI 'http://upperpaste.com/spark-1.2.0-bin-hadoop2.4.tgz'
I0208 12:57:45.415575 25720 fetcher.cpp:126] Downloading
'http://upperpaste.com/spark-1.2.0-bin-hadoop2.4.tgz' to
'/local/vdbogert/var/lib/mesos//slaves/20150206-110658-16813322-5050-5515-S1/frameworks/20150208-125721-906005770-5050-32371-0000/executors/0/runs/cb525b32-387c-4698-a27e-8d4213080151/spark-1.2.0-bin-hadoop2.4.tgz'
I0208 12:58:09.146960 25720 fetcher.cpp:64] Extracted resource
'/local/vdbogert/var/lib/mesos//slaves/20150206-110658-16813322-5050-5515-S1/frameworks/20150208-125721-906005770-5050-32371-0000/executors/0/runs/cb525b32-387c-4698-a27e-8d4213080151/spark-1.2.0-bin-hadoop2.4.tgz'
into
'/local/vdbogert/var/lib/mesos//slaves/20150206-110658-16813322-5050-5515-S1/frameworks/20150208-125721-906005770-5050-32371-0000/executors/0/runs/cb525b32-387c-4698-a27e-8d4213080151’
Mesos slaves' stdout are empty.
And I can confirm the spark distro is correctly extracted:
$ ls
spark-1.2.0-bin-hadoop2.4 spark-1.2.0-bin-hadoop2.4.tgz stderr stdout
The spark-submit log is here:
http://pastebin.com/ms3uZ2BK
Mesos-master
http://pastebin.com/QH2Vn1jX
Mesos-slave
http://pastebin.com/DXFYemix
Can somebody pinpoint me to logs, etc to further investigate this, I’m feeling
kind of blind.
Furthermore, do the executors on mesos inherit all configs from the spark
application/submit? E.g. I’ve given my executors 20GB of memory through a
spark-submit "—conf” parameter. Should these settings also be present in the
spark-1.2.0-bin-hadoop2.4.tgz distribution’s configs?
If, in order to be helped here, I need to present more logs etc, please let me
know.
Regards,
Hans van den Bogert
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
