I wasn’t thorough, the complete stderr includes:
g++: /usr/lib64/libaprutil-1.so: No such file or directory
g++: /usr/lib64/libapr-1.so: No such file or directoryn
(including that trailing ’n')
Though I can’t figure out how the process indirection is going from the
frontend spark application to mesos executors and where this shared library
error comes from.
Hope someone can shed some light,
Thanks
On 08 Feb 2015, at 14:15, Hans van den Bogert <hansbog...@gmail.com> wrote:
> Hi,
>
>
> I’m trying to get coarse mode to work under mesos(0.21.0), I thought this
> would be a trivial change as Mesos was working well in fine-grained mode.
>
> However the mesos tasks fail, I can’t pinpoint where things go wrong.
>
> This is a mesos stderr log from a slave:
>
> Fetching URI 'http://upperpaste.com/spark-1.2.0-bin-hadoop2.4.tgz'
> I0208 12:57:45.415575 25720 fetcher.cpp:126] Downloading
> 'http://upperpaste.com/spark-1.2.0-bin-hadoop2.4.tgz' to
> '/local/vdbogert/var/lib/mesos//slaves/20150206-110658-16813322-5050-5515-S1/frameworks/20150208-125721-906005770-5050-32371-0000/executors/0/runs/cb525b32-387c-4698-a27e-8d4213080151/spark-1.2.0-bin-hadoop2.4.tgz'
> I0208 12:58:09.146960 25720 fetcher.cpp:64] Extracted resource
> '/local/vdbogert/var/lib/mesos//slaves/20150206-110658-16813322-5050-5515-S1/frameworks/20150208-125721-906005770-5050-32371-0000/executors/0/runs/cb525b32-387c-4698-a27e-8d4213080151/spark-1.2.0-bin-hadoop2.4.tgz'
> into
> '/local/vdbogert/var/lib/mesos//slaves/20150206-110658-16813322-5050-5515-S1/frameworks/20150208-125721-906005770-5050-32371-0000/executors/0/runs/cb525b32-387c-4698-a27e-8d4213080151’
>
> Mesos slaves' stdout are empty.
>
>
> And I can confirm the spark distro is correctly extracted:
> $ ls
> spark-1.2.0-bin-hadoop2.4 spark-1.2.0-bin-hadoop2.4.tgz stderr stdout
>
> The spark-submit log is here:
> http://pastebin.com/ms3uZ2BK
>
> Mesos-master
> http://pastebin.com/QH2Vn1jX
>
> Mesos-slave
> http://pastebin.com/DXFYemix
>
>
> Can somebody pinpoint me to logs, etc to further investigate this, I’m
> feeling kind of blind.
> Furthermore, do the executors on mesos inherit all configs from the spark
> application/submit? E.g. I’ve given my executors 20GB of memory through a
> spark-submit "—conf” parameter. Should these settings also be present in the
> spark-1.2.0-bin-hadoop2.4.tgz distribution’s configs?
>
> If, in order to be helped here, I need to present more logs etc, please let
> me know.
>
> Regards,
>
> Hans van den Bogert
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
