esos slaves also run Spark executor docker container in
>>>>>>> --net=host mode, so that the AWS profile of Mesos slaves will take
>>>>>>> effect.
>>>>>>>
>>>>>>> Hope it helps,
>>>>>>
ogic.com> wrote:
>>>>>
>>>>>> Hi Sathish,
>>>>>>
>>>>>> The docker image is normal, no AWS profile included.
>>>>>>
>>>>>> When the driver container runs with --net=host, the driver host's AWS
>&g
runs with --net=host, the driver host's AWS
>>>> profile will take effect so that the driver can access the protected s3
>>>> files.
>>>>
>>>> Similarly, Mesos slaves also run Spark executor docker container in
>>>> --net=host mode, so that th
host's AWS
>>>>> profile will take effect so that the driver can access the protected s3
>>>>> files.
>>>>>
>>>>> Similarly, Mesos slaves also run Spark executor docker container in
>>>>> --net=host mode,
es also run Spark executor docker container in
>>> --net=host mode, so that the AWS profile of Mesos slaves will take effect.
>>>
>>> Hope it helps,
>>> Mao
>>>
>>> On Jan 26, 2016, at 9:15 PM, Sathish Kumaran Vairavelu <
>>> vsathish
On Jan 26, 2016, at 9:15 PM, Sathish Kumaran Vairavelu <
>> vsathishkuma...@gmail.com> wrote:
>>
>> Hi Mao,
>>
>> I want to check on accessing the S3 from Spark docker in Mesos. The EC2
>> instance that I am using has the AWS profile/IAM included.
Jan 26, 2016, at 9:15 PM, Sathish Kumaran Vairavelu <
> vsathishkuma...@gmail.com> wrote:
>
> Hi Mao,
>
> I want to check on accessing the S3 from Spark docker in Mesos. The EC2
> instance that I am using has the AWS profile/IAM included. Should we build
> the docker image with any A
=host
mode, so that the AWS profile of Mesos slaves will take effect.
Hope it helps,
Mao
> On Jan 26, 2016, at 9:15 PM, Sathish Kumaran Vairavelu
> <vsathishkuma...@gmail.com> wrote:
>
> Hi Mao,
>
> I want to check on accessing the S3 from Spark docker in Mesos. The E
Hi Mao,
I want to check on accessing the S3 from Spark docker in Mesos. The EC2
instance that I am using has the AWS profile/IAM included. Should we build
the docker image with any AWS profile settings or --net=host docker option
takes care of it?
Please help
Thanks
Sathish
On Tue, Jan 26
Hi there,
I am trying to run Spark on Mesos using a Docker image as executor, as
mentioned
http://spark.apache.org/docs/latest/running-on-mesos.html#mesos-docker-support
.
I built a docker image using the following Dockerfile (which is based on
Hi Mao,
Can you try --jars to include those jars?
Best Regards,
Jerry
Sent from my iPhone
> On 26 Jan, 2016, at 7:02 pm, Mao Geng wrote:
>
> Hi there,
>
> I am trying to run Spark on Mesos using a Docker image as executor, as
> mentioned
>
image already has
them), but that's ok - I am happy seeing Spark + Mesos + Docker + S3 worked
together!
Thanks,
Mao
16/01/27 02:54:45 INFO Executor: Using REPL class URI: http://172.16.3.98:33771
16/01/27 02:55:12 INFO CoarseGrainedExecutorBackend: Got assigned task 0
16/01/27 02:55:12 INFO Exec
12 matches
Mail list logo