Can you check on the NFS server to see if the filesystem has been exported
with the rootsquash option? That's a commonly used option which converts
root uid on NFS clients to nobody on the server.
-- Paul Brett
On May 10, 2015 5:15 PM, "Adam Bordelon" <a...@mesosphere.io> wrote:
> Go ahead and run `env` in your script too, and see if there are any
> interesting differences when run via Marathon vs. directly.
> Maybe you're running in a different shell?
>
> On Sun, May 10, 2015 at 2:21 PM, John Omernik <j...@omernik.com> wrote:
>
>> I believe the slave IS running as root. FWIW when I ran the script from
>> above as root, it did work as intended (created the files on the NFS
>> share).
>>
>> On Sun, May 10, 2015 at 9:08 AM, Dick Davies <d...@hellooperator.net>
>> wrote:
>>
>>> Any idea what user mesos is running as? This could just be a
>>> filesystem permission
>>> thing (ISTR last time I used NFS mounts, they had a 'root squash'
>>> option that prevented
>>> local root from writing to the NFS mount).
>>>
>>> On 9 May 2015 at 22:13, John Omernik <j...@omernik.com> wrote:
>>> > I am not specifying isolators. The Default? :) Is that a per slave
>>> setting?
>>> >
>>> > On Sat, May 9, 2015 at 3:33 PM, James DeFelice <
>>> james.defel...@gmail.com>
>>> > wrote:
>>> >>
>>> >> What isolators are you using?
>>> >>
>>> >> On Sat, May 9, 2015 at 3:48 PM, John Omernik <j...@omernik.com>
>>> wrote:
>>> >>>
>>> >>> Marco... great idea... thank you. I just tried it and it worked
>>> when I
>>> >>> had a /mnt/permtesting with the same permissions. So it appears
>>> something
>>> >>> to do with NFS and Mesos (Remember I tested just NFS that worked
>>> fine, it's
>>> >>> the combination that is causing this).
>>> >>>
>>> >>> On Sat, May 9, 2015 at 1:09 PM, Marco Massenzio <ma...@mesosphere.io
>>> >
>>> >>> wrote:
>>> >>>>
>>> >>>> Out of my own curiousity (sorry, I have no fresh insights into the
>>> issue
>>> >>>> here) did you try to run the script and write to a non-NFS mounted
>>> >>>> directory? (same ownership/permissions)
>>> >>>>
>>> >>>> This way we could at least find out whether it's something related
>>> to
>>> >>>> NFS, or a more general permission-related issue.
>>> >>>>
>>> >>>> Marco Massenzio
>>> >>>> Distributed Systems Engineer
>>> >>>>
>>> >>>> On Sat, May 9, 2015 at 5:10 AM, John Omernik <j...@omernik.com>
>>> wrote:
>>> >>>>>
>>> >>>>> Here is the testing I am doing. I used a simple script (run.sh) It
>>> >>>>> writes the user it is running as to stderr (so it's the same log
>>> as the
>>> >>>>> errors from file writing) and then tries to make a directory in
>>> nfs, and
>>> >>>>> then touch a file in nfs. Note: This script directly run works
>>> on every
>>> >>>>> node. You can see the JSON I used in marathon, and in the sandbox
>>> results,
>>> >>>>> you can see the user is indeed darkness and the directory cannot
>>> be created.
>>> >>>>> However when directly run, it the script, with the same user,
>>> creates the
>>> >>>>> directory with no issue. Now, I realize this COULD still be a
>>> NFS quirk
>>> >>>>> here, however, this testing points at some restriction in how
>>> marathon kicks
>>> >>>>> off the cmd. Any thoughts on where to look would be very helpful!
>>> >>>>>
>>> >>>>> John
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> Script:
>>> >>>>>
>>> >>>>> #!/bin/bash
>>> >>>>> echo "Writing whoami to stderr for one stop logging" 1>&2
>>> >>>>> whoami 1>&2
>>> >>>>> mkdir /mapr/brewpot/mesos/storm/test/test1
>>> >>>>> touch /mapr/brewpot/mesos/storm/test/test1/testing.go
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> Run Via Marathon
>>> >>>>>
>>> >>>>>
>>> >>>>> {
>>> >>>>> "cmd": "/mapr/brewpot/mesos/storm/run.sh",
>>> >>>>> "cpus": 1.0,
>>> >>>>> "mem": 1024,
>>> >>>>> "id": "permtest",
>>> >>>>> "user": "darkness",
>>> >>>>> "instances": 1
>>> >>>>> }
>>> >>>>>
>>> >>>>>
>>> >>>>> I0509 07:02:52.457242 9562 exec.cpp:132] Version: 0.21.0
>>> >>>>> I0509 07:02:52.462700 9570 exec.cpp:206] Executor registered on
>>> slave
>>> >>>>> 20150505-145508-1644210368-5050-8608-S0
>>> >>>>> Writing whoami to stderr for one stop logging
>>> >>>>> darkness
>>> >>>>> mkdir: cannot create directory
>>> `/mapr/brewpot/mesos/storm/test/test1':
>>> >>>>> Permission denied
>>> >>>>> touch: cannot touch
>>> `/mapr/brewpot/mesos/storm/test/test1/testing.go':
>>> >>>>> No such file or directory
>>> >>>>>
>>> >>>>>
>>> >>>>> Run Via Shell:
>>> >>>>>
>>> >>>>>
>>> >>>>> $ /mapr/brewpot/mesos/storm/run.sh
>>> >>>>> Writing whoami to stderr for one stop logging
>>> >>>>> darkness
>>> >>>>> darkness@hadoopmapr1:/mapr/brewpot/mesos/storm$ ls ./test/
>>> >>>>> test1
>>> >>>>> darkness@hadoopmapr1:/mapr/brewpot/mesos/storm$ ls ./test/test1/
>>> >>>>> testing.go
>>> >>>>>
>>> >>>>>
>>> >>>>> On Sat, May 9, 2015 at 3:14 AM, Adam Bordelon <a...@mesosphere.io>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> I don't know of anything inside of Mesos that would prevent you
>>> from
>>> >>>>>> writing to NFS. Maybe examine the environment variables set when
>>> running as
>>> >>>>>> that user. Or are you running in a Docker container? Those can
>>> have
>>> >>>>>> additional restrictions.
>>> >>>>>>
>>> >>>>>> On Fri, May 8, 2015 at 4:44 PM, John Omernik <j...@omernik.com>
>>> wrote:
>>> >>>>>>>
>>> >>>>>>> I am doing something where people may recommend against my
>>> course of
>>> >>>>>>> action. However, I am curious if there is "a way" basically I
>>> have a process
>>> >>>>>>> being kicked off in marathon that is trying to write to a nfs
>>> location. The
>>> >>>>>>> permissions of the user running the task and the nfs location
>>> are good. So
>>> >>>>>>> what component of mesos or marathon is keeping me from writing
>>> here ? ( I
>>> >>>>>>> am getting permission denied). Is this one of those things that
>>> is just not
>>> >>>>>>> allowed, or is there an option to pass to marathon to allow
>>> this? Thanks !
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>> Sent from my iThing
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> James DeFelice
>>> >> 585.241.9488 (voice)
>>> >> 650.649.6071 (fax)
>>> >
>>> >
>>>
>>
>>
>
