Oops, that should've been:
* {*
* "credentials": [*
* {*
* "principal": "mesos-mach5-beta",*
* "secret": " cGFzc3dvcmQ="*
* }*
* ]*
* }*
On Tue, Sep 29, 2015 at 8:20 PM Michael Park <mcyp...@gmail.com> wrote:
> I'll look into what's happening with the framework registration, but
> meanwhile I've also taken a look at what's going on with the master
> endpoints.
>
> It looks like the issue is around authentication rather than the dynamic
> reservation endpoints.
>
> When using *JSON-based* credentials file, the password should be given in
> *base64
> encoded* format.
> So your credentials file should be:
>
> * {*
> * "credentials": [*
> * {*
> * "principal": "mesos-mach5-beta",*
> * "secret": " cGFzc3dvcmQ"*
> * }*
> * ]*
> * }*
>
> It looks like this behavior is not documented well on the master. I'll be
> fixing that shortly.
>
> Thanks,
>
> MPark.
>
>
> On Tue, Sep 29, 2015 at 4:27 PM DiGiorgio, Mr. Rinaldo S. <
> rdigior...@pace.edu> wrote:
>
>> MPark,
>>
>> Thanks for your identification of something that was not configured. I
>> am using the mesos-plugin in Jenkins. I had not specified a principal. I
>> added the principal and it appears to register with that principal if I
>> don’t provide a password from the meson-plugin. When I try to perform a
>> reservation I get the authentication issue. So I thought perhaps the
>> framework must register with a password. I added the password and restarted
>> both jenkins and the meson-master. I get the following in the logs.
>>
>>
>> W0929 13:17:56.672214 346357760 master.cpp:5165] Failed to authenticate
>> scheduler-a6d69250-48dc-4474-8af8-89bb35fabb92@10.133.69.83:49817:
>> Refused authentication
>> *** Aborted at 1443557876 (unix time) try "date -d @1443557876" if you
>> are using GNU date ***
>> PC: @ 0x7fff8ad419a4 _pthread_mutex_check_init
>> *** SIGSEGV (@0x1) received by PID 62883 (TID 0x114ad3000) stack trace:
>> ***
>> @ 0x7fff8d09f5aa _sigtramp
>>
>> I am sure ntp is the same since I have set the time with ntp and both the
>> mesos master and jenkins are on the same machine. The authentication
>> process requires accurate clocks — since it is just the framework
>> registering I have not looked at the slaves.
>>
>>
>>
>> Rinaldo
>>
>>
>> On Sep 29, 2015, at 3:03 PM, Michael Park <mcyp...@gmail.com> wrote:
>>
>> Hi Rinaldo,
>>
>> Sorry that you're having trouble using dynamic reservations.
>>
>> I see that you're specifying the *mesos-mach5-beta* principal on the
>> resources, but I'm not sure if your framework is registered with the
>> *mesos-mach5-beta* principal? The framework must set the *
>> FrameworkInfo::principal* to be registered under that *principal*.
>>
>> Please let me know whether that is the case or not, and I'll follow up
>> with you to resolve the issue.
>>
>> Thanks,
>>
>> MPark.
>>
>> On Tue, Sep 29, 2015 at 1:53 PM DiGiorgio, Mr. Rinaldo S. <
>> rdigior...@pace.edu> wrote:
>>
>>> Joseph,
>>>
>>> I thought I tried that. So I must still not following the
>>> directions. Here is what I have?
>>>
>>> mesos master running on OS X 10.10.5 mesos 0.26
>>>
>>> I perform the following curl operation below.
>>>
>>> server reads credentials file
>>>
>>> I0929 10:48:42.062871 291536896 credentials.hpp:37] Loading credentials
>>> for authentication from '/etc/mesos-master/attributes/credentials'
>>> I0929 10:48:42.065512 291536896 master.cpp:467] Using default 'crammd5'
>>> authenticator
>>>
>>>
>>> The result of trying to reserve is: *Could not authenticate
>>> 'mesos-mach5-beta'*
>>>
>>> ======= the credentials file is =======
>>>
>>> {
>>> "credentials": [
>>> {
>>> "principal": "*mesos-mach5-beta*",
>>> "secret": "*password*"
>>> }
>>> ]
>>> }
>>> ===============================
>>>
>>> ========================User curl post to reserve a slave not a
>>> framework =============
>>> SLAVE_ID="efb748eb-e1ce-423d-a795-7589c92b2a32-S1"
>>> OPERATOR_PRINCIPAL="mach5"
>>> CPUS="3"
>>> MESOS_HOST="scaaa979.us.oracle.com:5050"
>>> curl -u "*mesos-mach5-beta*:password" -d slaveId="$SLAVE_ID" -d @- -X
>>> POST http://$MESOS_HOST/master/reserve <<HERE
>>> resources=[
>>> {
>>> "name": "cpus",
>>> "type": "SCALAR",
>>> "scalar": { "value": 8 },
>>> "role": "mach5",
>>> "reservation": {
>>> "principal": "*mesos-mach5-beta*"
>>> }
>>> },
>>> {
>>> "name": "mem",
>>> "type": "SCALAR",
>>> "scalar": { "value": 4096 },
>>> "role": "mach5",
>>> "reservation": {
>>> "principal": * "mesos-mach5-beta*"
>>> }
>>> }
>>> ]
>>> =================================================================
>>>
>>> On Sep 29, 2015, at 12:34 PM, Joseph Wu <jos...@mesosphere.io> wrote:
>>>
>>> Rinaldo,
>>>
>>> The principle is taken from authentication, rather than from the body of
>>> the resources. In this case, you'll be using Basic Authentication:
>>> https://en.wikipedia.org/wiki/Basic_access_authentication#Client_side
>>>
>>> With curl, you'd add something like: -H "Authorization: Basic
>>> bWVzb3MtbWFjaDUtYmV0YTpwYXNzd29yZA=="
>>> That base64 blurb is the encoded version of "mesos-mach5-beta:password".
>>>
>>> ~Joseph
>>>
>>> On Mon, Sep 28, 2015 at 8:25 PM, DiGiorgio, Mr. Rinaldo S. <
>>> rdigior...@pace.edu> wrote:
>>>
>>>>
>>>> On Sep 28, 2015, at 8:03 PM, Joseph Wu <jos...@mesosphere.io> wrote:
>>>>
>>>> Hi Rinaldo,
>>>>
>>>> I'd like to point out a small error in your ACLs.
>>>>
>>>> If you want to specify "ANY", you should set the "type" field. i.e.
>>>> For the RegisterFramework ACL:
>>>> "register_frameworks": [
>>>> {
>>>> "principals": { "values": "mesos-mach5-beta" },
>>>> "roles": { "type": 1 }
>>>> }
>>>> ]
>>>>
>>>>
>>>> Thanks — can’t keep my eyes open any more. This is the response I get
>>>> to the following request.
>>>>
>>>> *Invalid RESERVE operation: Cannot reserve resources without a
>>>> principal. *
>>>>
>>>> The example shows -u principal:password in curl which is
>>>> an auentycation string for the browser so I am totally confused on how to
>>>> provide a principal. The documentation for the framework reserve
>>>>
>>>>
>>>>
>>>> curl -i -d slaveId="$SLAVE_ID" -d @- -X POST
>>>> http://$MESOS_HOST/master/reserve <<HERE
>>>> resources=[
>>>> {
>>>> "name": "cpus",
>>>> "type": "SCALAR",
>>>> "scalar": { "value": 8 },
>>>> "role": "mach5",
>>>> "reservation": {
>>>> "principal": "mach5"
>>>> }
>>>> },
>>>> {
>>>> "name": "mem",
>>>> "type": "SCALAR",
>>>> "scalar": { "value": 4096 },
>>>> "role": "mach5",
>>>> "reservation": {
>>>> "principal": "mach5"
>>>> }
>>>> }
>>>> ]
>>>> <<HERE
>>>>
>>>>
>>>> The ANY "type" is part of an enumeration, defined here:
>>>>
>>>> https://github.com/apache/mesos/blob/master/include/mesos/authorizer/authorizer.proto#L33-L45
>>>>
>>>> Hope that helps,
>>>> ~Joseph
>>>>
>>>> On Mon, Sep 28, 2015 at 2:51 PM, DiGiorgio, Mr. Rinaldo S. <
>>>> rdigior...@pace.edu> wrote:
>>>>
>>>>>
>>>>> On Sep 28, 2015, at 5:27 PM, Marco Massenzio <ma...@mesosphere.io>
>>>>> wrote:
>>>>>
>>>>> Hi Rinaldo,
>>>>>
>>>>> sorry about the trouble you're having in getting this to work!
>>>>> If I got this one right, the original requirement was...
>>>>>
>>>>> I have some tasks that need to run on different types of agents.
>>>>>
>>>>>
>>>>> for that, I think you can use either (or both) of `roles` and
>>>>> `attributes` (see the Configuration doc [0] for more info).
>>>>>
>>>>> If you would like to run a 0.24 Mesos on your Mac for testing, you
>>>>> could use the Mesosphere published packages[1] or, if Vagrant is more your
>>>>> thing, feel free to "take inspiration" form [2].
>>>>>
>>>>> Marco,
>>>>>
>>>>> Thanks — We are running 0.23, 0.24 and the current branch as of
>>>>> this morning in three mesos environments with linux and mac nodes and
>>>>> working on porting Solaris. We have had various issues with building but
>>>>> are past most of them. We are making progess on the Solaris build and
>>>>> there is an issue with libsvn-1 as you mentioned with OL7.
>>>>>
>>>>>
>>>>> *Why do we need Dynamic Reservations?*
>>>>>
>>>>> We are also working with the mesos-plugin 0.8 and 0.9 and would like
>>>>> to change some of the behaviors of the plugin. One of the changes we want
>>>>> to make and we may move this out of the meson-plugin into workflow plugin
>>>>> in jenkins is to be able to reserve all the resources we need before we
>>>>> start a series of tasks. That is what we want to use dynamic reservations
>>>>> for. There may be issues with the jenkins workflow architecture in that
>>>>> “slaves” have to be requested via plugins. Mesos is new and I am sure it
>>>>> will provide a framework to innovate on all the following currently
>>>>> supported scheduling options in LSF.
>>>>>
>>>>> Fair share, preemptive, backfill and SLA scheduling
>>>>> High throughput scheduling
>>>>> Multicluster scheduling
>>>>> Topology-, resource-, and energy-aware scheduling
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I am trying to ask for a reservation and maybe I just don’t understand
>>>>> the definitions. I seem to be unsure about what a principal is. Maybe
>>>>> that
>>>>> is the root of my current issue. Unfortunately I am also a teacher so I
>>>>> notice things like I still can’t find a definition of *principal* on
>>>>> all those web pages.
>>>>>
>>>>> Thanks for all the links below but Docker is not a good technology for
>>>>> us because it has the usual linuxism’s runs best and mostly on Linux.
>>>>> Vagrant has the same issues so we will have to put more ports on our list.
>>>>> Docker don’t have separation that is equal to the task so we need to match
>>>>> the resources of the machine to the size of the task and not share in some
>>>>> circumstances. Our apps tend to open lots of ports and use advanced
>>>>> features of the operating system that may not be supported in Docker
>>>>> native, but may actually work in Docker on a VM. Containers have different
>>>>> definitions of separation.
>>>>>
>>>>> Rinaldo
>>>>>
>>>>>
>>>>> Finally, to build on OSX, you'll need to install libsvn-1 as described
>>>>> in [3].
>>>>>
>>>>> I'm afraid I don't know enough about Dynamic Reservation to really be
>>>>> able to help here; but I suspect that, if you run *without*
>>>>> authentication enabled, it will accept *any* principal (did you try
>>>>> that already? what error did you get?)
>>>>>
>>>>> Feel free to drop me a line if you're still having trouble.
>>>>>
>>>>>
>>>>> [0] http://mesos.apache.org/documentation/latest/configuration/
>>>>> [1] http://mesosphere.com/downloads
>>>>> [2] https://github.com/massenz/zk-mesos/tree/develop/vagrant
>>>>> [3] http://mesos.apache.org/gettingstarted/ (see the OSX section; in
>>>>> particular:
>>>>> `$ brew install autoconf automake libtool subversion maven`)
>>>>>
>>>>> *Marco Massenzio*
>>>>>
>>>>> *Distributed Systems Engineer http://codetrips.com
>>>>> <http://codetrips.com/>*
>>>>>
>>>>> On Mon, Sep 28, 2015 at 1:59 PM, DiGiorgio, Mr. Rinaldo S. <
>>>>> rdigior...@pace.edu> wrote:
>>>>>
>>>>>>
>>>>>> On Sep 21, 2015, at 7:33 PM, Guangya Liu <gyliu...@gmail.com> wrote:
>>>>>>
>>>>>> HI Rinaldo,
>>>>>>
>>>>>> I think that you can use dynamic reservation feature to achieve this:
>>>>>> You can launch your tasks after reservation succeeds. Actually, all of
>>>>>> the
>>>>>> dynamic reservation feature with endpoint has been finished except ACL
>>>>>> part, so you can use this feature now if you do not care ACL part.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> Hi Guangya,
>>>>>>
>>>>>> I have bene trying to get dynamic reservations to work. I downloaded
>>>>>> the latest from git and created a small environment on OS X 10.10. I am
>>>>>> trying to use reservations and I am not making much progress. I tried to
>>>>>> get it to work without authentication and was unable to. I used the ANY
>>>>>> option and it still required a principal. I am unable to configure the
>>>>>> master to work without authentication. Do you have some simple configs
>>>>>> for
>>>>>> starting a master with no authentication required so that it can be used
>>>>>> to
>>>>>> set dynamic reservations.
>>>>>>
>>>>>> The output below is for authentication. I tried to authenticate from
>>>>>> a slave and it failed with a coredump.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I start mesos like this:
>>>>>>
>>>>>> mesos-master.sh —ip=nnn,nnn,nnn,nnn --work_dir=/var/lib/mesos
>>>>>> --acls=$BASE/acls --credentials=$BASE/credentials
>>>>>>
>>>>>> bash-3.2# cat attributes/acls
>>>>>> {
>>>>>> "register_frameworks": [
>>>>>> {
>>>>>> "principals": { "type": "mesos-mach5-beta" },
>>>>>> "roles": { "values": "ANY" }
>>>>>> }
>>>>>> ],
>>>>>> "run_tasks": [
>>>>>> {
>>>>>> "principals": { "values": "ANY" },
>>>>>> "users": { "values": "ANY" }
>>>>>> }
>>>>>> ],
>>>>>> "shutdown_frameworks": [
>>>>>> {
>>>>>> "principals": { "values": "mesos-mach5-beta" },
>>>>>> "framework_principals": { "values": "ANY" }
>>>>>> }
>>>>>> ]
>>>>>> }
>>>>>>
>>>>>> bash-3.2# cat attributes/credentials
>>>>>> {
>>>>>> "credentials": [
>>>>>> {
>>>>>> "principal": "mesos-mach5-beta",
>>>>>> "secret": "password"
>>>>>> }
>>>>>> ]
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>> When I try the following I am told I am not authorized.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Guangya
>>>>>>
>>>>>> On Tue, Sep 22, 2015 at 6:32 AM, DiGiorgio, Mr. Rinaldo S. <
>>>>>> rdigior...@pace.edu> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have some tasks that need to run on different types of agents.
>>>>>>> I don’t want the tasks to run unless I am going to have all the
>>>>>>> resources.
>>>>>>> Can someone suggest how I could accomplish that with mesos. I read
>>>>>>> about
>>>>>>> reservations here:
>>>>>>> http://mesos.apache.org/documentation/latest/reservation/
>>>>>>>
>>>>>>> I could iterate over all the resources I need and if I get them
>>>>>>> proceed.
>>>>>>>
>>>>>>> Is that the only way to do it?
>>>>>>>
>>>>>>> Any idea when coming soon will be available?
>>>>>>>
>>>>>>> /reserve (*Coming Soon*)
>>>>>>>
>>>>>>> Suppose we want to reserve 8 CPUs and 4096 MB of RAM for the ads role
>>>>>>> on a slave with id=<slave_id>. We send an HTTP POST request to the
>>>>>>> /reserve HTTP endpoint like so:
>>>>>>>
>>>>>>>
>>>>>>> Rinaldo
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
