On Wed, Mar 30, 2016 at 3:58 PM, Joshua Cohen <jco...@apache.org> wrote:
> Job names are not unique though, what would happen if multiple jobs had the
> same name (either across roles or across environments in the same role)?
>
Good point. They would conflict with each other, and I guess in that case
Mesos DNS should not be used with the cluster.
An alternative is {role}-{job name}, although there are still ways to
create conflict in such case (e.g. "role-dumy/test/job" and
"role/test/dummy-job" generates the same name).
I think the correct long term approach is to allow some way to configure
this information by task or job. I'm a bit hesitant to include new thrift
structures for this experiment, and maybe the idea of "TaskInfoDecorator"
(see my previous posts) would be more flexible?
>
> On Wed, Mar 30, 2016 at 5:33 PM, Zhitao Li <zhitaoli...@gmail.com> wrote:
>
> > Stephan,
> >
> > So I've managed to run the official Mesos DNS docker container
> > <https://hub.docker.com/r/mesosphere/mesos-dns/> under the Aurora
> vagrant
> > environment and get some SRV/A recorded pulled from Mesos master from
> > Aurora.
> >
> > Because Mesos DNS uses 'name' field if set with some string manipulation,
> > for the job 'vagrant/test/http_example_docker', my prototype generates
> > these DNS records:
> >
> > A record: vagranttesthttp-exampled.twitterscheduler.mesos
> > SRV record: _vagranttesthttp-exampled._tcp.twitterscheduler.mesos.
> >
> > If we want to make current prototype useful for Mesos DNS, I suggest we
> > change the name field to job name, which would generate record like:
> > A: http_example_docker.twitterscheduler.mesos
> > SRV: _http_example_docker._tcp.twitterscheduler.slave.mesos
> >
> > I'll update my patch after getting some signal from you. Thanks.
> >
> > On Fri, Mar 25, 2016 at 1:49 PM, Zhitao Li <zhitaoli...@gmail.com>
> wrote:
> >
> > > Hi Stephan,
> > >
> > > Thanks for looking at that prototype patch.
> > >
> > > I'll update the patch with the review comments, and probably add a
> global
> > > flag of "populate_discovery_info" to toggle this behavior.
> > >
> > > About the optional fields: I think it'll be hard to come up a good set
> of
> > > rules applicable to all orgs using Aurora + Mesos, because cluster
> > > management and service discovery stack could differ from org to org.
> > >
> > > In a recent Mesos work group, some experience folks (Jie Yu and Ben
> > > Mahler) mentioned some ideas of *TaskInfoDecorator, *which is some
> > > optional and configurable plugin on Aurora scheduler side to allow
> > operator
> > > to set additional fields before sending the message to Mesos. I like
> such
> > > idea because it would enable Aurora users to experiment faster. Do you
> > > think this is an interesting idea worth pursuing?
> > >
> > >
> > > On Fri, Mar 25, 2016 at 1:42 PM, Erb, Stephan <
> > stephan....@blue-yonder.com
> > > > wrote:
> > >
> > >> I had a closer look at the Mesos documentation, and a design document
> > >> might be unnecessary. Most of the values are optional. We can
> therefore
> > >> leave them out until we have a proper usecase for them.
> > >>
> > >> I left a couple of comments in the review request.
> > >> ________________________________________
> > >> From: Zhitao Li <zhitaoli...@gmail.com>
> > >> Sent: Tuesday, March 22, 2016 21:15
> > >> To: dev@aurora.apache.org
> > >> Subject: Re: Populate DiscoveryInfo in Mesos
> > >>
> > >> Hi Stephan,
> > >>
> > >> Sorry for the delay on follow up on this. I took a quick look at
> Aurora
> > >> code, and it's actually quite easy to pipe this information to Mesos
> > (see
> > >> https://reviews.apache.org/r/45177/ for quick prototype).
> > >>
> > >> I'll take a stab to see how I can get Mesos-DNS to work with this
> > >> prototype.
> > >>
> > >> IMO, if this is something the community is interested, the main
> > questions
> > >> would be 1) how various fields would be mapped in different Aurora
> > usages,
> > >> and 2) to which level should opt-in/opt-out configured for populating
> > such
> > >> information.
> > >>
> > >> I actually don't have too much insights on how these usage conventions
> > >> would be set (through command line of scheduler or job configuration?)
> > >>
> > >> Do you think a design doc is the best action here, or a more involved
> > >> questionnaire about which fields would be useful for community, or
> what
> > >> value they should take?
> > >>
> > >> On Mon, Mar 7, 2016 at 1:00 AM, Erb, Stephan <
> > stephan....@blue-yonder.com
> > >> >
> > >> wrote:
> > >>
> > >> > That sounds like a good idea! Great.
> > >> >
> > >> > If you go ahead with this, please be so kind and start by posting a
> > >> short
> > >> > design document here on mailinglist (similar to those here
> > >> >
> https://github.com/apache/aurora/blob/master/docs/design-documents.md
> > ,
> > >> > but probably shorter).
> > >> >
> > >> > This will allow us to split the discussion of the design from
> > discussing
> > >> > the actual implementation. I believe this is necessary, as the
> > >> > DiscoveryInfo protocol is quite flexible (
> > >> >
> > >>
> >
> http://mesos.apache.org/documentation/latest/app-framework-development-guide/
> > >> > ).
> > >> >
> > >> > Thanks,
> > >> > Stephan
> > >> >
> > >> >
> > >> > ________________________________________
> > >> > From: Zhitao Li <zhitaoli...@gmail.com>
> > >> > Sent: Monday, March 7, 2016 00:05
> > >> > To: dev@aurora.apache.org
> > >> > Subject: Populate DiscoveryInfo in Mesos
> > >> >
> > >> > Hi,
> > >> >
> > >> > It seems like Aurora does not populate the "discovery" field in
> either
> > >> > TaskInfo or ExecutorInfo in mesos.proto
> > >> > <
> > >> >
> > >>
> >
> https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L438
> > >> > >
> > >> > .
> > >> >
> > >> > I'm considering adding this to support retrieving port map in Mesos
> > >> > directly. This would enable us to discovery this information
> directly
> > >> from
> > >> > Mesos side, and also enables us to build one universal service
> > discovery
> > >> > solution for multiple frameworks including Aurora.
> > >> >
> > >> > If no objection, I'll create a JIRA ticket for this task.
> > >> >
> > >> > Thanks.
> > >> > --
> > >> > Cheers,
> > >> >
> > >> > Zhitao Li
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Cheers,
> > >>
> > >> Zhitao Li
> > >>
> > >
> > >
> > >
> > > --
> > > Cheers,
> > >
> > > Zhitao Li
> > >
> >
> >
> >
> > --
> > Cheers,
> >
> > Zhitao Li
> >
>
--
Cheers,
Zhitao Li
