[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738636#comment-14738636 ] Qian Zhang commented on MESOS-2841: --- Created a separate ticket [MESOS-3408|https://issues.apache.org/jira/browse/MESOS-3408] for it. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: James DeFelice > Labels: mesosphere > Fix For: 0.24.0 > > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738595#comment-14738595 ] James DeFelice commented on MESOS-2841: --- Yes. Should there be a separate ticket for that? -- James DeFelice 585.241.9488 (voice) 650.649.6071 (fax) > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: James DeFelice > Labels: mesosphere > Fix For: 0.24.0 > > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738212#comment-14738212 ] Qian Zhang commented on MESOS-2841: --- I see Labels has been added into include/mesos/mesos.proto, but not in include/mesos/v1/mesos.proto, do we want to add it into v1 too? > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: James DeFelice > Labels: mesosphere > Fix For: 0.24.0 > > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696913#comment-14696913 ] James DeFelice commented on MESOS-2841: --- No worries. Really want this in the next release since other projects have deps on this. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695852#comment-14695852 ] Neil Conway commented on MESOS-2841: [~jdef] Thanks for finishing this off! My apologies for being flaky (vacation/travel, etc.). I'm back now, in case any more work is needed here. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695826#comment-14695826 ] James DeFelice commented on MESOS-2841: --- I've addressed the outstanding review comments and submitted an updated patch for review: https://reviews.apache.org/r/37443/ > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650512#comment-14650512 ] James DeFelice commented on MESOS-2841: --- Thanks, that's perfect. Would *love* to see this land in 0.24. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644683#comment-14644683 ] Adam B commented on MESOS-2841: --- A framework can change some of the basic metadata (including labels after Neil's patch) just by reregistering with the master. For libmesos-based frameworks, this requires restarting the schedulerDriver, but for a pure-language binding, or the new HTTP API, it's just a matter of sending a new ReregisterFramework message with an updated FrameworkInfo. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1464#comment-1464 ] James DeFelice commented on MESOS-2841: --- The addition of a Labels field should be just fine for my primary use case (mesos-dns plugin for k8s). It sounds like [~BenWhitehead] wanted something a bit more dynamic and perhaps a structure that is more rich (had suggested "capabilities"). Has there been anything thinking around updating/patching FrameworkInfo metadata when a scheduler wants to change what's advertised? I think I ended up with the Meta-based proposal to try incorporating (a) an envelope for shared data, as per [~nnielsen]; (b) supports alternate implementations in the future (generic vs. something more rich); (c) an initial implementation (generic) that provided the simple Labels that I need. I don't want to over-engineer anything. Just trying to find a level of abstraction that balanced concerns. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644419#comment-14644419
]
James DeFelice commented on MESOS-2841:
---
Is it acceptable to expect that a framework could advertise a k/v pair, with a
v that is a URL that returns a info based on the the tasks that have been
started on the cluster (and supply a minimal http server for providing that
info)? For example:
{code}
kafka.brokerlist.url=http://kafka.marathon.mesos/discovery/brokerlist
{code}
Now that I write this it feels hackier than I thought at first.
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Improvement
>Reporter: James DeFelice
>Assignee: Neil Conway
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644060#comment-14644060 ] Adam B commented on MESOS-2841: --- This feels over-engineered. Why not just add a top-level Labels field (as Neil did in his RR below)? If we want to add more structured metadata, that can go at the top-level of FrameworkInfo too, just like webui_url, capabilities, name, etc. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644056#comment-14644056 ] Adam B commented on MESOS-2841: --- [~BenWhitehead] Re: what if mesos allowed frameworks to advertise Resources that could then be part of resource offers made to frameworks. We're considering this for Clusterwide resources (MESOS-2728). Since all resources are currently tied to specific slaves, we would first need to solve the problem of how the allocator offers up these non-local resources (optimistically on every offer, one piece at a time). > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice >Assignee: Neil Conway > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591265#comment-14591265
]
James DeFelice commented on MESOS-2841:
---
Better? (I added the "subject" concept because it seems useful while still
being very generic)
{code}
message FrameworkInfo {
...
// discovery envelope, a framework metadata warehouse, currently
// supports a super-generic key-value format.
message Meta {
enum Type {
GENERIC = 1;
}
message Generic {
required string subject = 1 [default = "*"]; // what does the kv list
describe?
optional Labels labels = 2;
}
required Type type = 1;
optional Generic generic = 2;
}
repeated Meta meta = 13; // 13, or whatever comes next
... // other framework fields
}
{code}
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Epic
>Reporter: James DeFelice
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590805#comment-14590805
]
Ben Whitehead commented on MESOS-2841:
--
I think structured and unstructured can be useful and it probably makes sense
to start with the latter.
I do feel like it is odd to have the field on {{FrameworkInfo}}, since the only
way to send framework info to mesos is to pass it when the scheduler driver is
constructed.
This info seems more dynamic than a registration time thing.
For example if you have a framework that would like to list info based on tasks
that are started in the cluster (kafka broker list for example). This is
something that isn't known at constructions time, and can change over the life
of the process.
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Epic
>Reporter: James DeFelice
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590801#comment-14590801 ] Ben Whitehead commented on MESOS-2841: -- Makes sense, I used the word "Resource" in an abstract sense not in a mesos sense, none of Handle, Endpoint or Service seemed appropriate to me. Resource is probably also wrong given the connotation it already has in mesos. Sorry for the confusion. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590640#comment-14590640 ] Niklas Quarfot Nielsen commented on MESOS-2841: --- I am a bit confused about reusing the notion of 'DiscoveryInfo' taken it is used in the task info's. It should probably follow the format like QoSController or Operation :) > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590617#comment-14590617
]
James DeFelice commented on MESOS-2841:
---
It sounds like you're advocating for a structure that looks something like:
{code}
message FrameworkInfo {
...
// discovery envelope, a framework metadata warehouse currently
// represented in a super-generic key-value format.
message DiscoveryInfo {
enum Type {
GENERIC = 1;
}
required type Type = 1;
optional labels Labels = 2;
}
repeated discoveryInfo DiscoveryInfo = 13; // or whatever comes next
... // other framework fields
}
{code}
Does that capture the envelope concept? (do we even need a nested Type?)
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Epic
>Reporter: James DeFelice
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590546#comment-14590546 ] Niklas Quarfot Nielsen commented on MESOS-2841: --- Labels are trading off structure for generality. I think both key/value pairs _and_ structured capabilities could be useful. I have a hard time reasoning that these are resources though and the resource math becomes complicated. It is small change to enable the framework info to set these key value pairs until we have widespread use-cases for structured 'capabilities' (for example; messages describing 'I am a storage framework and can be accessed this way, etc.') Wrapping Labels in a envelope message would give us freedom to do that. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590482#comment-14590482 ] James DeFelice commented on MESOS-2841: --- My initial goal was to get a simple message/structure into Framework that we could use for discovery purposes. Understand your concerns about long term affects. The DiscoveryInfo message has Ports -- doesn't really make sense for frameworks. With respect to the other fields, it's arguable whether they're all valuable for framework discovery. But I do like the Labels :) Perhaps I misunderstand how the term Resource is used within mesos -- has this changed recently? My understanding was that Resources are allocatable/freeable things that are advertised to other frameworks within resource offers. In the above examples, the "resources" you've listed look more like advertisements for *usable* things that don't have any specific capacity constraints. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590162#comment-14590162
]
Ben Whitehead commented on MESOS-2841:
--
Ah, missed the {{uri}} vs {{url}} subtlety. Read Endpoint and though URL.
URN is interesting, though looking at the examples you've show each of the
scenarios looks like a new message type to me. I think mesos has taken the
approach of specific message types rather than abstract URNs for representing
things.
For example
{code}
// define a storage driver
storageResource = mesos.Storage{
acl: {
visibility: Mesos,
role: '*'
},
version: "v1.*"
}
// define an API resource
apiResource = mesos.HttpResource{
acl: {
visibility: Cluster,
role: '*'
},
version: "v1",
name: "api",
// declare where the api is accessible
locations: [
"http://:/service/api",
"http://:/service/api"
]
}
{code}
Even with this view for me, perhaps I'm completely missing your intention, and
you're trying to get a struct/message into mesos that doesn't have a complex
structure so that you can do with it what you need to without mesos having to
be updated. I can see how this would make things faster to get going, but I'm
leery of its long term effects. Perhaps having an unstructured option can help
us flush out use cases people have for wanting to advertise things. If that's
the case {{DiscoveryInfo}} already has
[{{Labels}}|https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L1260]
as one of it's properties. I would argue that adding the ability for a
framework to add/configure {{DiscoveryInfo}}s for itself independent of
executors/tasks. This would allow for replacement of {{webui_url}} and
definitely help with frameworks that want to advertise multiple things, or
things that aren't UIs.
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Epic
>Reporter: James DeFelice
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589318#comment-14589318
]
James DeFelice commented on MESOS-2841:
---
Agree that it's useful for API or UI (http(s) things). URI's don't have to be
URL's though, they could just be unique names that announce capabilities. For
example:
{code}
// define a handle that describes a storage driver capability of a framework
storageCap = mesos.Handle{
uri: "urn:mesos:capability:storage", // declares specific capability of
framework
labels: { { "version", "v1" }, { "version", "v1.1" } }, // supported
capability API versions
visibility: { VisibilityMesos }, // only mesos masters and slaves can see
this handle
}
// define a handle that describes a plugin capability of a framework
mesosDnsPlugin = mesos.Handle{
uri: "urn:mesos:discovery:plugin", // declares specific capability of
framework
// supported capability API versions, name of the plugin, location of plugin
discovery metadata
labels: { { "version", "v1" }, { "name", "k8s-services-discovery-plugin" }, {
"location": "http:///service/kubernetes/mesos/discovery" } },
visibility: { VisibilityCluster }, // visible within the mesos cluster, not
outside the cluster
}
{code}
/cc [~joerg84]
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Epic
>Reporter: James DeFelice
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14589093#comment-14589093 ] Ben Whitehead commented on MESOS-2841: -- I think something like that would probably be useful for API, UI (http compatible things) but there are others that still wouldn't be covered. I also don't know what form this would take in terms of mesos message changes. Perhaps [~kozyraki] has some guidance on this. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[
https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587365#comment-14587365
]
James DeFelice commented on MESOS-2841:
---
What about exposing Endpoints that are labeled?
type Endpoint struct {
uri string // something locatable
labels Labels // describe the uri
visibility Visibility // who sees this info
}
I think frameworks that offer resources is a cool idea. Not sure how well
it aligns with discovery. Unless you were talking about how to advertise
resource capabilities through some endpoint?
Maybe endpoint isn't quite right. Perhaps Handle? Service (already
overloaded term)?
> FrameworkInfo should include a Labels field to support arbitrary, lightweight
> metadata
> --
>
> Key: MESOS-2841
> URL: https://issues.apache.org/jira/browse/MESOS-2841
> Project: Mesos
> Issue Type: Epic
>Reporter: James DeFelice
> Labels: mesosphere
>
> A framework instance may offer specific capabilities to the cluster: storage,
> smartly-balanced request handling across deployed tasks, access to 3rd party
> services outside of the cluster, etc. These capabilities may or may not be
> utilized by all, or even most mesos clusters. However, it should be possible
> for processes running in the cluster to discover capabilities or features of
> frameworks in order to achieve a higher level of functionality and a more
> seamless integration experience across the cluster.
> A rich discovery API attached to the FrameworkInfo could result in some form
> of early lock-in: there are probably many ways to realize cross-framework
> integration and external services integration that we haven't considered yet.
> Rather than over-specify a discovery info message type at the framework level
> I think FrameworkInfo should expose a **very generic** way to supply metadata
> for interested consumers (other processes, tasks, etc).
> Adding a Labels field to FrameworkInfo reuses an existing message type and
> seems to fit well with the overall intent: attaching generic metadata to a
> framework instance. These labels should be visible when querying a mesos
> master's state.json endpoint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587291#comment-14587291 ] Ben Whitehead commented on MESOS-2841: -- I like the idea of a framework being able to "advertise" things about itself, but I'm not sure labels is the best way to go about doing that. webui_url is one such thing, but in practice it is really too limited and it would be nice if a framework could advertise it's UI in a better form. A few other things I can think of that would be great to advertise are: # An HTTP Api (Cassandra Framework API) # An admin server for a framework scheduler process (Maybe expose metrics, health checks etc) # An event stream (Marathon events) # A DNS Record of how to locate a scheduler process (Where is the kafka api running so I can check the status of my brokers?) # Special URLs that other things would useful (Kafka broker url) I think ultimately in a lot of ways, this particular problem falls in the space of service discovery. A few reasons why I think labels probably aren't enough to satisfy what we're going to want: # Labels aren't structure and will lead to lots of odd, non-uniform usages and formats of labels # Writing things that will want to integrate with this information, it would be great if they were more structured and easier to reason about # Allowing other frameworks access to the information # Defining ACLs to restrict the information that can be seen by other things in the cluster I don't know that DiscoveryInfo is going to be sufficient for all scenarios, but I doubt labels will be capable of expressing the different things that we'll want frameworks to be able to express. One thing I just thought of, what if mesos allowed frameworks to advertise Resources that could then be part of resource offers made to frameworks. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582738#comment-14582738 ] Aaron Bell commented on MESOS-2841: --- I believe [~BenWhitehead] had some thoughts on this. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Epic >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580916#comment-14580916 ] James DeFelice commented on MESOS-2841: --- I think I like the idea of a "shared info" type that, perhaps, contains a Labels field (generic :) ). It feels more explicit: this is information that will be shared with mesos and/or mesos clients/frameworks/consumers/etc. I'm not certain about the visibility/scope of the Capability message - who is meant to consume this? It appears to be intended for mesos itself, not the wider cluster. I'm not so sure that stuffing Labels into Capability fits foreseeable use cases: I can imagine scenario(s) where an admin may want to label frameworks with metadata that has nothing to do with capabilities. As you mentioned, webui_url also seems like a good candidate for inclusion in a "shared info" field. > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice > Labels: mesosphere > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2841) FrameworkInfo should include a Labels field to support arbitrary, lightweight metadata
[ https://issues.apache.org/jira/browse/MESOS-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580896#comment-14580896 ] Niklas Quarfot Nielsen commented on MESOS-2841: --- Great idea [~jdef]! Let us take a look at how this would look like. Changing those during reregistration may need a special path and we need to make sure they are accessible through the state endpoints. We introduce the notion of webui_url - maybe they should both be encapsulated in a SharedFrameworkInfo message (without any ties to the name - feel free to come up with another über message to encapsulate this in). We also have a 'framework capability' message now (introduced during the oversubscription work). Would it make sense to put in there perhaps? I have no strong preference at this point, but just exploring different approaches :) > FrameworkInfo should include a Labels field to support arbitrary, lightweight > metadata > -- > > Key: MESOS-2841 > URL: https://issues.apache.org/jira/browse/MESOS-2841 > Project: Mesos > Issue Type: Improvement >Reporter: James DeFelice > > A framework instance may offer specific capabilities to the cluster: storage, > smartly-balanced request handling across deployed tasks, access to 3rd party > services outside of the cluster, etc. These capabilities may or may not be > utilized by all, or even most mesos clusters. However, it should be possible > for processes running in the cluster to discover capabilities or features of > frameworks in order to achieve a higher level of functionality and a more > seamless integration experience across the cluster. > A rich discovery API attached to the FrameworkInfo could result in some form > of early lock-in: there are probably many ways to realize cross-framework > integration and external services integration that we haven't considered yet. > Rather than over-specify a discovery info message type at the framework level > I think FrameworkInfo should expose a **very generic** way to supply metadata > for interested consumers (other processes, tasks, etc). > Adding a Labels field to FrameworkInfo reuses an existing message type and > seems to fit well with the overall intent: attaching generic metadata to a > framework instance. These labels should be visible when querying a mesos > master's state.json endpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
