[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165261#comment-14165261 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- Let's create a sub-task for flags parsing and take it from there :-) Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Kapil Arya Fix For: 0.21.0 I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140977#comment-14140977 ] Bernd Mathiske commented on MESOS-1384: --- [~tstclair] We'll only support absolute, complete paths in the first patch. Good idea to handle lib exts automatically. We'd like to put that into the next patch then. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141570#comment-14141570 ] Bernd Mathiske commented on MESOS-1384: --- A patch has been submitted: https://reviews.apache.org/r/25848/ Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139097#comment-14139097 ] Timothy St. Clair commented on MESOS-1384: -- Having a pluggable architecture would enable folks to do the following: 1. Test PoC ideas in a clean way without impacting mainline. 2. Enable Service providers to write custom interfaces that may only apply to their workflow. *This is the big one* 3. Prevents mesos from accreating too much into it's core without having well thought out boundaries on interfaces and adaptability over time. By forcing the step, it helps to define clear boundaries. ... Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139349#comment-14139349 ] Timothy St. Clair commented on MESOS-1384: -- Folks - I think this is ready for review. You might want to make a couple of minor changes around named loading: e.g. libFoo.so, libFoo.dylib The load could check for extension, and in absence do the right thing. load (Foo) Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139390#comment-14139390 ] Bernd Mathiske commented on MESOS-1384: --- [~tstclair] Thanks for the vote of confidence! We will make a code improvement pass now and also remove non-essentials to get to a minimal viable first patch. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139412#comment-14139412 ] Timothy St. Clair commented on MESOS-1384: -- Keep it simple for now, as I fully expect this to iterate over time. It's also auxiliary and nothing depends on it yet, so until that point happens there can be refinement. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139553#comment-14139553 ] Vinod Kone commented on MESOS-1384: --- Please have the flag as JSON. It's easy to maintain. Our JSON flag parser accepts a file with JSON or raw JSON string. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139575#comment-14139575 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- [~vinodkone] +1 Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139841#comment-14139841 ] Bernd Mathiske commented on MESOS-1384: --- OK, JSON it is then. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[
https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137280#comment-14137280
]
Bernd Mathiske commented on MESOS-1384:
---
[~karya], [~nnielsen], and I have yet another design that I will describe here.
We also have a prototype that seems to work for at least some simple example
modules. We stripped down the API for the module writer to as little as
possible and we also reduced versioning to as little as we could think of that
still makes sense and can be expanded later.
Definitions
--
Mesos version := Mesos releases.
Module API version := Bumped when the module management system changes.
Role := The purpose that a module fulfills. In a given Mesos implementation
this is tied to a specific object type, e.g. “Allocator”, “Isolator”,
“Authenticator”.
The API from the point of view of a module writer
To write a Mesos module, include the module API header file and a module role
type declaration, then place one macro call inside your library source code to
capture version information and add another macro call to declare a module.
Then add a function body that returns a module instance. That’s all. Here is a
minimal example that declares a module with the role “Calculator” and the name
“example”.
#include mesos/module.hpp // module system API
#include calculator.hpp // module role type declaration is in here
MESOS_MODULE_LIBRARY() // declares the module library (and its versions)
class ExampleCalculator: public Calculator
{
public:
// An example function that the module implements.
virtual int compute(char a, long b)
{
return a + b;
}
};
MESOS_MODULE(Calculator, example) // declares the module
{
return new ExampleCalculator(); // creating and returning the module instance
}
Additional compatibility checks
By default the above only works when the module client Mesos version and the
Mesos version against which the library has been compiled are exactly the same.
However, with this extra declaration, you can enable backwards compatibility
controlled by a table inside Mesos that allows earlier library versions.
MESOS_IS_MODULE_COMPATIBLE(example)
{
return true;
}
Here, instead of simply returning true, the module could also perform its own
checks, and return false under certain conditions. In the latter case the
module will not be admitted, no matter what the results of any other checks by
Mesos indicate. The module’s own checks are open-ended. In particular, they may
include queries about other loaded libraries and modules and their respective
versions. We will later provide an API for that.
The API from the point of view of a Mesos implementor
Only modules with pre-declared roles and binding sites can be loaded by Mesos.
There is no means of dynamic discovery or binding of extra roles. To declare a
loadable module, a Mesos developer needs to specify an abstract class with at
least one virtual method. Here is an example that matches the module code above.
class Calculator {
public:
Calculator() {}
virtual ~Calculator() {}
virtual int foo(char a, long b) = 0;
};
That’s all. To employ a specific instantiation of this module role/type, you
can write something like this.
#include module/module_manager.hpp
TryCalculator* module =
mesos::ModuleManager::loadModuleCalculator(example);
if (module.isError()) {
… // error handling
}
Calculator calculator = loadedModule.get();
And then you can already use the module.
int n = calculator.compute(‘A’, 10);
Module Libraries and Module Versioning
Before loading the above module, a dynamic library that contains the module
needs to be loaded into Mesos. This happens early in Mesos startup code. The
Mesos developer does not need to touch that code when introducing new module
roles.
However, the developer is responsible for registering what versions of any
given module are expected to remain compatible with Mesos as it progresses.
This information is maintained in a table in src/module/module_manager.cpp. It
contains an entry for every possible module role that Mesos recognizes, each
with a corresponding Mesos release version number. This number needs to be
adjusted by the Mesos developer to reflect the current Mesos version whenever
compatibility between Mesos and modules that get compiled against it gets
broken. Given that module implementation for older Mesos versions can still be
written in the future, this may be impossible to tell and so in doubt it is
best to just bump the required module version to the current Mesos version. But
if one can be reasonably sure, assuming cooperative module developers, that a
certain kind of module will continue to function across several Mesos versions,
the table provides an easy way to specify this.
Add support for loadable MesosModule
Key: MESOS-1384
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137283#comment-14137283 ] Bernd Mathiske commented on MESOS-1384: --- And here is a list of Assumptions, design decisions, design goals, and explanations - Modules can be used by all constituents of Mesos, in particular slaves as well as masters. Different sets of modules may or may not apply to either. Each Mesos version of a module client program defines its own finite set of module bindings, i.e. module injection points. - Modules come packed in dynamic libraries (dynlibs). - An installed module is a C++ object created by a call into a module implementation in a dynlib. So far there is no well-defined set of features that work or don’t work. However, we use a dynamic_cast with RTTI to verify compatibility of a module with binding into Mesos. At a minimum, this restricts Module types to abstract classes with at least one virtual function. In case RTTI turns out not to be water-proof, we will eventually have to enumerate features that do work. This still seems less tedious than breaking our very simple API up into a much longer series of C-only constructs. - A compiler version and flag check may become necessary. - If really needed, an hourglass interface (http://cppcon2014.sched.org/event/e659d8c088904f7a1540524b196afbe9#.VBiSd0sjMeE, https://github.com/CppCon/CppCon2014/raw/master/Presentations/Hourglass%20Interfaces%20for%20C%2B%2B%20APIs%20-%20Stefanus%20Du%20Toit%20-%20CppCon%202014/Hourglass%20Interfaces%20for%20C%2B%2B%20APIs%20-%20Stefanus%20Du%20Toit%20-%20CppCon%202014.pdf) can be added later, with an extra Mesos Module System version. - Module installation has these phases: 1) Dynlib loading, 2) verification including dependency checking, 3) instantiation, 4) binding (assignment to an l-value that gets used somehow). - All modules are named in one command line flag, which gets parsed early. After all dynlibs have been loaded, all verification is run. Instantiation happens later, in various places in Mesos, wherever modules are involved. This is driven by other command line flags which then reference identifiers given by the module flag. At first we only need to support a very simple naming scheme where the module name is used directly and it is expected that there are never conflicting module names. Example: slave --modules=”/root/path1:module1,path2:module2” --allocator=”module1” --auth=”module2” - There can be multiple modules in a given dynlib. This allows shared implementation and data elements and potential packaging convenience. - Each dynlib must indicate what version of MMS it is built for. Then future MMSs can determine whether to use a given pre-existing dynlib or not. Conversely, an older Mesos/MMS can determine that a dynlib relies on a later MMS version. This also serves as a handshake between Mesos and any arbitrary dynlib to ensure it is dealing with a Mesos module dynlib at all. - Once it is thus established that Mesos is dealing with a proper module dynlib with a compatible version, the dynlib is trusted to behave cooperatively and non-maliciously. - Each dynlib indicates what minimum Mesos version it is compatible with. - Each module indicates its “role”, e.g. Isolator, Allocator, Authenticator. - There can be multiple modules for the same role in the same dynlib, especially isolator modules. - User code does not need to instantiate and bind all modules in a dynlib. It can cherry-pick. Limitations and Simplifications - If you build a dynlib against a certain Mesos version, it will not be allowed to be used with older Mesos versions. So for simplicity we bake the respective Mesos version into the dynlibs. - For each Role, a corresponding mesos version is kept in a table. If the role changes in a non-compatible way, its version must be bumped by the responsible Mesos developer to match the current mesos version. - To use the module API, The module writer only faces a header file. Nothing needs to be linked in for this purpose. However, to implement the specific payload features of any given module, it may have to reference any number of other parts of Mesos, including having to link against those. - We consider corralling all Mesos features that are used by modules into a facade layer. Thus backward compatibility could be maintained with more ease, albeit at the cost of erecting the facade. Things that we may want to support in the future and that should not be impeded by the first MMS implementation - Modules should be able to express and check interdependencies and mutual compatibility. - We do not check module versions beyond their type for now. In addition we may want to give them a version number that gets
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[
https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137370#comment-14137370
]
Benjamin Hindman commented on MESOS-1384:
-
This is great [~bernd-mesos]! I've got some clarifiying questions below. I'm
also curious if it makes more sense to throw this into a document or on a wiki
so it's easier to evolve the content? Questions:
Is the idea that we'll use Java package naming convention for the module names,
e.g., 'io.mesosphere.calculator.example'?
Can you elaborate on why we need to use macros versus function calls or
inheritance as had been in the previous design?
I don't fully understand the section starting with ... information is
maintained in a table in src/module/module_manager.cpp. Perhaps some concrete
examples that show the semantics as interfaces and versions change would help
explain?
Why do we need RTTI? Was the casting in the previous design insufficient?
Can we please make the command line flag use JSON rather than inventing another
parsing format (like I've already made the mistake of doing numerous times in
Mesos)? ;-) This will likely mean we'll have more flexibility as well, so we
can just have a single --modules='{ ... }' flag and not have to have any extra
flags like --allocator= and --auth=.
Instead of describing a module as having a 'role', can we just call it a
'module type', or any other name since 'role' already has a well defined
meaning in Mesos.
Add support for loadable MesosModule
Key: MESOS-1384
URL: https://issues.apache.org/jira/browse/MESOS-1384
Project: Mesos
Issue Type: Improvement
Affects Versions: 0.19.0
Reporter: Timothy St. Clair
Assignee: Niklas Quarfot Nielsen
I think we should break this into multiple phases.
-(1) Let's get the dynamic library loading via a stout-ified version of
https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h.
-
*DONE*
(2) Use (1) to instantiate some classes in Mesos (like an Authenticator
and/or isolator) from a dynamic library. This will give us some more
experience with how we want to name the underlying library symbol, how we
want to specify flags for finding the library, what types of validation we
want when loading a library.
*TARGET*
(3) After doing (2) for one or two classes in Mesos I think we can formalize
the approach in a mesos-ified version of
https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h.
*NEXT*
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137499#comment-14137499 ] Dominic Hamon commented on MESOS-1384: -- I'm missing some context here, I think. What problem is this trying to solve? The designs above seem to be reinventing some architecture and adding significant complexity that might already have a well-known solution elsewhere. If the goal is to split the monolithic mesos library into parts, then that's a build-time thing that can be done with well-defined public APIs between pieces. Is there more to it than that? Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137523#comment-14137523 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- [~dhamon] it is not (only) a build time thing :-) Will be happy to elaborate ([~bernd-mesos], [~benjaminhindman] or [~karya] can chime in too). Here is some text I wrote for the first ticket on the subject: Adding a first class primitive, abstraction and process for dynamic library writing and loading can make it easier to extend inner workings of Mesos. Making it possible to have dynamic loadable resource allocators, isolators, containerizes, authenticators and much more. I think this could be a powerful feature, as we get even more extensible and flexible ways of setting up Mesos - but also for isolating dependencies and complexity in external libraries and to ease experimentation with new features. For example, imagine a loadable allocators which contains a VM (Lua, Python, ...) which makes it possible to try out new allocator algorithms without forcing those dependencies into the project. BenH and I chatted about the concrete details and I will follow up with small write-up/proposal. We are having a con call at 10am PST you are more than welcome to join. We will follow up with further design docs and notes after the call. [~karya] Can you share the proof-of-concept branch of the latest design? Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137535#comment-14137535 ] Bernd Mathiske commented on MESOS-1384: --- [~benjaminhindman] What's the preferred place for a doc like this? The naming convention for modules is up for grabs. I am not aware that we implied Java package naming. We just came up with what is easy and quick (20 lines of code) to implement, because smaller patches are preferred by code reviewers. There seemed to be a lot of precedent in Mesos for the command line format we are using, and I would suggest to put JSON in files only since it is relatively verbose. But if Mesos is switching to JSON on command lines in general, then so should the module API. The macros hide all the machinery with dlsym and version checking so the developer can focus on the module code. Modules do not need to inherit anything from the module system in this new design, so that their development is more unencumbered. And you don't see any traces of the module API in Mesos code that does not have anything to do with modules. Here is an example version backward compatibility table in module_manage.cpp: roleToVersion[TestModule]= 0.21.0; roleToVersion[Isolator] = 0.21.0; roleToVersion[Authenticator] = 0.21.0; roleToVersion[Allocator] = 0.21.0; Yes, we need to change the variable name roleToVersion. I left it as is so you can find it in our prototype branch that [~karya] will post shortly :-) We don't need RTTI, but it provides an extra safety check. Yes, role should be renamed, but type would be incorrect. Each role can have many types that represent it. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137552#comment-14137552 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- I'd suggest to use the wiki https://cwiki.apache.org/confluence/display/MESOS/Index :-) BTW - we should make that more visible; couldn't find it through our website. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137580#comment-14137580 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- Let's use this wiki page: https://cwiki.apache.org/confluence/display/MESOS/DRAFT+Design+Doc+-+Mesos+Modules?showComments=true Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137648#comment-14137648 ] Kapil Arya commented on MESOS-1384: --- Here is the github repo where we have placed the proof-of-concept implementation: https://github.com/karya0/mesos/tree/modules_poc Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085637#comment-14085637 ] Craig Hansen-Sturm commented on MESOS-1384: --- MI=Multiple Inheritance. I'll dig up a good reference on using template abstract factories to make this concrete. There are also completely canned designs we can make of, in particular, XPCOM, resting at the heart of the Mozilla architecture, is worth looking at. It is more heavy weight for sure, but extremely powerful. https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085667#comment-14085667 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- Great point - does http://blog.nuclex-games.com/tutorials/cxx/plugin-architecture/ capture some of the thoughts you had on the proper factory pattern? Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077879#comment-14077879 ] Bernd Mathiske commented on MESOS-1384: --- I like Niklas' multi-inheritance design. It expresses as much as possible with pre-existing language features, avoiding extra boiler plate. If no dynamic library were ever explicitly addressed in code that uses modules, that would simplify things and also answer Ben's question about library lifetime: it gets loaded when the module gets loaded and it can be released when the module is released. Potentially, one could have multiple modules in a library, but that should be hidden under the rugs. (Yes, reference counting comes to mind, and that should also be under the rugs, if it comes to that.) The less internal mechanics come to the surface, the better. Easier to use and less complexity to implement. In that spirit, it would be ideal if there were no library path or library identifier of any kind involved at the surface. Just a module identifier (plus versioning). This should also simplify managing Mesos configuration. Maybe Module libraries could always be found at an expected path relativ to the Mesos executable/library. One can have flags/env vars etc. for changing that, but there should be a simple default IMHO, and what could be a reason to load a module from anywhere else anyway? Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078269#comment-14078269 ] Bernd Mathiske commented on MESOS-1384: --- Answering my own question: the reason to load a module from somewhere else than the default path would be to try something without having to have access to the Mesos installation path where alternative pre-installed modules reside. Instead, you could just point to a newly generated module on your local dev path. So, I think we want both options. Your syntax for that looks OK to me. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078477#comment-14078477 ] Craig Hansen-Sturm commented on MESOS-1384: --- I definately like the compactness and generally understandability of this solution, but is it general enough ? I believe this depends on what the requirements are (perhaps everyone is assuming this is clear from previous discussions). One example:Are mesos modules expected to behave like plugins ? If so, one might expect modules to be runtime-discoverable/enumerable, and loadable without actually statically linking in the init/create stub-code.Much of the design hinges on whether support for this is important or not. From a pure design perspective, the module abstraction exists, primarily, to expose the init method, load the binary, call the c-linkage entry constructor function, and then return a pointer to the object. This is completely generic - even type independent (which is a good thing) and, it serves as basically boostrapping code which doesn't need to be associated with the underlying object for all time. Given this, an external factory pattern might be more appropriate, as MI ties the bootstrapping code to the object which is being created. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078611#comment-14078611 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- bq. Given this, an external factory pattern might be more appropriate, as MI ties the bootstrapping code to the object which is being created. Can you expand a bit - do you have an example of how this would be implemented? And what is MI an abbreviation for? :-) Here is another proposal; the meta-data can encode a module subsystem version too that let's us evolve into more advanced schemes (that might be entirely different), without overengineering up-front? Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077372#comment-14077372 ] Benjamin Hindman commented on MESOS-1384: - This looks awesome Niklas. Only one question from me: What about the lifetime of the DynamicLibrary constructed by IsolatorModule::create()? When does it get deleted? I'm assuming it needs to stick around for as long as IsolatorModule (and thus Isolator) is valid. Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair Assignee: Niklas Quarfot Nielsen I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[
https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074885#comment-14074885
]
Niklas Quarfot Nielsen commented on MESOS-1384:
---
Here is a small write up on the proposed design:
h2. Motivation
The motivation behind mesos modules is to make it possible to implement
abstract interfaces to internal Mesos components outside the Mesos binary, as
dynamic loadable libraries.
Instead of mapping each and every call to C++ mangled (or self mangled)
symbols: http://mentorembedded.github.io/cxx-abi/abi.html and
https://github.com/nqn/dylib .
We can use the already existing vtable layout to dispatch to implementations
within a loaded library (just like our current use of JNI).
{code}
Mesos + External Library (.so / .dylib)
|
|
Implementation
Pure virtual class +--+ |
+-+
| IsolatorModule | | | IsolatorModuleImpl :
IsolatorModule |
| | | |
|
| + Module metadata| | | + Module metadata (Version:
XXX)|
| + create(library)| | |
|
| | | |
|
| +--+ | | |
+-+ |
| |Isolator | | | | | IsolatorImpl
| |
| | | | | | |
| |
| | | | | | |
| |
| | | | | | |
| |
| | | | | | |
| |
| +--+ | | |
+-+ |
+-+--+-+ |
+-+-+-+
|
|
Tryshared_ptrIsolator isolator =| extern C {
IsolatorModule::create(libxxx.so); |void*
create_isolator_module(void*) {
// Use isolator as if it was hosted by Mesos | return new
IsolatorModuleImpl;
|}
| }
|
+
{code}
This design has been implemented as a proof-of-concept in this branch:
https://github.com/nqn/mesos/tree/niklas/mesos_module
The sequence is as follows (taken IsolatorModule as an example):
1) Use stout's DynamicLibrary abstraction to get a library object
2) Call (new and dedicated) module shims initialize method
(IsolatorModule::create(), AllocatorModule::create(),
AuthenticatorModule::create(), ...)
2.a) Behind the scenes, the create() method localize the proper symbol in the
library
2.b) calls it (with optional arguments as void*)
2.c) Verifies it through metadata located between the isolator module object
start and the isolator offset.
2.d) returns a try to the isolator (compiler rebase from isolator module to
isolator)
3) Use returned isolator object as usual.
h3. Notes to drawing
The definition of IsolatorModule is shared (included) by both Mesos and library
implementer.
The IsolatorModule::create() is static and knows which symbol to look for.
A standard scheme of create_X_module could be used, so one library can be
used to multiple module implementations.
create() is called init() in the proof-of-concept:
https://github.com/nqn/mesos/tree/niklas/mesos_module
h2. QA
h4. 1) Why do we return a void pointer across the library border?
We need to be able to locate the right symbol in the library to generate a
module object.
This means that we can / should not use mangled names (which C++ types, like
classes) requires if it were a return type.
h4. 2) Why don't we cast the void pointer with dynamic_cast for type
verification.
GCC doesn't support this on void pointers unfortunately.
h4. 3) Why does create() / init() require a path to the library file instead of
a DynamicLibrary type?
We want to be able to control the life-cycle of the loaded library separately
from the life-cycle of a module object.
Multiple modules can be provided by one library file.
A short-cut could be to overload the create() / init() methods and add the path
constructor as a 2nd option.
h4. 4) Why do we use multiple inheritance? How about having the isolator as a
member variable instead?
We get an _IsolatorModule_ from the
[jira] [Commented] (MESOS-1384) Add support for loadable MesosModule
[ https://issues.apache.org/jira/browse/MESOS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065781#comment-14065781 ] Niklas Quarfot Nielsen commented on MESOS-1384: --- Sorry I missed this - must have been in the issues@ interim. I have no strong opinion on a first interface to support. I think the most likely one to be used frequently would be isolator modules. I may be wrong of course. Let's get the ball rolling on this one. I will rebase and start digging into https://github.com/nqn/mesos/tree/niklas/mesos_module again Add support for loadable MesosModule Key: MESOS-1384 URL: https://issues.apache.org/jira/browse/MESOS-1384 Project: Mesos Issue Type: Improvement Affects Versions: 0.19.0 Reporter: Timothy St. Clair I think we should break this into multiple phases. -(1) Let's get the dynamic library loading via a stout-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/DynamicLibrary.h. - *DONE* (2) Use (1) to instantiate some classes in Mesos (like an Authenticator and/or isolator) from a dynamic library. This will give us some more experience with how we want to name the underlying library symbol, how we want to specify flags for finding the library, what types of validation we want when loading a library. *TARGET* (3) After doing (2) for one or two classes in Mesos I think we can formalize the approach in a mesos-ified version of https://github.com/timothysc/tests/blob/master/plugin_modules/MesosModule.h. *NEXT* -- This message was sent by Atlassian JIRA (v6.2#6252)
