Re: [BULK]Discuss the possible technical directions of Mesos
Hey, Le lun. 24 mai 2021 à 14:12, Qian Zhang a écrit : > > several fixing bugs which basically make Mesos unusable on a recent Linux > distro > Can you please elaborate a bit on this? Do you mean Mesos not working on a > recent Linux distro? If so, I think we can start to fix the issues and > maybe do a patch release for that. Yes, there are several issues on recent Linux distributions, e.g. Debian Bullseye: - https://github.com/apache/mesos/pull/387: compilaiton error, although it's only in master not in the last release - https://github.com/apache/mesos/pull/388: problem with the freezer cgroup based task killer which causes over a dozen test to fail and can leave the freezer frozen, tasks in uninterruptible state etc - https://github.com/apache/mesos/pull/384: problem parsing ld.so.cache which also breaks a lot of things You were tagged in some of this MRs, I tagged you in all of them, it'd be great if you could have a look :). Cheers, > > > Regards, > Qian Zhang > > > On Fri, May 21, 2021 at 2:57 AM Charles-François Natali > wrote: > > > Hey, > > > > Sorry for being a killjoy and repeating myself, but as mentioned in > > the past, I don't think that technical direction is the most important > > problem right now - community is. > > Coming up with medium/long-term technical roadmap doesn't do much if > > there are no contributors to implement them, and users to use them. > > > > The following issues which have been brought up are still not resolved: > > - very few committers willing to review and merge MRs - currently only > > Andrei Sekretenko is doing that, and I'm sure he's busy with his day > > job so only has that much bandwidth > > - very few people contribute MRs and triage/address JIRA issues - > > AFAICT it's pretty much Andreas and me > > > > So I think the first thing to do would be to address those problems. > > Some suggestions which come to mind: > > - to the remaining committers who'd still like to salvage the project, > > please take some time to review and merge MRs - > > https://github.com/apache/mesos/pulls has a few open, several fixing > > bugs which basically make Mesos unusable on a recent Linux distro > > - to the various users who've said they were interested in keeping the > > project alive: start contributing. It doesn't have to be anything big, > > just get familiar with the code base: > > * start going through JIRA and triage bugs, closing invalid/stale > > ones, tackling small issues > > * submit MRs so that the test suite passes on your OS > > * submit MRs to merge various commits you have in your private repos > > if applicable > > > > Then in a few months, once the project is back to having a small > > active contributors base, they can together decide how to take the > > project forward, and start addressing larger projects. > > > > Cheers, > > > > Charles > > > > > > > > > > > > > > Le jeu. 20 mai 2021 à 18:16, Gregoire Seux a écrit : > > > > > > Hi, > > > > > > Interesting set of suggestions! Here are a few comments: > > > > > > * Mesos feels simple to deploy (only very few components: zookeeper, > > masters and agents), customization is done mostly through configuration > > files. I don't think there is a strong need to make it easier (even though > > I've used Mesos for years, so I'm pretty used to the difficulty if any) > > > * Having to manage Zookeeper adds some complexity but since > > Zookeeper piece is required to operate Marathon (which is our main > > framework), I don't see much value in the investment required to get rid of > > this dependency. > > > * Taking advantage of NUMA topology by default would be a good > > addition although I don't see it as strategic (at least we have solved this > > on our clusters with custom modules) > > > * I would love to see improvement on masters scalability for large > > clusters (our largest cluster is 3500 nodes and may start to suffer from > > the actor model) > > > > > > Something that I see as a very significant drawback to the ecosystem at > > large is the difficulty to write frameworks. In addition to this, most > > open-source frameworks feel abandoned. Without good frameworks, Mesos value > > really decreases a lot (although it is very technically strong). > > > I think, making Mesos thrive would necessarily go through a solution to > > this issue. > > > > > > Something that I'd see as strategic would be the ability to deploy > > complex workloads on Mesos without having to write a new framework. Random > > idea: make Mesos really usable as a backend for Kubernetes (as a virtual > > kubelet). This would remove a lot of barriers to use Mesos as a strong > > engine to operate a fleet of servers while allowing to use the Kubernetes > > API that apparently everybody loves. > > > > > > What do you think? > > > > > > -- > > > Grégoire Seux > > > > >
Re: [BULK]Discuss the possible technical directions of Mesos
Hi Charles, I agree that we should re-activate the community first, both Andrei and I can help review patches, we have complementary Mesos knowledge, he can help on the patches for Mesos master/allocator and I can do for Mesos agent/containerizer. > several fixing bugs which basically make Mesos unusable on a recent Linux distro Can you please elaborate a bit on this? Do you mean Mesos not working on a recent Linux distro? If so, I think we can start to fix the issues and maybe do a patch release for that. Regards, Qian Zhang On Fri, May 21, 2021 at 2:57 AM Charles-François Natali wrote: > Hey, > > Sorry for being a killjoy and repeating myself, but as mentioned in > the past, I don't think that technical direction is the most important > problem right now - community is. > Coming up with medium/long-term technical roadmap doesn't do much if > there are no contributors to implement them, and users to use them. > > The following issues which have been brought up are still not resolved: > - very few committers willing to review and merge MRs - currently only > Andrei Sekretenko is doing that, and I'm sure he's busy with his day > job so only has that much bandwidth > - very few people contribute MRs and triage/address JIRA issues - > AFAICT it's pretty much Andreas and me > > So I think the first thing to do would be to address those problems. > Some suggestions which come to mind: > - to the remaining committers who'd still like to salvage the project, > please take some time to review and merge MRs - > https://github.com/apache/mesos/pulls has a few open, several fixing > bugs which basically make Mesos unusable on a recent Linux distro > - to the various users who've said they were interested in keeping the > project alive: start contributing. It doesn't have to be anything big, > just get familiar with the code base: > * start going through JIRA and triage bugs, closing invalid/stale > ones, tackling small issues > * submit MRs so that the test suite passes on your OS > * submit MRs to merge various commits you have in your private repos > if applicable > > Then in a few months, once the project is back to having a small > active contributors base, they can together decide how to take the > project forward, and start addressing larger projects. > > Cheers, > > Charles > > > > > > > Le jeu. 20 mai 2021 à 18:16, Gregoire Seux a écrit : > > > > Hi, > > > > Interesting set of suggestions! Here are a few comments: > > > > * Mesos feels simple to deploy (only very few components: zookeeper, > masters and agents), customization is done mostly through configuration > files. I don't think there is a strong need to make it easier (even though > I've used Mesos for years, so I'm pretty used to the difficulty if any) > > * Having to manage Zookeeper adds some complexity but since > Zookeeper piece is required to operate Marathon (which is our main > framework), I don't see much value in the investment required to get rid of > this dependency. > > * Taking advantage of NUMA topology by default would be a good > addition although I don't see it as strategic (at least we have solved this > on our clusters with custom modules) > > * I would love to see improvement on masters scalability for large > clusters (our largest cluster is 3500 nodes and may start to suffer from > the actor model) > > > > Something that I see as a very significant drawback to the ecosystem at > large is the difficulty to write frameworks. In addition to this, most > open-source frameworks feel abandoned. Without good frameworks, Mesos value > really decreases a lot (although it is very technically strong). > > I think, making Mesos thrive would necessarily go through a solution to > this issue. > > > > Something that I'd see as strategic would be the ability to deploy > complex workloads on Mesos without having to write a new framework. Random > idea: make Mesos really usable as a backend for Kubernetes (as a virtual > kubelet). This would remove a lot of barriers to use Mesos as a strong > engine to operate a fleet of servers while allowing to use the Kubernetes > API that apparently everybody loves. > > > > What do you think? > > > > -- > > Grégoire Seux > > >
Re: [BULK]Discuss the possible technical directions of Mesos
Hey, Sorry for being a killjoy and repeating myself, but as mentioned in the past, I don't think that technical direction is the most important problem right now - community is. Coming up with medium/long-term technical roadmap doesn't do much if there are no contributors to implement them, and users to use them. The following issues which have been brought up are still not resolved: - very few committers willing to review and merge MRs - currently only Andrei Sekretenko is doing that, and I'm sure he's busy with his day job so only has that much bandwidth - very few people contribute MRs and triage/address JIRA issues - AFAICT it's pretty much Andreas and me So I think the first thing to do would be to address those problems. Some suggestions which come to mind: - to the remaining committers who'd still like to salvage the project, please take some time to review and merge MRs - https://github.com/apache/mesos/pulls has a few open, several fixing bugs which basically make Mesos unusable on a recent Linux distro - to the various users who've said they were interested in keeping the project alive: start contributing. It doesn't have to be anything big, just get familiar with the code base: * start going through JIRA and triage bugs, closing invalid/stale ones, tackling small issues * submit MRs so that the test suite passes on your OS * submit MRs to merge various commits you have in your private repos if applicable Then in a few months, once the project is back to having a small active contributors base, they can together decide how to take the project forward, and start addressing larger projects. Cheers, Charles Le jeu. 20 mai 2021 à 18:16, Gregoire Seux a écrit : > > Hi, > > Interesting set of suggestions! Here are a few comments: > > * Mesos feels simple to deploy (only very few components: zookeeper, > masters and agents), customization is done mostly through configuration > files. I don't think there is a strong need to make it easier (even though > I've used Mesos for years, so I'm pretty used to the difficulty if any) > * Having to manage Zookeeper adds some complexity but since Zookeeper > piece is required to operate Marathon (which is our main framework), I don't > see much value in the investment required to get rid of this dependency. > * Taking advantage of NUMA topology by default would be a good addition > although I don't see it as strategic (at least we have solved this on our > clusters with custom modules) > * I would love to see improvement on masters scalability for large > clusters (our largest cluster is 3500 nodes and may start to suffer from the > actor model) > > Something that I see as a very significant drawback to the ecosystem at large > is the difficulty to write frameworks. In addition to this, most open-source > frameworks feel abandoned. Without good frameworks, Mesos value really > decreases a lot (although it is very technically strong). > I think, making Mesos thrive would necessarily go through a solution to this > issue. > > Something that I'd see as strategic would be the ability to deploy complex > workloads on Mesos without having to write a new framework. Random idea: make > Mesos really usable as a backend for Kubernetes (as a virtual kubelet). This > would remove a lot of barriers to use Mesos as a strong engine to operate a > fleet of servers while allowing to use the Kubernetes API that apparently > everybody loves. > > What do you think? > > -- > Grégoire Seux >
Re: [BULK]Discuss the possible technical directions of Mesos
Hi, Interesting set of suggestions! Here are a few comments: * Mesos feels simple to deploy (only very few components: zookeeper, masters and agents), customization is done mostly through configuration files. I don't think there is a strong need to make it easier (even though I've used Mesos for years, so I'm pretty used to the difficulty if any) * Having to manage Zookeeper adds some complexity but since Zookeeper piece is required to operate Marathon (which is our main framework), I don't see much value in the investment required to get rid of this dependency. * Taking advantage of NUMA topology by default would be a good addition although I don't see it as strategic (at least we have solved this on our clusters with custom modules) * I would love to see improvement on masters scalability for large clusters (our largest cluster is 3500 nodes and may start to suffer from the actor model) Something that I see as a very significant drawback to the ecosystem at large is the difficulty to write frameworks. In addition to this, most open-source frameworks feel abandoned. Without good frameworks, Mesos value really decreases a lot (although it is very technically strong). I think, making Mesos thrive would necessarily go through a solution to this issue. Something that I'd see as strategic would be the ability to deploy complex workloads on Mesos without having to write a new framework. Random idea: make Mesos really usable as a backend for Kubernetes (as a virtual kubelet). This would remove a lot of barriers to use Mesos as a strong engine to operate a fleet of servers while allowing to use the Kubernetes API that apparently everybody loves. What do you think? -- Grégoire Seux
