Re: modularity: (my) expectations vs. reality

[stan]

On Thu, 24 Aug 2017 08:21:57 +0200
Tomas Tomecek  wrote:

> ​We would need to develop a dedicated, non-trivial tooling to enable
> this functionality.​ And honestly, I can't even imagine how this
> could be even possible to implement for all ecosystems (compiled
> languages, interpreted languages).

Thanks.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: modularity: (my) expectations vs. reality

[Tomas Tomecek]

On Wed, Aug 23, 2017 at 6:16 PM, stan  wrote:

> On Wed, 23 Aug 2017 07:51:57 +0200
> Michal Novotny  wrote:
>
> > I guess I am missing something but I don't see how modularity adds
> > flexibility. rpm, yum repos, ansible, dnf seem to be quite flexible
> > even now and having that + something else on top seems to be less
> > flexible. I am just speaking my mind here.
>
> I'm not involved in modularity, and I'm speaking as an observer.  But
> it seems that it would be a lot more effective to put the libraries in
> containers, and keep applications in rpms.
>
> That is, say there is a python container, and it contains the various
> formats of python, 2.6, 2.7, 3.5, 3.6, etc.  Then any application that
> needs python just specifies the python it needs, and the OS links it
> with the proper library from the container when it runs.
>

​We would need to develop a dedicated, non-trivial tooling to enable this
functionality.​ And honestly, I can't even imagine how this could be even
possible to implement for all ecosystems (compiled languages, interpreted
languages).


​Tomas
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: modularity: (my) expectations vs. reality

[stan]

On Wed, 23 Aug 2017 07:51:57 +0200
Michal Novotny  wrote:

> I guess I am missing something but I don't see how modularity adds
> flexibility. rpm, yum repos, ansible, dnf seem to be quite flexible
> even now and having that + something else on top seems to be less
> flexible. I am just speaking my mind here.

I'm not involved in modularity, and I'm speaking as an observer.  But
it seems that it would be a lot more effective to put the libraries in
containers, and keep applications in rpms.

That is, say there is a python container, and it contains the various
formats of python, 2.6, 2.7, 3.5, 3.6, etc.  Then any application that
needs python just specifies the python it needs, and the OS links it
with the proper library from the container when it runs.

Or, gtk, with all its variations in a container, and again, applications
get the version they need to run transparently.

This would eliminate all the redundant library duplication that
containers as they are now envisioned would entail.  There would still
be redundancy, but it would be the minimal set for the applications that
are installed.  And it would still allow the upstream of applications to
specify specific libraries without worrying about
variations in distributions.  

This doesn't address security concerns if an application upstream
specifies a library that is known to be insecure or unmaintained, and
it is in a container.  But that's just a policy decision of a
distribution as to whether they allow a library container to contain
insecure versions in order to permit applications to install.

Just some thoughts.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: modularity: (my) expectations vs. reality

[Michal Novotny]

Hello Langdon,

On Tue, Aug 22, 2017 at 11:42 PM, Langdon White 
wrote:

>
>
> On Tue, Aug 22, 2017 at 2:40 PM Michal Novotny  wrote:
>
>> I would like to publicly note that I had completely different idea about
>> this project first time I have encountered it at the last Flock.
>>
>> My idea was that the project will target runtime rather than build-time
>> and will try to provide a set of packages that would be able to
>> e.g. spawn and run an httpd stack almost at one command.
>>
>> So that we will basically have e.g. this functionality:
>> https://hub.docker.com/r/p0bailey/docker-flask/ but integrated directly
>> in
>> distribution.
>>
>> This would make lots of sense to me because it would work similarly to
>> how upstream projects get
>> into Fedora (and RHEL) eventually. They need to pass certain criteria for
>> being accepted and then
>> there is is a group of people that maintains it in the given operating
>> system.
>>
>> Similarly for the modularity (as I saw it), there would be e.g. a docker
>> image that would slowly make
>> it into being a full-fledged system component that can be run in a
>> container or even natively on the host.
>>
>> The concrete implementation would probably include ansible where the
>> modules (being standard rpms)
>> would together make a big ansible playbook archive (e.g. placed under
>> /etc/playbook) that would include
>> default templates describing default sysadmin use-cases (like importing
>> db and web application start)
>> that could be overridden by a custom user template being placed at the
>> same subpath under e.g. /etc/custom-playbook.
>>
>> This would be a killer tool for sysadmins that could script their common
>> use-cases in a simple manner and completely
>> avoid any nerve-wrecking direct config manipulations on a running system
>> that (almost) each of us is familiar with. It would actually be
>> a common sysadmin (and power-user) language on RHEL based distros and I
>> think you can imagine this can be pretty cool.
>>
>> This approach would be also very flexible because you could define your
>> resources that you want to launch individual module
>> components on by altering a default resource setup and switch from
>> spawning everything natively to spawning everything in docker
>> containers locally to spawning everything in docker containers in
>> OpenShift staging instance to spawning everything in OpenShift production
>> instance.
>>
>> This also makes lots of sense from the human-effort point of view. We
>> could apply very well our sysadmin expertise in a certain
>> areas and transmit it directly to our users through code that we would
>> have written. Packagers would be config masters and they
>> would maintain default configuration to provide user with the most
>> effortless way to make something run (e.g. dovecot + postfix combo).
>>
>> And we could then also maintain custom playbooks for specific customers
>> meaning we would be able to create system cut directly to
>> their needs and have means to maintain it at large scales. And this would
>> basically mean having an rpm somewhere being named after
>> our customer e.g. mailserver-intel with a set of specific intel
>> mailserver ansible playbooks.
>>
>> After time I started to understand that the focus of modularity is
>> different and it rather focuses on having tight control over
>> the build process and lifecycle syncing across different system
>> components.
>>
>> I am not saying this is wrong or anything. It probably just means I was
>> mislead in my expectations but still I would like to share this
>> particular point of view as it makes lots of sense to me and it is
>> something that makes me excited.
>>
>> Sorry for not being technical or anything. You can consider this to be a
>> short story that doesn't deserve much attention.
>> clime
>>
>> ...Oh yeah, I started very basic PoC at https://pagure.io/lamp and
>> needed to postpone it a bit. It might be worth continuing
>>
>
> So.. in short, I think this is a cool idea. Also, I think this is the kind
> of thing that modularity could enable.
>

The question is if this isn't already enabled.


> We had in the game plan, way back, allowing super sophisticated "post
> install scripts" as part of the install profile concept. Basically, we had
> the idea that you could have an ansible playbook as part of the install
> profile. However, it is just on the "someday" list right now.
>

I think that having ansible-playbook as part of installation process is
quite demanding at the packaging guidelines.
Putting configuration files in the right places could be certainly alright
during installation but systemctl
starting services or creating db users and databases would probably not be
ok as it might interfere with user setup.
For that I think it would be good to have certain control component for
each module - basically just a command-line app
(lamp, django, flask, mailserver, ...) with man pages etc. 

Re: modularity: (my) expectations vs. reality

[Matthew Miller]

On Tue, Aug 22, 2017 at 08:37:59PM +0200, Michal Novotny wrote:
> My idea was that the project will target runtime rather than
> build-time and will try to provide a set of packages that would be
> able to e.g. spawn and run an httpd stack almost at one command.

I think what you describe is something we can help enable with
modularity, and it's kind of like what I expect the future path of
Fedora Server and ansible-based roles to be.



-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: modularity: (my) expectations vs. reality

[Langdon White]

On Tue, Aug 22, 2017 at 2:40 PM Michal Novotny  wrote:

> I would like to publicly note that I had completely different idea about
> this project first time I have encountered it at the last Flock.
>
> My idea was that the project will target runtime rather than build-time
> and will try to provide a set of packages that would be able to
> e.g. spawn and run an httpd stack almost at one command.
>
> So that we will basically have e.g. this functionality:
> https://hub.docker.com/r/p0bailey/docker-flask/ but integrated directly in
> distribution.
>
> This would make lots of sense to me because it would work similarly to how
> upstream projects get
> into Fedora (and RHEL) eventually. They need to pass certain criteria for
> being accepted and then
> there is is a group of people that maintains it in the given operating
> system.
>
> Similarly for the modularity (as I saw it), there would be e.g. a docker
> image that would slowly make
> it into being a full-fledged system component that can be run in a
> container or even natively on the host.
>
> The concrete implementation would probably include ansible where the
> modules (being standard rpms)
> would together make a big ansible playbook archive (e.g. placed under
> /etc/playbook) that would include
> default templates describing default sysadmin use-cases (like importing db
> and web application start)
> that could be overridden by a custom user template being placed at the
> same subpath under e.g. /etc/custom-playbook.
>
> This would be a killer tool for sysadmins that could script their common
> use-cases in a simple manner and completely
> avoid any nerve-wrecking direct config manipulations on a running system
> that (almost) each of us is familiar with. It would actually be
> a common sysadmin (and power-user) language on RHEL based distros and I
> think you can imagine this can be pretty cool.
>
> This approach would be also very flexible because you could define your
> resources that you want to launch individual module
> components on by altering a default resource setup and switch from
> spawning everything natively to spawning everything in docker
> containers locally to spawning everything in docker containers in
> OpenShift staging instance to spawning everything in OpenShift production
> instance.
>
> This also makes lots of sense from the human-effort point of view. We
> could apply very well our sysadmin expertise in a certain
> areas and transmit it directly to our users through code that we would
> have written. Packagers would be config masters and they
> would maintain default configuration to provide user with the most
> effortless way to make something run (e.g. dovecot + postfix combo).
>
> And we could then also maintain custom playbooks for specific customers
> meaning we would be able to create system cut directly to
> their needs and have means to maintain it at large scales. And this would
> basically mean having an rpm somewhere being named after
> our customer e.g. mailserver-intel with a set of specific intel mailserver
> ansible playbooks.
>
> After time I started to understand that the focus of modularity is
> different and it rather focuses on having tight control over
> the build process and lifecycle syncing across different system
> components.
>
> I am not saying this is wrong or anything. It probably just means I was
> mislead in my expectations but still I would like to share this
> particular point of view as it makes lots of sense to me and it is
> something that makes me excited.
>
> Sorry for not being technical or anything. You can consider this to be a
> short story that doesn't deserve much attention.
> clime
>
> ...Oh yeah, I started very basic PoC at https://pagure.io/lamp and needed
> to postpone it a bit. It might be worth continuing
>

So.. in short, I think this is a cool idea. Also, I think this is the kind
of thing that modularity could enable.

We had in the game plan, way back, allowing super sophisticated "post
install scripts" as part of the install profile concept. Basically, we had
the idea that you could have an ansible playbook as part of the install
profile. However, it is just on the "someday" list right now.

You could, very easily, have streams or install profiles per target
audience, you can (in theory) do this now, but, we don't want to have the
branching and the like get out of control until we actually understand how
this stuff is all going to come together. So, you could do it in COPR, once
module builds come online, but perhaps not in dist-git.

We (fedora, not just modularity) also have been looking/implementing
"system containers[1]" which are kind of like a "native feeling
containerized application". Containers are already being built (sorta, need
the infra to be finished) from modules providing more flexibility by
allowing simple stream changing without modifying the dockerfiles.

So, I think your idea is exactly what we want the modularity flexibility
for.