If you want to see this all working you can just fetch the "no-at-db4"
branch at https://github.com/ibuildthecloud/cloudstack.git
Plugin composes multiple modules. If modules are siblings they can't
inject from each other. But a plugin can augment another module if it
chooses to. The reality is that the core cloudstack is a tangled mess of
dependencies such that most of the core code can't be modularized as it
stands. So there exists a context towards the top of the hierarchy called
"core" that a lot of jars contribute to it. Here is the full hierarchy
right now. I'll probably rename a bunch of things, but this gives you an
idea.
bootstrap
system
core
allocator
allocator-server
planner
api-planner
baremetal-planner
explicit-dedication
host-anti-affinity
implicit-dedication-planner
server-planner
user-concentrated-pod-planner
random-allocator
api
acl-static-role-based
rate-limit
server-api
user-authenticator-ldap
user-authenticator-md5
user-authenticator-plaintext
user-authenticator-sha256salted
backend
alert-adapter-server-backend
compute
alert-adapter-server-compute
baremetal-compute
fencer-server
investigator-server
kvm-compute
ovm-compute
server-compute
xenserver-compute
network
baremetal-network
elb
midonet
nvp
ovs
server-network
ssp
storage
alert-adapter-server-storage
allocator-storage
baremetal-storage
secondary-storage
server-storage
storage-image-default
storage-image-s3
storage-image-swift
storage-volume-default
storage-volume-solidfire
template-adapter-server
discoverer
baremetal-discoverer
discoverer-server
ovm-discoverer
xenserver-discoverer
If you look at the baremetal hypervisor plugin that is pretty cross cutting
to most of ACS. So the modules it contributes to are as follows
resources/META-INF/cloudstack/baremetal-storage/spring-context.xml
resources/META-INF/cloudstack/baremetal-compute/spring-context.xml
resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml
resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml
resources/META-INF/cloudstack/baremetal-planner/spring-context.xml
resources/META-INF/cloudstack/baremetal-network/spring-context.xml
So it creates child contextes of storage, compute, network, planner, and
discoverer to add its extensions where it needs to be. Additionally,
you'll notice, it adds some beans to the core context from the file
resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml. This
is because it has some manager class that is used by multiple contexts.
Frank, I understand the scare that we are going too complex, but do you
have some other suggestion? I don't like the idea of one gigantic spring
context. So I feel I am making it as simple as I can while maintaining
some order. People just need to create one or more spring xml files and a
properties files that says where to put it in the hierarchy.
Additionally, by putting forcing people to put beans in certains modules it
forces them to think about what is the role of the code. For example,
today in ACS the *ManagerImpl classes are a huge mess. They implement too
many interfaces and the code crosses to many architectural boundaries. Its
about time we start splitting things up to be more maintainable.
If you have some time, please check out what I have on github.
Darren
On Wed, Sep 18, 2013 at 1:56 PM, Frank Zhang <frank.zh...@citrix.com> wrote:
> I am not against boundary, I am just against making things unnecessary
> complex to enable boundary.
> If we are going this way, I hope we can make it as much as transparent to
> developers. That means, as
> a developer, all a plugin I need to do is 1) provide my separate spring
> xml 2) inject beans I want (valid beans)
> in my bean and code business logic 3) compile to a jar and put to some
> place designated by CloudStack. That's it.
>
> I raise this topic because I have seen some projects to create boundary
> making things horrible complex. And
> sometimes developers are hurt by wrong boundaries, as a result, to
> overcome these limitations people write
> lots of ugly code which makes thing even worse.
>
> However, I am still worry about if we can make things so simpler.
> For example, we may have an orchestration context that contains major
> beans needed by almost every
> plugin, this context can be easily set as parent context for all plugin
> contexts when bootstrap. However, if a plugin A needs
> to access some bean defined in plugin B, given they are sibling, how
> plugin framework resolves the dependency ?
>
> > -----Original Message-----
> > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> > Sent: Wednesday, September 18, 2013 11:53 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [PROPOSAL] Modularize Spring
> >
> > I'm not for OSGi either, but contexts are quite useful and will lead to
> better
> > things. First off, we don't want one gigantic spring XML config file
> like we have
> > today. I think we can all agree on that. So each plugin will have to
> supply its
> > own XML. So the obstacles you mention, will largely be just that for
> people.
> >
> > With Spring it is really simple to just inject dependencies and cross
> > architectural boundaries. Its currently everywhere in ACS. You can't
> just say
> > we should review code and make sure nobody doesn't do bad things. A
> little bit
> > of framework to enforce separation is a good thing. But I'm guessing
> you will
> > disagree with me there.
> >
> > Here are some random points on why contexts are good. Say I want to use
> > Spring AOP or Spring TX in my plugin. With your own context you can
> ensure
> > that you won't screw with anybody else code by accidentally having you
> > pointcut match their bean. You don't have to worry about bean name
> conflicts.
> > If two config files specify bean X, Spring will gladly just use the last
> one. I've
> > already found multiply defined beans in ACS, but things still just
> happen to work.
> > Having multiple contexts also defines initialization order. We can
> ensure that
> > the framework is loaded and ready before child context are loaded and
> started.
> > (we kind of do this today with ComponentLifeCycle, but its a hack in my
> mind).
> > Additionally, when things start you will know, loading context "crapping
> plug X".
> > If spring fails to initialize, the issue it there. Today, if spring
> fails to start, it
> > could be one of over 500 beans causing the weird problem. The list goes
> on
> > and on.
> >
> > Finally, this is the big one and why I really want contexts. I have
> some notes on
> > the wiki [1] that you might want to read through. Basically I want to
> get to a
> > more flexible deployment model that allows both a single monolithic JVM
> as
> > today and also a fully distributed system. Having contexts in hierarchy
> will
> > enable me to accomplish this. By selecting which contexts are loaded at
> > runtime will determine what role the JVM takes on. The contexts help
> people
> > better understand how the distributed architecture will work too, when
> we get
> > there.
> >
> > Frank, trust me, I hate complex things. I don't want OSGi, classloader
> magic,
> > etc. But I do like organization and a little bit of framework so that
> people don't
> > accidentally shoot themselves in the foot. I personally like knowing
> that I
> > couldn't have screwed something up, because the framework won't even
> allow
> > it. If we separate everything as I want today, and then tomorrow we say
> this is
> > way too much overhead, moving to a flat context is simple. Don't think
> we are
> > on some slippery slope to classloaders and dependency hell.
> >
> > Darren
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Nothing+to+see+her
> > e...#Nothingtoseehere...-DeploymentModels
> >
> >
> >
> > On Wed, Sep 18, 2013 at 11:22 AM, Frank Zhang
> > <frank.zh...@citrix.com>wrote:
> >
> > > What's the point in using separate spring context per plugin?
> > > Separate class loader is the thing I hate most in OSGI, I am afraid we
> > > are on the same way.
> > > Frankly speaking, I never see benefits of this *separate* model, our
> > > project(or most projects) is not like Chrome which has to create
> > > sandbox for extensions in order to avoid bad plugin screws up the
> > > whole browser(however, I still see bad plugins screw up my Chrome
> > > well).
> > > Projects like CloudStack using plugin to decouple architecture should
> > > not introduce many isolations to plugin writer, the point preventing
> > > wrong use of some components is not making much sense to me. If a
> > > plugin not following guide(if we have
> > > it) we should kick it out, instead of making obstacles for 99% good
> people.
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> > > > Sent: Wednesday, September 18, 2013 10:33 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Re: [PROPOSAL] Modularize Spring
> > > >
> > > > Right, component isn't a thing. I probably shouldn't use that term.
> > > > I
> > > want to
> > > > standarize on the naming convention of plugin, module, and extension.
> > > It is
> > > > explained a bit on the wiki [1] but I'll try to do a little better
> > > > job
> > > here. So a
> > > > plugin is basically a jar. A jar contains multiple modules. A
> > > > modules
> > > ends up
> > > > being a spring application context that composes multiple
> > > > configuration
> > > files.
> > > > Modules are assembled into a hierarchy at runtime. Extensions are
> > > > implementations of interfaces that exist in a module. A maven
> > > > project produces a jar, so a plugin ends up being a maven project
> also.
> > > >
> > > > So currently today we don't have a strong definition of Plugin and I
> > > hope to
> > > > address that.
> > > >
> > > > Darren
> > > >
> > > > [1]
> > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug-
> > > > ins%2C+Modules%2C+and+Extensions
> > > >
> > > >
> > > > On Wed, Sep 18, 2013 at 4:25 AM, Daan Hoogland
> > > > <daan.hoogl...@gmail.com>wrote:
> > > >
> > > > > sounds great Darren,
> > > > >
> > > > > By component, you mean maven project or some larger chunk like
> > > > > distribution package? (did i miss this definition somewhere or do
> > > > > we define the components now?)
> > > > >
> > > > > regards,
> > > > > Daan
> > > > >
> > > > > On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd
> > > > > <darren.s.sheph...@gmail.com> wrote:
> > > > > > Currently ACS code is fairly modular in that you can add
> > > > > > plug-ins to ACS
> > > > > to
> > > > > > extend most functionality. Unfortunately ACS is not packaged in
> > > > > > a
> > > > > modular
> > > > > > way. It is still delivered essentially as one large unit.
> > > > > > There are
> > > > > many
> > > > > > reason for this but one large barrier to modularizing ACS is
> > > > > > that the Spring configuration is managed as one large unit.
> > > > > >
> > > > > > I propose that we break apart the Spring XML configuration such
> > > > > > that each component contributes its own configuration.
> > > > > > Additionally each component will be loaded into its own Spring
> > > > > > ApplicationContext such that its beans will not conflict with
> > > > > > the wiring of other beans in ACS. This change
> > > > > will
> > > > > > lay the foundation for a richer plugin model and additionally a
> > > > > > more distributed architecture.
> > > > > >
> > > > > > The technical details for this proposal can be found on the wiki
> [1].
> > > > > >
> > > > > > Darren
> > > > > >
> > > > > > [1]
> > > > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+
> > > > > Spri
> > > > > ng
> > > > >
> > >
>
