Yes this is one of the many things this is trying to address. Adding a plugin should be plopping your jar in a directory and restarting. You pointed out two things I didn't think about though, command.properties and log4j xml. Let me think about those twos as they should be address also. Basically you should never have to edit a file that is packaged as part of ACS. Only add your artifacts to some directory, ideally just a jar.
Darren On Wed, Sep 18, 2013 at 5:46 PM, SuichII, Christopher < chris.su...@netapp.com> wrote: > I've been following this conversation somewhat and would like to throw in > some background as a plugin writer. > > One thing that concerns me in the current plugin model is the number of > XML/text files that need to be edited to deploy my plugin. > -applicationContext must be edited to add our PluginMangerImpl. > -commands.properties file must be edited to included the permissions for > the APIs we contributed. > -componentContext.xml & nonossComponentContext.xml must be edited to add > our Storage Subsystem Provider API. > -log4j-cloud.xml must be edited to ensure that our logger is enabled and > logging to our necessary default level. > > I know our situation is a bit different than the current plug in model, > but I think it is something we, as a community, are going to begin seeing > more of. For a variety of reasons that I won't get in to right now, our > plugin will be closed source and kept separate from the ACS source tree. We > want our users to be able to simply drop in our jar file to the CS > directory or run and installer and have it picked up by the MS upon a > restart. > > It sounds like what you are proposing here would be very beneficial to > plugins that are targeting a deployment model like this. > > Is this something you're looking/hoping/would like to solve, Darren? > > -Chris > -- > Chris Suich > chris.su...@netapp.com > NetApp Software Engineer > Data Center Platforms – Cloud Solutions > Citrix, Cisco & Red Hat > > On Sep 18, 2013, at 6:44 PM, Darren Shepherd <darren.s.sheph...@gmail.com> > wrote: > > > I'm not a committer > > > > > > On Wed, Sep 18, 2013 at 3:24 PM, Frank Zhang <frank.zh...@citrix.com> > wrote: > > > >> Well. The codes explain more than words. > >> It seems the only extra work is adding a property file that specifies > >> parent context and current context name, it's not much complex. > >> BTW: any reason for working on repo outside ACS? > >> > >>> -----Original Message----- > >>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] > >>> Sent: Wednesday, September 18, 2013 2:43 PM > >>> To: dev@cloudstack.apache.org > >>> Subject: Re: [PROPOSAL] Modularize Spring > >>> > >>> If you want to see this all working you can just fetch the "no-at-db4" > >>> branch at https://github.com/ibuildthecloud/cloudstack.git > >>> > >>> Plugin composes multiple modules. If modules are siblings they can't > >> inject > >>> from each other. But a plugin can augment another module if it chooses > >> to. > >>> The reality is that the core cloudstack is a tangled mess of > >> dependencies such > >>> that most of the core code can't be modularized as it stands. So there > >> exists a > >>> context towards the top of the hierarchy called "core" that a lot of > jars > >>> contribute to it. Here is the full hierarchy right now. I'll probably > >> rename a > >>> bunch of things, but this gives you an idea. > >>> > >>> bootstrap > >>> system > >>> core > >>> allocator > >>> allocator-server > >>> planner > >>> api-planner > >>> baremetal-planner > >>> explicit-dedication > >>> host-anti-affinity > >>> implicit-dedication-planner > >>> server-planner > >>> user-concentrated-pod-planner > >>> random-allocator > >>> api > >>> acl-static-role-based > >>> rate-limit > >>> server-api > >>> user-authenticator-ldap > >>> user-authenticator-md5 > >>> user-authenticator-plaintext > >>> user-authenticator-sha256salted > >>> backend > >>> alert-adapter-server-backend > >>> compute > >>> alert-adapter-server-compute > >>> baremetal-compute > >>> fencer-server > >>> investigator-server > >>> kvm-compute > >>> ovm-compute > >>> server-compute > >>> xenserver-compute > >>> network > >>> baremetal-network > >>> elb > >>> midonet > >>> nvp > >>> ovs > >>> server-network > >>> ssp > >>> storage > >>> alert-adapter-server-storage > >>> allocator-storage > >>> baremetal-storage > >>> secondary-storage > >>> server-storage > >>> storage-image-default > >>> storage-image-s3 > >>> storage-image-swift > >>> storage-volume-default > >>> storage-volume-solidfire > >>> template-adapter-server > >>> discoverer > >>> baremetal-discoverer > >>> discoverer-server > >>> ovm-discoverer > >>> xenserver-discoverer > >>> > >>> > >>> > >>> If you look at the baremetal hypervisor plugin that is pretty cross > >> cutting to > >>> most of ACS. So the modules it contributes to are as follows > >>> > >>> resources/META-INF/cloudstack/baremetal-storage/spring-context.xml > >>> resources/META-INF/cloudstack/baremetal-compute/spring-context.xml > >>> resources/META-INF/cloudstack/baremetal-discoverer/spring-context.xml > >>> resources/META-INF/cloudstack/core/spring-baremetal-core-context.xml > >>> resources/META-INF/cloudstack/baremetal-planner/spring-context.xml > >>> resources/META-INF/cloudstack/baremetal-network/spring-context.xml > >>> > >>> So it creates child contextes of storage, compute, network, planner, > and > >>> discoverer to add its extensions where it needs to be. Additionally, > >> you'll notice, > >>> it adds some beans to the core context from the file resources/META- > >>> INF/cloudstack/core/spring-baremetal-core-context.xml. This is because > >> it has > >>> some manager class that is used by multiple contexts. > >>> > >>> Frank, I understand the scare that we are going too complex, but do you > >> have > >>> some other suggestion? I don't like the idea of one gigantic spring > >> context. So I > >>> feel I am making it as simple as I can while maintaining some order. > >> People > >>> just need to create one or more spring xml files and a properties files > >> that says > >>> where to put it in the hierarchy. > >>> > >>> Additionally, by putting forcing people to put beans in certains > modules > >> it > >>> forces them to think about what is the role of the code. For example, > >> today in > >>> ACS the *ManagerImpl classes are a huge mess. They implement too many > >>> interfaces and the code crosses to many architectural boundaries. Its > >> about > >>> time we start splitting things up to be more maintainable. > >>> > >>> If you have some time, please check out what I have on github. > >>> > >>> Darren > >>> > >>> > >>> On Wed, Sep 18, 2013 at 1:56 PM, Frank Zhang <frank.zh...@citrix.com> > >>> wrote: > >>> > >>>> I am not against boundary, I am just against making things unnecessary > >>>> complex to enable boundary. > >>>> If we are going this way, I hope we can make it as much as transparent > >>>> to developers. That means, as a developer, all a plugin I need to do > >>>> is 1) provide my separate spring xml 2) inject beans I want (valid > >>>> beans) in my bean and code business logic 3) compile to a jar and put > >>>> to some place designated by CloudStack. That's it. > >>>> > >>>> I raise this topic because I have seen some projects to create > >>>> boundary making things horrible complex. And sometimes developers are > >>>> hurt by wrong boundaries, as a result, to overcome these limitations > >>>> people write lots of ugly code which makes thing even worse. > >>>> > >>>> However, I am still worry about if we can make things so simpler. > >>>> For example, we may have an orchestration context that contains major > >>>> beans needed by almost every plugin, this context can be easily set > >>>> as parent context for all plugin contexts when bootstrap. However, if > >>>> a plugin A needs to access some bean defined in plugin B, given they > >>>> are sibling, how plugin framework resolves the dependency ? > >>>> > >>>>> -----Original Message----- > >>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] > >>>>> Sent: Wednesday, September 18, 2013 11:53 AM > >>>>> To: dev@cloudstack.apache.org > >>>>> Subject: Re: [PROPOSAL] Modularize Spring > >>>>> > >>>>> I'm not for OSGi either, but contexts are quite useful and will lead > >>>>> to > >>>> better > >>>>> things. First off, we don't want one gigantic spring XML config > >>>>> file > >>>> like we have > >>>>> today. I think we can all agree on that. So each plugin will have > >>>>> to > >>>> supply its > >>>>> own XML. So the obstacles you mention, will largely be just that > >>>>> for > >>>> people. > >>>>> > >>>>> With Spring it is really simple to just inject dependencies and > >>>>> cross architectural boundaries. Its currently everywhere in ACS. > >>>>> You can't > >>>> just say > >>>>> we should review code and make sure nobody doesn't do bad things. A > >>>> little bit > >>>>> of framework to enforce separation is a good thing. But I'm > >>>>> guessing > >>>> you will > >>>>> disagree with me there. > >>>>> > >>>>> Here are some random points on why contexts are good. Say I want to > >>>>> use Spring AOP or Spring TX in my plugin. With your own context you > >>>>> can > >>>> ensure > >>>>> that you won't screw with anybody else code by accidentally having > >>>>> you pointcut match their bean. You don't have to worry about bean > >>>>> name > >>>> conflicts. > >>>>> If two config files specify bean X, Spring will gladly just use the > >>>>> last > >>>> one. I've > >>>>> already found multiply defined beans in ACS, but things still just > >>>> happen to work. > >>>>> Having multiple contexts also defines initialization order. We can > >>>> ensure that > >>>>> the framework is loaded and ready before child context are loaded > >>>>> and > >>>> started. > >>>>> (we kind of do this today with ComponentLifeCycle, but its a hack in > >>>>> my > >>>> mind). > >>>>> Additionally, when things start you will know, loading context > >>>>> "crapping > >>>> plug X". > >>>>> If spring fails to initialize, the issue it there. Today, if spring > >>>> fails to start, it > >>>>> could be one of over 500 beans causing the weird problem. The list > >>>>> goes > >>>> on > >>>>> and on. > >>>>> > >>>>> Finally, this is the big one and why I really want contexts. I have > >>>> some notes on > >>>>> the wiki [1] that you might want to read through. Basically I want > >>>>> to > >>>> get to a > >>>>> more flexible deployment model that allows both a single monolithic > >>>>> JVM > >>>> as > >>>>> today and also a fully distributed system. Having contexts in > >>>>> hierarchy > >>>> will > >>>>> enable me to accomplish this. By selecting which contexts are > >>>>> loaded at runtime will determine what role the JVM takes on. The > >>>>> contexts help > >>>> people > >>>>> better understand how the distributed architecture will work too, > >>>>> when > >>>> we get > >>>>> there. > >>>>> > >>>>> Frank, trust me, I hate complex things. I don't want OSGi, > >>>>> classloader > >>>> magic, > >>>>> etc. But I do like organization and a little bit of framework so > >>>>> that > >>>> people don't > >>>>> accidentally shoot themselves in the foot. I personally like > >>>>> knowing > >>>> that I > >>>>> couldn't have screwed something up, because the framework won't even > >>>> allow > >>>>> it. If we separate everything as I want today, and then tomorrow we > >>>>> say > >>>> this is > >>>>> way too much overhead, moving to a flat context is simple. Don't > >>>>> think > >>>> we are > >>>>> on some slippery slope to classloaders and dependency hell. > >>>>> > >>>>> Darren > >>>>> > >>>>> [1] > >>>>> > >>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Nothing+to+see+ > >>>> her > >>>>> e...#Nothingtoseehere...-DeploymentModels > >>>>> > >>>>> > >>>>> > >>>>> On Wed, Sep 18, 2013 at 11:22 AM, Frank Zhang > >>>>> <frank.zh...@citrix.com>wrote: > >>>>> > >>>>>> What's the point in using separate spring context per plugin? > >>>>>> Separate class loader is the thing I hate most in OSGI, I am > >>>>>> afraid we are on the same way. > >>>>>> Frankly speaking, I never see benefits of this *separate* model, > >>>>>> our project(or most projects) is not like Chrome which has to > >>>>>> create sandbox for extensions in order to avoid bad plugin screws > >>>>>> up the whole browser(however, I still see bad plugins screw up my > >>>>>> Chrome well). > >>>>>> Projects like CloudStack using plugin to decouple architecture > >>>>>> should not introduce many isolations to plugin writer, the point > >>>>>> preventing wrong use of some components is not making much sense > >>>>>> to me. If a plugin not following guide(if we have > >>>>>> it) we should kick it out, instead of making obstacles for 99% > >>>>>> good > >>>> people. > >>>>>> > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] > >>>>>>> Sent: Wednesday, September 18, 2013 10:33 AM > >>>>>>> To: dev@cloudstack.apache.org > >>>>>>> Subject: Re: [PROPOSAL] Modularize Spring > >>>>>>> > >>>>>>> Right, component isn't a thing. I probably shouldn't use that > >> term. > >>>>>>> I > >>>>>> want to > >>>>>>> standarize on the naming convention of plugin, module, and > >> extension. > >>>>>> It is > >>>>>>> explained a bit on the wiki [1] but I'll try to do a little > >>>>>>> better job > >>>>>> here. So a > >>>>>>> plugin is basically a jar. A jar contains multiple modules. A > >>>>>>> modules > >>>>>> ends up > >>>>>>> being a spring application context that composes multiple > >>>>>>> configuration > >>>>>> files. > >>>>>>> Modules are assembled into a hierarchy at runtime. Extensions > >>>>>>> are implementations of interfaces that exist in a module. A > >>>>>>> maven project produces a jar, so a plugin ends up being a maven > >>>>>>> project > >>>> also. > >>>>>>> > >>>>>>> So currently today we don't have a strong definition of Plugin > >>>>>>> and I > >>>>>> hope to > >>>>>>> address that. > >>>>>>> > >>>>>>> Darren > >>>>>>> > >>>>>>> [1] > >>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Plug- > >>>>>>> ins%2C+Modules%2C+and+Extensions > >>>>>>> > >>>>>>> > >>>>>>> On Wed, Sep 18, 2013 at 4:25 AM, Daan Hoogland > >>>>>>> <daan.hoogl...@gmail.com>wrote: > >>>>>>> > >>>>>>>> sounds great Darren, > >>>>>>>> > >>>>>>>> By component, you mean maven project or some larger chunk like > >>>>>>>> distribution package? (did i miss this definition somewhere or > >>>>>>>> do we define the components now?) > >>>>>>>> > >>>>>>>> regards, > >>>>>>>> Daan > >>>>>>>> > >>>>>>>> On Wed, Sep 18, 2013 at 12:10 AM, Darren Shepherd > >>>>>>>> <darren.s.sheph...@gmail.com> wrote: > >>>>>>>>> Currently ACS code is fairly modular in that you can add > >>>>>>>>> plug-ins to ACS > >>>>>>>> to > >>>>>>>>> extend most functionality. Unfortunately ACS is not > >>>>>>>>> packaged in a > >>>>>>>> modular > >>>>>>>>> way. It is still delivered essentially as one large unit. > >>>>>>>>> There are > >>>>>>>> many > >>>>>>>>> reason for this but one large barrier to modularizing ACS is > >>>>>>>>> that the Spring configuration is managed as one large unit. > >>>>>>>>> > >>>>>>>>> I propose that we break apart the Spring XML configuration > >>>>>>>>> such that each component contributes its own configuration. > >>>>>>>>> Additionally each component will be loaded into its own > >>>>>>>>> Spring ApplicationContext such that its beans will not > >>>>>>>>> conflict with the wiring of other beans in ACS. This change > >>>>>>>> will > >>>>>>>>> lay the foundation for a richer plugin model and > >>>>>>>>> additionally a more distributed architecture. > >>>>>>>>> > >>>>>>>>> The technical details for this proposal can be found on the > >>>>>>>>> wiki > >>>> [1]. > >>>>>>>>> > >>>>>>>>> Darren > >>>>>>>>> > >>>>>>>>> [1] > >>>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modular > >>>>>>>> ize+ > >>>>>>>> Spri > >>>>>>>> ng > >>>>>>>> > >>>>>> > >>>> > >> > >
