On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote:
On 30/11/06, Simon Laws <[EMAIL PROTECTED]> wrote:
>
> On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote:
> >
> > On 30/11/06, Pete Robbins <[EMAIL PROTECTED]> wrote:
> > >
> > >
> > >
> > > On 30/11/06, Simon Laws <[EMAIL PROTECTED]> wrote:
> > > >
> > > > On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote:
> > > > >
> > > > > On 30/11/06, Pete Robbins <[EMAIL PROTECTED]> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 30/11/06, Jean-Sebastien Delfino < [EMAIL PROTECTED]>
> > wrote:
> > > > > > >
> > > > > > > Pete Robbins wrote:
> > > > > > > > Our current method of packaging and loading an extension is
> > > > fairly
> > > > > > > > simple:
> > > > > > > > we load all schema and libraries in the extensions path.
> This
> > > > has a
> > > > > > > > number
> > > > > > > > of problems.
> > > > > > > >
> > > > > > > > 1. An extension may consist of more than one library e.g.
> > > > > > > > libmy_extension.so
> > > > > > > > and libmy_extension_utils.so. Our current loading scheme
> will
> > > > > attempt
> > > > > > > to
> > > > > > > > load both of these and may fail on the one that doesn't
> > provide
> > > > the
> > > > > > > > extension initialize method. On MacOS the output of our
> build
> > > > > produces
> > > > > > > a
> > > > > > > > libx.dylib and a series of symlinks to this called
> > libx.0.dylib,
> > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these which
> > doesn't
> > > > > cause
> > > > > > > > problems as they are all the sam library and just repeatedly
> > > > > register
> > > > > > > to
> > > > > > > > handle the same requests. Very inefficient though!
> > > > > > > >
> > > > > > > > 2. Control of whether or not to load an extension library is
> > > > > currently
> > > > > > > by
> > > > > > > > renaming the library so the runtime doesn't find it. An
> > example
> > > > is
> > > > > > > > that we
> > > > > > > > ship our python extension as
> libtuscany_sca_python.so.diabled.
> > > > This
> > > > > is
> > > > > > >
> > > > > > > > horrible and error prone.
> > > > > > > >
> > > > > > > > We could improve this by having a system configuration file
> > that
> > > > > lists
> > > > > > > > the
> > > > > > > > required extensions but the I like the self contained
> package
> > > > > approach
> > > > > > > > that
> > > > > > > > we have now. I'd like to implement an improved scheme for
> > > > packaging
> > > > > an
> > > > > > > > extension by introducing a per extension configuration file.
> > > > > Something
> > > > > > > > like:
> > > > > > > >
> > > > > > > >
> > > > > > > > tuscany_sca_ws_binding.extension
> > > > > > > >
> > > > > > > >
> > > > > > > > <scacpp:extension name="ws binding" enabled="true">
> > > > > > > > <library name="tuscany_sca_ws_reference"/>
> > > > > > > > <library name="tuscany_sca_ws_service"/>
> > > > > > > > </scacppp:extension>
> > > > > > > >
> > > > > > > > So the package would look like:
> > > > > > > >
> > > > > > > > extensions/
> > > > > > > > ws/
> > > > > > > > tusany_sca_ws_binding.extension
> > > > > > > > lib/
> > > > > > > > xsd/
> > > > > > > > other_folder/
> > > > > > > > ...
> > > > > > > >
> > > > > > > > The .extension configuration file is saying to load the
> > library
> > > > > which
> > > > > > > is
> > > > > > > > located somewhere in the package... the runtime will find
> > it...
> > > > no
> > > > > > > > need to
> > > > > > > > specify a path.
> > > > > > > >
> > > > > > > > Taking this further the configuration file could list the
> > schema
> > > > to
> > > > > be
> > > > > > > > loaded. Currently the runtime will just load any it finds
> but
> > > > these
> > > > > > > > may not
> > > > > > > > be needed by the runtime e.g. the schema may be for some
> > > > extension
> > > > > > > > implementation specific purpose.
> > > > > > > >
> > > > > > > > I think it would also be good for the extension
> > initialization()
> > > > > > > > method to
> > > > > > > > take as a parameter the root of the extension e.g.
> > > > > > > > extension("/tuscany/extensions/ws"). This would allow the
> > > > extension
> > > > > > > > package
> > > > > > > > to contain any configuration information that it needs.
> > > > > > > >
> > > > > > > > I'd like to start by at least introducing the .extension
> file
> > > > for
> > > > > each
> > > > > > > > extension and loading only the specified library(ies) if the
> > > > > extension
> > > > > > > is
> > > > > > > > enabled.
> > > > > > > >
> > > > > > > > Any thoughts?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > Two thoughts:
> > > > > > > - convention over configuration
> > > > > > > - the runtime should be consumable without having to go tweak
> > XML
> > > > > > > configuration files
> > > > > > >
> > > > > > > If I remember correctly, renaming the dlls to .disabled was a
> > last
> > > >
> > > > > > > minute change to work around DLL loading errors with our M2
> > > > release on
> > > > > > > Windows.
> > > > > > >
> > > > > > > I agree that we should do better than renaming to .disabled,
> but
> > > > I'd
> > > > > > > like to understand better the actual issues that we faced
> before
> > > > > > > inventing yet another XML based runtime configuration
> language
> > :)
> > > > > > >
> > > > > > > I'm aware of the following issues:
> > > > > > > 1. We need the runtime to load extension libs only, not other
> > libs
> > > > > under
> > > > > > >
> > > > > > > the extension directory which are not actual extensions
> > > > > > > 2. Same for XSDs, we need to load XSDs that contribute to
> SCDL,
> > > > and
> > > > > > > leave other XSDs under the extension directory alone
> > > > > > > 3. Extensions that cannot be loaded because some of their
> > > > dependencies
> > > > > > > are not there should no break the runtime
> > > > > > > 4. The system admin / installer should be able to disable
> > > > extensions
> > > > > > > that won't load because their dependencies are not there (I'm
> > not
> > > > yet
> > > > > > > convinced that this is still an issue if we manage to solve
> > issue
> > > > #3)
> > > > > > >
> > > > > > > Did we run into any other big issues?
> > > > > >
> > > > > >
> > > > > > No, that's about it.
> > > > > >
> > > > > > 2. XSD loading can be done by convention (schemas for the
> runtime
> > > > are
> > > > > > always in a folder called 'xsd'
> > > > > > 1. Could be solved in a similar way by only loading libraries in
> a
> > > > > folder
> > > > > > called ???
> > > > > > 3. Can be solved by just ignoring the load errors/issuing a
> > warning
> > > > > rather
> > > > > > than giving up
> > > > > > 4. Can be solved by solution to 3.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > I recall now why 3. was a big problem. Windows sometimes throws up
> > an
> > > > > error
> > > > > dialog when the load fails so it was not just a case of the
> runtime
> > > > > handling
> > > > > the load failure so we had to "disable" the extension.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > --
> > > > > Pete
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pete
> > > > >
> > > > > Pete, just summarizing so that I understand. It seems there are
> two
> > > > parts
> > > > to this.
> > > >
> > > > A/ Building and installing an extension in order that it can be
> > consumed
> > > > by
> > > > the runtime. Building and installing extensions is potentially a
> > > > separate
> > > > activity from building and installing the runtime itself. Looking at
> > the
> > > > M2
> > > > download they all come together at the moment.
> > >
> > >
> > > Yes. It just happens that we are the only providers of extensions at
> the
> > > moment.
> > >
> > > B/ Optionally consuming/enabling an extension in the runtime, once the
> > > > extension has been installed, in such a way that the runtime is able
> > to
> > > > find
> > > > and load it.
> > >
> > >
> > > Yes.
> > >
> > > If I understand the previous posts (and the current code) A/ is
> achieved
> > > > by
> > > > building the extension in the deployed folder layout
> > > > extension type/
> > > > bin
> > > > lib
> > > > xsd
> > >
> > >
> > > The current runtime does not mandate any layout. It will seek and
> > > destroy.. I mean seek and load any library it finds and any .xsd it
> > finds.
> > > That layout is just how we package extensions in ou build at the
> moment.
> > >
> > >
> > > and Pete you are proposing that this is adjusted so that there are
> > > > directories that hold just the stuff that the contains the extension
> > > > rather
> > > > than the things it depends on.
> > > > extension type/
> > > > bin
> > > > extensionbin?
> > > > lib
> > > > xsd
> > > > extensionxsd?
> > > >
> > > > You are also proposing that B/ is achieved by ensuring that this
> > > > directory
> > > > structure be placed in a location that the the runtime can search
> for
> > > > active
> > > > extensions (as it does at the moment in the deploy directory)
> > >
> > >
> > > it currently loads any extensions from under the
> > > <sca_install_dir>/extensions folder
> > >
> > > and that the
> > > > runtime ignores any badly configured extensions if possible.
> Removing
> > an
> > > > extension's directory from the deploy
> > >
> > >
> > > extensions not deploy
> > >
> > > directory structure has the effect of
> > > > disabling it so I guess this is the fallback if the runtime can't
> > > > continue.
> > >
> > >
> > > That is true in today's implementation.
> > >
> > > Anything more complex and, as Jean-Sebastien suggests, you start
> getting
> > > > into the full blown package and dependency management problem that
> > many
> > > > other systems try to solve in different ways, e.g. rpm or pear.
> > >
> > >
> > > Agreed... I'll give up on that plan ;-)
> > >
> > >
> > > What we need is to enable the runtime to identify which extension
> > > libraries to load and which schema to load. My suggestion is:
> > >
> > > my_extension/
> > > bin/
> > > extension/
> > > library_that_will_be_loaded_by_the_runtime.so
> > > lib/
> > > lib_my_extension.so
> > > include/
> > > ... some headers maybe
> > > xsd/
> > > my_binding.xsd
> > > any_other_folder/
> > > any_stuff_I_like.xsd
> > >
> > > So the runtime will load any xsd that is in a folder named xsd and
> > attempt
> > > to load any library in a folder called extension. It would not attempt
> > to
> > > load the any_stuff_I_like.xsd or the lib_my_extension.so
> > >
> > >
> > > Cheers,
> > >
> > > --
> > > Pete
> > >
> >
> >
> >
> > --
> > Pete
> >
> > Sounds like the right idea to me. What goes in the bin dir?.
Anything you like!
On windows, .exe's and .dll's are often found in bin, whereas the
.lib's (used for linking against at compile time) are generally found
in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which
is more unix-like, and, IMO, nicer)
Should the
> lib be...
>
> lib/
> lib_my_extension.lib
lif you like... again... anything you like.
Maybe you could stick with bin,include, lib,xsd and then have your "other"
> directory have everything else in it that the extension might rely on.
> Depends what you now anticipate being in bin and lib over and above the
> actual library exposing the extension.
>
> Will you continue to mandate a specific place where extensions are
> installed, i.e. from you post <sca_install_dir>/extensions folder?
Yes
So that
> you can move an extension out of there if you don't want it to load.
Well the idea here is you don't need to remove the whole package..just the
library from the extension dir
Simon
>
>
The layout of the extension package is whatever the package creater wants to
make it. We don't care. All we will care about is loading any library in a
folder anywhere in the extension package that is in a folder called
'extension' and expecting it to implement the extension initialize method.
We will also load any schema files in any folder called 'xsd'. An extension
only needs the xsd folder if it has schema required for the runtime model.
As an example here is the ws binding extension implemented for axis2c:
<sca_install>/extensions/
... somewhere in this tree... could be anywhere...
ws/
xsd/
ws-binding.xsd
reference/
lib/
libtuscany_sca_reference.so
service/
lib/
libtuscany_sca_service.so
OK... today the runtime would load the ws-binding.xsd as it loads any xsd it
finds. It would also load any .so it finds so both reference aand service
extensions would be loaded/initialized.
With my proposed change the xsd would be loaded (as it's in a folder named
'xsd' ) but the libraries would not. To get the extensions loaded you would
have to package with an 'extension' folder ... again anywhere in the tree...
which "contains" the libraries to load eg.
ws/
extension/
libload_this.so -> reference/lib/libtuscany_sca_reference.so
xsd/
ws-binding.xsd
reference/
lib/
libtuscany_sca_reference.so
service/
lib/
libtuscany_sca_service.so
Here the runtime loads anything in any folder named 'extension' so the
reference extension would be loaded/initialized.
Windows doesn't really do symlinks - I guess on windows we'd have the
tuscany_sca_reference.dll in extension/ and nothing in reference/lib/
or reference/bin/? For extensions that have dependencies on other
libraries, those dll's could go in lib/
I would package the cpp language extension as
cpp/
bin/
lib/
include/
xsd/
extension/
The bin, lib, include are exactly what you would expect from a package that
you might want to build against. The extension/ folder would hold the
library that implements the extension interface and xsd holds... well xsds!
All sounds good to me. +1
Andy
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
