Hi, Felix--
Thanks for your reply. I have some clarifications and responses below ...
On Tue, Jan 15, 2013 at 12:42 PM, Felix <
fe...@call-with-current-continuation.org> wrote:
> > I am developing a project which I expect will involve a number of
> extension
> > libraries, or plugins (a large number, many of them provided by third
> > parties, if my project ever becomes popular). For several reasons (which
> I
> > will explain on request if anyone is curious), I feel it is best *not* to
> > implement these libraries as eggs. So I am trying to work out a
> reasonable
> > method for deploying the libraries and loading them at runtime.
>
> Sure. What does "plugin" mean, though? A dynamically loadable library?
> I assume it is compiled code? Written in Scheme?
>
Actually, I have realized that my question conflates two similar but
distinct issues. So here's what's going on, specifically: I am developing a
web CMS that is intended to be somewhat like Drupal, only better. Yes, I'm
probably quite insane, but at least you can't say I lack ambition ;-)
The immediate problem I'm working on is a database layer. And although I am
currently supporting just one database, that one is Redis, and I am
anticipating that some users will be more comfortable with a traditional
RDBS, say PostgresQL or SQLite (just don't say the M word to me, 'kay?). So
I would like to be able to build in the capability to choose different
backends. And since it is often not possible for people to compile code in
a web hosting environment, it should be possible to make the choice at
runtime. In any case, the database code will be compiled Chicken libraries
installed in a known location. So there shouldn't be any problem finding
the libraries; the issue with this component is how to load one or another
implementation of a unified interface on demand, and how to manage the
namespace. The ideal (which I know doesn't work and is not supposed to
work), would be something like:
(module sappari-lib ; main library for the framework
.... ; various exports
(let ((backend
(string->symbol
(or
(get-environment-variable "SAPPARI_DB")
"redis")))
(case backend
((redis) (import (prefix sappari-db-redis db:)))
((pgsql) (import (prefix sappari-db-pgsql db:)))
...
[BTW .... my understanding of the module system is a bit vague, but I've
seen that the docs say something like "modules are purely a syntactic
construct" ... so am I right in thinking that means that modules really
don't exist at runtime ... and thus, if conditional imports were possible
at all, they would only be possible at compile time?]
The other problem is loading the arbitrary extension libraries that users
of my CMS can install at will from a huge online collection (there's my
megalomania talking again ;). Although there would be a potentially
infinite number of different extensions, [I think] they would really all be
fancy HTTP request handlers. Here the main problem is just having a
reliable way to find the appropriate library in the filesystem. Regarding
compilation or lack thereof, my plan is to recommend compiling all
libraries, but as mentioned above, I expect there are use cases where users
would not be able to compile any code, so I would like to give them the
option of installing a binary that could load the extensions as interpreted
code--if it's not unreasonably difficult to do so.
> > 1) Is there a way to set an arbitrary search path, such that REQUIRE (or
> > LOAD, or some such thing) will work when the library is in a non-standard
> > location?
>
> You could implement your own routine (probably based on "load"), which
> scans a number of directories. "repository-path" is for eggs, and "require"
> respects it. You can also add directories to "##sys#include-pathnames".
> I think "require" will try this too.
>
I tried this, and on my system it only works for 'include', not for
'require'. BTW, I have Chicken 4.8.0 on Arch Linux. I compiled the package
myself, but only in order to have the latest version; i.e. I didn't hack
anything. There were no patches, and the build() function in the PKGBUILD
script is:
build() {
cd ${pkgname}-${pkgver}
# parallel builds are not supported..
export MAKEFLAGS="${MAKEFLAGS/-j?/}"
# some yummy make options there. check out README in the tarball.
make PLATFORM=linux PREFIX=/usr
}
Nothing unusual there, right?
> > 2) Is it possible to load a library selected at runtime (via an
> environment
> > variable, config file, command-line argument ... the exact method isn't
> > that important) AND have the symbols defined in that library included in
> a
> > module?
>
> You can compute arbitrary file-paths and pass them to "load". To make
> the symbols available, you have to make sure the import-libraries for
> the modules are available. How do you use the symbols in the
> libraries? Do you evaluate code at runtime?
>
See above re: compilation/evaluation. As far as using symbols goes, I've
haven't gotten to a detailed plan yet, but I think each plugin would
register itself/its functions in a data structure established by the main
library. With an open-ended collection of arbitrary components, I can't see
any other sane way to do it.
>
> Hm. You can copy the "C_path_to_executable(argv[ 0 ])" in "chicken.h",
> and use it via the foreign-function interface" to get the pathname
> of an executable.
Ah, thank you! I was planning to look for a function something like that,
but had no idea what its name might be. I'll try it. Are there any
non-obvious pitfalls I should be aware of in using this?
> > 4) Is LOAD-RELATIVE broken? I wrote some test code to try to use that
> > procedure, but as far as I can tell it behaves just like LOAD, i.e. any
> > relative path I give it is determined relative to the current directory
> > from which the program is invoked. Though I would note that the
> > documentation isn't entirely clear to me: "loads FILE relative to the
> path
> > of the currently loaded file." ...? What does "the currently loaded file"
> > mean? I will be happy to post my code if this is not a known issue.
>
> It means that if loading is chained, nested "load-relative" calls with
> load files relative to the outer "load"/"load-relative" invocations. e.g.
file a.scm:
>
> ...
> (load "foo/b.scm")
> ...
>
> file foo/b.scm:
>
> ...
> (load-relative "c.scm") ; <- will load "foo/c.scm"
> ...
>
Ohhhh, I see. This works for interpreted but not compiled code. At least
that's how it is for me. Is that expected behavior? I didn't understand
that from the docs.
> Perhaps, if you describe the implementation of your plugins and the
> way the code therein is invoked, we can give more detailed advice.
>
Okay, I'll be glad to describe it once I get to that stage, which should be
within the next couple of months, if death or the short-term need for
income don't get in the way.
Thanks for all the info!
--
Matt Gushee
_______________________________________________
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users
