I'm so sorry to have taken so long to answer: work has been terribly
busy.
I think it would be best to post this as an issue to the ASDF git repo
[here](https://gitlab.common-lisp.net/asdf/asdf/-/issues). I have never
even used the bundling operations (except for the one that assembles
`build/asdf.lisp`), much less looked into the code.
I'd suggest reading [ASDF issue
23](https://gitlab.common-lisp.net/asdf/asdf/-/issues/23) since it was
the attempt to fix this issue that led to the commit you pointed to.
I think in the short term you are best off creating the uber system.
On 17 Sep 2024, at 18:50, Kasper Gałkowski wrote:
Dear users and developers,
I'm having issues using the bundling ops, deliver-asd-op to be
specific.
Apparently, since commit c213e319fd9f6dc53570f0a05034ee31a24ae1ae, the
generated system definitions get mangled when deliver-asd-op is used
on slashy subsystems. Instead of foo/bar, the system name becomes
foo--bar. Previously, the pathname-name was set to the full complex
component name (sneakily including the forward slash character).
Consider this ABCL session (ASDF 3.3.7):
(load "https://common-lisp.net/project/asdf/archives/asdf.lisp";)
(ensure-directories-exist "~/common-lisp/testsys1/")
(uiop:with-output-file (o "~/common-lisp/testsys1/testsys1.asd")
(print '(defsystem "testsys1"
:components ((:file "test1")))
o)
(print '(defsystem "testsys1/sub"
:components ((:file "test2")))
o))
(uiop:with-output-file (o "~/common-lisp/testsys1/test1.lisp")
(print '(defpackage test1 (:use :cl)) o)
(print '(in-package test1) o)
(print '(defun test1 () 42) o))
(uiop:with-output-file (o "~/common-lisp/testsys1/test2.lisp")
(print '(defpackage test2 (:use :cl)) o)
(print '(in-package test2) o)
(print '(defun test2 () 43) o))
(asdf:clear-configuration)
(asdf:operate 'asdf:deliver-asd-op "testsys1")
(asdf:operate 'asdf:deliver-asd-op "testsys1/sub")
(uiop:println
(list
(asdf:output-files 'asdf:compile-bundle-op "testsys1")
(asdf:output-files 'asdf:deliver-asd-op "testsys1")
(asdf:output-files 'asdf:compile-bundle-op "testsys1/sub")
(asdf:output-files 'asdf:deliver-asd-op "testsys1/sub")))
The result:
(("~/.cache/common-lisp/.../testsys1/testsys1--system.abcl")
("~/.cache/common-lisp/.../testsys1/testsys1.asd")
("~/.cache/common-lisp/.../testsys1/testsys1--sub--system.abcl")
("~/.cache/common-lisp/.../testsys1/testsys1--sub.asd"))
Concatenating these .asd files and replacing the invalid system names
becomes an epic hack. At which point it might be easier to implement a
replacement asdf:operation. But it might be better to do it in ASDF
itself.
To keep these systems loadable, their definitions should land in the
parent .asd file. But if ASDF checks whether an operation is already
completed based on probe-file, it becomes a problem when multiple
slashy systems are involved. It could cause early cutoff. The
deliver-asd-op would have to check for the existence of a
corresponding defsystem form in the file, rather than the mere
existence of that file.
The incoming defsystems would have to be appended. I don't know
whether that should be automatic or require an explicit
invoke-restart. There might be some validation required to verify that
the primary system is also being delivered. (Correct?)
Fasls can probably stay mangled, since they can be targeted
independently, as opposed to .asd files... well, that's unless someone
decides to create a (defsystem "testsys1--sub") - though it does seem
unconventional. Maybe that would be an acceptable limitation.
Here are the results from SBCL, which still uses uses ASDF 3.3.1:
(("~/.cache/common-lisp/.../testsys1/testsys1--system.fasl")
("~/.cache/common-lisp/.../testsys1/testsys1.asd")
("~/.cache/common-lisp/.../testsys1/testsys1/sub--system.fasl")
("~/.cache/common-lisp/.../testsys1/testsys1/sub.asd"))
(By the way, I'm not sure how did the testsys1/testsys1 subdirectory
manage to get created without an error, since it seems that
testsys/sub is the pathname-name, and I believe
ensure-directory-exists works with pathname-directory.)
In this case, I'm lucky that the testsys1 project doesn't contain an
inner testsys1 directory containing another ASD file "sub.asd" - that
would cause another conflict, but I digress.
To summarize, regarding deliver-asd-op:
- The slashy system names and the .asd pathname-names are being
mangled
- The system definitions need to stay in one file
- DIY merging un-mangling them afterwards is too hairy
- Let's discuss encapsulating that hairiness in ASDF itself
The goal: to support binary deployment of slashy systems (non-slashy
systems already work), so that the generated .asd and .fasl can be
installed, without source, on client systems, all the while being
ASDF-discoverable.
An alternative workaround might be to create an ephemeral uber system
just for delivery, one that depends-on the parent system and every
desired slashy system. But that prevents fine grained updates and also
creates one more system to remember.
Robert P. Goldman
Research Fellow
Smart Information Flow Technologies (d/b/a SIFT, LLC)
319 N. First Ave., Suite 400
Minneapolis, MN 55401
Voice: (612) 326-3934
Email: rpgold...@sift.net
