Re: monolithic-concatenate-source-op misses a few dependencies

2017-10-14 Thread Ben Vulpes
Fare et al.,

The Postmodern/usocket problem I encountered was in fact due to
cl-postgres not setting a feature flag in an :execute context:
https://github.com/marijnh/Postmodern/blob/57dc8cb4acc4599aee757e8f81999a0fa83c7111/cl-postgres/features.lisp#L8
. To digress from the list topic for half a second, this confuses me,
as that code refused to execute until I patched cl-postgres at that
point to also eval-when in :execute, even when loading the
concatenated sources with sbcl ... --load full-concatenation.lisp

More relevant to the list, other observations include: chipz mutates
*features* in :chipz-system, a package defined in chipz.asd, which
causes problems because the .asd files are not concatenated with the
sources (not suggesting this is something asdf should even concern
itself with); and both drakma and usocket call (asdf:component-version
(asdf:find-system ...)) at read time, which fails to find the package
in question when loading the concatenated source on a machine without
those packages in *central-registry*, because (I believe) as noted
above, monolithic-concatenate-source-op doesn't include system .asd
files. Again, I'm not suggesting that this behavior change.

Yours,
Ben

On Sat, Oct 14, 2017 at 11:52 AM, Faré <fah...@gmail.com> wrote:
> On Sat, Oct 14, 2017 at 1:17 PM, Robert Goldman <rpgold...@sift.info> wrote:
>> Will you please clarify for my benefit, since I don't actually use any of
>> the image operations.
>>
>> Is the problem that somewhere in the process of loading Postmodern, or one
>> of its dependencies, some bit of code invokes REQUIRE? Or is this an issue
>> with ASDF's REQUIRE-SYSTEM.
>>
> monolithic-concatenate-source-op is supposed to create a source file
> that has "all the (transitive) source code required to load a system".
> Problem is, it is naive in that the current implementation believes
> there are only source files to concatenate. And so it doesn't handle
> (:require "foo") dependencies which would require to generate a
> temporary "source file" that contains (require "FOO") in it — or
> better a refactoring such that instead of source files, you use
> arbitrary arguments to vomit-output, and for require dependencies you
> use a function that outputs that instead of a pathname  (and wrap a
> map () 'vomit-output in a with-output-file).
>
>> If it's the former, then I believe this is simply a bad implementation in
>> the relevant system.
>>
>> If it's the latter, we should do something about it. However, I believe that
>> REQUIRE-SYSTEM, despite its name, doesn't actually use REQUIRE, instead it
>> "acts like require."
>>
> Well, there are unhappily two things called "require-system": the
> class that backs the (:require "foo") dependencies, and the function
> that is now deprecated because it didn't have a good composable
> meaning in presence of multiple build phases. The one that matters
> here is the former.
>
> —♯ƒ • François-René ÐVB Rideau •Reflection• http://fare.tunes.org
> All problems in computer science can be solved by another level of indirection
> — David Wheeler
> Almost all programming can be viewed as an exercise in caching
> — Terje Mathisen, well-known programming optimization guru



-- 
Ben Vulpes
503-313-5838
MAVN



monolithic-concatenate-source-op misses a few dependencies

2017-10-13 Thread Ben Vulpes
On SBCL 1.3.11, when producing a monolithic source concatenation with
the library "Postmodern", asdf produces a file that needs a Lisp image
to need manual calls to (require :usocket) and (require :md5) in order
to load completely.

Given:

concatenatrix.asd as:

(asdf:defsystem :concatenatrix
  :build-operation monolithic-concatenate-source-op
  :build-pathname "build/full-concatenation"
  :depends-on (:postmodern)
  :components
  ((:file "concatenatrix")))

concatenatrix.lisp as:

(defpackage :concatenatrix
  (:use :cl :postmodern))
(in-package :concatenatrix)

(defun wat (it)
  (format t "~A~%" it))

Concatenated sources produced with:

(asdf:make :concatenatrix)

Loading tested with:

sbcl --noinform --disable-debugger --load build/full-concatenation.lisp

Produces:

Unhandled SB-C::INPUT-ERROR-IN-LOAD in thread #:
  READ error during LOAD:

Package SB-ROTATE-BYTE does not exist.

  Line: 221, Column: 29, File-Position: 8706

  Stream: #


Backtrace for: #
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK # #)
1: (SB-DEBUG::RUN-HOOK SB-EXT:*INVOKE-DEBUGGER-HOOK*
#)
2: (INVOKE-DEBUGGER #)
3: (ERROR #)
4: (SB-C:COMPILER-ERROR SB-C::INPUT-ERROR-IN-LOAD :CONDITION
# :STREAM #)
5: (SB-C::%DO-FORMS-FROM-INFO # #
SB-C::INPUT-ERROR-IN-LOAD)
6: (SB-INT:LOAD-AS-SOURCE # :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
7: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) # NIL)
8: (LOAD #P"build/full-concatenation.lisp" :VERBOSE NIL :PRINT NIL
:IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
9: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:LOAD .
"build/full-concatenation.lisp")))
10: (SB-IMPL::TOPLEVEL-INIT)
11: ((FLET #:WITHOUT-INTERRUPTS-BODY-90 :IN SB-EXT:SAVE-LISP-AND-DIE))
12: ((LABELS SB-IMPL::RESTART-LISP :IN SB-EXT:SAVE-LISP-AND-DIE))

There is another complaint about usocket, which is confusing, as
cl-postgres explicitly doesn't require usocket on sbcl by my read (
http://marijnhaverbeke.nl/git/?p=postmodern;a=blob;f=cl-postgres.asd;h=683edf0f131a4ebe172b44425f597d7f67656e70;hb=HEAD#l16
).

Loading is resolved by requiring both libraries in question, as:

sbcl --eval "(require :md5)" --eval "(require :usocket)" --load
build/full-concatenation.lisp

Yours,
Benjamin