Re: [racket-users] Compilation/Embedding leaves syntax traces

['Paulo Matos' via Racket Users]

On 25/09/2018 20:11, Alexis King wrote:
> (Sorry, Paulo, for the duplicate message; I forgot to Reply All the
> first time.)
> 
> This is sort of subtle. When we consider a macro-enabled language, we
> often imagine that `expand` takes a program with some phase ≥1 code,
> expands all the macros in the program by running the phase ≥1 code, and
> produces a fully-expanded program with only phase 0 code left. There is
> some truth to this, but it doesn’t paint the whole picture.
> 
> [snip] [snip]

Alexis, thanks for the thorough reply. I understood everything, at least
up until this point.


> The above explains why Racket retains some phase ≥1 code. However, it
> may be unsatisfying: while it’s true that the phase ≥1 code might be
> necessary for compilation of other modules, once you have compiled your
> whole program, it shouldn’t be necessary to keep that information
> around, right? 

Right! That's exactly what I was thinking...

> No other modules will ever need to be compiled against
> the macro-providing module. However, this is not necessarily true!
> Racket provides a set of reflective operations for compiling modules at
> runtime, and it makes no assumptions that all modules will be loaded
> from compiled code. In this sense, Racket includes an “open-world
> assumption” when compiling modules, and it retains any phase ≥1 code
> necessary for compiling new modules at any time.
> 

OK, so I understand now that what I want is an unimplemented feature,
but in most compilers these days and certainly those based in LLVM and
GCC there's a feature called whole-program optimization or link time
optimization. Basically the compiler will get the whole program in
memory after compiling each module/file and run the optimizations on the
whole thing again therefore being able to optimize things it wasn't able
to optimize before when it only had a module/file view.

Now, in Racket when I compile an executable, although it's true there
might be dynamic-requires, if you look at the example I posted there's
not even one. Surely it's possible to remove all the phase>=1 code,
correct? Is it just the case that this kind of global optimization is
not yet implemented?

Even with dynamic-requires, if the dynamic-require depends on a compile
time variable that contains the path, after compilation the
dynamic-require won't change and will always require the same file,
therefore we can do the same kind of phase >= 1 code cleanup.

Am I missing any subtlety here or are these feasible but we are just
missing these optimizations?

> This sort of thing is necessary to implement tools like DrRacket, which
> frequently compile new modules at runtime, but admittedly, most programs
> don’t do any such thing. Personally, I would appreciate a way to ask
> Racket to strip all phase ≥1 code and phase ≥1 dependencies from a
> specified program so that I can distribute the phase 0 code and
> dependencies exclusively. However, to my knowledge, Racket does not
> currently include any such feature.
> 

Again, here I assume that in some cases, like the ones I mentioned above
you wouldn't even have to ask. It could be done automatically.

> For more information on declaring, instantiating, and visiting modules,
> and how that relates to compilation, see this very helpful section in
> The Racket Guide:
> 
>    http://docs.racket-lang.org/guide/macro-module.html
> 

Thank you for the reference.

Paulo Matos
> 
>> On Sep 25, 2018, at 07:32, 'Paulo Matos' via Racket Users 
>> <racket-users@googlegroups.com> wrote:
>>
>>
>> Hi,
>>
>> I reached a point at which I don't think I am exactly understanding how
>> the racket compilation pipeline works.
>>
>> My software has several compile time options that use environment
>> variables to be read (since I can't think of another way to do it) so I
>> define a compile time variable as:
>>
>> (define-for-syntax enable-contracts?
>> (and (getenv "S10_ENABLE_CONTRACTS") #true))
>>
>> And then I create a macro to move this compile-time variable to runtime:
>> (define-syntax (compiled-with-contracts? stx)
>> (datum->syntax stx enable-contracts?))
>>
>> I have a few of these so when I create a distribution, I first create an
>> executable with (I use create-embedding-executable but for simplicity,
>> lets say I am using raco):
>> S10_ENABLE_CONTRACTS=1 raco exe ...
>>
>> I have a bunch of other options that don't matter for the moment.
>>
>> One of the things I noticed is that in some cases when I run my
>> executable, compile time code living inside begin-for-syntax to check if
>> a variable has been defined during compilation or not is triggered. At a
>> point, which I didn't expect any more syntax expansion to occur.
>>
>> I can't really reproduce the issue with a small example yet but I
>> noticed something:
>>
>> main.rkt:
>>
>> #lang racket
>>
>> (require (file "arch-choice.rkt"))
>>
>> (module+ main
>> (printf "arch: ~a~n" (get-path)))
>>
>> arch-choice.rkt:
>>
>> #lang racket
>>
>> (provide get-path)
>>
>> (begin-for-syntax
>>
>> (define arch-path (getenv "ARCH"))
>>
>> (unless arch-path
>>   (raise-user-error 'driver "Please define ARCH with a suitable path")))
>>
>> (define-syntax (get-path stx)
>> (datum->syntax stx arch-path))
>>
>> Then just to make sure nothing is compiled I remove my zos:
>> $ find . -type f -name '*.zo' -exec \{\} \;
>>
>> Then compile it:
>> $ ARCH=foo raco exe main.rkt
>>
>> In this case if you run ./main you'll get 'arch: foo' back which is fine
>> so I can't reproduce what I see in my software which is with some
>> combinations of compile time options, I see:
>> 'driver: Please define ARCH environment variable'
>>
>> which should even be part of the executable because it's a compile time
>> string (or so I thought).
>>
>> So I did on the above example:
>> $ strings main | grep ARCH
>> PLANET-ARCHIVE-FILTER
>> ARCH"
>> ''Please define ARCH with a suitable path
>>
>>
>> OK, so, this agrees with what I see in my program: compile-time error
>> strings still exist in the code. Why is that? I thought that only fully
>> expanded code (compiled-code) would make it to the executable file.
>>
>> Another thing that might help me understand what's going on, is there a
>> way to extract the bytecode from the executable and decompile it?
>>
>> Thanks,
>>
>>
>> -- 
>> Paulo Matos
> 

-- 
Paulo Matos

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.