Re: UTF16 encoding adds BOM everywhere?

[Mark H Weaver]

Hi,

Jean Abou Samra  wrote:

> With this code:
> 
> (let ((p (open-output-file "x.txt")))
>    (set-port-encoding! p "UTF16")
>    (display "ABC" p)
>    (close-port p))
> 
> the sequence of bytes in the output file x.txt is
> 
> ['FF', 'FE', '41', '0', 'FF', 'FE', '42', '0', 'FF', 'FE', '43', '0']
> 
> FFE is a little-endian Byte Order Mark (BOM), fine.
> But why is Guile adding it before every character
> instead of just at the beginning of the string?
> Is that expected?

No, this is certainly a bug.  It sounds like the
'at_stream_start_for_bom_write' port flag is not being cleared, as it
should be, after the first character is written.  I suspect that it
worked correctly when I first implemented proper BOM handling in 2013
(commit cdd3d6c9f423d5b95f05193fe3c27d50b56957e9), but the ports code
has seen some major reworking since then.  I guess that BOM handling was
broken somewhere along the way.

I would suggest filing a bug report.  I don't have time to look into it,
sorry.  I don't work on Guile anymore.  I only happened to see your
message by chance.

 Regards,
   Mark

-- 
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about .

Re: Weird behavior of hash-table

[Mark H Weaver]

Hi tomás,

 wrote:

> So the best thing for one's brain is to train it to read #(...)
> as some weird relative of '(...)?

Yes.  They are both literals, and no part of a literal is evaluated.
#(...) is actually a shorthand for '#(...), and incidentally, one which
was not standardized until the R7RS.  In portable R[3456]RS code, you
must explicitly quote vector literals.

It's important to distinguish between an expression and the value(s) it
evaluates to.  It's easy to confuse them in Scheme, where every
expression is a value, but not necessarily the same value that it
evaluates to.  For example, the expression (+ 1 1) is a list of length 3
which evaluates to the number 2.  Similarly, 'A1, which is a shorthand
for (quote A1), is a list of two symbols which evaluates to the symbol
A1.  Only the "self-evaluating" values evaluate to themselves.

In general, you should not put a quote (') anywhere that you could not
write (+ 1 1) in place of 2, unless of course you *want* a list starting
with the symbol 'quote'.

> Is there a corresponding weird relative of `(...)?

Yes.  Remember that #(...) is a shorthand for '#(...).  You can replace
the "'" with "`", just as you would for a quasiquoted list.  For
example:

  scheme@(guile-user)> `#(1 2 3)
  $1 = #(1 2 3)
  scheme@(guile-user)> `#(1 ,(+ 1 1) 3)
  $2 = #(1 2 3)
  scheme@(guile-user)> `#(1 (+ 1 1) 3)
  $3 = #(1 (+ 1 1) 3)

  Regards,
Mark

Re: Weird behavior of hash-table

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> I've noticed a strange behavior of hash tables. I put in symbols as keys
> and integers as values, but when I try to get the integers out again by
> using the same symbol, I get back a #f instead. Here is the code I am using:
>
> 
> (use-modules
>  ;; SRFI 60: procedures for treating integers as bits
>  (srfi srfi-60)
>  (ice-9 hash-table))
>
>
> (define SQUARES
>   ;; vector, constant time access
>   #('A1 'B1 'C1 'D1 'E1 'F1 'G1 'H1
> 'A2 'B2 'C2 'D2 'E2 'F2 'G2 'H2
> 'A3 'B3 'C3 'D3 'E3 'F3 'G3 'H3
> 'A4 'B4 'C4 'D4 'E4 'F4 'G4 'H4
> 'A5 'B5 'C5 'D5 'E5 'F5 'G5 'H5
> 'A6 'B6 'C6 'D6 'E6 'F6 'G6 'H6
> 'A7 'B7 'C7 'D7 'E7 'F7 'G7 'H7
> 'A8 'B8 'C8 'D8 'E8 'F8 'G8 'H8))

I guess that you meant for this to be a vector of symbols.  In fact, it
is a vector of lists of the form (quote ), for which '
is a shorthand.

  scheme@(guile-user)> (vector-ref SQUARES 0)
  $10 = (quote A1)

To fix the problem, remove all of the apostrophes (') from the elements
of the vector literal above.  Like list literals, vector literals take
raw values, not expressions.

> (hash-ref SQUARES-TO-INTEGERS 'A1)

  scheme@(guile-user)> (hash-ref SQUARES-TO-INTEGERS 'A1)
  $11 = #f
  scheme@(guile-user)> (hash-ref SQUARES-TO-INTEGERS ''A1)
  $12 = 1

   Mark

Re: guile-json, SRIFs and licenses

[Mark H Weaver]

Hi John,

John Cowan  writes:

> [...]  There are two traditional arguments
> in favor of putting libraries under the GPL:

[...]

> 2) "Benefit GPLed programs":  if a library is GPLed, it supposedly gives
> the advantages of using that library only to GPLed applications.

You're mistaken.  A GPLed library can be used by any program covered by
a *GPL-compatible* license.

> I think history shows that this doesn't work very well in the long run:
> rather than accepting the GPL, a lot of duplicative effort was put into
> libedit/editline,

It's a mistake to conclude from this that it didn't work well.  Avoiding
duplicative effort is _not_ a core goal of the Free Software Movement.

A vastly more important goal is that free software should exist for all
of the jobs that people want to use their computers for, so that users
need not sacrifice their freedom to do their computing.  The GNU GPL has
helped *enormously* with that goal.  For example, we gained our first
free C++ compiler thanks to the GNU GPL, because they built it on GCC.
Ditto for Objective C, and CLISP.

Those are major historical examples, but the same thing happens quite
frequently and unremarked in smaller examples, whenever a boss would
reflexively tell an employee that a program they wrote should be kept
proprietary, but is compelled to make it free software because it
depends on a copylefted library (such a Guile-JSON).

See  for more on this.

Providing an inducement to others make more software into free software
is in fact the main goal of the GNU GPL, and you didn't even include it
in your list of "traditional arguments in favor of putting libraries
under the GPL".

> Whether or not a GPLed JSON library requires the Scheme implementation
> to be itself GPL depends on the implementation, but certainly a
> stand-alone *application* that uses it would have to be.

Again, you are mistaken.  Check your facts, please.  See
.

  Thanks,
Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

John Cowan  wrote:
> Dual licensing is also a possibility, so that users who prefer the GPLv3
> license would still have it.

Dual licensing with a lax license option would eliminate the main
advantage of copyleft, namely that it offers an inducement to those who
wish to use Guile-JSON to release their code as free software.

See  for more.

 Thanks,
   Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

Hi John,

John Cowan  writes:

> Distinguo.  I have not *pressured* the author to dual license; that is, I
> have not exercised force or undue influence (of which I have none).

According to , "pressure"
means: "To encourage or heavily exert force or influence", and according
to , "encourage" means
(among other senses) "To spur on, strongly recommend."

As an example use of the word "pressure", wiktionary offers this:

  Do not let anyone pressure you into buying something you do not want.

For another source, according to
:

  "If you put pressure on someone, you try to cause that person to do
  something by persuading or threatening them"

Here, I used the word "pressure" in the sense of to "persuade" or to
"spur on, strongly recommend."

Therefore, I think I used the word correctly here.

  Thanks,
Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

Hi Aleix,

Aleix Conchillo Flaqué  wrote:
> If at some point I decide to switch to the MIT license I assume
> the projects that currently use guile-json would still be able to use it
> (probably renamed to some SRFI number) the same way they use other SRFIs,
> right?

Yes, as long as it's GPL-compatible.  The X11 and Expat licenses are
both GPL-compatible.  The ambiguous term "MIT license" usually refers to
one of those two licenses.

  Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

I wrote:
> Zelphir could propose a SRFI for the API only, with no reference
> implementation.  The existing implementation could remain copylefted.

John Cowan  replied:
> Unfortunately not.  A SRFI must have a sample implementation to get
> finalized (otherwise it is feared there would be too many WIBNI SRFIs).
> The implementation doesn't have to be portable, but it has to have the
> correct license.

That's unfortunate.  I stand corrected.

Nonetheless, I find it regrettable that you've chosen to pressure a
library author on a GNU project forum to abandon copyleft licensing.
Personally, I don't think that SRFIs are important enough to justify
bowing to such pressure.  Guile-JSON's association with Guile and Guix
is enough for it to gain wide adoption, whether it is a SRFI or not.
That said, it is of course Aleix's decision how to license his library.

  Thanks,
Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

John Cowan  writes:

> +1.  If only it weren't GPL3, which makes it ineligible to be a SRFI
> implementation

Zelphir could propose a SRFI for the API only, with no reference
implementation.  The existing implementation could remain copylefted.

   Mark

Re: Diversification [ branched from Re: conflicts in the gnu project now affect guile]

[Mark H Weaver]

Hi Todor,

Todor Kondić  writes:

> [...]  I've set up my workflows around Guix, git(lab)
> and a customised Emacs installation (instead of R Studio). My small
> team of science students (majority female, various cultural
> backgrounds), never previously exposed to a GNU system to such an
> extent, managed to get a handle on it quite impressively.
>
> But, I doubt any of them would find it natural to take a step further
> and participate in GNU itself (ugh, now I sound like a preacher of a
> new age religion). To my knowledge, interaction within GNU communities
> is still mostly mailing lists and IRC. This _not_ my students' natural
> digital habitat. I am probably not saying anything new, though ...

You raise an important issue.  If we can improve the situation without
causing other problems, I think we should.  I don't know of any modern
replacement for mailing lists that has the properties we need, but I
*do* think there's a very promising alternative for live chat: Matrix.
Amirouche mentioned it elsewhere in this thread.

  https://matrix.org/

Matrix is supported by a very large and diverse set of free clients,
from modern Web-based interfaces to simple text programs, multiple
Emacs-based clients, and several gateways to other protocols such as
IRC, so that old-timers can use their preferred IRC client if they
prefer.

  https://matrix.org/clients/

Incidentally, there was recently an internal GNU project discussion
about how to better communicate with one another, and Matrix was
identified as an option that would meet our requirements.

The client that would likely be most attractive for the younger
generation is Riot.im:

  https://about.riot.im/

What do you think?

Thanks,
  Mark

Re: Interactive Debugging

[Mark H Weaver]

"Thompson, David"  writes:

> On Fri, Oct 18, 2019 at 8:40 AM Matt Wette  wrote:
>>
>> On 10/17/19 9:39 PM, Christopher Howard wrote:
>> > Hi, it seems like with the flexibility of Guile, I should be able to do
>> > something like this:
>> > ```(define (foo)  (let ((a 1)(b 2)(c 3))(jump-into-
>> > debugging-repl)))```
>> > And have access to a, b, and c and their values. But I'm not quite
>> > figuring out how to this.
>> >
>>
>> Define dump-into-debugging-repl as
>>
>> (start-repl #:debug (make-debug (stack->vector (make-stack #t)) 0
>> "trap!" #t))
>
> I think something like this should be shipped in the (system repl
> debug) module.  I'm used to quick and easy access to a debugger in
> other languages, such as Ruby, and having to wrap your head around the
> REPL, debug, and stack code is too much to ask of any new Guiler.

For what it's worth, I think something like this would be a welcome
addition.  However, my knowledge of the Guile debugger is currently
quite weak, so I'd prefer to hear from Andy before proceeding.

 Thanks,
   Mark

Re: Stepping back up as a co-maintainer

[Mark H Weaver]

zx spectrumgomas  writes:

> Of your words: “RMS has not yet appointed me as a co-maintainer.” , the
> word “yet” I guess that means if he would have had his permission then you
> wouldn't have needed it of the current maintainers, as I also deduce from
> here:
> “Frankly, it should not be your decision to make.”
> https://lists.gnu.org/archive/html/guile-devel/2019-10/msg00021.html

Yes, and I stand by that statement.  I made my case clearly in the
message cited above.

That said, I agree that it would have been improper for RMS to appoint
me without first consulting the existing maintainers.  In fact,
discussions are currently ongoing with Andy and Ludovic to resolve the
situation, and I'm hopeful that we'll reach a mutually acceptable
resolution.

Regarding my phrase "In light of recent events", which I cited as the
reason for my decision to return, I should now be clear what I meant by
that:

GNU is far more fragile than I had supposed.  It's like if your mother
has a heart attack, and you suddenly realize that although you've always
taken her presence in your life for granted, her mortality has now
become real to you in a way that it wasn't before.

I realize now that I took GNU's continued existence and integrity for
granted, like a steady rock that I naively assumed would be there and
stay strong regardless of whether I was a part of it or not.  Now I see
that it will depend on all of us who care about software freedom to do
what we can to keep GNU strong.

I'm not interested in dividing the Guile community.  That would
obviously be disastrous.  I fully support Andy's work, and I recognize
that he is the main driver in core Guile development.  His continued
participation is crucial.  Still, we must guard against Guile becoming a
one-man show if it can possibly be avoided.  Keeping GNU strong means,
among other things, resolving our differences peacefully and working
together to make Guile better.  That's what I intend to do.

I hope that clarifies things.

  Thanks,
Mark

Re: Stepping back up as a co-maintainer

[Mark H Weaver]

zx spectrumgomas  writes:
> It's milk
> https://lists.gnu.org/archive/html/guile-devel/2019-10/msg00031.html

I'm sorry, but I don't understand what this means.  If you are accusing
me of something, can you state your case clearly, without reference to
proverbs?

   Mark

Re: Weird Guile Scheme Behaviour

[Mark H Weaver]

Hi Philip,

phi...@warpmail.net (Philip K.) writes:

> I was reading a thread on an imageboard[0] the other day, then I came
> across a most peculiar "bug", if it even is one. Since the original
> example was a bit dense (it tried to solve a problem someone else had
> posted, that's not relevant here), I tried to construct a minimal
> working example to discuss here.
>
> Compare
>
> (define (reverse-iota-1 max)
>   (let ((numbers '(start)))
> (let loop ((val 0))
>   (append! numbers
>(if (< max val)
>'()
>(begin
>  (loop (1+ val))
>  (list val)
> numbers))
>
> and
>
> (define (reverse-iota-2 max)
>   (let ((numbers '(start)))
> (let loop ((val 0))
>   (append! numbers
>(if (< max val)
>'()
>(begin
>  (loop (1+ val))
>  (list val)
> (cdr numbers)))
>
> (I know, the style is horrible, but that's not the point. Also, both
> have an internal state, so you have to re-eval the function every time
> before starting the function itself.)
>
> The only difference is in the last line. The first function returns the
> entire list (with the start symbol), and the second tries to chop it
> off.
>
> But what happens is that (reverse-iota-1 4) evals to '(start 3 2 1 0)
> while (reverse-iota-2 4) just returns '()!

The problem here is that '(start) is a *literal* list, and it's an error
to modify literals in Scheme.  In other words, modifying a literal
results in undefined behavior.  Like C string literals, Scheme literals
are actually part of the program text itself.

To fix this problem, instead of initializing 'numbers' to point to a
literal list, you must construct a fresh mutable list.  One way is by
replacing '(start) with (list 'start).

What's happening here is this: Since it's an error to modify a literal
list, and since the 'numbers' variable is never 'set!' within its
lexical scope (the only scope where a variable can possibly be set! in
Scheme), Guile's compiler is allowed to assume that 'numbers' will
always point to the literal '(start), and therefore that (cdr numbers)
can be optimized to '().

The other problem, as Neil and Vladimir correctly pointed out, is that
'append!' is permitted, but not required, to modify the original list.
Therefore, you must always do (set! numbers (append! numbers ...)) to
avoid relying on unspecified behavior.

Note that it's impossible in Scheme to implement an 'append!' procedure
that always modifies the original list.  Specifically, an empty list
cannot be destructively modified.  It can't be done for the same reason
that you cannot implement a C function with the following specification:

  struct pair { void *item; struct pair *next; };
  void appendx (struct pair *list_to_modify, struct pair *list_to_add);

This C function can only be implemented in the case where
'list_to_modify' has at least one element.  In that case, the 'next'
field of the last pair can be modified.  If 'list_to_modify' is NULL, it
can't be done, because 'appendx' doesn't have access to the variable
that contained NULL.

In Scheme, the same issues apply.  If there's at least one element in
the first list passed to 'append!', it can use 'set-cdr!' to modify the
last pair of the list.  If the first argument is '(), it can't be done.

In practice, that's the reason why 'append!' is specified the way it is.
However, I would advise against assuming that 'append!' will always
modify the original list if it's nonempty, because that fact, although
true of the current implementation, is not actually specified.

  Regards,
Mark

Stepping back up as a co-maintainer

[Mark H Weaver]

Hello all,

In light of recent events, I've decided to step back up as a
co-maintainer of GNU Guile.

 Thanks,
   Mark

Re: How to use-modules within macro?

[Mark H Weaver]

Hello again,

I wrote earlier:
> So, instead of using 'match' on the result of 'syntax->datum', you
> should instead use 'syntax-case' on the syntax object itself, like this
> (untested):
>
>   (let loop ((e #'exp))
> (syntax-case e ()
>   (num
>(number? (syntax->datum #'num))
>#'(1+ num))
>   ((x ...)
>(map loop #'(x ...)))
>   (y
>#'y)))

I should mention that the use of 'map' directly on a syntax object is
only allowable in certain cases.  Here, we assume that the syntax object
produced by #'(x ...) is a normal Scheme list.  That is _not_ the case
for arbitrary syntax objects that correspond to a list.  For example, it
would _not_ be safe to pass 'e' as the list argument to 'map', although
'e' is in some sense equivalent to #(x ...) in the second clause above.

The reason is that syntax objects contain information about the
associated lexical environment which starts at the top of the expression
tree, and is *lazily* pushed down into the subexpressions as the tree is
taken apart using 'syntax-case' and put back together using 'syntax'
(a.k.a. "#'").

As a result, there are only a few cases when you can safely assume that
the top structure of a syntax object is a normal list or pair, and they
are spelled out in the documentation for 'syntax' in the R6RS Standard
Libraries specification:

  http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-13.html#node_sec_12.4

Here's the relevant excerpt:

  The output produced by syntax is wrapped or unwrapped according to the
  following rules.

  * the copy of ( . ) is a pair if  or  contain any
pattern variables,

  * the copy of ( ) is a list if  contains any pattern
variables,

  * the copy of #( ... ) is a vector if any of , ..., 
contain any pattern variables, and

  * the copy of any portion of  not containing any pattern variables
is a wrapped syntax object.

A "wrapped syntax object" is one where the lexical environment
information has not yet been pushed down into the subexpressions.  It is
a special kind of object that you can only take apart using
'syntax-case'.

So, in the clause above where 'map' is used, 'e' might be a "wrapped
syntax object", but when the elements are extracted from it using the
'syntax-case' pattern (x ...) and then put back together using
#'(x ...), you can then assume that the resulting syntax object is
a normal Scheme list of syntax objects, and therefore it is safe to
use 'map' on it.

   Mark

Re: extract documentation from (define ...

[Mark H Weaver]

Hi Jesse,

Jesse Gibbons  writes:

> I am trying to generate documentation for a project written in guile. I
> tried "guild doc-snarf" [0] but apparently it only recognizes
> documentation in double-semicolon comments. A lot of the project is
> documented in a string like one would document an emacs-lisp function.
> Is there a program to extract that kind of documentation?
>
> [0]: http://git.savannah.gnu.org/cgit/guile.git/tree/module/scripts/doc
> -snarf.scm

I would use 'object-documentation' procedure from (ice-9 documentation),
which is what underlies the ,describe REPL command.

   Best,
Mark

Re: How to use-modules within macro?

[Mark H Weaver]

Hi Florian,

"pelzflorian (Florian Pelz)"  writes:

> To retain unhygienic references, I am now using datum->syntax instead
> of local-eval.  It is much better.  For example, to make a macro that
> increments all numbers in a given program by one:
>
> (use-modules (ice-9 match))
> (define-syntax one-more
>   (lambda (x)
> (syntax-case x ()
>   ((_ exp)
>(datum->syntax
> #'exp
> (let loop ((y (syntax->datum #'exp)))
>   (match y
> ((? number?) (1+ y))
> ((? list?) (map loop y))
> (else y
>
> (let ((four 4))
>   (one-more (* 2 3 four)))
>
> Yields 48.  I hope this is the right approach for rewriting programs.

There are some problems above:

(1) The first argument to 'datum->syntax' must be an identifier, which
is the syntax object corresponding to a symbol.  Here, you are
passing an entire expression, and in the example usage above, #'exp
will be the syntax object corresponding to (* 2 3 hour).  Guile
should ideally raise an error in this case.

(2) The way you are doing things here destroys hygiene within the
expression that you are rewriting.  You convert the entire
expression with 'syntax->datum', process the datum, and then convert
the rewritten expression using 'datum->syntax'.  The problem here is
that 'syntax->datum' discards all of the extra information about
lexical environments of identifiers that were kept in the syntax
object.  This will cause severe problems when 'one-more' is used in
combination with other macros, including unintended variable
capture.

To do this properly, you must do the rewriting on the syntax objects
themselves.  It's okay to convert a syntax object to a datum to test
whether it's a literal number, but the important thing is that all
*identifiers* in the rewritten code should be preserved.

So, instead of using 'match' on the result of 'syntax->datum', you
should instead use 'syntax-case' on the syntax object itself, like this
(untested):

  (let loop ((e #'exp))
(syntax-case e ()
  (num
   (number? (syntax->datum #'num))
   #'(1+ num))
  ((x ...)
   (map loop #'(x ...)))
  (y
   #'y)))

Finally, I should mention that macro expansion is always done from the
outside in, meaning that when 'one-more' is expanded, its operand will
not yet have been expanded.  In general, this means that it's impossible
to comprehend the code within a macro's operands unless the parsing code
knows about every macro that might be used within the operands.  It's
not even possible to know which subparts are expressions and which are
other things like variable binding lists.

For this reason, I think it's generally a mistake to try to parse code
within a macro's operands.  It normally only makes sense for macros to
inspect the parts of operands that are considered part of the macro's
syntax.  For example, it makes sense for a 'let' macro to parse its
binding list, or for a 'match' macro to parse its patterns and
templates, but it does *not* make sense for a macro to try to parse
general subexpressions passed to the macro.

If you could give me a birds-eye view of what you're trying to do here,
I might be able to suggest other approaches.

   Best,
Mark

Re: How to use-modules within macro?

[Mark H Weaver]

Hi Florian,

"pelzflorian (Florian Pelz)"  writes:

> I am writing a Guile macro to manipulate Scheme code and am stuck on
> what I hope is a simple problem and it would be nice if you could
> explain.  I try:
>
> (define-syntax O
>   (lambda (x)
> (syntax-case x ()
>   ((_)
>#`(begin (use-modules (ice-9 local-eval))
> (local-eval 42 (the-environment)))
> (pk (O))

This approach is misguided and unnecessary.  You don't need to include
'use-modules' in your macro expansion, and it's best avoided.

The references to 'local-eval' and 'the-environment' in the macro
template above will refer to bindings present in the module where 'O' is
defined, *not* the module where 'O' is used.  This is part of what it
means to say that 'O' is a "hygienic" macro.

Therefore, all you need to do is make sure (ice-9 local-eval) is
imported in the module where 'O' is defined, like this:

--8<---cut here---start->8---
(define-module (my-module-that-exports-O)
  #:use-module (ice-9 local-eval)
  #:export (O))

(define-syntax O
  (lambda (x)
(syntax-case x ()
  ((_)
   #`(local-eval 42 (the-environment))
--8<---cut here---end--->8---

Does that work for you?

FYI, the way this works internally is that the macro expander operates
on "syntax objects" instead of plain S-expressions.  The main difference
is that "syntax objects" keep additional information about the lexical
environments where the embedded identifiers were originally found.  So
when a use of (O) expands into (local-eval 42 (the-environment)), the
identifiers 'local-eval' and 'the-environment' are looked up in the
proper environment.

By the way, another consequence of hygiene, which you probably don't
want here, is that the (the-environment) above will capture the lexical
environment where 'O' was *defined*, instead of the environment where
(O) is used.  In other words, in (let ((x 5)) (O)), the captured lexical
environment will not include 'x'.

I should also mention that using (the-environment) will pretty much
disable most compiler optimizations that would otherwise occur with that
top-level form.  That entire mechanism is best avoided if at all
possible.

Can you tell me more broadly what you are trying to accomplish here?
I may be able to suggest an alternate approach.

 Best,
  Mark

Re: Problem installing the intended version

[Mark H Weaver]

Hi Thomas,

Thomas Morley  writes:

> In the light of your explanations I think the current problem only
> occurs for _equal_ major/minor with _different_ micro versions.
> Correct?

Yes.

>> and finally "ldconfig" as root.
>
> Doing so I get:
> /sbin/ldconfig.real: /usr/local/lib/libguile-2.0.so.22.8.1-gdb.scm is
> not an ELF file - it has the wrong magic bytes at the start.
> /sbin/ldconfig.real: /usr/local/lib/libguile-2.2.so.1.3.1-gdb.scm is
> not an ELF file - it has the wrong magic bytes at the start.
>
> And I've got even more notifications of this kind before I started to
> do the "make uninstall"-thingy.
> Is this expected?

Yes.  The warnings are harmless, although mildly annoying.

The issue is that GDB wants us to put that .scm file in the same
directory as libguile, in order to have GDB auto-load Guile debugging
support for executables linked with libguile.  See section 23.4.1 (The
'OBJFILE-gdb.EXT' file) in "Debugging with GDB":

  
https://sourceware.org/gdb/current/onlinedocs/gdb/objfile_002dgdbdotext-file.html

Unfortunately, 'ldconfig' warns about it.  I seem to recall that we
reported the problem to the GDB developers at some point in the past.
Nowadays, most of the Guile developers seem to be using GNU Guix, where
'ldconfig' is never run, so we tend to forget about it.

 Regards,
   Mark

Re: Problem installing the intended version

[Mark H Weaver]

Hi Thomas,

Thomas Morley  writes:

> recently I tested building LilyPond against guile-2.2.6 without
> success[*] and tried to investigate whether the problem is on the
> guile-side or at LilyPond (there are some patches which may or may not
> cause the problem).
>
> Though I stumbled across a problem which may be caused by guile or my
> naivity ...
>
> I had built guile-2.2.6 from the git repository and installed it via
> `sudo make install´
> Then I did (in/for a fresh repo):
>
>   git clone git://git.sv.gnu.org/guile.git ~/guile-2.2.4
>   cd guile-2.2.4/
>   git checkout v2.2.4
>   git checkout -b guile-2.2.4
>   sh autogen.sh
>   ./configure
>   make
>
> All successfull, then some tests:
>
>   ./meta/guile --version
>   -->  guile (GNU Guile) 2.2.4 [...]
>   ./meta/guile-config --version
>   --> guile-config - Guile version 2.2.4
>
> Then installed it systemwide
>
>   sudo make install
>
> And made some testings:
>
>   which guile
>   --> /usr/local/bin/guile
>   /usr/local/bin/guile --version
>   --> guile (GNU Guile) 2.2.6.1-a69b5
>
> Obviously 2.2.4 was _not_ installed but 2.2.6 from another repo.
>
> What am I missing?

What's happening here is that although the executable
/usr/local/bin/guile is from 2.2.4, it is being dynamically linked to
the newer libguile-2.2.so.1.4.1 from 2.2.6 that you installed earlier.
The libguile from guile 2.2.4 is libguile-2.2.so.1.3.1.

This happens because you never uninstalled the newer
libguile-2.2.so.1.4.1, and the library versioning information that we
assigned indicates that libguile-2.2.so.1.4.1 is a drop-in replacement
for libguile-2.2.so.1.3.1.  In other words, to our knowledge, any
executable linked with libguile-2.2.so.1.3.1 should also work properly
when linked with libguile-2.2.so.1.4.1.  That's because we added new
interfaces but did not change or remove any existing interfaces.

If you want to prevent libguile-2.2.so.1.4.1 from guile 2.2.6 from being
used, you should delete it, and then run 'ldconfig' as root to update
the links and caches.  Hopefully after that, libguile-2.2.so.1 and
libguile-2.2.so should be symlinks to libguile-2.2.so.1.3.1.

A more thorough approach, to make sure that no components of 2.2.6 are
being used, would be to run "make uninstall" from the Guile 2.2.6 build
directory (if you still have it), then rerun "make install" from the
Guile 2.2.4 build directory, and finally "ldconfig" as root.

   Best,
Mark

Re: yet another Guile tutorial for Linux (work in progress)

[Mark H Weaver]

Hi Basile,

Basile Starynkevitch  writes:

> I am writing yet another Guile tutorial. It is work in progress and
> available on http://starynkevitch.net/Basile/guile-tutorial-1.html
> under CC-BY-SA 3 license.
>
> Your comments are welcome.

Thanks for this.  I will try to find time to review it more carefully at
some point, but for now I have just one comment:

When referring to the operating system, please give GNU proper credit by
calling the system "GNU/Linux".  Linux is a kernel, not an operating
system.  The practice of calling the entire system "Linux" does great
harm to the GNU project, and arguably to the free software movement as a
whole, by greatly reducing the number of users of GNU/Linux systems who
learn about the crucially important freedoms that we are trying to
protect.

Anyway, I'm not sure why GNU/Linux is relevant to much (if any) of this
tutorial.  From a quick skim, I don't see anything here that wouldn't
work on FreeBSD, Solaris, or OS X.  Most of it would be the same on
Windows, for that matter.  The kernel is certainly not relevant.

Also, what do you mean by a "Linux developer"?  Do you mean to refer to
free software developers, or POSIX developers, or GNU/Linux developers,
or software developers in general?

It seems to me that most of the references to "Linux" are superfluous,
because almost all of this will work on any system that Guile supports.

What do you think?

  Regards,
Mark

Re: syntax-case identifier name predicate in guard expression

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> Hi Guile Users!
>
> I made some progress in writing a procedure defining macro for creating
> procedures which talk to an API.
>
> I now have working code, which checks the name of an identifier in a
> guard expression:
>
> 8<8<8<
> (use-modules (web uri)
>  (web client)
>  (json)
>  (ice-9 iconv)
>  (ice-9 regex))
>
>
> (define* (send-request-to-docker-socket request-url docker-socket
> my-content-type #:key (data #f))
>   (call-with-values
>   (lambda ()
>     (http-get request-url
>   #:port docker-socket
>   #:version '(1 . 1)
>   #:keep-alive? #f
>   #:headers `((host . ("localhost" . #f))
>   (content-type . (my-content-type (charset
> . "utf-8"
>   #:body (scm->json-string data)
>   #:decode-body? #t
>   #:streaming? #f))
>     (lambda (response response-text)
>   (let ([resp-text-as-string (bytevector->string response-text
> "utf-8")])
>     (cons response resp-text-as-string)
>
>
> (define-syntax define-api-route
>   (lambda (stx)
>     (define (identifier-name->string id)
>   (symbol->string (syntax->datum id)))
>
>     ;; Not needed yet.
>     ;; (define (identifier->symbol id)
>     ;;   (syntax->datum id))
>
>     (define (contains-url-template-variable? route-as-string)
>   (string-match "<[^>]+>" route-as-string))
>
>     ;; We do not need a macro to produce syntax. Instead we need to use
> a procedure to produce the
>     ;; syntax, because we want to use it while evaluating another macro.

Right.  Before, you were trying to perform the check within a procedural
macro, whose code was run when the 'define-api-route' macro was being
_defined_, which is too early.  Here you have the right idea.

>     (define make-simple-api-route-definition-syntax
>   (lambda (route http-method my-content-type route-as-string)
>     (syntax
>  (quote simple-route
>
>     (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
>   [(_ route GET my-content-type)
>    (contains-url-template-variable? (identifier-name->string (syntax
> route)))
>    (syntax (quote aaa))]
>   ;; an else branch basically
>   [(_ route GET my-content-type) #t
>    (syntax
>     (define* (route docker-socket #:key (data #f))
>   (call-with-values
>   (lambda ()
>     ;; GET request because syntax GET was specified
>     (http-get request-url
>   #:port docker-socket
>   #:version '(1 . 1)
>   #:keep-alive? #f
>   #:headers `((host . ("localhost" . #f))
>   (content-type . (my-content-type
> (charset . "utf-8"
>   #:body (scm->json-string data)
>   #:decode-body? #t
>   #:streaming? #f))
>     (lambda (response response-text)
>   (let ([resp-text-as-string (bytevector->string
> response-text "utf-8")])
>     (cons response resp-text-as-string))])))
>
>
> (define* (connect-to-docker-socket #:key (socket-path
> "/var/run/docker.sock"))
>   (let ([docker-sock-addr (make-socket-address AF_UNIX socket-path)]
>     [docker-sock (socket PF_UNIX SOCK_STREAM 0)])
>     (setsockopt docker-sock SOL_SOCKET SO_REUSEADDR 1)
>     (connect docker-sock docker-sock-addr)
>     docker-sock))
>
>
> (display
>  (define-api-route /containers/json GET
> "application/x-www-form-urlencoded"))
>
>
> ;; (let ([dock-sock (connect-to-docker-socket)])
> ;;   (let ([resp-and-resp-text
> ;;  (/containers/json dock-sock
> ;;    #:data '(("all" . "true")
> ;; ("filters" . (("name" . #("db"))
> ;;   ("status" .
> #("running" "exited"))])
> ;; (display resp-and-resp-text)
> ;; (newline)))
>
>
> (display
>  (define-api-route /container/json GET "application/x-www-form-urlencoded"))
> 8<8<8<
>
> However, it seems, that now I have a new problem. It seems I cannot use
> a define form inside a syntax-case case!

No, that's not it.  The problem is not with the macro itself, but with
how you're attempting to use it.

The problem is simply that the macro expands into a definition, and you
are trying to use it within an expression context, where definitions are
not allowed, here:

> (display
>  (define-api-route /containers/json GET
> "application/x-www-form-urlencoded"))

This fails for the same reason that the following fails:

  (display (define foo 1))

> While it was possible to have a define form inside a syntax-rules,
> suddenly it seems impossible inside a syntax-case?

For purposes of this issue, it is irr

Re: srfi-9 vs make-record-type

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> Hi All
> In experiments converting legacy code to use srfi-9 records, I'm finding
> the latter doesn't travel well across modules.
>
> See simple code below -- m1.scm runs fine however m2.scm borks when
> creating srfi-9 record object
>
> Any clue why srfi-9 can't be exported?
>
> For various reasons I must keep (load "module.scm") mechanism
> m1.scm follows
> (use-modules (srfi srfi-9))
>
> (define-record-type 
>   (make-person name age)
>   person?
>   (name person-name set-person-name!)
>   (age person-age set-person-age!))
>
> (define  (make-record-type "pet" '(name age)))
> (define make-pet (record-constructor ))
> (define pet? (record-predicate ))
> (define pet-name (record-accessor  'name))
> (define pet-age (record-accessor  'age))
> (define set-pet-name! (record-modifier  'name))
> (define set-pet-age! (record-modifier  'age))
> (export make-person)
> (export make-pet)
>
> (display "pet ")
> (let ((pet2 (make-pet "milou" 7)))
>   (display (pet-name pet2)))
> (display ", person ")
> (let ((person2 (make-person "james" 54)))
>   (display (person-name person2)))
> (newline)
> m2.scm follows
> (load "m1.scm")
> (display "in m2:")
> (newline)
> (display "pet ")
> (let ((pet2 (make-pet "milou" 7)))
>   (display (pet-name pet2)))
> (display ", person ")
> (let ((person2 (make-person "james" 54)))
>   (display (person-name person2)))
> (newline)
> 
> $guile m1.scm runs successfully
>
> pet milou, person james
>
> $guile m2.scm first runs m1.scm but fails to recognise the srfi-9
> make-person exists:
> pet milou, person james
> in m2:
> pet milou, person Backtrace:
>6 (apply-smob/1 #)
> In ice-9/boot-9.scm:
> 705:2  5 (call-with-prompt _ _ #)
> In ice-9/eval.scm:
> 619:8  4 (_ #(#(#)))
> In ice-9/boot-9.scm:
>2312:4  3 (save-module-excursion _)
>   3831:12  2 (_)
> In /home/chris/sources/caca/m2.scm:
>  8:15  1 (_)
> In unknown file:
>0 (_ "james" 54)
>
> ERROR: Wrong type to apply: #

The problem here is that 'make-person' is a macro.  Macros are expanded
at compile time, and must therefore be available at compile time.

When you compile 'm2.scm', those macros are not available, because
'load' only has effects at run-time, by design.  Since there is no macro
definition for 'make-person' at compile-time, Guile's compiler generates
code that assumes it will be a procedure.

In summary, you cannot use 'load' to import macros from another module.

> For various reasons I must keep (load "module.scm") mechanism

If you can share those reasons, perhaps I can help you find another
solution.

  Regards,
Mark

Re: Write a macro which defines a procedure

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> At first I had the macro use `define`, but then I thought: "What if I
> want to conditionally define a route procedure?". My guess is, that then
> the define form would be inside some `cond` or `if` form and then it
> would not work

You're right, it would not work.  Nonetheless, I would strongly
discourage you from doing this kind of thing unless there's a very
compelling reason for it.

Note that we have 'cond-expand' from SRFI-0 for conditional inclusion of
definitions, if you need it.  'cond-expand' is roughly analogous to #if
in C.  It's less general than plain 'if', but there's an important
reason for that, and it has to do with the distinction between compile
time and run time.

If you use 'cond-expand', the condition is checked at compile time, and
consequently it is known at compile time which branch was taken and
therefore which definitions were chosen.

If you use 'cond', then the condition will be checked at run time, which
will normally rule out the possibility of optimization opportunities
such as inlining and partial evaluation.  At present, Guile is not able
to perform these optimizations for toplevel level definitions in any
case, but it might be possible in the future, at least for modules that
are not explicitly mutated using low-level operations like this.

Also, it's possible that a future version of Guile may be more
disciplined in its handling of phases, i.e. expand-time
(i.e. compile-time) vs run-time phases.  See section 7.2 (Import and
export levels) of the R6RS for details on one approach, which would
bring many advantages.  It's been a while since I looked at this
closely, and at this point I would need some time to refresh my memory
of the details, but it seems to me that the code you've written would
likely cause problems with such an approach.

Both the R6RS and R7RS have more declarative module systems, which bring
many benefits.  Added freedom in one area often implies a loss of
freedom somewhere else.  In this case, the freedom to treat modules as
arbitrarily mutable objects implies a loss of freedom in the compiler to
make any assumptions about what's in the module or even its set of
bindings.  That freedom carries a very significant cost.

   Mark

Re: Write a macro which defines a procedure

[Mark H Weaver]

Matt Wette  writes:

> On 7/19/19 3:51 PM, zelphirkaltstahl wrote:
>> (module-define! (current-module)
>>  ;; `route` should be `/container/json` for example.
>>  route
>
> (quote route)

Yes, or equivalently:  'route

As an aside, is there a reason to not define it more simply as follows?

 Mark

--8<---cut here---start->8---
(define-syntax define-api-route
  (syntax-rules ()
[(define-api-route route http-method my-content-type)
 ;; `route` should be `/container/json` for example.
 (define* (route docker-socket #:key (data #f))
   (call-with-values
   (lambda ()
 ;; Here the route needs to be given as a string to `http-get`.
 (http-get (variable-name->string route)
   #:port docker-socket
   #:version '(1 . 1)
   #:keep-alive? #f
   ;; Why is my quasiquote not working as expected?
   #:headers `((host . ("localhost" . #f))
   (content-type . (,my-content-type (charset . 
"utf-8"
   #:body (scm->json-string data)
   #:decode-body? #t
   #:streaming? #f))
 (lambda (response response-text)
   (let ([resp-text-as-string (bytevector->string response-text 
"utf-8")])
 (cons response resp-text-as-string)]))
--8<---cut here---end--->8---

Re: Now crashing [was Re: guile-2.9.2 and threading

[Mark H Weaver]

Hi Linas,

> Investigating the crash with good-old printf's in libguile/vm.c produces
> a vast ocean of prints ... that should have not been printed, and/or should
> have been actual errors, but somehow were not handled by scm_error.
> Using today's git pull of master, here's the diff containing a printf:
>
> --- a/libguile/vm.c
> +++ b/libguile/vm.c
> @@ -1514,12 +1514,23 @@ thread->guard); fflush(stdout); assert (0); }
>
>proc = SCM_SMOB_DESCRIPTOR (proc).apply_trampoline;
>SCM_FRAME_LOCAL (vp->fp, 0) = proc;
>return SCM_PROGRAM_CODE (proc);
>  }
>
> +printf("duuude wrong type to apply!\n"
> +"proc=%lx\n"
> +"ip=%p\n"
> +"sp=%p\n"
> +"fp=%p\n"
> +"sp_min=%p\n"
> +"stack_lim=%p\n",
> +SCM_FRAME_SLOT(vp->fp, 0)->as_u64,
> +vp->ip, vp->sp, vp->fp, vp->sp_min_since_gc, vp->stack_limit);
> +fflush(stdout);
> +
>vp->ip = SCM_FRAME_VIRTUAL_RETURN_ADDRESS (vp->fp);
>
>scm_error (scm_arg_type_key, NULL, "Wrong type to apply: ~S",
>   scm_list_1 (proc), scm_list_1 (proc));
>  }
>
> As you can see, shortly after my printf, there should have been an
> error report.

Not necessarily.  Note that what 'scm_error' actually does is to raise
an exception.  What happens next depends on what exception handlers are
installed at the time of the error.

> There is no error report... until 5-10 minutes later, when the error
> report itself causes a crash.  Before then, I get an endless
> high-speed spew of prints:

It looks like another error is happening within the exception handler.

   Mark

PS: It would be good to pick either 'guile-devel' or 'guile-user' for
continuation of this thread.  I don't see a reason why it should be
sent to both lists.

Re: Threading / Pipe Macro

[Mark H Weaver]

Hello again,

Chris Vine  writes:
> My version was hygienic when used in functions, but not when used as a
> macro in a macro.

Not to belabor the point, but I wanted to mention that the unhygienic
'->' macro may fail when used in procedures, even when '->' is never
used in the definition of another macro.  Here's an example, where 'cut'
and '->' are used in combination:

--8<---cut here---start->8---
scheme@(guile-user)> ,use (srfi srfi-26)
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
(syntax-case x ()
  [(k exp0 . exps)
   (let* ([reversed (reverse (cons (syntax->datum #'exp0)
   (syntax->datum #'exps)))]
  [out (let loop ([first (car reversed)]
  [rest (cdr reversed)])
 (if (null? rest)
 first
 (let ([func (car first)]
   [args (cdr first)])
   (append `(,func ,@args)
   (list (loop (car rest) (cdr rest)))])
 (datum->syntax #'k out))])))
scheme@(guile-user)> (define foo (cut -> <> (format #t "~A\n")))
;;; :17:12: warning: possibly unbound variable `t-15fc270a-2d'
scheme@(guile-user)> (foo 4)
:17:34: In procedure foo:
In procedure module-lookup: Unbound variable: t-15fc270a-2d

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> ,expand (cut -> <> (format #t "~A\n"))
$1 = (lambda (t-15fc270a-34-1)
  (format #t "~A\n" t-15fc270a-34))
scheme@(guile-user)> (define-syntax ->
   (syntax-rules ()
 ((-> exp)
  exp)
 ((-> exp ... (op args ...))
  (op args ... (-> exp ...)
scheme@(guile-user)> (define foo (cut -> <> (format #t "~A\n")))
scheme@(guile-user)> (foo 4)
4
$2 = #t
scheme@(guile-user)> ,expand (cut -> <> (format #t "~A\n"))
$3 = (lambda (t-15fc270a-59)
  (format #t "~A\n" t-15fc270a-59))
scheme@(guile-user)> 
--8<---cut here---end--->8---

So, more generally, unhygienic macros may cause problems when they are
used in combination with other macros.  Since macros are so ubiquitous
in Scheme, attempting to avoid such combinations is likely to be
brittle.

  Best,
   Mark

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Chris Vine  writes:
> My version was hygienic when used in functions, but not when used as a
> macro in a macro.

The word "hygienic" should not be used to describe your original macro.
It would be more accurate to say that although your original version was
unhygienic, in practice it would work in many common cases.  That is
generally the case for unhygienic macros.

 Best,
  Mark

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Here's a complete, unedited transcript with Guile 2.2.6:

--8<---cut here---start->8---
mhw@jojen ~$ guile
GNU Guile 2.2.6
Copyright (C) 1995-2019 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
(syntax-case x ()
  [(k exp0 . exps)
   (let* ([reversed (reverse (cons (syntax->datum #'exp0)
   (syntax->datum #'exps)))]
  [out (let loop ([first (car reversed)]
  [rest (cdr reversed)])
 (if (null? rest)
 first
 (let ([func (car first)]
   [args (cdr first)])
   (append `(,func ,@args)
   (list (loop (car rest) (cdr rest)))])
 (datum->syntax #'k out))])))
scheme@(guile-user)> (define t 'global-t)
scheme@(guile-user)> (define-syntax-rule (foo x)
   (-> x (format #t "[t=~A] ~A\n" t)))
scheme@(guile-user)> (let ((t 'inner-t))
   (foo t))
[t=global-t] global-t
$1 = #t
scheme@(guile-user)> (define-syntax ->
   (syntax-rules ()
 ((-> exp)
  exp)
 ((-> exp ... (op args ...))
  (op args ... (-> exp ...)
scheme@(guile-user)> (let ((t 'inner-t))
   (foo t))
[t=global-t] inner-t
$2 = #t
scheme@(guile-user)>
--8<---cut here---end--->8---

Chris Vine  writes:
> How strange.  Both your and my macro gives 'global-t' when I test them,

Can you show me a complete, unedited transcript that demonstrates what
you're seeing?

> which is the result I would expect.  (Maybe I am missing something here,
> but a result of 'inner-t' would seem to me to imply unhygiene.)

(foo EXPR) is supposed to print "[t=global-t] VAL", where VAL is the
result of evaluating EXPR.  With this in mind,

(let ((t 'inner-t))
  (foo t))

The argument to 'foo' here should refer to the lexical binding of 't',
i.e. the variable with value 'inner-t'.  I'm curious what would make you
expect otherwise.

On the other hand, the reference to 't' in the template of the 'foo'
macro should refer to the toplevel variable 't', because the template
does not appear within the 'let'.

This is a good example of why syntax objects are needed, to distinguish
between these two references to distinct variables named 't'.  When you
convert the references to datums, the distinctions are lost.

 Regards,
   Mark

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Chris Vine  writes:

> I have a pipeline macro which sort-of mimics ML's |> pipeline operator
> which I use a lot:
>
> (define-syntax ->
>   (lambda (x)
> (syntax-case x ()
>   [(k exp0 . exps)
>(let* ([reversed (reverse (cons (syntax->datum #'exp0)
>(syntax->datum #'exps)))]
>   [out (let loop ([first (car reversed)]
>   [rest (cdr reversed)])
>  (if (null? rest)
>  first
>  (let ([func (car first)]
>[args (cdr first)])
>(append `(,func ,@args)
>(list (loop (car rest) (cdrrest)))])
>  (datum->syntax #'k out))])))
>
> Because all the macro does is to rearrange input forms, this is hygienic
> without the need to manipulate syntax objects - you can convert to a datum,
> rearrange and then convert back to a syntax object again, as above.

This macro is *not* hygienic.  The calls to 'syntax->datum' strip all of
the context information from the syntax objects, and then build a new
expression using raw S-expressions.  The result is essentially the same
as if you used 'define-macro'.  This results various problems.

For example:

--8<---cut here---start->8---
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
(syntax-case x ()
  [(k exp0 . exps)
   (let* ([reversed (reverse (cons (syntax->datum #'exp0)
   (syntax->datum #'exps)))]
  [out (let loop ([first (car reversed)]
  [rest (cdr reversed)])
 (if (null? rest)
 first
 (let ([func (car first)]
   [args (cdr first)])
   (append `(,func ,@args)
   (list (loop (car rest) (cdr rest)))])
 (datum->syntax #'k out))])))
scheme@(guile-user)> (define t 'global-t)
scheme@(guile-user)> (define-syntax-rule (foo x)
   (-> x (format #t "[t=~A] ~A\n" t)))
scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
[t=global-t] global-t
$1 = #t
scheme@(guile-user)> 
--8<---cut here---end--->8---

I recommend reformulating the -> macro using 'syntax-rules' as follows:

--8<---cut here---start->8---
scheme@(guile-user)> (define-syntax ->
   (syntax-rules ()
 ((-> exp)
  exp)
 ((-> exp ... (op args ...))
  (op args ... (-> exp ...)
scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
[t=global-t] inner-t
$8 = #t
scheme@(guile-user)> 
--8<---cut here---end--->8---

This macro is hygienic, and also easier to comprehend (IMO).  Of course,
it could also be implemented using syntax-case.  The key is to always
work with the syntax objects.

Whenever you use 'syntax->datum' on expressions that are not purely
literals, you will be sacrificing hygiene.

  Regards,
Mark

Re: Self-evaluating function and closure

[Mark H Weaver]

Hi Vladimir,

Vladimir Zhbanov  writes:
> Great, I've adapted your example to our code and it works
> nice. Thank you very much!

You're welcome, and I'm glad to hear it :)

 Best,
  Mark

Re: [ANN] guile-gi 0.0.2 released

[Mark H Weaver]

Mike Gran  writes:

> In short, this library hopes to make GTK3 and WebKit2 available to
> Guile.
>
> This particular library is one of several different attempts and
> solving the interface between GTK+3 and Guile.  Its differentiating
> (mis)feature is that its bindings are created dynamically at runtime
> by parsing typelib files.

This is very important work.  Thank you!

  Mark

Re: Srfi-159/166 - monadic formatting for guile

[Mark H Weaver]

Hi John,

John Cowan  writes:

> On Sun, Jun 16, 2019 at 2:47 AM Mark H Weaver  wrote:
>
>
>> How do you implement 'written-shared', 'pretty-shared', and
>> 'trimmed/lazy'?  In particular, how do you avoid non-termination when
>> asked to print cyclic data, when the cycle includes a non-standard data
>> type printed using a custom printer?  For example, what if you print a
>> list that includes an object whose custom printer will print output that
>> includes the same list that you started with?
>>
>
> Of course it can't cope with that.  But consider an object whose custom
> printer outputs its instance variables where one of those instance variables
> has been assigned to the object itself.  No procedure other than the object
> printer itself is going to be able to break *that* loop.

The built-in 'write' and 'display' procedures can break the loop, which
is the approach that I chose when I last looked at this.

 Regards,
   Mark

Re: Self-evaluating function and closure

[Mark H Weaver]

Hello again,

Vladimir Zhbanov  writes:

> scheme@(guile-user)> (define (function-generator)
>(let ((func #f)) 
>  (lambda () (set! func (let a () a)) func)))

[...]

> - Is there a way to work around this (either using the above 'let'
>   construct or anything else)?

Ideally, the code would be reworked to not expect equivalent procedures
to be distinguishable.  However, I should probably offer a hacky but
expedient workaround.  Here's one way to make otherwise equivalent
procedures distinguishable:

Allocate a fresh tag using (list #f), and arrange for the procedure to
return that tag if it's called with a special input that's outside of
the normal domain.  Note that for Scheme procedures, the "input" is in
general a list of arguments of arbitrary length.  You could use
'case-lambda', which creates procedures that evaluate different body
expressions depending on how many arguments are passed to it.  Just add
a case for an arity that you will never use, which returns the unique
tag.

In the example you gave, (let a () a) is equivalent to:

  ((letrec ((a (lambda () a)))
 a))

The procedure returned by (let a () a) expects 0 arguments.  It will
raise an error otherwise.  We can repurpose the previously erroneous
arity-1 case to return the unique tag, as follows:

  (let ((unique-tag (list #f)))
((letrec ((a (case-lambda
   (() a)
   ((x) unique-tag
   a)))

Every time the above expression is evaluated, it will necessarily return
a unique procedure, which, if passed 0 arguments, behaves the same as
the procedure returned by (let a () a).

   Mark

Re: Self-evaluating function and closure

[Mark H Weaver]

Hi Thomas,

Thomas Morley  writes:

> always interested in guile developments with regard to lilypond I
> noticed some inconsistence with my local lilypond-using-guile-2.9.2
> installation.
> I think I could break it down to pure guile (no lilypond)
>
> (1) The already stated behaviour:
> ~$ guile
> GNU Guile 2.9.2
> Copyright (C) 1995-2019 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> (define (function-generator)
>   (let ((func #f))
> (lambda () (set! func (let a () a)) func)))
> scheme@(guile-user)> (define x (function-generator))
> scheme@(guile-user)> (define y (function-generator))
> scheme@(guile-user)> (write (version))
> "2.9.2"
> scheme@(guile-user)> (format #t "\nTEST: ~a" (eq? (x) (y)))
>
> TEST: #t
> $1 = #t
>
> (2) Doing it in a .scm-file (attached)
> ~$ guile --no-auto-compile eq-tst.scm
> "2.9.2"
> TEST: #f
>
> Did I something wrong or is it a bug?

Neither.  This is a case of unspecified behavior, and the behavior of
our compiler differs from that of our interpreter.  You will notice
similar discrepancies when comparing two literal lists or strings, where
our compiler will aggressively unify literals within a compilation unit,
and our interpreter does not:

--8<---cut here---start->8---
GNU Guile 2.2.4
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (eqv? "hello" "hello")
$1 = #t
scheme@(guile-user)> ,o interp #t
scheme@(guile-user)> (eqv? "hello" "hello")
$2 = #f
scheme@(guile-user)>
--8<---cut here---end--->8---

 Regards,
   Mark

Re: Self-evaluating function and closure

[Mark H Weaver]

Hi Vladimir,

Vladimir Zhbanov  writes:

> On Sat, Jun 15, 2019 at 08:36:34PM -0400, Mark H Weaver wrote:
>> Vladimir Zhbanov  writes:
>> 
>> > I have tried almost a textbook example with Guile 2.2.4:
>> >
>> > scheme@(guile-user)> (define (function-generator)
>> >(let ((func #f)) 
>> >  (lambda () (set! func (let a () a)) func)))
>> >
>> > scheme@(guile-user)> (define x (function-generator))
>> > scheme@(guile-user)> (define y (function-generator))
>> > scheme@(guile-user)> x
>> > $20 = #:562:25 ()>
>> > scheme@(guile-user)> y
>> > $21 = #:562:25 ()>
>> > scheme@(guile-user)> (x)
>> > $22 = #
>> > scheme@(guile-user)> (y)
>> > $23 = #
>> > scheme@(guile-user)> (eq? (x) (y))
>> > $24 = #t
>> >
>> > The result is unexpected for me, I expected a new self-evaluating
>> > procedure every time I run the function-generator procedure (and
>> > it works differently with Guile 2.0, IIUC, cannot check just now).
>> 
>> Why would you expect 'eq?' to return #false here?  Do you know of any
>> text in Guile's manual, or in any of the relevant Scheme standards, that
>> would lead you to expect this?
>> 
>> Since (let a () a) contains no free variable references, every procedure
>> returned by (let a () a) is operationally equivalent to every other
>> procedure returned by it.  Therefore, as I understand it, a conforming
>> Scheme implementation is permitted (but not required) to return the same
>> procedure object every time.
>
> If the procedure is defined inside a closure, should it be
> available at toplevel (especially with the same name it was
> defined inside a function)?

A variable bound within a local lexical environment certainly should not
be visible at toplevel.  In particular, the transcript that you provided
above should *not* result in 'a' being bound in the toplevel
environment.

When I type those precise commands into Guile 2.2.4 on my system, 'a' is
*not* bound at toplevel.  Here's what I see:

  mhw@jojen ~$ guile
  GNU Guile 2.2.4
  Copyright (C) 1995-2017 Free Software Foundation, Inc.
  
  Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
  This program is free software, and you are welcome to redistribute it
  under certain conditions; type `,show c' for details.
  
  Enter `,help' for help.
  scheme@(guile-user)> (define (function-generator)
  (let ((func #f)) 
(lambda () (set! func (let a () a)) func)))
  scheme@(guile-user)> (define x (function-generator))
  scheme@(guile-user)> (define y (function-generator))
  scheme@(guile-user)> x
  $1 = #:3:26 ()>
  scheme@(guile-user)> y
  $2 = #:3:26 ()>
  scheme@(guile-user)> (x)
  $3 = #
  scheme@(guile-user)> (y)
  $4 = #
  scheme@(guile-user)> (eq? (x) (y))
  $5 = #t
  scheme@(guile-user)> a
  ;;; : warning: possibly unbound variable `a'
  ERROR: In procedure module-lookup: Unbound variable: a
  
  Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
  scheme@(guile-user) [1]> 

>> If this were the case, it would certainly be a bug.  However, I cannot
>> reproduce it, and I strongly suspect that you had defined 'a' as a
>> toplevel variable earlier in your Guile session and forgot about it.
>
> No, I'haven't defined anything before.
>
> OK, let's start with a fresh session and use another name:
>
> 
>
> GNU Guile 2.2.4
> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> (define (function-generator)
>(let ((func #f))
>  (lambda () (set! func (let b () b)) func)))
> scheme@(guile-user)> (define x (function-generator))
> scheme@(guile-user)> (define y (function-generator))
> scheme@(guile-user)> x
> $4 = #:108:25 ()>
> scheme@(guile-user)> y
> $5 = #:108:25 ()>
> scheme@(guile-user)> (x)
> $6 = #
> scheme@(guile-user)> (y)
> $7 = #
> scheme@(guile-user)> (eq? (x) (y))
> $8 = #t

This new transcript does not include an evaluation of 'b' at toplevel.
Can you show me a complete transcript that demonstrates this behavior?

> scheme@(guile-user)> ,compile (define (functio

Re: Self-evaluating function and closure

[Mark H Weaver]

Hello again Vladimir,

Vladimir Zhbanov  writes:
> - Is there a way to work around this (either using the above 'let'
>   construct or anything else)?

I'm not quite sure how to answer this question because I don't know what
your requirements are.

If you need to generate unique tags, any mutable object will do, e.g. a
vector, list or string with at least one element that is not a literal.
For example, (list #f), (vector #f), and (string #\a) must allocate a
fresh object every time, but (list), (vector), (string), '(#f), #(#f),
and "foo" may return the same object every time.

If you need to generate a unique _procedure_, there's only one
future-proof way to do it: the new procedure must behave differently
than every other procedure, for some input.

It's a mistake to expect procedures with equivalent behavior to be
distinguishable in Scheme.  If you feel that you need this, I would like
to understand why.

Note that 'eq?' is the same as 'eqv?' when applied to procedures (and
most other types), and that Scheme 'eqv?' historically implements an
approximation of "operational equivalence".  That language was explicit
in the R3RS, which defined 'eqv?' this way:

  The eqv? procedure implements an approximation to the relation of
  operational equivalence.  It returns #t if it can prove that obj1 and
  obj2 are operationally equivalent.  If it can't, it always errs on the
  conservative side and returns #f.
  
with "operational equivalence" defined as follows:
  
  Two objects are operationally equivalent if and only if there is no
  way that they can be distinguished, using Scheme primitives other than
  eqv? or eq? or those like memq and assv whose meaning is defined
  explicitly in terms of eqv? or eq?.  It is guaranteed that objects
  maintain their operational identity despite being named by variables
  or fetched from or stored into data structures.

More recent Scheme standards have dropped this language, because the
Scheme authors were not entirely satisfied with this definition and were
unable to formulate a better one, but nonetheless "operational
equivalence" remains the closest thing I've seen to a unifying principle
of the meaning of Scheme 'eqv?'.

  Regards,
Mark

Re: Srfi-159/166 - monadic formatting for guile

[Mark H Weaver]

Hi Linus,

Linus Björnstam  writes:

> I am not sure I understand what you mean, but i suspect it is already
> solved, since for simple cases show will use display/write. So unless
> you try to do advanced formatting of circular lists it will do
> whatever display or write does. The same thing goes for custom
> printers.

How do you implement 'written-shared', 'pretty-shared', and
'trimmed/lazy'?  In particular, how do you avoid non-termination when
asked to print cyclic data, when the cycle includes a non-standard data
type printed using a custom printer?  For example, what if you print a
list that includes an object whose custom printer will print output that
includes the same list that you started with?

 Regards,
   Mark

Re: Self-evaluating function and closure

[Mark H Weaver]

Hi John,

John Cowan  writes:

> On Sat, Jun 15, 2019 at 4:31 AM Vladimir Zhbanov  wrote:
>
> The result is unexpected for me, I expected a new self-evaluating
>> procedure every time I run the function-generator procedure (and
>> it works differently with Guile 2.0, IIUC, cannot check just now).
>>
>
> I tested this on Chicken, Biwa, and Chbi, and all agree on producing new
> closures that are not eq?.  So I'd call it a bug in either Guile or
> the Scheme standard, but probably in Guile.

I don't understand your reasoning.  If you think it's a bug in Guile,
can you cite a specific requirement in a relevant Scheme standard that
we are violating here?

I also don't know why you would call it a bug in the Scheme standard.
Can you explain why you think this?

  Mark

Re: Self-evaluating function and closure

[Mark H Weaver]

Hi Vladimir,

Vladimir Zhbanov  writes:

> Greetings,
>
> I have tried almost a textbook example with Guile 2.2.4:
>
> scheme@(guile-user)> (define (function-generator)
>(let ((func #f)) 
>  (lambda () (set! func (let a () a)) func)))
>
> scheme@(guile-user)> (define x (function-generator))
> scheme@(guile-user)> (define y (function-generator))
> scheme@(guile-user)> x
> $20 = #:562:25 ()>
> scheme@(guile-user)> y
> $21 = #:562:25 ()>
> scheme@(guile-user)> (x)
> $22 = #
> scheme@(guile-user)> (y)
> $23 = #
> scheme@(guile-user)> (eq? (x) (y))
> $24 = #t
>
> The result is unexpected for me, I expected a new self-evaluating
> procedure every time I run the function-generator procedure (and
> it works differently with Guile 2.0, IIUC, cannot check just now).

Why would you expect 'eq?' to return #false here?  Do you know of any
text in Guile's manual, or in any of the relevant Scheme standards, that
would lead you to expect this?

Since (let a () a) contains no free variable references, every procedure
returned by (let a () a) is operationally equivalent to every other
procedure returned by it.  Therefore, as I understand it, a conforming
Scheme implementation is permitted (but not required) to return the same
procedure object every time.

I just refreshed my memory of the requirements of the R5RS, R6RS, and
R7RS on 'eq?' when applied to procedures.  Conforming implementations
are required to return #true if the procedures have the same "location
tags", and are required to return #false if the procedures would behave
differently (return different value(s) or have different side effects)
for some arguments.

> AFAICS, Guile creates a toplevel procedure "a" while it should not do
>so.
>
> scheme@(guile-user)> a
> $25 = #:422:25 ()>

If this were the case, it would certainly be a bug.  However, I cannot
reproduce it, and I strongly suspect that you had defined 'a' as a
toplevel variable earlier in your Guile session and forgot about it.

   Mark

Re: Srfi-159/166 - monadic formatting for guile

[Mark H Weaver]

Hi Linus,

Linus Björnstam  writes:

> I just ported the reference implementation of SRFI-159/166 (166 being
> the successor of 159) from the reference implementation in chibi
> scheme. The srfi document is here:
> https://srfi.schemers.org/srfi-166/srfi-166.html
>
> SRFI-166 is like (ice-9 format) on steroids, drenched in molasses. It
> can do a lot more than (ice-9 format), but about 3x slower. I haven't
> started trying to make it fast just yet, but there are some low
> hanging fruit (like speeding up simple cases). I will wait with
> optimization until the srfi is finalized (and also maybe hear with
> Alex what he has in mind before trying to half-ass it myself).
>
> Anyway, the repo is here: https://bitbucket.org/bjoli/guile-srfi-159
> . I haven't ported the tests, but playing with it in guile and chibi
> scheme produces the same result. Tests are up, but I'm travelling
> without computer the next week, so there is no time.

I would be in favor of integrating this into Guile at some point,
although first I would like to make a careful review of the
implementation to make sure that it integrates well with Guile's related
mechanisms, e.g. custom record printers, printer options, and print
states, and that it supports all Guile data types.  In particular, I
remember finding it difficult to properly implement cycle detection
while supporting Guile's existing mechanisms.  I'm not sure it can be
done without modifying Guile's built-in printer.

I'll also want to take a close look at the numeric formatter, to make
sure that it achieves the same properties as our built-in printer,
e.g. that it prints inexacts with the minimal number of digits required
to ensure that the same number will be read back in (unless specifically
asked to limit the precision).

Anyway, thanks very much for sharing this!

Regards,
  Mark

Re: guile-2.9.2 and threading

[Mark H Weaver]

Mark H Weaver  writes:

>> Two are stuck here:
>>
>> #0  __lll_lock_wait () at 
>> ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
>> #1  0x7f343ca69bb5 in __GI___pthread_mutex_lock (
>> mutex=mutex@entry=0x7f343d4f0f40 )
>> at ../nptl/pthread_mutex_lock.c:80
>> #2  0x7f343d213e20 in scm_gc_register_allocation (size=size@entry=16)
>> at ../../libguile/gc.c:591
>
> This is the global GC allocation lock, which might be an issue if your
> threads are performing a lot of heap allocation.

Sorry, I spoke too quickly here, although what I wrote is not far from
the truth.  'bytes_until_gc_lock' is used by Guile to protect its global
counter of how much plain 'malloc' memory has been allocated since the
last garbage collection.  Once it reaches a certain threshold, a garbage
collection is forced.

Why do we have this?  Sometimes the GC heap contains a large number of
small collectible objects which reference much larger objects in plain
'malloc' memory.  For example, this can happen if you're creating a lot
of bignums, which are small GC objects pointing to digit data stored in
'malloc' memory.  The garbage collector doesn't trigger because there's
plenty of free memory in the GC heap, and it doesn't know that those
small objects are keeping alive much larger objects elsewhere.

However, if this particular lock is a pain point for you, there are
things we could do to improve this.  One idea that comes to mind is to
keep smaller per-thread counters, which are only added to the global
counter after they reach a certain threshold value.  In this case, we
don't need a precise global count.

Mark

Re: guile-2.9.2 and threading

[Mark H Weaver]

Hi Linas,

Linas Vepstas  writes:

> I'm trying to understand how scm_jit_enter_mcode leads to
> scm_timed_lock_mutex ...

This simply means that 'lock-mutex' was called from Scheme, and
specifically from Scheme code that has been compiled to machine code by
our JIT compiler.

> I want to know who is attempting to lock, and why ... and how to work
> around this...

You'll need to look at the stack frames on the Scheme stack.  It can be
done from GDB if necessary, but it might be sufficient to use Guile's
debugger.

My first thought was to suggest ",break lock-mutex", but that doesn't
work, presumably because it's a C primitive (although we should fix
that), but I was able to patch in a pure Scheme wrapper for it, which
then allows us to set a Scheme breakpoint:

--8<---cut here---start->8---
mhw@jojen ~/guile-master$ meta/guile
GNU Guile 2.9.2.14-1fb399-dirty
Copyright (C) 1995-2019 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use (ice-9 threads)
scheme@(guile-user)> (define (test n) (par-for-each + (iota n) (iota n)))
scheme@(guile-user)> (define core-lock-mutex lock-mutex)
scheme@(guile-user)> (set! lock-mutex (lambda* (mutex #:optional timeout)
(core-lock-mutex mutex timeout)))
scheme@(guile-user)> ,break lock-mutex
Trap 0: Breakpoint at #.
scheme@(guile-user)> (test 1)
Trap 0: Breakpoint at #
Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In current input:
  2:0  2 (test 1)
In ice-9/threads.scm:
389:6  1 (_ #)
In current input:
 4:17  0 (lock-mutex #)
scheme@(guile-user) [1]> 
--8<---cut here---end--->8---

Hopefully this should be enough to figure out what's calling
'lock-mutex' in your case.

One possibility is that your code is accessing the module table too
often.  Does your code frequently call procedures like 'resolve-module',
'module-variable', 'module-ref', or any of the other procedures
documented in section 6.20.8 (Module System Reflection) of the Guile
manual?

> I'm using (ice-9 threads)   I see this:
>
> (define (n-par-for-each n proc . arglists)
>   (let ((m (make-mutex))
>(threads '()))
> (do ((i 0 (+ 1 i)))
>((= i n)
> (for-each join-thread threads))
>   (set! threads
>(cons (begin-thread
>  (let loop ()
>(lock-mutex m)
>(if (null? (car arglists))
>   (unlock-mutex m) 
>   (let ((args (map car arglists)))
> (set! arglists (map cdr arglists))
> (unlock-mutex m)
> (apply proc args)
> (loop)
> threads)
>
> Oh, I says to myself: bad bad mutex. Let me write a lock-less loop: it
> chops the list into n pieces, each in its own thread. (a bit sloppy,
> but adequate):
>
> (define (my-for-each n proc args)
>(define len (length args))
>(define quo (euclidean-quotient len n))
>(define rem (euclidean-remainder len n))
>(define threads '())
>(do ((i 0 (+ 1 i)))
>   ((= i n) (for-each join-thread threads))
>  (set! threads
> (cons
>(begin-thread
>   (for-each proc (take (drop args (* i quo)) quo)))
>threads)))
>(for-each proc (drop args (* n quo)))
> )
>
> Let me go hog-wild: (my-for-each 12 proc list)   (I have a cpu with that many 
> cores) So... what happens? A little better .. not much.  This time, gdb shows 
> that there are four threads in my app.
> Two are stuck here:
>
> #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
> #1  0x7f343ca69bb5 in __GI___pthread_mutex_lock (
> mutex=mutex@entry=0x7f343d4f0f40 )
> at ../nptl/pthread_mutex_lock.c:80
> #2  0x7f343d213e20 in scm_gc_register_allocation (size=size@entry=16)
> at ../../libguile/gc.c:591

This is the global GC allocation lock, which might be an issue if your
threads are performing a lot of heap allocation.

  Mark

Re: TeXmacs on Guile 2.2.4

[Mark H Weaver]

Massimiliano Gubinelli  writes:

> While trying some alternative solutions I noticed the following: the
> function scm_to_latin1_stringn calls out to scm_to_stringn with an
> argument NULL for the encoding:
>
> result = scm_to_stringn (str, lenp, NULL,
>  SCM_FAILED_CONVERSION_ERROR);
>
> this causes an error in scm_to_stringn since it contains the check:
>
>(c_strcasecmp (encoding, "UTF-8") == 0)
>
> and c_strcasecmp does not check for null pointers. Is this a bug? 

Yes, it is.  I'll fix it soon, thanks.

   Mark

Re: string-ports issue on Windows

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> Addendum - wish to confirm if guile bug (guile-2.2 on Windows):
> - set locale to non-Anglo so that (setlocale LC_ALL) returns
> "French_France.1252"
> - call (strftime "%B" 400) - that's 4x10^6 -- this should return
> "février 1970"
>
> but the following error arises:
> Throw to key `decoding-error' with args `("scm_from_utf8_stringn" "input
> locale conversion error" 0 #vu8(102 233 118 114 105 101 114 32 49 57 55
> 48))'.
>
> Is this a bug?

Yes, it is.  I've filed a bug report and provided more information here:

  https://bugs.gnu.org/35920

Thanks for letting us know about this.

 Regards,
   Mark

Re: Are there any way to mimic Bash's exit traps in Guile?

[Mark H Weaver]

Hello again,

Mark H Weaver  writes:

> Михаил Бахтерев  writes:
>
>> I use in one of my bash-scripts the following construction
>>
>>   trap "kill -s TERM 0" EXIT

In my earlier reply, I somehow failed to notice that you're killing the
entire process group by passing 0 to kill.  This simplifies things quite
a bit.

>> to terminate all spawned processes when master process exits or is
>> aborted by any reason. Is equivalent technique possible in Guile?
>
> Guile doesn't provide anything like this out-of-the-box, but you should
> be able to accomplish something similar in Guile by installing signal
> handlers for all terminating signals, and additionally wrapping the main
> program within (catch #t ...).  This should catch most exit modes,
> including calls within Scheme to 'exit' or 'quit'.

See below for an example Guile script that does something reasonably
close to what Bash does when you install an EXIT trap in a
non-interactive shell.

Note that in my example 'exit-handler' below, I arrange for SIGTERM to
be ignored in the current process before killing the process group, to
avoid killing the master process with SIGTERM.

In 'term-signal-handler', after running 'exit-handler', I restore the
default handler for the signal that we originally caught, and then send
that signal to the master process.  These last steps are exactly what
Bash does at the end of 'termsig_handler' in sig.c.

This way, if we receive a terminating signal, we report back to our
parent process (via waitpid) the original signal number that killed us.

  Mark


--8<---cut here---start->8---
(use-modules (srfi srfi-1)   ; list library
 (ice-9 match))

(define terminating-signal-names
  '(SIGHUP SIGINT SIGILL SIGTRAP SIGIOT SIGDANGER
SIGEMT SIGFPE SIGBUS SIGSEGV SIGSYS SIGPIPE
SIGALRM SIGTERM SIGXCPU SIGXFSZ SIGVTALRM
SIGLOST SIGUSR1 SIGUSR2))

(define terminating-signals
  (let ((mod (resolve-module '(guile
(filter-map (lambda (sig-name)
  (and=> (module-variable mod sig-name)
 variable-ref))
terminating-signal-names)))

(define (exit-handler)
  (sigaction SIGTERM SIG_IGN)
  (kill 0 SIGTERM)
  (display "exit-handler")
  (newline)
  (force-output))

(define (term-signal-handler sig)
  (restore-signals)
  (exit-handler)
  (sigaction sig SIG_DFL)
  (kill (getpid) sig))

(define (install-signal-handlers)
  (for-each (lambda (sig)
  (match (sigaction sig)
((handler . flags)
 (when (eqv? handler SIG_DFL)
   (sigaction sig term-signal-handler)
terminating-signals))

(define (call-with-exit-handler thunk)
  (install-signal-handlers)
  (catch #t
thunk
(lambda (key . args)
  (exit-handler)
  (apply throw key args)))
  (exit-handler))

(call-with-exit-handler
 (lambda ()
   (let loop ((i 0))
 (display i)
 (newline)
 (loop (+ i 1)
--8<---cut here---end--->8---

Re: Are there any way to mimic Bash's exit traps in Guile?

[Mark H Weaver]

Hi,

Михаил Бахтерев  writes:

> I use in one of my bash-scripts the following construction
>
>   trap "kill -s TERM 0" EXIT
>
> to terminate all spawned processes when master process exits or is
> aborted by any reason. Is equivalent technique possible in Guile?

Guile doesn't provide anything like this out-of-the-box, but you should
be able to accomplish something similar in Guile by installing signal
handlers for all terminating signals, and additionally wrapping the main
program within (catch #t ...).  This should catch most exit modes,
including calls within Scheme to 'exit' or 'quit'.

However, it should be noted that Guile provides many ways to exit a
process immediately without unwinding the stack, notably
'primitive-exit' and 'primitive-_exit'.  The signal handlers that you
install for terminating signals should probably call 'primitive-exit'
after killing the subprocesses.

Also note there's a synchronization detail worth paying attention to
here.  The signal handler must be able to safely access the current list
of spawned processes.

In a single threaded program, or more generally, if only one thread
needs to access the list of spawned processes, then it should suffice to
arrange for the Guile terminating signal handlers to run within the same
thread (see the last optional argument to 'sigaction' in Guile).  In
this case, you shouldn't need locks to protect the list of spawned
processes.  You can use 'call-with-blocked-asyncs' to prevent Guile
signal handlers from being run during critical sections when the list is
being modified.

If multiple threads must access the list of spawned processes, then you
will also need to protect it with a mutex.  However, you must guard
against the case that the Guile signal handler interrupts an existing
thread that's already holding the mutex and is within a critical
section.  This case can be avoided by locking the mutex only within
'call-with-blocked-asyncs'.

  Mark

Re: exporting GOOPS generic functions, was: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hello again,

> Brandon Invergo  writes:
>
>> I can try something like this:
>>
>> (let ((old-absolute-file-name? absolute-file-name?))
>>   (define-generic absolute-file-name?)
>>   (define-method (absolute-file-name? (f ))
>> (proper-list? (route f)))
>>   (define-method (absolute-file-name? (f ))
>> (old-absolute-file-name? f)))
>>
>> But that strangely gives me this upon compiling the module:
>>
>> While compiling expression:
>> Unbound variable: absolute-file-name?
>>
>> I'm not sure what to make of that.  A compile-time error, but why?
>
> It's because you tried to export a binding that doesn't exist at the
> top-level of your module.

I should explain more clearly what happened here.  The short answer is:
When you export a variable, it immediately shadows any imported bindings
with the same name.

The longer answer is that before a variable can be exported, we first
need a variable object to add to the public interface.  If you export a
variable before it has been defined (the usual case), Guile allocates a
fresh variable object and immediately adds it to the local module table,
although it marks the variable as "unbound", which essentially means
that the variable pretends not to exist.  However, the existence of this
"unbound" variable *does* have the effect of hiding any imports with the
same name.

   Mark

Re: exporting GOOPS generic functions, was: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hi Brandon,

Brandon Invergo  writes:

> I can try something like this:
>
> (let ((old-absolute-file-name? absolute-file-name?))
>   (define-generic absolute-file-name?)
>   (define-method (absolute-file-name? (f ))
> (proper-list? (route f)))
>   (define-method (absolute-file-name? (f ))
> (old-absolute-file-name? f)))
>
> But that strangely gives me this upon compiling the module:
>
> While compiling expression:
> Unbound variable: absolute-file-name?
>
> I'm not sure what to make of that.  A compile-time error, but why?

It's because you tried to export a binding that doesn't exist at the
top-level of your module.  That's because the definitions you gave above
are not in a top-level context.  They are local variables.

One easy solution is to use Guile's (@ module var) syntax to explicitly
reference a variable from a particular module.  For example:

(define-method (absolute-file-name? (f ))
  ((@ (guile) absolute-file-name?) f)))

Here, the reference (@ (guile) absolute-file-name?) explicitly asks for
the binding from the (guile) module.

Another option is to *rename* bindings while importing them, using
either #:prefix or #:select.  For example, see
module/system/repl/command.scm in Guile's source, which includes the
following lines in its 'define-module' form:

  #:use-module ((ice-9 pretty-print) #:select ((pretty-print . pp)))
  #:use-module ((system vm inspect) #:select ((inspect . %inspect)))

Those modules include 'pretty-print' and 'inspect' among their exported
bindings, but the bindings will be visible as 'pp' and '%inspect' within
command.scm.

When using #:select, the imports are limited to the bindings you
specifically asked for.  Sometimes it's more convenient to add a prefix
to every binding imported from a given module, like the following
example from Guile's module/ice-9/streams.scm:

(define-module (ice-9 streams)
  #:use-module ((srfi srfi-41) #:prefix srfi-41:)

Bindings can also be renamed when they are exported, and (ice-9 streams)
does this.

> ps - You say that I should use #:export and #:replace, but according to
> the manual, #:replace "[exports] all identifiers in LIST ... and mark[s]
> them as “replacing bindings”."  So, shouldn't be unnecessary to put the
> symbol in both lists?

Yes, you're right.  If a binding is listed in the #:replace list,
there's no need to add it to the #:export list.

* * *

Finally, I wanted to mention that, in my opinion, the degree to which
this library modifies existing Guile bindings is regrettable.  I'm glad
that Guile is hackable in this way, but I generally prefer to avoid
libraries that do these kinds of hacks, unless there is a compelling
reason to justify it.

In my opinion, it would be better to use plain strings as the
representation for file names, and instead provide procedures to
conveniently manipulate those strings.

Consider:

(1) Your  objects are much larger than plain strings in terms
of their memory use.

(2) Every time you pass a  object to a file system primitive,
it needs to be converted back into a string, which makes them slower to
use as well.

(3) If Guile ever extends our file system primitives in the future,
e.g. by adding additional keyword or optional arguments, your library
will likely cause problems with that.  For example, a few years ago we
added the #:guess-encoding and #:encoding keyword arguments to
'open-file' and several other similar procedures.

Also, I had a question: a big part of your library seems to be aimed at
supporting Windows drive letters and separators.  Have tested your
library on Guile running on Windows?

One thing I know is that there's no need for us to use backslash as the
file separator on Windows.  My understanding is that the POSIX
compatibility layer, used in Guile's Windows port, automatically handles
interpreting forward slashes as a directory separator.  Guile already
includes code to try to ensure that from the perspective of Guile code,
we will see and use forward slashes uniformly, regardless of platform.
For example, see 'scm_i_mirror_backslashes' in libguile/load.c and its
associated comment.

  Regards,
Mark

Re: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hi again,

I wrote:
> If you must override core procedures, then please use #:export and
> #:replace in the 'define-module' form, and simply 'define' the new
> binding in your module instead of using 'set!'.  That way, the bindings
> in (guile) will be left unchanged, and your new bindings will only be
> used in modules that import your module.

Sorry, I meant to give an example of this.  Please see srfi/srfi-45.scm
in the Guile source distribution for an example of the preferred way to
override core bindings, such that only modules that import your module
are affected.

  Thanks,
Mark

Re: [ANN] guile-file-names 0.2

[Mark H Weaver]

Hi Brandon,

Brandon Invergo  writes:

> I'm happy to announce the release of guile-file-names 0.2.
>
> The (file-names) module provides methods for manipulating file names.
> Its design distinguishes between the human-friendly string format of
> filenames ("/usr/bin/guile") and a more Scheme-friendly representation
> to take out all the little nuisances of working with file names.
>
> This release sees bugs fixed, under-the-hood improvements, and new
> features added.  See below for an excerpt from the NEWS file.

I see that you are using 'set!' to mutate several core bindings in the
(guile) module to much slower GOOPS generic functions.  For example, you
'set!' the core 'append' procedure to a GOOPS generic function that adds
support for appending together your  objects.

If you must override core procedures, then please use #:export and
#:replace in the 'define-module' form, and simply 'define' the new
binding in your module instead of using 'set!'.  That way, the bindings
in (guile) will be left unchanged, and your new bindings will only be
used in modules that import your module.

I can understand your decision to overload Guile's primitives that
accept file names.  However, I disagree with your decision to overload
'append' and 'append!'.  Those are not intended to be generic functions
in Scheme.  If they were, we wouldn't have 'string-append' and
'symbol-append'.

 Regards,
   Mark

Re: Guile assert macro

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> I was looking for an assert facility in Guile and found the following:
>
> https://www.gnu.org/software/guile/manual/html_node/rnrs-base.html#rnrs-base
>
> (Search for assert there to find it on the page.)
>
> However, while searching, I also found something that looks even better:
>
> http://okmij.org/ftp/Scheme/assert-syntax-rule.txt
>
> Is that anywhere implemented in Guile already?

No.

> (In case it is not implemented: What are the reasons for not providing
> that one? It looks a little bit more useful than the one I found.)

Well, no one has suggested it until now, and moreover I didn't even know
it existed :)  Do you know if it's seen much use?  Do any other Scheme
implementations include it?

Correct me if I'm wrong, but I get the impression that this is something
that Oleg hacked up in an hour and posted ~15 years ago, but that it's
never seen much use.  Also, I'm personally not fond of the API, which is
quite unconventional.

The thing about assertion macros is that (1) they are usually trivial,
and (2) I'm not aware of any consensus in the Scheme community on what a
non-trivial assertion API should look like.  I suspect this is why
there's no SRFI for assertion macros, despite the fact that anyone can
write a SRFI.

For those who find the R6RS 'assert' macro unsatisfactory, I'm inclined
to suggest that each project should feel free to define their own
assertion macro according to their needs and preferences.

Having said that, if you think you know a non-trivial assert API that
other Schemers would like to use, feel free to publish a library or
write a SRFI.  If it becomes popular, we could consider including it
with Guile.

  Regards,
Mark

Re: string-ports issue on Windows

[Mark H Weaver]

Hi again,

Earlier, I wrote:

> Christopher Lam  writes:
>
>> Hi Mark
>> Thank you so much for looking into this.
>> I'm reviewing the GnuCash for Windows package (v3.5 released April 2019)
>> which contains the following libraries:
>> - guile 2.0.14
>
> Ah, for some reason I thought you were using Guile 2.2.  That explains
> the problem.
>
> In Guile 2.0, string ports internally used the locale encoding by
> default, which meant that any characters not supported by the locale
> encoding would be munged.
>
> Guile 2.2 changed the behavior of string ports to always use UTF-8
> internally, which ensures that all valid Guile strings can pass through
> unmunged.
>
> So, this problem would almost certainly be fixed by updating to
> Guile 2.2.

It's probably a good idea to update to Guile 2.2 anyway, but I'd like to
also offer the following workaround, which monkey patches the string
port procedures in Guile 2.0 to behave more like Guile 2.2.

Note that it only patches the Scheme APIs for string ports, and not the
underlying C functions.  It might be that some code, possibly within
Guile itself, creates a string port using the C functions, and such
string ports may still munge characters.

Anyway, if you want to try it, arrange for GnuCash to evaluate the code
below, after initializing Guile.

  Mark


(when (string=? (effective-version) "2.0")
  ;; When using Guile 2.0.x, use monkey patching to change the
  ;; behavior of string ports to use UTF-8 as the internal encoding.
  ;; Note that this is the default behavior in Guile 2.2 or later.
  (let* ((mod (resolve-module '(guile)))
 (orig-open-input-string  (module-ref mod 'open-input-string))
 (orig-open-output-string (module-ref mod 'open-output-string))
 (orig-object->string (module-ref mod 'object->string))
 (orig-simple-format  (module-ref mod 'simple-format)))

(define (open-input-string str)
  (with-fluids ((%default-port-encoding "UTF-8"))
(orig-open-input-string str)))

(define (open-output-string)
  (with-fluids ((%default-port-encoding "UTF-8"))
(orig-open-output-string)))

(define (object->string . args)
  (with-fluids ((%default-port-encoding "UTF-8"))
(apply orig-object->string args)))

(define (simple-format . args)
  (with-fluids ((%default-port-encoding "UTF-8"))
(apply orig-simple-format args)))

(define (call-with-input-string str proc)
  (proc (open-input-string str)))

(define (call-with-output-string proc)
  (let ((port (open-output-string)))
(proc port)
(get-output-string port)))

(module-set! mod 'open-input-string   open-input-string)
(module-set! mod 'open-output-string  open-output-string)
(module-set! mod 'object->string  object->string)
(module-set! mod 'simple-format   simple-format)
(module-set! mod 'call-with-input-string  call-with-input-string)
(module-set! mod 'call-with-output-string call-with-output-string)

(when (eqv? (module-ref mod 'format) orig-simple-format)
  (module-set! mod 'format simple-format

Re: string-ports issue on Windows

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> Hi Mark
> Thank you so much for looking into this.
> I'm reviewing the GnuCash for Windows package (v3.5 released April 2019)
> which contains the following libraries:
> - guile 2.0.14

Ah, for some reason I thought you were using Guile 2.2.  That explains
the problem.

In Guile 2.0, string ports internally used the locale encoding by
default, which meant that any characters not supported by the locale
encoding would be munged.

Guile 2.2 changed the behavior of string ports to always use UTF-8
internally, which ensures that all valid Guile strings can pass through
unmunged.

So, this problem would almost certainly be fixed by updating to
Guile 2.2.

   Mark

Re: A problem with statprof

[Mark H Weaver]

Hi,

Tommi Höynälänmaa  writes:

> It seems that statprof gives some second counts in some other units
> than seconds. I'm using guile 2.9.1. Consider the following program:
>
> ---cut here---
> (import (statprof))
>
> (define (factorial n)
>   (if (= n 0) 1 (* n (factorial (- n 1)
>
> (define (main args)
>   (statprof-reset 0 5 #t)
>   (statprof-start)
>   (display "Hello\n")
>   (display (factorial 10))
>   (newline)
>   (statprof-stop)
>   (statprof-display))
> ---cut here---
>
> This program gives the following output:
> ---cut here---
> ...
> % cumulative   self
> time   seconds    seconds   calls   procedure
>  57.14  1.83  1.83    anon #x55f314c21e40
>  41.07 05.81  1.32
> /home/tohoyn/tyo/omat/ohj/scheme/muut/statprof-test2.scm:4:0:factorial
>   1.79  0.06  0.06    anon #x55f314c27a38
>   0.00  3.21  0.00 ice-9/eval.scm:618:6
>   0.00  3.21  0.00 ice-9/boot-9.scm:701:0:call-with-prompt
>   0.00  3.21  0.00
> /home/tohoyn/tyo/omat/ohj/scheme/muut/statprof-test2.scm:7:0:main
>   0.00  3.21  0.00    anon #x55f314c29930
>   0.00  1.83  0.00    anon #x55f314c1f7e0
> ---
> Sample count: 56
> Total time: 3.206961337 seconds (1.765094628 seconds in GC)
> ---cut here---
>
> Obviously the cumulative time for procedure factorial is not given in
> seconds. OTOH the time for procedure main is correct. What is wrong
> here?

I'm not sure there's anything wrong here, but I agree that it's
counterintuitive.  Here's a paragraph from the "Implementation notes"
for statprof in the Guile manual:

 Later, when profiling stops, that log buffer is analyzed to produce
  the “self seconds” and “cumulative seconds” statistics.  A procedure
  at the top of the stack counts toward “self” samples, and everything
  on the stack counts towards “cumulative” samples.

In this case, while the program runs, the call stack will be full of a
very large number of stack frames associated with 'factorial'.  This has
a magnifying effect on the cumulative total for 'factorial'.

At the deepest point in the recursion, there will be 100 thousand active
stack frames for 'factorial' on the call stack.  To simplify things a
bit, suppose that the profiler finds that we spent 1 millisecond at that
stack depth.  How will that 1 millisecond contribute to the statistics?

According to the text above, it is supposed to add 1 millisecond of time
to the "self" total for the procedure on the top of the call stack
(factorial), and it's also supposed to add 1 millisecond to the
"cumulative" total of each procedure on the call stack.  In other words,
it loops over every stack frame, and for each one it adds 1 millisecond
to the associated procedure.  In this case, since there are 100 thousand
active stack frames for 'factorial', that single millisecond of real
time will lead to 100 thousand milliseconds (i.e. 100 seconds) being
added to the cumulative total of 'factorial'.

So, I believe that 'statprof' is working as it was designed to.  Whether
this design makes sense is another question.  I admit that I don't know
much about profilers, so I'm not yet prepared to have a useful opinion
on this question.

  Regards,
Mark

Re: Continuations in custom port: "cannot invoke continuation from this context"

[Mark H Weaver]

Hi Caleb,

Earlier I wrote:
> Caleb Ristvedt  writes:
>
>> On Thu, Mar 28, 2019 at 12:58 PM Stefan Israelsson Tampe <
>> stefan.ita...@gmail.com> wrote:
>>
>>> I have not looked at this more carefully. But I suspect that the code is
>>> going to C land and then enters the scheme again
>>> via the hooks and then continuations won't work as expected.
>
>> Any idea why it would still be going to C land even with suspendable ports
>> installed? The manual says they're implemented in scheme for precisely this
>> reason.
>
> 'install-suspendable-ports!' replaces only a subset of the C port
> functions with Scheme implementations, namely the ones listed in
> 'port-bindings' in suspendable-ports.scm.
>
> The problem here is the custom port.  Custom ports are implemented in C,
> and are not reimplemented by suspendable-ports.scm.  When you perform
> I/O on the custom port, it enters C code in r6rs-ports.c which calls
> back into Scheme code to call your custom I/O handlers 'read!' and
> 'write!'.

Since writing this, I've written preliminary patches to add suspendable
I/O support for custom ports:

  https://lists.gnu.org/archive/html/guile-devel/2019-04/msg00019.html

I'd be curious to hear how it works in your use case.

  Mark

Re: string-ports issue on Windows

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> Bearing in mind majority of strings code in GnuCash handle Unicode just
> fine. However, there are some currencies e.g.TYR
> https://en.wikipedia.org/wiki/Turkish_lira need extended Unicode and are
> misprinted as ? in the reports.

I looked into Turkish lira, which according to the web page above was
recently (2012) assigned the Unicode code point U+20BA, included in
Unicode 6.2.  As that the same code point that's getting munged on your
end?

I tried passing this character through Guile 2.2's string ports and also
through 'format' on my system, and encountered no difficulties:

--8<---cut here---start->8---
mhw@jojen ~/guile-stable-2.2$ guile
GNU Guile 2.2.4
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (define lira #\x20BA)
scheme@(guile-user)> lira
$1 = #\₺
scheme@(guile-user)> (string lira)
$2 = "₺"
scheme@(guile-user)> (call-with-output-string
   (lambda (port) (display lira port)))
$3 = "₺"
scheme@(guile-user)> (call-with-output-string
   (lambda (port) (display (string lira) port)))
$4 = "₺"
scheme@(guile-user)> (call-with-input-string (string lira) read-char)
$5 = #\₺
scheme@(guile-user)> (call-with-input-string (string lira) read)
$6 = ₺
scheme@(guile-user)> ,use (ice-9 rdelim)
scheme@(guile-user)> (call-with-input-string (string lira) read-line)
$7 = "₺"
scheme@(guile-user)> (number->string (char->integer (call-with-input-string 
(string lira) read-char)) 16)
$8 = "20ba"
scheme@(guile-user)> (format #f "~a bla" lira)
$9 = "₺ bla"
scheme@(guile-user)> (format #f "~a bla" (string lira))
$10 = "₺ bla"
scheme@(guile-user)> ,use (ice-9 format)
scheme@(guile-user)> (format #f "~a bla" lira)
$11 = "₺ bla"
scheme@(guile-user)> (format #f "~a bla" (string lira))
$12 = "₺ bla"
scheme@(guile-user)> (simple-format #f "~a bla" lira)
$13 = "₺ bla"
scheme@(guile-user)> (simple-format #f "~a bla" (string lira))
$14 = "₺ bla"
scheme@(guile-user)> 
--8<---cut here---end--->8---

I'm not sure if you'll see it correctly in your mail client, but in the
above transcript, I see unmunged Turkish liras in every response from
Guile except $8.

If you type the same commands above into a fresh Guile session on your
system, does it behave correctly, or are the results munged?

If you're able to produce a small self-contained example that
demonstrates the problem, e.g. something along the lines of the
transcript above but which misbehaves on your system, that would be very
helpful.

Also, can you tell me precisely which version of Guile is misbehaving,
and what versions of libunistring and/or iconv is Guile linked with?

  Thanks,
Mark

Re: string-ports issue on Windows

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> I'm struggling with string-ports on Windows.
>
> Last para of
> https://www.gnu.org/software/guile/manual/html_node/String-Ports.html
> "With string ports, the port-encoding is treated differently than other
> types of ports. When string ports are created, they do not inherit a
> character encoding from the current locale. They are given a default locale
> that allows them to handle all valid string characters."
>
> This causes a string-sanitize function to not run correctly in Windows.
> (locale-encoding) says "CP1252" no matter what LANG or setlocale I try.
>
> The use case is to sanitize string for html, but on Windows it munges
> extended-unicode.

Can you explain more fully what the problem is?  I know a fair amount
about Unicode, but my knowledge of Windows is extremely weak.

What exactly is "extended-unicode" in this context?  References welcome.

  Thanks,
Mark

Re: string-ports issue on Windows

[Mark H Weaver]

Hi,

Eli Zaretskii  writes:

>> From: Christopher Lam 
>> Date: Tue, 16 Apr 2019 04:13:14 +
>> 
>> I'm struggling with string-ports on Windows.
>
> Which version of Guile are you using, and where/how did you obtain the
> Windows binary?
>
>> Last para of
>> https://www.gnu.org/software/guile/manual/html_node/String-Ports.html
>> "With string ports, the port-encoding is treated differently than other
>> types of ports. When string ports are created, they do not inherit a
>> character encoding from the current locale. They are given a default locale
>> that allows them to handle all valid string characters."
>> 
>> This causes a string-sanitize function to not run correctly in Windows.
>> (locale-encoding) says "CP1252" no matter what LANG or setlocale I try.
>> 
>> The use case is to sanitize string for html, but on Windows it munges
>> extended-unicode.
>
> Can you show the code that doesn't work for you as you expected?  At
> least the example in "String Ports" seems to work for me as I'd
> expect, with Guile 2.0.11.

The change in string ports that Christopher is referring to was first
introduced in Guile 2.2.0.

   Mark

Re: Continuations in custom port: "cannot invoke continuation from this context"

[Mark H Weaver]

Hi Caleb,

Caleb Ristvedt  writes:

> On Thu, Mar 28, 2019 at 12:58 PM Stefan Israelsson Tampe <
> stefan.ita...@gmail.com> wrote:
>
>> I have not looked at this more carefully. But I suspect that the code is
>> going to C land and then enters the scheme again
>> via the hooks and then continuations won't work as expected.

> Any idea why it would still be going to C land even with suspendable ports
> installed? The manual says they're implemented in scheme for precisely this
> reason.

'install-suspendable-ports!' replaces only a subset of the C port
functions with Scheme implementations, namely the ones listed in
'port-bindings' in suspendable-ports.scm.

The problem here is the custom port.  Custom ports are implemented in C,
and are not reimplemented by suspendable-ports.scm.  When you perform
I/O on the custom port, it enters C code in r6rs-ports.c which calls
back into Scheme code to call your custom I/O handlers 'read!' and
'write!'.

   Mark

Re: Unicode numeric value

[Mark H Weaver]

Mark H Weaver  writes:

> If the range of relevant code points is small enough, another approach
> would be to use a vector:
>
>   (define private-code-point-start #xE000)
>   (define private-code-point-end   #xF900)
>
>   (define (code-point-in-range? cp)
> (<= private-code-point-start
> cp
> private-code-point-end))

Sorry, the definition above is incorrect.  It should be:

   (define (code-point-in-range? cp)
 (<= private-code-point-start
 cp
 (- private-code-point-end 1)))

 Mark

Re: Unicode numeric value

[Mark H Weaver]

Hi,

Freeman Gilmore  writes:

> On Sun, Dec 16, 2018 at 3:15 AM Mark H Weaver  wrote:
>
>  Freeman Gilmore  writes:
>
>  > I am looking for a procedure that will read the numeric value, field 8, of
>  > an Unicode numeric character.   Has anyone written this procedure or know
>  > where I can find it?
>
>  The 'r7rs-wip' branch of the Guile git repository contains a procedure
>  that does this, with a lookup table derived from Unicode 6.3.0.
>
>
> https://git.savannah.gnu.org/cgit/guile.git/tree/module/scheme/char.scm?h=r7rs-wip
>
>  The file is written as an R7RS library form, which won't work on current
>  releases of Guile, but for now you could simply extract the
>  'digit-value' procedure from it, provided that you preserve the
>  copyright notice.
>
>Mark
>
> Thank you Mark:
>
> That is only half the battle, let me explain.  I do not want to read
> the standard Unicode table.  I want to directly read field 8 of a
> numeric character in the privet use area of the Unicode.
>
> This is not part of scheme.  The other half, I need to finger out how
> to put the numeric values in field 8 for the characters I want to use.

If the mapping from code points to numeric values is static, then you
could simply modify the lookup table in the code I suggested above.

If the mapping is dynamic, then you'll need a different strategy.  One
simple approach would be to use a hash table mapping from characters to
digit values:

  (define digit-value-table (make-hash-table))
  
  (define (set-digit-value! char value)
(hashv-set! digit-value-table char value))

  (define (digit-value char)
(hashv-ref digit-value-table char #f))

If the range of relevant code points is small enough, another approach
would be to use a vector:

  (define private-code-point-start #xE000)
  (define private-code-point-end   #xF900)

  (define (code-point-in-range? cp)
(<= private-code-point-start
cp
private-code-point-end))

  (define digit-value-table
(make-vector (- private-code-point-end
private-code-point-start)
 #f))

  (define (set-digit-value! char value)
(let ((cp (char->integer char)))
  (unless (code-point-in-range? cp)
(error "set-digit-value!: code point out of range:" cp))
  (vector-set! digit-value-table
   (- cp private-code-point-start)
   value)))

  (define (digit-value char)
(let ((cp (char->integer char)))
  (and (code-point-in-range? cp)
   (vector-ref digit-value-table
   (- cp private-code-point-start)

For a more compact representation, you could use a SRFI-4 homogeneous
numeric vector instead, although you'd need to designate a special
numeric value to represent "not a digit".

Regards,
  Mark

Re: Unicode numeric value

[Mark H Weaver]

Freeman Gilmore  writes:

> I am looking for a procedure that will read the numeric value, field 8, of
> an Unicode numeric character.   Has anyone written this procedure or know
> where I can find it?

The 'r7rs-wip' branch of the Guile git repository contains a procedure
that does this, with a lookup table derived from Unicode 6.3.0.

  
https://git.savannah.gnu.org/cgit/guile.git/tree/module/scheme/char.scm?h=r7rs-wip

The file is written as an R7RS library form, which won't work on current
releases of Guile, but for now you could simply extract the
'digit-value' procedure from it, provided that you preserve the
copyright notice.

  Mark

Re: a network client

[Mark H Weaver]

Hi,

Catonano  writes:

> I wrote a small server in Guile scheme
>
> I used the command "nc" to test it
>
> It seems to work
>
> Then I wrote a small client in Guile
>
> It doesn't work and I don't understand why

I'm sorry, but there's not enough information in your email for me to
understand what's happening.

It would be most helpful if you could provide a small, self-contained
example of the server and client code, so that we can try to reproduce
the problem on our systems.

 Regards,
   Mark

Re: Need help with macro

[Mark H Weaver]

Hi Mike,

Mike Gran  writes:

> Hey all,
>
> I need help making a macro.
>
> I have an existing procedure of the form
>
> (call-method self method (...))
>
> Note that SELF is a struct, METHOD is a string, and the ellipses can
> be anything.
>
> I would like to make a macro that transforms into the above, but, is
> of the form
>
> (send self (method ...))
>
> where SELF is the same struct, METHOD is a literal symbol, and the
> ellipses are unchanged.
>
> For example, I would like to call
>
> (send date (get-year))
> (send date (set-year 2018))
> (send date (set-dmy 1 1 2018))
>
> and have it be
>
> (call-method date "get-year" '())
> (call-method date "set-year" (list 2018))
> (call-method date "set-dmy" (list 1 1 2018))
>
> I get hung up on figuring out how to handle the literal symbol and the
> ellipses.

Here's a syntax-rules macro to do it:

  (define-syntax send
(syntax-rules ()
  ((send self (method arg ...))
   (call-method self (symbol->string 'method) `(,arg ...)

However, in this implementation, the 'symbol->string' call is deferred
to run time.

Here's a syntax-case macro that does the conversion at compile time:

  (define-syntax send
(lambda (stx)
  (syntax-case stx ()
((send self (method arg ...))
 (identifier? #'method)
 (with-syntax ((method-str (symbol->string
(syntax->datum #'method
   #'(call-method self method-str `(,arg ...)))

This syntax-case macro also verifies at compile time that 'method' is a
bare identifier, via the guard (identifier? #'method).

Note that to generate the list of arguments, the simpler approach would
have been to write (list arg ...) instead of `(,arg ...).  I chose to
use quasiquote mainly to benefit from an optimization in the quasiquote
macro, namely that `() expands into '(), which is preferable to (list).

  Mark

Re: Guile 2.2.4 warnings

[Mark H Weaver]

Hi,

Tommi Höynälänmaa  writes:

> I just installed Guile 2.2.4 to a chroot sid (Debian)
> environment. Guile gives the following warnings on startup:
>
> ---cut here---
>
> guile: warning: failed to install locale
> warning: failed to install locale: Invalid argument
>
> ---cut here---

This indicates that setlocale(3) failed to set the locale specified by
your environment variable settings (LANG, LC_ALL, LC_CTYPE, etc),
probably because the associated locale data is not installed in the
chroot.

> Are these something important?

This is an area where my knowledge is relatively weak, but I guess the
implications of failing to set the locale include:

* ASCII encoding will likely be used by default by Guile, e.g. for file
  and terminal I/O, interpretation of command-line arguments and
  environment variables, and interpretation of file names on disk.
  Normally, your locale encoding (probably UTF-8) would be used by
  default for these purposes.

* Certain operations such as locale-aware string comparisons, formatting
  of dates, numbers, monetary amounts, etc, would normally be affected
  by your locale setting.  If the locale cannot be set, I guess it will
  use the default behaviors, primarily based on US English.

If only ASCII characters are used, and if you don't mind the US English
locale conventions being used, it probably doesn't matter.

  Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Alex,

Alex Vong  writes:
> Btw, this bug report[0]
> explains why the guile bindings were removed. Although it's mark as
> wontfix, should we encourage the maintainers to re-enable the guile
> bindings in the experimental repository?

Based on:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147#10

I think the Debian maintainers have made it clear that they will not
re-enable the Guile bindings until the underlying bug is fixed, which
apparently causes intermittent failures in GnuTLS's test suite when
Guile is built with certain compile flags, as described here:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863#84

> The problems could perhaps be solved by building the bindings with
> guile 2.2.

Sure, this would be a worthwhile thing to try, but we should probably
try it ourselves before asking the Debian developers to spend time on
it.  I can't try it easily because I use GuixSD, but if there are
interested users of Debian and Guile here, it would be a great help to
try reproducing the problem described in Debian bug 805863, and seeing
if the problem can be avoided by building GnuTLS with Debian's Guile-2.2
package.

  Regards,
Mark


> [0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863147

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Alex,

Alex Vong  writes:

> Maybe what I wrote is not clear. What I mean is that since Debian build
> of gnutls does not include the guile bindings, we have to build gnutls
> from the source tarball ourselves.
>
> The flag "--with-guile-site-dir=/usr/local/share/guile/site/2.2"
> instructs the build script to install the guile bindings into
> "/usr/local/share/guile/site/2.2".

Sure.  This is reasonable, but for the sake of completeness, I'll point
out two disadvantages with this approach, namely that (1) it entails
effectively overriding Debian's GnuTLS library with your manually built
version, which potentially affects the operation of any Debian package
that links to GnuTLS and (2) it means staying on top of security updates
yourself, i.e. recompiling and installing new versions of GnuTLS or the
bundled copies of libtasn1 and libunistring when security flaws are
discovered in those versions.  Ditto for the other packages that you
build and install manually.

 Regards,
   Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Alex,

Alex Vong  writes:

> Zelphir Kaltstahl  writes:
>
>> I took another look at the page you linked to. The issue is, that I
>> would like to try Guix package manager to install GNUTLS + Guile
>> bindings, but Guix itself has the requirement of GNUTLS + Guile
>> bindings. See:
>> https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
>>
>> So I guess I would need to somehow get GNUTLS and Guile bindings working
>> before getting Guix and then when I have Guix, I can probably somehow
>> link to the Guix installed GNUTLS and Guile bindings.
>>
>> Now I don't know where I would get the Guile bindings from. Are they
>> included in the releases on https://gnutls.org/ ?
>>
> As Mark has pointed out, if you use the binary installation of guix,
> then you can install gnutls using guix. However, I prefer to build guix
> from source. Therefore, I use the standard "configure, make, make
> install" method with appropriate CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS
> (I include some hardending flags[0][1]):
>
>   ./configure --with-included-libtasn1 --with-included-unistring 
> --with-guile-site-dir=/usr/local/share/guile/site/2.2
>   make
>   make check
>   make install
>
> The above works in Debian. Apart from it, you also have to build
> scheme-bytestructure, guile-git and guile-sqlite3 from source using the
> same method. Besides, it is important to keep gnutls up to date (since
> it is a piece of security-sensitive software).

Interesting.  I haven't tried building Guix from source on Debian in a
few years, but it's not clear to me how your suggestion above addresses
the original problem that Zelphir mentioned, namely that Guix requires
the Guile bindings for GnuTLS, which are not included in Debian.  I
don't see any mention of building those bindings above.  Am I missing
something?

   Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Zelphir Kaltstahl  writes:

> I took another look at the page you linked to. The issue is, that I
> would like to try Guix package manager to install GNUTLS + Guile
> bindings, but Guix itself has the requirement of GNUTLS + Guile
> bindings. See:
> https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html#Requirements
>
> So I guess I would need to somehow get GNUTLS and Guile bindings working
> before getting Guix and then when I have Guix, I can probably somehow
> link to the Guix installed GNUTLS and Guile bindings.

If you install Guix using the binary installation method, this will not
be an issue, because the binary install tarball is self-contained, i.e.
it includes all of Guix's dependencies, including Guile, GnuTLS,
sqlite3, guile-sqlite3, guile-json, libgc, libffi, etc, all the way down
to the C library, even low-level things like the dynamic linker, all in
/gnu/store.  You should be able to unpack it on any system with a Linux
kernel on one of our supported architectures, no matter how bare its
userspace environment is, follow the remaining install steps described
on the following page, and it should just work.

https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html

   Mark

Re: How to build GNUTLS Guile bindings on Xubuntu

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> I read in the docs at
> https://www.gnu.org/software/guile/manual/html_node/Web-Client.html that
> Guile will dynamically make use of GNUTLS for HTTPS requests. However, I
> don't seem to have the bindings installed. There is a link to a guide on
> the same docs page, but the link is dead:
> https://www.gnu.org/software/guile/manual/gnutls-guile/Guile-Preparations.html#Guile-Preparations
> . More searching showed me the following:
>
> There seems to have been some bug which prevented some tests on Debian
> from completing:
>
> Bug discussions seem to indicate the bug was fixed?:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821457
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=805863
>
> Then there is a commit removing Guile bindings from the gnutls-dev
> package on Debian:
>
> https://salsa.debian.org/gnutls-team/gnutls/commit/ebb7130b47dc08311c1de2c189758a73bbaeca27#58ef006ab62b83b4bec5d81fe5b32c3b4c2d1cc2_35_35

If you look closely, those two bugs were "fixed" by the above commit,
i.e. by simply disabling the Guile bindings in Debian's GnuTLS package.

> Apart from that, I could not find a download link or repository of the
> bindings, like I could for other libraries I so far used in Guile and
> run configure make make install in it. So I do not know how to get the
> bindings for my system, so that my Guile will be able to make requests
> using HTTPS as well as HTTP.

The bindings in question are not a separate package.  They are built and
installed by GnuTLS itself, unless explicitly disabled by passing
"--disable-guile" to the GnuTLS configure script, as Debian now does.

> The bindings seem to be available on GUIX:
>
> https://gitlab.digitalcourage.de/htgoebel/guix/commit/1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

Note that the repository above is not the official Guix repository, but
rather someone's personal variant.  Here's the same commit in the
official Guix repository:

  
https://git.sv.gnu.org/cgit/guix.git/commit/?id=1dbe3a8db0a3e5a8e5f9b30e6f6a6bbfb699275b

> How can I make it work on a Xubuntu 18.04?

Yes, an easy solution would be to install Guix on top of Xubuntu, and
installing the 'guile' and 'gnutls' packages from Guix.  Note that the
binaries from Guix are self-contained and independent of the host
system, in the sense that they are installed in different directories
(within /gnu/store) and don't use any libraries from the host system,
not even the C library.  Guix installs only in /gnu, /var/guix,
/var/log/guix, /etc/guix, ~/.config/guix, and ~/.guix-profile.  You must
set your environment variables to point within ~/.guix-profile, e.g. add
~/.guix-profile/bin to your PATH, to enable the use of software from
Guix.  If you'd like to try it, the instructions are here:

  https://www.gnu.org/software/guix/manual/en/html_node/Installation.html

  Mark

Re: http-request procedure missing or not exported from web client

[Mark H Weaver]

Zelphir Kaltstahl  writes:

> I recently noticed, that the procedure ~http-request~
> (https://www.gnu.org/software/guile/manual/html_node/Web-Client.html)
> does not seem to be available when I do:
>
> ~~~
>
> (use-modules (web client))
> (http-request ...)
>
> ~~~
>
> However, the specialised procedures like ~http-get~ are available.
>
> Are the docs wrong or is my Guile (2.2.3) too old or did I misunderstand
> the docs?

Guile-2.2.4 is the first version to export 'http-request'.

   Mark

Re: What's up with 'current-load-port'

[Mark H Weaver]

Thomas Morley  writes:

> what's up with 'current-load-port'?
>
> Simply checking in a guile-prompt I get:
> guile-1.8: #
> guile-2.0.14: #
> guile-2.2.4 and guile-2.9.1:
> ;;; : warning: possibly unbound variable `current-load-port'
> ERROR: In procedure module-lookup: Unbound variable: current-load-port
>
> It's in the manual, though, without any hint it could be deprecated or
> disabled or the need to use a certain module.

Good catch.  I agree that this was mishandled.

Out of curiosity, what do you use 'current-load-port' for?

Although it's in the manual, the description doesn't really describe
what the procedure returns, or when one can rely upon it being set.  It
only says that it's "used internally by 'primitive-load'".

  Mark

Re: Beginner questions

[Mark H Weaver]

Hi,

swedebugia  writes:

> I would like to learn more scheme and I would like to make a small CLI
> program that runs in the terminal and prompts the user for input and
> evaluates it.

This sounds like a description of Guile's REPL itself.  Are you looking
to implement your own simple REPL for educational purposes?

> Is that possible with guile? In the REPL?

Yes and yes.

> Can someone point me in the right direction for succeding with that?

Here are some relevant sections of the Guile manual to get started:

  https://www.gnu.org/software/guile/manual/html_node/Scripting-Examples.html
  
https://www.gnu.org/software/guile/manual/html_node/Read_002fLoad_002fEval_002fCompile.html
  https://www.gnu.org/software/guile/manual/html_node/Line_002fDelimited.html
  https://www.gnu.org/software/guile/manual/html_node/Strings.html

The first link gives several small examples of Guile scripts.  The
second documents procedures to read, evaluate, and print S-expressions,
and in particular the 'read', 'eval', and 'write' procedures, which are
the three main components of a REPL.  The third link documents
procedures to read a single line of input as a string, and the fourth
documents the string operations.

Would you like to start by looking over these sections of the manual,
and then asking more questions as they arise?

  Mark

Re: A macro containing a mini-macro?

[Mark H Weaver]

I wrote:
> It turns out that pure 'syntax-rules' macros are turing complete, but
> they are limited in the ways that they can inspect the syntax objects
> given to them as operands.  In particular, they cannot inspect atomic
> expressions, except to compare them with the finite set of literals in
> the first operand to 'syntax-rules'.  This is not sufficient to
> interpret an arbitrary integer literal.  It could only be done with
> 'syntax-rules' macros if the 'count' field were represented using a
> finite set of literals and/or list structure.  E.g. it could be done if
> the count were represented as a list of decimal digits like (1 4 2) for
> 142.

Correction: the finite set of literals that can be recognized by
'syntax-rules' macros must be identifiers, so a representation with bare
decimal digits like (1 4 2) would not work.  Something like (_1 _4 _2)
would be possible, though.

   Mark

Re: A macro containing a mini-macro?

[Mark H Weaver]

Hi,

HiPhish  writes:

> Hello Schemers,
>
> I have written a small macro for writing test specifications:
>
> (define-syntax test-cases
>   (syntax-rules ()
> ((_ title
>(given (byte byte* ...))
>...)
>  (begin
>(test-begin title)
>(call-with-values (λ () (open-bytevector-output-port))
>  (λ (out get-bv)
>(pack given out)
>(let ((received (get-bv))
>  (expected (u8-list->bytevector '(byte byte* ...
>  (test-assert (bytevector=? received expected)
>...
>(test-end title)
>
> The idea is that I can specify a series of test cases where each case 
> consists 
> of an object and a sequence of bytes which this object is to be serialized to:
>
> (test-cases "Single precision floating point numbers"
>   (+3.1415927410125732 (#xCA #b0100 #b01001001 #b #b11011011))
>   (-3.1415927410125732 (#xCA #b1100 #b01001001 #b 
> #b11011011)))
>
> This works fine, but sometimes there is a sequence of the same bytes and it 
> would be more readable if I could write something like this:
>
> ((make-vector 16 0) (#xDC (16 #x00)))
>
> instead of writing out 16 times `#x00`. This would require being able to make 
> a distinction in the pattern whether `byte` is of the pattern
>
> byte
>
> or
>
> (count byte)
>
> and if it's the latter construct a list of `count` `byte`s (via `(make-list 
> count byte)` for example) and splice it in. This distinction needs to be made 
> for each byte specification because I want to mix actual bytes and these "RLE-
> encoded" byte specifications.
>
> So I guess what I'm looking for is to have a `syntax-rules` inside a `syntax-
> rules` in a way. Can this be done?

It cannot be done with pure 'syntax-rules' macros, but it can certainly
be done with procedural macros, sometimes called 'syntax-case' macros.
Procedural macros are quite general, allowing you to write arbitrary
Scheme code that runs at compile time to inspect the macro operands and
generate arbitrary code.

I'll describe how to do this with macros further down, but let me begin
with the simple approach.

As rain1 suggested, this can be accomplished most easily by writing a
normal procedure to convert your compact bytevector notation into a
bytevector, and then having your macro expand into code that calls that
procedure.  Here's working code to do that:

--8<---cut here---start->8---
(use-modules (ice-9 match)
 (srfi srfi-1)
 (rnrs bytevectors))

(define (compact-bytevector segments)
  (u8-list->bytevector
   (append-map (match-lambda
 ((count byte)  (make-list count byte))
 (byte  (list byte)))
   segments)))

(define-syntax test-cases
  (syntax-rules ()
((_ title
(given (seg ...))
...)
 (begin
   (test-begin title)
   (call-with-values (λ () (open-bytevector-output-port))
 (λ (out get-bv)
   (pack given out)
   (let ((received (get-bv))
 (expected (compact-bytevector '(seg ...
 (test-assert (bytevector=? received expected)
   ...
   (test-end title)


scheme@(guile-user)> (compact-bytevector '(#xDC (16 #x00)))
$2 = #vu8(220 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0)
scheme@(guile-user)> ,expand (test-cases "Single precision floating point 
numbers"
 ((make-vector 16 0) (#xDC (16 #x00
$3 = (begin
   (test-begin
 "Single precision floating point numbers")
   (call-with-values
 (lambda () (open-bytevector-output-port))
 (lambda (out get-bv)
   (pack (make-vector 16 0) out)
   (let ((received (get-bv))
 (expected (compact-bytevector '(220 (16 0)
 (test-assert (bytevector=? received expected)
   (test-end
 "Single precision floating point numbers"))
scheme@(guile-user)> 
--8<---cut here---end--->8---

Now, suppose it was important to do more of this work at macro expansion
time.  For example, if efficiency was a concern, it might not be
acceptable to postpone the conversion of '(#xDC (16 #x00)) into
#vu8(220 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) until run time.

It turns out that pure 'syntax-rules' macros are turing complete, but
they are limited in the ways that they can inspect the syntax objects
given to them as operands.  In particular, they cannot inspect atomic
expressions, except to compare them with the finite set of literals in
the first operand to 'syntax-rules'.  This is not sufficient to
interpret an arbitrary integer literal.  It could only be done with
'syntax-rules' macros if the 'count' field were represented using a
finite set of literals and/or list structure.  E.g. it could be done if
the count were represented as a list o

Re: Comparing two hash tables for equality?

[Mark H Weaver]

Aleksandar Sandic  writes:
> I did not know about HAMTs, looks interesting. It's not relevant to my use-
> case, but it's an interesting topic to look into. Thanks.

There's an implementation of HAMTs that works with Guile in the
following "Purely Functional Data Structures in Scheme" library:

  https://github.com/ijp/pfds

I'm not sure off-hand how well optimized they are.

  Mark

Re: A value for "nothing"

[Mark H Weaver]

Hi John,

John Cowan  writes:

> On Tue, Aug 28, 2018 at 11:40 AM Mark H Weaver  wrote:
>  
>  That's the phrase used in R7RS-small, which fails to define it, as you
>  noted, but that shortcoming is limited to R7RS.
>
> The relevant sentences in R5RS and R7RS are identical: " If 
> yields a false value and no  is specified, then the result
> of the expression is unspecified."  Likewise, the paragraph from 1.3.2
> you quote below is identical in both standards.  So either they both
> define it or they both don't.

Good point, you're absolutely right about that.  I confess that I didn't
bother to check R7RS carefully, because I simply assumed that you had
good knowledge of it.  I only looked carefully at R5RS and R6RS, but I
guess we came to different conclusions from the same text.

>  In R6RS, section 11.4.3 (Conditionals) provides this example:
>
> Unlike Wil Clinger, and apparently you, I don't believe that examples
> in specs are normative.  But setting that aside for the moment:
>  
>  I take the use of the singular form of "value" here to imply that it
>  returns only one value.
>
> In R6RS 11.13, vector-set! is said to return unspecified values (note
> plural), but in the examples appears "⇒ unspecified", showing that
> this notation can be used where multiple unspecified values (or zero
> values) are allowed.

This is also a good point.  This looks like a mistake to me in R6RS.
Someone should probably notify them.

Anyway, these ambiguities in the informal descriptions underline the
importance of formal semantics.  Thankfully, the core constructs are
unambiguously described by them in the standards, including 'if'.

Thank you for your tireless efforts to promote Scheme standardization,
and also for helping me break through stone walls that were placed in my
way during the R7RS-small process.  I'm grateful for your work.

   Mark

Re: A value for "nothing"

[Mark H Weaver]

John Cowan  writes:

> On Tue, Aug 28, 2018 at 11:40 AM Mark H Weaver  wrote:
>  
>  That's the phrase used in R7RS-small, which fails to define it, as you
>  noted, but that shortcoming is limited to R7RS.
>
> The relevant sentences in R5RS and R7RS are identical: " If 
> yields a false value and no  is specified, then the result
> of the expression is unspecified."  Likewise, the paragraph from 1.3.2
> you quote below is identical in both standards.  So either they both
> define it or they both don't.
>
>  In R6RS, section 11.4.3 (Conditionals) provides this example:
>
> Unlike Wil Clinger, and apparently you, I don't believe that examples
> in specs are normative.

Alright, well, the formal denotational semantics makes it 100%
unambiguous, as I noted in my previous email.

   Mark

Re: A value for "nothing"

[Mark H Weaver]

Mark H Weaver  writes:

> John Cowan  writes:
>
>> On Tue, Aug 28, 2018 at 3:01 AM Mark H Weaver  wrote:
>>  
>>  In RnRS, (define a (if #f #f)) is allowed and guaranteed to assign
>>  *some* object to 'a' without signalling an error.  
>>
>> Actually, the phrase used is "the result is unspecified", which unfortunately
>> is not defined in any RnRS.
>
> That's the phrase used in R7RS-small, which fails to define it, as you
> noted, but that shortcoming is limited to R7RS.

Actually, the behavior _is_ clearly defined, in the formal denotational
semantics in both R5RS and R7RS.  If you learn how to read those, you'll
see that there's no question that (if #f #f) is guaranteed to return
exactly one unspecified value.

 Mark

Re: A value for "nothing"

[Mark H Weaver]

John Cowan  writes:

> On Tue, Aug 28, 2018 at 3:01 AM Mark H Weaver  wrote:
>  
>  In RnRS, (define a (if #f #f)) is allowed and guaranteed to assign
>  *some* object to 'a' without signalling an error.  
>
> Actually, the phrase used is "the result is unspecified", which unfortunately
> is not defined in any RnRS.

That's the phrase used in R7RS-small, which fails to define it, as you
noted, but that shortcoming is limited to R7RS.

In R6RS, section 11.4.3 (Conditionals) provides this example:

  (if #f #f) ===> unspecified

whose meaning is defined in section 6.6 (Evaluation examples), which
states:

  Moreover, the "===>" symbol is also used to explicitly say that the
  value of an expression is unspecified.  For example:

(eqv? "" "") ===> unspecified

I take the use of the singular form of "value" here to imply that it
returns only one value.

R5RS is even more clear.  It states "If  yields a false value and
no  is specified, then the result of the expression is
unspecified."

Section 1.3.2 of R5RS makes it crystal clear what that means:

  If the value of an expression is said to be "unspecified," then the
  expression must evaluate to some object without signalling an error,
  but the value depends on the implementation; this report explicitly
  does not say what value should be returned.

> Racket produces a syntax error in this situation at least in its
> default language.

Racket has diverged from Scheme quite a bit, to the point that they
don't even call the language "Scheme" anymore, but rather "Racket".

   Mark

Re: Comparing two hash tables for equality?

[Mark H Weaver]

Hi Aleksandar,

Aleksandar Sandic  writes:

>> An equality test on hash tables needs to know how to compare the keys
>> and how to compare the values.  There's no way to pass those additional
>> arguments to 'equal?', so it can't do that job.
> It has to compare the values, but not the keys.

There's an implicit equality test on keys every time you perform a hash
table lookup.  By using 'hash-ref' and 'hash-get-handle', you are
implicitly using 'equal?' to compare the keys in the hash table with the
key that you're asking to look up.

Guile's native hash tables are unusual in that they do not internally
keep track of which equality test on keys to use.  Instead, it is your
responsibility to consistently use the functions corresponding to same
equality test in all accesses to a given hash table.  As the Guile
manual states:

  Like the association list functions, the hash table functions come in
  several varieties, according to the equality test used for the keys.
  Plain ‘hash-’ functions use ‘equal?’, ‘hashq-’ functions use ‘eq?’,
  ‘hashv-’ functions use ‘eqv?’, and the ‘hashx-’ functions use an
  application supplied test.

 A single ‘make-hash-table’ creates a hash table suitable for use
  with any set of functions, but it’s imperative that just one set is
  then used consistently, or results will be unpredictable.

  


So, your 'hash-table-equal?' predicate implicitly assumes that the hash
tables passed to it were populated using 'hash-set!' or
'hash-create-handle!'.  If it is applied to a hash table that was
populated with the 'hashq-*!', 'hashv-*!', or 'hashx-*!' procedures,
then the results will be unpredictable.

My 'hash-table=?' predicate makes the same implicit assumption.

  Mark

Re: A value for "nothing"

[Mark H Weaver]

Matt Wette  writes:

> Is it reasonable to expect that if a value can be assigned to a variable
> then a predicate exists to test for that value type?  So, if
>
>   (define a (if #f #f))
>
> does not signal an error then there should be a predicate to indicate the
> value associated with a is unspecified?
>
> If the define allowed in RnRS?  I don't believe there is a predicate to
> test for this. (I could be wrong.)

In RnRS, (define a (if #f #f)) is allowed and guaranteed to assign
*some* object to 'a' without signalling an error.  However, it's not
specified what object will be assigned.  It could be 2 or (foo bar) or
"the cow jumps over the moon".

So, in RnRS, there's no predicate that you could apply to 'a' and be
assured that the result will be #t on all conforming implementations.

Guile has always had a predicate 'unspecified?' since at least 1996, but
personally I would advise against relying on its continued existence in
the future.

Regarding your question whether it's reasonable to expect that every
object has an associated predicate to test for it, I don't know.  It's
an interesting question, but I wonder what would be the practical use of
such an expectation?

Given the fact that there are several mechanisms to add new types that
are distinct from all other types, and that we occasionally add new core
types to Guile that no previously extant predicate would answer #t for,
you certainly cannot rely on being able to write a 'cond' statement that
tests an arbitrary object using some set of predicates and be assured
that at least one of those predicates will answer #t.  You could write
such a 'cond' statement today with that property, but for a future
version of Guile you might find an object for which none of those
predicates returns #t.

 Regards,
   Mark

Re: A value for "nothing"

[Mark H Weaver]

John Cowan  writes:
> However, in formats like JSON where map keys are always strings,
> it can save a lot of space to represent them as symbols, since
> they are often repeated from one object to the next.  There is no such
> limitation in MessagePack, although I bet strings are the most common
> type of map keys.

Agree that this is a point in favor of using symbols to represent
MessagePack strings.

   Mark

Re: A value for "nothing"

[Mark H Weaver]

John Cowan  writes:

> On Mon, Aug 27, 2018 at 12:54 AM Mark H Weaver  wrote:
>  
>  However, in most cases, symbols are precisely what's needed to represent
>  distinguished atomic objects such as this.
>
> The problem with symbols in Scheme is that they are not namespaced, so
> two different modules can use the same symbols in different ways.
> Exported variable bindings can be managed with the module system.

Right, but this is only a problem if the same variable might contain
either a MessagePack or some other data type that uses the same symbol
to mean something different.  In other words, this is only a problem if
you need to be able to distinguish between a MessagePack and some other
data type.

However, given that a MessagePack can also be a boolean, integer, float,
string, etc, and that the plan is apparently to use the corresponding
native Scheme objects to represent these things (which I agree is a good
idea), it will already be infeasible to distinguish between a
MessagePack and anything else.

So, while I agree that this is something to keep in mind when using
symbols in Scheme, I think it's a moot point in this case.

On the other hand, one disadvantage with using record types is that they
cannot be written and then read back in with the standard Scheme reader.
If this is considered desirable, it might be a point in favor of using
symbols instead of records.

  Mark

Re: A value for "nothing"

[Mark H Weaver]

I wrote:
> If he would like people to be able to write code that uses his library
> and works on multiple Scheme implementations, then it will certainly be
> an impediment to use a Racket-specific value for Nil on Racket, and a
> Guile-specific value for Nil on Guile.  It would be much better to use a
> portable Scheme data structure uniformly.

One might object, and point out that since the library will presumably
include procedures to construct and test for nil, it shouldn't matter if
they are represented by different objects under the hood.

The problem is, if #nil is used, it's quite likely that some code built
on top of this library and developed using Guile will end up relying on
the fact that #nil is considered false, and that (null? #nil) returns
true.  That would lead to possibly subtle breakage when the same code is
then used on another implementation.

   Mark

Re: A value for "nothing"

[Mark H Weaver]

 writes:

> On Sun, Aug 26, 2018 at 04:07:13PM -0400, Mark H Weaver wrote:
>
>> I would also avoid Guile's #nil.  That is a very special value, for one
>> purpose relating to Elisp compatibility, and ideally it should not be
>> used for anything else.
>
> I must admit that I'm... pretty confused about this very prescriptive
> tone.

You're right, I should have explained.  I'm overburdened at the moment.

For one thing, HiPhish has already indicated that his library will
support both Guile and Racket, so I conclude that he's at least somewhat
interested in portability to other Scheme implementations.

If he would like people to be able to write code that uses his library
and works on multiple Scheme implementations, then it will certainly be
an impediment to use a Racket-specific value for Nil on Racket, and a
Guile-specific value for Nil on Guile.  It would be much better to use a
portable Scheme data structure uniformly.

More generally, even for people only interested in supporting Guile, if
they would like their libraries to be usable from Elisp code on Guile,
which may become important some day if Guile-Emacs matures, then it's
problematic to use #nil for anything that needs to be distinguished from
'() or #f.

Alternatively, if Guile-Emacs dies and people give up on that project,
which is entirely possible, then we should probably deprecate #nil at
some point and eventually remove it.  Supporting #nil has costs, both in
performance and in the size of generated native code, when checking
whether an object is true (in every 'if' or 'cond') and when checking
for '().  That performance cost is insignificant in an interpreter or VM
implementation, but as we move to native code generation it may become
significant, as both of the aforementioned operations are quite
ubiquitous.

   Mark

Re: A value for "nothing"

[Mark H Weaver]

l...@gnu.org (Ludovic Courtès) writes:

> I would suggesting returning zero values, using:
>
>   (values)
>
> That way, if a caller wrongfully attempts to get at the return value of
> that procedure, it’ll get an error.
>
> Fibers does that in several places, and I think it’s a good convention
> as it conveys exactly what you want.

You cannot store (values) in a variable or data structure, so it
wouldn't work here.

The issue under discussion is how to represent MessagePack's "nil",
which is one of the possible values that a MessagePack can have,
alongside booleans, integers, floats, strings, arrays, etc.

MessagePack is similar to JSON, so the question we're discussing is
analogous to the question of how to represent JSON's "null" in Scheme.

  Mark

Re: A value for "nothing"

[Mark H Weaver]

John Cowan  writes:

> Well, you could use #nil, a Guile-specific unique object that is both falsy
> (like #f) and answers #t to the null? predicate.  It is used to emulate
> Common Lisp's and Elisp's nil.

As I wrote earlier, I would avoid using #nil for anything outside of its
intended use case.

> But a more portable approach would be to define a record type with no
> slots and make just one instance of it.

If _all_ symbols must be distinguishable from the "nothing" object, then
John's suggestion is a fine solution.  To avoid unnecessary heap
allocation, you should indeed arrange to make only one instance of it,
as John said.

However, in most cases, symbols are precisely what's needed to represent
distinguished atomic objects such as this.

I looked at  and I don't see
why a symbol couldn't be used to represent MessagePack's 'nil'.

I hope that HiPhish is not planning to use symbols to represent
MessagePack strings.  That would be an abuse of symbols, IMO.  Symbols
are intended to represent atomic objects (i.e. objects containing no
internal structure) whose only essential operation is to compare them
for equality.

A practical problem with using symbols to represent strings is that
symbols need to be "interned", i.e. stored in a global hash table, to
ensure that any two symbols containing the same sequence of characters
are represented by the same object.  A consequence of this is that every
time a new symbol is encountered, it must be added to the global hash
table, which in turn must be protected by a mutex.  This would slow down
creation of MessagePack strings, especially if several threads are doing
it concurrently, to no good end that I can see.

   Mark

Re: Comparing two hash tables for equality?

[Mark H Weaver]

Hi Aleksandar,

Aleksandar Sandic  writes:

> I have been looking through the reference manual, but there does not seem to 
> be a procedure for comparing two hash tables for equality. The procedure 
> 'equal?' only returns true if the two tables are also 'eq?':
>
> scheme@(guile-user)> (define h1 (make-hash-table))
> scheme@(guile-user)> (define h2 (make-hash-table))
> scheme@(guile-user)> (equal? h1 h2)
> $1 = #f
> scheme@(guile-user)> (equal? h1 h1)
> $2 = #t
>
> Is this the intended behaviour?

Yes.  That might seem surprising.  I have a few things to say on that,
in no particular order:

An equality test on hash tables needs to know how to compare the keys
and how to compare the values.  There's no way to pass those additional
arguments to 'equal?', so it can't do that job.

Hash table implementations typically don't offer an equality test on the
hash tables themselves, and I don't recall anyone ever asking for such
an operation before now.  I guess that's because in the use case where
hash tables are beneficial -- when the tables are very large --
comparing them for equality is expensive.

While hash tables are indispensible for certain use cases, they also
have very significant downsides, and I tend to avoid them whenever
possible.  Their most severe downside, in my opinion, is that they are
fundamentally incompatible with a functional programming style.  They
invariably force all code that uses them to be written in an imperative
style.  I cannot stress enough how important this is.

It's also not possible to efficiently create a new hash table based on
an existing one, but with one or more additional entries.  In Scheme,
you can efficiently 'cons' or 'append' some entries onto the front of an
existing list without modifying the original list.  That includes
association lists.  To do the same with a hash table, you must make a
copy of the entire hash table and then add the new entries to the copy.

There can be no sharing of storage between multiple hash tables, as can
be done with lists, association lists, or balanced trees.  Even if you
don't need sharing, hash tables also use more space than those other
data structures.

So, I would ask yourself whether the benefits of hash tables truly
outweigh the downsides in your particular use case.

If you're comfortable sharing details, I'd be curious to hear what
you're trying to accomplish here, at a higher level.  Maybe there's a
better way.

Anyway, if you really need to compare hash tables for equality, here's
some suggested code that's simpler than the code you wrote, and should
be quite a bit more efficient as well.  In particular, this code
performs no heap allocation:

--8<---cut here---start->8---
(use-modules (ice-9 match))

(define (hash-table-subset? val=? h1 h2)
  "H1 and H2 must be hash tables which use equal? as the equality test
for keys.  Return #t if and only if for every entry (K . V1) in H1,
there exists an entry (K . V2) in H2 such that (VAL=? V1 V2),
otherwise return #f."
  (hash-fold (lambda (k v1 equal-so-far)
   (and equal-so-far
(match (hash-get-handle h2 k)
  ((_ . v2) (val=? v1 v2))
  (#f   #f
 #t
 h1))

(define (hash-table=? val=? h1 h2)
  "H1 and H2 must be hash tables which use equal? as the equality test
for keys.  Return #t if H1 and H2 have the same keys (in the sense of
equal?) and each key has the same value (in the sense of val=?),
otherwise return #f."
  (and (hash-table-subset? val=? h1 h2)
   (hash-table-subset? val=? h2 h1)))

(define (equal*? a b)
  (or (equal? a b)
  (and (hash-table? a)
   (hash-table? b)
   (hash-table=? equal*? a b
--8<---cut here---end--->8---

You might not need 'equal*?' here.  I included it only to match more
closely the behavior of your code.

For most purposes, 'hash-table=?' is actually more flexible than
'equal*?', as long as the "type" of values stored in a given hash table
are known.  If the values stored in the hash table are numbers, then
'val=?' should be '='.  If the values are themselves hash tables, then
'val=?' should be (lambda (a b) (hash-table=? foo=? a b)) for some value
of 'foo=?'.  In some other cases, 'val=?' should be 'equal?'.

If you truly need hash tables whose values could contain either hash
tables or some other type, then something like 'equal*?' might be the
right tool.  However, keep in mind that 'equal?' is usually not the
desired equality test for numbers, because (equal? 1 1.0) => #false and
(equal? +0.0 -0.0) => #false.  'equal?' tests for _operational_
equivalence, which makes distinctions important for memoization and some
other purposes, whereas '=' tests for mathematical equality and is
usually what you want for numbers.

 Regards,
   Mark

Re: A value for "nothing"

[Mark H Weaver]

Hi,

HiPhish  writes:

> I am writing an implementation of MessagePack [1] for Guile and a part of the 
> spec is the presence of a "nil" data type. What would be a good value to 
> express "nothing" in Guile?

First of all, thank you very much for asking the question.  I often wish
that authors of Guile libraries would more often ask for design advice
here before committing to a particular API.

> I cannot use '() because that would be 
> indistinguishable from the empty list, so I thought that the return value of 
> a 
> function that returns nothing would be a good fit.

"The return value of a function that returns nothing" is a
self-contradictory notion, if you think about it :)

> The function `display` for 
> example returns an `#` value, but the only way of producing it 
> without side effects so for is the value of `(if #f #f)`. Is there a better 
> way?

*unspecified* is identifier syntax for (if #f #f), i.e. it expands into
the latter.

However, I would strongly advise against writing code (or worse, APIs)
that depend on (if #f #f) or *unspecified* returning a particular
distinguished value.

Quoting R5RS:

  If the value of an expression is said to be "unspecified," then the
  expression must evaluate to some object without signalling an error,
  but the value depends on the implementation; this report explicitly
  does not say what value should be returned.

It's true that Guile historically has a special object distinct from all
other objects, which (if #f #f) and various other expressions return,
and which prints as "#".

However, the fact that some existing code out there might depend on the
existence of this distinguished object, and that certain expressions in
Guile return it, is historical baggage which carries non-zero costs as
we move to native code generation.

I would also argue that it carries a terrible conceptual cost, in that
it leads to confusion between the concept of a truly unspecified return
value (as in R5RS) and this distinguished value in Guile that is called
"the unspecified value", a non-sensical notion.

I would also avoid Guile's #nil.  That is a very special value, for one
purpose relating to Elisp compatibility, and ideally it should not be
used for anything else.

> In Racket there is the `(void)` [2] procedure which returns a `#` 
> object, so that's what I am using there [3][4]. Any suggestions for Guile?

I would suggest using a symbol.  How about 'nil?

  Mark
 

Re: Change interpreter from octets to hex?

[Mark H Weaver]

Hi Joshua,

Joshua Datko  writes:

> Is it possible to change the default interpreter behavior to display
> hex instead of octets for bytevectors and u8-lists? And if so, how to
> do so?
>
> The default, as y'all know is:
>
> scheme@(guile-user)> #vu8(#x0a #xaa)
> $6 = #vu8(10 170)
>
> But I would like instead the output to be:
> $6 = #vu8(#x0a #xaa)

The printer used by the Guile REPL can be customized using the ",option"
REPL command.  For example:

--8<---cut here---start->8---
mhw@jojen ~$ guile
GNU Guile 2.2.3
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,help system
System Commands [abbrev]:

 ,gc   - Garbage collection.
 ,statistics   [,stat] - Display statistics.
 ,option [NAME] [EXP] [,o] - List/show/set options.
 ,quit[,q ,continue ,cont] - Quit this session.

scheme@(guile-user)> ,option
  compile-options   (#:warnings (unbound-variable 
macro-use-before-definition arity-mismatch format duplicate-case-datum 
bad-case-datum))
  trace #f
  interp#f
  prompt#f
  print #f
  value-history #t
  on-error  debug
scheme@(guile-user)> (use-modules (rnrs bytevectors)
  (ice-9 format))
scheme@(guile-user)> (define (format-octet octet)
   (format #f "#x~2,'0x" octet))
scheme@(guile-user)> (define (print-bytevector bv)
   (display "#vu8(")
   (display (string-join (map format-octet 
(bytevector->u8-list bv
   (display ")"))
scheme@(guile-user)> (define (my-repl-print repl val)
   (if (bytevector? val)
   (print-bytevector val)
   (write val))
   (newline))
scheme@(guile-user)> ,o print my-repl-print
scheme@(guile-user)> #vu8(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20)
$1 = #vu8(#x00 #x01 #x02 #x03 #x04 #x05 #x06 #x07 #x08 #x09 #x0a #x0b #x0c #x0d 
#x0e #x0f #x10 #x11 #x12 #x13 #x14)
scheme@(guile-user)> 
--8<---cut here---end--->8---

This simple strategy won't work for bytevectors nested within another
data structure.  However, you could handle some specific cases by
supporting more data structures in your custom printer.  For example, if
you enhanced the custom printer to print ordinary scheme lists and to
call itself recursively for the elements of those lists, then you could
support printing bytevectors in your preferred format as long as the
surrounding data structure is composed of normal list structure only.

Guile's built-in printer does not support customizing its formatting of
core data structures, so you cannot handle the general case here without
modifying Guile itself.  If you wanted to do that,
'scm_i_print_bytevector' in bytevectors.c is the function to modify.

Another option would be to wrap your bytevectors within your own record
type, in which case you could install your own custom printer for that
type using 'set-record-type-printer!'.

   Mark

Re: reading from input pipe is slow?

[Mark H Weaver]

Hi Thomas,

Thomas Danckaert  writes:

> I've notice that reading a child process' stdout from a port opened
> with (open-input-pipe "command") is very slow.  In fact, it's faster
> to redirect the child process' output to a temporary file, and then
> read this temporary file from Guile, than to directly read the child
> process' output using a port (I've added some example code below where
> you can see this).
>
> Is there an explanation for this?  Or even better: a solution?

For some reason that I don't know, possibly historical, the pipes
created by procedures in (ice-9 popen) are unbuffered by default.  When
reading or writing from an unbuffered port, a system call is made for
every byte.

To enable buffering on pipe ports, call (setvbuf port 'block) after
opening it.  Something like this, where (rnrs io ports) is added to the
module imports you already had, for 'call-with-port':

  (call-with-port (open-input-pipe my-command)
(lambda (port)
  (setvbuf port 'block)
  (string-take-right (get-string-all port)
 15)))

Hope this helps,

  Mark

Re: Trouble trying to use some modules from the docs (Matt Wette)

[Mark H Weaver]

Hi,

Zelphir Kaltstahl  writes:

> Something must be up with either the Guile REPL or the Guile REPL inside
> Emacs' M-x shell. I tried to run socket in it and it could not find
> that. I also ran other code before that, so maybe the issue was, that
> that somehow prevented the REPL from finding the bindings. However, now
> that I tried what you wrote in a completely new REPL in a XFCE terminal,
> it works flawlessly. And such a thing has happened to me before with
> another module inside Emacs M-x shell mode.

Can you please post a transcript demonstrating the problem that you're
seeing while trying to run 'socket' in Emacs shell mode, starting from
the beginning of the REPL session?

In general, it's much more useful to show us the precise commands you
typed, and the precise error message, than to write "I tried to run
socket in it and it could not find that".  I'd like to understand what's
going on here.  I use the Guile REPL from within Emacs all the time, and
it's far more convenient to use from within Emacs than within a bare
terminal.  It would be good to get to the bottom of this.

 Regards,
   Mark

Re: macroexpand-1

[Mark H Weaver]

Hi,

Catonano  writes:

> 2018-05-30 3:07 GMT+02:00 Mark H Weaver :
>
>  This is just a toy, and not very useful in practice.
>  Here's the equivalent formulation for Guile:
>
>(use-modules (system syntax)
> (srfi srfi-11))
>
>(define (syntax-local-value id)
>  (let-values (((type value) (syntax-local-binding id)))
>value))
>
>(define-syntax expand1
>  (lambda (stx)
>(syntax-case stx ()
>  [(_expand1 form)
>   (syntax-case #'form ()
> [(id . more)
>  (identifier? #'id)
>  (let ([transformer (syntax-local-value #'id)])
>(with-syntax ([expansion (transformer #'form)])
>  #''expansion))]
> [_
>  #''form])])))
>
>  (I usually prefer to avoid using square brackets in this way, but for
>  sake of comparison, I used them in the definition of 'expand1' above.)
>
>  Anyway, it works the same way as in Racket for this simple example:
>
>scheme@(guile-user)> (expand1 (or 1 2 3))
>$2 = (let ((t 1)) (if t t (or 2 3)))
>
> This is surprising to me
>
> When I saw that example made in Racket for the first time I instantly
> identified "syntax-local-value" as problematic

You're right, it is problematic, and it's good that you noticed that.
It exposes internal details of Guile's implementation, which is quite
likely to change in the future.  Do not use this interface if you can
avoid it, and expect code that uses it to break in future versions of
Guile.  That said, it can be useful for writing things like macro
steppers.

> Will Guile have anything equivalent ? I asked myself
>
> Now you show me the "(system syntax)" namespace (or module)
>
> I didn't  suspect it existed
>
> Does the manual mention it anywhere ? I didn' t see it

Do you know how to search the manual or its index?  Press 'i' from
either the Emacs or standalone info browsers to search the index, where
you can find 'syntax-local-binding'.

You can also search the entire manual text by pressing 's'.  You can
find (system syntax) that way.

> Or maybe does it belong to any scheme standard ?

No, certainly not.

> Do any more (system ) namespaces exist ? 
>
> How would I know ?

Look in the "module" subdirectory of the Guile source tree for modules
that come with Guile itself, or more generally in the directories of
%load-path after installation.  The directory structure mirrors the
module namespaces.  The module (foo bar baz) is found in
/foo/bar/baz.scm, where  is a component of %load-path.  For
example, (system syntax) is in /system/syntax.scm.  In the Guile
source tree, it's in module/system/syntax.scm.

  Mark

Re: conditional code segments

[Mark H Weaver]

Hi Matt,

Matt Wette  writes:

> In C I can use `#ifdef' .. `#endif' to "comment out" code segments.
>
> In Scheme, one can use `#|' and '|#' which is OK but requires dealing with 
> both ends of the
> segment to switch on / off.  And emacs (v 24.5) scheme mode does not always 
> fontify the buffer
> correctly with #|...|#.

There's another kind of comment in standard Scheme (both R6RS and R7RS)
that's supported by Guile, but unfortunately it doesn't seem to be
documented in our manual.  Simply put the two characters "#;" before any
datum, and the entire datum will be skipped by the reader.  Whitespace
may appear between "#;" and the datum.  So, for example:

#;
(let ()
  ...)

will cause the entire 'let' form (which is a datum) to be skipped.
Emacs and paredit understand this syntax, and will act accordingly.

It works not only at top-level, but also nested arbitrarily deeply
within datums.  So, for example:

  scheme@(guile-user)> (let ()
 (display "1\n")
 (display "2\n")
 #;
 (display "3\n"))
  1
  2
  scheme@(guile-user)> 

and if you put that code in an Emacs buffer in Scheme mode, the
(display "3\n") will be colorized as a comment, but the final closing
parenthesis will have the default color.

> I tried using cond-expand but it does not work as expected:
>   scheme@(guile-user)> (cond-expand-provide (current-module) '(abc))
>   $1 = (abc)
>   scheme@(guile-user)> (cond-expand (abc #t))
>   While compiling expression:
>   Syntax error:
>   unknown file:2:0: cond-expand: unfulfilled cond-expand in form (cond-expand 
> (abc #t))

It didn't work because when a 'cond-expand' form is used in module FOO,
it looks for the features (e.g. 'abc') in all of the modules that are
imported by FOO, but not in FOO itself.

   Mark

Re: macroexpand-1

[Mark H Weaver]

Hi,

Catonano  writes:

> 2018-05-29 17:01 GMT+02:00 Mark H Weaver :
>  > what's the problem with macroexpand-1 and syntax-case ?
>
>  In Guile 1.x, 'macroexpand-1' performed a single macro expansion step at
>  the top-level form of an expression, using its old non-hygienic macro
>  expander.  There are several problems with trying to provide such an
>  interface in a Hygienic macro expander, and especially in the
>  'syntax-case' expander with its support for 'datum->syntax'.  For one
>  thing, our modern macro expander doesn't even work with the plain
>  S-expressions which 'macroexpand-1' accepted and produced.  It works
>  with "syntax objects", which effectively annotate every identifier with
>  extra information needed to determine which binding it references, and
>  also extra information needed to implement 'datum->syntax'.  This in
>  turn requires detailed knowledge of the lexical environment in which
>  expansion is taking place, whereas 'macroexpand-1' provides no way for
>  the user to provide this information.
>
> Mark
>
> I have been reading this document about the scheme higienic macros
> https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf
>
> I stopped reading it when I read that the implementation relies on a
> previously bootstrapped version of another macro expansion
> implementation.

That's not an inherent limitation of the 'syntax-case' design.  It's
merely an unfortunate attribute of the psyntax _implementation_ of
'syntax-case', apparently because they didn't care enough about
bootstrapping issues to write psyntax without the benefit of macros.

'syntax-case' could certainly be implemented without using a
pre-existing macro expander.

> But Racket has some facilities to step and debug macros, as you can
> see here https://docs.racket-lang.org/macro-debugger/index.html
>
> Aren' t Racket macros higienyc ?

Yes, of course, and we could certainly implement similar macro stepping
facilities in Guile.  But that's not what you asked about in your
previous message.  You asked about 'macroexpand-1', and my answer was
specifically about that.  I don't see any procedure similar to
'macroexpand-1' in the document you referenced above.

> In this question I've been promptly suggested a quick solution to
> perform a single macro expansion step
>
> https://stackoverflow.com/questions/50073207/macro-expansion-in-guile-scheme/50515880#50515880

For posterity, here's the quick solution suggested in the link above:

  (define-syntax (expand1 stx)
(syntax-case stx ()
  [(_expand1 form)
   (syntax-case #'form ()
 [(id . more)
  (identifier? #'id)
  (let ([transformer (syntax-local-value #'id)])  
(with-syntax ([expansion (transformer #'form)])
  #''expansion))]
 [_
  #''form])]))

This is just a toy, and not very useful in practice.
Here's the equivalent formulation for Guile:

  (use-modules (system syntax)
   (srfi srfi-11))
  
  (define (syntax-local-value id)
(let-values (((type value) (syntax-local-binding id)))
  value))
  
  (define-syntax expand1
(lambda (stx)
  (syntax-case stx ()
[(_expand1 form)
 (syntax-case #'form ()
   [(id . more)
(identifier? #'id)
(let ([transformer (syntax-local-value #'id)])
  (with-syntax ([expansion (transformer #'form)])
#''expansion))]
   [_
#''form])])))

(I usually prefer to avoid using square brackets in this way, but for
sake of comparison, I used them in the definition of 'expand1' above.)

Anyway, it works the same way as in Racket for this simple example:

  scheme@(guile-user)> (expand1 (or 1 2 3))
  $2 = (let ((t 1)) (if t t (or 2 3)))

So, what's the problem?  The first problem is that when quoting the
resulting expansion, the binding information associated with identifiers
in the syntax objects are lost, so hygiene is lost.  For example:

  scheme@(guile-user)> (expand1 (or 1 2 t))
  $3 = (let ((t 1)) (if t t (or 2 t)))

Moving on, let's use this to try to investigate how 'define-record-type'
works from SRFI-9 in Guile:

  scheme@(guile-user)> ,use (srfi srfi-9)
  scheme@(guile-user)> (expand1 (define-record-type 
  (box value)
  box?
  (value unbox set-box!)))
  $4 = (%define-record-type #f (define-record-type  (box value) box? 
(value unbox set-box!))  (box value) box? (value unbox set-box!))
  scheme@(guile-user)> (expand1 (%define-

Re: CPS mystery

[Mark H Weaver]

Hi Matt,

Matt Wette  writes:

> I'm trying to generate CPS to feed to the compiler tower.
> The following program is supposed to evaluate, expressed in Scheme, `(+ 1 2)'.
> That is, essentially,
>   scheme@(guile-user)> (apply + '(1 2))
>   $1 = 3
> I get the error message following.  Any clues what is going on?
>
> (0 . #)
> (1 . #)
> (2 . #)
> (3 . #)
> (4 . #)
> (5 . #)
> (6 . #)

The problem here is that 'resolve' returns a variable object, but 'call'
expects a procedure.  So, you need another continuation between
continuations 6 and 5 above, which does (primcall box-ref <>) to extract
the procedure from the variable object.

Also, I believe the (values 1) in continuation 2 above is incorrect.  It
expects its argument to be a list, whereas in this case it would be a
number.  It's not needed here.

> And if I fire up Guile and turn off optimization, I get this, where I don't
> even see the + resolved.
> scheme@(guile-user)> (define v (compile '(+ 1 2) #:from 'scheme #:to 'cps))
> scheme@(guile-user)> (disp-cps v)
> (0 . #)
> (1 . #)
> (2 . #)
> (3 . #)
> (4 . #)

'+' is converted into a primitive operation in an early phase of
compilation.  For these purposes, it's probably better to choose a
procedure that won't be recognized specially by the compiler.

--8<---cut here---start->8---
scheme@(guile-user)> ,use (system base compile)
scheme@(guile-user)> (define intmap->alist (@@ (language cps intmap) 
intmap->alist))
scheme@(guile-user)> (define v (compile '(floor/ 3 7) #:from 'scheme #:to 'cps))
scheme@(guile-user)> ,pp (intmap->alist v)
$2 = ((0 . #)
  (1 . #)
  (2 . #)
  (3 . #)
  (4 . #)
  (5 . #)
  (6 . #)
  (7 . #)
  (8 . #)
  (9 . #))
scheme@(guile-user)>
--8<---cut here---end--->8---

Admittedly, I manually cleaned up the output of ,pp here.  Anyway, see
the 'box-ref' expression above, which was missing from yours, and also
note that in the CPS above, 'call' returns directly to the $ktail,
without an intervening 'values'.

Hope this helps,

   Mark

Re: German translation of R5RS

[Mark H Weaver]

Hi Florian,

"pelzflorian (Florian Pelz)"  writes:
> Please accept my patches adding a German language info file for R5RS
> and fixing some small errors in English R5RS.  I believe the patches
> to be reasonably correct.

I very much appreciate this enormous amount of work that you've done!

My primary concern is that I lack the ability to review this, because I
have not (yet?) learned to read German.

A large technical specification like this is quite hard to produce
without introducing many mistakes.  Even when everyone in a standards
committee is focused on the same draft document, errors are frequently
introduced.

Errors in technical specifications can have quite bad consequences over
long time frames, because they lead to misunderstandings in the minds of
developers, and thus bugs in code, which may persist for a long time.

At present, most users who wish to read the R5RS have no choice but to
read the original document written in English.  On the one hand, this is
obviously a barrier and hardship for people who do not know English
well, but on the other hand, at least they can be assured that the
document they are reading has been widely reviewed by a large number of
people over several decades.

If we now invite them to read a German translation which has not been
reviewed at all (except for the translator), I fear the resulting
misunderstandings and buggy code that this might entail.

So, I would feel much more comfortable if someone else with good
knowledge of Scheme would step forward to carefully review this
translation.  It need not be someone from the Guile community, which
brings me to the next issue:

Although we distribute a copy of R5RS with Guile, we do not maintain
R5RS, and it is not specific to us.  To me, this situation seems
somewhat analogous to submitting a large contribution for the Sqlite3
library to the Firefox developers.  Firefox distributes a bundled copy
of Sqlite3, but they do not maintain that library.

However, I admit that this case is far more muddled, because R5RS is an
historical document, and there's no longer any group that maintains it
as far as I know.

Are there any German speakers here with good knowledge of Scheme who
would be willing to make a careful review this translation?

   Mark