Re: timestamp

[Eli Zaretskii]

> From: Ricardo Wurmus 
> Date: Sat, 12 Feb 2022 14:49:20 +0100
> Cc: guile-user@gnu.org
> 
> > The argument of lack of examples in the Guile/Guix documentation has
> > been made several times now, or at least, I've seen it being made
> > several times
> 
> The Guix documentation contains examples, but it’s already very long and
> intimidating, so we started the Cookbook for more extensive tutorials
> and examples.
> 
> It hasn’t seen as much uptake by contributors as I hoped, but it’s there
> and can be extended with more examples, big and small.

I don't think just adding examples will cut it, although it will
certainly help.

My take of the issue is that the Guile reference manual is little more
than the collection of the doc strings of the various Guile APIs.  The
doc strings are by their very nature quite terse and lack the "glue":
the text which will explain the relative merits and demerits of each
API wrt the other, the factors to consider when deciding which APIs to
use, etc.  It also doesn't make it easy to have helpful
cross-references.  Moreover, adding examples to the doc strings will
have a disadvantage: it will increase the memory footprint of running
Guile applications.

Which is why I think the effort should be in the direction of
extending the parts of the manual that aren't produced from the doc
strings, but instead are written by humans to help other humans.

Re: timestamp

[Eli Zaretskii]

> Date: Sat, 12 Feb 2022 10:01:25 +0100
> From: 
> 
>   (strftime "%Y-%m-%d %H:%m:%s" (localtime 1607841890))
>   => "2020-12-13 07:12:1607845490"

I guess you meant %H:%M:%S, not %H:%m:%s...

Re: timestamp

[Eli Zaretskii]

> From: Ricardo Wurmus 
> Date: Sat, 12 Feb 2022 12:49:10 +0100
> Cc: guile-user@gnu.org
> 
> 
> Hi adriano,
> 
> I’ve got no good answers as to “why” things are the way they are, but
> the manual explains the range of these values:
> 
> > It seesm to be
> >
> > (tm:mon %3)
> >
> > This returns
> >
> > 11
> >
> > I expected 12 but ok, I recognize this kind of weirdness
> 
>  -- Scheme Procedure: tm:mon tm
>  -- Scheme Procedure: set-tm:mon tm val
>  Month (0-11).
> 
> > I'm unhappy with 
> >
> > (tm:year $3)
> >
> > This returns
> >
> > 120
> >
> > it's 2020
> >
> > Why would 120 represent 2020 ?
> 
>  -- Scheme Procedure: tm:year tm
>  -- Scheme Procedure: set-tm:year tm val
>  Year (70-), the year minus 1900.
> 
> I don’t know why this would be useful, but that’s what it is ¯\_(ツ)_/¯

It is simply how the C function 'localtime' returns its data,
basically for historical reasons.  Guile hands it to the application
without any changes.

Re: Double-precision floating point arithmetics behaves differently in Guile and in C

[Eli Zaretskii]

> From: Panicz Maciej Godek 
> Date: Sun, 26 Sep 2021 13:55:20 +0200
> 
> I forgot to mention that I run Guile in an Emacs session running in a WSL
> console on Windows 10.
> The tests of the C code that I've been performing so far were executed in
> an MSYS terminal, but I have just tried running them in WSL console, and
> the radii get ridiculous values.
> 
> While the values used to generate the points are the following:
> 
> center '[-65.12 -50.54 88.66]:
> radii: '[83.95 47.13 45.56]
> rotation: '[[9.633871e-001   1.363818e-001   -2.308359e-001]
> [-1.734094e-001  9.735887e-001   -1.485064e-001]
> [2.044857e-001   1.830983e-001   9.615927e-001]]
> 
> and the values reconstructed in Guile are:
> 
>  (ellipsoid
>   #:center (65.10194623013226 -88.58460232582514 -50.721825868168324)
>   #:radii (83.94677717528019 45.56525864722978 47.12037877216948)
>   #:rotation ((-0.9633227088329351 0.2307406879308479 0.13699669185777974)
>(0.173638969876562 0.14674930301995573
> 0.9738142277679884)
>(-0.20459439578586436 -0.961885324244193
> 0.18143251146545056)))
> 
> (the signs and the order of radii differ, but it seems that the rotation
> matrix compensates for that)
> 
> in the case of MSYS (MinGW 64-bit) I get the following values:
> 
> approx_e.center = [90.13, -68.76, -42.46]
> approx_e.radius = [178.83, 97.08, 100.43]
> approx_e.rotation(3x3) =
> -9.634121049254586e-001 1.734020884333972e-001
>  -2.043742444879810e-001
> -2.314281253565038e-001 -1.535574849495593e-001
> 9.606566096217422e-001
> -1.351966674037161e-001 -9.728061546592428e-001
> -1.880692600613582e-001
> 
> but in the case of the WSL console, I get
> 
> approx_e.center = [90.13, -68.76, -42.46]
> approx_e.radius = [1347844355973136335704668472606720.00,
> 731678657375689861259088782950400.00, 756961648396782369967223865344000.00]
> approx_e.rotation(3x3) =
> -9.634121049254428e-01  1.734020884334219e-01
> -2.043742444880351e-01
> -2.314281253566119e-01  -1.535574849499134e-01
>  9.606566096216593e-01
> -1.351966674036449e-01  -9.728061546591822e-01
>  -1.880692600617213e-01

Forgive me for saying this, but frankly the most probable reason for
this is some bug in converting the Scheme code to C.  SVD is a
remarkably stable algorithm in the face of numerical roundoff errors,
that's one of its strong points.  So if the issue is with some minor
numerical inaccuracies, SVD results should be insensitive to it, as
long as the condition numbers of the matrices that you decompose are
not preposterously large, like 10^12 or something.  (Did you calculate
the condition numbers, btw? what are they?)

So I would check and recheck your C code, as IMO that's the first
suspect in this story.

Also, please note that on Intel CPUs double-precision FP calculations
usually keep intermediate results in 10-byte (80-bit) wide "long
double" format, unless your program is built with the GCC switch that
disables that (few programs do, so I don't expect you to do that,
eneither while building Guile, nor when compiling your C
implementation of SVD).  But that cannot by itself explain the
problem, because using 80-bit FP values should make the results more
accurate, not less accurate.

So again, I suspect your C program.  Does it work well when you throw
simple problems at it, like systems of linear equations where the
result is known in advance and the condition number of the matrix is
around 1?

HTH

Re: Guile Hacker Handbook - Character sets

[Eli Zaretskii]

> From: Jérémy Korwin-Zmijowski 
> Cc: guile-user@gnu.org
> Date: Fri, 19 Feb 2021 11:20:54 +0100
> 
> Le vendredi 19 février 2021 à 10:10 +0200, Eli Zaretskii a écrit :
> > Thanks for your work, but would it make sense to coordinate this with
> > the Emacs project, so that, for example, these packages could be
> > available from the GNU ELPA site, if not in the Emacs tarball?
> 
> Hello Eli,
> 
> It was my intention. ;-)

Great, thanks.  Looking forward for hearing from you on the Emacs
mailing lists.

Re: Guile Hacker Handbook - Character sets

[Eli Zaretskii]

> From: Ricardo Wurmus 
> Date: Thu, 18 Feb 2021 22:54:36 +0100
> Cc: guile-user@gnu.org
> 
> Jérémy Korwin-Zmijowski  writes:
> 
> > For those who missed it, I am also starting a work towards a guile-ide.el 
> > for Emacs.
> >
> > https://framagit.org/Jeko/guile-ide
> 
> Forgot to include a screenshot of Guile Studio:
> 
>https://elephly.net/downies/guile-studio.png

Thanks for your work, but would it make sense to coordinate this with
the Emacs project, so that, for example, these packages could be
available from the GNU ELPA site, if not in the Emacs tarball?

Re: No Guile on Windows? (was: My Guile Hacker Handbook)

[Eli Zaretskii]

> From: Christopher Lam 
> Date: Sat, 25 Jul 2020 13:56:28 +
> Cc: guile-user , help-g...@gnu.org
> 
> Gnucash 4.0 in windows is successfully using libguile-2.2-1.dll from MSYS2.

Is Gnucash a MinGW build or an MSYS2 build?  If the latter, it's
expected.

Re: No Guile on Windows? (was: My Guile Hacker Handbook)

[Eli Zaretskii]

> Date: Sat, 25 Jul 2020 00:48:35 -0300
> From: David Pirotte 
> Cc: Dmitry Alexandrov , guile-user@gnu.org, help-g...@gnu.org
> 
> fwiw, i've used msys2 (not so much anymore, but i still would if i had
> to ...), easy to install, update, well maintained, very friendly on irc
> when i needed to ask for help ...:
> 
>   https://www.msys2.org/
> 
> it has guile-2.2.7-1, threaded (which I've used. it works ...)

AFAIU, that's not a native Windows port, that's an MSYS2 port, which
is almost the same as a Cygwin build (MSYS2 is a fork of Cygwin).

IOW, you cannot link the MSYS2 libguile with a native MinGW program,
such as GDB or Gnu Make (or Lilypond, or any other application
mentioned in this thread).  Right?

Re: No Guile on Windows? (was: My Guile Hacker Handbook)

[Eli Zaretskii]

> From: Dmitry Alexandrov 
> Date: Fri, 24 Jul 2020 03:06:53 +0300
> Cc: Guile User , Guix Help 
> 
> > ## Installation
> 
> > ### On Windows
> >
> > No solution yet.
> 
> Is that true?  Itʼs true (and a pity) that there no official packages, of 
> course, but ‘no solution’?
> 
> I vaguely recall, there was a Guile on MinGW a couple of years ago, does it 
> no longer build?  And what happened with Cygwin package after all?  
> cygwin.com says, that 3.0.4 is there [1].

  
https://sourceforge.net/projects/ezwinports/files/guile-2.0.11-2-w32-bin.zip/download

Mind you, this is a 32-bit build, and it is configured without threads
(because building with threads produces Guile that crashes for almost
any non-trivial operation, you can find the details in the Guile
mailing list archives).

I never tried to build a newer Guile, as doing a fully-functional
MinGW port is a non-trivial endeavor that consumes a lot of time, and
I don't have that time now.


Sadly, Guile seems to care only about one OS: GNU/Linux, and more or
less disregard the rest.  Features are added that clearly cannot
easily work on other OSes, let alone non-Posix ones, and with each
such new feature producing a working MinGW port becomes harder and
harder, even for experienced hackers.

Please don't bother replying to this rant, it's just FYI, to convey my
personal recollections and experiences from doing the 2.0.x port, and
I don't intend to argue about it.

Re: Guile Studio's goals (and home)

[Eli Zaretskii]

> Date: Tue, 03 Mar 2020 13:31:38 -0500
> From: sirgazil 
> Cc: "Eli Zaretskii" , "Ricardo Wurmus" ,
>   "guile-user" 
> 
>  > But I guess it wouldn't hurt to try. If something good comes out of it 
> (e.g. a default guile-ide), maybe Guile Studio could also benefit from it (if 
> there is still a need for a Guile Studio).
>  > 
>  > Maybe I'll send something...
> 
> I reported a feature request for a "guile-mode": 
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=39888

Thanks.

Re: Guile Studio's goals (and home)

[Eli Zaretskii]

> From: Arne Babenhauserheide 
> Date: Sat, 29 Feb 2020 22:15:05 +0100
> 
> TLDR: It would be nice if Emacs could at startup offer users to select a
> customization for a specific use-case.

Please report this using "M-x report-emacs-bug", so that an issue is
open with the Emacs issue tracker.

Thanks.

Re: Guile Studio's goals (and home)

[Eli Zaretskii]

> Date: Fri, 28 Feb 2020 11:02:00 -0500
> From: sirgazil 
> Cc: "Ricardo Wurmus" , "guile-user" 
> 
>  > My point is that a need for extensive customizations might mean that
>  > some more general issue exists that the Emacs developers may need to
>  > address, either by default or as customizable options that aggregate
>  > several related options (thus freeing the users from the need to
>  > customize each one separately).  Describing those issues could
>  > therefore be beneficial both for Emacs and for Guile.  So I still
>  > think a detailed bug report could be a good idea, if only to have the
>  > issues recorded in the Emacs issue tracker.
> 
> Personally, I avoid sending these kinds of issue reports to Emacs because I 
> think to myself, "Emacs is GNU, Guile is GNU; the IDE functionality I want is 
> common across IDEs; Emacs is very old; if this functionality is not in Emacs 
> it's because they don't use it, or don't want it." 

Emacs is very old, but new packages and features are being added to it
all the time.  Just look at the NEWS file of any Emacs release, and
you will see that clearly.

> Maybe I'll send something...

Please do, and TIA.

Re: Guile Studio's goals (and home)

[Eli Zaretskii]

> From: Ricardo Wurmus 
> Cc: sirga...@zoho.com, guile-user@gnu.org
> Date: Tue, 25 Feb 2020 13:47:36 +0100
> 
> > As an Emacs co-maintainer, I was quite surprised to read the above,
> > since AFAIK none of these issues were ever communicated to the Emacs
> > developers.  If they were reported (using the Emacs built-in
> > bug-reporting command), I'm quite sure we would be very attentive to
> > such reports and amenable to adding/tweaking features so as to make
> > Emacs a better basis for Guile use and development.  After all, GNU
> > projects should help each other.
> >
> > So I urge you to report the problems on which you hint above, and
> > suggest changes or improvements that would remove at least some of the
> > need to tweak Emacs for being a "Guile Studio".
> 
> These are not necessarily problems with Emacs.  They are just annoyances
> that result from a mismatch of expectations. [...]
> 
> I don’t feel strongly enough about these things that I would push to
> change the defaults in Emacs.  But as an experienced Emacs user I feel
> confident enough to judge what Emacs features contribute to confusion,
> and to configure them in a way that mitigates or avoids the confusion.
> 
> The goal here is to point new Guilers to an IDE they can use without
> further configuration instead of telling them to install Emacs, install
> Geiser, install Company mode, install Flycheck and configure a Guile
> syntax checker, etc.  Guile Studio accomplishes this goal by configuring
> Emacs — not by forking Emacs.
> 
> I don’t think Emacs should have its defaults changed to be the best
> Guile editor out of the box.  It’s already the best editor (and more),
> in my opinion, but it may need extensive configuration to get reach its
> potential in different domains.

A "problem" doesn't necessarily have to be a bug or a deficiency, it
could also be a missing feature.  In this case, a missing feature
could perhaps be described as a lack of guile-ide.el package in Emacs,
which users of the Guile Studio could simply load, and magically have
their confusion taken care of.  Or maybe Emacs could have a
simple-ide.el package which would target more than just Guile users
(assuming at least part of the customizations you think are needed are
not specific to Guile).  Or it could be something else; or a
combination of several features separately selectable by users.

My point is that a need for extensive customizations might mean that
some more general issue exists that the Emacs developers may need to
address, either by default or as customizable options that aggregate
several related options (thus freeing the users from the need to
customize each one separately).  Describing those issues could
therefore be beneficial both for Emacs and for Guile.  So I still
think a detailed bug report could be a good idea, if only to have the
issues recorded in the Emacs issue tracker.

TIA

Re: Guile Studio's goals (and home)

[Eli Zaretskii]

> From: Ricardo Wurmus 
> Date: Tue, 25 Feb 2020 10:12:31 +0100
> Cc: guile-user@gnu.org
> 
> The pretentiously named “Guile Studio” arose from the observation that
> we often tell new Guile users to learn how to use Emacs first in order
> to get a comfortable Guile development experience.  Since Emacs has
> really quirky defaults that don’t mesh with the expectations of people
> who learn a new programming language, this can be so discouraging
> that the person abandons the initial goal of learning Guile.
> 
> Many people in the past had the idea to “fix” Emacs.  “Guile Studio” is
> not intended to be yet another Emacs starter kit like Prelude, Doom, or
> Spacemacs.  Instead it tries to focus on just Guile, going as far as to
> remove “Emacs” from the window title and the menus.
> 
> My goal was to provide a friendly editor that does not require any
> configuration to use and play with Guile.  You install Guile Studio and
> get started right away.  It tries to be what Dr Racket is for Racket and
> what RStudio is for R.
> 
> This is why Guile Studio comes with the picture language and immediately
> spawns a Geiser session where it can be used.  It hides Emacs clutter
> from the menus and adds menu items that are relevant to new Guile users,
> such as a link to the Guile manual.  It aims to handle documentation
> buffers specially to avoid the confusion that comes with Emacs-typical
> “buffer clutter”.  It uses CUA key bindings to avoid annoying surprises.

As an Emacs co-maintainer, I was quite surprised to read the above,
since AFAIK none of these issues were ever communicated to the Emacs
developers.  If they were reported (using the Emacs built-in
bug-reporting command), I'm quite sure we would be very attentive to
such reports and amenable to adding/tweaking features so as to make
Emacs a better basis for Guile use and development.  After all, GNU
projects should help each other.

So I urge you to report the problems on which you hint above, and
suggest changes or improvements that would remove at least some of the
need to tweak Emacs for being a "Guile Studio".

TIA

Re: scheme with vim

[Eli Zaretskii]

> From: zimoun 
> Date: Wed, 5 Feb 2020 16:39:00 +0100
> Cc: Julien Lepiller , guile-user@gnu.org
> 
> > > Is 'ctags -L' a valid option?
> > > Because I do not find the documentation about it.
> >
> > --help of Exuberant Ctags 5.9~svn20110310 say:
> >
> >   -L 
> >A list of source file names are read from the specified file.
> 
> Héhé. This option is not in my version:
> 
> ctags --version
> ctags (GNU Emacs 26.3)
> Copyright (C) 2019 Free Software Foundation, Inc.
> This program is distributed under the terms in ETAGS.README

You can instead redirect the file from stdin:

  ctags -

will read the list of files from stdin, one file name per line of
input.  See "ctags --help".

Re: Stepping back up as a co-maintainer

[Eli Zaretskii]

> Date: Thu, 17 Oct 2019 12:33:33 +0200
> From: 
> Cc: zx spectrumgomas , guile-user 
> ,
>  guile-devel 
> 
> 
> On Thu, Oct 17, 2019 at 12:24:03PM +0200, Mikael Djurfeldt wrote:
> > I think we should trust what Mark says and not second guess him.
> 
> Definitely. I think this should count for *all*. Second-guessing anyone
> in this difficult situation can only add more difficulties on top.

Agreed.

Re: string-ports issue on Windows

[Eli Zaretskii]

> From: Christopher Lam 
> Date: Tue, 16 Apr 2019 04:13:14 +
> 
> I'm struggling with string-ports on Windows.

Which version of Guile are you using, and where/how did you obtain the
Windows binary?

> Last para of
> https://www.gnu.org/software/guile/manual/html_node/String-Ports.html
> "With string ports, the port-encoding is treated differently than other
> types of ports. When string ports are created, they do not inherit a
> character encoding from the current locale. They are given a default locale
> that allows them to handle all valid string characters."
> 
> This causes a string-sanitize function to not run correctly in Windows.
> (locale-encoding) says "CP1252" no matter what LANG or setlocale I try.
> 
> The use case is to sanitize string for html, but on Windows it munges
> extended-unicode.

Can you show the code that doesn't work for you as you expected?  At
least the example in "String Ports" seems to work for me as I'd
expect, with Guile 2.0.11.

Re: c99 support

[Eli Zaretskii]

> From: Andy Wingo 
> Cc: guile-us...@gnu.org,  guile-de...@gnu.org
> Date: Sat, 23 Jun 2018 22:07:24 +0200
> 
> > MS-Windows (MinGW) doesn't have a C99 compliant C library, although
> > quite a few of what's needed is present.
> 
> Hard to say :)  I think my questions are limited to, in decreasing order
> of importance:
> 
>   * Is there any system that we target that doesn't have C99 stdint.h
> and stddef.h ?
> 
>   * Is there any system that we target that doesn't support C99 inline
> functions?
> 
>   * C99 mixed decls and statements?
> 
>   * C99 one-line comments (// foo) ?
> 
>   * C99 compound literals? ((struct x) { 1, 2 }) ?
> 
>   * stdbool.h

All of the above is available with reasonably recent versions of MinGW.

> I assume MinGW uses GCC.  What version?  I see that the version 6 series
> is available on mingw.org.

Current versions are 6 and 7, I have 5.3 installed on one of my
machines.  All of them support what you mentioned.

Re: How to notice abrupt tcp connection losses in server/client?

[Eli Zaretskii]

> Date: Fri, 22 Jun 2018 00:09:16 +0100
> From: Chris Vine 
> 
> I think you will need to use recv! with windows, because you cannot
> read from sockets in windows using POSIX read().

What makes you say that?  It isn't true; Gawk does that on Windows,
and it works very well.  The only two issues, which are easy to
overcome with wrappers or macros, are:

  . you need to create the socket with 'WSASocket; rather than
'socket', since the latter creates overlapped sockets that cannot
be used with file I/O APIs;
  . you need to convert the SOCKET type (which is a handle in
disguise) into a file descriptor and back using a pair of library
functions

Perhaps Guile doesn't yet do that (I didn't look, it could be in
Gnulib functions), but it would be easy to add if so.

Re: File traversal on FAT32 or NTFS

[Eli Zaretskii]

> From: Richard Shann 
> Cc: guile-user@gnu.org
> Date: Fri, 08 Jun 2018 12:27:37 +0100
> 
> > > Should I expect the procedure ftw to work on Microsoft file
> > > systems?
> > > A call that works on Unix seems to hang on M/S
> > 
> > Up front, I cannot see why should it not work.  Can you show an
> > example that hangs?
> Thanks for the reassurance - I just wanted to check that I wasn't
> wasting my time before starting to dig into it - my example is very
> complex - but I've now run it under wine and I see it's not hanging but
> complaining about
> (#f Unbound variable: ~S (getuid) #f)
> (this is Guile 1.8). As long as those Microsoft file systems are, in
> principle, supported, I'll carry on digging.

If that's Guile 1.8, get version 2.x.  There were lots of fixes in
2.0.x specifically for MS-Windows, so I wouldn't expect 1.8 to work
well on Windows.

FWIW, getuid works fine here in Guile 2.0.11 compiled for Windows.

> > > And, for good measure, what is the "command processor" that will be
> > > invoked by the system and system* procedures?
> > 
> > The same one invoked by the C library function 'system', i.e. cmd.exe
> > by default.
> 
> Ok, well thinking about it I can use system* which then by passes that,
> I don't use any globbing.

I think system and system* both had "issues" before Guile 2.0.12, so
once again, I suggest to upgrade.

Re: File traversal on FAT32 or NTFS

[Eli Zaretskii]

> From: Richard Shann 
> Date: Thu, 07 Jun 2018 14:22:12 +0100
> 
> Should I expect the procedure ftw to work on Microsoft file systems?
> A call that works on Unix seems to hang on M/S

Up front, I cannot see why should it not work.  Can you show an
example that hangs?

> And, for good measure, what is the "command processor" that will be
> invoked by the system and system* procedures?

The same one invoked by the C library function 'system', i.e. cmd.exe
by default.

Re: Building Guile from git sources stuck during make

[Eli Zaretskii]

> From: Jeremy Korwin-Zmijowski 
> Date: Sat, 24 Mar 2018 12:21:11 +0100
> 
> I am trying to build Guile from git sources (master branch) but when I
> issue make command, the process is stuck here :
> 
>   GEN  guile-procedures.texi
> make[3]: Leaving directory `/home/jeko/Builds/guile/libguile'
> make[2]: Leaving directory `/home/jeko/Builds/guile/libguile'
> Making all in bootstrap
> make[2]: Entering directory `/home/jeko/Builds/guile/bootstrap'
>   BOOTSTRAP GUILEC ice-9/eval.go
> wrote `ice-9/eval.go'
>   BOOTSTRAP GUILEC ice-9/psyntax-pp.go

Are you sure it's stuck?  How long did you wait?  The initial
compilation of Guile files takes a long time, so you should be
patient.

Re: libguile-xxx.gdb.scm is not an ELF file

[Eli Zaretskii]

> From: David Kastrup 
> Date: Sat, 20 Jan 2018 20:14:25 +0100
> 
> > Every time I install something I get the following message:
> >
> >     /sbin/ldconfig.real: /usr/local/lib/libguile-2.2.so.1.3.0-gdb.scm
> > is not an ELF file
> 
> Why would that not be in /usr/local/share instead?

Because GDB first looks for such scripts in the same directory where
the corresponding object file lives, I guess.

Re: guix is the guildhall that we always wanted!

[Eli Zaretskii]

> From: l...@gnu.org (Ludovic Courtès)
> Cc: wi...@pobox.com,  guix-de...@gnu.org,  guile-user@gnu.org
> Date: Sun, 19 Mar 2017 16:57:22 +0100
> 
> > I'm saying that when cross-compiling, it's easy to produce binaries
> > that are unusable on Windows because Guile is not run natively during
> > the build, and so problems inherent to the Windows port are not
> > revealed during the build.
> 
> Right; that would be a bug in Guile’s build system.  Is pthread support
> detection broken when cross-compiling to MinGW?

Not sure what you are asking here.  I never cross-built Guile, I
always build the MinGW port natively.  And pthreads detection is not
broken, the build process correctly detects that I have pthreads
installed.  But if I want to produce a working Guile (or even get the
build to run to completion, since that involves running Guile to
compile Scheme files), I need to disable threads.

Re: guix is the guildhall that we always wanted!

[Eli Zaretskii]

> From: l...@gnu.org (Ludovic Courtès)
> Cc: wi...@pobox.com,  guix-de...@gnu.org,  guile-user@gnu.org
> Date: Sat, 18 Mar 2017 15:04:52 +0100
> 
> I think Andy was referring to the possibility of cross-compiling
> packages that use Guile to MinGW.  That is already possible thanks to
> the work of Jan Jan Nieuwenhuizen, and since yesterday, we can create
> “packs” that contain binaries cross-compiled for MinGW:
> 
>   https://lists.gnu.org/archive/html/guix-devel/2017-03/msg00525.html

I'm saying that when cross-compiling, it's easy to produce binaries
that are unusable on Windows because Guile is not run natively during
the build, and so problems inherent to the Windows port are not
revealed during the build.  Multithreading is one such issue, quite a
significant one.  Unless I disable multithreading, I cannot build
Guile natively, because every non-trivial Guile program crashes (I
reported that years ago, but no one was able to explain why that
happens or point me to a workaround).

So if the above binaries include threads, I'm afraid they might not be
very useful on Windows.

Re: guix is the guildhall that we always wanted!

[Eli Zaretskii]

> From: l...@gnu.org (Ludovic Courtès)
> Date: Fri, 17 Mar 2017 12:30:57 +0100
> Cc: guix-de...@gnu.org, guile-user@gnu.org
> 
> > Open questions would be, what about other targets like macOS or Windows
> > or whatever?  There I don't know.  I suspect that if Guix becomes
> > popular enough, someone will make a thing whereby you can cross-compile
> > to macOS from GNU/Linux at least.  I know you can do that already with
> > MinGW.  As for native "guildhall" operation on these targets, I don't
> > know.  But I think that is a cost that people will pay for in time.
> 
> We could surely create cross-compiled packs for MinGW, though I suspect
> a number of packages do not support cross-compilation.

The MinGW build doesn't support multithreading in Guile, so any such
cross-compiled packages should be aware of that.  Also, there are some
Posix functionality not supported, like primitive-fork, so if Guix
uses that, those parts won't work.

Re: Guile foreign object interface

[Eli Zaretskii]

> From: Andy Wingo 
> Cc: l...@gnu.org (Ludovic Courtès),  guile-user@gnu.org
> Date: Thu, 09 Mar 2017 20:56:09 +0100
> 
> > That's what I'm trying to tell you: there's no aggression.
> 
> I understand that different people can have different reactions and it's
> great that you can look through "style" to the substance.  I and a
> number of other contributors (evidently including Ludovic) find it hard
> to do so, and the only reason we try is because we care about Guile.
> It's really weird though to try to ignore this "style" when the style
> often says precisely that we _don't_ care, in those words, and other
> times in as many words.

Given the certain amount of frustration over the real problems that
didn't get solved until now, you can understand that, don't you?

I had my share of problems reported here, mostly with working patches,
some of which took many moons, sometimes years to get solved upstream.
So I think I understand some of David's frustration, although the
problems I reported were nowhere near the gravity of those Lilypond
has.

So maybe the project leadership should try to improve the efficiency
of handling bug reports and of catering to the problems raised by
projects using Guile, as a means to lower frustration and make the
environment friendlier?

Re: Guile foreign object interface

[Eli Zaretskii]

> From: l...@gnu.org (Ludovic Courtès)
> Cc: guile-user@gnu.org
> Date: Thu, 09 Mar 2017 18:26:09 +0100
> 
> > FYI, I've communicated (and occasionally disagreed) with David for
> > many years, and I can assure you that you see something that simply
> > isn't there.  He sometimes uses such "colorful" descriptions to make a
> > point more clear, that's all.  People should be allowed to use their
> > personal style when writing, without being reprimanded, IMO.
> 
> I’m all for personal style, but I’m against passive-aggressive or downright
> aggressive style.

That's what I'm trying to tell you: there's no aggression.  I once
thought like you, but experience of communicating with David taught me
I was wrong.

> The problem is not whether the person who writes is well-meaning or not;
> the problem is how others perceive it.

I'm saying that you are reading into David's words something that
isn't there, not only in meaning, but also not in form.  You are
misinterpreting his words, which might not be surprising, given that
we all are trying to communicate in a language that is not our first
one.  Things might sound aggressive when they really aren't.

> If I, as an old-timer and maintainer, feel attacked when reading
> these messages

I don't see why you should feel attacked.  FWIW, I see no attack in
David's messages.  He's opinionated, that's for sure.  But there's a
difference between that and an attack.

> We’d be doing a disservice to our group by sending the message that it’s
> OK to be harsh to others.

Indeed, we would.  But there's nothing particularly harsh in David's
messages.  I think it would make this list friendlier if people's
words are not taken as an attack just because they disagree with the
project leaders.  We should be able to disagree and still stay
colleagues in our common quest.

Re: Guile foreign object interface

[Eli Zaretskii]

> From: l...@gnu.org (Ludovic Courtès)
> Date: Thu, 09 Mar 2017 13:09:40 +0100
> 
> >> As an aside, please keep the tone friendly as is the norm on this
> >> mailing list.
> >
> > Disagreement is not the same as unfriendliness.
> 
> I agree.  However I found the tone of your messages patronizing and
> aggressive, assuming bad faith and incompetence on the side of the Guile
> developers (“planning for wreckage”, “pretty pointless”, etc. etc.)

FYI, I've communicated (and occasionally disagreed) with David for
many years, and I can assure you that you see something that simply
isn't there.  He sometimes uses such "colorful" descriptions to make a
point more clear, that's all.  People should be allowed to use their
personal style when writing, without being reprimanded, IMO.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Mon, 27 Feb 2017 20:24:19 + (GMT)
> From: Jan Wedekind <j...@wedesoft.de>
> cc: Eli Zaretskii <e...@gnu.org>, guile-user@gnu.org
> 
> The encoding support of the Ruby programming language [1] is IMHO pretty 
> good. It can handle different encodings for source code, input/output, 
> string variables, and regular expressions. UTF-8 is the preferred encoding 
> but other encodings are required. E.g. Ruby is used a lot in Japan and 
> there are many "Kanji" which are currently not covered by UTF-8.

Emacs solves the latter problem as well, by using codepoints beyond
the end of the Unicode range.  (Don't forget that the Emacs m17n
features were designed and implemented by people who came from Japan.)
The advantage of the Emacs solution is that the internal
representation is still (a superset of) UTF-8, even though the byte
sequences for these codepoints could be longer than the maximum of the
standard UTF-8.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Andy Wingo <wi...@pobox.com>
> Cc: Chris Vine <ch...@cvine.freeserve.co.uk>,  guile-user@gnu.org
> Date: Sun, 26 Feb 2017 21:58:00 +0100
> 
> On Wed 15 Feb 2017 18:07, Eli Zaretskii <e...@gnu.org> writes:
> 
> > the [Emacs] MS-Windows port pretends towards Emacs internals that file
> > names are encoded in UTF-8, and shadows relevant system APIs that
> > accept or return file names, like fopen, opendir/readdir, stat,
> > etc. with its own versions that convert UTF-8 to and from UTF-16
> > before calling the real OS APIs.
> >
> > Once again, just use that experience, and maybe even some
> > infrastructure code.
> 
> FWIW we are up for good suggestions.  It's clear that file names (and
> command line arguments and environment variables) aren't handled ideally
> in Guile as they aren't fundamentally strings of characters in any
> particular encoding, and hence this class of bug.

Let me know what kind of suggestions would help.  E.g., if you need a
more detailed descriptions of how Emacs goes about these issues, I can
do that (guile-devel is probably a better place for that).

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Cc: d...@gnu.org,  guile-user@gnu.org
> Date: Fri, 17 Feb 2017 10:46:32 +0200
> 
> > IMO, it makes no sense to limit this to file names, because (a) you
> > don't always know on all levels of the code which string is a file
> > name or a part thereof; and (b) because situations where non-ASCII
> > bytes cannot be properly decoded into Unicode happen with text that is
> > not file names, and users still expect Emacs to silently produce the
> > same byte stream on round-trip operations, e.g., when copying text
> > from one file to another.
> 
> Python just barfs:
> 
>$ python3 -c "import sys; print(sys.stdin.read(30))" <<<$'\xdd'
>Traceback (most recent call last):
>  File "", line 1, in 
>  File "/usr/lib64/python3.5/codecs.py", line 321, in decode
>(result, consumed) = self._buffer_decode(data, self.errors, final)
>UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdd in position \
>0: invalid continuation byte

Which is a bad idea, IME: users won't appreciate their program to barf
when all they want is to copy a chunk of text from one place to
another, without changing anything in it.

> The situation is a bit difficult to recover from.

If you assume valid UTF-8 everywhere, certainly.  The world is more
complex than that.  Emacs is known to be used for, e.g., searching
binary executable files for text patterns; if it required the user to
say in advance that the file was binary, so that Emacs could handle it
as a byte array, that would be a major annoyance, and worse: it would
prevent the users from searching valid non-ASCII text in such a binary
file.  So Emacs allows treating binary files as text files with a
certain encoding that have some raw bytes which don't fit that
encoding.  IMO, Guile will do its users a service if it provides
similar features, because applications with similar needs are entirely
reasonable in today's world.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: Eli Zaretskii <e...@gnu.org>,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 23:13:35 +0200
> 
> Python uses the surrogate hole in the middle of the Unicode range to
> represent such stray bytes, but only when naming files.

IMO, it makes no sense to limit this to file names, because (a) you
don't always know on all levels of the code which string is a file
name or a part thereof; and (b) because situations where non-ASCII
bytes cannot be properly decoded into Unicode happen with text that is
not file names, and users still expect Emacs to silently produce the
same byte stream on round-trip operations, e.g., when copying text
from one file to another.

> Internally, CPython (the principal implementation) has Latin-1, UCS-2
> and UCS-4 strings to optimize memory use while maintaining fixed-width
> character representation.

Emacs uses a superset of UTF-8 internally.  We have found that the
variable-length encoding doesn't slow down Emacs enough to worry
about, because the need to go back in a string or buffer text is rare.
It wasn't worth the complication of maintaining different
representations, with the corresponding risk of bugs (because it is
very easy in Emacs to gain access to the internal representation of
text).

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: David Kastrup <d...@gnu.org>
> Cc: Marko Rauhamaa <ma...@pacujo.net>,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 21:52:48 +0100
> 
> Eli Zaretskii <e...@gnu.org> writes:
> 
> > Yes, to be viable in real-life situation, Guile needs to support
> > character strings with occasional embedded raw bytes that cannot be
> > interpreted as characters.
> 
> They can be interpreted as "characters", just not inside the _Unicode_
> character range.

Yes.  Emacs considers them to belong to some imaginary character set,
called "eight-bit".  My point is, those are not human-readable
characters.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Cc: d...@gnu.org,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 21:35:12 +0200
> 
> >> If emacs managed to restore a binary/text unification (and infect Guile
> >> in the process), that would be quite an accomplishment.
> >
> > I don't understand what "binary/text unification" means, sorry.
> 
> I say filenames are byte strings. Guile says they are character strings.
> You are saying they are both at once.

Yes, to be viable in real-life situation, Guile needs to support
character strings with occasional embedded raw bytes that cannot be
interpreted as characters.  Which means string implementation needs to
have a special representation for these raw bytes that would allow
lossless round-trip, and at the same time avoid the pitfalls some of
which were mentioned here.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: d...@gnu.org,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 20:38:31 +0200
> 
> Eli Zaretskii <e...@gnu.org>:
> 
> > In any case, this is unrelated to how strings are implemented, because
> > the basic level of string implementation _must_ support binary,
> > character by character (and byte by byte) comparison. Otherwise, you
> > won't be able to compare file names equal, for example, at least on
> > Unix and Windows (macOS is another matter).
> 
> Your statement is true only if you want to use character strings when
> interfacing the operating system.

Why, because I mentioned comparison of file names?  That's just one
example that came to my mind within 5 sec of thought; there are many
others.

My point is that there's place for both types of string comparisons,
and therefore both should be available.  Which means the lowest level
of string implementation should not automatically normalize strings.
(You could also have a separate string variant where normalization
happens automatically, but that has a disadvantage that you need to
decide which variant you want in advance, where you don't necessarily
know enough yet.)

> You could leave character strings to application libraries for
> newsreaders, IRC clients etc, and have a separate byte string data
> type for the system interface.

I don't know what you mean by "application libraries", but if that's
something applications should provide, and Guile shouldn't, then I
disagree: application writers will generally not know enough to
implement this non-trivial functionality.

> If emacs managed to restore a binary/text unification (and infect Guile
> in the process), that would be quite an accomplishment.

I don't understand what "binary/text unification" means, sorry.

> Now, if we only could get rid of locales while we are at it...

Dream on.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: d...@gnu.org,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 18:38:48 +0200
> 
> Eli Zaretskii <e...@gnu.org>:
> 
> > Why is that a problem?  Unicode generally mandates that equivalent
> > character (a.k.a. "codepoint") sequences shall be handled the same by
> > applications, both while processing the text (e.g., searching it etc.)
> > and when displaying it.
> 
> As I just said in another reply, emacs 25.1 isn't handling them the same
> even though it maybe should.

Yes, it does -- where that's TRT.  For example, when displaying them.
And sometimes this is a user option; e.g., see character-folding in
Isearch.

> Now, should Guile handle them the same?

IMO, Guile should provide the facilities to handle them the same, and
leave for the higher-level code to use whatever is suitable.

In any case, this is unrelated to how strings are implemented, because
the basic level of string implementation _must_ support binary,
character by character (and byte by byte) comparison.  Otherwise, you
won't be able to compare file names equal, for example, at least on
Unix and Windows (macOS is another matter).

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: guile-user@gnu.org
> Date: Thu, 16 Feb 2017 18:35:48 +0200
> 
> Eli Zaretskii <e...@gnu.org>:
> 
> > You assume that Emacs concatenates strings by just splicing its bytes.
> > But that's a far cry from what Emacs does, precisely to countermand
> > such problems.
> 
> Good to hear. If Guile is to adopt a similar approach, it should pay
> attention to these details as well.

Indeed.

> > The important point for Guile is that the solution is there, in Free
> > Software, documented well enough, and people who understand the
> > implementation and can explain its subtleties are still here, ready to
> > help. All it takes is for Guile to decide it wants to implement
> > something similar.
> 
> It would be important for Guile to be a sufficient basis for emacs.

That's not my point.  My point is that the Emacs model, or some minor
variant thereof, should be a good model for Guile (or any other
environment that seeks to support complex multi-lingual applications),
_regardless_ of whether Guile will ever become the core of the Emacs
Lisp interpreter.  IOW, it's good for Guile itself.

> On the other hand, emacs' needs might be far too high for any simple
> string type. For example, Guile might treat strings as simple
> sequences of code points while emacs might impose some Unicode
> normalization requirements or vice versa.
> 
> For example, what should
> 
>(string= "Åström" "Åström")
> 
> return?
> 
> Emacs 25.1 doesn't see the strings as equal.

As it should, IMO.  Testing strings for equivalence under canonical or
compatibility decompositions is not the job of string=, it requires a
separate API.  (Emacs provides in ucs-normalize.el the functionality
required for that.)  There are situations where you want the former,
and others where you want the latter.

That's why Unicode normalization is not implemented in Emacs on the
same level as the string data type, and the application needs to
explicitly request normalization in order for it to happen.

In general, string equivalence is in many use cases an
application-level feature (think interactive text searching), and
needs to be language- and locale-sensitive to satisfy users (e.g., it
turns out users of Spanish locales don't consider "ñ" (one character),
to be equivalent to "ñ" (two characters)).

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: to...@tuxteam.de,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 09:02:09 +0200
> 
> Eli Zaretskii <e...@gnu.org>:
> 
> >> From: Marko Rauhamaa <ma...@pacujo.net>
> >> Cc: to...@tuxteam.de,  guile-user@gnu.org
> >> Date: Thu, 16 Feb 2017 08:15:57 +0200
> >> 
> >> It is possible to have illegal Unicode even in Windows filenames, ie,
> >> filenames not expressible using Guile's strings.
> >
> > Is it really possible? Can you show a code example that would create
> > such an illegal filename on Windows?
> 
> I have rely on hearsay since I don't have Windows at my disposal:
> 
>NTFS allows any sequence of 16-bit values for name encoding (file
>names, stream names, index names, etc.) except 0x.
> 
>https://en.wikipedia.org/wiki/NTFS#Internals>
> 
> Not all sequences of 16-bit values are legal UTF-16.

Of course.  But unlike on Unix, it is much harder to create such file
names, because Windows APIs won't allow that.  You probably will have
to access the directory entries on a very low level.

Anyway, this is not the issue at hand.  I only mentioned UTF-16
encoding on Windows because Tomás thought file names on Windows can be
encoded in several different encodings.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Date: Thu, 16 Feb 2017 14:14:41 +0200
> Cc: guile-user@gnu.org
> 
> (On the other side of the equation, expressing a filename in Unicode may
> not produce an unambiguous code point sequence...  http://unicode.org/faq/normalization.html>)

Why is that a problem?  Unicode generally mandates that equivalent
character (a.k.a. "codepoint") sequences shall be handled the same by
applications, both while processing the text (e.g., searching it etc.)
and when displaying it.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Cc: guile-user@gnu.org
> Date: Thu, 16 Feb 2017 09:16:21 +0200
> 
> If I understood it correctly, someone just told us emacs maps illegal
> UTF-8 to another form of illegal UTF-8 and back. That's better in that
> it's bytes to bytes (leaving Unicode out), but it's not immediately
> obvious to me why you have to transform the byte sequence at all.

Because it allows to solve all the problems you raise in the rest of
this thread.

> Look at the problem of concatenation. We could have a case where two
> illegal UTF-8 (or UTF-16) snippets are concatenated to get valid UTF-8
> (or UTF-16). That operation fails if you try to translate the snippets
> to strings before concatenation. Such concatenation operations are
> commonplace when dealing with filenames (eg, split(1)).

You assume that Emacs concatenates strings by just splicing its bytes.
But that's a far cry from what Emacs does, precisely to countermand
such problems.  I think David described enough of what's happening to
explain why Emacs is not susceptible to such failures.

These tricks, which all happen seamlessly and transparently, are
exactly why it took Emacs so long to get where it is today.  It takes
many moons to see the problems, analyze them, devise solutions that
don't break, and implement both the 90%-successful heuristics and the
opt-in solutions for the other 10%.  The important point for Guile is
that the solution is there, in Free Software, documented well enough,
and people who understand the implementation and can explain its
subtleties are still here, ready to help.  All it takes is for Guile
to decide it wants to implement something similar.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Thu, 16 Feb 2017 08:29:14 +0200
> From: Eli Zaretskii <e...@gnu.org>
> Cc: guile-user@gnu.org
> 
> > From: Marko Rauhamaa <ma...@pacujo.net>
> > Cc: to...@tuxteam.de,  guile-user@gnu.org
> > Date: Thu, 16 Feb 2017 08:15:57 +0200
> > 
> > It is possible to have illegal Unicode even in Windows filenames, ie,
> > filenames not expressible using Guile's strings.
> 
> Is it really possible?  Can you show a code example that would create
> such an illegal filename on Windows?

Btw, if by "UCS-2" you meant to say that only characters within the
BMP are supported in file names on Windows, then this is wrong: since
Windows XP, NTFS volumes support file names with characters outside of
the BMP.  I've just successfully created files with such file names on
Windows XP using Emacs.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Cc: to...@tuxteam.de,  guile-user@gnu.org
> Date: Thu, 16 Feb 2017 08:15:57 +0200
> 
> It is possible to have illegal Unicode even in Windows filenames, ie,
> filenames not expressible using Guile's strings.

Is it really possible?  Can you show a code example that would create
such an illegal filename on Windows?

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Wed, 15 Feb 2017 22:15:52 +0100
> From: to...@tuxteam.de
> Cc: guile-user@gnu.org
> 
> > > > A possible solution would be to decode each mount point's part as it
> > > > is being resolved.
> > > 
> > > ...which can only be based on guesswork: there's no reliable info on
> > > the encoding used for that file system (if it's consistent at all).
> > 
> > You could maintain a database of encodings per file system, perhaps
> > user-defined, or derived by some other means.  E.g., for volumes that
> > physically reside on Windows or macOS the encoding is pretty much
> > known in advance.
> 
> This is what I mean by "voodoo".

Such "voodoo" is what Emacs does, more or less (not in this particular
use case, though).  This is what makes it so useful and successful.
Refusing to use such techniques because they are theoretically
imperfect is an obstacle to making useful software systems that
support multi-lingual environments.

> We don't even know the encoding to be consistent whithin one file
> system.

In almost all cases, it is.  Once again, the 99% vs 1% issue.

> An example would be the home dirs of different users running under
> different locales (an extreme example: they may have different 8 bit
> locales!).

Did you ever see such a use case in practice?

Besides, my suggestion works there as well, given a large enough
database that users can augment.

> Anyway, having an encoding à la Emacs eases things a lot, since a
> string can at least survive unharmed a plain round trip.

That's a basic requirement, yes.

> The problem of properly displaying that remains unsolved.

This must be solved sufficiently in the majority of use cases; doing
that is not hard.  For the rest, there should be optional
settings/commands to get the correct display.  Example: the (now
largely unnecessary) rmail-redecode-body command in Rmail.

> Plus operations on that string (concatenation, e.g.).

No, this can be easily coded to support raw bytes.  Emacs does that.

> > No.  At the file system level (for NTFS volumes at least) Windows file
> > names are always UTF-16 encoded, and Windows just "knows" that.
> > Windows converts that to the locale's codepage when you access files
> > via an API that communicates file names encoded in that codepage.  (If
> > the conversion fails, you get question marks instead of the characters
> > that couldn't be converted.)
> 
> I see. That means that Windows has to use surrogates for everything
> beyond the BMP, right?

Yes.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: to...@tuxteam.de,  guile-user@gnu.org
> Date: Wed, 15 Feb 2017 23:04:52 +0200
> 
> Eli Zaretskii <e...@gnu.org>:
> 
> > At the file system level (for NTFS volumes at least) Windows file
> > names are always UTF-16 encoded, and Windows just "knows" that.
> 
> Hm, I had the impression NTFS filenames were UCS-2 ( https://en.wikipedia.org/wiki/Talk%3AUTF-16/UCS-2>).

What is the difference, in the context of this discussion?

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Wed, 15 Feb 2017 21:20:56 +0100
> From: to...@tuxteam.de
> Cc: guile-user@gnu.org
> 
> > > Most notably, the whole path might cross several mount points, thus
> > > the whole path can well have fragments coming from several file systems.
> > 
> > A possible solution would be to decode each mount point's part as it
> > is being resolved.
> 
> ...which can only be based on guesswork: there's no reliable info on
> the encoding used for that file system (if it's consistent at all).

You could maintain a database of encodings per file system, perhaps
user-defined, or derived by some other means.  E.g., for volumes that
physically reside on Windows or macOS the encoding is pretty much
known in advance.

> > > I think the only sane way to see a Linux file system path is the way
> > > Linux sees it: as a byte string.
> > 
> > This would lose a lot in 99% of use cases.  You are, in effect,
> > suggesting a "reverse optimization", whereby the majority of use cases
> > is punished in favor of a small minority, based on theoretical
> > intractability.
> 
> I feel queasy doing some voodoo whithout the application having
> a word on it. In the Emacs context it's a bit easier, because in
> the "normal" case things are pretty quickly deferred to the user
> (usually).

Not really, there are a lot of internal operations that access files
and directories, and would wreak major havoc if they don't succeed,
silently, in the absolute majority of uses.

> > > NT has done that too.
> > 
> > Windows can do that because it also transparently translates file
> > names to the locale's encoding when files are accessed with ANSI APIs.
> > Without such translation, this kind of decision is unwise, IMO.
> 
> I guess (I don't *know*) Windows stores information about the encoding
> at file system level (and keeps that consistent).

No.  At the file system level (for NTFS volumes at least) Windows file
names are always UTF-16 encoded, and Windows just "knows" that.
Windows converts that to the locale's codepage when you access files
via an API that communicates file names encoded in that codepage.  (If
the conversion fails, you get question marks instead of the characters
that couldn't be converted.)

> Linux hasn't that, it just keeps out of it. It hasn't even a place
> to state the encoding used.

Exactly.  Which is why forcing a single file-name encoding on
Linux/Unix filesystems is IMO a bad idea.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Wed, 15 Feb 2017 21:07:53 +0100
> From: to...@tuxteam.de
> Cc: to...@tuxteam.de, d...@gnu.org, guile-user@gnu.org
> 
> > It took many years because those smart, experienced, and patient
> > people made bad decisions, twice, and had to correct them later, which
> > required rewriting several important internal mechanisms.  Which tells
> > you that smarts, experience, and patience are not enough to get this
> > right the first time.
> 
> That's in my view part of being smart (and yes, you are one of those
> smart people I had in mind: thanks *a lot* for that!).

Actually, at the time I was a relatively passive observer.  I did very
little for the related code itself (with the exception of the Windows
handling of file names, which was done years later).  So it isn't my
credit to take.

But thanks anyway.

> And thanks for your insights.

You are welcome.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Wed, 15 Feb 2017 10:18:32 +0100
> From: 
> 
> > Filenames and locales are not necessarily related.  When you access a
> > networked file system, you get the filename encoding you are given,
> > which may or may not be the same as the particular locale encoding on
> > your particular machine on one particular day, and may or may not be a
> > unicode encoding.  Glib, for example, enables you to set this with the
> > G_FILENAME_ENCODING environmental variable [...]
> 
> which is, btw., "just a better approximation", but still wrong: the
> application creating a directory might have been "in" a different
> locale (and thus having a different encoding) that the one creating
> the file whithin that directory.
> 
> Most notably, the whole path might cross several mount points, thus
> the whole path can well have fragments coming from several file systems.

A possible solution would be to decode each mount point's part as it
is being resolved.

> I think the only sane way to see a Linux file system path is the way
> Linux sees it: as a byte string.

This would lose a lot in 99% of use cases.  You are, in effect,
suggesting a "reverse optimization", whereby the majority of use cases
is punished in favor of a small minority, based on theoretical
intractability.

> Sure, some helper infrastructure to try to make characters of that
> mess will be welcome, but that should be absolutely robust wrt.
> unexpected input e.g. bad UTF-8) and leave control to the application.

Most applications won't like this burden, because most application
programmers don't know enough about the issue to solve them correctly,
especially for users of other OSes and locales.

> > But if OpenBSD requires all _filenames_ to be in valid UTF-8, that
> > is a bad decision in my view.
> 
> NT has done that too.

Windows can do that because it also transparently translates file
names to the locale's encoding when files are accessed with ANSI APIs.
Without such translation, this kind of decision is unwise, IMO.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Mon, 30 Jan 2017 20:42:38 + (UTC)
> From: Mike Gran 
> Cc: "guile-user@gnu.org" 
> 
> Earlier in the 2.0.x release series, Guile had a hack where it started
> up in a Latin-1 encoding, which would be capable of storing any
> 8-bit string of bytes, even if they weren't Latin-1.

Latin-1 has holes in the 0..255 range, so it isn't very appropriate in
this situation.

> And what was supposed to happen after setlocale was called?

What Emacs does is explicitly decode any variable produced until that
moment that is known to hold unibyte strings.

> As an aside, GTK and GLIB based applications often use a method where
> you may need to set the environment variable G_FILENAME_ENCODING
> if your filename encoding is different from your locale encoding.
> GTK/GLIB also likes to store strings internally as UTF-8, and will
> convert to UTF-8 from either the locale or the G_FILENAME_ENCODING-
> specified encoding.

Emacs stores all environment variables in their original
locale-specific encoding, as unibyte strings, and only decodes them
when they are actually used or handed to Lisp.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> Date: Mon, 30 Jan 2017 21:32:41 +0200
> From: Eli Zaretskii <e...@gnu.org>
> Cc: guile-user@gnu.org
> 
> > Hm, I know that XEmacs-Mule emphatically does not have unibyte strings
> > (and Stephen considers them a complication and abomination that should
> > never have been left in Emacs), so it must be possible to get away
> > without them.
> 
> I doubt that's possible, at least not in general.  (You could get away
> if you assumed UTF-8 encoded file names.)  Some translation tables for
> some encodings must load files using the likes of load-path, and if
> that includes non-ASCII file names, you are screwed unless you can use
> unibyte strings.

Actually, the need arises even sooner.  Consider how load-path is set
up during startup: it starts with the directory from which Emacs was
invoked, either from argv[0] or by looking up PATH.  Either way, you
get a file name that is encoded in the locale-specific encoding.  Then
you cons load-path by expanding file names relative to the startup
directory.  So you immediately need to be able to create file names
from directories, check whether a file exists and is a directory,
etc. -- all of that before you even know in what locale you started,
so you cannot decode these file names into the internal
representation, before using them.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Date: Mon, 30 Jan 2017 21:01:31 +0200
> Cc: guile-user@gnu.org
> 
> UTF-8 beautifully bridges the interpretation gap between 8-bit character
> strings and text. However, the interpretation step should be done in the
> application and not in the programming language.

You can't do that in an environment that specifically targets
sophisticated multi-lingual text processing independent of the outside
locale.  Unless you can interpret byte sequences as characters, you
will be unable to even count characters in a range of text, let alone
render it for display.  And you cannot request applications to do
those low-level chores.

> Support libraries for Unicode are naturally welcome.

Well, in that case Emacs core is one huge "support library".  And I
don't see why Guile couldn't be another one; it should, IMO.

> Plain Unicode text is actually quite a rare programming need. It is
> woefully inadequate for the human interface, which generally requires
> numerous other typesetting effects.

You do need "other typesetting effects", naturally, but that doesn't
mean you can get away without more or less full support of Unicode
nowadays.  You are talking about programming, but we should instead
think about applications -- those of them which need to process text,
or even access files, as this discussion shows, do need decent Unicode
support.  E.g., users generally expect that decomposed and composed
character sequences behave and are treated identically, although they
are different byte-stream wise.

> But is also causing unnecessary grief in the computer-computer
> interface, where the classic textual naming and textual protocols
> are actually cutely chosen octet-aligned binary formats.

The universal acceptance of UTF-8 nowadays makes this much less of an
issue, IME.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: David Kastrup <d...@gnu.org>
> Cc: ma...@pacujo.net,  guile-user@gnu.org
> Date: Mon, 30 Jan 2017 20:00:03 +0100
> 
> Eli Zaretskii <e...@gnu.org> writes:
> 
> > One other crucial detail is that Emacs also has unibyte strings
> > (arrays of bytes), which are necessary during startup, when Emacs
> > doesn't yet know how to decode non-ASCII strings.  Without that, you
> > wouldn't be able to start Emacs in a directory whose name includes
> > non-ASCII characters, because it couldn't access files it needs to
> > read to set up some of its decoding machinery.
> 
> Hm, I know that XEmacs-Mule emphatically does not have unibyte strings
> (and Stephen considers them a complication and abomination that should
> never have been left in Emacs), so it must be possible to get away
> without them.

I doubt that's possible, at least not in general.  (You could get away
if you assumed UTF-8 encoded file names.)  Some translation tables for
some encodings must load files using the likes of load-path, and if
that includes non-ASCII file names, you are screwed unless you can use
unibyte strings.  That is why all Emacs primitives that accept file
names support both unibyte and multibyte strings as file names.

> And I don't think that the comparatively worse Mule implementation
> of XEmacs is due to that decision.

Emacs 20 vintage Mule didn't have all the sophisticated Unicode
support machinery we have today, so maybe for that subset the above
wasn't necessary.  Then again, Emacs couldn't be safely built or
started in a non-ASCII directory until just a few years ago, so
perhaps no one bothered to test that thoroughly with XEmacs, except in
ISO 2022 locales.

Re: guile can't find a chinese named file

[Eli Zaretskii]

> From: David Kastrup 
> Date: Mon, 30 Jan 2017 19:32:14 +0100
> Cc: guile-user@gnu.org
> 
> Emacs uses an UTF-8 based encoding internally: basically, valid UTF-8 is
> represented as itself, there is a number of coding points beyond the
> actual limit of UTF-8 that is used for non-Unicode character sets, and
> single bytes not properly belonging to the read encoding are represented
> with 0x00...0x7f, 0xc0 0x80 ... 0xc0 0xbf and 0xc1 0x80 ... 0xbf (the
> latter two ranges are "overlong" encodings of 0x00...0x7f and
> consequently also not valid utf-8).

One other crucial detail is that Emacs also has unibyte strings
(arrays of bytes), which are necessary during startup, when Emacs
doesn't yet know how to decode non-ASCII strings.  Without that, you
wouldn't be able to start Emacs in a directory whose name includes
non-ASCII characters, because it couldn't access files it needs to
read to set up some of its decoding machinery.

Re: Can't use dynamic-link with any shared object file

[Eli Zaretskii]

> Date: Mon, 26 Dec 2016 22:24:21 + (GMT)
> From: Jan Wedekind 
> Cc: guile-user@gnu.org
> 
> I think "libc" is normally loaded already. E.g. the C function "abs" can 
> be accessed as follows:
> 
>  (define main (dynamic-link))
>  (define cabs (pointer->procedure long (dynamic-func "abs" main) (list 
> long)))
>  (cabs -42)
>  ; 42

This is non-portable, and doesn't work outside of the Posix world.
E.g., on MS-Windows (with Guile 2.0.x):

  D:\usr\eli\data>guile
  GNU Guile 2.0.11
  Copyright (C) 1995-2014 Free Software Foundation, Inc.

  Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
  This program is free software, and you are welcome to redistribute it
  under certain conditions; type `,show c' for details.

  Enter `,help' for help.
  scheme@(guile-user)> ,use (system foreign)
  scheme@(guile-user)> ,use (rnrs bytevectors)
  scheme@(guile-user)> (define main (dynamic-link))
  scheme@(guile-user)> (define cabs (pointer->procedure long (dynamic-func 
"abs" main) (list long)))
  ERROR: In procedure dynamic-func:
  ERROR: In procedure dynamic-pointer: Symbol not found: abs

  Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
  scheme@(guile-user) [1]> ,q

But (msvcrt.dll is the C runtime library on Windows):

  scheme@(guile-user)> (define libc (dynamic-link "msvcrt.dll"))
  scheme@(guile-user)> (define cabs (pointer->procedure long (dynamic-func 
"abs" libc) (list long)))
  scheme@(guile-user)> (cabs -42)
  $1 = 42
  scheme@(guile-user)>

Re: Formatted output with locale

[Eli Zaretskii]

> From: l...@gnu.org (Ludovic Courtès)
> Date: Thu, 17 Nov 2016 12:02:06 +0100
> 
> It seems to work as advertised for me:
> 
> --8<---cut here---start->8---
> scheme@(guile-user)> ,use(ice-9 i18n)
> scheme@(guile-user)> (number->locale-string 1.01 2 (make-locale LC_ALL 
> "fr_FR.utf8"))
> $13 = "10 000,01"
> scheme@(guile-user)> (number->locale-string 1.01 2 (make-locale LC_ALL 
> "de_DE.utf8"))
> $14 = "10.000,01"
> scheme@(guile-user)> ,use(ice-9 format)
> scheme@(guile-user)> (setlocale LC_ALL "de_DE.utf8")
> $15 = "de_DE.utf8"
> scheme@(guile-user)> (format #f "~12,2h" 1.01)
> $16 = "   10.000,01"
> --8<---cut here---end--->8---
> 
> That’s on GNU/Linux (glibc 2.24).

It works for me as well, on MS-Windows (although I need to use a
non-UTF-8 codeset in the locales).

Re: Guile on MinGW

[Eli Zaretskii]

> Date: Sun, 9 Oct 2016 16:57:35 + (UTC)
> From: Mike Gran 
> 
> I guess this question is really for Eli.  I hope he's lurking.

I am.

> So, I'm trying to put together the next release for the
> Guile-ncurses bindings, and I want to drop 1.8.x compat. MinGW
> seems to be stuck on Guile 1.8.8. 
> But MinGW seems to be splitting into two camps, as far as I can
> tell: mingw.org and mingw-w64.org.
> If I wanted to test my upcoming release, which MinGW should
> I using to compile guile-2.0.x and test my new release of
> guile-ncurses?

I only use mingw.org's MinGW, and that's what I used to build Guile
2.0.x (I think I've built 3 different versions over the last few
years).  That said, I don't think the differences between the two
runtimes are so significant as to prevent you from building with one
or the other.  A perhaps more significant issue could be the Windows
64-bit build, because I'm guessing that Guile sources assume that a
'long' is as wide as a pointer, which is not true for 64-bit Windows.
So attempting a 64-bit Windows build might expose some more places
where porting and patches are needed.

You could also decide to just use my precompiled 32-bit MinGW binaries
of Guile 2.0.11, from here:

  https://sourceforge.net/projects/ezwinports/files

instead of building Guile on your own, which involves building its
dependencies first, or finding pre-build binaries of them somewhere.
(I assume that building and using guile-ncurses doesn't actually
require building Guile itself first.)

Good luck!

Re: Building guile-2.0.11 on a new cygwin 64

[Eli Zaretskii]

> Date: Sun, 29 May 2016 16:17:29 + (UTC)
> From: Mike Gran 
> 
> And I had to build guile with the "--without-threads" configure
> option.  When I use threads, it hangs in scm_i_pthread_cond_wait.
> 
> And with that, it builds and runs and passes the majority of the
> "make check" tests, except for these.
> 
> FAIL: 00-socket.test: AF_UNIX/SOCK_STREAM: accept 
> FAIL: 00-socket.test: AF_UNIX/SOCK_STREAM: accept (bis) 
> 
> FAIL: 00-socket.test: AF_INET6/SOCK_STREAM: accept 
> FAIL: i18n.test: character mapping: char-locale-upcase Turkish 
> FAIL: i18n.test: character mapping: char-locale-downcase Turkish 
> FAIL: i18n.test: string mapping: string-locale-upcase Turkish 
> FAIL: i18n.test: string mapping: string-locale-downcase Turkish 
> FAIL: i18n.test: number->locale-string: French: integer 
> FAIL: i18n.test: number->locale-string: French: fraction 
> FAIL: i18n.test: number->locale-string: French: fraction, 1 digit 
> FAIL: i18n.test: monetary-amount->locale-string: French: integer 
> FAIL: i18n.test: monetary-amount->locale-string: French: fraction 
> 
> 
> And the standalone test "test-ffi" fails with
> 
> ERROR: In procedure dynamic-func: 
> ERROR: In procedure dynamic-pointer: Symbol not found: strerror 
> 
> 
> I know this information is all too casual to be a either a bug
> report or build instructions, but, I wanted to document what I had
> before I moved on to other things.  Hopefully it'll be of use
> to someone.
> 
> Maybe I'll get back to this again soon.

I'm surprised you have almost the same problems in a Cygwin build
that I had in the MinGW build (see several discussions back in June
2013 and some more in 2014).  Cygwin is much more Posix compatible, so
at least the i18n tests should have passed.  But maybe I'm missing
something; if you look at the problems I reported back then, perhaps
you will be able to uncover the root causes for these problems.

Re: How to get the preferred environment variable path separator?

[Eli Zaretskii]

> Date: Mon, 28 Mar 2016 10:24:56 -0400
> From: "Thompson, David" 
> Cc: Guile User , Chris Marusich 
> 
> The environment variable path separator is *not* defined depending on
> the OS.  It is up to the programs that interpret these search paths to
> specify what the separator should be.  ":" is the most common
> separator, but that is just convention.  A search path is opaque to
> the operating system, where environment variables are just strings
> with no inherent meaning.

We are not talking about the OS, we are talking about the programs
that set and query environment variables using 'getenv', 'putenv', and
other similar APIs, followed by simple string processing.  And those
are definitely _not_ treating the separator as opaque, something you
can easily verify both by looking at the sources of the respective
applications, and by simple experiments.

And in that sense, the path separator character is always ':' on Posix
systems and ';' on MS-Windows.  So I think Guile ought to have such a
variable; Emacs, for one, does.

Re: filename(CJK utf8) problem on OSX.

[Eli Zaretskii]

> From: Park SungMin 
> Date: Mon, 28 Mar 2016 10:21:46 +0900
> 
> (define filename "/Users/byul/Desktop/사진.gif")
> 
> (define my-open-file
>   (lambda (filename)
> (let* ((fd ((pointer->procedure
>int
>(dynamic-func "open" (dynamic-link))
>(list '* int))
>   (bytevector->pointer
>(string->utf8 filename))
>   O_RDONLY)))
>   (when (= fd -1)
>   (throw 'can't-find-file))
>   (fdes->inport fd
> 
> (define port (my-open-file filename))
> 
> (get-bytevector-n port 10)  ;; OK. I get bytevector.
> 
> ;;but….
> (file-exists? filename) ;; #f
> (open-file filename "r") ; ERROR: In procedure open-file: No such file or 
> directory: "/Users/byul/Desktop/사진.gif"

Isn't this because OS X filesystems store file names in decomposed
form?

Re: Running script from directory with UTF-8 characters

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Date: Tue, 22 Dec 2015 23:39:28 +0200
> Cc: guile-user@gnu.org
> 
> > No, they aren't, not as file names. E.g., you cannot meaningfully
> > downcase or upcase such "characters", you cannot count characters (as
> > opposed to bytes), you cannot calculate how much screen estate will be
> > needed to display them, with some Far Eastern encodings you cannot
> > correctly search them for some specific ASCII characters (because they
> > can be part of a multibyte sequence), etc. etc. IOW, you cannot work
> > with file names as human-readable text, which is something many
> > programs need to do.
> 
> You can, in a roundabout way. You do the low-level file I/O in Latin-1.
> Then, you reencode into UTF-8

IOW, from the application-level perspective, the file names are
encoded in UTF-8 (in this example).  The low-level reading as byte
stream (NOT Latin-1!) is out of scope as long as you consider a Guile
Scheme program that needs to manipulate the file names.

So we are in violent agreement.

> Otherwise, you may not even be able to remove a file with a non-UTF-8
> name.

What do you mean by a non UTF-8 file name?  A file name that includes
byte sequences that are not valid UTF-8?  For that, Guile needs to
acquire a capability of representing raw bytes, similar to what Emacs
does.  This capability is an add-on, it should not be instead of being
able to interpret file names as character strings encoded in some
recognizable encoding, either forced by the application or deduced
from some meta-data, user preferences, locale's defaults, etc.

> > They are strings because _people_ name files and give them meaningful
> > names and extensions.
> 
> The Linux kernel just doesn't care, and shouldn't.

Guile is not an OS kernel.  Guile is an environment for writing
applications.  On the application level, you _should_ care, or else
you won't be able to manipulate file names in meaningful ways.

> It's acceptable for Guile to create a higher-level illusion, but it
> shouldn't sacrifice completeness while doing so. You should be able to
> manipulate every conceivable filename from Guile code.

We are again in violent agreement about the goal.  But the means
towards that goal is NOT to abandon interpretation of file names as
strings of characters, the means is to be able to represent raw bytes
on top of a meaningful character representation.

> > If Guile cannot easily work with file names encoded in a codeset other
> > than the current locale's one, then Guile should be extended to allow
> > a program to tell it in which encoding to interpret a particular name.
> 
> A program usually has no clue how a pathname has been encoded.

The programmer does, or should be.  The user does, sometimes (e.g.,
the capability presented in many browsers and editors to force text
encoding).  Some encodings can be deduced by analyzing the bit stream.
And there are locale defaults if nothing else works.  If none of that
is done, the program cannot manipulate these file names in any
meaningful way.  The kernel can duck that problem because it's the
kernel: it doesn't interact with users, and its filesystem layer is
not required to understand the meaning of, say, the file-name
extensions.  We have no such luxury on the application level.  So we
cannot simply copycat the kernel techniques into Guile, it won't work.
It also won't work to expect applications do that, as that is too
complex and subtle (and tedious) for application to do it right every
time.

Once again, I suggest to study how Emacs solves this very problem.
The solution used there is satisfactory, and fits all of your
requirements above.  It's not without some subtleties in rare cases,
but the problem is complex and there's no way around that complexity.

> > (I think Guile already supports that, but maybe I misremember.) But
> > lobbying for treating file names as byte streams, let alone Latin-1
> > characters, is a large step backwards, to 1990s when we didn't know
> > better. We've come a long way since then and learned a lot on the way.
> 
> At least our backwardness allowed Linux to jump directly to UTF-8 and
> not be afflicted by UCS-2 like Windows and Java.

Once again, Guile is not an OS kernel.  It cannot simply adopt kernel
solutions.

> I'm not saying bytevectors are elegant, but we should not replace them
> with wishful thinking.

No need for wishful thinking.  Study what Emacs does and do something
similar.

> I'm a bit sorry that Guile repeated Python 3's mistake and brought
> (Unicode) strings to the center.

Everybody does that mistake.  Emacs did it as well, but that was years
ago, and since then the mistakes were identified and corrected.  The
basis must be Unicode, the trick is to build additions on top of that
which allow raw bytes and Unicode text strings to coexist, more or
less transparently to the application level.  ("More or less" because
handling raw bytes as part of strings requires some care; fortunately,
such use 

Re: Running script from directory with UTF-8 characters

[Eli Zaretskii]

> From: Marko Rauhamaa <ma...@pacujo.net>
> Cc: guile-user@gnu.org
> Date: Wed, 23 Dec 2015 21:18:28 +0200
> 
> Eli Zaretskii <e...@gnu.org>:
> 
> >> From: Marko Rauhamaa <ma...@pacujo.net>
> >> The Linux kernel just doesn't care, and shouldn't.
> >
> > Guile is not an OS kernel. Guile is an environment for writing
> > applications. On the application level, you _should_ care, or else you
> > won't be able to manipulate file names in meaningful ways.
> 
> To me, a programming language is a medium of writing programs for an
> operating system. I don't think a programming language should "shield"
> me from the OS. Instead, it should make the whole gamut of the OS
> facilities available to me.

I see no contradiction here, as long as you acknowledge that Guile
should be good for more than just OS level stuff.

> >> I'm not saying bytevectors are elegant, but we should not replace
> >> them with wishful thinking.
> >
> > No need for wishful thinking. Study what Emacs does and do something
> > similar.
> 
> Why don't you tell me already what emacs does?

I did, you elided that.  It represents text as superset of UTF-8, and
uses high codepoints above the Unicode space for raw bytes.

> >> Guile 1.x's and Python 2.x's bytevector/string confusion was actually
> >> a very happy medium. Neither the OS nor the programming language
> >> placed any interpretation to the byte sequences. That was left to the
> >> application.
> >
> > And that is wrong. Applications cannot handle that, they need some
> > heavy help from the infrastructure.
> 
> That can be managed through support libraries.

Guile is one huge support library, so it should include that built-in.

Re: Running script from directory with UTF-8 characters

[Eli Zaretskii]

> From: Marko Rauhamaa 
> Date: Tue, 22 Dec 2015 22:36:07 +0200
> Cc: guile-user@gnu.org
> 
> By setting the character set artificially to Latin-1 in Guile, all
> pathnames are accessible to it.

No, they aren't, not as file names.  E.g., you cannot meaningfully
downcase or upcase such "characters", you cannot count characters (as
opposed to bytes), you cannot calculate how much screen estate will be
needed to display them, with some Far Eastern encodings you cannot
correctly search them for some specific ASCII characters (because they
can be part of a multibyte sequence), etc. etc.  IOW, you cannot work
with file names as human-readable text, which is something many
programs need to do.

File names _are_ strings, there's no way around that.  They are
strings because _people_ name files and give them meaningful names and
extensions.  If Guile cannot easily work with file names encoded in a
codeset other than the current locale's one, then Guile should be
extended to allow a program to tell it in which encoding to interpret
a particular name.  (I think Guile already supports that, but maybe I
misremember.)  But lobbying for treating file names as byte streams,
let alone Latin-1 characters, is a large step backwards, to 1990s when
we didn't know better.  We've come a long way since then and learned a
lot on the way.

Re: Any New Guile Release Planned ?

[Eli Zaretskii]

> Date: Mon, 2 Nov 2015 15:10:15 +0300
> From: Vladimir Zhbanov 
> 
> > The reason I ask is that we rely on Gnulib for these portability
> > things.  The  in libguile/iselect.h is supposed to do the
> > right thing; if it’s not, we should (1) update our Gnulib copy, and (2)
> > fix the problem in Gnulib if it’s still there.
> 
> OK, on my system (Debian jessie (stable)) guile builds well under
> MinGW.  However, when I'm trying to compile geda-gaf it complains:
> 
> In file included from
> /home/user/minipack/result/include/guile/2.0/libguile/threads.h:31:0,
>  from
> /home/user/minipack/result/include/guile/2.0/libguile/async.h:29,
>  from
> /home/user/minipack/result/include/guile/2.0/libguile.h:37,
>  from ./../include/libgeda_priv.h:4,
>  from scheme_init.c:26:
> /home/user/minipack/result/include/guile/2.0/libguile/iselect.h:31:24:
> fatal error: sys/select.h: No such file or directory
>  #include 
> ^
> compilation terminated.
> make[4]: *** [scheme_init.x] Error 1
> make[3]: *** [all-recursive] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all] Error 2
> 
> AFAICS, mingw package in Debian has the winsock2.h headers while I
> don't see any select.h

MinGW indeed doesn't have sys/select.h, but gnulib, included in Guile,
provides it.

So I think Ludovic is right, and showing the complete compiler command
line will give some hints.  I'm guessing something related to -I
switches or maybe even -isystem.

Re: Any New Guile Release Planned ?

[Eli Zaretskii]

> Date: Mon, 2 Nov 2015 15:21:00 +0300
> From: Vladimir Zhbanov 
> Cc: guile-user@gnu.org
> 
> > http://sourceforge.net/projects/ezwinports/files/guile-2.0.11-2-w32-bin.zip/download
> >
> 
> I know, thanks :)
> 
> Thank you for your great work on the guile port for Windows.
> Unfortunately your prebuilt version didn't work for me last time I
> tried it.

Strange.  It certainly works for me and others in GDB and in Make (and
of course as a stand-alone interpreter).  It also passes all the
tests.  So I suspect some snafu with the dependent DLLs or some such.

> (BTW, headers in your email are broken in my MUA, so I couldn't
> read it there, while web interface shows it well.)

That's the first time someone complains about that.  Broken how?

Re: Any New Guile Release Planned ?

[Eli Zaretskii]

> Date: Wed, 28 Oct 2015 21:15:33 +0300
> From: Vladimir Zhbanov 
> 
> The stable-2.0 guile branch contains changes that some of our users
> (that is, users of gEDA/gaf) have been waiting for years, especially
> after guile 2.0 became necessary to build our package. I mean those
> users who have been waiting for Windows support. (Some of them insist on
> moving to using Python instead of Guile these days, since they have been
> tired trying to cross-compile with guile-2.0 support.) A while ago I was
> eventually able to build our toolkit using only one patch for guile [1]
> (almost one-liner, in essence), and several work-around patches for some
> of our programs, which can be found at [2] and are actually the same
> code blocks (declaring an array and doing its initialisation). I don't
> know why the latter patches work (geda-gaf with guile 1.8 worked without
> them), however, I am sure the issue is in guile 2.0, though I cannot
> even found the culprit by bisecting (because of too many changes in both
> guile and geda-gaf repositories). Having said all that, I'd like to see
> at least the patch [1] committed before the next guile release in order
> to have guile cross-compilation for windows working. Are there any
> chances for that?

Just so you and your users know: there's a 32-bit MinGW build of Guile
2.0.x here:

  
http://sourceforge.net/projects/ezwinports/files/guile-2.0.11-2-w32-bin.zip/download

Re: System Scheme (was Re: GOOPS Terminal Class - RnRS POSIX support)

[Eli Zaretskii]

 From: Marko Rauhamaa ma...@pacujo.net
 Date: Tue, 23 Jun 2015 11:08:23 +0300
 Cc: guile-user@gnu.org

 Michael Tiedtke michele.ti...@o2online.de:
 POSIX isn't that important or useful anymore but full access to
 POSIX system calls it has never been.
 What I'd like is a way to communicate open file descriptors between
 processes.
 That's highly non-portable, and is bound to produce Guile code that
 works only on some systems.
 I'd like to produce Guile code that works on Linux. As it stands, I
 can't.

Of course you can: write it in C and load it via FFI.

Re: GOOPS Terminal Class - RnRS POSIX support

[Eli Zaretskii]

 From: Marko Rauhamaa ma...@pacujo.net
 Date: Tue, 23 Jun 2015 11:08:23 +0300
 Cc: guile-user@gnu.org
 
 Michael Tiedtke michele.ti...@o2online.de:
 
  POSIX isn't that important or useful anymore but full access to POSIX
  system calls it has never been.
 
 What I'd like is a way to communicate open file descriptors between
 processes.

That's highly non-portable, and is bound to produce Guile code that
works only on some systems.

Re: Unable to compile Guile

[Eli Zaretskii]

 Date: Sun, 9 Nov 2014 23:30:07 -0500
 From: jeremy chen yng...@gmail.com
 
 Hi, I am trying to compile guile on windows with Mingw-w64.
 The file libguile/numbers.c signal an error in this function:
 
 static SCM
 scm_i_inum2big (scm_t_inum x)
 {
 /* Return a newly created bignum initialized to X. */
 SCM z = make_bignum ();
 #if SIZEOF_VOID_P == SIZEOF_LONG
 mpz_init_set_si (SCM_I_BIG_MPZ (z), x);
 #else
 /* Note that in this case, you'll also have to check all mpz_*_ui and
 mpz_*_si invocations in Guile. */
 #error creation of mpz not implemented for this inum size
 #endif
 return z;
 }
 
 sizeof(void*) is 8 and sizeof(long) is 4 on my machine.
 Anyway to get around this? Thanks.

This is a bug in Guile: it assumes the LP64 data model, where both
'long' and a pointer are 64-bit wide.  But 64-bit Windows uses the
LLP64 model, where 'long' is a 32-bit data type, and the 64-bit
integer data type is 'long long' or '__int64'.

You need to fix Guile to support the LLP64 model, at least in this
module, and probably elsewhere as well.

Re: [ANN] guile-colorized-0.0.2 released

[Eli Zaretskii]

 From: Nala Ginrut nalagin...@gmail.com
 Date: Wed, 17 Sep 2014 18:11:59 +0800
 Cc: guile-devel guile-de...@gnu.org
 
 Hi folks!
 Here's the latest release of guile-colorized:
 https://github.com/NalaGinrut/guile-colorized/releases
 
 guile-colorized is a repl plugin to make your repl color.

Thanks.

Too bad it requires an ANSI-capable terminal, and therefore will not
work on MS-Windows.

Re: [ANN] guile-colorized-0.0.2 released

[Eli Zaretskii]

 From: Nala Ginrut nalagin...@gmail.com
 Cc: guile-user@gnu.org, guile-de...@gnu.org
 Date: Wed, 17 Sep 2014 18:33:50 +0800
 
  Too bad it requires an ANSI-capable terminal, and therefore will not
  work on MS-Windows.
 
 Oh, I'm not familiar with Windows, what's the solution on it?

You cannot do that from Scheme, I think, you need to call special
functions to set foreground and background colors from the C level.

This would require Scheme procedures to set console colors of the
text, and then your package would need to call them, instead of
constructing SGR sequences on the flight.

 Or patches welcome. ;-)

I'll see what I can do.

Thanks.

Re: compilation-mode support for Guile backtrace

[Eli Zaretskii]

 From: Jan Nieuwenhuizen jann...@gnu.org
 Date: Fri,  8 Aug 2014 13:05:53 +0200
 
 Still wondering about my previous set of Guile/Emacs integration
 patches

For the Emacs side: please be patient.

Re: [PATCH 1/3] Support Guile backtraces in compilation mode.

[Eli Zaretskii]

 From: Jan Nieuwenhuizen jann...@gnu.org
 Date: Fri,  8 Aug 2014 13:05:54 +0200
 
 +(defcustom compilation-guile-load-path '(/usr/share/guile/2.0 
 /usr/share/guile/site)

Not a good idea, IMO.  This is inherently system-dependent, and
doesn't support Guile installations with non-default $(prefix).  IOW,
it's unreliable.

How about invoking the Guile executable and asking it to emit the
path?  That would be reliable and portable.

 +  Path to locate Guile soure files for incomplete (Guile %load-path based) 
 error messages.  If compilation-guile-load-path-p is non-nil, it is 
 dynamically set from Guile.

Please break the doc string into lines that are shorter than 75
columns, that's our coding practice.

Thanks.

Re: [PATCH 1/3] Support Guile backtraces in compilation mode.

[Eli Zaretskii]

 From: Jan Nieuwenhuizen jann...@gnu.org
 Cc: guile-user@gnu.org, emacs-de...@gnu.org
 Date: Fri, 08 Aug 2014 15:53:11 +0200
 
 +(defcustom compilation-guile-get-load-path-command guile -c '(write 
 %load-path)'

OK, but for portability to non-Posix platforms, please use only ..
quotes, as '..' quoting is Posix-specific.

 Thanks for your comments!  I'm working on a patch right now for Guile to
 use GNU standard error/info messages as per Stefan's suggestion.

Thank you for your work.

Re: Does Guile have a thread limit?

[Eli Zaretskii]

 From: Diogo F. S. Ramos d...@riseup.net
 Date: Sat, 05 Apr 2014 03:28:25 -0300
 
 The following program is aborted:
 
 --8---cut here---start-8---
 (define number-of-thread 1000)
 
 (do ((i number-of-thread (- i 1)))
 ((zero? i))
   (call-with-new-thread (lambda () (sleep 42
 --8---cut here---end---8---

On what OS?

Re: GNU Guile 2.0.10 released

[Eli Zaretskii]

 Date: Thu, 20 Mar 2014 11:34:53 +0100
 From: Panicz Maciej Godek godek.mac...@gmail.com
 Cc: bd...@lists.opendylan.org, guile-user@gnu.org guile-user@gnu.org
 
  I did remove the only reference to mkstemp.c that appeared in the
  Makefile.am, then run autoreconf and configure, but it turned out that
  there were still some dependencies on libguile_2.0_la-mkstemp.o in the
  generated libguile/Makefile. After removing those, I went through that
  part of compilation, but the process stalled on the GEN
  guile-procedures.texi.
 
  OK.  How long did you wait?  That part can take a bit of time (I'd say
  30s to 1mn maybe) because it runs the yet-to-be-compiled Guile.
 
 I decided to run the whole process once again, and GEN
 guile-procedures.texi has been running for almost an hour now, and it
 clearly uses no CPU.
 
 I had a process hacker tool installed, and it allows me to do some
 introspection (although I understand very little). There are two
 threads running in the youngest guile.exe (#2820), the first one
 with start address
 guile.exe+0x1570
 and the second with
 msvcrt.dll!endthreadex+0x3a
 
 When I inspect the first thread, I get:
 
 0, ntkrnlpa.exe!KiDeliverApc+0xb3
 1, ntkrnlpa.exe!ZwYieldExecution+0x19ae
 2, ntkrnlpa.exe!NtWaitForSingleObject+0x38c
 3, ntkrnlpa.exe!KeReleaseInStackQueuedSpinLockFromDpcLevel+0xb14
 4, ntdll.dll!KiFastSystemCallRet
 5, kernel32.dll!WaitForMultipleObjects+0x18
 6, pthreadGC2.dll!pthreadCancelableTimedWait+0x4b
 7, pthreadGC2.dll!sem_timedwait+0x16f
 8, 0x79a50
 
 The inspection of the second thread shows:
 
 0, ntkrnlpa.exe!KiDeliverApc+0xb3
 1, ntkrnlpa.exe!ZwYieldExecution+0x19ae
 2, ntkrnlpa.exe!NtWaitForSingleObject+0x9a
 3, ntkrnlpa.exe!KeReleaseInStackQueuedSpinLockFromDpcLevel+0xb14
 4, ntdll.dll!KiFastSystemCallRet
 5, ntdll.dll!RtlEnterCriticalSection+0x46
 6, ntdll.dll!KiUserApcDispatcher+0x7
 
 When I use the analyze  wait option on the first thread, I get:
 Thread is waiting (alertable, wait all) for:
 Handle 0x79a50 (Semaphore): (unnamed object)
 Handle 0x7b4 (Event): (unnamed object)

Isn't this the same problem I reported last year, starting here:

  http://lists.gnu.org/archive/html/guile-user/2013-06/msg00022.html

I suggest to configure Guile --without-threads, and see if that
resolves the issue.

Re: GNU Guile 2.0.10 released

[Eli Zaretskii]

 Date: Wed, 26 Mar 2014 19:44:53 +0100
 From: Panicz Maciej Godek godek.mac...@gmail.com
 Cc: Ludovic Courtès l...@gnu.org, 
   bd...@lists.opendylan.org, guile-user@gnu.org guile-user@gnu.org
 
 2014-03-26 19:24 GMT+01:00 Eli Zaretskii e...@gnu.org:
  I had a process hacker tool installed, and it allows me to do some
  introspection (although I understand very little). [...]
 
  Isn't this the same problem I reported last year, starting here:
 
http://lists.gnu.org/archive/html/guile-user/2013-06/msg00022.html
 
  I suggest to configure Guile --without-threads, and see if that
  resolves the issue.
 
 I suspect that it is. But I wanted to rebuild my SLAYER framework for
 Windows and make a new release with Guile 2.0.10 and threads enabled
 -- because then it would make more sense to distribute it.

I know what you mean.  I also hate it when programs lose features when
ported.

 I wish I knew a way to find out whether the problem is on Guile's or
 GC's or mingw's or pthreads-win32's side.

Indeed.  At the time, no one here was able to help me locate the
problem.

 Some guys who had similar problem (i.e. their process also stopped at
 ntdll!KiUserApcDispatcher+0x7) suggest that it is a busy resource:
 http://stackoverflow.com/questions/2214465/debugging-the-dreaded-application-has-failed-to-initialize-error

Last June I reported many details about the hang, you might find there
something about this stuff.  See, for example, these messages:

  http://lists.gnu.org/archive/html/guile-user/2013-06/msg00030.html
  http://lists.gnu.org/archive/html/guile-user/2013-06/msg00076.html
  http://lists.gnu.org/archive/html/guile-user/2013-06/msg00079.html

The last one leads to this:

  http://lists.gnu.org/archive/html/guile-devel/2011-06/msg00034.html

 So now, instead of just compiling --without-threads, I'd like to
 somehow get the threaded version working, even it takes much more time
 and more digging.

It would be wonderful if you succeed in solving this.  Mark told me
that someone succeeded in building MinGW Guile with pthreads, and he
built pthreads himself.  So maybe you should try that, among other
things.

Thanks.

Re: GNU Guile 2.0.10 released

[Eli Zaretskii]

 Date: Wed, 26 Mar 2014 12:02:21 -0700 (PDT)
 From: Mike Gran spk...@yahoo.com
 Cc: Ludovic Courtès l...@gnu.org,
   guile-user@gnu.org guile-user@gnu.org
 
 The Guile build uses the in-tree Guile executable in
 the GEN guile-procedure.texi step.  Backed when I hacked on Guile,
 failing at that step was pretty common if the executable is less
 than 100%.  But a version of Guile that isn't correct enough
 to pass that step may still be functional enough to run much
 of 'make check'.
 
 Thus, I often found it useful hand edit the Makefile to strip out the
 document generation steps so that the build would continue as far as
 possible.

Well, I'm sure we all tried this route here and there.  Sure enough,
back last year, I tried that as well -- but to no avail.  Guile would
get stuck during the next step, when compiling the *.scm files.  See
http://lists.gnu.org/archive/html/bug-guile/2013-05/msg6.html for
more details.

Thanks.

Re: Filename encoding

[Eli Zaretskii]

 Date: Thu, 16 Jan 2014 15:07:43 +0100
 From: John Darrington j...@darrington.wattle.id.au
 Cc: Eli Zaretskii e...@gnu.org, guile-user@gnu.org
 
 If you know that the filename was always obtained using the Guile's 
 interface then the issue is never pertinent.The problem comes when a 
 function
 is aske to open a non-ascii named file, without any information about where 
 that
 filename came from.
 
 
 There is no answer to this general problem.

I think storing file names in some Unicode based encoding internally
is that answer.  If you disagree, please tell why.

 This works up to a point, but eventually there comes an interface where the 
 crucial 
 information is missing.  For example, what happens if the filename is in a 
 text file.

Then the encoding of that file is your clue.

 One has to decide on an approach which will give the lowest probability of 
 surprises.

I think if Guile provides sensible defaults and convenient ways to
override those defaults for a specific operation, that should be
enough.

Re: Filename encoding

[Eli Zaretskii]

 From: l...@gnu.org (Ludovic Courtès)
 Cc: guile-user@gnu.org
 Date: Thu, 16 Jan 2014 14:03:05 +0100
 
 Eli Zaretskii e...@gnu.org skribis:
 
  From: l...@gnu.org (Ludovic Courtès)
  Date: Thu, 16 Jan 2014 00:29:06 +0100
  
  Does anyone know of systems where the file name encoding is commonly
  different from locale encoding?  Is it the case on Windows?
 
  Windows stores file names on disk encoded in UTF-16, but converts them
  to the current codepage if you use Posix-style interfaces like 'open'
  and 'rename'.
 
 So in practice, given that Guile uses the POSIX interfaces, the
 assumption that file names are in the locale encoding is valid on
 Windows.

For now, yes.  But I was under the impression that this thread
discusses future designs, and for that, I don't recommend to rely on
this situation to remain for long.  After all, providing wrappers for
a few Posix interfaces is not such a hard job.

Re: Filename encoding

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Date: Wed, 15 Jan 2014 13:14:39 -0500
 Cc: guile-user@gnu.org
 
 My hope is that this will become less of an issue over time, as systems
 increasingly standardize on UTF-8.  I see no other good solution.
 
 Thoughts?

MS-Windows filesystems will not standardize on UTF-8 in any observable
future.

Likewise, in some Far Eastern cultures, non-UTF encoding are still
widely used.

An other good solution is to decode file names into Unicode based
representation (which can be UTF-8) for internal handling, then encode
them back into the locale-specific encoding when passing them to
system calls and library functions that receive file names.  This is
what Emacs does.

Re: Filename encoding

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: ch...@cvine.freeserve.co.uk,  guile-user@gnu.org
 Date: Wed, 15 Jan 2014 16:34:26 -0500
 
 Well, I understand that MS has standardized on UTF-16 (right?)

Right.

 but what matters from Guile's perspective is the encoding used by
 the POSIX-style interfaces that Guile uses, such as 'open'.  Do you
 know what encoding that is on Windows?

It's the current system codepage, a.k.a. ANSI encoding.  But
sticking to that means you can never support file names with
characters outside of the current locale.  So IMO this is not good
enough for Guile.

The POSIX-style interfaces issue can be worked around by providing
wrapper functions.  A few projects already do that: Emacs, msysgit.

Re: Filename encoding

[Eli Zaretskii]

 Date: Wed, 15 Jan 2014 21:42:57 +
 From: Chris Vine ch...@cvine.freeserve.co.uk
 Cc: m...@netris.org, guile-user@gnu.org
 
 I am not sure what you mean, as I am not talking about internal use.

Then I probably didn't understand why you mentioned the external
encoding.  How is that relevant to the issue at hand?

I'm saying that Guile does needs to know how to convert a file name
when it needs to pass it to library functions and system calls.  So
The POSIX system calls may be encoding agnostic, but Guile simply
cannot be.

 As it happens (although this is beside the point) using a byte value or
 sequence in a filename which the operating system reserves as the '/'
 character, for a purpose other than designating a pathname, or a NUL
 character for designating anything other than end of filename, is not
 POSIX compliant and will not work on any operating system I know of,
 including windows.

Windows is not Posix-compliant, so all bets are off.  As a matter of
fact, there _are_ DBCS codepages where the second byte can be '\'.

Re: Filename encoding

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Date: Wed, 15 Jan 2014 16:47:45 -0500
 Cc: guile-user@gnu.org
 
  All guile needs to know is what encoding the person creating the
  filesystem has adopted in naming files and which it needs to map to.
 
 Right, but how does it know that?  The closest thing we have to a
 standard way to tell programs what encoding to use is via the locale.  I
 believe that's what most existing internationalized programs do, anyway.

You can take the defaults from the locale, but you need to allow the
user or application to change those defaults.

 It seems to me that each system must standardize on a single encoding
 for all filenames on that system, and the locale encoding is the defacto
 standard way of telling programs what that is.

But I can, for example, mount a file system from a different locale
which uses different encoding.  No need to prevent me from using file
names on that file system.

Re: Filename encoding

[Eli Zaretskii]

 From: l...@gnu.org (Ludovic Courtès)
 Date: Thu, 16 Jan 2014 00:29:06 +0100
 
 Does anyone know of systems where the file name encoding is commonly
 different from locale encoding?  Is it the case on Windows?

Windows stores file names on disk encoded in UTF-16, but converts them
to the current codepage if you use Posix-style interfaces like 'open'
and 'rename'.  (There are parallel APIs that accept UTF-16 encoded
file names.)

Re: guile+iconv fails on windows

[Eli Zaretskii]

 Date: Sun, 10 Nov 2013 10:16:51 +0100
 From: Panicz Maciej Godek godek.mac...@gmail.com
 Cc: guile-user@gnu.org guile-user@gnu.org
 
 The dependency walker shows that libiconv depends only on dlls supplied
 by Windows (kernel32.dll, msvcrt.dll and ntdll.dll). The dependency on
 libgcc appears only in libguile, but it seems to have nothing to do with
 iconv. Rebuilding libiconv didn't change anything

Then I guess you will have to run that compilation under GDB and see
why it thinks utf8 is invalid encoding.  (Is it really utf8 or
utf-8 or maybe UTF-8?)

 (I believe that since those are dlls and libguile plainly depends on
 libiconv-2.dll, there should be no need to rebuild guile
 afterwards?)

If you rebuilt libiconv-2.dll, you shouldn't need to rebuild guile.

Re: guile+iconv fails on windows

[Eli Zaretskii]

 Date: Sun, 10 Nov 2013 23:09:08 +0100
 From: Panicz Maciej Godek godek.mac...@gmail.com
 Cc: guile-user@gnu.org guile-user@gnu.org
 
 Well, I think that is the reason of the problem -- that for some reason
 Windows reports utf8 encoding, while iconv requires it to be UTF-8.

What do you mean by Windows reports?  Where did that utf8 come
from?

Re: guile+iconv fails on windows

[Eli Zaretskii]

 Date: Sat, 9 Nov 2013 23:35:33 +0100
 From: Panicz Maciej Godek godek.mac...@gmail.com
 
 ;;; compiling .\extra\common.scm
 ;;; compiling c:/guile2/share/guile/2.0\system\vm\frame.scm
 Backtrace:
 In unknown file:
?: 1 ;;; compiling system\vm\frame.scm
 Exception thrown while printing backtrace:
 ERROR: In procedure open_iconv_descriptors: invalid or unknown character
 encoding utf8
 
 Do you know what could be the reason and solution?
 Should I rebuild libiconv and then relink guile against it?
 Would that work? Or is there any quick workaround?

Assuming you link against libiconv as DLL, then if you run the
dependency walker on your libiconv DLL, does it show that it depends
on libgcc DLL?  If so, try a version of libiconv that doesn't depend
on libgcc, or build libiconv yourself that way.

Re: Guile-Emacs update

[Eli Zaretskii]

 From: Andy Wingo wi...@pobox.com
 Date: Sat, 07 Sep 2013 11:29:10 +0200
 Cc: guile-user@gnu.org, BT Templeton b...@hcoop.net, joa...@verona.se,
   emacs-de...@gnu.org
 
 On Thu 01 Aug 2013 00:17, taylanbayi...@gmail.com (Taylan Ulrich B.) writes:
 
  the dumping feature of Emacs is disabled
 
 Eventually Guile will support linking together multiple separately
 compiled files into one ELF image.  (The object file format in the
 development Guile branch is ELF -- we have our own linker and loader, so
 we currently plan on using this format even on non-ELF platforms.)

Was this ever tested on any non-ELF platform, such as MinGW?

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: dsm...@roadrunner.com,  guile-user@gnu.org
 Date: Mon, 26 Aug 2013 01:56:19 -0400
 
 Okay, again: How exactly do you suggest we determine the location of the
 libguile installation from an arbitrary executable that links to it?

Like this:

  Step 1: provide an API for the executable to tell that; failing that
  Step 2: look in ../share relative to the executable's argv[0]; finally
  Step 3: fall back on the default recorded at build time

Step 2 will have a Windows wrinkle, whereby we can look in ..share
relative to where Guile's shared library lives.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: Panicz Maciej Godek godek.mac...@gmail.com,  guile-user@gnu.org
 Date: Sun, 25 Aug 2013 12:59:43 -0400
 
  Anyway, I looked into this a bit.  I can confirm that the simple
  program you mentioned the first time aborts due to stack overflow
 [...]
  [...] it turns out that GC_get_stack_base, which
  is implemented in libgc, returns zero as the stack base.
 
 Thanks for looking into this, Eli!
 
 This raises the question: what's the relevant difference between
 Panicz's simple 'main' and Guile's 'main' (in libguile/guile.c) that
 causes one to (apparently) initialize the stack base properly, where the
 other fails?  It would be worthwhile to find out.

Thanks for the suggestion, here's some follow-up:

Comparison between libguile/guile.exe and the test program shows that
the former goes through a different initialization process, calling
libgc as part of it.  Here's the call stack from guilify_self_1, when
called from libguile/guile.exe:

 Breakpoint 1, guilify_self_1 (base=base@entry=0x28fe8c) at threads.c:533
 533 {
 (gdb) n
 541   t.pthread = scm_i_pthread_self ();
 (gdb) n
 542   t.handle = SCM_BOOL_F;
 (gdb) p t.pthread
 $1 = 0
 (gdb) bt
 #0  guilify_self_1 (base=base@entry=0x28fe8c) at threads.c:542
 #1  0x004330b7 in scm_threads_prehistory (base=base@entry=0x28fe8c)
 at threads.c:2176
 #2  0x00412c2d in scm_i_init_guile (base=base@entry=0x28fe8c) at init.c:386
 #3  0x0043249c in scm_i_init_thread_for_guile (base=0x28fe8c,
 parent=optimized out) at threads.c:835
 #4  scm_i_init_thread_for_guile (base=0x28fe8c, parent=optimized out)
 at threads.c:814
 #5  0x004324c4 in with_guile_and_parent (base=0x28fe8c, data=0x28feb4)
 at threads.c:901
 #6  0x709cae6f in ?? () from D:\usr\bin\libgc-1.dll
 #7  0x004326cc in scm_i_with_guile_and_parent (parent=optimized out,
 data=0x28fee0, data@entry=0x28feb0,
 func=func@entry=0x412abc invoke_main_func) at threads.c:951
 #8  scm_with_guile (func=func@entry=0x412abc invoke_main_func,
 data=data@entry=0x28fee0) at threads.c:957
 #9  0x00412beb in scm_boot_guile (argc=argc@entry=1,
 argv=argv@entry=0x28c6970,
 main_func=main_func@entry=0x4013d4 inner_main,
 closure=closure@entry=0x0) at init.c:320
 #10 0x004c5eab in main (argc=1, argv=0x28c6970) at guile.c:108
 (gdb) p t.base
 $2 = (SCM_STACKITEM *) 0x28fe8c

while the latter does not go through libgc:

 Breakpoint 1, guilify_self_1 (base=base@entry=0x2deff0c) at threads.c:533
 533 {
 (gdb) bt
 #0  guilify_self_1 (base=base@entry=0x2deff0c) at threads.c:533
 #1  0x00402f5f in scm_threads_prehistory (base=base@entry=0x2deff0c)
 at threads.c:2176
 #2  0x00404b89 in scm_i_init_guile (base=base@entry=0x2deff0c) at 
init.c:386
 #3  0x00402344 in scm_i_init_thread_for_guile (base=0x2deff0c,
 parent=optimized out) at threads.c:835
 #4  scm_i_init_thread_for_guile (base=0x2deff0c, parent=optimized out)
 at threads.c:814
 #5  0x00402519 in scm_init_guile () at threads.c:869
 #6  0x004013c0 in main () at guile-hello.c:13
 (gdb) p base-mem_base
 $1 = (void *) 0x0

In the former case, scm_i_with_guile_and_parent does this:

 static void *
 scm_i_with_guile_and_parent (void *(*func)(void *), void *data, SCM parent)
 {
   struct with_guile_args args;

   args.func = func;
   args.data = data;
   args.parent = parent;

   return GC_call_with_stack_base (with_guile_and_parent, args);
 }^^^

and the call to GC_call_with_stack_base correctly initializes the
stack base:

 GC_API void * GC_CALL GC_call_with_stack_base(GC_stack_base_func fn, void 
*arg)
 {
 struct GC_stack_base base;
 void *result;

 base.mem_base = (void *)base; 

I have verified in the debugger that base.mem_base gets a good value
here.

By contrast, in the test program, GC_call_with_stack_base is never
called.  Instead, GC_get_stack_base is called, which returns zero.
The reason for that seems to be that GC_get_stack_base is called
_before_ GC_init:

 (gdb) break GC_get_stack_base
 Breakpoint 1 at 0x4bbed8
 (gdb) break GC_init
 Breakpoint 2 at 0x4bbfa8
 (gdb) r
 Starting program: D:\usr\eli\utils\guile-2.0.9\guile-hello.exe
 [New Thread 11200.0x39ac]

 Breakpoint 1, 0x004bbed8 in GC_get_stack_base ()
 (gdb) bt
 #0  0x004bbed8 in GC_get_stack_base ()
 #1  0x00402508 in scm_init_guile () at threads.c:868
 #2  0x004013c0 in main () at guile-hello.c:13
 (gdb) c
 Continuing.

 Breakpoint 2, 0x004bbfa8 in GC_init ()
 (gdb) bt
 #0  0x004bbfa8 in GC_init ()
 #1  0x0043d53b in scm_storage_prehistory () at gc.c:653
 #2  0x00404b81 in scm_i_init_guile (base=base@entry=0x2deff0c) at 
init.c:385
 #3  0x00402344 in 

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: Panicz Maciej Godek godek.mac...@gmail.com,  guile-user@gnu.org
 Date: Sun, 25 Aug 2013 12:59:43 -0400
 
 Eli Zaretskii e...@gnu.org writes:
 
  I guess you didn't configure without threads on GNU/Linux, did you?
  If not, I suggest to try that, my impression is that Guile without
  threads is not used too much on Posix platforms.
 
 Hydra, a continuous integration system, runs Guile's make check with
 threads disabled on several POSIX platforms, so there's no need for
 Panicz to do this test.

Sorry, I disagree.  make check is not the issue here: Guile passes
it on my machine with flying colors (see my reports back then).  And
yet a simple program shown by Panicz aborts due to stack overflow.
I was suggesting to configure Guile on GNU/Linux without threads and
run that same program: perhaps it will also fail.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: godek.mac...@gmail.com,  guile-user@gnu.org
 Date: Sun, 25 Aug 2013 15:50:34 -0400
 
 Well, that's not generally how things are done on POSIX systems

As I said, some GNU projects already do that.  GCC and GDB are two
examples.

 furthermore I don't see how this could be made to work reliably on
 POSIX.

It's true that this is not 100% bulletproof on Posix, but it's close.
And when it doesn't work, we didn't lose anything, we are just back to
where we are now, right?

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: godek.mac...@gmail.com,  guile-user@gnu.org
 Date: Sun, 25 Aug 2013 15:56:53 -0400
 
 Remember that Guile is a library, not just an executable.  So argv[0]
 could point to any arbitrary executable that's linked with libguile.

We can provide an API for passing to the library the root of its
installation.

 Having said that, we should probably come up with a solution for
 relocatable builds on MacOS X as well.

Indeed.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 Date: Sun, 25 Aug 2013 23:33:51 +0300
 From: Eli Zaretskii e...@gnu.org
 Cc: guile-user@gnu.org
 
  From: Mark H Weaver m...@netris.org
  Cc: godek.mac...@gmail.com,  guile-user@gnu.org
  Date: Sun, 25 Aug 2013 15:56:53 -0400
  
  Remember that Guile is a library, not just an executable.  So argv[0]
  could point to any arbitrary executable that's linked with libguile.
 
 We can provide an API for passing to the library the root of its
 installation.

And btw, how is this different from GCC looking for its libgcc or GDB
looking for its Python scripts?

An executable linked with libguile will either be in /usr/bin or
somesuch, i.e. close to /usr/lib where libguile lives; or it will be
in some random place under the user's home directory, in which case
either libguile is in the default place, or it is near the binary.
The latter case is precisely the additional feature where looking for
the library nearby will be a benefit.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: godek.mac...@gmail.com,  guile-user@gnu.org
 Date: Sun, 25 Aug 2013 17:42:27 -0400
 
 Eli Zaretskii e...@gnu.org writes:
 
  From: Mark H Weaver m...@netris.org
  Cc: godek.mac...@gmail.com,  guile-user@gnu.org
  Date: Sun, 25 Aug 2013 15:56:53 -0400
  
  Remember that Guile is a library, not just an executable.  So argv[0]
  could point to any arbitrary executable that's linked with libguile.
 
  We can provide an API for passing to the library the root of its
  installation.
 
 I suppose, but that assumes that the main program knows the location of
 the libguile installation it's linked to.  How would it know this?

We are talking about the situation where libguile is _not_ installed
in the usual places.  Why would a program _not_ know where that is?

  And btw, how is this different from GCC looking for its libgcc or GDB
  looking for its Python scripts?
 
 GCC and GDB are programs, not libraries.  Finding out the location of
 the current executable is a much easier problem than finding out the
 install prefix of a particular library.

The issue is how to find the auxiliary files _given_ the location of
the executable, not how to find where the executable itself lives.

  And when it doesn't work, we didn't lose anything, we are just back to
  where we are now, right?
 
 I disagree.  If we advertise a new feature that cannot work without
 making dubious assumptions, then we're making promises we can't keep.
 That's a step in the wrong direction, IMO.

My turn to disagree.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 Date: Sun, 25 Aug 2013 19:24:17 -0400
 From:  dsm...@roadrunner.com
 Cc: guile-user@gnu.org
 
 On windows, I think you call GetModuleFileName() with the handle that was 
 passed to DllMain().
 Is it possible for libguile to do that?

I see no reasons why not.

 Seems like most windows apps install everything into their own subtree.  The 
 main application is not even on the system PATH!

That's not true.  On my machine, there's a single Posix-like tree
rooted at d:/usr, with everything below it.

Many (most?) installers will install the shared libraries into a
single system directory (e.g., C:/WINDOWS/system32), unless you
override the installation defaults, something the majority of users
won't do.

 On a POSIXy system, there are shared places for config files, libraries, 
 documentation, and so on.  It's just different.  I don't see why we would 
 need to be relocatable here.

One reason to be relocatable is to be able to easily run
applications from the build or source tree, while a different version
is installed on the system.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 Date: Fri, 23 Aug 2013 22:14:43 +0200
 From: Panicz Maciej Godek godek.mac...@gmail.com
 Cc: guile-user@gnu.org guile-user@gnu.org
 
  Configuring with --prefix=/usr is a mistake: there's no /usr on your
  system, as far as MinGW programs are concerned.
  [...]
 
 
 yes, it seems that this was the reason. So now the guile
 interpreter runs properly, but I still have a problem with
 running a program that, having been linked against
 libguile, would call scm_with_guile or scm_init_guile, e.g.
 ===
 #include libguile.h
 void hi(void *unused) { scm_puts(hello!\n, scm_current_output_port()); }
 int main() {
scm_with_guile(hi, NULL);
   return 0;
 }
 ===

Does this work on other platforms?  (I know almost nothing about
linking C programs with libguile; maybe your program has a bug, or you
are missing some function call necessary for this to work?)

 I compile it under msys in c:/guile-2.0/bin (i.e. prefix/bin) using
 $ gcc -o hello.exe hello.c -I ../include/guile/2.0 -lguile-2.0

There should be no need to compile under MSYS, you can compile from
the Windows cmd window.  I don't think that's the reason for your
problems, but it's worth a try.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 Date: Thu, 22 Aug 2013 22:25:02 +0200
 From: Panicz Maciej Godek godek.mac...@gmail.com
 
 I managed to go through the compilation process,
 and (after modifying meta/Makefile.am according to Eli's
 patch) to make install.

This compilation process includes compiling all the Scheme files that
come with Guile.  Since you say it succeeded, I don't understand what
you say later, see below.  Do you see a lot of *.go files in the
directory where you built Guile?

 When I ran guile-2.0, I got the following message:
 ===
 Throw without catch before boot:
 Throw to key misc-error with args (primitive-load-path Unable to find
 file ~S in load path (ice-9/boot-9) #f)Aborting.

This seems to say that Guile cannot find its Scheme files.

 Supplying GUILE_LOAD_PATH=/usr/share/guile/2.0 explicitly
 helped a little

If that helped to avoid the failure to load, you probably didn't
specify a correct --prefix at configure time, or your make install
somehow didn't DTRT.

 but only during the first run -- it compiled some files from that
 directory and proceeded to the prompt.

This is what I don't understand: which files it needed to compile, and
why?  The compilation of Scheme files is part of the build process,
which you say you ran successfully to completion.  What am I missing?

 The Windows environment doesn't seem to be particularly
 developer-friendly (or maybe I just don't know the right
 tools)

Which developer-friendly features do you lack?  Perhaps it's a matter
of installing or configuring what is already available out there.

 but I'd truly appreciate some help.

Let's start with the basics.  Please describe:

 . Which build of what version of the GC library did you use, and
   where did you get the Windows build of that library?  Likewise for
   other build dependencies, like libunistring, libiconv, etc. --
   please tell where you got each one of them.

 . How did you configure Guile?  If you used any --prefix argument,
   please tell to which Windows directory does that prefix map on your
   system?

 . Did you see any warning or error messages during the build, and if
   so, please show them.

 . Did you see all the Scheme files being byte-compiled?

 . What command(s) did you use to install the built Guile?

 . How do you run Guile after installing it?  In particular, do you
   run it from the MSYS Bash or from the Windows cmd window?

IOW, you didn't tell enough details about the build and the usage to
be able to guess what possible problems could be in your way.

FWIW, I've successfully ran Guile from the Windows prompt after
building it, and successfully built GNU Make with Guile support.  So
it's definitely possible to do that with MinGW.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 Date: Fri, 23 Aug 2013 11:29:27 +0200
 From: Panicz Maciej Godek godek.mac...@gmail.com
 Cc: guile-user@gnu.org guile-user@gnu.org
 
 When I enter the build directory and type
 $ find ./ -name '*.go'
 it seems that every .scm file from `modules' directory has its .go
 counterpart,

Are they also present in the tree where you installed Guile after
building it?

 I ./configured with --prefix=/usr (which is at the same mount point
 as / on my mingw configuration). When it comes to make install,
 I don't know what TRT would be

Configuring with --prefix=/usr is a mistake: there's no /usr on your
system, as far as MinGW programs are concerned.

You need to understand the difference between MinGW and MSYS programs
and their respective setups.  MSYS, like Cygwin, emulates a Posix
system, so it supports /usr etc.  It does so by mounting some Windows
directory (in your case, it seems to be C:/MinGW) as /.  But the Guile
you compiled is _not_ an MSYS program, it is a native Windows program,
so it doesn't know about this mount.  Since the build process records
the /usr prefix in the binary, the resulting Guile executables will
look for their files starting from the /usr directory on the current
drive, and that directory doesn't exist.

You need to reconfigure Guile with --prefix=c:/MinGW (assuming that is
where you want your installation tree to be rooted), and then rebuild
everything.  I would suggest make distclean followed by a new
configure command with the right prefix and then make and make
install.

 gc-7.2d compiled from the source from Hans Boehm's web site
 (I don't remember whether I finally configured it with pthreads
 or with windows threads, but I tried both for sure),
 gmp-5.2.1 from the source (I think it is shared version),
 libffi-3.0.13 from the source
 libunistring-0.9.3 from the source
 libiconv-1.14 and a few others bundled with mingw
 Everything built from the source was configured with --prefix=/usr

See above: you will therefore need to reconfigure and rebuild those as
well, using the correct Windows absolute directory name as prefix.

 Both / and /usr point to C:\MinGW\

Right, and this is the source of your problems.

 And would it be possible to distribute the binary release somehow?
 (in particular, the libguile.dll + scm/go modules that would be loaded
 from current directory)

Yes, it should be possible.  What I usually do for that install to a
separate empty directory, like this:

  make install prefix=d:/test

(you can use any other empty directory as value of prefix), then
create a tarball or a zip file from everything under d:/test.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

 From: Mark H Weaver m...@netris.org
 Cc: Eli Zaretskii e...@gnu.org,  guile-user\@gnu.org guile-user@gnu.org
 Date: Fri, 23 Aug 2013 11:13:19 -0400
 
 Of course, ideally we would allow the entire Guile install to be freely
 relocatable, as Windows users have come to expect.  This would require
 some help from a Windows developer.

Why should this feature be limited to Windows builds of Guile?  Why
not make it work for Posix platforms as well?  Some other projects
already do, they use argv[0] and PATH search to find the place where
the executable is installed, and record relative file names from
there.