Re: Deprecated current-milliseconds

[Lassi Kortela]

chicken time current-process-milliseconds


Thanks. Here's a grep of current-milliseconds in eggs-5-latest. Anybody 
got time to patch some?


dbus/0.97/tests/run.scm
gochan/5.2.10/chibi-compat.scm
gochan/5.2.10/chicken-module5.scm
gochan/5.2.10/gochan-record-printer.scm
gochan/5.2.10/gochan.scm
gochan/5.2.10/readme.md
gochan/5.2.10/tests/fake-search.scm
gochan/5.2.10/tests/run.scm
gochan/5.2.10/tests/stress.scm
hopefully/0.2.6/tests/run.scm
http-session/2.10/http-session.scm
matchable/1.1/matchable-test.scm
matrico/0.5rel/matrico.scm
nrepl/5.0.8/readme.md
openssl/2.2.5/openssl.socket.scm
pigeon-hole/0.2.8/tests/run.scm
postgresql/4.1.4/postgresql.scm
r7rs/1.0.9/scheme.time.scm
salmonella-feeds/0.1.1/salmonella-feeds.scm
sendfile/1.11/sendfile.scm
sendfile/1.11/strategies/mmap.scm
sendfile/1.11/strategies/rw.scm
simple-timer/0.1.2/timeout.scm
socket/0.3.3/socket.scm
sqlite3pth/0.2.6/sqlite3pth.scm
srfi-18/0.1.7/srfi-18.scm
srfi-18/0.1.7/tests/mutex-test.scm
srfi-19/4.9.5/srfi-19-tm.scm
test/1.2/test-support.scm

Deprecated current-milliseconds

[Lassi Kortela]

Hi,

Version 5.3 csc gives the warning:

Use of deprecated identifier `current-milliseconds' from module 
`chicken.time'.


for (at least) these modules:

* scheme.time (r7rs)
* sendfile
* srfi-18

Is there a drop-in replacement for this procedure?

-l

Re: Question about how to check a symbol is bound

[Lassi Kortela]

  (define-record egg-info
    name author desc)

  (define (show-egg-info egg)
    (define (symbol-value sym)
  (##sys#slot sym 0))
    (define (getter field-name)
  (symbol-value
   (string->symbol
    (format #f "egg-info-~a"
    field-name
    (let ((fields '(name author desc)))
  (for-each
   (lambda (f)
     (format #t "~a: ~a~%"
     f
     ((getter f) egg)))
   fields)))

  (show-egg-info (make-egg-info
  "F-operator"
  "Kon Lovett"
  "Shift/Reset Control Operators"))


What you're looking for is record inspection (aka introspection). That 
has been standardized in R6RS, but Chicken doesn't have it AFAIK.


The right points of comparison in Common Lisp are:

* slot-value
* slot-boundp
* slot-exists-p

Symbol-value is not a good point of comparison, as it has to do with 
packages, not records. Neither Scheme not CL programmers tend to solve 
problems by poking around the symbol table. It's a last resort.


Your best bet in Scheme as it stands, is to use a hash table or 
association list instead of a record type. Or wrap a hash table in a record.


I think it is a very common idiom in languages from Lisp family. So it 
is important to know

how to check symbol is bound and get its value.


It's not idiomatic at all. It's useful mainly for tools that support 
interactive development, e.g. via REPL or Emacs.

Re: srfi-tools egg

[Lassi Kortela]

I was originally thinking about doing that, but since the srfi program 
needs the greater srfi-common file tree it's part of to be present in 
$SRFI_HOME to be able to run effectively, I changed my mind. Seemed 
easier to just check out the repository into the right spot and run 
chicken-install in it.


Good point. Perhaps there should be an `srfi init` subcommand that 
git-clones srfi-common under $SRFI_HOME?

srfi-tools egg

[Lassi Kortela]

We now have an up-to-date .egg file for the `srfi` command line tool, a 
new program maintained by the SRFI Editor and volunteers to help run the 
SRFI process.


The source tree is here: 
https://github.com/scheme-requests-for-implementation/srfi-common/tree/master/srfi-tools


Typing `chicken-install` in that directory builds a working `srfi` 
executable. (We should have ironed out most of the bugs by now, but a 
few doubtless remain.)


Would it make sense to add this egg to the Chicken coop? We can write 
the requisite wiki page for it.

Re: Different versions of syntax-rules

[Lassi Kortela]

I seem to remember that in R7RS, an underscore matches anything, which
is not entirely compatible with R5RS, where you could actually use an
underscore as a variable name.


That seems correct. I can find the underscore rule in neither R5RS nor 
SRFI 46, but it is present in R6RS and R7RS.



Besides that one nitpick, I'd be fine with copying it over to core.


Here's the diff (with `diff -uw`, ignoring whitespace differences): 
https://misc.lassi.io/2023/chicken-synrules.diff


It looks like the changes are light.

Of particular note is that the R7RS code adds this:

+   ;; R7RS support: underscore matches anything
+   (define (underscore? x)
+ (c x (r '_)))

If you wish to support the R5RS semantics where underscore isn't 
special, this procedure could check some kind of state variable saying 
which flavor of syntax-rules to use. `underscore?` could always return 
#f unless the R7RS version is in use.

Different versions of syntax-rules

[Lassi Kortela]

Chicken currently has two different implementations of syntax-rules:

- synrules.scm in chicken-core

- synrules.scm in the r7rs egg

I'm far from an expert, but at a glance it looks like the r7rs egg's 
version is indeed R7RS compliant whereas the chicken-core version is r5rs.


R7RS (and R6RS) = R5RS + SRFI 46 (Basic Syntax-rules Extensions)?

Both of the above versions of synrules.scm claim the following lineage:

Copyright (c) 1993-2001 by Richard Kelsey and Jonathan Rees.

;;; [Hacked slightly by Taylor R. Campbell to make it work in his
;;; macro expander `riaxpander'.]

;; [Hacked even more by Felix L. Winkelmann to make it work in his
;; Hi-Lo expander]

I can't tell whether these hacks affect the feature set offered to 
users, or just the internals.


The version in the r7rs egg additionally says:

;; This is a slightly modified copy of core syntax-rules, enhanced
;; with underscore "wildcard" patterns and the ellipsis (... ...)
;; "quoting" mechanism from R7RS.

Is there a desire to stick to r5rs features only in the chicken-core 
expander, or is the intention to fold the R7RS / SRFI 46 features back 
into the chicken-core expander at some point in the future?

General comments

[Lassi Kortela]

Arguably that's a feature - in Scheme you'll always be sure of the
performance characteristics of the type you're working with, and you'll
always know what type you're working with.  Also, by not having generics
for collection types you avoid quite a performance overhead related to
dispatch. But I understand many people disagree that this is desirable.


While RnRS collection procedures are monomorphic, users can easily add 
generic ones. IMHO the latter should be standardized in the future.


(RnRS number procedures have always been generic, so I see generic 
collection procedures as a natural evolution of Scheme.)



Besides just being "etched in stone", the SRFI process has been hijacked
by a small group of people and is producing SRFIs at breakneck speed with
perhaps not enough community input.  Usually by the time I learn about a
SRFI about a subject I find interesting, it's already finalized ;)
That makes SRFIs rather "take it or leave it".  And often they're not
especially ergonomic either.


Agreed by many people, myself included.

To those who have a few spare cycles, please help me try to alleviate 
the problem. There is now an experimental alternative at 
https://groups.scheme.org/review/ to ease the constraints that cause 
problems with SRFI. Namely, the time pressure to finish proposals and 
the focus on "requests for implementation" are gone.


All that's needed to get going is for people to submit a couple of 
proposals. It would even be possible to source eggs from the proposals' 
git repos.


I announced this on srfi-discuss which generated a lot of discussion but 
no holy war ensued. The thread is here: 
https://srfi-email.schemers.org/srfi-discuss/msg/21408058/

Re: Partial automation of egg dependencies

[Lassi Kortela]

I think I understand the problem, but I'm not sure I understand the
solution (I'm probably missing something). :-)

* Regarding the usability problem mentioned, I still don't see how a
   user could easily see all the dependencies of an egg if the egg file
   has only `(auto-dependencies)'.

* Regarding the problem related to salmonella reports, I suppose
   salmonella would have to rely on some file emitted by chicken-install
   to be able to figure out dependencies.

* Regarding the import problem, another thing to keep in mind is that we
   don't have a 1-1 mapping between egg and module names.  Module names
   don't have to be the same as egg names and one egg might ship zero,
   one or multiple modules.  If you have `(import foo)' it doesn't mean
   that the `foo' module is shipped by an egg called `foo'.

* How can `auto-dependencies' deal with egg versions?


Sorry, I was probably being unclear. The idea is to only auto-detect 
dependencies between components of the same egg.


Dependencies between eggs would still be written manually. (Perhaps the 
groveler could also help detect when you forgot to specify some of them 
in the .egg file. But it wouldn't be smart enough to figure them out by 
itself.)


In any case, yes - there would have to be some command to view the 
expanded dependencies.

Re: Partial automation of egg dependencies

[Lassi Kortela]

Actually, perhaps the best way would be to use a custom build
using csm to build the components of the egg. A little bit of
inventiveness might be required, but I think it should be possible.


This would make the egg depend on csm. I don't mind this for programs, 
but for eggs that are just libraries, isn't this a bit "opinionated" / 
heavy given that all the build steps (apart from the import groveling) 
can be done using the current chicken-install?


Perhaps the csm groveler could be extracted into a library? Perhaps it 
could even be shipped in chicken-core at some point, since after all 
Chicken has to be able to trace all imports and includes in order for 
the compiler and interpreter to work.



I added a "-compile-imports" to csm today to force import libs to
be compiled, which may be a start. Perhaps we can cover this
particular scenario without too much extra effort.


This is above my paygrade :) I'll try to understand the patch.

Re: Partial automation of egg dependencies

[Lassi Kortela]

My take on that as a user: I care about dependencies and tend to avoid
eggs with many dependencies.  If I see `(auto-dependencies)' in an egg
file, I don't know the dependencies of that egg.  I would not like that.

Also, that would break the [reverse] dependency ranks of salmonella
reports.


auto-dependencies should only cover things that are possible to compute 
statically by looking at the source files. So you'd have a simple, 
predictable tool that expands them to manual dependencies at will.


The main source of trouble is that this:

(import foo bar baz)

specifies stuff that is then duplicated in the .egg file. When the 
imports change, the .egg has to be changed too. Else the build may break.


This is the same problem that C/C++ programmers face in keeping 
makefiles synced with #include. Many of them concluded that thy can't 
maintain correct makefiles by hand, and wrote tools like makedepend.

Re: Partial automation of egg dependencies

[Lassi Kortela]

It's ambiguous. There can be (and are) eggs that export procedures of the same name. 


Procedures are exported from modules, not from eggs, right? Where's the 
conflict?



Dependencies shouldn't get out of sync, and updated versions may change what is 
exported... so I would question exactly what you are doing that would require 
such functionality and whether there would be a better development methodology 
that wouldn't shift the burden from paying attention to what one is using to a 
psychic programme that does what you mean (not what you tell it).


Here it is: 
https://github.com/scheme-requests-for-implementation/srfi-common/blob/master/srfi-tools/srfi-tools.egg


For practical purposes, that file is impossible to keep in sync even 
though I wrote a tool to help.


I expect the best way to do it is manually list all the components of an 
egg. Then give the "root" source file for each component, and it'd 
automatically traverse starting from the roots.


AFAICT this is a good balance between automatic and manual - should give 
quite predictable results. Library/module definitions have a known 
grammar so you can find all the includes and imports.


The only complication I can see is cond-expand. Different cond-expand 
clauses can have different imports/includes, so the code walker would 
have to pin down the feature list.



That is quite a lot of stuff... You mention csm already, which does
most of this, but for packaging eggs I think this is a bit too much
automatic processing for my taste. I don't think such a feature belongs
into the core system, but a preprocessor (perhaps by scavenging code
from csm) or a mode for csm to emit egg files might be appropriate.


Fair enough.

If csm could generate .egg files I'd be happy with that. But csm's idea 
is to work without any "makefile" or .egg style specification, right? So 
it's even more automated than what I suggest.

Partial automation of egg dependencies

[Lassi Kortela]

I have a .egg file for a portable Scheme program substantial enough and 
fast-moving enough that the dependencies are constantly getting out of sync.


Would it be possible to add a feature something like this?

(extension foo.bar
  (source "bar.sld")
  (auto-dependencies)  ; <-- The new feature.
  (csc-options "-R" "r7rs" "-X" "r7rs"))

auto-dependencies would compute the deps by traversing the given source 
file. It would look at csc-options and notice that r7rs is being used. 
Then expect an R7RS-style define-library form in bar.sld, look at 
(import ...) libraries matching those in the .egg and add those to 
component-dependencies, as well as traverse all (include "...") and add 
those to (source-dependencies ...).


If csc-options doesn't say r7rs, it would do the same, but expect 
Chicken module syntax instead.


I wrote some code to grovel r7rs libraries, and there's probably similar 
stuff in csm and Akku.scm too. So if the feature is ok we could stitch 
together the implementation from these.

Re: New egg: gmi

[Lassi Kortela]

I wrote an egg for the Gemini network protocol at 
. I haven't ended up as an 
active user of Gemini, so if someone more active wants to take over the 
egg, I can move the repo under your account.


Last we checked, the openssl egg had trouble with some Gemini sites. We 
didn't manage to find the root cause.


Keeping the protocol and the markup format in separate eggs is a good 
idea IMHO.

Re: CHICKEN version dependent expansion

[Lassi Kortela]

#;1> (cond-expand (chicken-5 'a) (chicken 'b) (else 'c))
a

This only seems to let me differentiate between major versions and not minor 
versions, am I correct? I would need to decide based on minor versions.


#;2> (cond-expand (chicken-5.3 'a) (else 'b))
a

To find these, I used the r7rs egg (since the r7rs standard mandates a 
`features` procedure).


$ chicken-install r7rs
$ csi -R r7rs
#;1> (features)
(r7rs srfi-13 srfi-14 ...)

Re: CHICKEN version dependent expansion

[Lassi Kortela]

I am trying to define a function depending on the version of CHICKEN.  I was thinking to 
use "cond-expand" for this purpose, but found no suitable feature in the 
documentation


#;1> (cond-expand (chicken-5 'a) (chicken 'b) (else 'c))
a

Re: Bare-Metal Suitability

[Lassi Kortela]

as a beginner
you're probably better off finding an implementation that requires
less of an OS footprint.


- Loko Scheme runs on bare metal x86-64.

- Gambit Scheme has been used to compile bare metal code.

- Ribbit is a new ultralight Scheme from Gambit's author.

- There are various microcontroller Schemes - from memory, most recent 
last: BIT, PICBIT, PICOBIT, Armpit, Microscheme. Papers written about at 
least the first three.

Re: What are the long-term goals for R7RS in Chicken?

[Lassi Kortela]

But I"d like to stress the fact that run-time and compile-time file
locations may differ, also things like dynamic loading vs static
linking come into play. In the end this is a build issue, not necessarily
a question of run-time semantics.


Sorry, I was being ambiguous. The natural strategy when dealing with 
R7RS is to compile each .sld file [i.e. each (define-library ...) form] 
into one .o or .so file, traversing any (include "...") files at compile 
time, not at runtime, so that the code in the included files becomes 
part of the object code corresponding to the .sld file.


So you'd check foo.sld into git, and compile it and its dependencies 
into foo.so for dynamic linking, or into foo.o for static linking.



Actually, the physical locations of code are secondary, as long as libraries
are found, resolved and evaluated properly. Once again, I can not help but point
towards "csm"[1], as a natural and convenient (yet, apparently sadly ignored),
tool-based solution to this problem.


I wasn't aware of the tool. Looks like good thinking went into it!

For portable code, the main obstacle in practice is that a different 
wrapper file is required per each standard and/or implementation. For 
example, R6RS and R7RS typically require separate .sls and .sld files. 
(R6RS has many features that are not in R7RS-small, but a lot of useful 
code would be easily portable if it weren't for the different (library 
...) vs (define-library ...) wrappers, as well as the 
implementation-specific module wrappers in Scheme implementations that 
predate R6RS. It would help a lot if one .sld file could cover all 
R7RS-capable implementations, even if (cond-expand ...)s have to be 
written inside the file.


Reading csm's manpage, it seems to handle .sld files just as well as 
other kinds, but since it's not part of core Chicken, people tend to 
write eggs and other code in .scm files using (module ...) forms which 
complexifies the task of porting that code.


Here's an example of a real library that's provided as both a Chicken 
egg and a R7RS (and R6RS) library in the same git repo: 
https://gitlab.com/weinholt/packrat. The implementation code that does 
the heavy lifting is shared among all three (packrat-r5rs.scm) but the 
library wrappers are all different (), and there's a little magic in 
packrat.egg and packrat.setup to enable things.

Re: What are the long-term goals for R7RS in Chicken?

[Lassi Kortela]

Note that include-files and loaded libraries are two different things,
also in CHICKEN, libraries are usually compiled, so the .sld convention
is only partially useful.


The convention (observed at least by Chibi, Gambit, and Gauche) is that 
each .sld file contains one define-library form.


Gambit can compile something like hello.sld:

(define-library (hello)
  (import (scheme base))
  (begin (write-string "Hello world\n")))

into an object file hello.o1 via "gsc hello.sld". The gsi interpreter 
can load either the original hello.sld or the compiled hello.o1. (I'm 
not sure why it appends a running number to the ".o" suffix.) The same 
arrangement would probably work for Chicken.


The code to implement simple libraries is often written directly into 
the .sld file itself. Complex libraries tend to have the implementation 
parcelled out into one or more separate .scm files, which are (include 
"...") from the .sld file. Non-portable files may be included via 
cond-expand.


(include "...") must always specify the file name extension of the file 
being included, though in practice that tends to always be ".scm". The 
file foo/bar.sld files is imported via (import (foo bar)) and the import 
never specifies the file name extension; ".sld" is implied, though 
something else could be used as well; R6RS tends to use ".sls".


IMHO Gambit's R7RS support is simple, works well, and would probably be 
a reasonable model for Chicken.

Conditional include

[Lassi Kortela]

Lassi: The fact that different Schemes have different conventions for 
where their library files are is actually a great convenience to me when 
developing SRFIs.  All the actual code goes into files named *-impl.scm 
(or multiple names if there are multiple files).  Then each library file 
or equivalent, which is where the differences are concentrated, is in an 
implementation-specific place.  The library file for (foo bar) will be at:


Chibi:  foo/bar.sld
Chicken: foo.bar.scm
Guile: foo/bar.scm
Ypsilon: foo/bar.sls, or foo/bar.ypsilon.sls if it is Ypsilon-specific

And since each of these needs to be slightly different, that's a Good Thing.


This is suboptimal: the meaning is obscured, and things work by accident 
since you have only 4 Schemes and all of them happen to use different 
naming conventions. If you add more Schemes, some of them will 
eventually clash.


R6RS implementations have a convention of appending the implementation's 
name to the filename. For example, 
 has the files:


parameters.sls  ; Portable version of the code.
parameters.chezscheme.sls
parameters.ikarus.sls
parameters.ironscheme.sls
parameters.mzscheme.sls
parameters.ypsilon.sls

This is clearer and more likely to avoid clashes.

Best of all is the R7RS solution:

(define-library (foo)
  (cond-expand (chicken (include "foo.chicken.scm"

Re: What are the long-term goals for R7RS in Chicken?

[Lassi Kortela]

Currently you can switch to R7RS more by installing the egg and running csc
with "-R R7RS -X R7RS", which is a mouthful, and could be abbreviated, that
doesn't look to me like too much hassle.

Anybody for a "-r7rs" option that does the above?


+1 for an easy command line flag!

Gauche and Sagittarius Scheme have an "-r" flag so you can run 
"sagittarius -r 6" or "gosh -r 7". Using one flag gives simple upward 
compatibility to "-r 8", "-r 9" in the future, and also non-RnRS 
standards or modes, should the need arise. Neither csc nor csi is yet 
using the "-r" flag for anything, so the uniformity would be nice. It 
also has nice symmetry with the upper case "-R" flag for extensions.



A good question. I think the Chicken "native" language and module system
is likely to be used more often than a R7RS mode, so I don't think there are
any attempts to make Chicken "more R7RS". As you say, the differences
are not that substantial.


If there are few substantial differences, it would be a boon to writing 
portable code if the same syntax is eventually used as for standard R7RS 
libraries, and the same filename conventions are supported as other R7RS 
implementations (the .sld filename extension for an R7RS library is 
becoming a de facto standard). You already support (include "...") and 
(cond-expand ...) in a similar manner as R7RS.


Here's how a typical module would look, before:

;;-
(module lowdown

  (markdown->sxml
   markdown->sxml*
   markdown-sxml->html-sxml
   markdown->html
   markdown-html-conversion-rules*)

  (import scheme)

  (cond-expand
   (chicken-4
(import chicken)
(use data-structures
 irregex
 srfi-1
 clojurian-syntax
 comparse
 sxml-transforms
 lowdown-lolevel))
   (chicken-5
(import (chicken base)
(chicken irregex)
(scheme)
(srfi 1)
(clojurian syntax)
(comparse)
(lowdown lolevel)
(sxml-transforms

  ...code here...)
;;-

And after:

;;-
(define-library (lowdown)
  (export markdown->sxml
  markdown->sxml*
  markdown-sxml->html-sxml
  markdown->html
  markdown-html-conversion-rules*)
  (cond-expand
   (chicken-5
(import (scheme base)
(chicken base)
(chicken irregex)
(srfi 1)
(clojurian syntax)
(comparse)
(lowdown lolevel)
(sxml-transforms
  (begin ...code here...))
;;-

Functors could use the existing syntax:

(functor (squaring-functor (M (multiply))) (square)
  (import scheme M)
  (define (square x) (multiply x x)))

or spell it `define-functor` for consistency.

It should be possible to preserve the old (module ...) and (functor ...) 
syntax for backward compatibility so current modules keep working with 
no code changes.

What are the long-term goals for R7RS in Chicken?

[Lassi Kortela]

Is there a consensus on how deeply to integrate R7RS into Chicken? It 
seems mostly R7RS compliant as it stands. Are there any technical or 
ideological blockers to basing core Chicken on R7RS-small in the future?


The main point of divergence seems to be the native module system, which 
offers a superset of the features of the R7RS library system. The common 
features look more or less compatible, and this impression is reinforced 
by the fact that you already have a useful r7rs egg.

Re: Build static binaries of chicken apps in a docker container

[Lassi Kortela]

Currently is uses my own Chicken docker images (Images: 
https://hub.docker.com/r/lattay/chicken , Dockerfile: 
https://github.com/Lattay/chicken_docker ) so it can work on AMD64 or 
ARMv7 (nice for RPi 2 and 3), but it is very simple and can easily be 
used for other base images.


Your Dockerfile looks almost identical to ours at 
https://github.com/scheme-containers/chicken 
(https://hub.docker.com/r/schemers/chicken). Contributions to our 
container(s) are welcome, in case you feel like it.

Re: chicken-doc instructions recommend extracting tar file as root

[Lassi Kortela]

For a proper fix, could chicken-doc be modified to download the tar 
file, sanity-check its contents, and unpack it safely into the user's 
home directory instead?


Alternatively, if the documentation is shipped in some kind of file 
format with an index for fast lookup, it doesn't need to be extracted 
into multiple files at all. There are reasonably simple databases like 
CDB and Berkeley DB for jobs like this.


CDB is a very simple format that could be ported to Scheme. There's a 
Common Lisp library for it here: https://github.com/xach/zcdb


Alternatively, I tried

tar --to-stdout -xvf chicken-doc-repo-5.tgz >chicken-doc-repo-5.scm 2>&1

and it produced a 12 MiB file. (The -v and 2>&1 cause tar to output the 
filenames of the extracted files into the dump in addition to the 
contents of those files.)


The following script slurrrp.scm:

(define (read-all)
  (let loop ()
(let ((form (read)))
  (unless (eof-object? form)
(loop)

(with-input-from-file "chicken-doc-repo-5.scm" read-all)

clocked using `time csi -script slurrrp.scm` rips through the file in 2 
seconds on average on my computer. This could be acceptable performance.


What you're storing in the tar file is just 2000 small S-expression 
files. You could vastly simplify the system by concatenating those files 
into one. You can still gzip it too if you want to, though does 12 MiB 
(uncompressed) vs 2 MiB (gzipped) really matter? "What's an order of 
magnitude between friends?" as the saying goes.


The chicken-doc wiki page has another section:


Cleaning up old repository crust
Occasionally, or when the repository format changes significantly, you should 
wipe out your repository before extracting a new one, to get rid of dead wood. 
Simply delete the directory shown by the following command:

$ csi -R chicken-doc -p "(locate-repository)"
/usr/local/share/chicken/chicken-doc
If you have chicken-doc-admin installed, just do instead:

$ chicken-doc-admin -D


That cleanup is not needed if the whole thing is just one S-expression 
file instead of 2000 files in a tarball.


Yet another solution is to ship the tarball but store it as-is and have 
chicken-doc extract it in memory instead of having the user extract it 
manually. This is not hard either: uncompressed tar is a simple format 
and you doubtless have a Scheme library for it in the egg collection.


But it seems shipping one big S-expression file would be simplest.

chicken-doc instructions recommend extracting tar file as root

[Lassi Kortela]

Currently https://wiki.call-cc.org/eggref/5/chicken-doc instructs users 
to run:


curl https://3e8.org/pub/chicken-doc/chicken-doc-repo-5.tgz | sudo tar zx

in a directory that's often located within /usr. This is not ideal from 
a security perspective, especially given that that the remote file 
changes daily so some users can be expected to repeat the command lots 
of times.


An immediate safeguard is to edit the wiki page to add the verbose flag 
to the suggested tar command, causing it to show the pathnames of all 
the files it extracts.


For a proper fix, could chicken-doc be modified to download the tar 
file, sanity-check its contents, and unpack it safely into the user's 
home directory instead?


Alternatively, if the documentation is shipped in some kind of file 
format with an index for fast lookup, it doesn't need to be extracted 
into multiple files at all. There are reasonably simple databases like 
CDB and Berkeley DB for jobs like this.


-l

Re: filter

[Lassi Kortela]

I suggest, to include a filter routine with two values, the sublist
which passes the test and the sublist which fails


In Scheme that procedure is called `partition`, also from SRFI 1:
https://srfi.schemers.org/srfi-1/srfi-1.html#partition

`filter` returns one list containing the values that pass the test:
https://srfi.schemers.org/srfi-1/srfi-1.html#filter

You can define `partition` in a few lines:

(define (partition pass? xs)
  (let loop ((xs xs) (pass '()) (fail '()))
(if (null? xs) (values (reverse pass) (reverse fail))
(if (pass? (car xs))
(loop (cdr xs) (cons (car xs) pass) fail)
(loop (cdr xs) pass (cons (car xs) fail))

(partition even? '(0 1 2 3 4 5 6 7 8 9)) => (0 2 4 6 8); (1 3 5 7 9)

Re: Web page with quick links to all C4 and C5 egg versions

[Lassi Kortela]

I wonder if I'm missing the actual meessage that page is trying to
convey.  Is it about showing CHICKEN 4 eggs that are "missing" in
CHICKEN 5 or vice-versa?


That's one use of it -- or it shows if C5 has a different version of the 
same egg than C4.



* Some extensions that are eggs in CHICKEN 5 are core units in CHICKEN 4.


Not sure if it does the trick, but those are documented here:
https://wiki.call-cc.org/porting-c4-to-c5#begin-with-the-easy-part-replacing-module-imports


Thanks. I added a "Core" marker to those SRFIs.

Re: Web page with quick links to all C4 and C5 egg versions

[Lassi Kortela]

Isn't that page similar to https://eggs.call-cc.org/4/ and
https://eggs.call-cc.org/5/ in terms of content?


It collects both Chicken 4 and Chicken 5 versions of the information on 
the same page, which is its main point.



Regarding to the link to the eggs-[45]-latest repos, I think that can be
a bit misleading to people trying to find out the repository of eggs.
Those repositories are merely a representation of the egg caches in git
format -- they are NOT the canonical repositories of eggs.  The
canonical repositories should be linked from the egg documentation.


Good point. Could link to the origin repos instead.


Regarding versions being up to date or not, the intent of that
information is not clear to me (it's very probable that I'm just missing
something).


The page is geared toward egg authors and maintainers, not for people 
who just want to use an egg. That's why there's so much detail, and is 
probably the main cause of confusion.



Some aspects that may be relevant when comparing egg
versions between CHICKEN 4 and 5:

* Some egg authors have deliberately stopped updating eggs for CHICKEN 4
   once they got ported to CHICKEN 5.

* Some egg authors have deliberately decided not to port some eggs from
   CHICKEN 4 to CHICKEN 5.

* Most people who are creating new eggs are not making them available
   for CHICKEN 4.


Sure. FWIW, some C4-only eggs have been ported to C5 upon request.


* Some extensions that are eggs in CHICKEN 5 are core units in CHICKEN 4.


This would be useful to indicate somehow.

Web page with quick links to all C4 and C5 egg versions

[Lassi Kortela]

Here's a listing of all eggs with

- egg name
- description
- license
- latest egg version number for Chicken 5
- latest egg version number for Chicken 4
- link to egg documentation (wiki) and git repo for both versions
- color indicator saying whether each version is up to date or not

https://misc.lassi.io/2020/chicken-eggs.html

Generated by https://github.com/chicken-contrib/chicken-quick-links

If you have a better place to upload HTML files, or some cron job where 
the script could run, feel free to hack on it.

Re: Source code formatter?

[Lassi Kortela]

The problem with pretty printer is that it is not designed for source:
it does not keep comments and blank lines.


+1

Scheme reader that preserves them:

- https://github.com/weinholt/laesare

Work-in-progress formatters:

- https://github.com/paines/scmfmt (uses Chicken pretty printer)

- https://github.com/lispunion/code-formatter

Chez Scheme, Gambit, Gauche, Chibi-Scheme also ship a pretty-printer.

At the moment, most people probably use indent-region in Emacs. Emacs 
can also run in batch mode. There's probably something similar for Vim.


If any one feels like solving the problem for real, I'm happy to advise. 
This is the outline:


- Expand a reader like laesare to cover all the variants of S-expression 
syntax (Scheme, CL, Emacs Lisp, Clojure are very similar; no sense in 
having a different algorithm for each).


- Add a matching printer by consulting "the literature". Marc Feeley of 
Gambit has written a popular pretty-printer. Wadler has written a paper 
from first principles: 
.


- A good pretty printer should automatically figure out where to put 
line breaks, joining and splitting lines of code as needed. This is by 
far the hardest part of pretty-printing; naive approaches are 
susceptible to combinatorial explosion.

Re: Chicken on Debian

[Lassi Kortela]

dnormandin@ASUSLAP:~$ sudo apt install chicken-bin=5.2.0
E: Version '5.2.0' for 'chicken-bin' was not found


5.2.0 is in Debian Testing and Unstable. 
https://repology.org/project/chicken/versions

Re: New egg: SRFI-87: => in case clauses

[Lassi Kortela]

 currently doesn't mention 
SRFI 87.

Re: New egg: unveil

[Lassi Kortela]

I'd like to add the following new egg, which provides unveil(2)
support for CHICKENs running under OpenBSD


Nice feature!

unveil() works together with pledge(). There already seems to be a 
pledge egg at . Any chance the 
two eggs could be merged?

Re: Establishing Git repos for miscellaneous Chicken tools

[Lassi Kortela]

Sure.  What should it be called?


Is "chicken-tools" ok? Or "chicken-integrations" or "chicken-contrib"? 
Or even more generically, "chicken-scheme" which doesn't exist yet.


It could be "chicken-emacs" but Dan suggested:


an organization that focuses on Chicken tool integration would be good; not 
just for Emacs. I'd love to see LSP support, for instance

Re: Establishing Git repos for miscellaneous Chicken tools

[Lassi Kortela]

Sounds good to me.  Thanks for organizing this.


Great. Mario, can you create the org since from our initial group of 
contributors you are the one most closely associated with Chicken?

Re: Establishing Git repos for miscellaneous Chicken tools

[Lassi Kortela]

Please note that what I wrote was just a suggestion to prevent future
problems.  I don't mean to impose anything.  If you think using GitHub,
GitLab or whatever git host available would be a good solution, that's
absolutely fine by me.  I use GitHub myself.  It was just a small remark
to avoid falling in the trap of having a lot of dependencies on
vendor-specific features that might just vanish (like what happened with
Sourceforge, Google Code etc.).


No problem, I agree that it's important to avoid vendor lock-in.


And, sure, the CHICKEN bug tracker, mailing lists, wiki and IRC channel
can be used for CHICKEN-related stuff.


+1


Currently we don't have e-mail notifications.  Our use-case is very
modest for today's standards of fanciness.  Our git traffic is very low.
We basically use the mailing lists to communicate.  It's a very simple
setup -- as it must be, otherwise we wouldn't be able to maintain it.


Given:

- a GitHub organization would be preferred for ease of maintenance by 
Dan and Vasilij who have written some of the Chicken Emacs support so far


- you and I don't object to GH

- nobody else has said anything

- a decision seems hard to make

should we just make a `chicken-tools` org on GitHub? That would let us 
resolve the current stagnation in maintaining these packages, and if we 
disable the wikis and issue trackers we can move the repos to another 
host later on.

Re: Establishing Git repos for miscellaneous Chicken tools

[Lassi Kortela]

Switching git hosting solutions should be just a matter of changing the git URL.


AFAICT this can be achieved by turning off the issue tracker and wiki on 
GitHub repos. This would mean we use the mailing list, Chicken's issue 
tracker, or email/irc for coordination. Since it's just a few tools, 
maybe not worth overthinking this?


Another option would be to put all the tools in one git repo on 
code.call-cc.org, and give write access to that repo to the few people 
who need it. Is there some way to have it send email notifications, 
similar to the GitHub "watch this repo" feature?

Establishing Git repos for miscellaneous Chicken tools

[Lassi Kortela]

We spoke with Mario and Dan about gathering Git repos for all the 
Chicken Emacs packages under one roof, and Dan suggested adding other 
tools such as an LSP server into the mix.


We thought it'd be convenient to start an organization on GitHub or 
GitLab to do this, but are not sure whether the community accepts this. 
Any suggestions to the contrary? We could use code.call-cc.org, but is 
it simple to create organizations and teams there and receive 
notifications about commits?

Re: Determining the correct, most up-to-date source/document for scraping CHICKEN SRFI metadata from

[Lassi Kortela]

Another option is to add an S-expression like (provides-srfi 1 2 3) into 
the .egg file of each egg that implements one or more SRFIs. This would 
make the information independent of the package name, and the 
`eggs-5-latest` Git repo already aggregates all eggs, so an indexer 
could clone that repo and read all the egg files to arrive at the full 
list of SRFIs. Can the existing tools handle a new form in those files?


This has the downside that all egg authors need to be contacted. Having 
a policy of `srfi-NNN` package names sounds reasonable as well.

Re: New egg for SRFI 193: Command line

[Lassi Kortela]

Thanks for your contributions to CHICKEN.


No problem, sorry for taking so long!


You can edit the wiki anonymously, but since you are maintaining eggs,
I'd highly recommend using a personal wiki account (reason being every
time we see an anonymous edit we must check whether it is spam or not).

To get an account, please send me a private message with the username
you want to use and the hash of your password.  It can be generated
with:

 $ openssl passwd -apr1


OK: lassik and $apr1$12JSSXLI$h0R26idN83V9EsUJTcHtF/


All the best.
Mario


You wrote test-new-egg and a ton of other stuff. Thank you for all your 
efforts for Scheme; Chicken's infrastructure works really well!

Re: New egg for SRFI 193: Command line

[Lassi Kortela]

Everything seems to be in order, I’ve added the egg to the repository! :)


Thank you very much.

Unfortunately it has a bug (my fault): chicken-install pre-compiles it 
into a .so and (import (srfi 193)) in csi loads that .so instead of 
loading the source .scm file. The library uses cond-expand to figure out 
whether it's being loaded from csi or csc, but that doesn't work since 
csc has already expanded it when compiling the .so.


Is there a way to figure out if running in csi at runtime without using 
cond-expand?


The offending cond-expands are on this line: 



and this line: 




The wiki page looks fine too, please upload it to the wiki when you have the 
time.


I don't have an account to edit the wiki.

New egg for SRFI 193: Command line

[Lassi Kortela]

Based on the code posted to chicken-hackers previously, here's an egg 
for SRFI 193: .


test-new-egg says:

< srfi-193 (1 of 1) 
  Fetching[ ok ] 0s
  Reading .egg[ ok ] 0s
  Checking dependencies...[ ok ] 0s
  Checking category...[ ok ] 0s
  Checking license[ ok ] 0s
  Checking author.[ ok ] 0s
  Installing..[ ok ] 1s
  Checking version[ -- ]
  Testing.[ ok ] 12s
  Checking documentation..[fail] 0s

An attempt at a wiki page is here: 
.


Is this enough to go on? Let me know if more work is needed.

Best way to track and isolate project dependencies?

[Lassi Kortela]

Does Chicken support installing eggs into a project-specific directory, 
and using only those eggs for that project (ignoring system-wide ones)? 
I.e. something like Python's venv 
(https://docs.python.org/3/library/venv.html) and similar tools for 
other languages.


General-purpose libraries are shipped as eggs, which can depend on other 
eggs. Is the eggs facility intended only for these public libraries, or 
can it be used to track the dependencies of an application in 
development as well?

Docker container of Chicken's git master

[Lassi Kortela]

docker run -it schemers/chicken:head

Currently updated manually. The release containers also continue to be 
maintained:


docker run -it schemers/chicken:4
docker run -it schemers/chicken:5

Docker images updated

[Lassi Kortela]

Congrats on the new release.

We updated our Docker image to match:

$ docker run -it weinholt/chicken
CHICKEN
(c) 2008-2020, The CHICKEN Team
(c) 2000-2007, Felix L. Winkelmann
Version 5.2.0 (rev 317468e4)

There's now also a Chicken 4 image:

$ docker run -it weinholt/chicken:4
CHICKEN
(c) 2008-2017, The CHICKEN Team
(c) 2000-2007, Felix L. Winkelmann
Version 4.13.0 (rev 68eeaaef)

Re: Trying to understand chicken limitations

[Lassi Kortela]

There's a persistent myth among programmers that interpreters are slow.


I disagree that this is a myth.  Interpreted code is typically anywhere from 5x 
to 100x slower than compiled code depending on the features of the interpreter 
and quality of the compiled code.  The real issue is that developers are 
notoriously bad at predicting where performance matters in their code and in 
particular the impact of the interpreter's “slowness” on the overall 
performance of the application.  If the interpreted code is mainly ordering the 
execution of rather long-running operations that are well implemented in C then 
the slowness of the interpreter will not be observable by the user (for example 
typical shell scripts executed by an interpreter-based shell).


Your numbers sound correct. The myth is not about the slowdown factor; 
it's that the slowdown matters to users. Many (perhaps most) code is I/O 
bound so users don't perceive a difference. On the other hand, we easily 
perceive a difference in how fast the program loads when you click its 
icon or type its name in the shell (fancy runtimes have slower startup). 
Also, programmers notice a big difference in how fast they can build and 
run the program after each change they make. Programs that are easy and 
fast to build are fun to tinker with, which leads to more improvements.


There are still a lot of programmers who use languages like C++, Java, 
Haskell or Rust by default because they are perceived as "real 
languages" that run fast (even when their program is 1000 lines and I/O 
bound). These come with heavy toolchains which are difficult and slow to 
install and operate.


Emacs is a good example of a huge, mostly interpreted application whose 
interpreter is not stellar. And still almost none of the day-to-day 
slowdown when using Emacs is due to the interpreter; mostly you're 
waiting for some external program that is not responding.


Even an old version of Microsoft Excel was interpreted. The team had 
their own C compiler that generated bytecode instead of machine code. 
Joel Spolsky has a blog post about it.



But the performance of code execution by the embedded language matters for some 
applications and usually the developper only knows this late in the development 
process (after the application’s goals have evolved) which is after the 
embedded language implementation has been selected and much code written around 
it.  In such a situation it is usually too expensive to change the embedded 
language implementation, so instead more and more functionality gets 
implemented in the low-level language.

So code execution performance should not be overlooked when selecting an 
embedded language implementation, otherwise the development benefits of the 
high-level language may eventually be lost.


These are very good points. Indeed a good experience with interpreters 
requires moving performance-sensitive parts to C. Emacs is harmed by 
this as well.


Then again, using an exotic compiled language is certainly far from 
problem-free. Using C for the fast parts and a self-contained 
interpreter for the slow parts offers a nice mix of portability and 
expressive power. For example, it might make sense to rewrite Emacs in 
Common Lisp (since Emacs Lisp is basically a subset of CL already) but 
the result would probably be less portable, and there would be problems 
with CL implementations that they don't have with C. It's a trade-off.


The best of both worlds is a standardized language supported by many 
interpreters and compilers of different levels of sophistication - for 
example, a certain long-lived language in the Lisp family ;-) Then one 
can start with something simple and move up with a minimum of friction.


Lua is still a big inspiration to me as an interpreter that is actually 
as easy to drop into a C codebase as they advertise. TinyScheme and S7 
are currently the closest thing to Lua in the Scheme world. Other 
Schemes run code faster with more features and with ready-to-use 
libraries, but at a cost in simplicity.

Re: Trying to understand chicken limitations

[Lassi Kortela]

As for S7, I would use Chibi instead: it's more modern and probably faster.


It's a little-known fact that S7's author, Bill Schottstaedt still 
actively maintains it (https://ccrma.stanford.edu/software/s7/). S7 is 
the scripting language for three audio/music programs: Snd, Radium, and 
Common Music. No matter which Scheme implementation you end up using, 
you may be able to bounce ideas off Bill, or ask him about who you 
should talk to about Scheme and Max. He is from the academic music 
community; with luck, he may even know somebody who has already used 
Scheme with Max. The Snd subversion repository (at SourceForge, linked 
from that page) has some S7 Scheme code for various audio tasks. You may 
be able to repurpose it for Max, and with some work also for Scheme 
implementations other than S7.


S7 is a heavily modified version of TinyScheme, which itself is a 
heavily modified version of the venerable Mini-Scheme dating all the way 
back to 1989. Mini-Scheme is an awesome feat of C coding - a Scheme 
interpreter with proper lexical scope and tail calls in only 2000 lines 
of C. However, it would be too bare-bones for your needs. TinyScheme may 
be enough, but S7 would probably be better since it has already been 
used for music.


TinyScheme should be fully re-entrant so it can be used in a 
multi-threaded C program. All of the interpreter state is stored in one 
C struct; you can make as many interpreters as you like and run each one 
in a different thread. I'm not sure whether or not S7 is re-entrant.


IIRC, TinyScheme (or a slight derivative of it) is the current scripting 
language of the GIMP image editor.


All of these Schemes start up instantly and due to their simplicity are 
the easiest to embed into C with no external dependencies. Their 
internals are easy to modify as needed. But they are also the slowest 
interpreters available. They don't even have a bytecode compiler; they 
store your procedures as lists and recurse through the lists each time 
you call the procedure. Chicken, Chibi-Scheme, etc. have bytecode 
interpreters that are almost certainly faster if your Scheme code has 
loops with lots of iterations. However, they are also more complex.


There's a persistent myth among programmers that interpreters are slow. 
On the contrary, some interpreters are remarkably fast and even the slow 
ones are good enough for plenty of applications. The simplest 
interpreters use little RAM, start up instantly and load code instantly; 
these advantages are more important than run-time speed for lots of 
programs, but are almost always overlooked.


It may be a good idea to start with S7 or TinyScheme and switch to 
Chicken once you outgrow it. S7 and TS are both very self-contained and 
designed to be easily embedded in a C codebase with no dependencies, so 
they are easy to try; if they don't work for your use case, you won't 
have wasted a lot of effort. Basic Scheme code is quite portable so you 
won't have to make many changes if you switch implementations.


Finally, Chicken has the best community of any programming language 
implementation I've come across, and also has tons of libraries ("eggs") 
ready to use. It is also well documented. From that standpoint, it's by 
far the best Scheme for your use case.

Re: RAM usage of applications?

[Lassi Kortela]

I added (let loop () (loop)) after the (print "Hello world") so I could 
watch the process, and Windows is charging it with 2.6 MB.  I don't know 
how, if at all, Windows allocates DLL memory to applications with the 
DLL open.


The DLL file can be substantially larger than the runtime image if it 
contains debug information, PE resources or other things that are not 
loaded by default. Not sure whether that's the case here, or how to 
check it on Windows.

Re: RAM usage of applications?

[Lassi Kortela]

The Chicken DLL is about 3.4MB, of which a good deal will be relocation tables 
and such.  I'm a Cygwin user, so my C library is included in cygwin1.dll, which 
is about the same size [...]  The native Windows C library, also a DLL, is 
about 1MB, depending on the OS version [...]


So the RAM footprint at run time for the executable code (excluding the OS) 
will be on the order of 12KB + 3.4MB + 3.4MB + 1MB, so about 8MB.  I think that 
is a better measure when estimating the amount of RAM needed to run an 
application.


The RAM taken by the DLLs ought to be shared with other running programs 
using the same DLLs. If we assume the user's program is the only Chicken 
program running at the time, but there are other Windows/Cygwin 
programs, the Chicken program would be 12KB + 3.4MB on top of those.

Re: [Chicken-users] Snowtar C5 patch

[Lassi Kortela]

Additionally, if anyone knows of a mirror for scheme now! packages/code, I'd 
love to explore these and package some up for C5 (I'm awaiting on a response 
from the person who maintained the website, as far as I can tell).


Are you talking about ? It's maintained by Alex 
Shinn of Chibi-Scheme fame. He does reply to email, but he is quite busy 
so it may take a while.



Looking on web archive, there's some pretty fun and interesting sounding stuff 
in there. Maybe we could create a snow compatibility egg to avoid code 
duplication across snow eggs.


Again, not sure whether you are talking about . 
But if you are, all (or almost all) of that should be R7RS code, and 
should hence run in Chicken's R7RS mode. If a separate compatibility egg 
is needed, that's a sign of a problem that should be fixed upstream.


Some of the libs there contain Chibi-Scheme FFI stub files which would 
have to be ported to Chicken.


In case you are talking about the old Scheme Now! instead of snow-fort, 
IIRC most of those packages have been imported into snow-fort and/or 
some other, more modern package collection. Might want to diff the old 
snow and snow-fort package listings.


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Docker container for Chicken

[Lassi Kortela]

A prebuilt container based on Debian Linux is now available. To boot 
into csi:


$ docker run -it weinholt/chicken

To explore the container's Linux system:

$ docker run -it weinholt/chicken bash

The container's size is 87MB.

Dockerfile at .

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Changing the chickadee home page to C5

[Lassi Kortela]

I'd like to take this opportunity to thank the people who made Chickadee 
possible. It is a pleasure to use. We aspire to eventually index 
documentation for all Scheme standards, implementations and libraries in 
one place, and Chickadee is a big inspiration for the interface.


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] GraphQL libraries coming soon

[Lassi Kortela]

On 25.07.2019 10.36, Mark Janssen wrote:
I am currently working on a GraphQL PoC at work. The chicken deployment 
story combined with GraphQL suppport would be very nice.


Great, welcome aboard! It's always exciting to get things done faster.

1. Think about N+1 early, most real life projects will need some sort of 
dataloader. It's nice if it's integrated.


You have much more background for thinking about query performance than 
I do, so all suggestions are warmly welcome.


2. Allow a schema first approach. This helps with interop, I have seen 
dotnet graphql schemas which are easily recognizable as such.


I don't understand what any of that means :)

Current repo here: .

We should find some communication channels so we don't spam 
chicken-users too much. I suggest we talk on the schemeweb mailing list 
(https://srfi.schemers.org/scheme-lists-subscribe.html). It aims to 
develop a portable Scheme web stack, so a GraphQL implementation would 
fit right in. It's very low-traffic ATM, so maybe more traffic would 
inspire more people to hack on web projects :) I personally like GitHub 
issues as well. Private email is fine. I don't normally use IRC but I can.


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Status of srfi-64 egg

[Lassi Kortela]

Seems there used to be a srfi-64 egg ("A Scheme API for test suites"). 
But there's currently no page for it in the Chicken wiki - neither for 
Chicken 4 or Chicken 5. Does anybody know about its status, and whether 
it's being brought back to Chicken 5? I'll volunteer to help with this 
if needed.


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] GraphQL libraries coming soon

[Lassi Kortela]

We're exploring making a GraphQL API for the Schemedoc project 
( and 
 - new members welcome :)


Since we want to do as much as possible in Scheme, I'll write GraphQL 
client and server libraries to support the work (including an 
alternative S-expression syntax for GraphQL documents that we've been 
thinking about with John Cowan). The libraries are meant to be portable 
but initial experiments show that Peter's great web libraries make this 
easy on Chicken, so I'll start with a Chicken-native implementation and 
then gradually port it to other Schemes.


In case anyone wants to advise or to test this early for your own 
project, let me know. Otherwise I'll announce when some actually 
reusable code has been put together :)


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Easiest way to find origin repos of Chicken eggs

[Lassi Kortela]

Is there a global index (one file or web page) that links to the git/hg 
origin repo of every Chicken egg?


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Interaction of subprocess and condition handling

[Lassi Kortela]

Sorry, that code cleared the FD_CLOEXEC flag instead of setting it. 
Fixed code at  .


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Interaction of subprocess and condition handling

[Lassi Kortela]

Obviously in Chicken you would not exit the parent, but you'd exit the 
child with code 126 as here. In Chicken you'd probably want some fancy 
way to propagate the errno value from the child to the parent after a 
failed execve(). It's unwise to do something like _exit(errno); One 
solution would be to open a close-on-exec pipe from parent to child at a 
known fd number and pass the errno value via the pipe. The parent would 
probably have to poll() or select() on the pipe to check for error, but 
I haven't thought this through.
I tried that technique and it worked surprisingly easily! Source code, 
below, also downloadable at .


Beware that threads and signals may require extra caution! This test 
program only has one thread and I didn't think about signals at all. I 
also don't handle EINTR.


--

#include 

#include 
#include 
#include 
#include 
#include 
#include 
#include 

static void warn(const char *msg) {
fprintf(stderr, "%s\n", msg);
}

static void warnsys(const char *msg) {
fprintf(stderr, "%s: %s\n", msg, strerror(errno));
}

static void diesys(const char *msg) {
warnsys(msg);
exit(1);
}

static void die(const char *msg) {
warn(msg);
exit(1);
}

static void
set_close_on_exec(int fd)
{
int flags;

if ((flags = fcntl(fd, F_GETFD)) == -1) {
diesys("cannot get file descriptor flags");
}
flags &= ~FD_CLOEXEC;
if (fcntl(fd, F_SETFD, flags) == -1) {
diesys("cannot set close-on-exec");
}
}

static void spawn(char **argv) {
pid_t child;
int status, exec_errno;
int fds[2];
struct pollfd pollfd;
ssize_t nbyte;

if (pipe(fds) == -1) {
diesys("cannot create pipe");
}
set_close_on_exec(fds[0]);
set_close_on_exec(fds[1]);
if ((child = fork()) == -1) {
diesys("cannot fork");
}
if (!child) {
// We are in the new child process.
close(fds[0]);
execvp(argv[0], argv);
exec_errno = errno;
nbyte = write(fds[1], _errno, sizeof(exec_errno));
if (nbyte == (ssize_t)-1) {
warnsys("completely borked (child)");
} else if (nbyte != (ssize_t)sizeof(exec_errno)) {
warn("completely borked (child)");
}
_exit(126);
}
// We are in the old parent process.
close(fds[1]);
memset(, 0, sizeof(pollfd));
pollfd.fd = fds[0];
pollfd.events = POLLIN;
if (poll(, 1, -1) == -1) {
diesys("cannot poll");
}
exec_errno = 0;
if (pollfd.revents & POLLIN) {
nbyte = read(pollfd.fd, _errno, sizeof(exec_errno));
if (nbyte == 0) {
// We don't get any data, means the pipe was closed.
} else if (nbyte == (ssize_t)-1) {
warnsys("completely borked (parent)");
} else if (nbyte != (ssize_t)sizeof(exec_errno)) {
warn("completely borked (parent)");
}
}
if (waitpid(child, , 0) == -1) {
diesys("cannot wait for child process");
}
if (!WIFEXITED(status)) {
die("child process did not exit normally");
}
if (exec_errno != 0) {
errno = exec_errno;
warnsys("cannot execute command");
} else if (WEXITSTATUS(status) == 0) {
// Success, all good.
} else {
fprintf(stderr, "Child exited with code %d\n",
(int)WEXITSTATUS(status));
}
}

int main(int argc, char **argv) {
if (argc < 2) {
die("usage: spawn command [args...]");
}
spawn([1]);
return 0;
}

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Interaction of subprocess and condition handling

[Lassi Kortela]

I think you'd basically have to replicate the following C traditional 
programming technique in whatever mixture of Scheme and C is appropriate 
for Chicken. So when you run "./spawn date" it succeeds and prints 
output from the standard Unix "date" utility. But when you run "./spawn 
nonexistent" it writes an error message to stderr (the child process 
does the writing since it has access to the precise errno value after 
execve() whereas the parent does not, so the child can show a more 
precise error message). And the exit code from the parent is the nonzero 
code 1 to indicate error.


Obviously in Chicken you would not exit the parent, but you'd exit the 
child with code 126 as here. In Chicken you'd probably want some fancy 
way to propagate the errno value from the child to the parent after a 
failed execve(). It's unwise to do something like _exit(errno); One 
solution would be to open a close-on-exec pipe from parent to child at a 
known fd number and pass the errno value via the pipe. The parent would 
probably have to poll() or select() on the pipe to check for error, but 
I haven't thought this through.


Source code, also downloadable at :

--

// The child process uses the magic exit code 126 to indicate to the
// parent process that execve() failed. You could use another code,
// but IIRC this is the traditional choice.

#include 

#include 
#include 
#include 
#include 
#include 

static void warn(const char *msg) {
fprintf(stderr, "%s\n", msg);
}

static void warnsys(const char *msg) {
fprintf(stderr, "%s: %s\n", msg, strerror(errno));
}

static void diesys(const char *msg) {
warnsys(msg);
exit(1);
}

static void die(const char *msg) {
warn(msg);
exit(1);
}

static void spawn(char **argv) {
pid_t child;
int status;

if ((child = fork()) == -1) {
diesys("cannot fork");
}
if (!child) {
// We are in the new child process.
execvp(argv[0], argv);
warnsys("cannot exec child process");
_exit(126);
}
// We are in the old parent process.
if (waitpid(child, , 0) == -1) {
diesys("cannot wait for child process");
}
if (!WIFEXITED(status)) {
die("child process did not exit normally");
}
switch (WEXITSTATUS(status)) {
case 0:
// Success, all good.
break;
case 126:
// Exec failed. Child already printed error message. Since
// this we are the parent, we stay silent -- there's no need
// to write a duplicate error message. But propagate the error
// to the process that started _us_ by exiting with a nonzero
// code anyway.
exit(1);
default:
// Some other error.
fprintf(stderr, "Child exited with code %d\n",
(int)WEXITSTATUS(status));
break;
}
}

int main(int argc, char **argv) {
if (argc < 2) {
die("usage: spawn command [args...]");
}
spawn([1]);
return 0;
}


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Interaction of subprocess and condition handling

[Lassi Kortela]

OK, I perused the chicken-core git repo. posixunix.scm defines the 
'process' procedure. It's a wrapper for the 'process-impl' procedure in 
the same file. process-impl does a lot of setup, then:


  (chicken.process#process-fork
(lambda ()
  ;; ...set up input and output pipes...
  (chicken.process#process-execute cmd args env)))

And chicken.process#process-execute is also defined in the same file. I 
presume it's the execve() wrapper. The execve() error is handled in it 
as follows:


   (when (fx= r -1)
 (posix-error #:process-error 'process-execute
  "cannot execute process" filename))

If 'process-execute' is run from within 'process', then 
'process-execute' is running in the forked child process, and the above 
error is also signaled in that child process instead of the parent.


Since the code that caused the bug for me had my own 'condition-case' 
exception handler wrapped around the 'process' call, it would catch an 
exception in the child process, not in the parent.


And since normal control flow proceeds after a 'condition-case' 
exception handler has run, the forked child would continue running my 
Scheme program concurrently with the parent!


Probably you should change the implementation of 'process' so it calls a 
version of 'process-execute' that's specialized to run inside a child 
process forked with the sole purpose of "execve() or die trying" :)


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Interaction of subprocess and condition handling

[Lassi Kortela]

Thanks for helping out :)


*Normally*, closing both ports returned from "process" will do a
waitpid(2) (or the equivalent) and throw an exception if WIFEXITED
returns false.


So WIFEXITED checks for a normal exit (instead of being terminated by 
signal, e.g. segfault or kill -9). But what if the execve() itself fails 
so the forked child process doesn't get the chance to replace the 
Chicken process image with the other program that it was asked to start?


When coding in C, the forked child process is usually programmed to call 
_exit() in case the execve() fails. But _exit() causes a normal exit of 
the child process - it doesn't terminate abnormally by signal. This is 
tricky for the parent process, which can't tell the difference between a 
failed execve() and a successful execve() of another program that then 
chooses to exit with a non-zero exit code.


I'm under the impression that a special exit code (arbitrarily chosen 
magic number, often 126?) is usually reserved to indicate a failed 
execve(). What does Chicken do - if execve() fails, will the lingering 
Chicken child process call _exit() with a particular exit code to notify 
the parent? My test suggests that the child will keep running Scheme 
code instead of exiting. But I may have messed something up myself.


Sorry for not posting the verbatim source code of a program that 
reproduces the bug. Here is one:


 csc -R r7rs test.scm && ./test

(import (r7rs) (chicken process) (chicken process-context posix))

(define (show . xs)
  (for-each display xs)
  (newline))

(define (demo)
  (receive (from-child to-child child)
  (condition-case
  (process "does-not-exist-so-execve-will-fail" '())
(_ ()
   (show "exception handler entered in pid " (current-process-id))
   (values #f #f #f)))
(and child
 (let ((output (read-string #f from-child)))
   (close-port from-child)
   (close-port to-child)
   output

(show "subprocess result = " (demo))
(show "current-process-id = " (current-process-id))

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Interaction of subprocess and condition handling

[Lassi Kortela]

How do you handle errors signaled by the 'process' procedure from the 
(chicken process) module in Chicken 5?


When I try things like

  (receive (from-child to-child child)
  (condition-case
  (process command args)
(_ ()
   (values #f #f #f)))
...)

I get weird results - I have some 'display' calls, and according to 
'current-process-id' some of them are coming from a different PID (I 
assume 'process' does fork/exec internally, so somehow some of the 
display code actually ends up running in the forked child process.


I guess normally the unix execve() call succeeds so the child process 
stops running the Chicken programs and executes the desired command. But 
when execve() fails (e.g. due to a missing executable file) then the 
child process continues running Chicken code instead of exiting?


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Building the openssl egg on MacOS

[Lassi Kortela]

I maintain the openssl egg these days.


Very nice, thanks for your efforts!


Ugh.  I've always had the impression macOS gets worse with each release,
but this is ridiculous, almost as if they expect everyone to use XCode
for development...


I find that the usability gets better with each release (still dreading 
the inevitable downfall) but from a programmer's point of view it does 
indeed get worse. Mojave locked down some files so even with sudo you 
can't touch them, deprecated other stuf, and retroactively made it so 
you need the App Store login to install the full XCode.



The easiest workaround is to install a copy of OpenSSL or LibreSSL from the
popular Homebrew package manager and build the egg using that copy:


This is what I recommend to everyone who has to work on that kind of
system.  It's sad, but the least painful way of getting work done.


Cool, I didn't know that.

Homebrew is actually really nice to use, but your point stands about the 
state of MacOS itself.



I hereby reiterate my point that doing development on macOS involves
much sadness, such as creating a developer account to do development.
I'm afraid there isn't much else you can do, unless you somehow get gcc
and the rest of the toolchain working without that.


The command-line compilers (clang, swiftc) are available without an App 
Store login as before ("sudo xcode-select --install"). But the XCode IDE 
is apparently not. As for the OpenSSL headers, I don't know whether they 
are intentionally gone for good or only available from the App Store 
(and can't tell which is worse :)



would it make sense to add MacOS-specific checks to the build-openssl script?


Before you do that, there is some work I've done on a few more eggs I
maintain, I got fed up with writing user-unfriendly shell scripts that I
rewrote the non-Windows version to use a Scheme program instead


That sounds like a wise move. And for us who maintain Scheme-related 
infrastructure, it's nice to get the chance to actually write some 
Scheme code every now and then :) It so often happens that the C/shell 
parts take the most time.



basic version detection, falling back to environment variables and
finally bailing out with an error.  You can find the latest version of
it at the breadline repository [1].  Please let me know if that fulfills
your wishes and if not, whether it can be made to do so.  If yes, then
I'd be willing to migrate the openssl egg towards such a script as well.
The reason I haven't done so is because unlike the other eggs I maintain
it's something I'd rather not touch unnecessarily, breakages to it will
be far more annoying to handle than anything else.  And honestly
speaking, OpenSSL isn't nice to deal with either :>


Thank you for inviting feedback! I'll definitely help work things out.

I was thinking, even without considering OpenSSL at all, dealing with 
pkg-config by itself is complicated enough that it might be worth baking 
special pkg-config support into chicken-install (or whatever the right 
internal module is that chicken-install uses).


Some of the BSD's re-implemented the original glib-based pkg-config as 
"pkgconf". Luckily, pkgconf installs a pkg-config command as a 
compatibility wrapper and it seems quite feature-complete (in the same 
way that libressl is with openssl, or clang is with gcc). I haven't had 
any problems with pkgconf.


But several OSes don't ship with pkg-config by default, so users have to 
install it. In order to be user-friendly, it'd be nice if Chicken 
advised them on how to do it. (If Chicken has all the pkg-config 
business in one place, instead of separately in each egg, then it's a 
very modest amount of code to provide a user-friendly experience.)


I would tentatively recommend supporting pkg-config as the only way to 
specify compiler and linker flags for C libraries. I've always thought 
that users should have the option to give flags manually instead of 
relying on pkg-config. But today I realized that pkg-config .pc files 
are basically simple .ini files that contain little more than those same 
flags. So if users want manual overrides, we could just ask them to 
write their own .pc file and set PKG_CONFIG_PATH to find it.


Also .pc files specify the library version. E.g. the openssl egg 
requires openssl 1.0.2 or later, which it can check from the .pc file. 
(It's not reliable to check using the "openssl" shell command, since 
that may be a different version.) With user-supplied CFLAGS and LDFLAGS 
we can't easily make this version check. The .pc file is unlikely to 
have the wrong version number unless the user deliberately messes it up 
or makes up an arbitrary version number.


But is stuff detailed enough that it should be discussed on the 
"hackers" mailing list?


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Building the openssl egg on MacOS

[Lassi Kortela]

I spent a _little_ while digging into the details of this a few months
ago, and my understanding is this:

* Apple were suffering version-skew hell because (reportedly) the
OpenSSL folk kept changing the ABI for the library.
* So they lost patience and deprecated OpenSSL on macOS


Sounds quite plausible.


* ...to the extent that there are now compiler-visible deprecation
markers in the relevant still-visible header files.
* The doctrine is that one should use Apple's own encryption
frameworks (as you noted, Vasilij)
* (which work fine, in the pretty basic uses I've made of them, but
they don't pretend to be OpenSSL)
* ...and act as if there were no OpenSSL library at all on macOS.
* As Lassi noted, there is still a LibreSSL library on the system in
fact, but I believe it is intended to be strictly for legacy use.


This is the impression I got as well.

The thing is, not only are the Apple frameworks non-portable to any 
other OS but Apple keeps changing, deprecating and merging frameworks 
from release to release. They just deprecated OpenGL (and didn't adopt 
Vulkan) in favor of some brand new framework that doesn't work on any 
non-Apple OS! And Swift may hold the world record for the number of 
deprecation warnings in any tool. So Apple's own programming interfaces 
are some of the least stable ever :)



So if you have a tool which depends on Open/LibreSSL, then you need to
get it on your machine either from source, or using the package manager
of your choice, and not even try using the system one.


Based on the non-portability and difficulty of using Apple's frameworks, 
and their history of changing them endlessly, I suggest adopting a 
simple policy to direct all Mac users to Homebrew :)


Of SSL libraries, LibreSSL sounds like it will be the most stable and 
secure choice. The OpenSSL compatibility makes it extra nice.


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Mapping the idiosyncrasies of building the OpenSSL egg on many operating systems

[Lassi Kortela]

I investigated how to get openssl working on all the major BSD-derived 
operating systems. Here are the results: 



The shell script used and the build logs I got are here: 



Only the OpenBSD-packaged version of chicken-install (4.13.0) shows 
compiler warnings. It's also the only chicken-install I have that 
advertises in its help screen that it has a "-debug" flag (which I 
enabled for the build). Is there an equivalent flag to show compiler 
stuff in the never versions? The -verbose flag doesn't seem to do it.


Adding some flag(s) to chicken-install to easily get (mostly) 
reproducible builds would be a big help for testing like this. Currently 
I do "chicken-install; chicken-uninstall; chicken-install". The first 
install to ensure that all dependencies are installed; then uninstall to 
remove the package itself but keep all its dependencies installed; then 
the third install to get the build output from installing the package 
but not its dependencies. In 5.x versions, would


chicken-install -no-install -no-install-dependencies -purge openssl

do the trick?

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Building the openssl egg on MacOS

[Lassi Kortela]

On MacOS Mojave, "chicken-install openssl" fails because the OS doesn't 
ship any pkg-config definition file for its version of the openssl 
library. The pkg-config definition is supposed to be in a file called 
'openssl.pc' but 'sudo find / -name openssl.pc 2>/dev/null' turns up no 
such file for the OpenSSL that comes with the OS.


(In this version of MacOS the openssl library is actually the OpenSSL 
compatibility wrapper of the LibreSSL library: "/usr/bin/openssl 
version" says "LibreSSL 2.6.5". Even though the library is LibreSSL, 
it's still designed to ship with a file named 'openssl.pc' for 
compatibility with OpenSSL. But MacOS doesn't have that file.)


The error chicken-install gives is:

OpenSSL >= 1.0.2 seems to be unavailable

Using the helpful verbose mode of chicken-install I figured out that the 
source of the error is the pkg-config invocation in the shell script 
<~/.cache/chicken-install/openssl/build-openssl>.


The easiest workaround is to install a copy of OpenSSL or LibreSSL from 
the popular Homebrew package manager and build the egg using that copy:


brew install openssl
export PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig"
chicken-install openssl

Or:

brew install libressl
export PKG_CONFIG_PATH="$(brew --prefix libressl)/lib/pkgconfig"
chicken-install openssl

In principle one could use the openssl library that ships with MacOS to 
build the openssl egg. However, on this OS version I can't find the 
 C header file anywhere in the file system, even though 
the library itself is installed as . I installed 
Apple's command-line developer tools using "sudo xcode-select 
--install". As far as I can tell, I don't have the full GUI version of 
XCode anymore with this OS upgrade. I think the GUI version is still 
available free of charge but it may now require a Mac App Store login to 
install. From Chicken's point of view, we unfortunately can't assume 
that people who use Chicken have the full version of XCode.


Many/most MacOS users of intarweb might stumble onto this problem now 
that HTTPS websites are everywhere, so would it make sense to add 
MacOS-specific checks to the build-openssl script? Since it seems tricky 
to reliably find the system OpenSSL header files, maybe it should 
suggest that people use Homebrew as the easiest alternative. I can write 
and test a patch for the 'build-openssl' shell script if it helps (well, 
I already wrote most of it :-)


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users