Thanks for the initiative Harold!
I'm pressed for time, but quick comments:
One thing that's become clear in the discussions of auxiliary SRFIs
embedded in other official SRFIs is that testing in Scheme needs a bit
of work.
+1
It's also a special case for SRFIs, ideally the sample
implementations will always have a test suite, and for many SRFIs this
could be the only non-standard, higher numbered SRFI they depend on.
https://github.com/srfi-explorations/srfi-test is a repo I started a
while back to collect tests for as many SRFIs as we can in one place.
The tests in that repo use the SRFI 64 test definition primitives. Shiro
and Alex graciously donates many of their excellent tests under the MIT
License so the licensing situation for the repo can be as simple as the
situation with the SRFI documents themselves. Marc Feeley also donated
any tests in Gambit and is converting them piecemeal to SRFI 64.
The ultimate idea would be that the repo can have the test suites in a
format that Scheme implementations can simply copy those files verbatim
into their source trees. Doing whatever it takes to get them into a form
that implementors find agreeable. (include "...") in (define-library)
does wonders to enable portable .scm files. I haven't yet had the energy
to pursue this since there are so many things to do, but it'd be awesome
if you do. Aggregating the tests already uncovered a bunch of bugs; I
expect doing it more would uncover more and save everyone time in the
long run.
So in the spirit of other Scheme topics, and to limit the demands on
Arthur, I've created a GitHub Schemetest organization, into which we can
put repositories with testing libraries, documentation, examples, etc.
+1
Ideally it will in due course have a working SRFI 64 library for every
Scheme implementation people care about, and whatever other testing
libraries people like to use.
+1
Don't know if this warrants a Schemetest mailing list, but I'll throw
that idea out.
No opinion.
First question: how to subdivide the space of testing libraries and
Schemes they run on. A general repo, page, or file can be consulted to
tell you "if you want to use SRFI 64 in Chibi Scheme, then use /this/
library", rather than having a repo or sub-directory per implementation
with messy duplication. So ... no great reason not to do it in grab bag
fashion, after people's names or user ids, or whatever???
The situation basically is that 3 test definition frameworks dominate:
- SRFI 64
- Chicken's `test` egg <https://wiki.call-cc.org/eggref/5/test>
- The (chibi test) library
<http://snow-fort.org/s/gmail.com/alexshinn/chibi/test/0.9.0/index.html>,
a port of Chicken `test`.
All of these provide almost the same set of primitives for defining
tests. All of the main stuff is identical or near-identical. Therefore
the most viable strategy is to take this API as-is.
SRFI 64 defines two parts in one SRFI: the part where you define tests,
and the part where you run tests. Most of the reluctance to ship SRFI 64
with a Scheme implementation comes from the fact that you have to ship
the whole bundle, and some people want a differetn test runner than the
one in it.
The problem could be solved by writing a new SRFI that:
- copies (most of) the test definition parts from SRFI 64
- adds the new test definition stuff from the Chicken/Chibi test
library, which is like 1 or 2 new primitives IIRC.
- adds a couple more primitives that are none of those. Gambit needs one
to test whether an exception is raised in tail position or not.
Since the SRFI process is clogged, I suggest first writing a library
that does the above. Once the library has been used for a while,
immortalize its interface in a SRFI.
Here's a very recent GitHub issue on this topic:
<https://github.com/srfi-explorations/srfi-test/issues/3>, with many of
the usual suspects chiming in at length :)
Anyone out there interested in this who's already running a bunch of
different Scheme implementations so we can do some testing? If not,
I've got a bit of spare time to devote to that.
We have Docker containers for 45 Scheme implementations at
<https://hub.docker.com/u/schemers>, including a container build from
git master for many of the actively developed Schemes. I can give ssh
access to people I know from the lists to a Linux server with Docker
pre-configured.
