Updating Packages

[allan]

Hi there! I am new to software development, but I would like to contribute. I 
want to start by updating some rust packages to newer versions.

So far I have just been making scheme files in a seperate directory and running 
"guix build -f $SCHEME_FILE". I am trying to follow the [guidlines in the 
manual](https://guix.gnu.org/manual/en/html_node/Contributing.html), but git is 
still new to me and I am a little confused how to make this all work.

What should I do to get package builds to recognize the updates I make to 
dependencies?

Re: libvirt: A potential problem in the package and looking for guidance

[Allan Adair]

Thanks! Looks like I was on a good track. The only reference that I could
find to /usr/libexec/qemu-bridge-helper was in the meson.build file of the
libvirt package. I wrote a package variant that looks like this, to test:

(define-public libvirt-with-qemu-bridge-helper
  (package
(inherit libvirt)
(name "libvirt-with-qemu-bridge-helper")
(arguments
 (substitute-keyword-arguments (package-arguments libvirt)
   ((#:phases phases)
#~(modify-phases #$phases
;; libvirt needs to be able to find qemu's bridge helper
;; this is hacky, perhaps a patch on meson.build would be
better.
(add-after 'unpack 'find-qemu-bridge-helper
  (lambda* (#:key inputs #:allow-other-keys)
(let* ((qemu (assoc-ref inputs "qemu")))
  (substitute* "meson.build"
(("/usr/libexec/qemu-bridge-helper")
 (format #f "~a/libexec/qemu-bridge-helper"
qemu))
(inputs
 (modify-inputs (package-inputs libvirt)
(append qemu)

However the effort was fruitless, and my efforts on this are currently on
pause. If I am able to fix it I plan to submit a patch.

libvirt: A potential problem in the package and looking for guidance

[Allan Adair]

Hi Guix!

I am using a libvirt-service-type in my system configuration and it seems
to be working. I can launch VMs using virt-manager, for example. However I
have run into a potential problem, and I think it has to do with the
libvirt package.

Additionally, I would like to present a plea to others on this list that
perhaps someone could share their system configurations for libvirt-
service-type if they use it in a similar way that I am attempting.

My end goal is that I would like to be able to launch several VM guests,
for them to be able to see each other through a local network, and I would
like to be able to SSH from my my host to VM guests.

The configuration for my service-type might be relevant here, so I will
share it here. I don't think that "mdns-adv?" and "log-level" are
particularly relevant to my problem, just keeping it exactly as it is
defined on my system. My user is part of the "libvirt" group.

(service libvirt-service-type
  (libvirt-configuration
(unix-sock-group "libvirt")
(mdns-adv? #t)
(log-level 1)))

I am trying to do everything through qemu://session in user space, not
qemu://system. However it's not that big of a deal to use either. I just
haven't gotten either to work!

The first thing that I am trying to do is to SSH into a single running
guest from my host, and I have so far been unsuccessful.

So, the potential problem that I want to mention is that after having
poked around a bit and created the default network under a qemu://system
connection, a bridge device called "virbr0" is created.

And then for my guest VM which runs under a qemu://session connection, it
has a virtual network interface defined in XML. It was generated by the
virt-manager GUI:



  
  
  
  



When I try to restart the VM with this modification, I get the following
error:


Error starting domain: '/usr/libexec/qemu-bridge-helper' is not a suitable
bridge helper: No such file or directory

Traceback (most recent call last):
  File "/gnu/store/s15ahi35w0i274lima3dk41a7vr3kdbn-virt-manager-
3.2.0/share/virt-manager/virtManager/asyncjob.py", line 65, in cb_wrapper
callback(asyncjob, *args, **kwargs)
  File "/gnu/store/s15ahi35w0i274lima3dk41a7vr3kdbn-virt-manager-
3.2.0/share/virt-manager/virtManager/asyncjob.py", line 101, in tmpcb
callback(*args, **kwargs)
  File "/gnu/store/s15ahi35w0i274lima3dk41a7vr3kdbn-virt-manager-
3.2.0/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in
newfn
ret = fn(self, *args, **kwargs)
  File "/gnu/store/s15ahi35w0i274lima3dk41a7vr3kdbn-virt-manager-
3.2.0/share/virt-manager/virtManager/object/domain.py", line 1329, in
startup
self._backend.create()
  File "/gnu/store/g2x37cxh2ag5h66f0p9zaz9pkz2vcvgg-python-libvirt-
7.9.0/lib/python3.9/site-packages/libvirt.py", line 1353, in create
raise libvirtError('virDomainCreate() failed')
libvirt.libvirtError: '/usr/libexec/qemu-bridge-helper' is not a suitable
bridge helper: No such file or directory


Because the package is searching specifically for "/usr/libexec/qemu-
bridge-helper", I think this is a bug in the package definition. "/usr"
should probably be a prefix from the store, right?

Any comments or suggestions?


Thanks in advance for a response,

Allan

Re: How to find binaries in libexec dir?

[Allan Adair]

Hi!

I'm not sure if the following suggestion is a canonical way of doing
things (I'm a scheme newb), but it is possible to define a package variant
of diffoscope with a G-expression that uses wrap-program (see
guix/build/utils.scm) to include paths to dependencies as part of an
environment variable.

From the docstring:

"This is useful for scripts that expect particular programs to be in
$PATH, for programs that expect particular shared libraries to be in
$LD_LIBRARY_PATH, or modules in $GUILE_LOAD_PATH, etc."

As Liliana said, it is possilbe to iterate over package-inputs to compute
the paths in the package definition of a diffoscope variant. So until the
problem with xb-tool is fixed, maybe this is a good way to go?


signature.asc
Description: This is a digitally signed message part

Re: Guix on macOS

[Christopher Allan Webber]

Ricardo Wurmus writes:

> Ludovic Courtès  writes:
>
>> Windows recently gained an in-kernel Linux syscall emulation, which
>> means that (GNU/)Linux binaries can run unmodified on Windows.
>>
>> If macOS had a similar feature, that’d be perfect: we wouldn’t have
>> anything to do.  Perhaps Docker-for-Mac actually provides something
>> close to that?  I really don’t know.
>
> IIUC, Docker for Mac uses macOS virtualization libraries.  Previously,
> it would require an installation of VirtualBox, but this is no longer
> the case.

Oh, I see you already replied with essentially the same message I
replied to you with... :)

So effectively "if it is (or was) good enough for Docker, why couldn't
this be good enough for us as a path forward?"

Re: Guix on macOS

[Christopher Allan Webber]

Ricardo Wurmus writes:

> Christopher wrote this:
>
>> Is there a way to maybe run Guix in some sort of namespaced or some
>> variant of "virtualized" or "contained" way that we could recommend for
>> OSX users, without having to bend over backwards to accomodate a
>> different libc and etc?
>
> In order to use the Hypervisor framework that macOS 10.10 and higher
> provide we would need to use Xcode, so we couldn’t even build a tool
> like that without relying on proprietary software.  But we don’t have to
> build a tool like that, because it already exists in the form of virtual
> machine applications (or even Docker for Mac).
>
> The best we could do on macOS is to run applications in a GNU virtual
> machine and try hard to make them blend in.  That’s not really
> appealing, neither technically nor practically, in my opinion.

I think someone told me this is exactly how Docker does it, or used to
do it, for quite some time.  So we'd hardly be alone... :)

Re: Guix on macOS

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hi Chris,
>
> Chris Marusich  skribis:
>
>> I want to get Guix working on macOS.  I recently had a need to do this,
>> and I was sad to find that although Nix works on macOS, Guix isn't quite
>> there yet.  The manual makes it sound like this should be fairly
>> straightforward, and I intend to give it a shot [1].  But before I
>> begin, I wanted to know: has anyone done this already?  Is there
>> interest?  I've checked the email lists, and I didn't find much
>> discussion about this.
>
> First of all, it’s never been a goal of Guix to run on non-GNU systems.
> Now, I have nothing against it in principle, as long as (1) this can be
> achieved in a maintainable way, and (2) the targeted user-land software
> is free and buildable from source.
>
> I suspect macOS fails criterion #2.  Back in the day (not sure if that’s
> still the case), Nix would bootstrap using the system’s compiler and C
> library (which meant that things were likely to break in subtle ways on
> macOS upgrades.)
>
> As for criterion #1, to me, that pretty much means sticking to the GNU
> libc.  From my experience on Nixpkgs, having to deal with different C
> libraries is a real burden.  It also leads to a situation where you have
> second-class systems because they use an alternate libc and it’s not
> uncommon for packages to fail to build against that libc.  To put it
> differently: it’s already difficult enough to have *one* OS working.
>
> I’m afraid this is not the answer you were looking for.  WDYT?
>
> Ludo’.

Is there a way to maybe run Guix in some sort of namespaced or some
variant of "virtualized" or "contained" way that we could recommend for
OSX users, without having to bend over backwards to accomodate a
different libc and etc?

 - Chris

Re: Scope of support for Guix on other distros

[Christopher Allan Webber]

Mark H Weaver writes:

> Hi Chris,

Hi Mark,

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Mark H Weaver writes:
>>
>>> Tobias Platen <trisq...@platen-software.de> writes:
>>>
>>>> On 02.10.2017 20:52, Christopher Allan Webber wrote:
>>>>> Since these don't provide Guix in the main repo (and Debian won't
>>>>> because we violate the FHS with /gnu/) we could probably auto-generate
>>>>> the .deb or .rpm from some gexp? 
>>>>> 
>>>> Debian also includes GNUstep which lives in /usr/gnustep. Maybe /gnu
>>>> could be moves to /usr/gnu on Debian.
>>>
>>> All of our binary substitutes are built with a store prefix of /gnu.
>>> Using a different store prefix requires bootstrapping the entire system
>>> from source code, once per core-updates cycle, and also doing frequent
>>> rebuilds of upper layers of the stack to keep up with the updates on our
>>> master branch.  Most users won't want to do this.
>>
>> I still think a hilarious option would be to "graft" every output to
>> /usr/gnu or /opt/gnu but I think nobody else is on board with this ;)
>
> I've had similar thoughts in the past, but upon further reflection, it
> would raise serious technical difficulties.
>
> So far, grafting only attempts to change individual components of file
> names (i.e. between two '/'s), which is prudent because it's entirely
> possible that some software stores the components separately, or uses a
> different directory separator internally.  I don't know how much of a
> problem this would be in practice, but stranger things have happened,
> e.g. <https://bugs.gnu.org/24703>.
>
> If we neglect that issue, the next problem is that grafting obviously
> cannot change the length of file names embedded within executables and
> data files of arbitrary format, so in order to add 4 characters to the
> beginning of file names, we'd have to discard 4 characters from
> somewhere else.
>
> Discarding 4 characters from the hash would seem to be the obvious
> choice, but that would entail making it about a million times easier to
> generate a hash collision.  If someone can successfully cause a hash
> collision, that could be used to breach the security of the system.
> There's also the fact that quite a bit of code in Guix assumes a fixed
> hash length, and it would take some effort to lift that limitation.
>
> The safer approach would be to discard 4 characters from the
> human-readable part of the file names (i.e. the package name and/or
> version number), which would obviously hinder readability.
>
> This would be a lot of fuss for very little benefit, in my opinion.  The
> sole benefit would be to allow Debian users to install a (very old)
> version of Guix from Debian's own repository instead of having to use
> our own (up-to-date) .deb file.

That's an interesting summary of things.  Okay, I'm convinced the "graft
our problem away" route won't work here.

> As a long-time Debian user myself, I can very much sympathize with the
> desire to avoid using outside repositories.  My main reason for this is
> that on a Debian system, I've already put full trust in the Debian
> developers and infrastructure, and I'd rather avoid trusting anyone else
> if I can avoid it.  However, that reason is not applicable here, because
> it's not possible to use Guix without putting tremendous trust in the
> Guix developers, regardless of where the original .deb came from.
>
> If we were to try to add a package to Debian, it might be better to add
> a package that downloads, authenticates, and installs our latest binary
> installer.  Most importantly, this would allow us to leverage Debian's
> existing infrastructure to securely distribute a copy of our signing key
> to new users, and allow slightly more convenient install without
> condemning users to starting with an ancient version of Guix.
>
> What do you think?

Hm.  I know I personally am wary of packages that are there just to be
installers for other packages (even though in effect if you do a guix
pull with a base package, you're kind of doing that anyway).  I'd
suspect we'll get more mileage at this point by just generating .deb
files and hosting them on our own site.

Thanks for the insightful comments!

Re: Scope of support for Guix on other distros

[Christopher Allan Webber]

Ricardo Wurmus writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Since these don't provide Guix in the main repo (and Debian won't
>> because we violate the FHS with /gnu/) we could probably auto-generate
>> the .deb or .rpm from some gexp?
>
> I was thinking about adding support for the “deb” package format to
> “guix pack”.  We can already create fat tarballs, so it shouldn’t be too
> much effort to create fat Debian archives.

I think this would be a huge win!

Re: Scope of support for Guix on other distros

[Christopher Allan Webber]

Mark H Weaver writes:

> Tobias Platen <trisq...@platen-software.de> writes:
>
>> On 02.10.2017 20:52, Christopher Allan Webber wrote:
>>> Since these don't provide Guix in the main repo (and Debian won't
>>> because we violate the FHS with /gnu/) we could probably auto-generate
>>> the .deb or .rpm from some gexp? 
>>> 
>> Debian also includes GNUstep which lives in /usr/gnustep. Maybe /gnu
>> could be moves to /usr/gnu on Debian.
>
> All of our binary substitutes are built with a store prefix of /gnu.
> Using a different store prefix requires bootstrapping the entire system
> from source code, once per core-updates cycle, and also doing frequent
> rebuilds of upper layers of the stack to keep up with the updates on our
> master branch.  Most users won't want to do this.

I still think a hilarious option would be to "graft" every output to
/usr/gnu or /opt/gnu but I think nobody else is on board with this ;)

Re: Scope of support for Guix on other distros

[Christopher Allan Webber]

ng0 writes:

> Christopher Allan Webber transcribed 0.6K bytes:
>> Konrad Hinsen writes:
>> 
>> > On 02/10/2017 11:18, David Seaward wrote:
>> >
>> >> To what extent is support for other distros a priority for the Guix
>> >> project? In other words, explicitly planning to make Guix available as
>> >> an alternate installation source on non-Guix-SD distros.
>> >
>> > That's already possible right now. Go to the download page and check for 
>> > "GNU Guix 0.13.0 Binary". This is Guix for other Linux distros, as a 
>> > look at the installation instructions will confirm. I am using this on a 
>> > Ubuntu 16.04 LTS installation on my laptop.
>> >
>> > Konrad.
>> 
>> It would still be nice if we provided .deb/.rpm/etc files on that page.
>
> By my own observation, systems including Guix either officially
> or by third-party / community methods:

Cool!

> Archlinux: https://aur.archlinux.org/packages/guix/
> Gentoo: https://packages.gentoo.org/packages/sys-apps/guix

These provide Guix in their own upstream repository so I think it would
be good to mention on the Download page.

> Debian: from past discussion and on request from Whonix iirc it is currently
> not possible due to Debian Packaging Standards (expected package
> behavior) or something along the lines, see guix-devel archives.
> Fedora: https://copr.fedorainfracloud.org/coprs/lantw44/guix/
> Slackware: https://slackbuilds.org/repository/14.2/system/guix/
>is on 0.12, needs an update. Any slacker up for that task?
>Otherwise, ping the maintainer:
>> Maintained by: Hunter Sezen

Since these don't provide Guix in the main repo (and Debian won't
because we violate the FHS with /gnu/) we could probably auto-generate
the .deb or .rpm from some gexp?

Re: Scope of support for Guix on other distros

[Christopher Allan Webber]

Konrad Hinsen writes:

> On 02/10/2017 11:18, David Seaward wrote:
>
>> To what extent is support for other distros a priority for the Guix
>> project? In other words, explicitly planning to make Guix available as
>> an alternate installation source on non-Guix-SD distros.
>
> That's already possible right now. Go to the download page and check for 
> "GNU Guix 0.13.0 Binary". This is Guix for other Linux distros, as a 
> look at the installation instructions will confirm. I am using this on a 
> Ubuntu 16.04 LTS installation on my laptop.
>
> Konrad.

It would still be nice if we provided .deb/.rpm/etc files on that page.

Re: Status of "GuixOps"?

[Christopher Allan Webber]

Hartmut Goebel writes:

> Hi,
>
> in Ludo's presentation at GHM he presented "GuixOps" on a slide. What is
> the status of this approach? I'm very interested in trying it out and
> contributing.
>
> I contributed to DebOps when it was "young". So my point of view is
> influenced by how DebOps works. DebOps is a collection of interoperating
> role/recipes for Ansible. Debops has become quite complex and I would
> like to migrate to GuixSD for new systems.
>
> Q1: I did not follow the development closely, but I seem to recall that
> there is some guix sub-command for configuring a remote system. But
> grepping the manual for "remote", I did not find it, neither one of the
> commands did attract me. How is it called?

There's a vey out of date branch on git origin called wip-deploy.
It needs a lot more work!

> Q2: DebOps has some tooling to securely store credentials, certificates,
> etc. It uses a gpg-encrypted container which is mounted using FUSE. When
> I unlock this container, the appropriate data is transferred to the
> target system. How can this be handled with GuixSD? AFAIU with GuixSD
> all data in the system-configuration is world-readable in the store. So
> how can I automatically transfer e.g. passwords and private keys the the
> target system?

Not sure the right answer for this one :)
But the right system might be user-hackable since Guix is Just Scheme (TM)?
Probably the right route is to remote-copy the files while pushing the
new state of the system over.  Maybe having a loopback device with that
data mounted in it is indeed a good idea, I don't know.

> Q3: One of DepOps' main features for me is easy use and the automatic
> refresh of Let's Encrypt certificates. Basically I just say: "Create
> certificates for hostnames A, B, C" and everything happens
> automatically: Configuration of nginx, creating the CSR, requesting the
> certificate, renewal, etc. What is the status for something like this
> for GuixSD?

There's a wip-lets-encrypt branch on origin too!  In fact I'm using it
on a server!

I'd really like to work on guix-deploy but I won't be able to until next
year.  It sounds like you have experience hacking similar systems; maybe
look at wip-deploy and read David Thompson's old thread about it?  (I'm
too tired to look it up...)

Happy hacking!  I'm off for happy sleeping. :)
 - Chris

Re: Removing the need for ./configure --localstatedir=/var ?

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Every now and then I rebuild Guix and hit the problem where I run
>> ./configure and then I am reminded that I need to instead do:
>>
>>   ./configure --localstatedir=/var
>>
>> IIRC this is because of GNU standards about default paths and pretty
>> much no other reason that we don't default to /var.  But I think Guix is
>> a special case.
>>
>> Should we ask maybe for an exception?
>
> No need to ask.  :-)
>
>> There's no reason for users to bump into this.  It's annoying even for
>> me, and at least I know (or think I do) why.
>
> I think it would be surprising to users who install from source if
> localstatedir is not under $prefix by default (someone who wants to
> install to /opt/foo would be surprised to find out that some files go to
> /var instead of /opt/foo/var.)
>
> There’s also the question of “make dist”, which should still DTRT.
>
> We could have $prefix default to /… but that’s not quite what one might
> expect either.
>
> I think the current situation is a good middle ground, even though I can
> also sympathize with your frustration.
>
> Thoughts?
>
> It’s weird how a simple question like this can be seemingly this complex
> to address.  :-)
>
> Ludo’.

Okay!  If you feel confident it's the right thing, then I'm not going to
argue about it.  The warning helps... it's just a little bit annoying :)

Removing the need for ./configure --localstatedir=/var ?

[Christopher Allan Webber]

Every now and then I rebuild Guix and hit the problem where I run
./configure and then I am reminded that I need to instead do:

  ./configure --localstatedir=/var

IIRC this is because of GNU standards about default paths and pretty
much no other reason that we don't default to /var.  But I think Guix is
a special case.

Should we ask maybe for an exception?  There's no reason for users to
bump into this.  It's annoying even for me, and at least I know (or
think I do) why.

 - Chris

Re: News from the GNU Hackers Meeting

[Christopher Allan Webber]

Thanks for the update!  Truly wish I was there.

Ludovic Courtès writes:

> Our next meeting point will be FOSDEM in February I guess.
>
> Until then, happy hacking!

Hopefully I can make that, if I can put up patience with the growing
unpleasantness of international border crossing!

Re: The future of 'guix environment'

[Christopher Allan Webber]

Thompson, David writes:

> Ricardo, you are correct that we would lose the ability to use the
> guix.scm file for both 'guix environment' and 'guix build'.  In
> practice I don't actually use my guix.scm file this way, so I think
> it's worth breaking, but maybe you (or someone else) actually uses
> this and we should think more about it?

I do use guix.scm for testing `guix build'.  Maybe I ma the only one
though.  It turns out to be very useful to find out whether or not I've
done something that borked my package that's not obvious because I have
some compiled .go file around in working directory or something.

> I wasn't very clear about whether ephemeral or cached would be the new
> default.  I don't think there is one default for all cases, I think
> it's more context sensitive.

I'm a bit confused by caching, but IMO maybe the right solution is
actually guix environment for "environment profiles".  Ie, I want to be
able to update my guix environment and if it turns out I made a mistake
easily roll back, just like I can with my user's profile.  Currently
you can use --root, but you don't have the generations support.

Re: The future of 'guix environment'

[Christopher Allan Webber]

Not going to reply in detail, but I'm a devoted user of `guix
environment' and I support all these changes!

Re: Using IPFS to host mirrors of source packages

[Christopher Allan Webber]

Pjotr Prins writes:

> By hosting source packages on IPFS they become content addressable and
> should scale for downloads. It would also become a safe way of
> distributing data.
>
> https://github.com/ipfs/ipfs
>
> Essentially the interplanetary file system allows for distributed
> storage servers and a distributed DNS style resolution for finding
> files. It allows people to share storage and by pooling storage we
> can provide resilient access to all source packages. Especially the
> ones that prove dodgy now.
>
> This should be fairly trivial to implement as long as some of us can
> commit some storage. We could use the build farm.
>
> Next we can do the same for binary Guix packages. It would make for
> way faster downloads when the Guix-deamon supports the protocol.
>
> Can we look into this? Who of us can host IPFS and expose port 4001? I
> can put in some.
>
> Pj.

I like the idea... a lot!

Though I think we'll have to address, how do we *get* the packages from
IPFS?  Do we use an http gateway (and whose)?  It would be even better
if by using Guix you automatically could enable contributing to the IPFS
pool, but I guess that would require a) having an IPFS client in Guile
and b) having the Nix daemon replaced with Guile to do efficiently.
(But maybe I'm overthinking it?)

 - Chris

Re: guile-git trouble (building from source)

[Christopher Allan Webber]

Pjotr Prins writes:

>> But yeah, the problem remains for someone building from source.  :-/
>
> I am facing issues with guile-git. 
>
> I wish we had a bullet proof way of installing from source. I had it
> with guix environment and now that won't work because guile-git is
> missing and I can't bootstrap...
>
> Pj.

Save us, content addressed storage! :)

Re: [PATCH 001/303] gnu: Add opencolorio.

[Christopher Allan Webber]

ng0 writes:

> Christopher Allan Webber transcribed 9.3K bytes:
>> I have a friend who's a Blender user who said they'd like to see
>> opencolorio support in the Blender package... this is an old patch,
>> but maybe worth trying to get in again?
>
> Was this directed specifically at myself, or just a bump for the thread?
> I welcome any further work on this, but I'm not able to do this
> currently or in a foreseeable future.

Directed at whoever has the time :)

Re: [PATCH 001/303] gnu: Add opencolorio.

[Christopher Allan Webber]

I have a friend who's a Blender user who said they'd like to see
opencolorio support in the Blender package... this is an old patch,
but maybe worth trying to get in again?

ng0 writes:

> From: ng0 
>
> * gnu/packages/graphics (opencolorio): New variable.
> * gnu/packages/patches/opencolorio-avoid-bundled-includes.patch
> ---
>  gnu/local.mk   |  1 +
>  gnu/packages/graphics.scm  | 73 
> ++
>  .../opencolorio-avoid-bundled-includes.patch   | 72 +
>  3 files changed, 146 insertions(+)
>  create mode 100644 
> gnu/packages/patches/opencolorio-avoid-bundled-includes.patch
>
> diff --git a/gnu/local.mk b/gnu/local.mk
> index a7006cb..bc52f1a 100644
> --- a/gnu/local.mk
> +++ b/gnu/local.mk
> @@ -701,6 +701,7 @@ dist_patch_DATA = 
> \
>%D%/packages/patches/nvi-db4.patch \
>%D%/packages/patches/ocaml-CVE-2015-8869.patch \
>%D%/packages/patches/ocaml-findlib-make-install.patch  \
> +  %D%/packages/patches/opencolorio-avoid-bundled-includes.patch \
>%D%/packages/patches/openexr-missing-samples.patch \
>%D%/packages/patches/openjpeg-CVE-2015-6581.patch  \
>%D%/packages/patches/openjpeg-CVE-2016-5157.patch  \
> diff --git a/gnu/packages/graphics.scm b/gnu/packages/graphics.scm
> index d0df830..1f04da2 100644
> --- a/gnu/packages/graphics.scm
> +++ b/gnu/packages/graphics.scm
> @@ -5,6 +5,7 @@
>  ;;; Copyright © 2016 Ricardo Wurmus 
>  ;;; Copyright © 2016 Efraim Flashner 
>  ;;; Copyright © 2016 Andreas Enge 
> +;;; Copyright © 2016 ng0 
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -46,18 +47,21 @@
>#:use-module (gnu packages compression)
>#:use-module (gnu packages multiprecision)
>#:use-module (gnu packages boost)
> +  #:use-module (gnu packages ghostscript)
>#:use-module (gnu packages gl)
>#:use-module (gnu packages glib)
>#:use-module (gnu packages graphviz)
>#:use-module (gnu packages gtk)
>#:use-module (gnu packages gnome)
>#:use-module (gnu packages image)
> +  #:use-module (gnu packages java)
>#:use-module (gnu packages jemalloc)
>#:use-module (gnu packages photo)
>#:use-module (gnu packages python)
>#:use-module (gnu packages qt)
>#:use-module (gnu packages readline)
>#:use-module (gnu packages sdl)
> +  #:use-module (gnu packages serialization)
>#:use-module (gnu packages video)
>#:use-module (gnu packages xml)
>#:use-module (gnu packages xorg))
> @@ -510,3 +514,72 @@ and understanding different BRDFs (and other component 
> functions).")
>  It supports sub-pixel resolutions and anti-aliasing.  It is also library for
>  rendering SVG graphics.")
>  (license license:gpl2+)))
> +
> +(define-public opencolorio
> +  (package
> +(name "opencolorio")
> +(version "1.0.9")
> +(source (origin
> +  (method url-fetch)
> +  (uri (string-append 
> "https://github.com/imageworks/OpenColorIO/;
> +  "archive/v" version ".tar.gz"))
> +  (sha256
> +   (base32
> +"0zzacmfs4k6lk4yd9v8f6p1616k7sm5wmhk0np93qx8m3ilixj17"))
> +  (patches (search-patches
> +"opencolorio-avoid-bundled-includes.patch"))
> +  (file-name (string-append name "-" version ".tar.gz"
> +(build-system cmake-build-system)
> +(arguments
> + `(#:configure-flags
> +   (list "-DOCIO_BUILD_JNIGLUE=OFF" "-DOCIO_BUILD_SHARED=ON"
> + "-DOCIO_BUILD_NUKE=OFF" ; Nuke is commerical software
> + "-DOCIO_BUILD_STATIC=OFF" "-DOCIO_STATIC_JNIGLUE=OFF"
> + "-DOCIO_BUILD_TRUELIGHT=OFF" ; Another external software
> + "-DUSE_EXTERNAL_LCMS=ON" "-DUSE_EXTERNAL_TINYXML=ON"
> + "-DUSE_EXTERNAL_YAML=ON" ;;"-DOCIO_BUILD_DOCS=ON"
> + "-DOCIO_BUILD_APPS=ON" ;;"-DOCIO_BUILD_PDF_DOCS=ON"
> + "-DOCIO_BUILD_PYGLUE=ON")
> +   ;;"-DOCIO_BUILD_TESTS")))
> +   #:phases
> +   (modify-phases %standard-phases
> + (add-before 'configure 'fix-fixes
> +   (lambda _
> + (substitute* "docs/CMakeLists.txt"
> +   (("add_dependencies(doc Sphinx)") "")
> +   (("add_dependencies(pdf latex)") "")))
> +(inputs
> + `(("lcms" ,lcms)
> +   ("openimageio" ,openimageio)
> +   ("glew" ,glew)
> +   ("freeglut" ,freeglut)
> +   ("python" ,python)
> +   ("yaml-cpp" ,yaml-cpp)
> +   ("yaml" ,yaml)
> +   ("boost" ,boost)
> +   ("tinyxml" ,tinyxml)
> +   ("python-sphinx" ,python-sphinx)
> +   ("mesa" ,mesa)
> +   ("libxi" ,libxi)
> +   ("libxmu" ,libxmu)
> +   ("pkg-config" ,pkg-config)
> +   ("python" ,python)

Re: Mes 0.9 released

[Christopher Allan Webber]

Jan Nieuwenhuizen writes:

> I am pleased to announce the release of Mes 0.9, representing 107
> commits over 5 weeks.  Mescc now compiles to a surprisingly readable
> stage0 M1 macro assembler output format.  Also, mescc can now compile a
> modified TinyCC into a running [mostly segfaulting] executable.  This is
> a major milestone as tcc can compile GCC.

Amazing stuff, Jan!  You are doing the work of saints.

Re: New git signing key for Jan Nieuwenhuizen?

[Christopher Allan Webber]

Jan Nieuwenhuizen writes:

> Mark H Weaver writes:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
>>> commit ebfb71d45615698040818a68b7dc34996ff4c046
>>> gpg: Signature made Fri 16 Jun 2017 07:22:49 AM EDT
>>> gpg:using RSA key DB34CB51D25C9408156FCDD6A12F87978D701B99
>>> gpg: Can't check signature: No public key
>>> Author: Jan Nieuwenhuizen 
>>> Date:   Thu Jun 15 19:22:00 2017 +0200
>>>
>>> tests: Allow setting of qemu memory-size for system tests.
>>>
>>> * gnu/system/vm.scm (common-qemu-options): Remove hardcoded "-m 256".
>>> (system-qemu-image/shared-store-script): New keyword argument: 
>>> #:memory-size.
>>
>> This commit was signed using a different key than the one you've been
>> using for your other commits, and different from the one you have
>> registered on Savannah.
>>
>> Is DB34CB51D25C9408156FCDD6A12F87978D701B99 your key?
>
> Yes, that's my new key.  Oops.  I have updated my key on savannah.
> janneke
> -BEGIN PGP SIGNATURE-
>
> iQIzBAEBCAAdFiEE2zTLUdJclAgVb83WoS+Hl41wG5kFAllGFv4ACgkQoS+Hl41w
> G5mcHg/+Pd3AvBLSmoPUrXrhFcFrhQyVdq/d6f2R0JiSkrPy402P003Bp6pzEdD8
> OxQzT/ahk7XYRrD0KywCzn6Vd3b7xd+pnBzAJ3TV7lzyXKg97qm6CAAMEdaLNd/3
> 18dKOUmGI5oTRAvTc7Ubdwc4E1qmI4ckBT3QW2LBDOzPaMY6wiLgIA8vETN2bq+S
> vIMN9XDQRbtdwwA+k2fXQUB8b5N+bWdc1/YH5DmuqhKUiaHVCHHBc2K5L8YEydPr
> 0o+MrbwpmlxXjmKQTYFX8L5Wc1DARWECbPIa70TB53uWIxSQdfQQha/Oqd1fMYU0
> H9jPBF0PJHjJMyrTwmvMrZFC9Xi4u3UnKdCOPOCz2DXTttOGsGRsP16so34e9UzR
> DyLD7kR9K/m5djxSYLcIr8mMAfPJ0g/mk1Jyj5Yv9MZVCmFggcfeamirgcvnClxe
> Nhkfyf7SPeq/qpKIVpIOabthtuuoYiH2imnlLCL+TS6qGZYtBkhxlKUjRYMFWm+F
> bTwo3c+FdwOeyWdKn4a6QwIzfaJ70o1EeIYS4MUCyB1TKzcsJWE/t5R0SOqz1SX5
> ofLsn8amwE0dnoCKQhX92sse+sq4ITj3oD5YpehBl4pvzeIs+7LpLHJSVDSFRfwT
> N1Xy/PscHohFgcbfD8dZn/xtxRTN8dz1ysrhwLRBl8zc/tRKYPA=
> =xmQy
> -END PGP SIGNATURE-

Though I didn't confirm in-person, Janneke and I have had other
interactions using this key to make me reasonably confident it's their
key.



signature.asc
Description: PGP signature

Re: User-Friendlyness of Guix and non-scaryness, printing messages

[Christopher Allan Webber]

Arun Isaac writes:

>> But I think 'guix package' can be made a lot more user-friendly by not
>> showing the build output (by default).  The summary of what will be
>> installed, and some progress indicator or at least something moving to
>> show the user it's doing something, and eventually the success or
>> failure message is enough.
>
> Is it possible to have `guix package' NOT build from source? I really
> hate it when big packages like icecat, epiphany, or libreoffice start
> building on my machine. It takes about a day for each one of these
> packages to build on my machine, and they generally swap so much into
> HDD, that I can hardly do anything else when building. Because of this
> inconvenience, I keep postponing upgrading my packages.
>
> Also, sometimes having to "--fallback" to building source is such a
> pain. I think the end user needs a Debian-like experience (or like other
> conventional non-source distros) where they can ALWAYS download the
> latest packages from the repos.

There's a bug I opened about adding an --only-substitutes option:
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26608

Ludo seems to support it, so I guess really someone just needs to
implement it.  It would result in a lot less churn for a lot of us :)

"your installation is old" when running via a git checkout

[Christopher Allan Webber]

Hi!  I've done this hack so I can run the Guix I'm hacking on from git
as I know some others are as well:

  ln -s ~/devel/guix ~/.config/guix/latest

However, I am getting:

  guix package: warning: Your Guix installation is 233 days old

It looks like it's looking at the age of the "latest" directory, which
wouldn't be useful information since I've symlinked it intentionally.

I wonder if there's a way to turn this off for people who have
intentionally symlinked guix to their git checkout?

Re: postgres 9.6.2 update breakage

[Christopher Allan Webber]

Jan Nieuwenhuizen writes:

> Roel Janssen writes:
>
>> So, it would be something like:
>> postgres pg_upgrade \
>> ...
>
> It's great to have a recipe `that works', so thanks!
>
> However, whether or not we automate this, I cannot help to wonder if
> we should support downgrading -- at least to the previous version
> in this case?
>
> If I'm not mistaken, everything else in GuixSD will run if I select a
> previous system generation in Grub...except for this?
>
> Is involving postgres developers an option, I'm sure a least one of
> the postgresql hackers[cc] are already looking at Guix[SD]?
>
> Greetings,
> janneke

There's a big difference in upgrading and downgrading between guix
revisions and doing so in highly stateful databases, unfortunately.

I can't speak for postgres specifically, but here's my experience with
migrations as the tech lead of MediaGoblin:

 - upgrades should be taken with extreme caution, and you should back up
   first.
 - downgrades should be taken with ten times the amount of caution of
   upgrades, a vat of coffee to work through the problems, and a barrel
   of whiskey for when it doesn't.  I say that as someone who's mostly
   given up coffee and doesn't drink alcohol.

State changes are bad enough when unidirectional.  Django, for instance,
provides an API that does both upgrades and downgrades.  Almost
everybody spends a bunch of time carefully crafting their upgrades, and
just leaves their downgrades as the stubs that come with it.  These are
stubs that drop columns entirely, possibly columns that data was moved
to in the migration.  Reverse course, and suddenly you don't have a lot
of data you used to.

What we really want to do is provide the option to snapshot things
*before* you do an upgrade, IMO...

Re: postgres 9.6.2 update breakage

[Christopher Allan Webber]

Roel Janssen writes:

> Jan Nieuwenhuizen writes:
>
>> Hi!
>>
>> I reconfigured my system and pulled in the postgres 9.6.2 update.  Now
>> postgres does not start, /var/log/messages has
>>
>> May 12 13:02:52 localhost postgres[451]: [1-1] FATAL:  database files 
>> are incompatible with server
>> May 12 13:02:52 localhost postgres[451]: [1-2] DETAIL:  The data 
>> directory was initialized by PostgreSQL version 9.5, which is not compatible 
>> with this version 9.6.2.
>>
>>
>> I have reverted the postgres update and everything is "fine" again.s
>
> I think database upgrades can be performed with 'pg_upgrade', which
> is included in the postgresql package.  The command's '--help' switch
> even includes an example.
>
> In my experience (9.2 > 9.3, 9.3 > 9.4, 9.4 > 9.5), the upgrades went
> just fine.  You have to stop the postgresql daemon, perform the
> upgrade, and start it again.

Could you provide the steps you used to upgrade using pg_upgrade?

It seems it needs to reference the old version of posgres, so we'll need
to keep the old version around every time we bump the postgres version.
We aren't doing that currently.  If I just do:

  cwebber@oolong:/tmp$ sudo -u postgres pg_upgrade
  
  You must identify the directory where the old cluster binaries reside.
  Please use the -b command-line option or the PGBINOLD environment variable.
  Failure, exiting

Well, ok, so maybe we could keep the previous verison of postgres
around.  But even then, this seems like it is going to be annoying for
users because you'll need to install both versions of postres and figure
out what the paths are to point pg_upgrade at.  Am I wrong?

Hence, I think having a wrapper script that does this for users (maybe
even through some dumb gexp) would be nice..

Re: postgres 9.6.2 update breakage

[Christopher Allan Webber]

Hi there!

Jan Nieuwenhuizen writes:

> Hi!
>
> I reconfigured my system and pulled in the postgres 9.6.2 update.  Now
> postgres does not start, /var/log/messages has
>
> May 12 13:02:52 localhost postgres[451]: [1-1] FATAL:  database files are 
> incompatible with server
> May 12 13:02:52 localhost postgres[451]: [1-2] DETAIL:  The data 
> directory was initialized by PostgreSQL version 9.5, which is not compatible 
> with this version 9.6.2.
>
>
> I have reverted the postgres update and everything is "fine" again.s
>
> How do we want to handle this?  I imagine that postgres has some way to
> update its database...and I probably can figure out how to do that.  But
> do our users need to know this?  And more importantly, if I upgrade,
> will I be able to revert to a previous generation of my system?

Yes, so therew as some conversation on this last year:

  https://lists.gnu.org/archive/html/guix-devel/2016-06/msg00917.html

Basically, we should have a package like "postgresql-upgrade".  It
should use:

  https://www.postgresql.org/docs/9.4/static/pgupgrade.html

... and do an upgrade between postgres versions.

Any volunteers want to work on this? :)

Re: certbot service experience

[Christopher Allan Webber]

[-bug]

Chris Marusich writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>>  - I was surprised that I was prompted for an email while doing guix
>>system reconfigure
>
> That does seem odd.  Why were you prompted for an email address?  Can
> that be fixed somehow?

It's the certbot initial-setup script firing off here.  The email
address is given so you can be notified for security updates, etc.
Maybe recovery?  Don't remember about that last bit.

>>   2) Enable the certbot-service-type (and mcron-service-type if you
>>  haven't already):
>>
>>(service certbot-service-type
>> (certbot-configuration
>>  ;; Replace these with your own domain and web root
>>  (hosts '("test.activitypub.rocks"))
>>  (webroot "/srv/activitypub.rocks/site/")))
>>;; if you don't have an mcron service already
>>(service mcron-service-type)
>
> Where is the certbot-service-type defined?  I couldn't find it in the
> master branch.  Also, why is mcron required?  I don't know much about
> LetsEncrypt, but I thought certbot was a one-time thing that you do
> manually...  Why is it a "service" here?

It's not in the master branch is why.  I was looking at the
wip-git-https branch. :)  I think it's also in the wip-potluck branch.

It's extending the mcron service so it can install an auto-update rule
for you, which is pretty cool!

>>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>>  did, *now* we can add the keys appropriately to the nginx config.
>>
>>(service nginx-service-type
>> (nginx-configuration
>>  (server-blocks
>>   (list
>>(nginx-server-configuration
>> ;; Again, adjust to your site
>> (server-name '("test.activitypub.rocks"))
>> (root "/srv/activitypub.rocks/site/")
>> (ssl-certificate
>>  
>> "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>> (ssl-certificate-key
>>  
>> "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))
>>
>>  Reconfigure and cross your fingers!
>>
>>   4) At this point I was surprised that it seemed like nginx should have
>>  been working with https since everything was in place, but I
>>  couldn't access it from my browser over https.  Frustrated, I
>>  restarted the server.
>>
>>  And then it worked! :)
>>
>> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
>> a restart, then it worked for me.  (Well, plus a few reconfigures where
>> nothing worked at all because I broke things of course. ;))  I wonder if
>> that can be improved?
>
> I wonder if it is possible to define a custom service which orchestrates
> the execution of nginx and certbot in the way you require, so that you
> can define it all in one place, at once, without needing to reconfigure
> multiple times?

It might be, I dunno!  Maybe in starting the service, if it sees that
the keys have not been generated yet, it pulls up nginx temporarily just
to do the registration with the nginx that doesn't have the keys in it,
does the generation of the keys and verifies them with letsencrypt, then
pulls up the proper nginx at last.

I don't know how easy/feasible this is.

>> That said, it's still really exciting to be able to describe these
>> things declaratively, and to have Guix take care of keeping things
>> renewed for me. :)  Excited to have this landing, and to be that much
>> closer to doing server deployment with GuixSD!
>
> Pretty cool!  Thanks for sharing your experience.  It's always neat to
> read about how people are using the system.

Yeah!  I'm excited to finally move to hosting my servers with GuixSD at
last, after ages of talking about it. ;)

certbot service experience

[Christopher Allan Webber]

I'm crossposting this to guix-devel, even though it's in reply to
guix-patches bug #26685 adding the cerbot service, because I think it's
more about my experiences with workflow and less about what might affect
that specific bug.  (If you reply on guix-devel, maybe remove the
debbugs address.)

Andy Wingo writes:

> Attached patch adds a certbot service extending nginx.  Only question
> is, how to ensure that nginx is runing when the certbot activation runs?
> In practice I bet this races so it's not a big issue but if there's a
> way to require nginx before activation, that would be nice.

So I got a server up and running using certbot and this.  Yay!  It went
pretty well, and I'm happy about it now that it's running, though I did
run into some bumps (which is okay, I ran into bumps doing this on
Debian too, I just think now's a good time to reflect on the experience
and see if we can do anything better... plus maybe it's useful to
explore for someone else too).  I figured I'd document what the process
was like for me.  Here's my observations:

 - I was surprised that I was prompted for an email while doing guix
   system reconfigure (and in being asked if I was willing to supply
   that info to the EFF so they can put me on their mailing lists, which
   I'm not against but was surprising to encounter in a reconfigure).
   IIRC this is the first time anything like that has happened.  I
   generally anticipate such a reconfigure to be "prompt-less".
   I wonder if there should be an email field we can provide, so that it
   doesn't do this prompt?  Granted, it's for the first time run only.
   Maybe it's okay to do things as they are, but we should document it?
   I dunno?

 - Not specifically something to do with this package, but nginx wasn't
   coming up at a few points (btw, default nginx configuration won't
   work, because it points at certificate filepaths that we don't
   "automatically" put in place).  When nginx doesn't come up, there's
   pretty much no information as to why; nothing in the shepherd logs,
   nothing useful from shepherd itself ("shepherd: Service nginx could
   not be started."), and nothing in the nginx logs either.

 - Getting the certbot stuff to work basically involved three stages:

   1) Enable nginx without cert certbot, and point the root somewhere
  which will be the same place you'll set certbot-configuration's
  webroot at in the next step.  This looks something like the
  following:

(service nginx-service-type
  (nginx-configuration
   (server-blocks
(list
 (nginx-server-configuration
  ;; Replace these with your own domain and web root
  (server-name '("test.activitypub.rocks"))
  (root "/srv/activitypub.rocks/site/")
  ;; Note that you have to disable https and any certificates
  ;; esp because our nginx config defaults to pointing at keys
  ;; that probably aren't there.  Without doing this nginx
  ;; will die mysteriously.
  (https-port #f)
  (ssl-certificate #f)
  (ssl-certificate-key #f))

  (You can maybe do this in the same phase as the next step but
  there's a risk of a race condition I think.)

  Anyway now do "guix system reconfigure" and nginx should come up
  and serve your stuff *without* https.

  2) Enable the certbot-service-type (and mcron-service-type if you
 haven't already):

   (service certbot-service-type
(certbot-configuration
 ;; Replace these with your own domain and web root
 (hosts '("test.activitypub.rocks"))
 (webroot "/srv/activitypub.rocks/site/")))
   ;; if you don't have an mcron service already
   (service mcron-service-type)

 Okay, with those added, run "guix system reconfigure" again.
 You'll be prompted for your email address and about whether or not
 you want to join the EFF mailing lists, so watch for that.

  3) Okay hopefully that went successfully!  It should say.  Assuming it
 did, *now* we can add the keys appropriately to the nginx config.

   (service nginx-service-type
(nginx-configuration
 (server-blocks
  (list
   (nginx-server-configuration
;; Again, adjust to your site
(server-name '("test.activitypub.rocks"))
(root "/srv/activitypub.rocks/site/")
(ssl-certificate
 
"/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
(ssl-certificate-key
 
"/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))

 Reconfigure and cross your fingers!

  4) At this point I was surprised that it seemed like nginx should have
 been working with https since everything was in place, but I
 couldn't access it 

Re: potluck status

[Christopher Allan Webber]

Andy Wingo writes:

> On Fri 28 Apr 2017 14:42, Hartmut Goebel  
> writes:
>
>> Am 28.04.2017 um 14:05 schrieb Andy Wingo:
>>>   5.15 Invoking ‘guix potluck’
>>
>> Please think about an other name for this command. "potlouk"  may be
>> common to native speakers but I never heard this word. Thanks.
>
> I thought about many things :)  Do you have a suggestion?
> https://guix-potluck.org/ does provide the definition.
>
> Andy

I personally like Potluck a lot; I know it might not be a particularly
common English word, but it may be a good one for people to learn if
they don't already know it.  It captures the spirit of things to me very
nicely.

Re: Services can now have a default value

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello,
>
> Carlo Zancanaro  skribis:
>
>> On Thu, Apr 20 2017, Ludovic Courtès wrote:
>>> There must be some sort of a mapping between service types and
>>> configuration types, indeed, but I’m not sure how to achieve it.
>>>
>>> One solution would be to have all the  records
>>> inherit (in the OO sense) from , or something along these
>>> lines.
>>
>> This was my first thought. I couldn't see how to do OO-style inheritance
>> with the SRFI-9 API, though. I'm not very experienced with Guile (or
>> scheme generally), so I might do some more reading about that.
>
> SRFI-99 supports inheritance, though there’s currently no SRFI-99 module
> in Guile proper:
>
>   https://srfi.schemers.org/srfi-99/srfi-99.html
>
> Oh and there’s also R6RS records, SRFI-35… no shortage of record APIs!
> :-)

A record interface with inheritance!  Neat.

Though hey, if you're going to add inheritance, we also have GOOPS, and
with the clone macro I sent to the mailing list a while ago that I
worked on with Janneke, we even have immutable GOOPS! ;)
Plus then you get generic methods!

 - Chris Webber, who may only be slightly trolling and knows Guix will
   probably never adopt using GOOPS

Re: Idea: 'ethical hosting' [formerly mailman service (free for FOSS projects)]

[Christopher Allan Webber]

Pjotr Prins writes:

> On Tue, Apr 18, 2017 at 06:11:41PM +, ng0 wrote:
>> > I don't think we should offer VPS (at first). But we can offer the
>> > choice of using other VPS'. The VPS marked is overcrowded and very
>> > competitive - I would not want to compete with that. I want to compete
>> > with companies that offer hosting, but do not provide the software
>> > stack. I also want people to have access to their own VPS. That is the
>> > ethical part.
>> 
>> Okay, then I do not understandand completely where you are going with this.
>> For a hosted service you need a host first. Do you want to offer consulting
>> work, so that this can be deployed at service (which furthermore would
>> require some legal annoyance), as a service? In other words, some of us
>> probably to cover the legal part with a contract at Guix Europe e.v. (if
>> the registered association law in France works similar to the ones in 
>> Germany)
>> will set up services on demand for interested people and organizations?
>
> This is one example for hosting mailman: https://www.mailmanhost.com/.
> They host it on some other VPS service.
>
> What I want to do is provide something similar with a slick web
> interface where people click on a service, say a VPS with mailman,
> configure it and launch it on a VPS hoster of their choice - we can
> offer multiple solutions on multiple hosting parties. Then the client
> has a running and configured server. We charge a fee on top of the VPS
> fee.
>

Note that that vision of building a web UI on top of a declarative
system to make deploying and maintaining servers easier is what
originally drew me to Guix.  Still hasn't left my mind as something
valuable.

Re: Getting rid of alpha label

[Christopher Allan Webber]

Ludovic Courtès writes:

> Heya!
>
> Pjotr Prins  skribis:
>
>> Can we get rid of the alpha status? It suggests packages are in alpha,
>> which they are not. 
>>
>> I have already had two different administrators in two environments
>> claiming that Guix packages can not be deployed on HPC systems because
>> it is alpha. 
>>
>> The excuse is too easy ;)
>
> I agree.  ;-)
>
> After FOSDEM Ricardo removed the big blue banner that was saying Guix is
> alpha/beta.  So now the only mentions of “beta” that are left are:
>
>   1. on the “TEST” button on the home page;
>   2. under GuixSD at .
>
> I think we can remove #1.  Is this what you had in mind?
>
> Cheers,
> Ludo’.

+1

Re: Getting rid of alpha label

[Christopher Allan Webber]

Pjotr Prins writes:

> Can we get rid of the alpha status? It suggests packages are in alpha,
> which they are not. 
>
> I have already had two different administrators in two environments
> claiming that Guix packages can not be deployed on HPC systems because
> it is alpha. 

I thought we were in beta?  That's what the website homepage says now...

Re: [PATCH] guix hash: -g hashes a git repository

[Christopher Allan Webber]

Andy Wingo writes:

> +   (pk "git" "clone" file checkout)
> +   (recursive-hash checkout)

One of those cursed pk's made it in here.

Exciting!  Look forward to having this option.

Re: "guix potluck", a moveable feast

[Christopher Allan Webber]

Andy Wingo writes:

> Christopher Webber asks about breakage due to version skew between peer
> channels and channels and Guix itself.  I think I would like to just
> ignore this problem for now: if you add channels and things break
> somehow due to an update in Guix or an update in some channel, then the
> workaround is to disable channels until developers fix things.

It isn't an impossible problem to get around anyway, at least from a
channel to channel perspective... since we're building channels around
git repositories, *eventually* we could allow a channel to include
another channel as a dependency with a specific hash.  But we can work
on that once we get things working.

Re: Staying on-topic

[Christopher Allan Webber]

Taylan Ulrich Bayırlı/Kammer writes:

> That's why I was worrying.  I think it's fine to ban misgendering, on
> the condition that, should a female-born person feel insulted at the use
> of feminine pronouns for a male-born person, that too should be banned,
> effectively forcing neutral pronouns so the project takes no sides.

The Code of Conduct's reflection of gendering, and how it applies to
(mis)gendering is referring to this type of thing:

  
https://en.wikipedia.org/w/index.php?title=Transphobia=773143601#Misgendering_and_exclusion

I think the code of conduct is pretty clear about it:

  We are committed to making participation in this project a
  harassment-free experience for everyone, regardless of
  [...] gender, gender identity and expression [...]

"Gender identity and expression" is pretty clear: trans-exclusionary
behavior doesn't belong here.

It's very simple: respect peoples' preferred pronouns and *drop this
subject*.  This topic is over, and that's been signaled as such from the
maintainers and many other people in this community.  Please don't
reopen it.

Re: "guix potluck", a moveable feast

[Christopher Allan Webber]

Andy Wingo writes:

> Hi!

Hi!

> potluck.guixsd.org needs to be isolated from other hosts because it will
> load potluck.scm files from untrusted sources; we hope the sandbox works
> but we need a bit of defense-in-depth.

Well now I see the motivation behind (ice-9 sandbox) ... :)

> As I mentioned, I think it would be nice to be able to install some
> potluck packages directly from git, without requiring those packages to
> make releases and update the potluck.scm.  But until then, we can make
> it so that the source is fixed in the potluck.scm as it is with other
> Guix packages, and therefore that any update to potluck.scm in the
> source git branch registered with potluck.guixsd.org constitutes a new
> release which replaces the old one.  A developer should signal
> potluck.guixsd.org about the update via a re-invocation of "guix potluck
> add".  Maybe "guix potluck add" could remember the branch, dunno.
>
> Anyway!  The result of the "guix potluck channel-manager" is a stream of
> guix modules as a continually updated git tree -- a guix channel.  I am
> thinking that we need to rewrite these files to be more "normal" -- like
> starting with a (define-module), but a #:pure module and an appropriate
> set of imports to enforce the sandbox.  We should be able to compile
> this module, to prevent the potluck channel from slowing things down.
> So basically the channel-manager rewrites the potluck.scm files.

It sounds nice!

One challenge though... what do we do about multiple channels
introducing version skew?  (Maybe I'm abusing that term?)  This isn't
something we've dealt with before in Guix... if my channel adds
something that depends on your channel's package definition, do I
explicitly set a revision for your channel?  Otherwise else, your
channel could change as you upgrade your software version, and that
might unexpectedly break my channel...

Re: On merging the npm importer

[Christopher Allan Webber]

Jan Nieuwenhuizen writes:

> Hi,

Hi Jan!

> We have a working importer for npm packages written by Jelle that I have
> been using for about half a year.  It can use some improvements and
> that's why I think we should merge it.
>
> Have alook at my npm branch here, rebased on master
>
> https://gitlab.com/janneke/guix

Would like to review soon, though I'll say that I think unless there are
serious problems, we should probably merge it.  Avoiding bitrot is prety
important, and at the very least I don't think it will hurt to have it
merged.

> I added a patch with several fixes for the importer and and build
> system.  So far, so good.
>
> There's a problem however with the --recursive option and the build
> system.  To quote Jelle[1]
>
>To start of with something that did not work out as well as I had
>hoped, getting a popular build system (e.g. Gulp, Grunt, Broccoli and
>others) packaged.  As mentioned in my earlier mails, the list of
>transitive dependencies of any of these suffer from at least the
>following:
>
>- It is a list with more than 4000 packages on it
>- It is a list with at some point the package itself on it
>
> Most nontrivial npm packages use a build system, and all build systems
> have circular development dependencies.  Not all development
> dependencies are always required to build a package, but some certainly
> are nd there's no way to tell which is which, afaik.
>
> That's why I added a --binary option to the importer: it will not
> try to use the build system and instead mimick `what npm does.'  This
> does provide, however, an amazing reproducibility feature to the
> dependency woes that npm hackers are familar with.
>
> I suggest to not add any npm package to Guix that is the result of using
> the --binary option and to build a base of full-source/sanitized npm
> packages.

Cool... makes sense to me to have this as something we don't use for
Guix packages, but which might make Guix more useful for people who have
to use npm in the awkward "real world" that is the current state of npm.

> Greetings, janneke
>
> [1] https://lists.gnu.org/archive/html/guix-devel/2016-08/msg01567.html

A "busy beaver" approach to assisting package definition updates

[Christopher Allan Webber]

Hello!

David Thompson and I had the good fortune to be able to hang out with
the esteemed Gerald Sussman in his office over tea.  Topics were wide
ranging, but one in particular has stuck in my mind as to how it could
apply to Guix.

What Sussman said is he thought version skew was a source of many
problems in practice (something I'm sure most of the Guix community
agrees with, and Guix avoids it somewhat by being much more precise).
Sussman also suggested something that I think he was thinking about in
terms of more distributed systems (but I'm not really sure); I'll give
an example of how it might apply at the end of the email since it's
maybe not applicable to Guix (but could be to an actor model or
propagator system).

Sussman's suggestions is that programs themselves should be able to
reach out and test to see if they can operate with the newer version of
some library and make the decision of which one they can link to.  And
they should be able to tell by having a reasonable test suite, and
checking to see if their test suite works once the linking happens.
It's not a perfect check of course, since tests aren't perfect, but it
would give programs more autonomy in linking by "exploring" their
surroundings.

Now obviously in Guix, we're trying to be very specific about linking,
and it's done at package build time, not at compile time.  But still, we
can mostly pull off the same thing using a continuous integration
system, assuming we have enough resources; we already have tooling to
look for the newest versions of packages, and we could possibly have a
CI system try to write out a new package definition based on the latest
version, try compiling it and see if it passes its tests... and then we
can try to see if the packages that depend on it can also compile
against it and pass their tests.  In this way we could automatically do
the work of detecting and suggesting upgrades for many packages, and
even finding out what they would break... *before* community members
have to spend the effort to try it themselves.  It wouldn't be perfect,
but it could be a big assist.  Perhaps there could be a web interface
that shows "Here's a new definition of this package, it seems to build.
Here's packages that depend on it which pass... these ones don't,
perhaps they should be pinned to the old version or be investigated?"

Of course, this would require a lot more build resources than the
already-strapped amount we have.  (If we had Guix running on "commodity
hosting" platforms, I suspect we could get some of these resources
donated.)

Helpful?  Not helpful?  Maybe it's obvious to everyone already, I dunno :)
 - Chris

BONUS CONTENT!  How would this apply to a distributed system, such as an
actor model (or propagator system or something), which might run into
other actors that it has to choose whether or not to link up to in real
time?  Actors in the system which become aware of other actors they
might interface with could either "test" linking up with the other
actors by running their tests (assuming they can interface
non-destructively) and also perhaps if there's a way to get some kind of
signature of what this actor's version or something identify'y looks
like, even if things go badly they could record information on whether
or not to link up to it in the future.

Re: Advice about GuixSD on Serveraptor?

[Christopher Allan Webber]

Leo Famulari writes:

> On Sun, Mar 12, 2017 at 08:32:52PM -0400, Leo Famulari wrote:
>> I had wanted to use the GuixSD installer because the image format
>> conversion is reproducible. So, nobody would have to trust me to provide
>> the right thing.
>>
>> Of course, it would be a lot simpler if we made a barebones system with
>> `guix system vm-image`.
>>
>> If we can't use the installer, how should we create and provide another
>> image?
>
> Any advice or guidance?
>
> I can easily create an image to use for this, but I don't want to do it
> if others think I am going beyond the level of trust placed in me by the
> Guix project.

So, if you provided the source scheme to generate the image, and signed
the image, people would both have the option to generate the image
themselves, or download your signed binary image if they trust you?

Honestly, at this point the most important thing is to get things to the
point where we have *a* documented process to install GuixSD on these
servers; once we have that, and assuming we also have documentation /
tooling where people could reproduce the whole process (even if they
used the image you provided, as long as they could reproduce that step
too) I think we're in a much better state than we are... and we could
refine further from there.

My opinion, anyway!
 - Chris

Re: Being excellent to one another

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hi there,
>
> Gentlefolks, please everyone calm down.  Being rude or insulting to
> fellow hackers is not acceptable on the project’s communication
> channels, period.  When you feel unable to express your disagreement in
> a constructive and respectful manner, please delay your reply until you
> can do that.

[...]

> Besides, while I appreciate it when native English speakers provide
> corrections and guidance, I think we as a project must tolerate English
> mistakes in our communication.  The reason for this is very simple: most
> contributors are not native English speakers.  English is our
> communication medium; it shouldn’t be a hindrance to the inclusion of
> contributors who do not master it.

About English though, I do agree with ng0 about they/them... as a
default pronoun, especially when you don't know.  It's good English,
with longstanding history:

  https://en.wikipedia.org/wiki/Singular_they

The important thing is to not assume someone's preferred pronouns
without knowing them.  Singular they isn't your only option; I also
happen to like Spivak pronouns:

  https://en.wikipedia.org/wiki/Spivak_pronoun

... which have the delightful connection to hacker culture in their
popularity with Lambdamoo, an oldschool MUD. :)  Singular they (or even
spivak) is also acceptable as a pronoun if someone chooses that.

Of course it's possible to make mistakes, but it *is* important to try
not to misgender people... both by not making assumptions, and
especially when using the right pronouns once you do know.  I have both
seen people break down into tears and also walk away from communities
from being misgendered.  That's an important sign of respect towards the
person...  and it doesn't cost you anything to do it.

Be excellent to each other indeed... and here's one critical way to do
it.

Re: guix is the guildhall that we always wanted!

[Christopher Allan Webber]

Andy Wingo writes:

> On Thu 16 Mar 2017 23:01, Mark H Weaver  writes:
>
>> If [Guix] starts encouraging a decentralized approach, that would
>> result in strong pressure on us to freeze our API, which includes even
>> such details as which module each package is exported from.  This
>> would drastically reduce the freedom Guix has to evolve the way its
>> packages are specified.
>
> I get what you are saying.  I think that if a future guildhall is
> decentralized but uses Guix it needs to minimize its burden on Guix.
> That could mean that the packages are actually specified in a different
> DSL with different stability characteristics -- for example that DSL
> could call specification->package under the hood for example, like
> Ludovic mentions.  (I should mention that this idea of using Guix and
> especially all its errors are my own -- haven't talked to others about
> it yet!)
>
> Which module a package definition is in is a good example of something
> not to depend on.

This makes sense to me... if it really is true that our scheme'y
Guildhall-style packages are so simple they're more data than code,
maybe we could even restrict them to... just data.  Just a list of what
files are being provided, etc.  That could easily be stored in some
minimal database.

I guess I'm saying +1.

Re: Summer of code!

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hi!
>
> l...@gnu.org (Ludovic Courtès) skribis:
>
>> Last week GNU was accepted as an organization for Google’s Summer of
>> Code (GSoC), which means we can have students working on Guix and the
>> Shepherd.
>>
>> Now’s the time to recruit!  If you’d like to mentor, please follow the
>> instructions in this email:
>>
>>   https://lists.gnu.org/archive/html/summer-of-code/2017-02/msg00026.html
>>
>> Also please take a look at
>> , update it as you
>> see fit, and email us if you’re a candidate.
>>
>> People who are already contributing to Guix and who are eligible are
>> welcome too!
>
> I’ve posted a news here:
>
>   https://gnu.org/software/guix/news/join-gnu-guix-for-gsoc-2017.html
>
> Spread the word!
>
> Ludo’.

Whoo!  I can mentor the async event loop one, but I added Fibers as an
option for the event loop... and Fibers may be a better fit, for this
than 8sync (but I left it on there).

Re: How to M-x debbugs-gnu with new guix-patches

[Christopher Allan Webber]

Pjotr Prins writes:

> I submitted my first patch on debbugs. It is a fairly trivial one for
> speedtest-cli which I need because I move around so much ;)
>
> More importantly I wrote some documentation:
>
>   
> https://github.com/pjotrp/guix-notes/blob/master/HACKING.org#making-a-patch-and-submit-to-the-guix-project
>
> Comments welcome (as usual).
>
> Pj.

This is really nice!  Maybe we should link to it from or put it on the
Guix website somewhere?

Re: GuixSD on commodity hosting platforms, hoster: IN-Berlin

[Christopher Allan Webber]

Leo Famulari writes:

> I'd like for someone to try this conversion themself and verify that it
> creates the same qcow2 file.
>
> If it does, then we can ask Serveraptor to make it available for testing
> on their platform.

Leo and I worked on this off-list, and verified!  I was able to make the
same thing.

Re: unmerged patches

[Christopher Allan Webber]

ng0 writes:

> Hi,
>
> I will re-submit my unmerged patches to the new guix-patches list. If
> someone lost track, there's an online view to keep track of open/closed
> things.
>
> I hope this helps to move forward the patches I still have control over
> (some are already dropped for reasons).

Great, thank you! :)

BTW I've found out that when I file bugs with patches there's a feature
where you can add the header:

  Tags: patch

in the email you compose, and then the patch tracker will mark it
appropriately.  Just a suggestion!

Re: Ideas for Summer of Code 2017

[Christopher Allan Webber]

Mathieu Lirzin writes:

> Hello,
>
> GNU is applying as an organization for Google Summer of Code 2017.  So
> this is time to start gathering ideas for possible Guix/Shepherd related
> projects.
>
> I have created a page based on the one from last year:
>
>   https://libreplanet.org/wiki/Group:Guix/GSoC-2017
>
> It's a Wiki, so feel free to contribute to it.
>
> TIA.

One interesting change from when this was defined last year and this
year is that the "extensible event loop" one has some candidates to pick
from, via guile a-sync, fibers, and 8sync.

However using any of those would require Shepherd being pinned to Guile
2.2.  Maybe not a problem; it could be a branch, and Guile 2.2 stable
looks not so far away.

Re: Fixing non-reproducibility in some guile packages

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>>   (let* ((out (assoc-ref %outputs "out"))
>> -(module-dir (string-append out "/share/guile/site/2.0"))
>> +(module-dir (string-append out "/share/guile/site/"
>> +   ,(if guile-2.2?
>> +"2.2" "2.0")))
>>  (source (assoc-ref %build-inputs "source"))
>
> Another approach, which is more future-proof but also more verbose, is
> to evaluate (effective-version) for the Guile that’s being used, on the
> “build side” (thus, no need to do the unquote thing above).
>
> The ‘guile-minikanren’ package does exactly that:
>
>  (let* ((out (assoc-ref %outputs "out"))
> (guile (assoc-ref %build-inputs "guile"))
> (effective (read-line
> (open-pipe* OPEN_READ
> (string-append guile "/bin/guile")
> "-c" "(display 
> (effective-version))")))
> (module-dir (string-append out "/share/guile/site/"
>effective)) …)
> …)
>
> We should probably factorize this somewhere (a new (guix build guile)
> module?), but for now that’s what we have.
>
> How does that sound?

It looks much better!  I've updated my patch to use this method.

Re: the new module, I agree it's a good idea.  We had some talks at
FOSDEM about maybe supporting more declarative guild'y type
configuration again, and maybe that's the right approach, I don't really
know.  It feels like "what should we do about easy to package guile
packages that don't necessarily use autotools" is still a conversation
to be had.  Not sure if that's a prerequisite for the module.

Anyway, okay to push?  I'd love to have the buggy package not be buggy
in master. :)

From 369582d7c8b5ce1249761337319e79cc117f161e Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Fri, 10 Feb 2017 19:24:57 -0600
Subject: [PATCH] guile-gdbm-ffi: Write to correct guile output directory and
 use guild.

* gnu/packages/guile.scm (guile-gdbm-ffi): Check guile for effective version
before writing to output path.  Also fixes a bug where the guild command was
not getting called, and instead was calling the internal guile compile-file
procedure.  This meant that the package produced was dependent on whatever
version of guile was powering Guix at the time.  Also set GUILE_AUTO_COMPILE
to 0 to avoid gnarly looking warnings during build.
---
 gnu/packages/guile.scm | 23 +--
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 3e8ab007b..75f561c03 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -844,11 +844,20 @@ inspired by the SCSH regular expression system.")
((guix build utils))
#:builder
(begin
- (use-modules (guix build utils)
-  (system base compile))
+ (use-modules (guix build utils))
+
+ (setenv "GUILE_AUTO_COMPILE" "0")
 
  (let* ((out (assoc-ref %outputs "out"))
-(module-dir (string-append out "/share/guile/site/2.0"))
+(effective-version
+ (read-line
+  (open-pipe* OPEN_READ
+  (string-append
+   (assoc-ref %build-inputs "guile")
+   "/bin/guile")
+  "-c" "(display (effective-version))")))
+(module-dir (string-append out "/share/guile/site/"
+   effective-version))
 (source (assoc-ref %build-inputs "source"))
 (doc (string-append out "/share/doc"))
 (guild (string-append (assoc-ref %build-inputs "guile")
@@ -856,7 +865,10 @@ inspired by the SCSH regular expression system.")
 (gdbm.scm-dest
  (string-append module-dir "/gdbm.scm"))
 (gdbm.go-dest
- (string-append module-dir "/gdbm.go")))
+ (string-append module-dir "/gdbm.go"))
+(compile-file
+ (lambda (in-file out-file)
+   (system* guild "compile" "-o" out-file in-file
;; Make installation directories.
(mkdir-p module-dir)
(mkdir-p

Re: Fixing non-reproducibility in some guile packages

[Christopher Allan Webber]

Andy Wingo writes:

> In some future (is it near or far?), the source -> compiled function
> needs additional inputs: checksums or timestamps of "build inputs" or
> so, so that when for-syntax definitions (like macros) change, users of
> those definitions will recompile.  That is a harder problem though.

Ah yeah, I've been bit by this before.  Though, I imagine we'd run into
a problem where we'd never know how to "garbage collect" anything in
~/.cache/guile/ .  "Maybe not a problem", except if you're hacking some files
constantly... :)

Maybe every guile hacker needs to get in the habit of
`rm -rf ~/.cache/guile/' though? :)

How to M-x debbugs-gnu with new guix-patches

[Christopher Allan Webber]

So!  We have a new debbugs tracking of guix-patches.  Great!  Those who
are emacs users in the know probably like to use the M-x debbugs-gnu
interface.  Here's what you need to do:

Add this to your .emacs:
  (add-to-list 'debbugs-gnu-all-packages "guix-patches")

Now open up debbugs-gnu:
  C-u M-x debbugs-gnu   guix-patches  n y

And now you have a nice emacs interface!

I found this documentation on debbugs useful:
  https://www.debian.org/Bugs/Reporting

And also, maybe you want to tag bugs or etc.  Use the 'C' key from the
emacs interface!

Happy bug triaging!

Fixing non-reproducibility in some guile packages (was: guile2.2-gdbm-ffi issue)

[Christopher Allan Webber]

Christopher Allan Webber writes:

> (Originally accidentally sent this to guile-de...@gnu.org ... sorry!)
>
> Hiya,
>
> I pushed guile2.2-gdbm because it was just a couple of lines and it
> seemed to work and didn't affect anything.  I made a mistake though that
> I didn't realize until I ran
> "guix environment --ad-hoc guile-next guile2.2-gdbm-ffi --pure".
> The trivial-build-system in guile-gdbm-ffi dumps the built module into
> (string-append out "/share/guile/site/2.0") ... oops!  Before I ran
> --pure, the 2.0 directory was on my load path, and thus I didn't bump
> into the error.  (Maybe I should have asked for review anyway...)
>
> I'm trying to think of how to fix this.  I have a very kludgy solution
> attached.  I don't feel great about it but I don't know how to signal to
> the builder whether it's using guile 2.2 or 2.0 otherwise.  Thoughts?

It turns out there was a more serious issue involved here.  The guile
input was never used... instead, we were compiling from whatever guile
process guix was using to run by using the compile-file procedure from
(system base compile).

I think this is a pretty serious bug.  It means that guile-gdbm-ffi was
never properly reproducible by our standard methods of determining
inputs.  This is entirely my fault, since I'm the one who put together
these packages.

The guile-wisp package also has this issue.  I'm going to work on a fix
for it.  In the meanwhile, here's a patch that both fixes
guile2.2-gdbm-ffi and fixes the forementioned reproducibility problem.

 - Chris

From b718720d72f1da7655db49ec6ec25b658b0ca27f Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Fri, 10 Feb 2017 19:24:57 -0600
Subject: [PATCH] guile-gdbm-ffi: Write to correct guile output directory and
 use guild.

* gnu/packages/guile.scm (make-guile-gdbm-ffi): New variable.
Adapts from the previous guile-gdbm-ffi definition.  Also fixes
a bug where the guild command was not getting called, and instead
was calling the internal guile compile-file procedure.  This meant
that the package produced was dependent on whatever version of
guile was powering Guix at the time.
(guile-gdbm-ffi, guile2.2-gdbm-ffi): Use make-guile-gdbm-ffi.
---
 gnu/packages/guile.scm | 29 +++--
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 3e8ab007b..a02e4887a 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -825,9 +825,11 @@ inspired by the SCSH regular expression system.")
 ;; There are two guile-gdbm packages, one using the FFI and one with
 ;; direct C bindings, hence the verbose name.
 
-(define-public guile-gdbm-ffi
+(define (make-guile-gdbm-ffi guile-2.2?)
   (package
-(name "guile-gdbm-ffi")
+(name (if guile-2.2?
+  "guile2.2-gdbm-ffi"
+  "guile-gdbm-ffi"))
 (version "20120209.fa1d5b6")
 (source (origin
   (method git-fetch)
@@ -844,11 +846,12 @@ inspired by the SCSH regular expression system.")
((guix build utils))
#:builder
(begin
- (use-modules (guix build utils)
-  (system base compile))
+ (use-modules (guix build utils))
 
  (let* ((out (assoc-ref %outputs "out"))
-(module-dir (string-append out "/share/guile/site/2.0"))
+(module-dir (string-append out "/share/guile/site/"
+   ,(if guile-2.2?
+"2.2" "2.0")))
 (source (assoc-ref %build-inputs "source"))
 (doc (string-append out "/share/doc"))
 (guild (string-append (assoc-ref %build-inputs "guile")
@@ -856,7 +859,10 @@ inspired by the SCSH regular expression system.")
 (gdbm.scm-dest
  (string-append module-dir "/gdbm.scm"))
 (gdbm.go-dest
- (string-append module-dir "/gdbm.go")))
+ (string-append module-dir "/gdbm.go"))
+(compile-file
+ (lambda (in-file out-file)
+   (system* guild "compile" "-o" out-file in-file
;; Make installation directories.
(mkdir-p module-dir)
(mkdir-p doc)
@@ -874,10 +880,10 @@ inspired by the SCSH regular expression system.")
   (assoc-ref %build-inputs "gdbm"
 
;; compile to the destination
-   (compile-file gdbm.scm-dest
- #:output-file gdbm.go-dest)
+   (compile-file gdbm.scm-dest gdbm.go-dest)
 (inputs
- `(("guile" ,guile-2.0)))
+ `(("guile"

Re: Debbugs handling of Guix patches

[Christopher Allan Webber]

Ludovic Courtès writes:

>> guix-patches isn't listed at
>> https://lists.gnu.org/mailman/listinfo/guix-patches
>
> It does show up at  though.
> I wonder if there’s just a delay somewhere (24h already), or if
> something’s wrong on Savannah.

Yeah, I tried submitting a patch to the new list and it didn't go
through.  I'm looking forward to it being in place, though! :)

guile2.2-gdbm-ffi issue

[Christopher Allan Webber]

(Originally accidentally sent this to guile-de...@gnu.org ... sorry!)

Hiya,

I pushed guile2.2-gdbm because it was just a couple of lines and it
seemed to work and didn't affect anything.  I made a mistake though that
I didn't realize until I ran
"guix environment --ad-hoc guile-next guile2.2-gdbm-ffi --pure".
The trivial-build-system in guile-gdbm-ffi dumps the built module into
(string-append out "/share/guile/site/2.0") ... oops!  Before I ran
--pure, the 2.0 directory was on my load path, and thus I didn't bump
into the error.  (Maybe I should have asked for review anyway...)

I'm trying to think of how to fix this.  I have a very kludgy solution
attached.  I don't feel great about it but I don't know how to signal to
the builder whether it's using guile 2.2 or 2.0 otherwise.  Thoughts?

From 711a23036417807d444729eaa778c9cadffa8646 Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Fri, 10 Feb 2017 19:24:57 -0600
Subject: [PATCH 3/3] guile-gdbm-ffi: Write to correct guile output directory.

* gnu/packages/guile.scm (make-guile-gdbm-ffi): New variable.
(guile-gdbm-ffi, guile2.2-gdbm-ffi): Use make-guile-gdbm-ffi.
---
 gnu/packages/guile.scm | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 3e8ab007b..3e62949f2 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -825,9 +825,11 @@ inspired by the SCSH regular expression system.")
 ;; There are two guile-gdbm packages, one using the FFI and one with
 ;; direct C bindings, hence the verbose name.
 
-(define-public guile-gdbm-ffi
+(define (make-guile-gdbm-ffi guile-2.2?)
   (package
-(name "guile-gdbm-ffi")
+(name (if guile-2.2?
+  "guile2.2-gdbm-ffi"
+  "guile-gdbm-ffi"))
 (version "20120209.fa1d5b6")
 (source (origin
   (method git-fetch)
@@ -848,7 +850,9 @@ inspired by the SCSH regular expression system.")
   (system base compile))
 
  (let* ((out (assoc-ref %outputs "out"))
-(module-dir (string-append out "/share/guile/site/2.0"))
+(module-dir (string-append out "/share/guile/site/"
+   ,(if guile-2.2?
+"2.2" "2.0")))
 (source (assoc-ref %build-inputs "source"))
 (doc (string-append out "/share/doc"))
 (guild (string-append (assoc-ref %build-inputs "guile")
@@ -877,7 +881,8 @@ inspired by the SCSH regular expression system.")
(compile-file gdbm.scm-dest
  #:output-file gdbm.go-dest)
 (inputs
- `(("guile" ,guile-2.0)))
+ `(("guile" ,(if guile-2.2?
+ guile-next guile-2.0
 (propagated-inputs
  `(("gdbm" ,gdbm)))
 (home-page "https://github.com/ijp/guile-gdbm;)
@@ -887,8 +892,11 @@ inspired by the SCSH regular expression system.")
 Guile's foreign function interface.")
 (license gpl3+)))
 
+(define-public guile-gdbm-ffi
+  (make-guile-gdbm-ffi #f))
+
 (define-public guile2.2-gdbm-ffi
-  (package-for-guile-2.2 guile-gdbm-ffi))
+  (make-guile-gdbm-ffi #t))
 
 (define-public guile-sqlite3
   (let ((commit "607721fe1174a299e45d457acacf94eefb964071"))
-- 
2.11.0



signature.asc
Description: PGP signature

Re: `guix pull` over HTTPS

[Christopher Allan Webber]

Leo Famulari writes:

> On Thu, Feb 09, 2017 at 04:55:12PM +0100, Leo Famulari wrote:
>> Does anyone have any specific concerns or advice about changing the
>> value of %snapshot-url in (guix scripts pull) to use the HTTPS URL?
>> Should the change be that simple, or should we do more?
>
> While testing, I realized that an X.509 certificate store is not a
> standard feature of GuixSD, so using Savannah's HTTPS URL will not work
> in all cases.

Maybe it should become standard?

> SSL_CERT_FILE and SSL_CERT_DIR appear to be set unconditionally in (gnu
> system operating-system-environment-variables), so it's not enough to
> test that they are set in order to decide which protocol to download the
> Guix source code with.
>
> Any advice on how to proceed?

Re: Add a generalized git-file? to Guix?

[Christopher Allan Webber]

Ludovic Courtès writes:

> Mathieu Lirzin  skribis:
>
>> From f104b3745097746d6ef89b6198ec7b81e8b679f4 Mon Sep 17 00:00:00 2001
>> From: Mathieu Lirzin 
>> Date: Sun, 29 Jan 2017 00:34:48 +0100
>> Subject: [PATCH] git-download: Add 'git-predicate'.
>>
>> * guix/git-download.scm (git-predicate): New procedure.
>> * gnu/packages/package-management.scm (current-guix): Use it.
>> (make-git-predicate): Remove.
>
> LGTM, thanks!
>
> Ludo'.

I just pushed this.  Switching my various projects over to using it now.
Thank you for doing the work to abstract it out Mathieu!

Re: FOSDEM 2017 what a success!!

[Christopher Allan Webber]

Ludovic Courtès writes:

> It confirmed my feeling that one of the greatest things about Guile and
> Guix is the people.  Thanks for the great talks and the good time we’ve
> had!

+1.  The Guile/Guix devroom has been some of the most fun I've ever had
at a conference.  I'm sure 2018 will be great too, and I'm looking
forward to all the things that will happen in the next year! :)

 - Chris

Re: Guix package incubator (later a channel)

[Christopher Allan Webber]

Pjotr Prins writes:

> On Mon, Feb 06, 2017 at 09:44:48PM +, ng0 wrote:
>> Just a reply on the notabug question (I don't have much time otherwise):
>> Notabug will eventually move to an instance of pagure.io, you can read
>> about this in their own issues where I asked about some question back
>> then (no link, sorry .. my name was 'ng0' back on there). Developing
>> with pagure might be easier (fedora and surrounding communities)
>> compared to the situation they describe in the issue.
>> I started packaging pagure for this reason, which is about ~85% done
>> (services + pagure itself left to wrap it up ... pagure itself is more
>> or less done, just the service are needed. I'm more than happy to pass
>> this on (could rebase the pagure patch), as I know I have taken on some
>> tasks which would be better handled by different people :)
>> That is in case you want to use a GuixSD as a host, otherwise you should
>> not have many problems.
>
> That would be a great result :). Also pagure looks very interesting -
> simpler than gitlab. I like simple.

It does look good:

  Open data: Sources, doc, ticket and pull-requests meta-data are
  available in the web interface but also in git repos which can thus be
  cloned and changed locally.

That's really appealing.

Re: [PATCH] gnu: Add angband. (almost working!)

[Christopher Allan Webber]

Kei Kebreau writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> At the risk of totally destroying my productivity, I thought I'd take a
>> short of packaging a common roguelike.  Unfortunately, it seems like
>> every roguelike I try to package runs into some snag.  In this case,
>> angband can't find ncurses.  I don't know why!
>>
>> Apparently this is related:
>> https://lists.freebsd.org/pipermail/freebsd-ports-bugs/2014-January/271203.html
>> ... but I don't really know enough.
>>
>> Anyone who wants to help contribute to the downfall of various GuixSD
>> users by assisting getting roguelikes into the distribution: this is
>> pretty close, if we can solve that one thing! :)
>>
>>  - Chris
>>
>> From 5a97d5dffa12a3282b57313db8aa609f27729074 Mon Sep 17 00:00:00 2001
>> From: Christopher Allan Webber <cweb...@dustycloud.org>
>> Date: Mon, 12 Sep 2016 20:28:51 -0500
>> Subject: [PATCH] gnu: Add angband.
>>
>> * gnu/packages/games.scm (angband): New variable.
>> ---
>>  gnu/packages/games.scm | 34 +-
>>  1 file changed, 33 insertions(+), 1 deletion(-)
>>
>> diff --git a/gnu/packages/games.scm b/gnu/packages/games.scm
>> index d386e2c..08c00b9 100644
>> --- a/gnu/packages/games.scm
>> +++ b/gnu/packages/games.scm
>> @@ -10,7 +10,7 @@
>>  ;;; Copyright © 2014, 2015 Mark H Weaver <m...@netris.org>
>>  ;;; Copyright © 2015, 2016 Andreas Enge <andr...@enge.fr>
>>  ;;; Copyright © 2015 David Hashe <david.ha...@dhashe.com>
>> -;;; Copyright © 2015 Christopher Allan Webber <cweb...@dustycloud.org>
>> +;;; Copyright © 2015, 2106 Christopher Allan Webber <cweb...@dustycloud.org>
> Should be 2016, I assume. ^
>>  ;;; Copyright © 2015 Ricardo Wurmus <rek...@elephly.net>
>>  ;;; Copyright © 2015, 2016 Alex Kost <alez...@gmail.com>
>>  ;;; Copyright © 2015 Paul van der Walt <p...@denknerd.org>
>> @@ -223,6 +223,38 @@ them, called Jean Raymond, found an old church in which 
>> to hide, not knowing
>>  that beneath its ruins lay buried an ancient evil.")
>>  (license license:gpl3+)))
>>  
>> +(define-public angband
>> +  (package
>> +(name "angband")
>> +(version "4.0.5")
>> +(source
>> + (origin
>> +   (method url-fetch)
>> +   (uri (string-append "http://rephial.org/downloads/4.0/;
>> +   "angband-" version ".tar.gz"))
>> +   (sha256
>> +(base32
>> + "0lpq2kms7hp421vrasx2bkkn9w08kr581ldwik3v0hlq6h7rlxhd"
>> +(build-system gnu-build-system)
>> +(arguments
>> + `(#:tests? #f ;no check target
>> +   ;; #:configure-flags '("--enable-sdl")
>> +   #:phases (modify-phases %standard-phases
>> +  (add-after
>> +  'unpack 'autogen.sh
>> +(lambda _
>> +  (zero? (system* "sh" "autogen.sh"))) 
>> +(native-inputs
>> + `(("autoconf" ,autoconf)
>> +   ("automake" ,automake)))
>> +(inputs `(("ncurses" ,ncurses)))
>> +(home-page "http://rephial.org/;)
>> +(synopsis "Dungeon exploration roguelike")
>> +(description "Classic dungeon exploration roguelike.  Explore the depths
>> +below Angband, seeking riches, fighting monsters, and preparing to fight
>> +Morgoth, the Lord of Darkness.")
>> +(license license:gpl2)))
>> +
>>  (define-public pingus
>>(package
>>  (name "pingus")
>
> I found the issue. You must substitute the instances of the string
> "ncursew5-config" with "ncursesw6-config" before running autogen.sh.
> This is because we only package version 6 of the ncurses library. By
> making this change, I was able to build angband reproducibly. Should we
> enable the SDL, X11 and GTK+ interfaces as well?

Wow nice!

I'm for enabling at least one of the visual interfaces, though I don't
think we need to enable all of them?  Perhaps they should be different
derived package versions?  WDYT?

Re: “Future of Guix” panel

[Christopher Allan Webber]

It looks good.  We won't run out of topics for sure. :)

Pjotr Prins writes:

> Heh. Long list.
>
> On Fri, Feb 03, 2017 at 12:11:15AM +0100, Ludovic Courtès wrote:
>> Hello Guix!
>> 
>> We’ll close FOSDEM with a discussion on the future of Guix and GuixSD:
>> 
>>   https://fosdem.org/2017/schedule/event/futureofguix/
>> 
>> Attached is a semi-structured list of topics that came to mind.  Please
>> feel free to bring your own, either at the event (this Sunday), or on
>> this list, or on IRC during the event maybe?
>> 
>> Ludo’.
>> 
>
>> #+TITLE: Future of Guix (panel)
>> #+STARTUP: content hidestars
>> 
>> https://fosdem.org/2017/schedule/event/futureofguix/
>> Sunday, 16:30, K.4.601
>> 
>> * Organization
>> 
>> ** sharing the work load
>> 
>> *** security updates
>> 
>> *** build farm
>> 
>> *** web site, posts, videos, etc.
>> 
>> *** handling branches: starting & merging ‘core-updates’, ‘staging’
>> 
>> *** release management: setting a date, choosing what goes in, publishing
>> 
>> *** GSoC mentoring
>> 
>> ** review
>> 
>> *** getting more reviewers
>> 
>> *** facilitating the process
>> 
>> *** encouraging newcomers
>> 
>> ** funding
>> 
>> *** development
>> 
>>   - how do we get core & maintenance work funded?
>>   - how do we get funding for new areas (like ‘guix deploy’)?
>> 
>> *** infrastructure (build farm)
>> 
>>   - sustaining infrastructure growth and maintenance
>>   - Jan. 2016 crowdfunding was enough for a start
>> 
>> * Publicity, outreach
>> 
>> ** high-performance computing (HPC)
>> 
>> *** competition: EasyBuild, Spack, Singularity, Shifter
>> 
>> *** stories: MDC, Utrecht, Brisbane
>> 
>> *** possible cooperation in the near future
>> 
>> ** reproducible science
>> 
>> *** how to help projects such as [[https://rescience.github.io/][ReScience]]?
>> 
>> *** make reproducible free software stacks the obvious choice
>> 
>> ** containers, deployment
>> 
>> *** competition: Docker, Kubernetes, NixOps
>> 
>> *** stories: gov.uk (Chris Baines), bayfront.guixsd.org
>> 
>> ** hold Guix User Gatherings (GUGs)?
>> 
>> * Code
>> 
>> ** 1.0 in 2017! (see roadmap in 
>> [[https://www.gnu.org/software/guix/guix-ghm-20160818.pdf][GHM 2016 talk]])
>> 
>> *** ‘guix pull’ and channels
>> 
>> *** Git checkout authentication
>> 
>> *** performance & usability improvements
>> 
>> *** GuixSD graphical installer (John)
>> 
>> *** self-hosted infrastructure
>> 
>>  Cuirass
>> 
>>  ‘guix publish’ improvements
>> 
>>  build machines running GuixSD
>> 
>> *** bug fixes
>> 
>> ** next big thing: ‘guix deploy’
>> 
>> ** scalability
>> 
>> *** better importers & updaters
>> 
>> *** better ‘guix lint -c cve’
>> 
>> ** reproducibility: reach 90% or so, and measure
>> 
>> ** bootstrapping
>> 
>> *** reduce the size of our bootstrap binaries (Bournish? Mes? …)
>> 
>> *** “do something” about compilers that cannot be bootstrapped

Re: FOSDEM social dinner

[Christopher Allan Webber]

Alex Sassmannshausen writes:

> Hello,
>
> To confirm, I've now placed a reservation for Saturday 4 February at
> 19:30 at a Lebanese restaurant called Al Jannah.  I haven't been there
> before, but the menu looks diverse, the prices good and the location
> relatively central.
>
> https://www.tripadvisor.com/Restaurant_Review-g188644-d2039550-Reviews-Al_Jannah-Brussels.html
>
> Address:
> Rue Blaes 59, Brussels 1000, Belgium
>
> There is a direct tram line from ULB to Louise, and then it's a 5-10 min
> walk.  From there it's probably around 15 min walk to the central
> station where there are trains to Antwerp.
>
> List of attendees:
> Leo, Catonano, Amirouche, Efraim, Tomáš, Thomas, Tobias, Christopher
> Baines, Manolis, Ludo, Ricardo, Matias and myself.

Is it possible to add me a bit late to the attendees? :)

Re: A postinst equivalent in Guix?

[Christopher Allan Webber]

John Darrington writes:

> On Wed, Jan 25, 2017 at 02:15:12PM +0100, Ludovic Court??s wrote:
>  Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>  > Ludovic Court??s writes:
>  >
>  >> Georgi Kirilov <kirilov.georg...@gmail.com> skribis:
>  >>
>  >>> On Sat, Jan 21, 2017 at 04:34:55PM +0100, Ludovic Court??s wrote:
>  >>>
>  >>>>To make things more concrete, we could discuss specific packages you 
> are
>  >>>>interested in and see how we could provide them in Guix{,SD}.
>  >>>
>  >>> The package is the old bsd-games bundle. Some of the games need to
>  >>> write score files under /var/lib/bsdgames/
>  >>> You can find attached my patch so far.
>  >>
>  >> The patch looks good to me!
>  >>
>  >> As for /var/lib/bsdgames, then it???s up to the admin to set the right
>  >> permissions on it.  We can ensure that it exists and has the right
>  >> permissions on GuixSD, but on foreign distros, there???s nothing we 
> can
>  >> do.
>  >>
>  >> We could also modify bsd-games such that it falls back to
>  >> ~/.local/bsdgames when /var/lib/bsdgames isn???t accessible (and it 
> would
>  >> be worth submitting upstream).  ISTR this was discussed for one of the
>  >> games present in Guix.
>  >>
>  >> WDYT?
>  >>
>  >> Ludo???.
>  >
>  > I'm a bit wary about GuixSD packages declaring being able to write to
>  > /var/ anything by default.  What would the permissions be?  I guess if
>  > it were world-writable to all "users" group users it would be okayish.
>  >
>  > Note that KoboDeluxe includes a patch snarfed from Debian that comments
>  > out the ability to save score files for this same reason, and it was
>  > marked in Debian as a security patch IIRC...
>
>  Yeah, I think scores in /var are a remnant of the past.  Unix just lacks
>  a good way to address this use case.
>
>  So it sounds best for games to use a score file under $HOME by default.
>
> I always thought the unix way was rather nice.   The scores file was owned by
> "games" and programs which wanted to write to them were setuid games.
>
> That way everyone on the system shares the same scores file.
>
> J'

It's fun but... does anyone still play games on the same shared machine
anymore and compare score files?

Except for maybe nethack on fencepost ... ;)

Re: gnu: gnutls: Add gnutls-3.5.5 and gnutls-with-guile-next.

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello!
>
> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Hello!  I was talking in #guile and realized that even though guile-next
>> now includes glorious https support via gnutls, few have tried it or
>> know how to use it.
>>
>> It turns out you need a specially packaged version of gnutls!  Or,
>> specially packaged until `guile-next' becomes just `guile'.
>>
>> Here's a patch that will give you that package.  You can give it a shot
>> like so:
>>
>>   $ guix environment --ad-hoc guile-next gnutls-with-guile-next --pure
>
> Good idea!  I ended up pushing a slightly different patch, calling the
> package ‘guile2.2-gnutls’ to match what ‘package-for-guile-2.2’ does:
>
>   
> http://git.savannah.gnu.org/cgit/guix.git/commit/?id=079f013be2a3601518a67d65f4e39f7fabddb053
>
> Let me know if anything’s wrong!

Yay!  Thanks for the push.  I'll test shortly.

>> Have fun hacking with https support,
>
> BTW, it would be nice :-) to port this to Guile as well:
>
>   
> http://git.savannah.gnu.org/cgit/guix.git/commit/?id=bc3c41ce36349ed4ec758c70b48a7059e363043a
>   
> http://git.savannah.gnu.org/cgit/guix.git/commit/?id=580deec5b44d623e994e59ef07e9e0c5496762fd
>
> Anyway, thanks for the heads-up!
>
> Ludo’.

Oh, those look important.  I've made a TODO item and I'll try to get to
them soon, but who knows if I will...

Re: A postinst equivalent in Guix?

[Christopher Allan Webber]

Ludovic Courtès writes:

> Georgi Kirilov  skribis:
>
>> On Sat, Jan 21, 2017 at 04:34:55PM +0100, Ludovic Courtès wrote:
>>
>>>To make things more concrete, we could discuss specific packages you are
>>>interested in and see how we could provide them in Guix{,SD}.
>>
>> The package is the old bsd-games bundle. Some of the games need to
>> write score files under /var/lib/bsdgames/
>> You can find attached my patch so far.
>
> The patch looks good to me!
>
> As for /var/lib/bsdgames, then it’s up to the admin to set the right
> permissions on it.  We can ensure that it exists and has the right
> permissions on GuixSD, but on foreign distros, there’s nothing we can
> do.
>
> We could also modify bsd-games such that it falls back to
> ~/.local/bsdgames when /var/lib/bsdgames isn’t accessible (and it would
> be worth submitting upstream).  ISTR this was discussed for one of the
> games present in Guix.
>
> WDYT?
>
> Ludo’.

I'm a bit wary about GuixSD packages declaring being able to write to
/var/ anything by default.  What would the permissions be?  I guess if
it were world-writable to all "users" group users it would be okayish.

Note that KoboDeluxe includes a patch snarfed from Debian that comments
out the ability to save score files for this same reason, and it was
marked in Debian as a security patch IIRC...

Re: Hacker Public Radio (podcast) interview about Guix

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello Guix!
>
> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Hello!  In late 2015 I was interviewed about Guix for Hacker Public
>> Radio.  Obviously, a lot has happened since then, but the ideas are
>> still relevant.  This is mainly aimed at people who are interested in
>> what Guix is and have maybe never even heard of what functional package
>> management is, and I think it covers that well enough.  But I think it
>> will still also be fun for seasoned Guix hackers who like to hear people
>> talk about Guix. ;)
>>
>>   http://hackerpublicradio.org/eps.php?id=2198
>
> I listened to it the other day and found it super interesting and fun!
> I like the description of how you came to Guile and Guix, the intro to
> functional programming, and the great overview of Guix.
>
> Recommended!  :-)
>
> Ludo’.

Thanks for the kind words! :)

Add a generalized git-file? to Guix?

[Christopher Allan Webber]

Thompson, David writes:

> Hi Christopher and Jan,
>
> On Tue, Jan 10, 2017 at 11:28 AM, Christopher Allan Webber
> <cweb...@dustycloud.org> wrote:
>> Jan Nieuwenhuizen writes:
>>
>>> Christopher Allan Webber writes:
>>>
>>>> Thanks!
>>>>
>>>> The file is updated, and even nicer now, since I'm using a hack from
>>>> guile-sdl2 which allows you to set the source to the whole checkout.
>>>
>>> Ohh!  That needs to go in the Guix manual... could git-file? be added
>>> to guix/utils?
>>
>> Maybe!  I think David knows more about the provenance?
>
> I took this code from Ludovic.  See make-git-predicate in
> gnu/packages/package-management.scm in the Guix source tree.
>
>> I agree it would be nice to have in Guix itself.
>
> Agreed.  A generalized and publicly available procedure would be great.
>
> - Dave

Hello!  See the above conversation... 8sync now uses `git-file?' in its
guix.scm, a predicate check which allows for checking out the whole
local directory as a "source" for testing a package.  I borrowed it from
Dave who originally adapted it from some code in Guix itself.  See:

  http://git.savannah.gnu.org/cgit/8sync.git/tree/guix.scm#n62

This is pretty handy; probably other projects would like to make use of
it.  What do we think of making it a generally available utility?

 - Chris

gnu: gnutls: Add gnutls-3.5.5 and gnutls-with-guile-next.

[Christopher Allan Webber]

Hello!  I was talking in #guile and realized that even though guile-next
now includes glorious https support via gnutls, few have tried it or
know how to use it.

It turns out you need a specially packaged version of gnutls!  Or,
specially packaged until `guile-next' becomes just `guile'.

Here's a patch that will give you that package.  You can give it a shot
like so:

  $ guix environment --ad-hoc guile-next gnutls-with-guile-next --pure
  $ guile
  GNU Guile 2.1.5
  Copyright (C) 1995-2016 Free Software Foundation, Inc.

  Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
  This program is free software, and you are welcome to redistribute it
  under certain conditions; type `,show c' for details.

  Enter `,help' for help.
  scheme@(guile-user)> (use-modules (web client) (web uri))
  scheme@(guile-user)> (http-get (string->uri "https://fsf.org;))
  $1 = #< a long response here>
  $2 = "the body, also long"

Have fun trying it with https://slashdot.org/ or whatever you feel like.
(Unless that thing is https://gnu.org/, which doesn't work, but
everything else seems to.  Why is that?)

Have fun hacking with https support,
 - Chris

>From e31e3e9784a978628655fcd7a1f3527e88d475ec Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Tue, 10 Jan 2017 13:25:26 -0600
Subject: [PATCH] gnu: gnutls: Add gnutls-3.5.5 and gnutls-with-guile-next.

* gnu/packages/tls.scm (gnutls-3.5.5, gnutls-with-guile-next): New variables.
---
 gnu/packages/tls.scm | 34 ++
 1 file changed, 34 insertions(+)

diff --git a/gnu/packages/tls.scm b/gnu/packages/tls.scm
index e577421fa..4d9d2135f 100644
--- a/gnu/packages/tls.scm
+++ b/gnu/packages/tls.scm
@@ -211,6 +211,40 @@ required structures.")
 (properties '((ftp-server . "ftp.gnutls.org")
   (ftp-directory . "/gcrypt/gnutls")
 
+(define-public gnutls-3.5.5
+  (package
+(inherit gnutls)
+(source
+  (let ((version "3.5.5"))
+(origin
+  (method url-fetch)
+  (uri (string-append "mirror://gnupg/gnutls/v"
+  (version-major+minor version)
+  "/gnutls-" version ".tar.xz"))
+  (sha256
+   (base32
+"0ag5q3dfxzv0dmqy7q0a8y74yc3m5yzvjrp324l6vqafh3klz6c6")))
+
+(define-public gnutls-with-guile-next
+  (package
+(inherit gnutls-3.5.5)  ; 3.5.5 introduces guile 2.1 / 2.2 support
+(name "gnutls-with-guile-next")
+(build-system gnu-build-system)
+;; Unfortunately we're inheriting this whole thing just to
+(arguments
+ (substitute-keyword-arguments (package-arguments gnutls)
+   ;; change the guile-site-dir to 2.2
+   ((#:configure-flags configure-flags)
+'(list (string-append "--with-guile-site-dir="
+  (assoc-ref %outputs "out")
+  "/share/guile/site/2.2")
+   ;; See the gnutls definition for explaination of these.
+   "--with-default-trust-store-dir=/etc/ssl/certs"
+   "--without-p11-kit"
+(inputs
+ `(("guile" ,guile-next)  ; we want guile-next
+   ("perl" ,perl)
+
 (define-public openssl
   (package
(name "openssl")
-- 
2.11.0

Re: Editing Scheme in the installation image

[Christopher Allan Webber]

Ludovic Courtès writes:

> Mike Gran  skribis:
>
>>>On Sunday, January 8, 2017 2:48 AM, Ludovic Courtès  wrote:
>
> [...]
>
>>>I think it would take more than elisp compilation support to port Emacs
>>>packages to Zile-on-Guile.  Essentially we’d need many APIs that Emacs
>>>provides.
>>
>> I looked at the current beta of paredit, and I quickly scanned
>> the code to look for *emacs* primitives.  Below please find a list
>> of the primitives that Zile is missing.  Note that this isn't the
>> missing *elisp* procedures.
>> Adding the majority of these is quite straightforward, but,
>> some have heretofore been out of scope for Zile. The missing
>> categories of concepts are
>> - lisp and scheme filling and indentation
>> - blinking the cursor or a matching parenthesis
>> - handling comments
>> - handling what emacs calls "lists" which is moving up and down
>>   balanced parentheses
>> Here's the list. I probably missed a few.
>
> Woow, thanks for investigating!
>
> Looks non-trivial.  I wonder how much of it could be trimmed by focusing
> just on the subset of Paredit relating to matching parens.
>
> Ludo’.

I mentioned this earlier in the thread, but I think it got lost...
mit-scheme's edwin includes a scheme-based paredit.  I haven't tried
looking at how much work it would be to port to Zile, but I'm guessing
since it's also scheme, it wouldn't be much work.

I've extracted it from the latest mit-scheme release and included it.
The code is in the public domain.

Maybe it can be of use?

Note: it might also be possible to pilfer some logic from edwin to "fill
in" the missing procedures to make Zile work.



paredit.scm
Description: Binary data

[PATCH] gnu: Add guile-8sync.

[Christopher Allan Webber]

This patch adds guile-8sync.  0.4.0 (well, and 0.4.1, since there was a
bug in 0.4.0's packaging) came out today.

Jan Nieuwenhuizen is also given a copyright line since this is based on
their contribution of the guix.scm in 8sync itself.

From 78683a5d2601d58445bfd82dc9c624e64f2231ed Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Sun, 8 Jan 2017 15:46:34 -0600
Subject: [PATCH] gnu: Add guile-8sync.

* gnu/packages/guile.scm (guile-8sync): New variable.
---
 gnu/packages/guile.scm | 35 ++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 9458ab714..e204ae84f 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -1,7 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017 Ludovic Courtès <l...@gnu.org>
 ;;; Copyright © 2014, 2015 Mark H Weaver <m...@netris.org>
-;;; Copyright © 2015 Christopher Allan Webber <cweb...@dustycloud.org>
+;;; Copyright © 2015, 2017 Christopher Allan Webber <cweb...@dustycloud.org>
 ;;; Copyright © 2016 Alex Sassmannshausen <a...@pompo.co>
 ;;; Copyright © 2016 Ricardo Wurmus <rek...@elephly.net>
 ;;; Copyright © 2016 Erik Edrosa <erik.edr...@gmail.com>
@@ -9,6 +9,7 @@
 ;;; Copyright © 2016 Alex Kost <alez...@gmail.com>
 ;;; Copyright © 2016 Adonay "adfeno" Felipe Nogueira <https://libreplanet.org/wiki/User:Adfeno> <adf...@openmailbox.org>
 ;;; Copyright © 2016 Amirouche <amirou...@hypermove.net>
+;;; Copyright © 2016 Jan Nieuwenhuizen <jann...@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1496,4 +1497,36 @@ enable -f ~/.guix-profile/lib/bash/libguile-bash.so scm
 and then run @command{scm example.scm}.")
   (license gpl3+
 
+(define-public guile-8sync
+  (package
+(name "guile-8sync")
+(version "0.4.1")
+(source (origin
+  (method url-fetch)
+  (uri (string-append "mirror://gnu/8sync/8sync-" version
+  ".tar.gz"))
+  (sha256
+   (base32
+"1fvf8d2s3vvg4nyskbqaiqmlm2x571hv7hizcnmny45zvalydr9h"
+(build-system gnu-build-system)
+(native-inputs `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("guile" ,guile-next)
+ ("pkg-config" ,pkg-config)
+ ("texinfo" ,texinfo)))
+(arguments
+ `(#:phases (modify-phases %standard-phases
+  (add-before 'configure 'setenv
+(lambda _
+  ;; quiet warnings
+  (setenv "GUILE_AUTO_COMPILE" "0"))
+(home-page "https://gnu.org/s/8sync/;)
+(synopsis "Asynchronous actor model library for Guile")
+(description
+ "GNU 8sync (pronounced \"eight-sync\") is an asynchronous programming
+library for GNU Guile based on the actor model.
+
+Note that 8sync is only available for Guile 2.2 (guile-next in Guix).")
+(license lgpl3+)))
+
 ;;; guile.scm ends here
-- 
2.11.0



signature.asc
Description: PGP signature

Hacker Public Radio (podcast) interview about Guix

[Christopher Allan Webber]

Hello!  In late 2015 I was interviewed about Guix for Hacker Public
Radio.  Obviously, a lot has happened since then, but the ideas are
still relevant.  This is mainly aimed at people who are interested in
what Guix is and have maybe never even heard of what functional package
management is, and I think it covers that well enough.  But I think it
will still also be fun for seasoned Guix hackers who like to hear people
talk about Guix. ;)

  http://hackerpublicradio.org/eps.php?id=2198

It's not a perfect interview, but I do think it's a lot of fun.
Hopefully you enjoy it!

 - Chris

Re: Graphical Installer - Call for Testing.

[Christopher Allan Webber]

Thompson, David writes:

> Hi John,
>
> On Wed, Jan 4, 2017 at 2:59 AM, John Darrington
>  wrote:
>> The wip-installer branch is ready for testing and general review.
>>
>> This branch provides a curses based graphical interface to the disk
>> images used for installing GuixSD. It is intended to allow users
>> unfamiliar with bash and other aspects of unix-like operating systems
>> to easily install GuixSD.   This means, it should allow you to install
>> GuixSD on a bare PC, without ever touching a shell (whilst still
>> providing that option for those who want it).
>
> I don't have any real feedback, but I just want to say that this is an
> awesome initiative that will make GuixSD accessible to many more
> people!  Thanks for working on this!
>
> - Dave

What Dave said.  Thank you :)

Re: Editing Scheme in the installation image

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello Guix!
>
> One issue that’s often reported is that it’s inconvenient to edit the
> config file with all its parentheses in the installation image given the
> available options (Zile, Nano, and nvi).
>
> Something like Paredit and ‘show-paren-mode’ in Emacs would help avoid
> mistakes such as unbalanced parenthesis.  However ‘emacs-minimal’ takes
> 180MiB and it would be unreasonable to include it.
>
> So I figured we could use Zile-on-Guile¹ (yes!) and extend it to have
> something that resembles Paredit, like:
>
>
> Of course, it takes more than these few lines to write a real Paredit,
> but still, wouldn’t it be cool?  :-)
>
> What do people (Mike in particular!) think?

Note that I saw that mit-scheme's edwin now includes some sort of
paredit:

https://www.gnu.org/software/mit-scheme/release.html
   New parenthesis-editing minor mode M-x paredit-mode. 

iirc edwin is written in scheme, so adapting their paredit-mode might be
less work than writing our own from scratch?

Re: Creating a docker image with Guix

[Christopher Allan Webber]

Ricardo Wurmus writes:

> Hi Guix,
>
> I just played around with Docker and built up a command to create a
> Docker image for Emacs.
>
> Can anyone find a more elegant way to do this?
>
> --8<---cut here---start->8---
> guix environment --ad-hoc \
> coreutils bash emacs-no-x-toolkit -- \
> sh -c 'tar -c $(guix gc --requisites $GUIX_ENVIRONMENT) | \
>docker import -c "ONBUILD RUN [\"$GUIX_ENVIRONMENT/bin/ln\", 
> \"-s\", \"$GUIX_ENVIRONMENT/bin\", \"/bin\"]" - emacs-base' \
>&& echo -e "FROM emacs-base\nCMD [\"/bin/emacs\"]" | \
>docker build -
> --8<---cut here---end--->8---
>
> What happens here is this:
>
> * using “guix environment” build a temporary profile containing
>   coreutils (for “ln”), bash (for “sh”), and emacs
>
> * inside the environment $GUIX_ENVIRONMENT points to the profile; take
>   all of the requisites for this profile (“guix gc --requisites”) and
>   send them to “docker import” using a tar pipe
>
> * modify the base image such that it links the profile’s “bin” directory
>   to “/bin” when a derivative image is built
>
> * name that base image “emacs-base”
>
> * then build a derivative of this base image with “docker build” (this
>   causes “ln -s” to be executed) and make it run the command
>   “/bin/emacs” by default
>
> The result is an unnamed image (“docker build” reports an image id such
> as “ae8d6281b56f”) that can be run interactively like this:
>
> docker run -it --rm ae8d6281b56f
>
> What do you think?  Gross?
>
> ~~ Ricardo

I won't comment on the specific application, but I think it's a good
thing to have.  I was watching the conversation about providing an Emacs
Docker image on emacs-devel, and despite my caveats about Docker, I
think having Guix be a good way to get a "reproducible" image here is a
good idea.  If nothing else, maybe it could become a good reason for
Docker-using people to explore Guix...

Thanks for exploring this, Ricardo!
 - Chris

Re: Graphical GuixSD Installer

[Christopher Allan Webber]

John Darrington writes:

> There is a new branch called "wip-installer" providing a graphical
> installer for GuixSD.  A screenshot of it in action is attached.
>
> We want GuixSD to be accessible to as many people as possible,
> including people who may not (yet) be comfortable using bash and/or
> guile.
>
> Testers and hackers are wanted!
>
> To try out the installer run:
>
>  guix system installer
>
> It uses guile-ncurses so that will have to be installed and be
> in your load path.

Wowee!  I'm so excited this is moving forward!

Re: Guix IceCat users have had early access to security fixes

[Christopher Allan Webber]

Mark H Weaver writes:

> Yesterday, Mozilla released Firefox ESR 45.6 and announced several CVEs
> fixed by it:
>
>   https://www.mozilla.org/en-US/security/advisories/mfsa2016-95/
>
> I'm pleased to announce that Guix users of IceCat have had early access
> all of these fixes.
>
> Since November 30 (commit 9689e71d2f2b5e766415a40d5f5ab267768d217d),
> we've had fixes for CVE-2016-9897, CVE-2016-9898, CVE-2016-9899,
> CVE-2016-9900, CVE-2016-9904, and 4 out of 11 patches for CVE-2016-9893.
>
> Since December 3 (commit 5bdec7d634ce0058801cd212e9e4ea56e914ca0c),
> we've had the fixes that were later announced as CVE-2016-9901,
> CVE-2016-9902, CVE-2016-9905, and another patch for CVE-2016-9893.
>
> On December 10 (commit 56c394ee4397015d6144dab002ee43fc7e32a331), I
> cherry-picked the remaining fixes from the not-yet-released Firefox
> ESR 45.6: CVE-2016-9895, and the final six patches for CVE-2016-9893.
>
>   Mark

Wow!  Thank you for staying on top of things, Mark!

Re: Berlin Guix gathering: notes!

[Christopher Allan Webber]

Ricardo Wurmus writes:

> Hi Guix!
>
> Yesterday a couple of Guix hackers got together in Berlin to discuss a
> wide range of Guix-related topics.  Attached are the notes I took during
> the meeting.  If anything is unclear about these notes please just ask
> (while the meeting is still fresh in our minds).
>
> Hope you’ll find this interesting!

I found it very interesting!  Thanks for taking the notes.
Looking forward to the presumably-next gathering of Guix people at
FOSDEM :)

Re: Guix gathering in Berlin, Dec. 12th

[Christopher Allan Webber]

Ludovic Courtès writes:

> John Darrington  skribis:
>
>> On Thu, Dec 08, 2016 at 10:57:59AM +0100, Ludovic Court??s wrote:
>>
>>  So far I think we???re just 4 or 5 people who said they would come.
>>  Please bring your own topics!
>>  
>>  Mine include: ???guix pull???, Guix in HPC, the new build farm,
>>  organizational issues.
>>
>> I have a prototype for a curses based installer, for which I'd appreciate
>> critique (hopefully constructive!)
>
> Woohoo!  Sounds cool.
>
> Ludo’.

Wow!  I definitely want to hear more about this. :)

[PATCH] Add guile-fibers (needs help)

[Christopher Allan Webber]

Hello,

I started a patch to add guile-fibers, which I know many of us are
intertested in giving a spin, but I haven't gotten things to build
right.  The funny thing is I can get it to build fine from the tarball
itself.  I'm missing something in the package, but I don't know what.

Here's the error I get during "make":

  Backtrace:
  In ice-9/boot-9.scm:
2788:17 19 (resolve-interface (fibers internal) #:select _ #:hide _ …)
2713:10 18 (_ (fibers internal) _ _ #:ensure _)
2989:16 17 (try-module-autoload _ _)
 2325:4 16 (save-module-excursion #)
3009:22 15 (_)
  In unknown file:
14 (primitive-load-path "fibers/internal" #)
  In ice-9/eval.scm:
 710:20 13 (primitive-eval (define-module (fibers internal) # (…) …))
  In ice-9/psyntax.scm:
1209:36 12 (expand-top-sequence ((define-module (fibers #) # # …)) …)
1156:24 11 (parse _ (("placeholder" placeholder)) ((top) #(# # …)) …)
 279:10 10 (parse _ (("placeholder" placeholder)) (()) _ c (eval) …)
  In ice-9/eval.scm:
 293:34  9 (_ #)
  In ice-9/boot-9.scm:
2849:10  8 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
2788:17  7 (resolve-interface (fibers epoll) #:select _ #:hide _ # …)
2713:10  6 (_ (fibers epoll) _ _ #:ensure _)
2989:16  5 (try-module-autoload _ _)
 2325:4  4 (save-module-excursion #)
3009:22  3 (_)
  In unknown file:
 2 (primitive-load-path "fibers/epoll" #)
 1 (load-extension "epoll" "init_fibers_epoll")
  In ice-9/boot-9.scm:
 753:26  0 (dispatch-exception _ _ _)
  
  ice-9/boot-9.scm:753:26: In procedure dispatch-exception:
  ice-9/boot-9.scm:753:26: In procedure dynamic-link: file: "epoll", message: 
"file not found"
  make[1]: *** [Makefile:1381: fibers/channels.go] Error 1
  make[1]: Leaving directory 
'/tmp/guix-build-guile-fibers-0.3.0.drv-0/fibers-0.3.0'
make: *** [Makefile:498: all] Error 2
  phase `build' failed after 3.4 seconds

Funny thing is, if I become the builder user and then try running
./configure && make after sourcing the environment variables, things
build fine.

One of the two must be true, afaict:
 - ... either there's something in the system environment that's being
   included that I'm missing
 - ... or Guix is patching the gnu-build-system stuff in such a way
   that things are different than when I run ./configure && make?

I'd really like this package to work but am not sure I have the time to
continue on it soon.  If someone wants to take over from here, go for
it.  Maybe what I'm missing is something obvious...

 - Chris

>From d7952e26edb3732516f1e1743383a4ef68832661 Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Mon, 5 Dec 2016 12:18:23 -0600
Subject: [PATCH] gnu: Add guile-fibers.

* gnu/packages/guile.scm (guile-fibers): New variable.
---
 gnu/packages/guile.scm | 30 +-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index 4572544..4568434 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -1,7 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2012, 2013, 2014, 2015, 2016 Ludovic Courtès <l...@gnu.org>
 ;;; Copyright © 2014, 2015 Mark H Weaver <m...@netris.org>
-;;; Copyright © 2015 Christopher Allan Webber <cweb...@dustycloud.org>
+;;; Copyright © 2015, 2016 Christopher Allan Webber <cweb...@dustycloud.org>
 ;;; Copyright © 2016 Alex Sassmannshausen <a...@pompo.co>
 ;;; Copyright © 2016 Ricardo Wurmus <rek...@elephly.net>
 ;;; Copyright © 2016 Erik Edrosa <erik.edr...@gmail.com>
@@ -1412,4 +1412,32 @@ type system, elevating types to first-class status.")
 dictionary and suggesting spelling corrections.")
 (license gpl3+)))
 
+(define-public guile-fibers
+  (package
+(name "guile-fibers")
+(version "0.3.0")
+(source (origin
+  (method url-fetch)
+  (uri (string-append "https://wingolog.org/pub/fibers/fibers-;
+  version ".tar.gz"))
+  (sha256
+   (base32
+"1dc6d2ncclrd7napzf5b0mbw3xxsv43kb471ciz9v7r22277n9m7"
+(build-system gnu-build-system)
+(native-inputs
+ `(("texinfo" ,texinfo)
+   ("pkg-config" ,pkg-config)))
+(inputs
+ `(("guile" ,guile-next)))
+(synopsis "Concurrent ML-like concurrency for Guile")
+(description
+ "Fibers adds concurrency to Guile, inspired by systems like Concurrent ML
+and Erlang.  Fibers are lightweight \"thread\"-like objects which communicate
+through channels.
+
+Note that Fibers makes use of some Guile 2.1/2.2-specific features and
+is not available for Guile 2.0.")
+(home-page "https://github.com/wingo/fibers;)
+(license lgpl3+)))
+
 ;;; guile.scm ends here
-- 
2.10.2

Re: Hosting a GuixSD server on commodity hosting platforms, a journey

[Christopher Allan Webber]

ng0 writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> Onwards and upwards!
>>  - Chris
>
> In theory we could also try to init GuixSD from within an Debian
> image (server), couldn't we?
>
> So we could also make a list of commonly offered virtual server
> distribution choices and try to init GuixSD from within them and
> see if it just works.
> Is there anyone who tried that before?

That's what most of the NixOS guides do.

(Btw, was moving this from help-guix to guix-devel for a specific
reason?)

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
> Adding a ‘config’ action where we see fit would certainly make sense,
> yes.  I guess it’ll have to be decided on a case-by-case basis, and
> perhaps we’ll see that this pattern makes sense for a whole class of
> services.
>
> Ludo’.

Ok!  Well, glad we had this discussion... if I bump into the right
itch-scratching moment maybe I'll get the exposing shepherd actions
implemented and we can go from there.  Otherwise others are welcome to!
:)

 - Chris

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Ludovic Courtès writes:

> Ricardo Wurmus <rek...@elephly.net> skribis:
>
>> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>>
>>>  - We could also have a shepherd action like "herd config mediagoblin"
>>>that would merely spit out the configuration file path... so someone
>>>could do something like:
>>>  $ foo-db dump-db --config `herd config foo-db`
>>
>> Here’s another similar idea:
>>
>> herd environment  -- some command
>>
>> The environment for a service may be represented by a bunch of
>> environment variables.  “some command” then runs within this
>> environment and can access the variables.
>>
>> So if “foo-db” can read its configuration file from FOO_DB_CONFIG the
>> environment definition would be something like
>>
>> #~(("FOO_DB_CONFIG" . (string-append #$foo-db "/etc/config")))
>
> Problem is that in many cases the config file is passed via a
> command-line option to the daemon.
>
> Ludo’.

Yeah, I'm not sure it would work.

But if we were looking for inspiration in what other things have been
done in Guix, we could do something along the lines of "guix system vm"
where it returns a path to a derivation with the appropriate wrapping
already done, something like:

  `herd cmd ` --some --args

It's a bit hacky in a different way though!

 - Chris

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Chris Marusich writes:

> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>> So shepherd actions are probably fine for something like "herd status
>> mcron" but for running slow and expensive operations, we need some way
>> to expose the config file.
>
> The speed and cost of an operation are orthogonal to whether or not that
> operation requires access to a config file.

But they aren't orthogonal to whether or not it's a shepherd action, if
a shepherd action is expected to be nonblocking.

>> SO, the two ways to do that seem to be:
>>  - Populate those configs in /etc/ (maybe providing prefix/suffix option
>>for multiple instances)... probably ok since we expect situations
>>that need these configs to be relatively rare.
>
> Putting stuff into /etc seems undesirable because (1) putting things
> outside the immutable store seems like an invitation to meddle with the
> files, and (2) the possibility of file name conflicts exists, which you
> are already aware of.  However, this might be the simplest solution, so
> if nothing else seems better, I think it would be reasonable to do.

Those are definitely concerns (though hopefully people wouldn't be
modifying things in /etc since "guix system reconfigure" will splat over
any changes).

>>  - We could also have a shepherd action like "herd config mediagoblin"
>>that would merely spit out the configuration file path... so someone
>>could do something like:
>>  $ foo-db dump-db --config `herd config foo-db`
>
> This seems less desirable than putting things into /etc because it
> doesn't seem to be in line with the intended use of Shepherd.  My
> understanding is that Shepherd's job is to nanny the system's processes.
> Responding to queries about the location of the services' config files
> doesn't seem germane to that job.

I agree that it seems a bit strange, but no solution really seems great.
However, I don't think it's totally outside the reasonable realm of
shepherd.  Shepherd actions are to execute some operation on a process
that's running (or which could run).  Returning the contextual
information of what config file is being used can fit that paradigm.
But it does feel a bit like a shoehorn.

> As I see it, there is another possible solution: Modify the
> service/daemon/tool so that it is no longer necessary to expose the
> config file location in the first place.  I'm still not sure which
> daemon/service we're talking about here, but surely the daemon/service
> knows where its configuration file is when it starts up.  Surely it
> could be made to remember that, if it doesn't already.  Surely a tool
> could be made which queries the service/daemon for that information if
> necessary.  Surely the service/daemon could be made to perform
> operations like a database dump without being told where its own
> configuration file is.

And where would the configuration file be?  And who would define it,
when?  That's effectively what imperative distributions do, and the
application "knows" to look in /etc/, for some user-mutated file.

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Ludovic Courtès writes:

>> True.  Though we still run, potentially, into problems where multiple
>> instances of some service are provided, eg multiple mediagoblin servers
>> or mail daemons or etc.
>
> Sure, definitely.  All I meant was that populating /etc can be done as a
> quick stop-gap measure when it makes sense, but it’s not a great
> solution, notably because of the multiple-instance problem you describe.

Right.

>> Speaking of I/O from commands, I wonder how you'd give any kind of
>> output back through an action to the herd?  Afaict the protocol supports
>> it and allows sending back "messages" that will be displayed, but
>> nothing uses it yet.  There's a  record type that afaict
>> nothing uses at all.
>
>  is used for every reply sent by the daemon, in
> (shepherd).
>
> However, this hasn’t been thought to provide interactive commands and
> such things; I’m not sure it would be a great idea to support
> interactive commands, dunno.

Yeah, I didn't think a "feeding back input" option was likely to be
possible.  I understand why it isn't.

> The protocol currently is just: you connect, you send a request, you get
> a reply, and you disconnect.  Actions are expected to be non-blocking.

Okay, an expectation of non-blocking behaviour is useful to know.
Especially because I'm not sure it's a guarantee we can provide.  Eg,
imagine one of the previous commands, such as dumpdb or gc, on a really
large database.  That could block for a bit.

So shepherd actions are probably fine for something like "herd status
mcron" but for running slow and expensive operations, we need some way
to expose the config file.

SO, the two ways to do that seem to be:
 - Populate those configs in /etc/ (maybe providing prefix/suffix option
   for multiple instances)... probably ok since we expect situations
   that need these configs to be relatively rare.
 - We could also have a shepherd action like "herd config mediagoblin"
   that would merely spit out the configuration file path... so someone
   could do something like:
 $ foo-db dump-db --config `herd config foo-db`

WDYT?
 - Chris

Re: [herd] Have status display services as a bulleted list.

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> I don't know about you, but I am totally unable to read the
>> "herd status" line as it exists in the current state.  I mean, I love
>> sexps, but even I don't like a completely flat and ungreppable list
>> printed to stdout.
>
> Agreed!
>
>> *** New output ***
>>
>> cwebber@oolong:~/devel/shepherd$ sudo ./herd -s /var/run/shepherd/socket 
>> status
>> Started:
>>  + file-system-/run/systemd
>>  + xorg-server
>>  + file-system-/sys/fs/cgroup/cpuacct
>>  + file-system-/mnt/debian
>>  + syslogd
>>  + term-tty1
>>  + root-file-system
>>  + file-system-/sys/fs/cgroup/blkio
>>  + avahi-daemon
>
> Works for me!  I’m just wondering if there are ideas we could/should
> follow more closely the output of ‘systemctl status’.  IIRC, it displays
> a couple of lines for each service; for instance, we could display the
> “running” value and dependencies of each service, indented below its
> name.
>
> WDYT?  Would that make sense?
>
> Ludo’.

It's a good idea, and I think we should shoot for better output.
Listing a tree of dependencies is a great idea.

But I'm unlikely to have time to get to it soon.  I think it should be
on our TODO list, but for now, the preceding patch will still be a nice
improvement.

If someone else wants to step up to make those changes, of course,
great!

 - Chris

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Chris Marusich writes:
>>
>>>>>  - Initializing a data store.  For example, in dirvish I need to run
>>>>>a command to initialize a "vault" where I will be storing my data.
>>>>>  - Manually invoking a garbage collection utility.
>>>>>  - Manually invoking an integrity check utility.
>>>>>  - Possibly some side effect involving querying the network.
>>>>>  - Running schema migrations.
>>>>>
>>>>> All sorts of things!  Most of them (all?) involve state or side effects,
>>>>> but plenty of our most important services will be "state overlords" of
>>>>> some type.
>>>
>>> Why do those activities need to be represented as actions in Shepherd?
>>> If we're running a daemon or service that already exposes a mechanism
>>> for manually running tasks like these, then can't we just use "the usual
>>> mechanism" for doing it?  For example, if we're running a daemon that
>>> already provides a script to perform garbage collection, can't we just
>>> invoke that script?  It isn't clear to me why we would need to model
>>> domain-specific actions like that in Shepherd, although I can see why it
>>> might be convenient.  Am I missing something?
>>
>> So sure, we can run "foo-db gc" occasionally (though system
>> administrators sometimes have to run these kinds of commands by hand).
>> But what about "foo-db dumpdb"?  That's not something we just run on a
>> cronjob.  You need access to that command.  And in order for the command
>> to do the right thing, it might need access to the config file.
>
> Oh I see, so the main issue is getting a hand on that config file.
>
> In that case, yes, a Shepherd action would a good way to achieve it.
>
> It’s also a situation where adding the config file to /etc would be
> reasonable (until Shepherd actions can actually be added :-)).

True.  Though we still run, potentially, into problems where multiple
instances of some service are provided, eg multiple mediagoblin servers
or mail daemons or etc.  Though maybe a prefix/suffix could be provided
by the user to the service definition, so it would really shouw up like
/etc/mediagoblin-foosite.org

It might also be possible to do "herd config foo-db" with "config" as an
action and get back your config file to stdout.

Note: I'm interested still in exploring the shepherd actions stuff
still... though I did realize this morning that it wouldn't help in the
rare commands that have interactive input... there's no way to send
input/output in that way through the herd afaict!  Oh well, that's
probably pretty rare.

Speaking of I/O from commands, I wonder how you'd give any kind of
output back through an action to the herd?  Afaict the protocol supports
it and allows sending back "messages" that will be displayed, but
nothing uses it yet.  There's a  record type that afaict
nothing uses at all.

>> There are some other things where we can try to initialize or run
>> migrations automatically, but that stuff can be very dangerous to just
>> do implicitly.  Now note, I *do* think we want to handle some of this
>> stuff behind the scenes as much as possible so that users don't have to
>> think about it.  But have you ever done a *really big* database schema
>> migration?
>
> I guess not ;-), but I think it may be best to simply prevent the DB
> service from starting when a migration needs to happen, and let the user
> handle it explicitly and restart the DB service when they’re done.
>
> Ludo’.

Yep!  Does require handing them a config, but I guess we've discussed
some reasonable ways to make that happen for now.

I did test out extending the etc-service-type this week; it was pretty
easy!

 - Chris

[herd] Have status display services as a bulleted list.

[Christopher Allan Webber]

I don't know about you, but I am totally unable to read the
"herd status" line as it exists in the current state.  I mean, I love
sexps, but even I don't like a completely flat and ungreppable list
printed to stdout.

*** Previous output ***

cwebber@oolong:~/devel/shepherd$ sudo herd -s /var/run/shepherd/socket status
Started: (file-system-/run/systemd xorg-server 
file-system-/sys/fs/cgroup/cpuacct file-system-/mnt/debian syslogd term-tty1 
root-file-system file-system-/sys/fs/cgroup/blkio avahi-daemon 
file-system-/sys/fs/cgroup upower-daemon console-font-tty6 console-font-tty3 
nscd file-system-/run/user file-system-/sys/fs/cgroup/memory 
file-system-/dev/pts loopback term-tty2 user-processes term-tty4 postgres 
file-system-/sys/fs/cgroup/perf_event file-system-/sys/fs/cgroup/cpuset 
device-mapping-home-luks guix-daemon file-system-/sys/fs/cgroup/elogind 
file-system-/sys/fs/cgroup/devices file-system-/dev/shm dbus-system 
console-font-tty5 term-tty6 host-name term-tty3 
file-system-/sys/fs/cgroup/hugetlb file-system-/sys/fs/cgroup/cpu 
file-system-/home networking ntpd console-font-tty4 urandom-seed 
user-file-systems file-system-/sys/fs/cgroup/freezer file-system-/gnu/store 
root console-font-tty2 term-tty5 console-font-tty1 udev)
Stopped: (ssh-daemon)

*** New output ***

cwebber@oolong:~/devel/shepherd$ sudo ./herd -s /var/run/shepherd/socket status
Started:
 + file-system-/run/systemd
 + xorg-server
 + file-system-/sys/fs/cgroup/cpuacct
 + file-system-/mnt/debian
 + syslogd
 + term-tty1
 + root-file-system
 + file-system-/sys/fs/cgroup/blkio
 + avahi-daemon
 + file-system-/sys/fs/cgroup
 + upower-daemon
 + console-font-tty6
 + console-font-tty3
 + nscd
 + file-system-/run/user
 + file-system-/sys/fs/cgroup/memory
 + file-system-/dev/pts
 + loopback
 + term-tty2
 + user-processes
 + term-tty4
 + postgres
 + file-system-/sys/fs/cgroup/perf_event
 + file-system-/sys/fs/cgroup/cpuset
 + device-mapping-home-luks
 + guix-daemon
 + file-system-/sys/fs/cgroup/elogind
 + file-system-/sys/fs/cgroup/devices
 + file-system-/dev/shm
 + dbus-system
 + console-font-tty5
 + term-tty6
 + host-name
 + term-tty3
 + file-system-/sys/fs/cgroup/hugetlb
 + file-system-/sys/fs/cgroup/cpu
 + file-system-/home
 + networking
 + ntpd
 + console-font-tty4
 + urandom-seed
 + user-file-systems
 + file-system-/sys/fs/cgroup/freezer
 + file-system-/gnu/store
 + root
 + console-font-tty2
 + term-tty5
 + console-font-tty1
 + udev
Stopped:
 - ssh-daemon

Whew!  I can read that finally.  And grep it too!

Speaking of grepping, the new bulleted behavior uses a "+" for services
enabled, and a "-" for disabled services.  That way if you do a grep,
you'll still know whether it was enabled/disabled.

 - Chris

>From d9391382286b1dc5a0719c9e1e6ac1e5941bbb87 Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cweb...@dustycloud.org>
Date: Wed, 23 Nov 2016 14:53:18 -0600
Subject: [PATCH] herd: Have status display services as a bulleted list.

* modules/shepherd/scripts/herd.scm (display-status-summary): Display
services as an ascii bulleted list.
---
 modules/shepherd/scripts/herd.scm | 13 +
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/modules/shepherd/scripts/herd.scm b/modules/shepherd/scripts/herd.scm
index 0ad52b2..06256c1 100644
--- a/modules/shepherd/scripts/herd.scm
+++ b/modules/shepherd/scripts/herd.scm
@@ -46,6 +46,11 @@ of pairs."
 
 (define (display-status-summary services)
   "Display a summary of the status of all of SERVICES."
+  (define (display-services header bullet services)
+(display header)
+(for-each (lambda (service)
+(format #t " ~a ~a~%" bullet (service-canonical-name service)))
+  services))
   (call-with-values
   (lambda ()
 (partition (match-lambda
@@ -53,10 +58,10 @@ of pairs."
   (car (assoc-ref properties 'running
services))
 (lambda (started stopped)
-  (format #t (l10n "Started: ~a~%")
-  (map service-canonical-name started))
-  (format #t (l10n "Stopped: ~a~%")
-  (map service-canonical-name stopped)
+  (display-services (l10n "Started:\n") "+"
+started)
+  (display-services (l10n "Stopped:\n") "-"
+stopped
 
 (define (display-detailed-status services)
   "Display the detailed status of SERVICES."
-- 
2.10.2

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Chris Marusich writes:

> Hi Chris!
>
> Thanks for entertaining my questions.
>
> Christopher Allan Webber <cweb...@dustycloud.org> writes:
>
>>> So sure, we can run "foo-db gc" occasionally (though system
>>> administrators sometimes have to run these kinds of commands by hand).
>>> But what about "foo-db dumpdb"?  That's not something we just run on a
>>> cronjob.  You need access to that command.  And in order for the command
>>> to do the right thing, it might need access to the config file.
>
> I'm having trouble understanding the motivation here.  Can you clarify
> why it's preferable to model the action as a Shepherd action, even
> though a mechanism already exists to perform the action (e.g., "foo-db
> dumpdb")?
>
> I don't know a lot about Shepherd (yet!), but it seems like you COULD
> model the action as a Shepherd action.  The question is: WHY would that
> be better than just performing the action via "the usual" mechanism?  If
> you can provide a more concrete example which illustrates the problem
> for a specific service/daemon, it might help me to understand where
> you're coming from.

Sure, sorry I'm not being clear.

So say you wanted to run `foo-db dumpdb'.  Now while most foo-db setups
point to /var/lib/foodb/, you have your foo-db setup mounted on some
external or network storage or something, so it's mounted at
/mnt/dbdisk/foodb/.  With me so far?

And here's how we configured that, in our GuixSD configuration:

  (operating-system
;; ...
(services (foo-db-service #:path "/mnt/dbdisk/foodb")))

Okay, cool!  So the way this works is that the foo-db-service procedure
sets up the service so that it passes in a configuration file that it
generated with the path set appropriately, as above.  So when shepherd
starts/stops this service, it uses this config, and it knows where the
data is.  (It probably does something like
  `foo-db run --config=/gnu/store/.../foodb.cfg')

So now you want to back up the database manually.  No problem!  You have
that `foo-db dumpdb' command.

But you run it... and it doesn't work!  foo-db assumed that your data
would be in the default location, /var/lib/foodb/.  The command you
*want* to run is something like:

  `foo-db dumpdb --config=/path/to/foodb.cfg'

Now... your GuixSD system has, somewhere buried in it, a config file
that it passes in when shepherd starts/stops the daemon.  It's somewhere
in /gnu/store/ ... but you don't know where it is!  GuixSD has cleverly
provided this config file to the closure of its own daemon managing
environment, but has given you no way to access it.

So when you try to back up your database, too bad!  It won't work!

Does that make sense now?

Anyway, I think exposing the `actions' slot in shepherd is the way to
go.  We could already use it for some things other than the theoretical
examples I've given.  Last night I wanted to do something like:

  sudo mcron --schedule=10

... which would, according to the mcron docs, show me the next ten
upcoming tasks that are scheduled!  I wanted to verify that I was
writing my mcron procedures right.  Likewise, my shepherd configuration
seems obscured (well sort of, I can figure it out by looking at
`ps auwwwx' but I don't want to do that).  It would be nice if I could
do:

  sudo herd list-schedule mcron 10

Anyway, hopefully I've made my case! :)
 
 - Chris

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Chris Marusich writes:
>
>>>>  - Initializing a data store.  For example, in dirvish I need to run
>>>>a command to initialize a "vault" where I will be storing my data.
>>>>  - Manually invoking a garbage collection utility.
>>>>  - Manually invoking an integrity check utility.
>>>>  - Possibly some side effect involving querying the network.
>>>>  - Running schema migrations.
>>>>
>>>> All sorts of things!  Most of them (all?) involve state or side effects,
>>>> but plenty of our most important services will be "state overlords" of
>>>> some type.
>>
>> Why do those activities need to be represented as actions in Shepherd?
>> If we're running a daemon or service that already exposes a mechanism
>> for manually running tasks like these, then can't we just use "the usual
>> mechanism" for doing it?  For example, if we're running a daemon that
>> already provides a script to perform garbage collection, can't we just
>> invoke that script?  It isn't clear to me why we would need to model
>> domain-specific actions like that in Shepherd, although I can see why it
>> might be convenient.  Am I missing something?
>
> So sure, we can run "foo-db gc" occasionally (though system
> administrators sometimes have to run these kinds of commands by hand).
> But what about "foo-db dumpdb"?  That's not something we just run on a
> cronjob.  You need access to that command.  And in order for the command
> to do the right thing, it might need access to the config file.
>
> There are some other things where we can try to initialize or run
> migrations automatically, but that stuff can be very dangerous to just
> do implicitly.  Now note, I *do* think we want to handle some of this
> stuff behind the scenes as much as possible so that users don't have to
> think about it.  But have you ever done a *really big* database schema
> migration?
>
> We run into two challenges:
>  - Now we're trying to "idempotently manage state", which it turns out
>is very hard (and the source of many bugs in devops tooling)
>  - Some commands either need to be run manually occasionally, or are
>never automatically run (see the dumpdb example).
>
> Does this make sense?
>  - Chris

FWIW, I do agree that in most cases, we can put these in the background
and automate them so that users don't need to see them.  Possibly that's
true for the key things I want to do for dirvish. :)  So we can and
should when we can!

Where we can't (a dropdb type command, or something like "borg mount")
I do think it would be nice to expose Shepherd actions.

 - Chris

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Chris Marusich writes:

>>>  - Initializing a data store.  For example, in dirvish I need to run
>>>a command to initialize a "vault" where I will be storing my data.
>>>  - Manually invoking a garbage collection utility.
>>>  - Manually invoking an integrity check utility.
>>>  - Possibly some side effect involving querying the network.
>>>  - Running schema migrations.
>>>
>>> All sorts of things!  Most of them (all?) involve state or side effects,
>>> but plenty of our most important services will be "state overlords" of
>>> some type.
>
> Why do those activities need to be represented as actions in Shepherd?
> If we're running a daemon or service that already exposes a mechanism
> for manually running tasks like these, then can't we just use "the usual
> mechanism" for doing it?  For example, if we're running a daemon that
> already provides a script to perform garbage collection, can't we just
> invoke that script?  It isn't clear to me why we would need to model
> domain-specific actions like that in Shepherd, although I can see why it
> might be convenient.  Am I missing something?

So sure, we can run "foo-db gc" occasionally (though system
administrators sometimes have to run these kinds of commands by hand).
But what about "foo-db dumpdb"?  That's not something we just run on a
cronjob.  You need access to that command.  And in order for the command
to do the right thing, it might need access to the config file.

There are some other things where we can try to initialize or run
migrations automatically, but that stuff can be very dangerous to just
do implicitly.  Now note, I *do* think we want to handle some of this
stuff behind the scenes as much as possible so that users don't have to
think about it.  But have you ever done a *really big* database schema
migration?

We run into two challenges:
 - Now we're trying to "idempotently manage state", which it turns out
   is very hard (and the source of many bugs in devops tooling)
 - Some commands either need to be run manually occasionally, or are
   never automatically run (see the dumpdb example).

Does this make sense?
 - Chris

 

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello!
>
> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> Christopher Allan Webber writes:
>>
>>> Hello,
>>>
>>> I'm writing a service for dirvish, and I realized that if I'm following
>>> current guix service routes, I might not be able to run all the
>>> operations I need to.  It seems that the current route for Guix is to
>>> have your service write out a config that more or less becomes part of
>>> the environment for starting / stopping a daemon via Shepherd.  But what
>>> if that's not all you need to do?
>>>
>>> Aside from just "running as a daemon", plenty of (especially
>>> applications which manage state) will need to have other commands that
>>> are unlikely to be run from shepherd.  For example:
>>>
>>>  - Initializing a data store.  For example, in dirvish I need to run
>>>a command to initialize a "vault" where I will be storing my data.
>>>  - Manually invoking a garbage collection utility.
>>>  - Manually invoking an integrity check utility.
>>>  - Possibly some side effect involving querying the network.
>>>  - Running schema migrations.
>>>
>>> All sorts of things!  Most of them (all?) involve state or side effects,
>>> but plenty of our most important services will be "state overlords" of
>>> some type.
>>>
>>> So it seems to me that one of two things will be needed... either:
>>>
>>>  - Expose the configuration file directly, possibly by putting in
>>>`${profile}/etc/foo'
>>>  - Expose "wrapped" utilities.  For example, instead of invoking
>>>"dirvish" directly, I might invoke a wrapped dirvish.
>>
>> Talking about this with Ricardo Wurmus on irc, the idea of launching
>> such a utility from shepherd itself came to mind.
>>
>> So imagine you want to run a tool like this:
>>
>>   foo-db --config=/path/to/foodb-config.cfg gc --aggressive
>>
>> It looks like shepherd has an "action" slot/method:
>>
>>   
>> https://www.gnu.org/software/shepherd/manual/html_node/Slots-of-services.html#Slots-of-services
>>   
>> https://www.gnu.org/software/shepherd/manual/html_node/Service-Convenience.html#Service-Convenience
>>
>> So I wonder if we could add additional actions, and do something like
>> this:
>>
>>   herd gc foo-db --aggressive
>>
>> Or, even more lazy (but maybe not as good?):
>>
>>   herd run-cmd foo-db gc --aggressive
>>
>> Anyway, either of these examples would call the appropriate command but
>> implicitly pass in the --config parameter appropriately.
>>
>> It looks like (gnu services shepherd) doesn't expose the "actions" slot,
>> but couldn't we do that?
>>
>> This seems like the right route.  What do others think?
>
> We could do that (and custom actions would probably be useful in other
> contexts), but like the other Chris ;-), I’m not sure whether this is
> necessary.
>
> So it seems to be that the state-management commands (initialization
> commands, DB schema migration commands, etc.) could be run either:
>
>   1. By an “activation snippet”, by extending ‘activation-service-type’
>  like many services do.
>
>   2. By an auxiliary Shepherd service, say ‘dirvish-init’, that the main
>  service, say ‘dirvish’, would depend on.
> 
> How does that sound?

That's probably fine in most cases; if you read my reply to the other
Chris Marusich you'll see that there are still some cases where you
might need to run commands manually... eg a "dumpdb" type command.
Maybe that could be handled by #2, but it seeems like in a dumpdb type
command you might need to be able to pass in arguments.  It seems like
Shepherd actions are the right case for such a thing?

 - Chris

Re: (Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Hello,
>
> I'm writing a service for dirvish, and I realized that if I'm following
> current guix service routes, I might not be able to run all the
> operations I need to.  It seems that the current route for Guix is to
> have your service write out a config that more or less becomes part of
> the environment for starting / stopping a daemon via Shepherd.  But what
> if that's not all you need to do?
>
> Aside from just "running as a daemon", plenty of (especially
> applications which manage state) will need to have other commands that
> are unlikely to be run from shepherd.  For example:
>
>  - Initializing a data store.  For example, in dirvish I need to run
>a command to initialize a "vault" where I will be storing my data.
>  - Manually invoking a garbage collection utility.
>  - Manually invoking an integrity check utility.
>  - Possibly some side effect involving querying the network.
>  - Running schema migrations.
>
> All sorts of things!  Most of them (all?) involve state or side effects,
> but plenty of our most important services will be "state overlords" of
> some type.
>
> So it seems to me that one of two things will be needed... either:
>
>  - Expose the configuration file directly, possibly by putting in
>`${profile}/etc/foo'
>  - Expose "wrapped" utilities.  For example, instead of invoking
>"dirvish" directly, I might invoke a wrapped dirvish.

Talking about this with Ricardo Wurmus on irc, the idea of launching
such a utility from shepherd itself came to mind.

So imagine you want to run a tool like this:

  foo-db --config=/path/to/foodb-config.cfg gc --aggressive

It looks like shepherd has an "action" slot/method:

  
https://www.gnu.org/software/shepherd/manual/html_node/Slots-of-services.html#Slots-of-services
  
https://www.gnu.org/software/shepherd/manual/html_node/Service-Convenience.html#Service-Convenience

So I wonder if we could add additional actions, and do something like
this:

  herd gc foo-db --aggressive

Or, even more lazy (but maybe not as good?):

  herd run-cmd foo-db gc --aggressive

Anyway, either of these examples would call the appropriate command but
implicitly pass in the --config parameter appropriately.

It looks like (gnu services shepherd) doesn't expose the "actions" slot,
but couldn't we do that?

This seems like the right route.  What do others think?

 - Chris

(Exposing?) config files and non-start/stop operations

[Christopher Allan Webber]

Hello,

I'm writing a service for dirvish, and I realized that if I'm following
current guix service routes, I might not be able to run all the
operations I need to.  It seems that the current route for Guix is to
have your service write out a config that more or less becomes part of
the environment for starting / stopping a daemon via Shepherd.  But what
if that's not all you need to do?

Aside from just "running as a daemon", plenty of (especially
applications which manage state) will need to have other commands that
are unlikely to be run from shepherd.  For example:

 - Initializing a data store.  For example, in dirvish I need to run
   a command to initialize a "vault" where I will be storing my data.
 - Manually invoking a garbage collection utility.
 - Manually invoking an integrity check utility.
 - Possibly some side effect involving querying the network.
 - Running schema migrations.

All sorts of things!  Most of them (all?) involve state or side effects,
but plenty of our most important services will be "state overlords" of
some type.

So it seems to me that one of two things will be needed... either:

 - Expose the configuration file directly, possibly by putting in
   `${profile}/etc/foo'
 - Expose "wrapped" utilities.  For example, instead of invoking
   "dirvish" directly, I might invoke a wrapped dirvish.

What are our thoughts on how to deal with this?  This is going to be
very important when moving towards GuixSD as a platform for server
deployment.

 - Chris

PS: Note that for either of the above suggested methods, this will be
extra tricky when we have users running multiple services of the same
application.  For example, we might have users running multiple
Wordpress installs or mail daemons or MediaGoblin instances on the same
machine, and I'm not really sure how you'd find which thing you're
looking for there (but I guess you have the same problem with which
identifier would be used in shepherd start/stop anyway).

Re: [PATCH] gnu: milkytracker: Update upstream location.

[Christopher Allan Webber]

Looks good.  Push it!

Re: GNU/Hurd support on core-updates

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello Manolis,
>
> I had almost forgotten about it (!), but ‘core-updates’ adds support for
> Guix on native GNU/Hurd, via commit
> d75acc293dd3e63db8739aa04c021df917aa1b80.
>
> Do I get it right that, on current core-updates, we have (1)
> cross-compilation support to i586-pc-gnu(?), and (2) native support
> works and all we need is to provide the bootstrap binaries?
>
> If so, I’d like to add cross-compilation jobs on Hydra to catch any
> regressions, and address (2) by providing the bootstrap binaries that
> you built on ftp://alpha.gnu.org and such.
>
> TIA!  :-)
>
> Ludo’.

Wow!  Thrilling!

Re: Web site news to move to Haunt

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello Guix!
>
> In the ‘wip-haunt’ branch of guix-artwork.git, I started switching the
> web site to use Haunt, as was suggested long ago:
>
>   https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00087.html
>
> The goal is to manage news using Haunt, and to have a page to display
> them on the web site, something nicer and more flexible than this:
>
>   https://savannah.gnu.org/news/?group=guix
>
> And I already enjoy Haunt!  :-)

Horray!  Haunt is great, and it is nice to see more users... means we
can collaboratively help tune Haunt to our collective needs more easily.

Re: Providing an alternative to setuid in GuixSD

[Christopher Allan Webber]

Ludovic Courtès writes:

> SSH is a complex protocol and its implementations are complex too.  I
> would find it unreasonable to replace ‘su’ and ‘sudo’ with something
> this complex, that goes through the TCP/IP stack, etc.

I agree.  We could maybe have a pseudo-sudo service that is built just
for this purpose though... let's call it "psudo". ;)  Thinking out loud:

So, you're running psudo, and this thing maybe accepts connections over
something more secure, *maybe* unix domain sockets... so restrict group
access to the socket to users in the "psudo" group.

>From there, maybe it could require PAM authentication while entering the
root password, or something.

It feels hard to know how psudo could "know" what user is accessing the
socket... I don't think that information is made available, right?
Maybe I'm wrong!  I guess postgres and etc do similar things?

Fun idea to think about anyway :)

Re: half-package of autossh

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> From c718ca3986750c255a58b87d9e6805e56c1ffc72 Mon Sep 17 00:00:00 2001
>> From: Christopher Allan Webber <cweb...@dustycloud.org>
>> Date: Mon, 8 Feb 2016 16:01:49 -0800
>> Subject: [PATCH] gnu: Add autossh.
>>
>> * gnu/packages/ssh.scm (autossh): New variable.
>
> [...]
>
>> +(synopsis "Automatically restart SSH sessions and tunnels")
>> +(description "autossh is a program to start a copy of ssh and 
>
> @command{ssh}
>
>> +monitor it, restarting it as necessary should it die or stop 
>> +passing traffic. The idea is from rstunnel (Reliable SSH Tunnel), 
>> +but implemented in C.")
>
> Maybe remove the last sentence?
>
>> + ;; copy of this license in their headers, but there's no seprate file
>
> “separate”
>
> Go for it!
>
> Thanks!
>
> Ludo’.

Pushed with those changes, thanks!

Re: #!/usr/bin/env in containers, or something

[Christopher Allan Webber]

Ludovic Courtès writes:

> Christopher Allan Webber <cweb...@dustycloud.org> skribis:
>
>> I'm currently looking at building a version of coreboot.  Unfortunately,
>> the mortal enemy of every GuixSD user, #!/usr/bin/env, lurks around
>> every corner.
>>
>> Wingo made an interesting suggestion on IRC today: maybe we could have
>> some "guix environment --container" CLI mapping /usr/bin/env.
>>
>> I know we want "purity" and it's annoying that /usr/bin/env even exists,
>> but there are times where you're trying to do some development on some
>> upstream package, and it's not possible if you can't get /usr/bin/env
>> into your development environment.  Sinc we can make containers --pure,
>> I think it would be okay to have /usr/bin/env in this circumstance.
>>
>> What do people think?
>
> There was some consensus that ‘guix environment --container’ could have
> an option to create /usr/bin/env:
>
>   https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00527.html
>
> ‘--expose’ already allows this, so essentially we’d provide a shorter
> option and take ‘env’ from the current ‘coreutils’ package—similar to
> how /bin/sh links to the current ‘bash’ already.
>
> How does that sound?
>
> Ludo’.

That sounds really great!

(I am unlikely to have time to do it myself, but maybe I'll turn out to
be wrong :))

Would like to see that happen though!
 - Chris