Re: How can we decrease the cognitive overhead for contributors?

[Danny Milosavljevic]

Hi,

> review" because of the amount of cognitive overhead required. I've written
> a script for myself that tries to perform all the steps including running
> the git command to submit the patch, and this has helped me, but that this
> is necessary for me to do might be something that, if addressed, could
> help others.

I agree that automating the technical steps of contributing to Guix
would be nice.

I suggest that we preinstall a script that does the equivalent of this:

if [ ! -d guix ]
  then
git clone --depth=1
https://git.savannah.gnu.org/git/guix.git guix
  else
(cd guix && git pull --rebase)
  fi \
  && cd guix
  && guix shell -C -D guix -- ./bootstrap \
  && guix shell -C -D guix -- ./configure --localstatedir=/var
  --disable-daemon \
  && guix shell -C -D guix -- make -j5 \
  && ./pre-inst-env guix edit "$@" \
  && git add gnu/packages/*.scm

Also, we should automate adding an smtp server (after ascertaining that port
25 localhost is not open or something? Not sure.):

git config --global sendemail.smtpencryption tls
git config --global sendemail.smtpserver mail.messagingengine.com
git config --global sendemail.smtpuser f...@example.com
git config --global sendemail.smtpserverport 587
git config --global sendemail.smtppass hackme

There's even a file guix/etc/git/gitconfig already--aha!

This would reduce the barrier to entry a lot.

It's unfortunate that for example pre-inst-env doesn't work correctly (for
me; checked on multiple computers) when you don't invoke "make" beforehand
(sometimes it just picks up the wrong guix and doesn't say anything!). On the
other hand, using a guix.scm anywhere else works without make.

If I do "guix edit xyz" it opens a file in /gnu/store that I can't edit.
Not sure what good that is.

It would be better to ask and then automatically suggest invoking the script
above that prepares a guix development environment.

It's kinda weird we don't just have a "guix contrib" subcommand or something
that automates basic things that any guix contributor needs.

> I can't ever seem to get the GNU style commit messages correct.

Writing the metadata into the commit messages is annoying. It totally should
be automated, especially since Scheme has pretty simple syntax (so it should
be easy to write such a thing/famous-last-words). It should just figure out
which procedures the changed lines were in, automatically.
(This probably does exist in emacs)

Because of the line-based nature of the diff tools and of a lot of git tools, we
do need to have which procedure was changed in the git commit message,
though.

> I don't use the email-based patch workflow day-to-day, so this is another
> area where I spend a lot of time trying to make sure I'm doing things
> correctly.

Especially since one needs to the debbugs dance where one first opens a new
bug report by sending an email to guix-patc...@gnu.org, waits for an ID to
be mailed to you, only after that to reply to @debbugs.gnu.org,
attaching the patches (copying the message-id from the old mail manually as
a reference). That is such a weird workaround for a debbugs limitation.

This definitely should be automated. Why not generate a UUID locally and send
a mail to @debbugs.gnu.org ? That may require changes to debbugs,
though.

> manually managing Guile imports is laborious. As a fledgling schemer, I
> often forget which imports I needed just for development. 

Yeah. I like that we now tell the user whether imports are unused, and also
which import it could most likely be if it's missing. But automating the cleanup
and adding like it is in Java IDEs would totally be awesome. With a language
as dynamic as Guile I think this problem is intractable since you can never be
sure someone dynamically does (or doesn't) need the import.
Can still be approximated.

I attached some changes to guix to make the contributor experience less bad.
It doesn't look nice but it does work and I even prefer that workflow myself 
now.

@Ludo: What do you think?

> Does this affect anyone else?

The basic setup affects one only when one changes workstations--which I suspect
is why it wasn't improved until now. It's just too rare a problem per person. 
But it's a
very bad barrier of entry for new contributors, and there's no reason for it to 
be there!

But the other annoyances totally affect me all the time, too.

Cheers,
 Danny

P.S. We could also do git config --local alias.send-patches 'git send-email -r 
origin/master --cover-letter --annotate --to=guix-patc...@gnu.org'
From 10f16b0b3cf47931db5c9607b95872f477550422 Mon Sep 17 00:00:00 2001
Message-Id: <10f16b0b3cf47931db5c9607b95872f477550422.1693350103.git.dan...@scratchpost.org>
From: Danny Milosavljevic 
Date: Wed, 30 Aug 2023 00:50:

Re: git guix checkout automation for contributors

[Danny Milosavljevic]

Also, this doesn't work:

On Sun, 25 Sep 2022 16:06:06 +0200
Danny Milosavljevic  wrote:

(1) install the guix system
(2) log in as regular user
(3c) git clone --depth=1 https://git.savannah.gnu.org/git/guix.git guix \
   && cd guix \
   && guix shell

Even after doing

echo /home/dannym/../guix >> 
/home/dannym/.config/guix/shell-authorized-directories

I get:

guix shell: loading environment from '/home/dannym/backup/g1/guix/guix.scm'...
Backtrace:
  14 (primitive-load "/home/dannym/.config/guix/current/bin/?")
In guix/ui.scm:
   2263:7 13 (run-guix . _)
  2226:10 12 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1752:10 11 (with-exception-handler _ _ #:unwind? _ # _)
  1752:10 10 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   656:37  9 (thunk)
   1295:8  8 (call-with-build-handler # ?)
In guix/status.scm:
815:4  7 (call-with-status-report _ _)
In guix/scripts/environment.scm:
320:4  6 (_)
In srfi/srfi-1.scm:
   673:15  5 (append-map _ _ . _)
   586:29  4 (map1 _)
   586:17  3 (map1 ((load package "/home/dannym/backup/g1/guix/?") ?))
In guix/scripts/environment.scm:
306:4  2 (packages->outputs _ _)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `match-error' with args `("match" "no matching pattern" 
#)'.


pgpgX7B2ZACvX.pgp
Description: OpenPGP digital signature

git guix checkout automation for contributors

[Danny Milosavljevic]

Hi,

So there were some comments on https://news.ycombinator.com/item?id=32960308
about getting started with contributing to Guix.

It turns out that the barrier of entry for editing existing packages is
kinda high (I tried it myself--putting myself into the mindset of a
beginner).

For example:

(1) Install the guix system
(2) Log in as regular user
(3) guix edit nano
(4) It opens a file from /gnu/store that you cannot edit.

Uhh. That's... not ideal. Maybe instead error out with a hint on what you need
to do in order to provide a local checkout? Or do we want viewing to work?

Ok, so let's say we are a little more knowledgeable:

(1) Install the guix system
(2) Log in as regular user
(3a) if [ ! -d src ]
  then
mkdir src
cd src
  else
cd src
  fi \
  && if [ ! -d guix ]
  then
git clone --depth=1 https://git.savannah.gnu.org/git/guix.git guix
  else
(cd guix && git pull --rebase)
  fi \
  && cd guix \
  && guix shell -C -D guix -- ./bootstrap \
  && guix shell -C -D guix -- ./configure --localstatedir=/var --disable-daemon 
\
  && guix shell -C -D guix -- make -j5 \ <--- takes forever to build
  && ./pre-inst-env guix edit nano \
  && git add gnu/packages/*.scm

Ok, that worked.

But there are a lot of ways that can be done wrong.

Weird way it can be done wrong:

(1) install the guix system
(2) log in as regular user
(3b) git clone --depth=1 https://git.savannah.gnu.org/git/guix.git guix \
  && cd guix \
  && guix shell -C -D guix -- ./bootstrap \
  && guix shell -C -D guix -- ./configure --localstatedir=/var --disable-daemon 
\
  && ./pre-inst-env guix edit nano

It will open the file in /gnu/store . WTF? I'm pretty sure that that's a bug.

./pre-inst-env does exist--but it does some weird things if you don't
compile beforehand.

Also, can we put a variant of (3a) into a command "guix prepare-edit" or
something?
It has always bothered me how manual and magical-commandy it is otherwise
(what you actually need to do also changed over the years--not good).


pgp_9jOkgNAMH.pgp
Description: OpenPGP digital signature

Re: branch master updated: gnu: python-lxml: Update to 4.6.5.

[Danny Milosavljevic]

Hi Mathieu,

On Mon, 15 Aug 2022 16:24:03 +0200
Mathieu Othacehe  wrote:

> > * gnu/packages/xml.scm (python-lxml): Update to 4.6.5.  
> 
> I had to revert it as it caused >15k rebuilds. I also cancelled all the
> triggered builds in the CI.

Thanks for doing it!

But guix refresh -l python-lxml reported one package O_o

Now I see that python-lxml-4.7 shadows it.

Note: This would have fixed two CVEs, CVE-2021-43818 and CVE-2021-43818.



pgpouVuP0WkYm.pgp
Description: OpenPGP digital signature

Re: How to find binaries in libexec dir?

[Danny Milosavljevic]

Hi,

On Sat, 16 Apr 2022 16:21:50 -0700
Vagrant Cascadian  wrote:

> But libxmlb ships the xb-tool binary in libexec, which diffoscope cannot
> find on guix.

> For Debian, diffoscope just basically adds the
> corresponding directory to PATH, and it's basically a single directory,
> but I'm not sure how I'd do that with guix ...
> 
> A workaround would be to adjust the libxmlb package so that xb-tool is
> also in PATH somehow, but maybe this is bad form?

Isn't the whole point of libexec so that executable is not in PATH?

FHS section 4.7 says:

>/usr/libexec includes internal binaries that are not intended to be executed 
>directly by users or shell scripts.

libexec is usually used to store tool executables that are invoked internally
by libraries (glibc, ssh, gpg etc).

I'd like to caution against just adding libexec to PATH, because that would
make the entire separation of libexec out of bin (and thus the reason of
existence of libexec) useless to begin with.

I've looked upstream https://github.com/hughsie/libxmlb and it says, very close
to the beginning (so it's a main feature apparently):

>$ xb-tool compile fedora.xmlb fedora.xml.gz

Well, for that to work, xb-tool should be in bin.

I'd file a bug report with libxmlb to move xb-tool to bin.


pgpwbDc9bzGui.pgp
Description: OpenPGP digital signature

Re: Package's inputs for developer?

[Danny Milosavljevic]

Hi,

On Sun, 06 Mar 2022 11:19:58 -0500
Olivier Dion via "Development of GNU Guix and the GNU System distribution." 
 wrote:

> I often find my self using inheritance of package to add native-inputs
> that are not stricly necessary for building the project, but are used
> for developement purpose:

It's very common to require extra tools for development, but those are
anything but standardized. In fact, finding elusive bugs is very much
helped by using different compiler versions, compiler vendors etc.

So I don't think adding those (which?) directly in the base package is a good 
idea.

Better:

$ guix shell -f guix.scm gdb@11.1 lcov


pgpY9yLdjKdWt.pgp
Description: OpenPGP digital signature

Re: How to handle package udev rules?

[Danny Milosavljevic]

Hi,

On Sun, 12 Dec 2021 21:58:14 +0100
g...@member.fsf.org wrote:

> If I change my operating-system config to inlcude udev-rules from
> package "projecteur" everything works fine - at least if I do it as a
> regular user. As soon as I sudo the guix system reconfigure command the
> package is known but it's code is not. Error message is:
> 
> > $ sudo guix system reconfigure ~/etc/config.scm
> > ice-9/boot-9.scm:3329:6: In procedure resolve-interface:
> > no code for module (projecteur)  
> 
> Could it be the case that sudo'ed the variable GUIX_PACKAGE_PATH is not
> known or not interpreted correctly? Does the package need to reside
> somewhere else than in GUIX_PACKAGE_PATH?

Yeah, sudo is very paranoid. You need to pass -E GUIX_PACKAGE_PATH to it:

   sudo -E GUIX_PACKAGE_PATH guix system reconfigure ~/etc/config.scm


pgp4jE7o4Rtsb.pgp
Description: OpenPGP digital signature

Re: How to handle package udev rules?

[Danny Milosavljevic]

Hello Alexander,

On Sun, 12 Dec 2021 13:12:50 +0100
Alexander Asteroth  wrote:

> I tried to import the libgudev module but that that only results in the
> package wanting to write to another write-protected directory from the
> store.

It's supposed to write those to $output/lib/udev/rules.d (s.t. there's
*.rules inside) instead.   There's usually a cmake variable to make it do
that--but you need to look at the CMakeLists.txt in question.

Guix will pick those up if they originate in something it can see and add
those to Guix's udev service automatically (via the service extension mechanism,
which allows you to extend service config from outside the udev service).

Are you using Guix as an operating system? Or on top of another distribution?

> As I understand, the udev-rules are usually created on system level.

Yes.

> That would mean I need to split the package into a service part
> and a package part? And remove the installation of the udev-file from
> the package install process?

Kinda not really--at least not exactly.  See below.

Example I'm using (that one definitely works--but I only add the custom
package because the upstream package doesn't do it![1]):

/etc/config.scm :

(define my-ledger-nano
  (package
(name "my-ledger-nano")
(version "1")
(source #f)
(build-system trivial-build-system)
(synopsis "")
(description "")
(license #f)
(home-page #f)
(arguments
 `(#:builder
   (begin
 (mkdir %output)
 (mkdir (string-append %output "/lib"))
 (mkdir (string-append %output "/lib/udev"))
 (mkdir (string-append %output "/lib/udev/rules.d"))
 (call-with-output-file (string-append %output 
"/lib/udev/rules.d/99-my-ledger-nano.rules")
   (lambda (port)
 (format port
"SUBSYSTEM==\"usb\", ATTRS{idVendor}==\"2c97\", MODE=\"0600\", 
OWNER=\"dannym\", TAG+=\"uaccess\", TAG+=\"udev-acl\"
KERNEL==\"hidraw*\", ATTRS{idVendor}==\"2c97\", MODE=\"0600\", 
OWNER=\"dannym\", SYMLINK+=\"ledger_%n\", TAG+=\"uaccess\", TAG+=\"udev-acl\"
")))
 #t)

(operating-system
  ...
  (services
(simple-service 'custom-udev-rules udev-service-type (list 
sane-backends my-ledger-nano)))

You can add your package reference there and it will work.

Or you can try adding your package reference into
  (operating-system (packages (list ...)))--should be enough.

If you mean adding your package's udev rules to the operating system
configuration without being the "root" user in-between: no, that would be a
(very bad! those rules run as root!) security problem.

In the case of your package, it seems[0] that they calculate the directory to
store the udev rules to from pkg-config--which will result in the udev
package's install directory. That won't work.

But in line 214 in [0] they seem to allow you to override it anyway.
So you can try to call cmake with

  -DCMAKE_INSTALL_UDEVRULESDIR=$output//lib/udev/rules.d

like this (in your package definition):

(package
  ...
  (arguments
 '(#:configure-flags (list (string-append "-DCMAKE_INSTALL_UDEVRULESDIR="
  (assoc-ref %outputs "out")
  "/lib/udev/rules.d"

[0] https://github.com/jahnf/Projecteur/blob/develop/CMakeLists.txt#L231
[1] https://github.com/LedgerHQ/udev-rules/pull/8


pgpJcpN9h9FfN.pgp
Description: OpenPGP digital signature

Re: trezor-agent: using trezor as gpg key

[Danny Milosavljevic]

Hi,

On Fri, 03 Sep 2021 07:55:19 +
Attila Lendvai  wrote:

> i'm failing to configure my gnupg to use Trezor (an USB hw key store).
> $ ls -l /dev/usb/
[...]
> crw-rw 1 root plugdev 180, 0 Sep 3 09:25 hiddev0
> 
> $ groups
> users plugdev [...]
> 
> $ trezor-gpg-agent &
> [1] 11223
> 
> $ gpg --sign /tmp/x
> 2021-09-03 09:41:19,129 ERROR Failed to enumerate WebUsbTransport. 
> USBErrorAccess: LIBUSB_ERROR_ACCESS [-3] [__init__.py:126]
> 2021-09-03 09:41:19,132 ERROR handler failed: Trezor not connected 
> [__init__.py:273]
[...]
> libagent.device.interface.NotFoundError: Trezor not connected
> gpg: Warning: not using 'Attila Lendvai ' as default 
> key: No secret key
> gpg: all values passed to '--default-key' ignored
> gpg: keydb_search failed: Broken pipe
> gpg: no default secret key: Broken pipe
> gpg: signing failed: Broken pipe
> 
> how can i see/debug here which file has the wrong permissions? the python 
> backtrace is not very useful without printing the variable values.

You can

 strace -f gpg --sign /tmp/x 2>&1 |grep /dev

but I suspect that that won't show anything interesting because it's actually 
trezor-gpg-agent doing the access.

Maybe

   strace -f trezor-gpg-agent 2>&1 |grep /dev

?

In any case, since I did pretty much the same with a Ledger Nano, check the 
permissions of /dev/hidraw*.  After fixing the permissions to the latter file, 
it works fine as a regular user.

See also https://github.com/LedgerHQ/udev-rules/pull/8 for how to make the 
change permanent.

Even then, it only works fine once one logs out and back in, and at the time of 
login have the usb device already be connected.  I think that's a bug in our 
elogind (sigh).

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Danny Milosavljevic]

Hi Vagrant,

On Sun, 06 Jun 2021 14:38:27 -0700
Vagrant Cascadian  wrote:

> So, I've managed to get a single image that supports booting both the
> Pinebook and Pinebook Pro reasonably well! I can pop the microSD card
> out of one and put it into the other, and it boots!

Awesome!

> The only problem with my single image for multiple pinebooks variants is
> it requires manually installing u-boot to different offsets for Pinebook
> Pro (e.g. idbloader.img at sector 2112 rather than sector 64), 

>as parts of the Pinebook bootloader are installed at overlapping offsets.

Uhhh?  Doesn't sound possible to me then to use the same image for both?
I'm probably missing something.

> But as I understand it, guix only supports a single bootloader entry. 

That is correct.

> To support all of the above, I would need three separate bootloader
> instances... one for Pinebook, one for Pinebook Pro, and lastly a
> grub-efi bootloader.

Stefan wrote a Guix chain bootloader that is in master--but it's meant
to be only used for bootloaders where there is a primary "bare-metal-loaded"
bootloader and the others are chain-loaded one-after-another from ONE filesystem
(for example Raspberry Pi and/or EFI bootloaders).

(It's currently used in order to use an EFI bootloader hosted on NFS--which
is an awesome way to manage embedded boards)

The chain bootloader itself is one Guix bootloader.

I advise you to search for mails by Stefan on the guix-devel mailing list--those
are very illuminating.

> Installing u-boot-pinebook uses:

> (define install-allwinner64-u-boot
>   #~(lambda (bootloader root-index image)
>   (let ((spl (string-append bootloader 
> "/libexec/u-boot-sunxi-with-spl.bin"))
> (u-boot (string-append bootloader 
> "/libexec/u-boot-sunxi-with-spl.fit.itb")))
> (write-file-on-device spl (stat:size (stat spl))
>   image (* 8 1024))
> (write-file-on-device u-boot (stat:size (stat u-boot))
>   image (* 40 1024)
> 
> Installing u-boot-pinebook-pro-rk3399 uses:
> 
> (define install-rockpro64-rk3399-u-boot
>   #~(lambda (bootloader root-index image)

#~(lambda* (bootloader root-index image #:key idb-destination-offset)

>   (let ((idb (string-append bootloader "/libexec/idbloader.img"))
> (u-boot (string-append bootloader "/libexec/u-boot.itb")))
> (write-file-on-device idb (stat:size (stat idb))
>   image (* 64 512))


image (or idb-destination-offset (* 64 512)))

> (write-file-on-device u-boot (stat:size (stat u-boot))
>   image (* 16384 512)
> 
> (define install-pinebook-pro-rk3399-u-boot install-rockpro64-rk3399-u-boot)
> 
> But now I need to figure out how to pass a non-default offset for the
> "idb" part for rockchip platforms.
> 
> In a system config.scm, you'd define it like so:
> 
>   (bootloader (bootloader-configuration
>(bootloader u-boot-pinebook-pro-rk3399-bootloader)
>(target "/dev/mmcblk0")))
> 
> u-boot-pinebook-pro-rk3399-bootloader is defined in
> gnu/bootloader/u-boot.scm, which inherits from u-boot-bootloader, which
> inherits from extlinux-bootloader in gnu/bootloader/extlinux.scm...
> 
> And somewhere along the way I've lost track of how we get to
> install-pinebook-pro-rk3399-u-boot...

It's in the "bootloader" record, field "disk-image-installer".

If you search for "bootloader-disk-image-installer" in the Guix source code,
you find the (two) callers of it.

> Is it possible to definte multiple bootloaders [in the same operating-system] 
> currently, 

No--and I don't think that would have any advantages.

If the thing is not a chain then Guix doesn't need to logically split the
bootloaders in the first place.

Ok: stage1 -> stage2 -> stage3 represented as (chain-bootloaders stage1 stage2 
stage3).

Not ok: stage1-arch1 stage1-arch2 represented as two different bootloaders
that both are specified in the same operating-system inside the same list.

Instead, I think it's fine to just model that as one "weird" bootloader
(arm64-pinebook-generic-bootloader)--especially if both use u-boot anyway and 
also
especially if they reuse some of their parts anyway (it's gonna have weird
interdependencies in any case--so any apparent modularity would just be a ruse).

> or if not,
> what would need to change to be able to support that? Where would one
> pass non-default offsets for a given platform?

Could you elaborate on what you mean?  Pass where?  On the Guix command line?

If you mean the disk-image-installer, that's just a procedure (which is run
build-side).  You can make it do whatever you want--including hardcoding weird
offsets.  I'm pretty sure it gets the entire image (one could potentially 
mangle)
as a parameter anyway.

> Strictly speaking, the extlinux.conf generation would be optional for an
> EFI generated image(as u-boot can load grub-efi), al

Re: RISC-V is giving away developer boards

[Danny Milosavljevic]

> Could interested developers raise their hands?  :-)

*raises hand*

In other news, tinycc has now an RISC-V assembler[1] :)

[1] https://lists.nongnu.org/archive/html/tinycc-devel/2021-04/msg9.html

meson-build-system cross compilation

[Danny Milosavljevic]

Hi Raghav,

since you have been using meson build system a lot, could you add support for
cross-compilation to guix/build-system/meson.scm please?  I would do it myself
if I knew how.

See https://issues.guix.gnu.org/44244

See also https://mesonbuild.com/Cross-compilation.html

As it is now, a lot of basic guix packages will break when cross-compiling
since they have been switched to meson-build-system.



pgpMcZkEc5YAW.pgp
Description: OpenPGP digital signature

Re: GNOME 3.34 in GNU Guix and security

[Danny Milosavljevic]

Hello,

core-updates is still in a pretty bad state.

I'd be glad to merge Raghav's patches (which he already reworked to be current 
for core-updates!) to core-updates--but right now, Guix packages don't build 
BEFORE or after applying these patches to core-updates.

Please, let's do something about that.

A short summary of what is at least broken:

[(1) FTP with IPv6--or with the new patch, some other FTP that don't support 
EPSV (the latter is a server problem, but the former had been bug in Guix).  So 
I guess that one is fixed.]
(2) Source files have been in-place replaced upstream with a lot of packages 
(see my bug report about the topic).  fldigi has such a problem but can just be 
updated.  This is easy to see by just building without substitutes--and it 
doesn't do anyone any good for me to file individual bug reports for each and 
every one of those
(3) libusb-for-axoloti build failure (guix build axoloti-patcher-next)
(4) bug applying patches in sources unpacked from zip files (I just posted 
patch 47203 to guix-patches)
(5) download failed 
"https://bioconductor.org/packages/release/bioc/src/contrib/DelayedArray_0.16.1.tar.gz";
 404 "Not Found"


That doesn't mean that those are the only problems.  It means I got frustrated 
and stopped trying, lest I find more problems (which would be easy).

My current rebuild command for build-testing the first of Raghav's patches is 
(from guix refresh -l with some editing--because it didn't work without 
editing.  Sigh):

./pre-inst-env guix build -K --no-substitutes foo2zjs docker localed jnettop 
raul libinstpatch hdup rdup connman rust-gobject-sys rust-gio rust-glib 
rust-gio-sys rust-gobject-sys rust-gio-sys rust-glib-sys rust-glib 
rust-glib-sys rust-gio rust-glib rust-gio 4store mdbtools ncdc 
american-fuzzy-lop sdcv duperemove libticalcs2 ecl-enchant imposm3 
spatialite-tools poly2tri-c gnome-shell-extension-hide-app-icon 
gnome-shell-extension-topicons-redux tiramisu gnome-shell-extension-paperwm 
gnome-shell-extension-clipboard-indicator gnome-shell-extension-dash-to-dock 
gnome-shell-extension-dash-to-panel linkchecker fortune-mod rust-gdk-pixbuf-sys 
rust-gdk-pixbuf rust-gdk-pixbuf rust-gdk-pixbuf-sys rust-gdk-pixbuf rust-atk 
rust-atk-sys ddcutil irssi rspamd emacs-mu4e-jump-to-list 
emacs-mu4e-conversation emacs-helm-mu emacs-mu4e-patch mc bitlbee-discord fna 
mojoshader-cs glyr remid-lv2 nbd ocaml-lwt-log brlaser igt-gpu-tools pscircle 
ztoolkit rust-cairo-sys-rs rust-cairo-rs rust-cairo-rs rust-cairo-sys-rs 
rust-cairo-rs libratbag kaldi-gstreamer-server playerctl python2-pyatspi 
guile-charting xplanet rust-pango rust-pango-sys rust-pango rust-pango 
rust-pango-sys font-culmus font-fantasque-sans font-meera-inimai 
font-linuxlibertine fntsample plymouth python-pygraphviz python2-pygraphviz 
pynac makefile2graph sigrok-cli ibutils rust-andrew rust-smithay-clipboard 
mesa-opencl-icd picom rakarrack flamp flrig flwrap tigervnc-client dillo 
tuxpaint-config infamous-plugins sorcer non-sequencer non-timeline non-mixer 
alex4 virtualgl celestia tao slim chipmunk aseprite python2-mapnik 
perceptualdiff megacmd feh sxiv screenfetch ranger eureka git-open privoxy 
fluxbox xmenu idesk xnotify hsetroot python-django-sortedm2m 
python-django-simple-math-captcha python-django-override-storage 
python-django-contrib-comments python-easy-thumbnails python-django-assets 
python-django-auth-ldap python-django-url-filter python-django-netfields 
python-django-contact-form python-django-logging-json python-django-rq 
python-django-debug-toolbar-alchemy patchwork python-django-statici18n 
font-cozette freedoom mcomix conda visidata gajim-omemo python2-ledgerblue 
nototools python2-anaconda-client python2-reportlab python2-qrcode 
libfreenect-examples perl-opengl setbfree xfe gmsh extempore sherlock-lv2 
zynaddsubfx dragonfly-reverb ninjas2 helm zam-plugins wolf-shaper patchmatrix 
wolf-spectrum maim xdriinfo glmark2 insight-toolkit insight-toolkit egl-wayland 
beignet intel-vaapi-driver libvdpau-va-gl mediasdk libva-utils vulkan-tools 
swayidle kanshi wlr-randr wl-clipboard wev foot wterm pass-otp pass-git-helper 
python2-xdo clipmenu libstdc++-doc libstdc++-doc python2-pydot darcs pdf2djvu 
osm2pgsql postgis netcdf-fortran python-netcdf4 nmoldyn domainfinder mes 
mes-rb5 python-anytree python2-dulwich python2-graphql-core python-ws4py 
python-locust python-pykka python-gipc python-pykafka uriparser libxmlplusplus 
pangomm emacs-w3m emacs-blimp emacs-gif-screencast emacs-image+ 
tango-icon-theme catimg chafa perl-catalyst-plugin-captcha ecl-ltk 
emacs-theme-magic skribilo solaar wla-dx keepalived python-symengine alacritty 
python-pylibmc python2-pylibmc rmlint libabigail python-cffi-documentation 
emacs-helm-notmuch notmuch-addrlookup-c patches neomutt muchsync afew mpd-mpc 
ncmpc python-cantools bpython python-robotframework-sshlibrary 
p

Re: Update on wip-arm-bootstrap

[Danny Milosavljevic]

Oops, I meant

> +#define SYS_ify(syscall_name)  (__NR_##syscall_name & 0xf)


pgpYQV9VCeoUb.pgp
Description: OpenPGP digital signature

Re: Update on wip-arm-bootstrap

[Danny Milosavljevic]

> Hmm, I'm not seeing this (I did see something like this before).  You
> *are* on
> 
> e44b6b7eed squash! commencement: binutils-mesboot0: Support ARM.
> 
> right?

Yes, but your gcc-mesboot.sh hardcoded a specific (older) path.  Sorry.

With updated gcc-mesboot.sh I get:

Program terminated with signal SIGSEGV, Segmentation fault.
(gdb) bt
#0  _IO_setb (f=0x4, b=0x4b , 
eb=0x4c , a=0) at genops.c:387
#1  0x8f1c in _IO_doallocbuf (fp=0x4) at genops.c:406
#2  0x0002409c in _IO_new_file_overflow (f=0x6cde0 <_IO_2_1_stdout_>, ch=0) at 
fileops.c:576
#3  0x000249b0 in _IO_new_file_xsputn (f=0x6cde0 <_IO_2_1_stdout_>, 
data=0xbecfd313, n=1) at fileops.c:962
#4  0x0001a09c in _IO_vfprintf (s=0x6cde0 <_IO_2_1_stdout_>, format=0xbecfd313 
"2\224\323Ͼ\001", 
ap=) at 
vfprintf.c:1523
#5  0x8828 in printf (format=0x525f0 "%d\n") at printf.c:33
#6  0x8214 in main (argc=, 
argv=) 
at test.c:6

Looks like a NULL pointer for a FILE* ?  Or is gdb just weird?

glibc-2.2.5/libio/genops.c:

void
_IO_setb (f, b, eb, a)
 _IO_FILE *f;
  char *b;
 char *eb;
 int a;
{
  if (f->_IO_buf_base && !(f->_flags & _IO_USER_BUF)) < here
FREE_BUF (f->_IO_buf_base, _IO_blen (f));
  f->_IO_buf_base = b;
  f->_IO_buf_end = eb;
  if (a)
f->_flags &= ~_IO_USER_BUF;
  else
f->_flags |= _IO_USER_BUF;
}

int
_IO_new_file_overflow (f, ch)
  _IO_FILE *f;
  int ch;
{
  if (f->_flags & _IO_NO_WRITES) /* SET ERROR */
{
  f->_flags |= _IO_ERR_SEEN;
  __set_errno (EBADF);
  return EOF;
}
  /* If currently reading or no buffer allocated. */
  if ((f->_flags & _IO_CURRENTLY_PUTTING) == 0 || f->_IO_write_base == 0)
{
  /* Allocate a buffer if needed. */
  if (f->_IO_write_base == 0)
{
  _IO_doallocbuf (f); <- here
  _IO_setg (f, f->_IO_buf_base, f->_IO_buf_base, f->_IO_buf_base);
}

Maybe add some debugging statements there?  But printf is broken, so maybe
use write(2, "foo\n", 4) and similar.

(I'm not sure about just defining -D__ARM_EABI__=1 as you are doing--after all,
it entails further stack alignment requirements.  Maybe try


+++ glibc-2.2.5/sysdeps/unix/sysv/linux/arm/sysdep.h2021-02-22 
00:44:43.300906460 +0100
@@ -29,8 +29,8 @@
of the kernel.  But these symbols do not follow the SYS_* syntax
so we have to redefine the `SYS_ify' macro here.  */
 #undef SYS_ify
-#define SWI_BASE  (0x90)
-#define SYS_ify(syscall_name)  (__NR_##syscall_name)
+#define SWI_BASE  (0x00)
+#define SYS_ify(syscall_name)  (__NR_##syscall_name & 0x8f)


as a more minimal patch?  Not sure...)


pgp61moZSYl4y.pgp
Description: OpenPGP digital signature

Re: Cuirass build notifications.

[Danny Milosavljevic]

Hi Mathieu,

that is awesome!  Thank you!


pgp9pGP0xSjH_.pgp
Description: OpenPGP digital signature

Re: Update on wip-arm-bootstrap

[Danny Milosavljevic]

Hi Janneke,

I get

gcc-mesboot.sh: line 20:  9438 Illegal instruction (core dumped) ./a.out

│   0x276b8 <__writev+32>   svc 0x0014 │
│  >0x276bc <__writev+36>   mov r4, r0 │

The actual problem is at the first line above--the svc with argument != 0.

#0  0x000276bc in __writev (fd=2, vector=0xbebc54c0, count=10)
at ../sysdeps/unix/sysv/linux/writev.c:51

glibc 2.2.5:sysdeps/unix/sysv/linux/writev.c:51

>  bytes_written = INLINE_SYSCALL (writev, 3, fd, CHECK_N (vector, count), 
> count);

Err hmm?!

You *are* patching INLINE_SYSCALL body--so I'd guess that __ARM_EABI__ for
some reason is not being defined.

I'd suggest to #error (or at least #warning) if !__ARM_EABI__.


pgpJ1qm1DXq5_.pgp
Description: OpenPGP digital signature

Re: Update on wip-arm-bootstrap

[Danny Milosavljevic]

Hi Janneke,

Nice!

Patch review:

>+#if __ARM_EABI__
>+#define CLEAR_CACHE(BEG,END)  \
>+{ \
>+  register unsigned long _beg __asm ("a1") = (unsigned long)(BEG);\
>+  register unsigned long _end __asm ("a2") = (unsigned long)(END);\
>+  register unsigned long _flg __asm ("a3") = 0;   
>\
>+  __asm __volatile ("mov r7, 0x9f0002\n\t"\

should be mov r7, 0xf0002 in EABI.
(SWI #(0x90 + syscallnr) is for OABI syscalls only)

>* sysdeps/unix/sysv/linux/arm/brk.c (__brk): Likewise.Likewise.

Duplicate "Likewise".

Otherwise LGTM!

>But something like puts or printf segfaults "Illegal
>instruction" in strlen:

>--8<---cut here---start->8---
>(gdb) disas /r
>Dump of assembler code for function strlen:
>   0xd160 <+0>:03 10 c0 e3 bic r1, r0, #3
>=> 0xd164 <+4>:04 20 91 e4 ldr r2, [r1], #4  
>--8<---cut here---end--->8---

Hmm... looks good to me?  Weird.

I'm building it right now and will have a look.


pgpheWF3N5kjV.pgp
Description: OpenPGP digital signature

Re: Update on wip-arm-bootstrap

[Danny Milosavljevic]

> > Does that mean there are no old reference binaries known to work on
> > Novena?  
> 
> Not if they are binaries from before Lenny (2009), unless we compile the
> kernel with OABI support, which as I understand from Danny, is a bad
> idea.  Weird.

It backdoors https://en.wikipedia.org/wiki/Seccomp if you enable the kernel
option CONFIG_OABI_COMPAT (it also breaks syscall auditing).

And it's very easy to just change the syscall facility in glibc to emit EABI
syscalls (load one new register with constant, and the swi number is different).

Also, binaries compiled with gcc already [say they] are EABI (while using
OABI syscalls :P).

So I think just using EABI syscalls is easier than figuring out how to get
gcc to emit OABI executables--which otherwise are an obsolete format.

GNU Mes for ARM does not use OABI for its own system calls either.

I mean as a test we could totally enable that kernel option and see
whether the bootstrap works.  But it's of limited long-term use.  If we want
regular people to be able to build our bootstrap binaries, we probably
shouldn't rely on them enabling obscure kernel options where the kernel docs
say you shouldn't enable those.


pgpmm7f7GOl5F.pgp
Description: OpenPGP digital signature

Re: Update on wip-arm-bootstrap

[Danny Milosavljevic]

Hi Janneke,

On Thu, 18 Feb 2021 22:52:57 +0100
Jan Nieuwenhuizen  wrote:

> # CONFIG_OABI_COMPAT is not set
> 
> ...certainly a lot easier to find when you know what you're looking
> for.
> 
> @Danny: I'm wondering if we could (should?) try a kernel with OABI
> compatibility?  I suppose it would be better to somehow target
> EABI...but I'm not sure that's possible with glibc-2.2.5 / gcc-2.95.3.
> Interesting choices here!

OABI is older than year 2000, and the kernel docs say not to enable it.
It also breaks seccomp.
I doubt that people have it enabled, and thus would have trouble reproducing
our stuff.  Since this only affects the syscall interface and since also our
ELF headers specify EABI, I would just change the syscalls to EABI:
Just put the syscall number into r7 and use svc 0.

I'd do it myself but I don't see what libc the gcc 2.95 we built has been
using.
Is it ours?
If so, how come it then uses svc 9... in the first place? 
We don't do that.

Or is it using glibc ?

How do I build that gcc on novena?  Where would the syscall headers that
I could change be?


pgp_fN0CcT3Ao.pgp
Description: OpenPGP digital signature

Re: ZFS on Guix

[Danny Milosavljevic]

I just wanted to say that I'm not ignoring your patch, I'm just not qualified
to review it.  I hope someone steps up to it--otherwise I can't really tell
whether (mbegin %state-monad...) inside a random service procedure is a good
idea.

Then again, provenance-service-type does it and there it seems to be fine...


pgpqoMwICNujI.pgp
Description: OpenPGP digital signature

Re: ARM Unified Assembly Language - GNU as does some weird stuff

[Danny Milosavljevic]

> (1) b #60
> 
> It seems that GNU as ignores the immediate entirely and just always encodes
> #0 (to test, do ".syntax unified" and then "b #60" in GNU as).  WTF?
> 
> Likewise with bl, blx.
> 
> It seems that the debug info still has the user-specified immediate value--but
> the executable object file does not.  So in order to test you should strip the
> resulting file--otherwise you are gonna be very confused.
> 
> No error or warning messages are printed by GNU as here.

With GNU as 2.34 on ARM the same happens for:

  b r3

It encodes as

 eafeb   0x0

Spooky...


pgpAXQ9qfpaA6.pgp
Description: OpenPGP digital signature

ARM Unified Assembly Language - GNU as does some weird stuff

[Danny Milosavljevic]

Hello Paul,

we are now implementing ARM inline assembly in TinyCC.

The traditional ARM inline assembler is finished now.  Now we started
implementing the Unified Assembly Language dialect.

I'd like to have some help which constructs are valid and which are not.

(GNU as 2.34 seems to have some bugs with Unified Assembly Language--so I
guess we shouldn't use "do what GNU as does" as a yardstick here)

The questionable--maybe invalid--constructs follow (the "#" are for clarity):



(1) b #60

It seems that GNU as ignores the immediate entirely and just always encodes
#0 (to test, do ".syntax unified" and then "b #60" in GNU as).  WTF?

Likewise with bl, blx.

It seems that the debug info still has the user-specified immediate value--but
the executable object file does not.  So in order to test you should strip the
resulting file--otherwise you are gonna be very confused.

No error or warning messages are printed by GNU as here.



(2) push #4

It works in GNU as--but is it specified by ARM to push the register r2 ?
I think exposing ISA implementation details like that is a leaky 
abstraction--and
no good can come from it.

Likewise with pop, stm*, ldm*.

No error or warning messages are printed by GNU as here.



(3) and r3, r4, LSL #5

This doesn't work in unified mode--but does work in non-unified mode.
Note that "and r3, r3, r4, LSL #5" always works--both in unified mode and in
non-unified mode.

I would have thought the LSL token would be a dead giveaway for disambiguating
what to do here--but apparently GNU as thinks otherwise.

Likewise with ands, eor, eors, sub, subs, sbc, sbcs, rsc, rscs, rsb, rsbs,
orr, orrs, bic, bics, cmp, cmns, mov, movs, mvn, mvns (all arithmetic
instructions), and with other modifiers (LSR, ASR, ROR, RRX).

GNU as fails to assemble these.



(4) lsl r1, #4, #2

GNU as encodes exactly the same as "lsl r1, #4"--drops the "#2" silently.

Likewise with lsr, lsrs, asr, asrs, orr, rors.

No error or warning messages are printed by GNU as here.



(5) vmov.f32 r2, r3, d1

f32 and two 32 bit ARM registers, and one 64 bit VFP register?  What does this
do?!

Likewise with "vmov.f32 d1, r2, r3".

No error or warning messages are printed by GNU as here.



Which of those are good (and should thus be implemented in the same way)
and which of those are bad (and should thus not be implemented the
same way)?


pgpQJAXIoSHjQ.pgp
Description: OpenPGP digital signature

Re: [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Hi Janneke,

On Fri, 8 Jan 2021 19:56:19 +0100
Danny Milosavljevic  wrote:

> The CI on nanana is currently building and running the tests.
> 
> I'm curious what it will say.

Tests succeeded.

Pushed to mes on savannah as commit 10c38e112f177bc0b01ecf107d193e4c6b13
("wip" branch).


pgpVlyVc2XTIm.pgp
Description: OpenPGP digital signature

Re: [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Hi Janneke,

On Fri, 08 Jan 2021 17:15:24 +0100
Jan Nieuwenhuizen  wrote:

> > +/* TODO: On armhf gcc, max_align_t is 16 Byte big instead.  Use that? */
> > +
> > +typedef double max_align_t;
> > +
> >  #endif // ! SYSTEM_LIBC  
> 
> Is this something you can get more info on, or do we just try it like
> this?

I would just try like this.  I mean I'm sure we could find out what the
16 Byte thing is (long double ?  Nope, not according to both gcc and tcc
on armhf)--but seriously, mescc can't represent that object anyhow--so
there's really no upside to making this bigger (if mescc and tinycc
don't emit it, it doesn't need to be aligned either :) ).

The CI on nanana is currently building and running the tests.

I'm curious what it will say.


pgpVcsGl0NRIz.pgp
Description: OpenPGP digital signature

Re: [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Hi Janneke,

I propose to, instead, change mes libc to align stuff malloc returns like this:

That should fix it.

diff --git a/include/stddef.h b/include/stddef.h
index a597c9bb..a682d726 100644
--- a/include/stddef.h
+++ b/include/stddef.h
@@ -37,6 +37,10 @@
 #endif // !__MESC__
 #endif // offsetof
 
+/* TODO: On armhf gcc, max_align_t is 16 Byte big instead.  Use that? */
+
+typedef double max_align_t;
+
 #endif // ! SYSTEM_LIBC
 
 #endif // __MES_STDDEF_H
diff --git a/lib/stdlib/malloc.c b/lib/stdlib/malloc.c
index f4be4de1..aaf99886 100644
--- a/lib/stdlib/malloc.c
+++ b/lib/stdlib/malloc.c
@@ -20,6 +20,8 @@
 
 #include 
 #include 
+#include 
+#include 
 
 /* FIXME: We want bin/mes-mescc's x86-linux sha256sum to stay the same.
Therfore we cannot remove stdlib/malloc from libc_SOURCES, which is
@@ -37,6 +39,8 @@ malloc (size_t size)
 {
   if (!__brk)
 __brk = (char *) brk (0);
+  /* align what we give back. */
+  __brk = (char*) (((uintptr_t) __brk + sizeof(max_align_t) - 1) & 
-sizeof(max_align_t));
   if (brk (__brk + size) == -1)
 return 0;
   char *p = __brk;


pgpgxAYoYNKpK.pgp
Description: OpenPGP digital signature

Re: [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Hi Janneke,

On Fri, 08 Jan 2021 07:25:52 +0100
Jan Nieuwenhuizen  wrote:

> > Alignment could be disabled on the CPU
> >
> >   
> > https://developer.arm.com/documentation/ddi0464/f/system-control/register-descriptions/system-control-register
> >
> > but I don't think EABI wants that.  
> 
> Hmm, what does this mean?  We are not really using EABI, or are we?

VFP is a floating point unit on ARM CPUs.  It has been designed to either
require aligned members and be fast, or not, depending on a CPU flag that
for example the kernel can set.

See also https://www.keil.com/support/man/docs/armcc/armcc_chr1359124231926.htm
for much more detail.

ARM is a famously mix-and-match CPU, so the client can choose whatever
they want.

If you choose aligned-only, you will get a bus error on misaligned access.
A single program really shouldn't be choosing--it's more the entire platform
doing the choosing.

EABI has standardized on particular settings, among which is required alignment.



> So alignment should be fixed, but that's more work and you propose a
> workaround, right?  

No, I wanted to find out what's going on first.

Now that grischka commented and I looked up the malloc docs, I suggest
to fix mes libc's malloc to align what it gives back.  That's all--that
should fix everything.

We should provide maxalign_t to make it known to the user what the
malloc alignment we are using is.


pgpTdY7kymXqu.pgp
Description: OpenPGP digital signature

Re: [Tinycc-devel] [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Hi grischka,
Hi Janneke,

On Fri, 08 Jan 2021 08:16:29 +0100
grischka  wrote:

> But no such thing happens in this case.  The 'ptr' in init_putv()
> comes from
> 
>  ptr = sec->data + c;
> 
> and it seems that if tcc is doing the right thing then 'c' cannot
> be misaligned, and if malloc/realloc on that system is doing the
> right thing,  then sec->data cannot be misaligned either.  So...?

How does tcc allocate dynamic memory?  I've tried to find out, but
tcc_malloc is defined to be "use_tcc_malloc", which I don't find
anywhere.  Does it use libc malloc for that ?

(With MEM_DEBUG defined, tcc_malloc_debug seems to use malloc.  But
what about without MEM_DEBUG defined ?)

If so, is libc malloc supposed to ensure alignment of allocated memory?

According to https://man7.org/linux/man-pages/man3/malloc.3.html yes.

@Janneke: So our mes libc malloc should be aligning the stuff--but it's not
doing it.  So it's a bug in our libc.


pgpchjZrn1jel.pgp
Description: OpenPGP digital signature

Re: [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Hi Janneke,

I just found the bug in tinycc that caused failed our ARM bootstrap to fail.

I use the following reproducer:

int main() {
double f = 1.0;
return 0;
}

and then invoke

  tcc a.c

on ARM (32) using your patched tcc.  (but it's also broken in the unpatched tcc)

(tcc cross compiler is not enough.  tcc has to actually itself be an ARM
EABI executable)

I get a bus error here:

│   0x24698 vstrd0, [r0]
  │

Debugging some more, I find:

tccgen.c:

/* store a value or an expression directly in global data or in local array */
static void init_putv(CType *type, Section *sec, unsigned long c)
{
[...]
size = type_size(type, &align);
section_reserve(sec, c + size); // c == 0, size == 8
ptr = sec->data + c; // sec->data == 0x24b01e, c == 0
switch(bt) {
/* XXX: when cross-compiling we assume that each type has the
   same representation on host and target, which is likely to
   be wrong in the case of long double */
case VT_BOOL:
vtop->c.i = vtop->c.i != 0;
case VT_BYTE:
*(char *)ptr = vtop->c.i;
break;
case VT_SHORT:
*(short *)ptr = vtop->c.i;
break;
case VT_FLOAT:
*(float*)ptr = vtop->c.f;
break;
case VT_DOUBLE:
*(double *)ptr = vtop->c.d;
break;
[... and so on]

tccelf.c:

/* reserve at least 'size' bytes from section start */
ST_FUNC void section_reserve(Section *sec, unsigned long size)
{
if (size > sec->data_allocated)  // both 8
section_realloc(sec, size);
if (size > sec->data_offset) // both 8
sec->data_offset = size;
}

Nothing here make sure that the VFP double is aligned to 8 Byte.

And indeed, (0x24b01e % 8) == 6, not 0.

Alignment could be disabled on the CPU

  
https://developer.arm.com/documentation/ddi0464/f/system-control/register-descriptions/system-control-register

but I don't think EABI wants that.

tinycc does have:

/* reserve at least 'size' bytes aligned per 'align' in section
   'sec' from current offset, and return the aligned offset */
ST_FUNC size_t section_add(Section *sec, addr_t size, int align)
{
size_t offset, offset1;

offset = (sec->data_offset + align - 1) & -align;
offset1 = offset + size;
if (sec->sh_type != SHT_NOBITS && offset1 > sec->data_allocated)
section_realloc(sec, offset1);
sec->data_offset = offset1;
if (align > sec->sh_addralign)
sec->sh_addralign = align;
return offset;
}

But that's not used for init_putv.

And section_reserve, which is used, doesn't care about alignment at all.

(it seems there's a reserved part and a data part in each section, and
it holds that the data part elements are aligned--but the reserved part
elements are NOT aligned.  I don't see how sec->data would be aligned
by the dynamic memory allocator either)

Other notes:

tccgen.c even has this:

>/* XXX: when cross-compiling we assume that each type has the
>   same representation on host and target, which is likely to
>   be wrong in the case of long double */

Yeah, and even when NOT cross-compiling, the alignment is wrong--which means
it sometimes won't work at all on ARM, depending on luck.

As a workaround, we can patch tcc to instead do the assignments on elements
on the stack and then copy those over, instead of doing

*(double *)ptr = vtop->c.d

(the latter of which emits VFP instructions that expect double-aligned
pointers).


pgp513JL8bZDS.pgp
Description: OpenPGP digital signature

Re: [bootstrappable] Re: wip-full-source-bootstrap: from a 357-byte `hex0' to 'hello'

[Danny Milosavljevic]

Oops, g...@gitlab.com:janneke/tinycc.git in branch "mes-0.23"




pgpUsxbrGINDP.pgp
Description: OpenPGP digital signature

Re: ZFS on Guix

[Danny Milosavljevic]

Hi,

the reason is that our "zfs" package uses

  ("util-linux" ,util-linux "lib")

and then does

   (substitute* "lib/libzfs/libzfs_mount.c"
 (("/bin/mount") (string-append util-linux "/bin/mount"))
 (("/bin/umount") (string-append util-linux "/bin/umount")))

.

That can't work.

zfs interna that will be patched by the Guix package are:

./lib/libzfs/libzfs_mount.c

do_mount(const char *src, const char *mntpt, char *opts)

char *argv[9] = {
"/bin/mount",
"--no-canonicalize",
"-t", MNTTYPE_ZFS,
"-o", opts,
(char *)src,
(char *)mntpt,
(char *)NULL };

Easy fix would be to change our package to have both

  ("util-linux:lib" ,util-linux "lib")

and

  ("util-linux" ,util-linux)

.


pgpA8InGZ7kW2.pgp
Description: OpenPGP digital signature

Re: gnu-build-system not handling pkg-config properly in package definition

[Danny Milosavljevic]

Hi,

try util-linux's "lib" output:

 (inputs
  `(("util-linux" ,util-linux "lib")))

After that, you need (string-append "CC=" ,(cc-for-target)) in make-flags, and
then add a phase before install that creates $output/sbin .


pgp6XAiuynXzX.pgp
Description: OpenPGP digital signature

Re: Question about Guix on Novena - mainly U-Boot

[Danny Milosavljevic]

For reference, the config.scm I used was:

(use-modules (gnu) (gnu bootloader u-boot))
(use-service-modules networking ssh desktop)
(use-package-modules bootloaders screen ssh linux)
;;; Note: Maybe better: linux-libre-arm-generic
;;; Note: Novena upstream uses Kernel 3.19
(operating-system
  (host-name "novena")
  (timezone "Europe/Vienna")
  (locale "en_US.utf8")
  (kernel-arguments '("console=ttymxc1,115200"))
  (initrd-modules (cons* "sdhci-esdhc-imx" "ahci_imx" "sd_mod" "i2c-dev" 
%base-initrd-modules))
  (bootloader (bootloader-configuration
   (bootloader u-boot-novena-bootloader)
   (target "/dev/sda")))
  (file-systems (cons (file-system
(device (file-system-label "novena-root"))
(mount-point "/")
(type "ext4"))
  %base-file-systems))
  (packages (append (list screen openssh) %base-packages))
  (services (append (list (service dhcp-client-service-type)
  (service openssh-service-type)
  (service elogind-service-type))
%base-services)))


pgpTUioCiBp85.pgp
Description: OpenPGP digital signature

Question about Guix on Novena - mainly U-Boot

[Danny Milosavljevic]

Hi,

I'm still in the process of setting up the Novena as a GNU Mes for ARM build
machine.

Guix system on Novena from sd card finally works.

Now I want to make it boot it from SATA instead.

So I did the following:

(1) Used cp -a to copy over everything from the sd card / to the sata drive
partition 2 (ext4 fs).
(2) Fixed up /etc/config.scm to say to install u-boot into target "/dev/sda".
(3) mounted /dev, /dev/pts, /sys, /proc and then chroot /mnt guix system
reconfigure /etc/config.scm
(4) Used novena-eeprom to enable booting from SATA according to
https://novena-guide.readthedocs.io/en/latest/tasks.html
(5) Power off
(6) Take sd card out
(7) Power on
(8) u-boot prompt on Novena serial line never appears

If I instead try to boot Guix from the SATA drive via the u-boot that's on the
microsd card, that fails too, probably because of the patch
https://lists.denx.de/pipermail/u-boot/2008-December/044716.html

That attempt was like this:

(1) Copy the hard drive's /boot/extlinux/extlinux.conf to the microsd's
/boot/extlinux/extlinux.conf, overwriting the latter
(2) Reboot into u-boot-from-the-sd-card
(3) Wait for autoboot in u-boot to commence
(4) It fails to read the initrd (or anything else) from SATA, eventually
failovers to sd card boot because there's still an old guix generation for it

So instead I did this:

(1) Stop autoboot
(2) Execute: sata init
(3) Execute: part list sata 0 -bootable devplist
(4) Execute: run sata_boot

Then it indeed boots Guix from sata starting from the u-boot from microsd.

But I cannot keep it like this because if there's a power outage and restore,
the build server won't come back up correctly.

Help?


pgprlOXroYRfm.pgp
Description: OpenPGP digital signature

Re: Questionable "cosmetic changes" commits

[Danny Milosavljevic]

Hi Raghav,

first, let me say that as far as I'm concerned, you did nothing wrong--although
it caused a lot of work for you to do the rearranging in the first place
(and also some work for us).

Guix irregularities also annoy the hell out of me.  You can check out some
earlier patches by me (when I joined the project) in the archives.  I wanted
(and still want!) to rearrange a LOT of stuff--because, let's face it, Guix's
module layout in general is a mess.  But for better or for worse, it's
not easy to do anything about it and also keep backward compatibility with
what external projects expect of Guix and also keep using the tools that we
have been using so far (git, diff, patch etc).  So even though I still would
prefer Guix module layout not to suck, it cannot be changed in the short term.
So I know where you are coming from.

But in general, arguing about *formatting* is bike-shedding at its best,
especially in a language where formatting does not matter.

However, we do use textual diff, blame and merge tools--and those do not
understand the tree structure of a Lisp program at all.  Rearranging stuff
especially will (and did) cause diff and patch to mistake the insertion
point for changes.  (It will patch it wrong if applied out of order or when
skipping patches--and will often NOT fail)

So for example it's very difficult to leave off the cosmetic patch and just
apply the non-cosmetic patches that came later.

Furthermore, understand that the package fields (and guix record fields in
general) can refer to previously defined fields--so a package with reordered
fields is NOT necessarily semantically equivalent to the original one.

Try these in guix repl:

(let ((name "OUTER"))
  (package-version
   (package
 (name "INNER")
 (version name)
 (source #f)
 (build-system #f)
 (synopsis #f)
 (description #f)
 (license #f)
 (home-page #f

(that gives "INNER")

vs

(let ((name "OUTER"))
  (package-version
   (package
 (version name)
 (name "name")
 (source #f)
 (build-system #f)
 (synopsis #f)
 (description #f)
 (license #f)
 (home-page #f

(that gives "OUTER")

That means each cosmetic patch of you required some extra manual review effort
by me in order to make sure that this does not introduce semantic changes.

That said, if people post reformatting patches (especially if part of a patchset
that was presumably already tested by that person) I usually do not say anything
about it because I don't want to cause extra work that is useful to nobody (and
potentially invalidate all the testing done).

FWIW, I do find it strange that Lisp projects, despite using a minimal-syntax
language (mostly in order to conserve its regular tree structure), do not
usually automatically format source code as they check in, but Go projects,
using the prime example of an irregular C-like language, DOES usually use
code formatters automatically when checking in.  That is some strange
reversal of strengths that I wouldn't have expected.


pgp8cS_qBS01E.pgp
Description: OpenPGP digital signature

Re: /etc/passwd & co. in Docker images

[Danny Milosavljevic]

On the other hand, we have this:

(define-public network-manager-openconnect
  (package
(name "network-manager-openconnect")
[...]
(properties `([...]
  ;; The 'etc/dbus-1/system.d/nm-openconnect-service.conf'
  ;; file refers to account "nm-openconnect".  Specify it here
  ;; so that 'network-manager-service-type' creates it.
  (user-accounts . ("nm-openconnect"))

So which is it?


pgpkMC5GnObef.pgp
Description: OpenPGP digital signature

Xen hypervisor

[Danny Milosavljevic]

Hi Janneke,

with our (gnu bootloader grub) already supporting multiboot, it would be easy
for us to add official support for the Xen hypervisor & Linux.

I've already packaged xen and it's ready to be used.

In order to boot Xen, the grub config should read something like this:

menuentry 'Guix with Xen Hypervisor' --class gnu-linux --class gnu --class os 
--class xen {
insmod part_msdos
insmod ext2
set root='(hd0,msdos1)'
search --no-floppy --fs-uuid --set=root 
c4291ab1-d3ad-4c70-ab97-e6c8cca0af2b
echo'Loading Xen 4.1-amd64 ...'
multiboot   /xen-4.1-amd64.gz placeholder  
echo'Loading Linux 3.10-3-amd64 ...'
module  /vmlinuz-3.10-3-amd64 placeholder root=/dev/mapper/disks-root 
ro resume=/dev/mapper/disks-swap quiet
echo'Loading initial ramdisk ...'
module  /initrd.img-3.10-3-amd64
}

Note especially that "multiboot" is "/xen-4.1-amd64.gz", and that "module"
statements are used for the linux kernel and for the initrd.

The work you have been doing for the Hurd already fetches both the "multiboot"
and the "module" parameters from the boot-parameters via
boot-parameters->menu-entry.

But I can't find how I can set the multiboot-modules in my operating-system
definition.

Like,

(operating-system
  (kernel (file-append xen "/boot/xen.gz"))
  (multiboot-modules (list linux-libre initrd)))

(Or find where it is set for the Hurd in the first place)

Also, how do I get guix's usual initrd into that multiboot-modules list ?

Do you think it would make sense to adapt boot-parameters->menu-entry
slightly so one could specify both a multiboot kernel and a linux kernel
at the same time?  Or would that be too magical?


pgpt98dAuBgGF.pgp
Description: OpenPGP digital signature

Re: Phases and snippets no longer returning a Boolean

[Danny Milosavljevic]

Hi Ludo,

On Wed, 25 Nov 2020 00:17:41 +0100
Ludovic Courtès  wrote:

> With these changes, the return value of origin snippets and build phases
> is now ignored.

Good riddance :)

No, seriously, good change.

> I only removed trailing #t in a couple of files unlikely to result in
> merge conflicts down the road, as Marius rightfully suggested on IRC.

I agree that we shouldn't graciously remove "#t"s in existing packages too
much.  It can just be done slowly as we touch each package anyway.

Since returning #f was effectively broken anyway for a year now (it wouldn't
cancel the build) I doubt we have any legitimate places in build phases where
it legitimately wants to cancel the build and thus returns #f, because that
wouldn't have worked anyway.

But the origin snippets, I'm not sure.  I guess those are lower risk anyway
since if the origin snippet doesn't like something, the result probably
doesn't have all the files anyway--cancelled or not.

Thanks for this change!


pgpQhGW5UPkni.pgp
Description: OpenPGP digital signature

Re: /etc/passwd & co. in Docker images

[Danny Milosavljevic]

Hi Ludo,

On Fri, 20 Nov 2020 12:34:07 +0100
Ludovic Courtès  wrote:

> Danny Milosavljevic  skribis:
> 
> > After Ryan Prior's E-Mail I'm pretty sure my workaround of creating /tmp,
> > /etc/passwd, /etc/group etc is what Docker actually expects one to do.  
> 
> I’ve been pondering whether to create those files in ‘guix pack -f
> docker’ and…
> 
> > ADD etc/passwd /etc
> > ADD etc/group /etc
> > ADD etc/services /etc
> > ADD with-guix-daemon.scm /
> > RUN ["/usr/local/bin/guix", "repl", "/set-mtimes.scm"]  
> 
> … what you’re doing here suggest that ‘guix pack’ should indeed create
> those files.
> 
> Thoughts?

If guix pack can be used to put multiple packages into one pack,
are then packs like profiles, or would that be too much?

Because the question is what to do if you invoke

  guix pack -f docker guix postgresql

.

Both need user accounts--and thus the total required user accounts (and thus
the contents of /etc/passwd) are the union of the respective required user
accounts of both packages.

So someone needs to merge those, and for that the packages need to require
these user accounts in the first place.

But usually in Guix it's Guix *services* that require user accounts and not
Guix packages.  (which makes sense!)

So I would suggest that

  guix system docker-image ...

create /etc/passwd by merging the required user accounts like described
above, but

  guix pack -f docker a b c

really can't do that.  I mean where would it know the requirements from?

But creating /tmp with the right permissions should be easy enough (?).

That leaves how we want to do a Guix release to a Docker registry.

I think if you pack guix-the-package-manager, the question is whether
there are scenarios (100% offloading for example) that allow you to
use guix without the guix daemon also running inside the docker container.

If so, it doesn't make sense to add /etc/passwd and /etc/services and so
on for guix-daemon in the guix pack case, especially since it singles out
one Guix package, guix, for special consideration.

Then you'd use the guix pack'ed image for using guix-the-package-manager
with 100% offload.

We should release a minimal guix-the-operating-system to a docker registry
so the guix-daemon also works (i.e. built using guix system docker-image).

You'd use this Docker system image when you want to use both guix and
guix-daemon.
 
Are there any downsides to just using a trimmed-down operating-system
definition in order to have a docker image with a working guix-daemon ?

Or we can instead add a static /etc/passwd in Docker after the fact and then
release that to a Docker registry--that's what guix-on-docker does now
(though that doubles the size of the result because Docker is being silly).


pgpEHNt0IdZr2.pgp
Description: OpenPGP digital signature

Re: guix depends on openldap?

[Danny Milosavljevic]

Hi Efraim,

On Thu, 19 Nov 2020 15:56:34 +0200
Efraim Flashner  wrote:

> I think a better question is can cmake-minimal depend on a new
> curl-minimal, or does it even need curl at all?

In the course of debugging the "32 bit ARM on 64 bit host can't read 
directories"
problem ("json-c") and starting to fix it, I had also asked myself that 
question.
I've researched back then why cmake needs curl in the first place and it turned
out it is in order to contact a CI dashboard server when tests run in ctest.

They do not support removing the library, but they do support disabling the
(or setting another) dashboard server to contact.


pgpRdGWAJmRS0.pgp
Description: OpenPGP digital signature

Re: Release: Docker Image? DockerHub? skopeo?

[Danny Milosavljevic]

Hi zimoun,

On Thu, 19 Nov 2020 10:21:16 +0100
zimoun  wrote:

> Let’s postpone this Docker image work and start a new fresh thread once
> v1.2.0 is published and the goal to have something for v1.3.0, well
> that’s my point of view.

After Ryan Prior's E-Mail I'm pretty sure my workaround of creating /tmp,
/etc/passwd, /etc/group etc is what Docker actually expects one to do.

So we can just create those--either at runtime, or maybe even have

  guix system docker-image

do it at build time (if it doesn't already).

I have no opinion on when we should do that (at this release or the next
one), except to state that I am certain that it works (and pretty easily),
because guix-on-docker does that already and guix works just fine there.

That still leaves to explain how to prevent Docker from keeping older layers
when it doesn't need to.

In guix-on-docker I have a Dockerfile like this

FROM alpine:3.12 AS bootstrap-guix-on-alpine
[...]

FROM scratch AS guix-on-docker
COPY --from=bootstrap-guix-on-alpine /etc/guix /etc/guix
COPY --from=bootstrap-guix-on-alpine /var/guix /var/guix
COPY --from=bootstrap-guix-on-alpine /gnu /gnu
COPY --from=bootstrap-guix-on-alpine /usr/local /usr/local
COPY --from=bootstrap-guix-on-alpine /root/.config/guix /root/.config/guix
ADD set-mtimes.scm /
ADD etc/passwd /etc
ADD etc/group /etc
ADD etc/services /etc
ADD with-guix-daemon.scm /
RUN ["/usr/local/bin/guix", "repl", "/set-mtimes.scm"]

in order to prevent Docker from keeping older layers[1].

The "set-times.scm" invocation there is in order to fix the timestamps.

"COPY --from=" does not preserve timestamps.  Then guile is very annoyed
because it can't use any of the ".go" files--because they are older than
the respective ".scm" files.

Using set-times.scm means it will create yet another layer where the only
difference is the timestamps--so it doubles the size of the resulting image.

But then it works.

[1] https://docs.docker.com/develop/develop-images/multistage-build/


pgpRHSyXoKbMP.pgp
Description: OpenPGP digital signature

Re: Release: Docker Image? DockerHub? skopeo?

[Danny Milosavljevic]

Hmm, maybe I'm misunderstanding what Docker compose does entirely.

Reading the docs it seems to just manage multiple isolated Docker images and
deploy new ones?

What then is used to do the equivalent of a guix profile with multiple packages
in a Docker image?


pgpxTzXrSUfaY.pgp
Description: OpenPGP digital signature

Re: Release: Docker Image? DockerHub? skopeo?

[Danny Milosavljevic]

Hi zimoun,

On Tue, 17 Nov 2020 18:30:57 +0100
zimoun  wrote:

> > Yeah, someone needs to create /tmp.  That someone is not Guix (it
> > would be weird).  
> 
> Created where?

Created when creating the docker image--presumably by Docker when evaluating
Composefile.

AFAIK Docker images are supposed to be composed of multiple things using a
Composefile.[1]

So Docker has to have some mechanism to set up the shared space that all those
multiple things need.

Ideally, we/the user should use this mechanism to add /tmp--not create it
manually by some weird script in the running container.

> If with the container:
> 
> --8<---cut here---start->8---
> docker exec guix mkdir -h
> OCI runtime exec failed: exec failed: container_linux.go:349: starting 
> container process caused "exec: \"mkdir\": executable file not found in 
> $PATH": unknown

You have to use guix repl, then it will work.
There's no coreutils in there, just guix.

That's what /with-guix-daemon.scm in guix-on-docker does (it's already in the
image), among other weird things.

I paste it in full here:

(use-modules (ice-9 match))
(if (not (file-exists? "/tmp"))
(mkdir "/tmp"))
(system "/root/.config/guix/current/bin/guix-daemon 
--build-users-group=guixbuild --disable-chroot &")
(sleep 2)
(match (command-line)
 ((me next-executable rest ...)
  (match (cons next-executable rest)
   ;; gitlab runner invokes "sh -c" for a shell existence check.
   ;; Replace it by bash.
   (("sh" "-c" rest ...)
(apply execlp "guix" '("guix" "environment" "--ad-hoc" "bash" "coreutils" 
"--" "bash")))
   ((next-executable rest)
(apply execlp next-executable (cons next-executable rest)
 ((me)
  ;; For gitlab runner, see 
.
  (apply execlp "guix" '("guix" "repl"
(exit 1)

It's used as entrypoint by me manually (because it's full of weird workarounds
like this it's NOT the default entrypoint), overriding docker's entrypoint by
["guix", "repl", "--", "/with-guix-daemon.scm"].

> Sorry to be so naive.

No, I don't use docker that much--and when I do, it's to run simple images
others have created.  So I just really don't know how this is supposed to
be set up.  I mean there has to be a way to set this up--that is one of the
first things anyone would need--shared /tmp, /etc/passwd, /etc/group,
/etc/services and so on.  The parts that are composed together by Docker have
to negotiate a common version of those, right?

> > /tmp is a common directory, so it does not belong to any of the parts that
> > have been composed, or maybe there should be a standard "/tmp" part?
> >
> > Is there such a standard "/tmp" part in Docker-land?
> >
> > Similarly, but worse, with /etc/passwd, /etc/group, /etc/services and so 
> > on.  
> 
> I do not know.

I hope that others will chime in explaining what the standard way to do this
is.  The workaround above *does* work, though (and is the wrong thing to do).

[1] https://docs.docker.com/compose/


pgpA5630q4mps.pgp
Description: OpenPGP digital signature

Re: Release: Docker Image? DockerHub? skopeo?

[Danny Milosavljevic]

Hi,

> Yeah, someone needs to create /tmp.  That someone is not Guix (it would be 
> weird).

There is a workaround in /with-guix-daemon.scm in that guix-on-docker container,
which is how I'm using it.

The entrypoint I'm using (with the unmodified guix-on-docker image) is

   ["guix", "repl", "--", "/with-guix-daemon.scm"]


pgpE5Bg6sRsUV.pgp
Description: OpenPGP digital signature

Re: Releasing guix binary in Docker format too?

[Danny Milosavljevic]

Hi,

On Sun, 15 Nov 2020 19:30:44 +0100
zimoun  wrote:

> $ docker exec guix guix pack hello
> user with UID 0 not found

Docker needs to generate a /etc/passwd with uid 0 and the guix build user 
accounts, and a /etc/group with the guixbuild group; and whatever other users 
the things that are composed together using docker compose[1] require.  How 
does this work in Docker-land ?

> substitute: guix substitute: warning: ACL for archive imports seems to be 
> uninitialized, substitutes may be unavailable

We need to configure substitute server acls (/etc/guix/acl).  So this one is 
totally our fault.

> substitute: guix substitute: warning: ci.guix.gnu.org: host not found: 
> Servname not supported for ai_socktype

Docker needs to generate /etc/services with a union of the required services of 
all the things that are composed together using docker compose.

All in all, these are all obvious things which guix-on-docker already works 
around (I don't know the official way that surely exists--hence the 
workarounds).

The question is: since Docker supports composition[1], how do they handle this 
standard case ?  How can we get Docker to generate /etc/services, /etc/passwd 
and /etc/group for the composed docker image ?

> BTW, the tool skopeo failed in the branch version-1.2.0 as ’weather’
> confirms.  Therefore the Cookbook recipe will be for the next release.

(I can't help with skopeo, never used it before)

[1] https://docs.docker.com/compose/


pgpwMyzo8St0f.pgp
Description: OpenPGP digital signature

Re: Release: Docker Image? DockerHub? skopeo?

[Danny Milosavljevic]

Hi,

On Sun, 15 Nov 2020 22:30:51 +0100
zimoun  wrote:

> Wow!  The image is huge:
> 
> 5.23GB registry.gitlab.com/daym/guix-on-docker latest 884b4a4ddf61 
> 
> Is it expected?

Yes, because Docker is silly.  It will continue to grow.
It should be better when you use an external volume as /gnu/store and /var/guix.

> 

> Then, the ’docker exec’ downloads \o/ and then fails with:

>   guix pack: error: getting status of `/tmp': No such file or directory
> 
> Hum, as in the other message, I am sure I am missing the obvious with
> Docker options.

Yeah, someone needs to create /tmp.  That someone is not Guix (it would be 
weird).

/tmp is a common directory, so it does not belong to any of the parts that
have been composed, or maybe there should be a standard "/tmp" part?

Is there such a standard "/tmp" part in Docker-land?

Similarly, but worse, with /etc/passwd, /etc/group, /etc/services and so on.

> Currently, it builds master, right?  I have not looked but do you think
> it could possible to track the both branches master and version-1.x.y?

Yes, right now it builds current master.

Sure, could easily make it do both.


pgpkCB9nh6Bl2.pgp
Description: OpenPGP digital signature

Re: A plan for parameterized packages

[Danny Milosavljevic]

Hi Ludo,

nice feature!

On Sun, 15 Nov 2020 17:33:28 +0100
Ludovic Courtès  wrote:

> An important question: do we have examples of packages for which we’d
> like to have parameters?

For the embedded/flash rom side:

* Enable/disable building the documentation.  I really don't need a 40 MiB
manual stored onto a 16 MiB firmware flash chip.  If that's better done as an
extra output, fair enough.

* Enable/disable obscure dependencies: Library packages that pull in 400 MiB Qt
into the closure for a thing no one (TM) uses should probably provide a switch 
to
disable that GUI.  Sometimes a package provides multiple GUIs for different
toolkits, in which case one should be able to choose one toolkit and not
build for the others.

* No, even in 2020, I won't start using AmigaFS, NFSv3 and whatever old
protocol/format is still around and superseded by other protocols.

The gexp-functionality of being able to select individual files of a package
is already very useful for use cases an embedded developer would have
(and that's there for a long time already).  So that's nice!

For the kind of feature flags I have in mind, it usually means I don't want
to have the feature *anywhere*--for example, if I don't want to have Qt or
Kerberos or whatever, that's because I want to save the space and thus
it should be able to be *globally* specified--at least per profile.

It doesn't help one bit for space savings if package A doesn't pull in
Kerberos or latex or or Haskell bindings for a Raspberry Pi serial port
library (guess why that's oddly specific ;) ), but package B does.

However, sometimes using static libraries is better and there's no reason to
build the shared libraries.  But that's very much local to whatever I'm
trying to do (at least local to a profile if not package).

I would advise against doing a grep -r -- --enable and introducing all those
as parameters.  Rather I would check the closure of stuff and if the closure
goes from 1200 MiB to 50 MiB, chances are a parameter would be nice there :)

I guess the kind of flags I envision would be set at profile level.

From experience with Gentoo before I can tell you that the combinatory
explosion is a real problem and most of the "more advanced" (toggled more
switches :) ) combinations did not work the majority of the time.


pgpick5BhadGn.pgp
Description: OpenPGP digital signature

Re: Discoverability at the REPL level

[Danny Milosavljevic]

Hi zimoun,

On Sun, 15 Nov 2020 14:02:04 +0100
zimoun  wrote:

> In an ideal world, the first ’,a’ could provide hint for the module to
> ’,use’

There is no "the" module.  Any number of modules could have your searched-for
symbol--and the procedures so found could do completely unrelated things.  One
of the points of using modules in the first place is in order to group together
related things (and in order not to have unrelated things together).

What would be nice is for the module names to be easy to understand so you know
which module to import.  That's currently not great.  For example I have no
idea when something goes into (guix packages) vs (gnu packages).

Also, it would be nice and easy to implement to actually have the Guile REPL
search for all possible loadable modules that contain some symbol if it
encounters an unknown symbol, and print those, too (Guix often already does
that anyway!).

It should be easy to add such a thing to the guile repl.  In addition to
",describe" and ",apropos" there would be ",search" which would loop through
all modules, find the specified symbol and then print the docstrings of each
of those, including the module to use for each.

But since these modules can contain code that runs at module import time,
that's maybe also not what you want to actually happen (it would execute
code of random modules that are in the search path).
Then again, guile has declarative modules, too.  If those don't do that,
maybe just search in those.

Also, maybe you don't want Guile to actually IMPORT things into your namespace
when you do ",search".  You just want guile to list them.  That would be the
only complication.

... in before someone points out that this already exists in guile ;)

> and the ’,d’ should provide both signature and docstring.

I agree.  It's useless to separate those.  I mean it's nice that ",a"
exists--but then having ",d" not list the signature is just causing more work
for the developer.


pgpeN4qWpJyJV.pgp
Description: OpenPGP digital signature

Re: Releasing guix binary in Docker format too?

[Danny Milosavljevic]

Hi,

On Fri, 06 Nov 2020 13:47:02 +0100
zimoun  wrote:

> On Fri, 06 Nov 2020 at 10:59, Ludovic Courtès  wrote:
> 
> >  guix pack guix -f docker -S /bin=bin \
> >--entry-point="bin/guix-daemon --disable-chroot"

Why --disable-chroot here?  (I'm not objecting to it)

> > might work, right?
> >
> > Someone needs to try because the devil is in the details.  
> 
> I will try to give a try.  If someone does not beat me.

Please do.

Note: guix-daemon needs a usergroup in order to actually usefully build
things, with at least one member user.

If Docker actually has first-class composition operators (I don't know), then
that should not go into the same image--but that means the end user has to 
provide a /etc/group and /etc/passwd with at least the "guixbuild" group and
at least one member in that group as a composable image[maybe 1] in order
for guix-daemon to actually work.

Also, guix needs /etc/services for http and https and so on to resolve.

Or you could just add those to the main guix docker image--but I think that
would be kinda weird (even though I'm doing it for mine--but it has a kind
of narrow use case where this is fine for the time being).

Or provide the host /etc/passwd using the "-v" command line option (that
would be kinda weird, too).

Or expect the user to always create a dockerfile and use "FROM" to derive
from the offical image.

[1] https://docs.docker.com/compose/compose-file/compose-file-v2/


pgpVEgIHkrtuM.pgp
Description: OpenPGP digital signature

Re: Release: Docker Image? DockerHub? skopeo?

[Danny Milosavljevic]

Hi zimoun,

On Wed, 04 Nov 2020 12:05:28 +0100
zimoun  wrote:

> Some days ago, we discussed on #guix about releasing Docker images
> ready-to-use.  And you pointed your project:
> 
>   https://gitlab.com/daym/guix-on-docker/
> 
> First question for Danny: is it ready to include it to the release?
> Include meaning communicate about it – with the status experimental.

Yes.  It's done.

For the use case I'm using it for, guix-on-docker is final now.
I doubt that I'll change anything in it, and certainly nothing major.

Right now, my gitlab project automatically tracks guix from git master on
savannah and rebuilds that and pushes the result to gitlab's docker
registry (every time someone commits to guix master).

If we wanted the gitlab project to track a release tag, we'd have to
configure it to do building from a tag, too.  If so, please tell me.


pgp5110WCvptK.pgp
Description: OpenPGP digital signature

Re: Releasing guix binary in Docker format too?

[Danny Milosavljevic]

Hi Ludo,

On Wed, 21 Oct 2020 17:42:36 +0200
Ludovic Courtès  wrote:

> zimoun  skribis:
> 
> > The tool is 'skopeo' and packaged in Guix.
> >
> > However, push to DockerHub requires an account (by Guix project) which
> > requires... probably non-free JS, at least once.  
> 
> Hmm OK.  Users of Docker wouldn’t have to create an account and run the
> non-free JS, so this is probably acceptable.
> 
> The next problem is: what would this image contain?  ‘guix pack guix’ is
> probably not enough because people expect the daemon to be running.
> Adding ‘--entry-point=bin/guix-daemon’ probably isn’t good either
> because ‘guix-daemon’ never exits, which I think is contrary to what
> users expect.

It really depends on how people use the images.

It's totally normal for docker containers to start services they need in
the foreground on container startup, and people then entering that
running container from the outside anyway (!) using docker tools.

The guix I packaged on https://gitlab.com/daym/guix-on-docker/ is
specifically used as a test runner, which is why I background
guix-daemon there.  Otherwise, the gitlab test runners would hang waiting
for guix-daemon to finish (which never happens).

If I wouldn't need that, I'd totally not background stuff.

> For the same reason, an image produced by ‘guix system docker-image’ may
> not be OK because ‘shepherd’ never returns.
> 
> Should we change ‘guix system docker-image’ such that ‘shepherd’ is
> started in the background?

Why?  Docker tools are perfectly capable of entering a docker container
where a foreground process is running.

Also, one has to be careful that the container doesn't get stopped since
the last process finished.


pgp74SOIIR5Yr.pgp
Description: OpenPGP digital signature

Re: GNOME in Guix

[Danny Milosavljevic]

Hi,

On Wed, 04 Nov 2020 10:45:06 +0100
Leo Prikler  wrote:

> But we already know all this from our earlier discussion.

I *know* you already know that--but "we" don't.  I want someone to actually
proceed further, because I cannot.  Hence I posted this on guix-devel
(for the first time with an easy short recipe to follow).

> > Now how do I get more info, using Guix tools?  
> What kind of information do you even want to gather?  To be honest, you
> lose me every time you end up establishing knowledge, that already
> exists between this thread and the mentioned Guile-GI issue.

I am hereby asking someone who knows how this part of guix works to find out
what happens here.

> I suppose if you want to look at how environments are built, building
> them with higher verbosity would be a start, no?

Thanks.  That is what I wanted to know.

I tried guix environment -v 99 -d 1 and it adds nothing useful, especially not
how the other libgobject got in there.

So I still want to know.

> For the purpose of this, it would probably suffice to look at something
> simpler than guile-gi in its totality, perhaps just gobject-
> introspection + glib (note, that you'd need Scheme code to access the
> latter).

Sure, sounds good.


pgpvXyQJ0YZ9v.pgp
Description: OpenPGP digital signature

Re: GNOME in Guix

[Danny Milosavljevic]

Hi,

I've checked guile-gi test/insanity.scm again to find "hard" evidence.

For that, I've just checked out guile-gi anew, then ran test/insanity.scm.

Steps:

(1) git clone https://github.com/spk121/guile-gi.git guile-gi
(2) cd guile-gi
(3) git checkout b454a99b65f927e947faab17d25bd3499829c1b4 # the current one
(4) guix environment --pure -l guix.scm --ad-hoc gtk+ guile
(5) ./bootstrap && ./configure && make
(6) tools/run-guile test/insanity.scm
 Starting test insanity.scm  (Writing full log to "insanity.scm.log")
/home/dannym/src/guile-gi/guile-gi/test/insanity.scm:19: FAIL GObject.Value
/home/dannym/src/guile-gi/guile-gi/test/insanity.scm:23: FAIL GObject.Closure
# of expected passes  1
# of unexpected failures  2
WARNING: (guile-user): imported module (gi) overrides core binding `quit'
(7) cat insanity.scm.log
 Starting test insanity.scm
Group begin: insanity.scm
Test begin:
  test-name: "GObject.Object"
  source-file: "/home/dannym/src/guile-gi/guile-gi/test/insanity.scm"
  source-line: 15
  source-form: (test-equal "GObject.Object"  (module-ref m (quote 
)))
Test end:
  result-kind: pass
  actual-value: #<  7f2d4e2ca180>
  expected-value: #<  7f2d4e2ca180>
Test begin:
  test-name: "GObject.Value"
  source-file: "/home/dannym/src/guile-gi/guile-gi/test/insanity.scm"
  source-line: 19
  source-form: (test-equal "GObject.Value"  (module-ref m (quote 
)))
Test end:
  result-kind: fail
  actual-value: #<  7f2d4e2ea680>
  expected-value: #<  7f2d4e2e0200>
Test begin:
  test-name: "GObject.Closure"
  source-file: "/home/dannym/src/guile-gi/guile-gi/test/insanity.scm"
  source-line: 23
  source-form: (test-equal "GObject.Closure"  (module-ref m (quote 
)))
Test end:
  result-kind: fail
  actual-value: #<  7f2d4e2ea900>
  expected-value: #<  7f2d4e2eae80>
Group end: insanity.scm
# of expected passes  1
# of unexpected failures  2

That's it.  It fails.

(8) Using attached dlopen logger (gcc -fPIC -shared -o block-open.so 
block-open.c, then edit tools/uninstalled-env bottom before the exec to export 
LD_PRELOAD=$PWD/block-open.so), I get:
dlopen libguile-gi
dlopen /home/dannym/src/guile-gi/guile-gi/./.libs/libguile-gi.so.5
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen 
/gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-glib-2.62.6/lib/libgobject-2.0.so.0

There it is again.

(9) ldd /home/dannym/src/guile-gi/guile-gi/./.libs/libguile-gi.so.5 |grep 
gobject
[...]
libgobject-2.0.so.0 => 
/gnu/store/4jl613j0d5y6icbxxmwij75fd0i7qpwn-profile/lib/libgobject-2.0.so.0 
(0x7fe2a0677000)

And there is the other one.

Now how do I get more info, using Guix tools?
#define _GNU_SOURCE
#include 
#include 
#include 
#include 

typedef void *(*dlopen_t)(const char *filename, int flags);

void *dlopen(const char *filename, int flags) {
	void* result;
	dlopen_t dlopen = dlsym(RTLD_NEXT, "dlopen");
	fprintf(stderr, "dlopen %s\n", filename);
	if (filename && strstr(filename, "/") == NULL && strstr(filename, "libcairo-gobject.so")) {
		fprintf(stderr, "dlopen blocked %s\n", filename);
		result = dlopen(filename, flags);
		if (result) {
			fprintf(stderr, "and would have been found! Aborting!\n");
			/*int* x = 0;
			*x = 5;
			abort();*/
		}
		//return NULL;
	}
	result = dlopen(filename, flags);
	return result;
}



pgp23zbw3XtGC.pgp
Description: OpenPGP digital signature

Re: GNOME in Guix

[Danny Milosavljevic]

Hi Leo,

On Tue, 03 Nov 2020 14:41:31 +0100
Leo Prikler  wrote:

> >   (note: "-l guix.scm")
> > 
> > seems to have fixed most of the problems.
> > (There is no automated diagnostic--so who knows whether it did fix
> > them for real?)  
> What diagnostic would you want here?

Whether there exist packages with the same name but different derivations in
the environment (especially if mixing hidden and non-hidden packages with
the same name).

Also, there's probably some kind of environment build file list printing in 
guix.
Using it, it could list libgobjects are in the environment without me having
to use a LD_PRELOAD to find which they are...

Does it exist?  If so, how to use it?

> Is that really the problem at hand? I don't see glib explicitly
> mentioned here, so you should not build glib-with-documentation here. 
> Adding glib to one of your --ad-hoc "chains" should yield a different
> profile.

It's not that direct--but yes, something like that is indeed the problem at 
hand.

If you want more detail, the guile-gi bug report is pretty detailed on it.

Also, better, the reason I put the "historical" section at the end of my
previous post is that it is reproducible later, so we can find out what exactly
happened here.

Following that you will get exactly the same situation--and using (only) the
LD_PRELOAD discussed on guile-gi, you will find the culprit, and you will see
that it is two different libgobjects being loaded.

We are now in a better situation because I do all my development using version
control--even throwaway stuff.  guile-gi devs had this happen before--but their
reproducers got lost to time.  Now we have reproducers that cause it every
single time (both the historical version of guix-gui mentioned and also
file test/insanity.scm in guile-gi by now).

> What is the meaning of "doesn't like" here?  Is it explicitly
> discouraged, poorly supported, work in progress, ...?

It means it does not work, and given what they are doing in the source code
GNOME evidently did not think about this case at all.

And at runtime using guix environment, this does (pretty much always) cause
problems without any special things to be done to cause it.

You can read a lot of details on the bug report on the guile-gi website.

> > This was both my fault, and guix's fault for being VERY obtuse.  

> I don't think looking for someone to blame is particularly useful.

We are engineers--so finding out what is to blame for a problem and
fixing that place is indeed what we do all day (that includes deciding
which projects should have the change, and which shouldn't.  If the
answer is "none" the situation will not improve.  So that's right out).

If you mean blaming people, I don't do that--that would be silly.

"Blaming" programs I will do all day every day (including mine).  Problems like
this need to be seen, otherwise us guix devs don't know that it's like this.

I know it from my own stuff I put into Guix master.  *I* did get those
to work by configuring stuff.  But is it usable for a regular user?

For this guix environment stuff, the answer is a definite no.  (there's
another thread on the guix-devel list about improving that--so it's not
like I'm the only one--it's just a bad interface.  It happens.)

>Regarding the safety aspect, I don't think that's what they're
> going for.  They might be thinking, that "--pure --no-grafts" is close
> enough to what you'd get inside `guix build`, so that you don't have to
> use `guix build` for debugging.

If grafts are broken (which seems likely according to guile-gi devs), grafting
should be fixed now, before a security problem happens and we miss it (...but
state that we fixed it).

I know that guile-gi argues for doing development with --no-grafts, but

(1) I don't have any readable characters when I do that
(2) If I did, it would still not help the end user because it is not the right 
fix

> > > Do you still encounter Guile-GI#96 after packaging?  
> > 
> > I've not finished packaging it yet (I tried--but that's pretty
> > difficult,
> > too!  Help wanted--see README and guix.scm in the guix-gui repo).  

> I'll see what I can do to help, but the only thing I can see are
> "FIXME" strings in guix.scm.  The README even suggests to run `guix
> build`.

Thanks!

If you do the guix build and then run the final script, then
you will see that it does not start up because it cannot find guile
modules.  I am not knowledgeable how to deploy large guile programs,
so I'd appreciate help in doing that.

(I can totally find a hacky way to make it work--but I'd rather use the
canonical way to find external guile modules that surely exists,
especially in guix (... which is written in guile and is a large program
with external guile modules))

> One could disagree about that based on the workflow you've posted here
> ;)

I can assure you this is what anyone would try, until he gets burned like
this and then does some of

(1) gives up and deletes guix
(2) posts a bad review to the m

Re: GNOME in Guix

[Danny Milosavljevic]

 "internal"
> > means.  
> Sure, but that's only if either of them ends up using a different
> version.  That should not be the case for Guix' GNOME stack apart from
> some bootstrap shenanigans, that don't really matter at the point of
> application packages.
> 
> In order to make use of any GI in Guix, you need
> - the GI bindings (one of python-gobject, g-golf, guile-gi, gjs, etc.),
> - *and* the packages of the libraries themselves
> as inputs to your package.  So the resulting graph for your package
> will always be complete and should only contain one relevant package
> per library.

It is good to know that guix guarantees to only have one libgobject
(i.e. "glib" package derivation) in this case.
I therefore withdraw the propagated-input proposal.

> [having two different libgobject in the same process] should not be the case
> for Guix' GNOME stack apart from [bootstrapping]

When using guix environment, it is easy for this to happen in Guix's
GNOME.

Grafting, different glib-with-or-without-documentation packages and propagated
inputs in profiles propagating cairo-gobject into my user profile (not to
mention stuff in the SYSTEM profile) and guix environment being
obtuse not only make this possible, but make it a regular occurence
when using guix environment.  I think the first step in fixing this
is actually automatically diagnosing this problem, which Guix (and
gobject-introspection) do not do right now.

As it is now, often there are two different libgobjects being loaded at the same
time, and those register their types in the respective type registry.  Depending
on who talks to which registry, they get different actual implementers back.

Which registry you talk to also determines whether you will get "the type
doesn't exist" back or not (which is how it should be (!), and fine if people
talked to the correct respective registry--which they don't).

So the current (and maybe perpetual) state is that GNOME does not support this
(and they really don't need to support it), so somebody (possibly
gobject-introspection) should be catching it, if necessary at runtime.
They don't.

> > Does anyone know how to make dlopen fail on duplicate global
> > variables?  
> No, but I don't think that is too relevant here.  See my earlier point
> on how it *is* possible to handle this with the methods GLib provides.

(1) This would make it extremely easy to find problems like this now.

So it's relevant in the sense of it could have spared both me and the guile-gi
devs a LOT of time.

(2) It would make it possible to find problems of this kind in the future.
I do not want to keep fixing the same thing over and over again.  I want there
to be an automated check in place so if there are problems, they are
automatically detected.
In this case I want dlopen to fail in this case and not be able to load a
second libgobject in the first place--neither directly nor indirectly.

(Evidently, programs and *GNOME's own libraries* are not designed for the
alternative of two libgobject libraries loaded in the same process--so
gobject-introspection should not allow it in the first place)

> Am Donnerstag, den 29.10.2020, 20:34 +0100 schrieb Danny Milosavljevic:
> > Or I could just use Gtk in C and use popen("guix ...").  
> You're probably joking, but you should know, that I had moderate
> success passing around just data from Guile to Gtk while not using g-
> golf or guile-gi.  You could either do all your conversions in C like I
> did back then or declare your own data models as C structs and then
> fill them from guile using e.g. guile-bytestructures.
> Obviously, it would be nicer if one could program the GUI itself in
> Guile, but you are free to make the choices you feel are fitting.

I don't want to maintain data structure marshallers that add no value to
anything.  One of the touted advantages of using guix as a package manager
is that you don't need to do that anymore--you just use guile as a
general-purpose programming language and not keep manually translating back
and forth between different incompatible environments, with all the
impedance mismatch, slowness, different semantics, protocol versioning and
bugs that that entails.

Using gtk on guile also already makes me sometimes force a square peg into a
round hole--that should be enough.

> [expose guix via dbus]

You should keep in mind for this approach: impedance mismatch, slowness,
different semantics, protocol versioning, and bugs in dbus (the latter
of which I also regularily experience in guix--dbus bus hangs etc).

The advantages you listed of better and easier privilege checking are
valid, though.

(In addition, I don't want a hard dependency to gtk+ in the guix package 
manager.
But that can be avoided in other ways, too).

I should note that I

Gtk via the web

[Danny Milosavljevic]

Hi Ludo,

On Sun, 01 Nov 2020 22:53:26 +0100
Ludovic Courtès  wrote:

> Lars-Dominik Braun  skribis:
> 
> Long ago Dave Thompson wrote guix-web, which allowed you to install
> packages (you’d run it as your user):
> 
>   
> https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://gitorious.org/guix-web/guix-web.git
> 
> It’s an interesting approach with several advantages (in particular it
> can be quickly developed), though I must say I remain unenthusiastic
> about using web browsers for local GUIs.

You can use Gtk applications in the web using Broadway.  That's why I enabled
Broadway in our gtk+ package a long time ago.

Try it:

broadwayd :1 &
GDK_BACKEND=broadway BROADWAY_DISPLAY=:1 gedit &
icecat http://localhost:8081/

(broadwayd is in gtk+'s "bin" output)

Not saying we have to do that--but it's possible.


pgpyuMpivDzvt.pgp
Description: OpenPGP digital signature

GNOME in Guix

[Danny Milosavljevic]

Hi,

On Mon, 02 Nov 2020 08:44:29 +0100
Pierre Neidhardt  wrote:

> Danny Milosavljevic  writes:
> > Not much more works yet because I've hit this (design) bug in Guix and/or 
> > GNOME:
> >
> > * https://github.com/spk121/guile-gi/issues/96  
> 
> Have you tried g-golf?  The Nomad browser uses it, it may not suffer
> from the aforementioned issue.
> Thoughts on that?

It really depends on what you mean.  If it also uses gobject-introspection, it
will have the same problem.

Even gobject-introspection links to glib internally, then registers a class in
that, THEN opens (potentially another) glib at runtime using dlopen in the same
address same.
I'm pretty sure that GNOME is not designed for that.  If you then use the
dlopened glib to use the first-mentioned class, good luck with that.

The only reason it didn't fall onto the head of users of other distributions
is because those don't support internal dependencies of packages at all--so
you always have to update all in lockstep.  So the following can only happen
in Guix:

If a library A has an input Q, and library B has the same input Q, and you
use A and B in your program, and then you (guix-) update just B to use Q'
(for example a newer version of Q) and recompile your program, then you have
both Q and Q' in your process address space!

Q is an internal dependency of A, and by coincidence it's an internal
dependency of B, too. Just because the internal dependency of B changed
shouldn't change the internal depenency of A--that's what "internal"
means.

Otherwise the modules aren't really modular (because they then don't isolate
from each other).

I think that the gobject type registry does not consider that it can happen
to have to classes with the same class name, registered in respective
internal dependencies.  The objects of the internal dependency shouldn't
be available to the outside program in the first place.

That means actually we should propagate a lot of GNOME inputs, which means
in practice what a regular user has in his profile will be very much like
in other distributions, and internal dependencies will become a weird corner
feature nobody uses.

Does anyone know how to make dlopen fail on duplicate global variables?

I tried to make it fail but I can't get it to work:

I tried:

File d1.c contains:
int x;

File d2.c contains:
int x;

File a.c contains:

#include 
#include 

// 
https://stackoverflow.com/questions/43582165/handling-global-variables-in-shared-object

int main() {
if (!dlopen("./d1.so", RTLD_NOW | RTLD_GLOBAL /*| RTLD_DEEPBIND*/)) {
fprintf(stderr, "dlerror: %s\n", dlerror());
return 1;
}
if (!dlopen("./d2.so", RTLD_NOW | RTLD_GLOBAL /*| RTLD_DEEPBIND*/)) {
fprintf(stderr, "dlerror: %s\n", dlerror());
return 2;
}
return 0;
}

Then I do:

$ gcc -fPIC -shared -Wl,--export-dynamic -o d1.so d1.c
$ gcc -fPIC -shared -Wl,--export-dynamic -o d2.so d2.c
$ gcc a.c -ldl
$ ./a.out
$ echo $?
0


pgprGMw9IB3hY.pgp
Description: OpenPGP digital signature

Re: A better way to access records.

[Danny Milosavljevic]

Hi Brendan,

On Fri, 30 Oct 2020 21:59:59 +1100
Brendan Tildesley  wrote:

> No I didn't want to specify the fields at all, just have all of them 
> automatically defined.

I think that that is a bad idea for maintenance reasons.

This totally would hide variables from the enclosing context without you
being able to tell that it does so from the lexical context.

For example let's say you have:

(define-record  size)

(let ((color 5))
  (with-record foo
color))

And later on you update  (but do not change the text of the with-record
usage at all), so in total you have:

(define-record  size color)

(let ((color 5))
  (with-record foo
color))

Now color is a different one!!

I don't even like unqualified imports for that reason: a change in a remote
place can affect what this module does without this module body referring
to it literally in the first place.

On the other hand, when directly specifying the fields

  (let ((color 5))
(with-record (foo size)
  color))

I'm all for that.


pgpBFk2lOijyG.pgp
Description: OpenPGP digital signature

Re: Outdated avr-toolchain

[Danny Milosavljevic]

Hi,

On Fri, 30 Oct 2020 12:47:18 +0530
Arun Isaac  wrote:

> Our avr-toolchain is very old and outdated. Our version is 5.5.0,
> whereas the latest seems to be 10.2.0. We also have multiple versions of
> the avr-toolchain packages along with functions to produce those
> multiple versions. Can we remove these functions and different versions,
> and keep only one latest version of the avr-toolchain?

As one of the people using avr-toolchain, +1

> I have an old patchset[1] to this effect, but I didn't want to proceed
> without a discussion. I was concerned that there might be a use, for all
> these different versions, that I did not comprehend.

I don't think so.  The AVRs are so tiny and development using them is usually
done by copying library source code into your project, so I cannot see any
reason to care about different gcc versions for AVR.

avrdude is another matter--it's very easy to break that if you don't actually
test it.


pgph5juYWeinq.pgp
Description: OpenPGP digital signature

Re: Guix Front End (GUI) and making it more mainstream, popular in scientific community.

[Danny Milosavljevic]

Hello everyone,

On Thu, 29 Oct 2020 21:55:10 +0530
Aniket Patil  wrote:

>If we build a GUI frontend then we can attract more
> people who don't identify themselves with CLI users. As such there are many
> people like writers, Graphic Designers, General users, etc. who don't
> bother about CLI, will love to see such a package manager.  And considering
> the fact GUIX lets rollback to the previous version of the package it will
> be very helpful in the scientific community, to maintain and work with
> different versions of packages.

I've got one at https://gitlab.com/daym/guix-gui/ .

As of commit 978f7d92b4486829fd4f5030baf2d42c001b0890 (current master),
guix-gui works well enough that you can see the current profile, you have
(slow) tooltips, you can type to search, you can view the previous generation,
and you can get a list of all installable packages in Guix.

Not much more works yet because I've hit this (design) bug in Guix and/or GNOME:

* https://github.com/spk121/guile-gi/issues/96

I hadn't noticed before just how bad a match Guix's private dependencies (which
I think are a good thing) and GNOME's object system is (see link above).

> P.S: I don't know whether GUI for Guix exists or not, to my knowledge it
> doesn't. If I am mistaken please enlighten me on this topic.

It does kinda exist--but working on GUIs in Guile is like pulling teeth.

I'm used to seamless integration like PyGTK is in Python--PyGTK feels
EXACTLY like a native Python library.  You can't tell that it's implemented
in C at all.

The way it is in Guile is not seamless.  Furthermore, there are fundamental
problems in how the package "gobject-introspection" is designed (see link
above)--so I can't see how this can be resilient or consistent at all.

(Also, I've used Delphi before--that's basically the holy grail of GUI
development.  That doesn't help making me less grumpy using this contraption
string-and-duct-tape barely-holding-together contraption instead)

In any case, with all the involuntary yak-shaving I have to do it's gonna
take a while.

Or I could just use Gtk in C and use popen("guix ...").


pgpSoL0oVDO2w.pgp
Description: OpenPGP digital signature

Re: Packaging Python projects managed with Poetry

[Danny Milosavljevic]

Hi,

On Thu, 22 Oct 2020 17:15:20 +0200
Tanguy Le Carrour  wrote:

> does not contain a `setup.py` file –because Poetry does not use it!—, and
>the `python-build-system` fails.
> I haven't wrap my head around this yet and I'm not sure what would be
> the proper way to do it?

>Write a `python-poetry-build-system`? I hope not!

Why not?

According to https://github.com/python-poetry/poetry they took inspiration
from existing build systems like cargo, and they just replaced setup.py by
pyproject.toml.

So what you could do is create a poetry-build-system that is just like
python-build-system (probably even inherits from it) but uses "poetry"
instead of "python setup.py".

If the author of a package replaces the build system used in his actual
project, he has to expect to also have to replace the build-system reference
in the guix package.  Why is that weird?

Or you could try to add it to the existing python-build-system--but the
poetry website doesn't sound like it's designed like that (it rather sounds
like they want to replace all other python build systems).

> Just put the d**n tests in the Python package? This would look like a
> failure to me! :-(

If the end user doesn't need the tests, the tests shouldn't make it into the
derivation of your package.  But they are there while the package is building
the derivation--so just run the tests then.


pgpK2Jd5ZKHV2.pgp
Description: OpenPGP digital signature

Importing package-overlay-for-Nix - How?

[Danny Milosavljevic]

Hi,

since the Raspberry Pi open firmware effort has unstalled

  https://github.com/librerpi/rpi-open-firmware/

I want to eventually package it in Guix.

It turns out that there are already config files for nix in the project above.

And we have a nix importer in Guix.

How do I use it

  guix import nix

in order to import those packages?

See

https://github.com/librerpi/rpi-open-firmware/blob/master/bcm2835.nix
https://github.com/librerpi/rpi-open-firmware/blob/master/bootloader.nix
https://github.com/librerpi/rpi-open-firmware/blob/master/default.nix
https://github.com/librerpi/rpi-open-firmware/blob/master/netboot.nix
https://github.com/librerpi/rpi-open-firmware/tree/master/nix

for what I want to import.


pgpOG5iNaBBHy.pgp
Description: OpenPGP digital signature

Re: Using #true and #false everywhere?

[Danny Milosavljevic]

Hi Ludo,

On Fri, 16 Oct 2020 12:38:23 +0200
Ludovic Courtès  wrote:

> As for the manual, I’m afraid it’ll make every msgid that contains
> @code{#t} stale.  So maybe now’s not a good time to make this change?

Now's definitely not a good time to make this change.

I think it's a good idea to make the change eventually, but I wouldn't change
it so short before a release.


pgps7Qk_esiV2.pgp
Description: OpenPGP digital signature

Re: Shipping more installer images?

[Danny Milosavljevic]

Hi Ludo,

On Tue, 13 Oct 2020 15:51:19 +0200
Ludovic Courtès  wrote:

> > IMO there are only very few RYF-worthy ARM devices--and we should support
> > at least those, if we support any ARM devices at all.  That includes
> > providing images for those few (at least A20-EOMA68, A?0*Olinuxino*, and
> > Novena).  
> 
> So that’s 3 more 1 GiB installer images, right?  If we target Oct. 29th,
> how are we going to test them in the meantime?  How long will it take to
> build them, even assuming we build on an OverDrive?

Sure, I didn't mean "provide them in this release".

I meant: eventually.


pgpluc_OUmEP7.pgp
Description: OpenPGP digital signature

Re: Shipping more installer images?

[Danny Milosavljevic]

Hi Ludo,

On Mon, 12 Oct 2020 13:47:25 +0200
Ludovic Courtès  wrote:

> Mathieu Othacehe  skribis:
> 
> Keep in mind that images use space at ftp.gnu.org and also take time to
> build (having CI up-to-date helps with that, but it doesn’t not
> eliminate build times due to the ‘update-guix-package’ dance that takes
> place during “make release”.)
> 
> Likewise, if we ship more images, we should update the “System
> Installation” section accordingly and be clear about what users can
> expect.
> 
> I guess all I’m saying is that we should not make such decisions lightly
> and be sure to examine all the consequences.

I agree.

IMO there are only very few RYF-worthy ARM devices--and we should support
at least those, if we support any ARM devices at all.  That includes
providing images for those few (at least A20-EOMA68, A?0*Olinuxino*, and
Novena).

(Now that we use genimage in order to create guix images anyway, long term
we can also provide an importer and import essentially all ARM devices
from buildroot--but that can come later)


pgpyRhadDcRdb.pgp
Description: OpenPGP digital signature

Re: Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

[Danny Milosavljevic]

Hi Ludo,

On Mon, 05 Oct 2020 14:20:08 +0200
Ludovic Courtès  wrote:

> Danny Milosavljevic  skribis:
> 
> > I'm trying to bootstrap current Guix (master) from Guix past (1.1.0 binary
> > tarball).
> >
> > The goal is: I want to have only guix-the-package-manager at a specific guix
> > commit (!) available inside a Docker image.  
> 
> Why build Guix from source?  I guess it’s enough to do:
> 
>   guix pull --commit=XYZ
> 
> if all you want is Guix at commit XYZ.  Or am I missing something?

I'm doing a project for Heads where we are trying to switch over their build
system to something that makes their builds more reproducible (for example
Guix).

They are using github and gitlab test runners for a lot of things, so one of
the ways we are trying to do continuous integration is to do the following:

(1) Have guix-the-package-manager be built and published on
repository.gitlab.com.  It eventually does "./pre-inst-env guix pack guix"
and then puts the result into a new docker container.  I can't see how to do 
that
after a guix pull.  Note that I don't want to also carry garbage (this entire
thing has to be verified for security eventually, so...).  Currently, guix
is being bootstrapped from Alpine, and I don't want Alpine to remain in there.

(2) Use the result in order to build boards using tiny Dockerfiles
which would just say

  FROM repository.gitlab.com/guix-on-docker
  RUN guix build heads-kgpe-d16

and throw away the derivation (or publish it, too?)--but keep the log file
and exit status.

Note that (1) should pin a specific Guix commit for a long time since Heads
does not want to build on a moving target since they do hash verification
on bootup, and firmware is hard to keep working (i.e. someone has to
manually verify, on real hardware, whether stuff still works after an
update of the toolchain).  And Heads basically is ONLY security-relevant
stuff.

But you are right--I'll now instead just guix gc and then copy /gnu and
/var/guix and /etc/guix into a "FROM scratch" Docker image.


pgpkeQX5Swr8I.pgp
Description: OpenPGP digital signature

Re: Release v1.2 timetable

[Danny Milosavljevic]

On Fri, 9 Oct 2020 20:25:57 +0200
Andreas Enge  wrote:

> On Tue, Oct 06, 2020 at 09:57:20AM +0200, Mathieu Othacehe wrote:
> > The CI is already building substitutes for two images
> > (hurd-barebones-qcow2-image and pine64-barebones-raw-image). We could
> > maybe release 1.2 version of those images.  
> 
> Nice! For completely selfish reasons, I would like to also see an
> image for the novena board.

+1

I have a Novena board now (for GNU Mes for ARM work)--but no Guix
for it yet.

In other news, I've disabled building 32 bit packages on 64 bit machines
in guix-maintenance now.

A Novena image must NOT use any 32 bit substitutes that had already been
built *on 64 bit machines* to make that Novena image--otherwise we would
be playing with fire.


pgpnLDnjf0cDp.pgp
Description: OpenPGP digital signature

Re: Release v1.2 timetable

[Danny Milosavljevic]

Hi,

On Wed, 7 Oct 2020 16:51:46 +0200
zimoun  wrote:

> BTW, is this Bug #43720 [5] blocking for the release too?

I'd say yes.  If homedirs are not created, one can't use the shiny new Guix
system, now can one?


pgpD0bhyarlf9.pgp
Description: OpenPGP digital signature

Re: Release v1.2 timetable

[Danny Milosavljevic]

Hi zimoun,

On Wed, 7 Oct 2020 17:39:44 +0200
Danny Milosavljevic  wrote:

> > BTW, is this Bug #43720 [5] blocking for the release too?
> > 
> > Well, what is the status and the plan about this armhf topic?  

The short term plan is to only build 32 bit releases on 32 bit machines and
64 bit releases on 64 bit machines (I chose my words very carefully here.
I mean exactly that.  To be clear, do not build armhf on x86_64, do not
build armhf on aarch64, and best also do not build i686 on x86_64).

If there are already substitutes built that way, make sure to take those
offline.

This is not a fix but only a workaround--see table [1].  It IS perfectly
possible and easy for a 32 bit kernel to return a 64 bit value to userspace.
And then userspace can fail--or worse, not fail but still have an error.

Long term, I am fixing it in guix branch wip-file-offset-bits-64-sledgehammer
by enabling large file support.  That means that off_t will be 64 bits on all
architectures (including 32 bits).  Once off_t is the same size on all
architectures, an architecture-dependent off_t size mismatch can't happen,
now can it?

All the 32 bit distributions I know how enabled large file support decades
ago--so it should work fine by now.

[1] http://issues.guix.gnu.org/issue/43591#30


pgphsOf0cuRSA.pgp
Description: OpenPGP digital signature

Weird things found while fixing basic Guix packages

[Danny Milosavljevic]

Hi,

because I don't have another place to put this, here are some weird things I
found while I was fixing basic Guix packages (for fixing bug #43513), which
have nothing specifically to do with bug #43513:

* curl[-minimal]: Needs (delete-file "tests/data/test1094") to fix a test 
failure:
  ;; "timeout: 5[LF]"(expected) vs "timeout: 6[LF]"(generated).
  What causes this?

* Why does cmake-bootstrap have curl-minimal as a dependency in the first place?

  It makes sense if bootstrap packages don't have extra dependencies that can
  be avoided.  And this one sounds really weird.  We want to have NETWORK
  CONNECTIVITY while bootstrapping??

  It turns out that that exist in order for ctest to automatically publish test
  results to a dashboard server.  It seems that that cannot be compiled out.
  Could upstream make it possible to remove curl?
  (If one specifies CMAKE_TESTS_CDASH_SERVER="NOTFOUND" then it won't publish.
  CMake_TEST_NO_NETWORK also exists)

* cmake and cmake-bootstrap could be updated to the newest release (3.18.3)
  with a tiny adaption to the snippet.  Do we want to?  It doesn't help us
  with the thing above, though.

* lz4 has "CC=gcc".  Shouldn't that be (string-append "CC=" (cc-for-target)) ?

* Long term, would it be possible to have some kind of check that the closure
  size of bootstrap packages can't exceed some amount?  This way, new
  weirdness like this cannot sneak in.  We need a LOT of checks of
  derivations anyway.  So many things come to mind.

* Grep and coreutils fail strerror_r and perror2 tests in gnulib-tests.
  test-perror2.c:84: assertion 'msg3 == msg4 || STREQ (msg3, str3)' failed
  test-strerror_r.c:170: assertion 'msg3 == msg4 || STREQ (msg3, str3)' failed


pgpOE7R9eG8WD.pgp
Description: OpenPGP digital signature

Continuous integration - automatic EMAIL

[Danny Milosavljevic]

Hi,

please, let's improve building and testing processes.

Please, can we have the build servers send build failures to guix-devel
instead of hoping that people check manually?  I have other things to do
in my life than to poll random servers every few hours.

If you don't want to send e-mail on all failures (probably an overwhelming
flood at first--that's how bad the current state is!), then I suggest to:

Daily do, on each supported architecture (and I mean those packages
specifically--it's not an example stand-in):

  guix pull
  guix build glibc || mail guix-devel@gnu.org -s "glibc failed"
  guix build gtk+ || mail guix-devel@gnu.org -s "gtk+ failed"
  guix build gnome-desktop || mail guix-devel@gnu.org -s "gnome-desktop failed"

and I mean automatically--not triggered manually by a person.

Maybe I'm missing something--but the amount of manual fiddling I have to do
which a computer could automate perfectly well is REALLY inefficient.

There's nothing fundamental missing--why can't the last obvious step be
added so it is actually automatic?


pgpMFM7LEPY9v.pgp
Description: OpenPGP digital signature

Re: Release v1.2 timetable

[Danny Milosavljevic]

Hi,

On Wed, 7 Oct 2020 16:51:46 +0200
zimoun  wrote:

> I have read Bug #43513 and Patch #43591 [1,2] and the table [3] is not
> really encouraging. :-(

This bug has serious repercussions for bootstrapping--and thus for the entire
GNU Guix distribution (not to mention those other distributions which do not
enable large file support).  If I were glibc I would be recalling their
last few releases and would be telling everyone not to use those releases.

> If I understand correctly, the plan is to disable some architectures
> with this Patch #43829 [4], right?

Yes, that is correct.

> BTW, is this Bug #43720 [5] blocking for the release too?
> 
> Well, what is the status and the plan about this armhf topic?

There's a branch wip-file-offset-64-sledgehammer (branched off core-updates)
where I'm trying to fix this bug for good.

I keep running into other problems (which have nothing to do with this bug) on
it--which is why I have to do other commits to fix those OTHER problems (to
core-updates).

Each one of those unrelated bugs delays me for a day each (first I have to
notice that one of the build servers stopped building everything on
wip-file-offset-64-sledgehammer, then I have to build core-updates to
see whether it has the problem too (also, chances are that the build of
core-updates on i686-linux-on-x86_64 and armhf-linux-on-aarch64 hangs because
the readdir problem causes an endless loop!), and then fix the problem on
core-updates, then rebase branch wip-file-offset-64-sledgehammer on that
core-updates, and finally restart the build of everything on all architectures
again).

(For example, I just fixed mesa--but it's not reproducible.  Was it not
reproducible before?  No one knows... WHY NOT?)

(Right now xorg-server has a problem with event (non-)ordering... again,
something that is definitely unrelated to file size.  So there I have to go
test that on core-updates again)

Not to mention that on the aarch64 build server I still don't have a working
home directory.  Otherwise I'd just do crontab -e and add the thing above
by myself.  But crontab -e probably doesn't work either...

This is so frustrating.


pgpnY2lTxdyyw.pgp
Description: OpenPGP digital signature

Re: u-boot for beagleboard

[Danny Milosavljevic]

Hi Vagrant,

On Thu, 01 Oct 2020 10:15:40 -0700
Vagrant Cascadian  wrote:

> Maybe make-u-boot-package should be extended to support passing a list
> of "common" names for boards, which could then be appended to the
> description?

Translators should say what they think about that.

(Long ago, I stopped doing string concatenations like this because
translating that into some languages isn't easy)

Other than that, sure, something like that would be nice.

Maybe just make it possible to add any sentence(s) to the description, though.
That should be translatable enough... I think.




pgpLXNAgmIJZO.pgp
Description: OpenPGP digital signature

Re: Release v1.2 timetable

[Danny Milosavljevic]

Hi Zimoun,

On Tue, 29 Sep 2020 14:16:30 +0200
zimoun  wrote:

>  b. fix bugs severity:important,serious [3,4] or report

Bug #43513 means that all armhf substitutes built for armhf on anything else
than armhf (especially those built on aarch64) are untrustworthy.

I'm working on fixing it (Patch #43591)--but it will entail a complete rebuild
of all packages at least on all 32 bit architectures.

If this bug is not fixed AND TESTED in time for the release, I would ask to
remove ARMHF and i686 from the list of supported systems for the time being,
and disable substitutes for armhf and i686--at least those built on a machine
that is not exactly the same (armhf and i686, respectively).

That bug is horrible.


pgpHXj7LREAIs.pgp
Description: OpenPGP digital signature

Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

[Danny Milosavljevic]

Hi,

I'm trying to bootstrap current Guix (master) from Guix past (1.1.0 binary
tarball).

The goal is: I want to have only guix-the-package-manager at a specific guix
commit (!) available inside a Docker image.

Because the package "guix" in guix is always behind a little bit, that
means I have to first check out that commit from source code, then use
make update-guix-package in order to update the guix package definition
in gnu/packages/package-management.scm to that commit and then use guix pack
in order to pack guix into a tarball, which I then extract and then copy into
a previously-empty docker image.

Or so I thought.

What I have currently is on https://gitlab.com/daym/guix-on-docker/ .
Especially see Dockerfile.

You can build it on your machine using

$ docker build --build-arg 
BOOTSTRAP_GUIX_COMMIT_ID=0b21bf6245b36ea21d2eeaf2340a99aa4d5894db .

(or a newer commit--it doesn't matter) and see it fail.

(it will first download 
https://ftp.gnu.org/gnu/guix/guix-binary-1.1.0.x86_64-linux.tar.xz
and then check out guix anonymously from 
https://git.savannah.gnu.org/git/guix.git at
the specified commit)

Or you can look at the CI logs on
https://gitlab.com/daym/guix-on-docker/-/pipelines and see it fail there.

The failure is when it does that:

&& ENV="guix environment --pure guix --ad-hoc git guile-readline guile-json 
guile-zlib guile
-lzlib bash which --" \
&& ${ENV} make update-guix-package \
&& ./pre-inst-env guix pack --verbosity=2 -f tarball --localstatedir -r 
/tmp/guix.tar.gz --profile-name=current-guix guix \

and it prints the following:

(see https://gitlab.com/daym/guix-on-docker/-/jobs/755694619 )

make[1]: Leaving directory '/master'
accepted connection from pid 17760, user root
git rev-parse HEAD
ff43f128b7c142ae3f758d34137e562e6f7ef0e0
./pre-inst-env "/gnu/store/s08nkx9dzpvn41s4k277p9b27abpcjvp-profile/bin/guile"  
\
   ./build-aux/update-guix-package.scm  \
   "`git rev-parse HEAD`"
accepted connection from pid 17766, user root
source code for commit ff43f128b7c142ae3f758d34137e562e6f7ef0e0: 
/gnu/store/3bz5q9g2p11pazy4g4vq8x8qp86c7ngh-guix-1.1.0-28.ff43f12-checkout (GC 
root: guix-1.1.0-28.ff43f12-checkout)
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;   or pass the --no-auto-compile argument to disable.
;;; compiling /master/./build-aux/update-guix-package.scm
;;; compiled 
/root/.cache/guile/ccache/3.0-LE-8-4.3/master/build-aux/update-guix-package.scm.go
;;; compiling /master/gnu/packages/package-management.scm
;;; compiled 
/root/.cache/guile/ccache/3.0-LE-8-4.3/master/gnu/packages/package-management.scm.go
Backtrace:
In ice-9/boot-9.scm:
  3223:13 19 (_)
In ice-9/threads.scm:
390:8 18 (_ _)
In ice-9/boot-9.scm:
  3507:20 17 (_)
   2806:4 16 (save-module-excursion _)
  3527:26 15 (_)
In unknown file:
  14 (primitive-load-path "guix/store" #)
In guix/store.scm:
 23:0 13 (_)
In ice-9/boot-9.scm:
   3380:4 12 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  3393:24 11 (_)
   222:29 10 (map1 (((guix utils)) ((guix config)) ((guix #)) ((?)) ?))
   222:29  9 (map1 (((guix config)) ((guix deprecation)) ((guix ?)) ?))
   222:29  8 (map1 (((guix deprecation)) ((guix memoization)) ((?)) ?))
   222:29  7 (map1 (((guix memoization)) ((guix serialization)) (#) ?))
   222:29  6 (map1 (((guix serialization)) ((guix monads)) ((# #)) ?))
   222:29  5 (map1 (((guix monads)) ((guix records)) ((guix #)) (#) ?))
   222:29  4 (map1 (((guix records)) ((guix base16)) ((guix #)) (#) ?))
   222:29  3 (map1 (((guix base16)) ((guix base32)) ((gcrypt #)) # ?))
   222:29  2 (map1 (((guix base32)) ((gcrypt hash)) ((guix #)) (#) ?))
   222:17  1 (map1 (((gcrypt hash)) ((guix profiling)) ((rnrs #)) # ?))
   3300:6  0 (resolve-interface (gcrypt hash) #:select _ #:hide _ # _ ?)

ice-9/boot-9.scm:3300:6: In procedure resolve-interface:
no code for module (gcrypt hash).

What now?

Am I doing it wrong?


pgplakbT9rMEL.pgp
Description: OpenPGP digital signature

Re: Releasing guix binary in Docker format too?

[Danny Milosavljevic]

Also, while doing that, using the guix binary 1.1.0 tarball from the website and
issuing guix pull (ONLY), a lot of weird stuff is updated, like libx11, fribidi,
graphviz, cairo, pixman, libjpeg-turbo, pango etc.  Is that really necessary?
I guess it's because of the profile hooks, but still... why.

I want to stress that the profile I'm building contains only guix, the package
manager.  Does that really need all those (presumably build-) dependencies?


pgpkWeUt8KUbp.pgp
Description: OpenPGP digital signature

Releasing guix binary in Docker format too?

[Danny Milosavljevic]

Hello,

I'm working with Heads on using Guix to build Heads' source code reproducibly.

Now, Heads has most of its stuff on CI servers that use Docker for a lot of
things.

Therefore, I used Guix pack in the usual way to create a recent Guix
binary release and then just loaded that into a docker container, using that to
run "guix build".

That got me thinking--this situation is pretty common.  Do you think it would
make sense to actually build and release a Docker container for Guix (not
guix system; just guix-the-package-manager)?

While I was researching gitlab, I came across a feature where they have their
own Docker container registry on there.  Apparently, the Docker CLI can fetch
from whatever server the user wants!

$ docker run [options] registry.gitlab.com/group/project/image [arguments]

That got me thinking, we could easily also release Guix on something like
that.  Is our HTTP webserver enough to have a Docker registry, without
installing extra stuff?

$ docker run localhost/foo
Unable to find image 'localhost/foo:latest' locally
docker: Error response from daemon: Get "http://localhost/v2/": dial tcp 
127.0.0.1:80: connect: connection refused.

Aha!

Do we want to do it?


pgpd8qGE7zuHC.pgp
Description: OpenPGP digital signature

Re: guile-sqlite3 trace support

[Danny Milosavljevic]

Hi Mathieu,

thanks for telling me.

For some reason I do not get notification E-Mails when someone posts a pull
request there.  I've searched the preferences, but nothing about that anywhere.
My E-Mail address IS set up there (this one, but with +a suffix).  I've now
changed it not to use a plus slgn now.

In any case, I've merged your patch to master.

Let's see how it goes and then I'll make a new release.


pgpF7bHyw6AL6.pgp
Description: OpenPGP digital signature

Spooky problem in "nfs-root-os" system test

[Danny Milosavljevic]

Hi,

I've got some spooky problem in the NFS root test (or in the VM setup by Guix,
really).

It seems to hang the system test in a part I cannot change (so it's not inside
the test script).

In order to reproduce, start with guix master commit
898fbb60b2354e82e5b7f259b44dbfed112a83aa--or any recent Guix master commit.

So far it's still "fine"-ish:

$ make TESTS="nfs-root-fs" check-system
[... works fine and eventually fails because port 20001 is waited for but is
never opened by the server--so far so good]

Now edit gnu/tests/nfs.scm to change

 ;(rpcmountd-port 20001)

to

 (rpcmountd-port 20001)

in order to fix the problem above--or so I thought.

$ make TESTS="nfs-root-fs" check-system
[... hangs in first VM--without even trying to start up the NFS server VM (as
far as I can tell)]

What is going on?  Also, how to debug this?

(same happens when enabling (rpcstatd-port 20002) only.
Also, doesn't rpcstatd have TWO ports--one to broadcast and one to listen?
How does one set up both using the operating-system configuration?)



pgpN7m4WqAWeA.pgp
Description: OpenPGP digital signature

Re: linux-module-builder leads to huge store items

[Danny Milosavljevic]

>Could you give it a try?

... I did.  See footnote 2.

[2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=43332


pgpTAWG_ImLeN.pgp
Description: OpenPGP digital signature

Re: linux-module-builder leads to huge store items

[Danny Milosavljevic]

Hi Ludo,

On Thu, 10 Sep 2020 18:26:57 +0200
Ludovic Courtès  wrote:

> > 150 MB 
> > a2fs24bgghjvlzq5lzr6zki7mqxx8mpi-linux-libre-module-builder-5.8.7/lib/modules/build/drivers/gpu/drm/amd/include/asic_reg
> > 30 MB 
> > a2fs24bgghjvlzq5lzr6zki7mqxx8mpi-linux-libre-module-builder-5.8.7/lib/modules/build/drivers/staging
> >
> > I don't think we need these in any potential Linux kernel module--but who
> > knows?  Maybe somebody does need those.  
> 
> So, I can think of the following ways to address that:
> 
>   1. Remove for example *.c, assuming the out-of-tree modules we package
>  only care about *.h + build machinery.

The biggest source files, see above, are header files
(drivers/gpu/drm/amd/include/asic_reg).

>   2. Don’t add all of the Linux source to the store, or at least not
>  under a new name each time.  One way to do that would be to unpack
>  the Linux-libre tree within the build tree of the out-of-tree
>  module.  We’d be reducing GC stress at the expense of a
>  (reasonable?) increase in module build time.

And patch it correctly etc just like make-linux-libre does?  That doesn't
sound safe.  The current setup is basically inheriting from the package
make-linux-libre gave us and changing the 'build phase slightly.  That means
it will be exactly the same otherwise.

The only way to reduce effort safely is to delete the set of source
files (especially header files) which we don't need for out-of-tree
modules.

I checked--that would reduce the size of linux-libre-builder by about 800 MB
(to 255 MB; see [1][2]).
That would mean that some modules can't be out-of-tree.  But I guess
that's okay--and we could not-delete some files later if need be.

Btw, linux-libre is currently not reproducible--see bug 43301.
That could introduce a security problem if the Linux kernel and
an out-of-tree module are compiled differently (they are) and then loaded
into the same kernel (even now).

[1] Toplevel files in linux-libre-module-builder would be then (sizes are 
included if size > 1 MB):

arch/   (134 MB)
block/  (2 MB)
certs/
COPYING
CREDITS
crypto/ (4 MB)
include/ (59 MB)
init/
ipc/
Kbuild
Kconfig
kernel/ (11 MB)
lib/ (6 MB)
LICENSES/
MAINTAINERS
Makefile
mm/ (5 MB)
Module.symvers (2 MB)
README
scripts/ (6 MB)
security/ (4 MB)
System.map (5 MB)
tools/ (14 MB)
usr/
virt/

[2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=43332


pgp_1_ch8NAmZ.pgp
Description: OpenPGP digital signature

Re: linux-module-builder leads to huge store items

[Danny Milosavljevic]

Hi Ludo,

On Thu, 10 Sep 2020 10:09:11 +0200
Ludovic Courtès  wrote:

> Surely we don’t need that much just to build a kernel module.  :-)

Since Linux is a monolithic kernel and since C has no modules: Yes, yes we
do--in the general case.

Linux kernel developers have no interest in supporting out-of-tree modules for
free (and no interesting in having the Linux kernel API frozen) so there's no
incentive to support a use case where you can compile a Linux kernel module
without having the full source of the Linux kernel (that would be just asking
for license abuse, too).

That would mean that we would have to both design and then support out-of-tree
build infrastructure ourselves in Guix.  And for what?  It's only used when
building the module--it will be thrown away afterwards anyway.

Or is it not garbage-collected afterwards?  That would be a serious bug.

>   ;; TODO: Only preserve the minimum, i.e. [Kbuild], Kconfig,
>   ;; scripts, include, ".config".
> 
> Which files exactly are needed?  Can we go ahead and implement this TODO?

Who knows?  Nobody--not even Linux kernel developers.  Given the design of
C, it's not possible in general to know what is the interface and what is
the implementation.  I guess for the Linux kernel we could kinda wing it
because of the social conventions the Linux kernel enforces where it kinda
almost does have a module separation into interface and implementation.

> I understand the need to “cut corners” while trying things out, but I’d
> argue that this specific corner should have been restored before getting
> into master.

If you want that, I think you need to argue that with the main Linux kernel
maintainers (Greg Kroah-Hartmann etc).  I don't think it is
practical for us to maintain a weird minimal module-building-only fork
of Linux.  I'd be glad to be proven wrong, though.

Even if we did, there are much worse build-time storage space wastes
(*cough* llvm *cough*).

That said, I did a quick size comparison and found those big items:

150 MB 
a2fs24bgghjvlzq5lzr6zki7mqxx8mpi-linux-libre-module-builder-5.8.7/lib/modules/build/drivers/gpu/drm/amd/include/asic_reg
30 MB 
a2fs24bgghjvlzq5lzr6zki7mqxx8mpi-linux-libre-module-builder-5.8.7/lib/modules/build/drivers/staging

I don't think we need these in any potential Linux kernel module--but who
knows?  Maybe somebody does need those.


pgp9bE4buqZmN.pgp
Description: OpenPGP digital signature

Re: Linux-libre git repository

[Danny Milosavljevic]

Hi Vagrant,

On Thu, 13 Aug 2020 09:47:21 -0700
Vagrant Cascadian  wrote:

> The source checkout was quite slow to download, and took up ~1GB in the
> store once completed. I'm not sure how guix's git origin works exactly;

git init
git remote add origin 
if git fetch --depth 1 origin 
then
  git checkout FETCH_HEAD
else
  echo "Failed to do a shallow fetch; retrying a full fetch..."
  git fetch origin
  git checkout 
fi
if ,recursive?
then
  git submodule update --init --recursive
  rm -rf .git for each submodule
fi
rm -rf .git

See guix/build/git.scm .

There exist git servers that have disabled fetching by commit hash for
"security" reasons (if you checked in a file containing a password and
then removed it again, and no branch or tag to it exists, nobody can
get to it even if he knew the commit hash).  We would always use the
fallback for those servers.

> if it downloads the entire git history even to perform a shallow
> checkout of a single commit, and then throws out the git history?

As a fallback if the above doesn't work.

> appear to be calling git with flags to perform a shallow checkout.

Yes.


pgpTiWBUhDOdD.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

pushed that patchset to wip-desktop.

I've changed liblouis output "louis" to "python" to do it the same way as other
packages do it.

Thanks!


pgpUTTccoZDv_.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi Raghav,

applied these to wip-desktop except for 0030-gnu-seed-Disable-tests.patch
(I still want to look into it) and
0026-gnu-flatpak-Update-package-definition.patch (there's a newer flatpak
in the master I rebased to).

Had to adapt the libostree patch slightly because half was already applied
on the master I rebased to.


pgpmBeacS57pF.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

pushed those to wip-desktop with small changes to the commit message.


pgpz3Y_typ_Q5.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

pushed this patchset to wip-desktop with small changes.

I have the following comments/point out the following problems:

* gtksourceview depends on icedtea (Java) (because it has groovy as 
native-input),
which is not reproducible and only works on x86_64.  Please make it not depend 
on
icedtea.
  * So does tepl, because it depends on gtksourceview
* libsecret fails two tests

libsecret test failure:
 9/12 test-item   FAIL 0.87 s (killed by signal 
11 SIGSEGV)
# GLib-DEBUG: posix_spawn avoided (workdir specified) (fd close requested) 
(child_setup specified) 
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used 
after threads are created
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after 
threads are created
[...]

 5/12 test-sessionFAIL 0.34 s (killed by signal 
5 SIGTRAP)
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used 
after threads are created
Bail out! GLib-FATAL-CRITICAL: g_source_ref: assertion 'g_atomic_int_get 
(&source->ref_count) >= 0' failed
--- stderr ---

(test-session:406): GLib-CRITICAL **: 22:02:56.436: g_source_ref: assertion 
'g_atomic_int_get (&source->ref_count) >= 0' failed

Both of those tests work fine outside nix build container and outside pure guix 
environment, while logged into X.

  * memphis doesn't build because of that
  * network-manager doesn't build because of that
  * gtksourceview doesn't build because of that
* gjs is not reproducible

gjs diffoscope:
├── bin
│ ├── .gjs-console-real
│ │ ├── readelf --wide --decompress --hex-dump=.data {}
│ │ │ @@ -51,15 +51,15 @@
│ │ │0x0040a540   94ab4000  ..@.
│ │ │0x0040a550 be714000    .q@.
│ │ │0x0040a560     
│ │ │0x0040a570     
│ │ │0x0040a580     
│ │ │0x0040a590     
│ │ │0x0040a5a0 2a353741    *57A
│ │ │ -  0x0040a5b0 f3a04bb6  30754000  ..K.0u@.
│ │ │ +  0x0040a5b0 48ed49b6  30754000  H.I.0u@.
│ │ │0x0040a5c0 00474000    .G@.
│ │ │0x0040a5d0     
│ │ │0x0040a5e0     
│ │ │0x0040a5f0     
│ │ │0x0040a600   0500  
│ │ │0x0040a610 60a24000    `.@.
│ │ │0x0040a620 2a353741    *57A
[...]
├── lib
│ ├── libgjs.so.0.0.0
│ │ ├── readelf --wide --decompress --hex-dump=.data {}
│ │ │ @@ -2687,15 +2687,15 @@
│ │ │0x0049e4c0 a0223a00    .":.
│ │ │0x0049e4d0     
│ │ │0x0049e4e0 40203a00  0c00  @ :.
│ │ │0x0049e4f0 4000  2cca3a00  @...,.:.
│ │ │0x0049e500 a0223a00    .":.
│ │ │0x0049e510     
│ │ │0x0049e520 2a353741    *57A
│ │ │ -  0x0049e530 e9a54bb6  00413a00  ..K..A:.
│ │ │ +  0x0049e530 00f249b6  00413a00  ..I..A:.

  * network-manager doesn't build reproducibly because of that, and doesn't 
build at all because of the libsecret test failures
* network-manager-applet doesn't build reproducibly because of that
  * tracker doesn't build reproducibly because of that
* tracker-miners doesn't build reproducibly because of that

Pushed your patches to wip-desktop anyway.

Please fix libsecret.

The other things are nice-to-have for now, but should also be fixed
eventually.

Did I apply all the patches you sent me or are some still pending?


pgpBKsAXld0Ys.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

applied the patches to wip-desktop, with libsigc++ and glibmm replaced by your
newer patches.

I'm still not sure about the propagated-inputs but I left them as-is for the
time being.


pgpVVtnO4xMh2.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

On Sat, 18 Jul 2020 16:18:53 -0400
Raghav Gururajan  wrote:

> Please find the revised patched for libsigc++ and glibmm, attached with
> this email.

Applied to wip-desktop


pgpY2zXGHscS7.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

On Sat, 18 Jul 2020 16:18:53 -0400
Raghav Gururajan  wrote:

> >> Also, please do not make libsigc++ public twice.  It's already in the
> >> module's #:export list.
> >>
> >> Same for glibmm and maybe others.  
> > 
> > Hmm. Without that, I get "package not found" while doing `./pre-inst-env
> > guix build foo`.  
> 
> It was for `./etc/indent-code.el`, not for `./pre-inst-env guix build`.

Whoops?  Please file a bug report at bug-g...@gnu.org


pgpzOAKQ3cKBV.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

since cairo is meant to be a portable and backend-agnostic vector graphics
library, I find it hard to believe it needs to propagate fontconfig, freetype,
glib, libpng, pixman, x11, xcb, xext and xrender.

Are you sure that cairo actually exposes those libraries' interfaces to its
users?

The cairo.pc file also suggests otherwise:

Requires.private:   gobject-2.0 glib-2.0 >= 2.14   pixman-1 >= 0.30.0
fontconfig >= 2.2.95 freetype2 >= 9.7.3   libpng xcb-shm xcb >= 1.6 xcb-render 
>= 1.6 xrender >= 0.6 x11 xext 
Libs: -L${libdir} -lcairo
Libs.private:-lz -lz   -lz  

"Requires.private" means that those are implementation details and not to be
exposed to the users of the library.

Also, please do not make libsigc++ public twice.  It's already in the
module's #:export list.

Same for glibmm and maybe others.

Pushed the pixman and hplip patches to wip-desktop.


pgpVmbjMtJeJ3.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Pushed to wip-desktop.

Comments:

* Why is libcamera in (gnu packages networking)?  It sounds like it has digital 
camera libraries.
* libcamera:doc is not reproducible.
* building zentiy has webkitgtk as dependencies.


pgpEy7dn9YaPc.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

When trying to build gtk+ I get:

/tmp/guix-build-libsoup-2.70.0.drv-0 
--- stderr ---
**
ERROR:../libsoup-2.70.0/tests/ssl-test.c:394:do_tls_interaction_test: 
Unexpected status 7 Connection terminated unexpectedly (expected 6 SSL 
handshake failed)
---

Full log written to 
/tmp/guix-build-libsoup-2.70.0.drv-0/build/meson-logs/testlog.txt
FAILED: meson-test 
/gnu/store/9m03qg3zrcfr3sxmr5592vkaa4p33sci-meson-for-build-0.53.2/bin/meson 
test --no-rebuild --print-errorlogs
ninja: build stopped: subcommand failed.
command "ninja" "test" failed with status 1

gtk+ had a problem:

TEST: tree-relationships... (pid=9358)
  /a11y/tree/focus:
(/tmp/guix-build-gtk+-3.24.20.drv-0/gtk+-3.24.20/testsuite/a11y/.libs/tree-relationships:9358):
 Gtk-WARNING **: 00:15:13.927: Could not find the icon 'pan-down-symbolic'. The 
'hicolor' theme
was not found either, perhaps you need to install it.
You can get a copy from:
http://icon-theme.freedesktop.org/releases
FAIL
GTester: last random seed: R02S4bb1d91fca6f2b72aa71362d1367bf22
(pid=9372)
  /a11y/tree/relations:OK
FAIL: tree-relationships

That's because hicolor-icon-theme was not found.  Adding it to native-inputs 
helped, so I did.

Pushed to wip-desktop with slight changes to mend these problems.

Would it be possible to remove the dependency on Java?
Java is not reproducible right now which makes building gtksourceview (any 
version) difficult for me.


pgp0SOdZRJz4K.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

I've applied most to wip-desktop, with the following changes/comments:

nanomsg: not reproducible
nng: not reproducible
libmpeg3 lots of "undeclared" warnings
libtimidity test failed "Could not initialise libTiMidity" in playmidi

directfb not reproducible
│ │ │ -DirectFB/Core: %s Application Core. (2020-07-05 00:00) %s%s
│ │ │ +DirectFB/Core: %s Application Core. (2020-07-04 23:57) %s%s

libde265 misses dependency to libvideogfx

mjpegtools:

>From 
>https://sourceforge.net/projects/mjpegtools/files/mjpegtools/2.1.0/mjpegtools-2.1.0.tar.gz...
>download failed 
>"https://sourceforge.net/projects/mjpegtools/files/mjpegtools/2.1.0/mjpegtools-2.1.0.tar.gz";
> 404 "Not Found"
Fallback works, though.

libopenmpt: It is not required to propagate mpg123 or vorbis (see .pc file).

srt: not reproducible; massive diff

flite: Applied with the following changes

diff --git a/gnu/packages/speech.scm b/gnu/packages/speech.scm
index 0968b015be..c7fbb986ac 100644
--- a/gnu/packages/speech.scm
+++ b/gnu/packages/speech.scm
@@ -66,7 +66,7 @@
 (arguments
  ;; XXX:
  ;; There numerous issues with the testsuite.
- ;; Enable them once they are fixed in upstream.
+ ;; Enable all of them once they are fixed in upstream.
  `(#:tests? #f
#:configure-flags
(list
@@ -76,25 +76,35 @@
"/lib"))
#:phases
(modify-phases %standard-phases
+ (add-after 'unpack 'patch-rpath
+   (lambda _
+ (substitute* "main/Makefile"
+  (("flite_LIBS_flags \\+= -Wl,-rpath [^ ]*")
+   "flite_LIBS_flags +="))
+ #t))
+ (delete 'check)
+ ;; Modifying testsuite/Makefile is not done in favor of
+ ;; overriding 'check.
+ ;; The path not taken would be:
+ ;; test:\n\t$(foreach x,$(subst tris1,,$(subst dcoffset_wave,,$(subst 
flite_test,,$(subst by_word,,$(subst bin2ascii,,$(subst lpc_resynth,,$(subst 
rfc,,$(subst compare_wave,,$(subst record_in_noise,,$(subst 
combine_waves,,$(patsubst play_%,,$(subst record_wave,,$(subst 
lex_lookup,,$(patsubst lpc_test%,,$(patsubst asciiS2U%,,$(patsubst 
asciiU2S%,,$(ALL),echo TEST $x && ./$x data.one && ) true
+ (add-after 'install 'check
+   (lambda _
+ (invoke "make" "-C" "testsuite")
+ (with-directory-excursion "testsuite"
+   (invoke "./token_test")
+   (invoke "./hrg_test")
+   (invoke "./regex_test")
+   (invoke "./nums_test")
+   (invoke "./lex_test")
+   (invoke "./utt_test")
+   (invoke "./multi_thread"))
+ #t))
  (add-after 'install 'remove-static-libs
(lambda* (#:key outputs #:allow-other-keys)
  (let* ((out (assoc-ref outputs "out")))
(for-each delete-file
- (list
-  (string-append out "/lib/libflite.a")
-  (string-append out 
"/lib/libflite_cmu_grapheme_lang.a")
-  (string-append out 
"/lib/libflite_cmu_grapheme_lex.a")
-  (string-append out "/lib/libflite_cmu_indic_lang.a")
-  (string-append out "/lib/libflite_cmu_indic_lex.a")
-  (string-append out "/lib/libflite_cmulex.a")
-  (string-append out "/lib/libflite_cmu_time_awb.a")
-  (string-append out "/lib/libflite_cmu_us_awb.a")
-  (string-append out "/lib/libflite_cmu_us_kal16.a")
-  (string-append out "/lib/libflite_cmu_us_kal.a")
-  (string-append out "/lib/libflite_cmu_us_rms.a")
-  (string-append out "/lib/libflite_cmu_us_slt.a")
-  (string-append out "/lib/libflite_usenglish.a"
- #t)
+ (find-files out "\\.a$"))
+   #t))
 (native-inputs
  `(("perl" ,perl)))
 (inputs

zxing-cpp: Applied with the following changes

diff --git a/gnu/packages/aidc.scm b/gnu/packages/aidc.scm
index 4921ae03a1..2c3df8f42b 100644
--- a/gnu/packages/aidc.scm
+++ b/gnu/packages/aidc.scm
@@ -28,6 +28,7 @@
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (gnu packages autotools)
+  #:use-module (gnu packages check)
   #:use-module (gnu packages imagemagick)
   #:use-module (gnu packages glib)
   #:use-module (gnu packages gtk)
@@ -54,9 +55,31 @@
(file-name (git-file-name name version))
(sha256
 (base32 "011sq8wcjfxbnd8sj6bf2fgkamlp8gj6q835g61c952npvwsnl71"
+(native-inputs
+ `(("googletest-source" ,(package-source googletest
 (build-system cmake-build-system)
 (arguments
- `(#:tests? #f)); No target
+ `(#:out-of-source? #f
+   #:phases
+   (modify-phases %standard-phases
+ (add-after 'u

Re: [GSOC 2020] network-boot-service

[Danny Milosavljevic]

Hi Brice,

On Thu, 02 Jul 2020 10:11:28 +
Brice Waegeneire  wrote:
> My issue building a static 'nfs-utils' is that it can't find
> 'getrpcbynumber{,_r}' “configure: error: Neither getrpcbynumber_r nor
> getrpcbynumber are available”. This function should be provided by the 
> libc
> or by libtirpc if it's not that first one. The problem is that 
> 'libtirpc'
> don't build it's own 'getrpcbynumber' because it find one in libc but
> nfs-utils can't find it...

Since glibc:static doesn't seem to contain it
(/gnu/store/rj4il3jjqg23cm3a0h17yzq3b5wa8llm-glibc-2.31-static/lib$
 objdump -t *.a |grep -i getrpcby returns nothing), we could make our
own variant of libtirpc that does enable their version.

> You can find the part of my work which is committed in the
> 'wip-network-boot' at https://git.sr.ht/~bricewge/guix.

Thanks!

I've checked it out, looks good so far!


pgpjOQbXaIJzM.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Please, when you remove patch references from (gnu packages ...), also remove
the actual patch files using "git rm -f" and also remove the reference to
the actual patch files from gnu/local.mk .

Likewise, when you are adding patch references to (gnu packages ...), also
add the reference to the actual patch files to gnu/local.mk .



pgptAMd6d11q5.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

On Thu, 25 Jun 2020 00:04:50 -0400
Raghav Gururajan  wrote:

> Please find the attached patches.

Thanks.

> > * gstreamer

Please add a comment on why this is here:

>+ (search-paths native-search-paths)

> > * libvisual

>+ (search-paths native-search-paths)

Same

> > * libvisual-plugins

>+ (search-paths native-search-paths)

Same

> > * gst-plugins-base

I didn't apply it because I didn't apply gstreamer.

> > * gst-plugins-good  

I didn't apply it because I didn't apply gstreamer.

I've applied the rest to wip-desktop.


pgpeJc6bbQGUn.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

* In ruby-slim: Why are tests disabled?  There are tests.
* In libmpeg3: Could you unbundle a52dec-0.7.3?  It's bundled right now.
* In flux, a git commit is used.  We usually don't package non-releases.
  Is there a good reason?

* 0019-gnu-Add-asciidoctor.patch does some other stuff to other packages, and 
that
fact isn't mentioned in the commit log in the patch.  Why not?

* If nng is compatible with nanomsg, why do we need the "nanomsg" package at 
all?

* In tesseract-ocr you propagate leptonica.  Why?
  If we did that, users that would install tesseract for the "tesseract" binary
  would get leptonica in their profile.
  Is it necessary?

* ccextractor still has a XXX.  Does it work?

* In libtimidity, what's the problem with libtimidity using its own "play"
binary?  I would expect the package to test itself like that...

* In faac, what do the comments "Newer code" and "Older code" mean?

* In flite, there are tests, but they fail.

* In libopenmpt, why does it propagate zlib?  That sounds wrong.

For ruby-slim there IS a "test" subdirectory--so why are they disabled?

For the time being I've not applied any of those patches since it's difficult
to leave some off and still be able to apply the others.

I have applied most of the patches in your patchset of your previous E-Mail.


pgp6VvTk0ERYh.pgp
Description: OpenPGP digital signature

Re: [OUTREACHY]: Integration of desktop environments into GNU Guix

[Danny Milosavljevic]

Hi RG,

I've changed guix wip-desktop to use guix master
commit c03cefc3d8dd826c4a74abbfe2b235525dca50fb as base (made a rebase).

Then I tested and applied the patches from your E-Mail this is an answer to.

That means a few of your patches didn't apply--which is not your fault.

Could you please clone wip-desktop anew and rebase the patches that didn't
apply on top manually and then re-send those?

The ones that didn't apply are:

glib: Patch doesn't apply.  Also: Why define-public now?
glib-networking 2.64.3 doesn't apply because of missing glib (because above 
didn't apply).
gobject-introspection: Patch didn't apply.
gstreamer: Patch didn't apply.
libvisual: Patch didn't apply.
libvisual-plugins: Didn't try, dependency now missing.
gst-plugins-base: Patch didn't apply.
gst-plugins-good: Didn't try, dependency now missing.

Some comments on the ones that did apply:

at-spi2-atk:

  * I've not applied the move of dbus to regular inputs.
Was it accidential or does it actually use dbus after it is installed?

cogl:

  I've had some merging trouble, so just FYI:

  * Phase 'fix-build-with-mesa-20 is now gone
  * Phase 'patch-docbook-xml is now new
  * arguments were in another spot than before--which confused git very much

geoclue:

  I've had some merging trouble, so did the following:

  * Kept geoclue-config.patch as before. Should I delete it for real?

esound:

  * tcp-wrappers' libwrap is not referenced for some reason.
  * Moved tcp-wrappers to regular inputs.

Please thus resend patches on top of wip-desktop for

* glib
* glib-networking 2.64.3 (which is the second update)
* gobject-introspection
* gstreamer
* libvisual
* libvisual-plugins
* gst-plugins-base
* gst-plugins-good


pgpzj794HQFPq.pgp
Description: OpenPGP digital signature

Re: Blog post on Further reduced bootstrap seed to 25%

[Danny Milosavljevic]

Hi Janneke,

On Mon, 15 Jun 2020 14:54:39 +0200
Jan Nieuwenhuizen  wrote:

> I’ve published a post about the second big reduction of the Guix
> bootstrap binaries
> 
> https://guix.gnu.org/blog/2020/guix-further-reduces-bootstrap-seed-to-25/

"again decided to sponsor this work" link is broken

Otherwise good :)


pgpF6DS0rJec8.pgp
Description: OpenPGP digital signature