Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Andreas Enge]

Hello,

On Thu, Jul 05, 2018 at 11:04:31AM +0200, Jonathan Brielmaier wrote:
> I just want to bring POWER up as a freedom-respecting architecture.
> Especially the TalosII from RaptorCS[0]. I know that guix does not work
> on ppc64le yet, but I'm working for it :) They tend to be quite
> expensive, but you get a decent performance on compiling and packing[1].

this is indeed an exciting architecture. Vikings had a machine on display
at their FOSDEM table earlier this year. But the price is still quite
steep, if you know anybody who would sponsor a machine...

Andreas

[PATCH] gnu: gzochi: Update to 0.12

[Julian Graham]

Hi Guix folk,

Find attached a patch that bumps the version of gzochi, a game
development framework I maintain. I've done a `guix build' test run
for amd64. Let me know if anything looks untoward.


Regards,
Julian
From d09752ea47bea9ef0a769e4356ffef5cf08242f8 Mon Sep 17 00:00:00 2001
From: Julian Graham 
Date: Thu, 5 Jul 2018 11:06:02 -0400
Subject: [PATCH] gnu: gzochi: Update to 0.12.

* gnu/packages/game-development.scm (gzochi): Update to 0.12.
---
 gnu/packages/game-development.scm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/game-development.scm b/gnu/packages/game-development.scm
index 96b15b565..d466384eb 100644
--- a/gnu/packages/game-development.scm
+++ b/gnu/packages/game-development.scm
@@ -7,7 +7,7 @@
 ;;; Copyright © 2016, 2017, 2018 Efraim Flashner 
 ;;; Copyright © 2016, 2017 Kei Kebreau 
 ;;; Copyright © 2016, 2018 Ricardo Wurmus 
-;;; Copyright © 2016, 2017 Julian Graham 
+;;; Copyright © 2016, 2017, 2018 Julian Graham 
 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice 
 ;;; Copyright © 2017 Manolis Fragkiskos Ragkousis 
 ;;; Copyright © 2017 Peter Mikkelsen 
@@ -225,14 +225,14 @@ PCM data.")
 (define-public gzochi
   (package
 (name "gzochi")
-(version "0.11.1")
+(version "0.12")
 (source (origin
   (method url-fetch)
   (uri (string-append "mirror://savannah/gzochi/gzochi-"
   version ".tar.gz"))
   (sha256
(base32
-"13j1m92zhxwkaaja3lg5x0h0b28mrrawdzk9d3hd19031akfxwb3"
+"0h8yvk7154kd8zdfa9nqy73blrjq2x19kv305jcnwlmm09vvss59"
 (build-system gnu-build-system)
 (arguments
  '(#:phases (modify-phases %standard-phases
-- 
2.11.0

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Kei Kebreau]

l...@gnu.org (Ludovic Courtès) writes:

> Hello Kei,
>
> Kei Kebreau  skribis:
>
>> I am interested in helping with non-x86_64 issues. Particularly, helping
>> with i686-related changes should be just a change in workflow, but I'm
>> interested in obtaining freedom-respecting non-x86 hardware (or at least
>> using a virtual machine as close as possible to real hardware
>> configurations). Any recommendation or links for where I can get a
>> Yeeloong laptop or what freedom-respecting armhf computers are
>> available?
>
> Without having actual hardware, you can use the qemu-binfmt service on
> your machine (info "(guix) Virtualization Services").  It’s slow, but it
> allows you to reproduce and debug issues for other architectures.
>
> Thanks,
> Ludo’.

Wow, this is excellent! Thanks for the tip.


signature.asc
Description: PGP signature

Re: shortening the git test suite

[Ricardo Wurmus]

Gábor Boskovits  writes:

> Mark H Weaver  ezt írta (időpont: 2018. júl. 2., H, 6:58):
>
>> Ricardo Wurmus  writes:
>>
>> > git takes a very long time to build, because it has an extensive test
>> > suite.  Most of the time is spent in running the SVN interoperability
>> > tests, though, which are not really all that interesting for most uses
>> > of git.
>> >
>> > The Makefile says this:
>> >
>> >   # Define NO_SVN_TESTS if you want to skip time-consuming SVN
>> interoperability
>> >   # tests.  These tests take up a significant amount of the total test
>> time
>> >   # but are not needed unless you plan to talk to SVN repos.
>> >
>> > What do you think about disabling the SVN tests in the git package?
>>
>> My preference would be to keep the SVN tests enabled, but it's not
>> something I feel very strongly about.
>>
>>   Thanks,
>> Mark
>>
>>
> Do we have a way to make running this SVN test suite optional? For example
> to run it only before release, or on demand?

No, and I would think that a mechanism to execute them optionally would
necessarily be a hack that undermines some of the guarantees that Guix
tries hard to provide.

--
Ricardo

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Andreas Enge]

Hello all,

of course I can all but agree that support for "exotic" hardware is very
desirable, especially since, as Mark pointed out, we would like it to become
more mainstream!

On Thu, Jul 05, 2018 at 08:38:19AM +0200, Ricardo Wurmus wrote:
> One thing that would help, in my opinion, is to purchase hardware and
> make it available to interested developers and/or join these new
> machines to the build farm.

If people want to look at armhf, one of the donated Novena boards is currently
running in my living room, under the name of redhill.guixsd.org. I could
of course create accounts for Guix developers who want to have access to
debug the architecture.

Andreas

Re: shortening the git test suite

[Ricardo Wurmus]

Marius Bakke  writes:

> When debugging the git build, I usually set #:parallel-tests? #t.  So
> far I haven't come across a failure related to that.  I wonder if
>  is still relevant.

That’s a good question.  I don’t know.

-- 
Ricardo

Re: shortening the git test suite

[Ricardo Wurmus]

Chris Marusich  writes:

> Ricardo Wurmus  writes:
>
>> Hi Guix,
>>
>> git takes a very long time to build, because it has an extensive test
>> suite.  Most of the time is spent in running the SVN interoperability
>> tests, though, which are not really all that interesting for most uses
>> of git.
>>
>> The Makefile says this:
>>
>>   # Define NO_SVN_TESTS if you want to skip time-consuming SVN 
>> interoperability
>>   # tests.  These tests take up a significant amount of the total test time
>>   # but are not needed unless you plan to talk to SVN repos.
>>
>> What do you think about disabling the SVN tests in the git package?
>
> This sounds similar to the discussion we had earlier about treating
> tests as a special case:
>
> https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00071.html
>
> I felt that the conclusion of that thread was basically that if someone
> is concerned about the build time, then they ought to be able to use
> substitutes to speed things up, and we should continue to run as many
> tests as possible in order to discover problems sooner.

What I’m worried about is availability of substitutes.  When we keep
changing packages on the “master” branch that have very expensive test
suites then we accept that people won’t have substitutes for a while.
The duration of that while depends on how quickly the build of this
package is started by our continuous integration software, and how long
it takes to complete the build.

Granted, disabling parts of the test suite in an attempt to shorten it
is really a technical fix to a social problem.

Personally, I think that the SVN tests are non-essential (after all,
we’re building Git here, we keep running the individual test suites of
Git and Subversion, and git-svn interop seems like a thing that only
upstream need to worry about), which is why I made this proposal.  But
as it seems that the people who responded to this message rather lean in
the other direction, let’s try to address the social problem instead:

How can we change our workflow to make sure that for packages with long
build/test times we can provide substitutes more quickly?  Currently,
our policy for pushing changes to “staging” and “core-updates” is based
on package counts.  I’m not suggesting that changes to Git be pushed to
“staging”.

What do you think of pushing some package updates only to feature
branches that follow a certain naming convention (e.g. “_update-foo” for
updating the “foo” package), which causes Cuirass to build them and
merge the branch into “master” (semi-)automatically when the build is
successful?

(Obviously, any kind of automation has to be thought out carefully, but
I’m sure we would be able to find a solution.)

What do you think?

--
Ricardo

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Jonathan Brielmaier]

On 7/5/18 12:32 AM, Kei Kebreau wrote:
>> I'm open to suggestions.  Do you see any solution to the problem of how
>> to attract more non-x86_64 users, given our current policies?
>>
>>  Thanks,
>>Mark
> 
> I am interested in helping with non-x86_64 issues. Particularly, helping
> with i686-related changes should be just a change in workflow, but I'm
> interested in obtaining freedom-respecting non-x86 hardware (or at least
> using a virtual machine as close as possible to real hardware
> configurations). Any recommendation or links for where I can get a
> Yeeloong laptop or what freedom-respecting armhf computers are
> available?

I just want to bring POWER up as a freedom-respecting architecture.
Especially the TalosII from RaptorCS[0]. I know that guix does not work
on ppc64le yet, but I'm working for it :) They tend to be quite
expensive, but you get a decent performance on compiling and packing[1].

Regarding ARM hardware: I have access to a bunch of performant ARM
machines (Cavium Thunder, AMD ARM stuff etc.) at work. So feel free to
drop me a mail or set me to CC, if you need something to be tested on ARM :)

[0] https://raptorcs.com/
[1] https://www.phoronix.com/scan.php?page=article&item=power9-talos-2&num=3

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Ludovic Courtès]

Hi again Mark,

Mark H Weaver  skribis:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver  skribis:
>>
>>> The end result is that the wishes of the x86_64-using majority are the
>>> only ones that seem to matter in this community, and other users are
>>> frequently left in a bad spot.  This makes it increasingly unlikely that
>>> we'll ever gain a significant number of non-x86_64 users.
>>
>> This kind of rant is really unhelpful.  You’re shouting at someone who
>> *is* doing the work of keeping things running.
>
> I wasn't actually shouting, but in retrospect I can see how it came off
> that way.  I apologize for any hurt feelings that I caused.

I think the error is to suggest that people genuinely don’t care about
the issues.

Often they’re unaware, and sometimes they make suboptimal tradeoffs, as
in the PatchELF case, simply because the status quo is worse than the
suboptimal tradeoff.

> However, I do feel frustrated by the fact that it's considered
> acceptable in this community to leave non-x86_64 users with broken
> systems in the name of "moving things forward" for x86_64 users.

Like I write, it’s not “considered acceptable.”  That’s just not the way
it works.

There’s an implicit rule that we should not break any architecture
badly, but just like sometimes packages fail to build, sometimes there
are architecture-specific issues; and just like an unpopular package
that fails to build is likely to remain that way, an unpopular
architecture is more likely to have issues.

We don’t have to take it as a fact of life, though.  We can work
proactively to mitigate that, and support for those architectures in the
build farm, along with heads-up from overseers (like you’ve been doing
to great effect!) can greatly help.  It won’t bring, say, MIPS to the
level of support of x86_64, but it can reduce damage.

> I'm open to suggestions.  Do you see any solution to the problem of how
> to attract more non-x86_64 users, given our current policies?

Efraim, Danny, Vagrant, Julien, Mathieu, etc. have done a lot of work
fiddling with ARMv7 and AArch64.  We should encourage that, and
providing timely substitutes for the arches is one way to do it, and
ultimately to attract more users and contributors.

Thanks,
Ludo’.

Re: Chrooting into GuixSD

[Chris Marusich]

swedebugia  writes:

> On 2018-06-14 17:01, Ricardo Wurmus wrote:
>> Oleg Pykhalov  writes:
>>
 Would any of you like to document this in the manual?
>>> I would because I posted articles like the current one sometime ago.
>> Excellent!
>>
>>> Maybe a separate ‘users-stories.texi’ should be as was discussed
>>> somewhere on a mailing list (as I remember)?
>> I think that this particular section on tools that Guix provides to
>> recover a broken system (and the peculiarities of chrooting into a
>> GuixSD system) can go right into the manual.  This does not preclude the
>> option to have a more detailed step by step guide in a
>> “user-stories.texi”.
> +1
>
> Does GuixSD by the way support dropping to a root shell by passing
> "single" as a kernel parameter? I once used a distro that had a
> feature like that.
>
> Ubuntu supports dropping to a root shell (and memtesting RAM?) among
> more options if "recovery" is choosen in the grub menu. I like that
> feature. If something like that had been possible on this old
> GuixSD-installation I would not have had to resort to chrooting.

If you pass the "--repl" option on the Linux kernel command line (e.g.,
from the GRUB menu), I believe you can do this.  See "(guix) Initial RAM
Disk" in the manual for details.

-- 
Chris


signature.asc
Description: PGP signature

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Ludovic Courtès]

Mark H Weaver  skribis:

> Another problem is that Guile 2.2's compiler has become so heavy that
> it's nearly unbearable to use on slower hardware.  Simply running "make"
> in my Guix git checkout after updating on my mips-based Yeeloong is so
> slow that I'm in the habit of letting it run overnight.

This is a topic on its own, and I really think we can and should do
better.  I posted observations on where the compiler is spending its
time and memory earlier; Andy pushed improvements, but I believe there’s
much more than can be done:

  https://lists.gnu.org/archive/html/guile-devel/2017-10/msg00035.html
  https://lists.gnu.org/archive/html/guile-devel/2017-05/msg00033.html

Ludo’.

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Ludovic Courtès]

Hello,

Ricardo Wurmus  skribis:

>> However, I do feel frustrated by the fact that it's considered
>> acceptable in this community to leave non-x86_64 users with broken
>> systems in the name of "moving things forward" for x86_64 users.
>
> I don’t think this is true.

What is true is that most Guix users use x86_64 primarily.  But there’s
also a lot of interest in ARM.

Guix doesn’t exist in a vacuum, and I think the situation of non-x86_64
support in Guix is just as good or bad as in other free software
projects.  We have fewer packages available on non-x86_64 architectures,
but that’s in large part due to upstream packages not supporting those
architectures in the first place.

I agree this is sad, but repeating it doesn’t help address it.

> I do agree with your laments about a lack of popularity of non-x86_64
> systems and thus developers, but I do think this has been getting better
> with the work this community has done to support Guix for the aarch64
> and armhf architectures, and by adding aarch64/armhf build servers to
> the build farm.  We can and should do more of this, but it won’t happen
> by decree.

Agreed.

> One thing that would help, in my opinion, is to purchase hardware and
> make it available to interested developers and/or join these new
> machines to the build farm.  We would need to come to an agreement about
> at least these things:
>
>   * what exact system configurations do we want?
>   * where would these systems be hosted?
>   * how many do we need / can we afford to buy and pay hosting fees for?
>
> The last time this has come up the discussion kinda tapered out.  It
> would be good if someone or a group of people would volunteer to take
> this on and drive this project to its conclusion.

I agree!  If someone cares about ARM, for instance, now’s the time to
tell us what we should buy and to offer to help out with
hosting/sysadmin.  That would be immensely helpful in maintaining
non-x86_64 up to speed.

Thanks,
Ludo’.

Re: shortening the git test suite

[Chris Marusich]

Ricardo Wurmus  writes:

> Hi Guix,
>
> git takes a very long time to build, because it has an extensive test
> suite.  Most of the time is spent in running the SVN interoperability
> tests, though, which are not really all that interesting for most uses
> of git.
>
> The Makefile says this:
>
>   # Define NO_SVN_TESTS if you want to skip time-consuming SVN 
> interoperability
>   # tests.  These tests take up a significant amount of the total test time
>   # but are not needed unless you plan to talk to SVN repos.
>
> What do you think about disabling the SVN tests in the git package?

This sounds similar to the discussion we had earlier about treating
tests as a special case:

https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00071.html

I felt that the conclusion of that thread was basically that if someone
is concerned about the build time, then they ought to be able to use
substitutes to speed things up, and we should continue to run as many
tests as possible in order to discover problems sooner.

I'm in favor of disabling tests if they fail for reasons other than bugs
in the program and there isn't an easy fix (e.g., they require root
privileges, or they fail due to randomness in the tests), but disabling
them simply because they take a "long" time doesn't seem like a great
reason, to me.  I'm sympathetic, since I don't like waiting around,
either, but substitutes should be sufficient for those who need faster
builds.

-- 
Chris


signature.asc
Description: PGP signature

Re: RFC: Portability should be a higher priority for Guix (was Re: 01/01: build-system/meson: Really skip the 'fix-runpath' phase on armhf.)

[Ludovic Courtès]

Hello Kei,

Kei Kebreau  skribis:

> I am interested in helping with non-x86_64 issues. Particularly, helping
> with i686-related changes should be just a change in workflow, but I'm
> interested in obtaining freedom-respecting non-x86 hardware (or at least
> using a virtual machine as close as possible to real hardware
> configurations). Any recommendation or links for where I can get a
> Yeeloong laptop or what freedom-respecting armhf computers are
> available?

Without having actual hardware, you can use the qemu-binfmt service on
your machine (info "(guix) Virtualization Services").  It’s slow, but it
allows you to reproduce and debug issues for other architectures.

Thanks,
Ludo’.

Re: GSoC: Adding a web interface similar to the Hydra web interface

[Danny Milosavljevic]

Hi Tatiana,

On Wed, 4 Jul 2018 22:54:46 +0200
Tatiana Sholokhova  wrote:

> If we want to maintain a link to
> the previous page we have to filter the database table entries with to
> types of filters: one with lower bound on the id, and the other with the
> upper bound. 

Yeah, I know what you mean.

I'd suggest one of the following alternatives for implementing "Previous page":

(1) Remember all the page boundaries in the query string (or maybe hidden
form elements).  So instead of finding out where the beginning of the
previous page was all over again just remember it from before.
(2) Reverse the ordering in the query and the boundary check and run
the query, reverse the result of the query.  Handle finished result as before.
(3) Just use the browser's back button.  In fact, you can just put a
"Previous" link that presses the back button via Javascript for the time being.

I suggest (3) - and implement one of the others later.

> The current implementation of pagination works correctly but it does not
> support link to the previous page (first, and next only).

Cool!


pgp3fOZK322cj.pgp
Description: OpenPGP digital signature