Heads-up from Linus -- potential bisection trainwreck: "A note on the 5.12-rc1 tag"

[Bengt Richter]

Hi,

Not so usual to be switching rc kernels for guix I suppose, but
this looked worth mentioning anyway:

LWN archive link [1]

[1]https://lwn.net/ml/linux-kernel/CAHk-=wjnzdlsp3odxhf9emtyo7gf-qjanlbuh1zk3c4a7x7...@mail.gmail.com/

In case of link problem, a couple of extractions:
--8<---cut here---start->8---
From:   Linus Torvalds 
To: Linux Kernel Mailing List 
Subject:A note on the 5.12-rc1 tag
Date:   Wed, 03 Mar 2021 12:53:18 -0800
Message-ID: 


Hey peeps - some of you may have already noticed that in my public git
tree, the "v5.12-rc1" tag has magically been renamed to
"v5.12-rc1-dontuse". It's still the same object, it still says
"v5.12-rc1" internally, and it is still is signed by me, but the
user-visible name of the tag has changed.

The reason is fairly straightforward: this merge window, we had a very
innocuous code cleanup and simplification that raised no red flags at
all, but had a subtle and very nasty bug in it: swap files stopped
working right.  And they stopped working in a particularly bad way:
the offset of the start of the swap file was lost.

Swapping still happened, but it happened to the wrong part of the
filesystem, with the obvious catastrophic end results.

[ -- snip discussion why all will not be hit -- ]

But I want everybody to be aware of because _if_ it bites you, it
bites you hard, and you can end up with a filesystem that is
essentially overwritten by random swap data. This is what we in the
industry call "double ungood".

--8<---cut here---end--->8---

-- 
Regards,
Bengt Richter

Re: mupdf vulnerable to CVE-2021-3407

[Kei]

I just patched this in commit 6891f957.  Thanks for the heads up!

On Wed, 2021-03-03 at 21:53 +0100, Léo Le Bouter wrote:
> CVE-2021-3407 24.02.21 00:15
> A flaw was found in mupdf 1.18.0. Double free of object during
> linearization may lead to memory corruption and other potential
> consequences.
> 
> mupdf has made no release yet, so you need to cherry-pick the commit: 
> https://git.ghostscript.com/?p=mupdf.git;a=log;h=cee7cefc610d42fd383b3c80c12cbc675443176a


signature.asc
Description: This is a digitally signed message part

Re: [Outreachy] Use of impure functional programming and use of vlists

[zimoun]

Hi Magali,

On Sat, 27 Feb 2021 at 00:10, Magali Lemes  wrote:

> As my Outreachy internship approaches an end, I'd like to know which of 
> the following options is better for walking and displaying the Git 
> commit history:
> 1) having a list and then using for-each it to display the commit 
> information;
> 2) display the list while building it.

What could be informative is to benchmark the two cases.  For an
example, see my previous tests on your code:

   

Profiling the 2 functions should be also helpful.  Feel free to ask on
guix-devel how to profile with Guile.


Cheers,
simon

Re: Release on April 18th?

[zimoun]

Hi Leo,

On Thu, 04 Mar 2021 at 14:07, Leo Famulari  wrote:
> On Thu, Mar 04, 2021 at 10:41:34AM +0100, zimoun wrote:
>> On Wed, 03 Mar 2021 at 13:51, Leo Famulari  wrote:
>> > * Update tzdata
>> 
>> “guix refresh tzdata -l” provides couple of dependants.  Is it
>> reasonable to update it for the next release?
>
> For me, I see 1765 dependents (a "couple" is 2). We have the capacity to
> rebuild them in this timeframe.

I should have had emphasized «couple». ;-)
Ah, I miss something because I thought this kind of upgrade was a
candidate for core-updates or staging.
Anyway. :-)

Added to the TODO. :-)


> The staging branch has been completed, along with the previous
> ungrafting. But now there are new grafts and, in my opinion, we don't
> have time to do another staging round before April 18.

Ungrafting as an instance of «Sisyphus stone». ;-)


>> From my point of view, the whole “ungrafting” process is unclear on two
>> sides: 1. how to effectively ungraft a package?  i.e., what are the
>> typical steps? and 2. what is the list of packages to ungraft?
>
> 1) Move the changes of the replacement packages into the packages that
> were being replaced. For example, we should move the patch
> 'python-2.7-CVE-2021-3177.patch' from the origin of python-2.7/fixed to
> the origin of python-2.7.
>
> 2) Grep on the master branch in gnu/packages for '(replacement'. That
> will show you every graft.

Thanks for the explanations.  I will try to contribute to the effort in
the next days.

Does it exist a way to list the grafts?  I mean there is “guix build
--no-grafts” but I do not know how to get what grafts which will be
applied beforehand.


Cheers,
simon

Re: Release on April 18th?

[Leo Famulari]

On Thu, Mar 04, 2021 at 11:18:19PM +0100, zimoun wrote:
> I should have had emphasized «couple». ;-)
> Ah, I miss something because I thought this kind of upgrade was a
> candidate for core-updates or staging.
> Anyway. :-)

It is usually is, but we can be confident that it won't break anything,
based on past experience updating tzdata.

> Thanks for the explanations.  I will try to contribute to the effort in
> the next days.

We should have a branch where we do these changes.

> Does it exist a way to list the grafts?  I mean there is “guix build
> --no-grafts” but I do not know how to get what grafts which will be
> applied beforehand.

Not that I know of. I always use `git grep`.

Re: Changes to the branching workflow

[zimoun]

Hi, Chris,

On Thu, 04 Mar 2021 at 16:05, Christopher Lemmer Webber 
 wrote:

> I wonder if we should formalize it.  What about adding a section to the 
> "Contributing" section of the manual explaining what the different
> branches are, and when you have a patch that's been approved, when to
> push it to which branch?

Do you mean something like #8 in [1] and then [2]?

1: 
2: 

In [2], it reads:

For patches that just add a new package, and a simple one, it’s
OK to commit, if you’re confident (which means you successfully
built it in a chroot setup, and have done a reasonable copyright
and license auditing). Likewise for package upgrades, except
upgrades that trigger a lot of rebuilds (for example, upgrading
GnuTLS or GLib).

which I understand as: the ’staging’ or ’core-update’ patches should go
to guix-patches and not be pushed directly.  Especially when one has
commit access and does not follow closely enough to know the status of
the very branch.


Cheers,
simon

Re: Changes to the branching workflow

[Christopher Lemmer Webber]

Hello,

As someone who jumps in and out of Guix development occasionally but who
does have commit access (probably for grantparented-in reasons, heh),
sometimes I don't know where I should commit code or what the current
branching policy is, and I might miss emails like this.

I wonder if we should formalize it.  What about adding a section to the 
"Contributing" section of the manual explaining what the different
branches are, and when you have a patch that's been approved, when to
push it to which branch?

I think that would help me, and maybe it would help others.  Thoughts?

 - Chris

Leo Famulari writes:

> Based on experiences with the last "staging" cycle and discussions at
> the most recent Guix Day meeting [0], we've changed the branching
> workflow.
>
> The default branch names remain "core-updates" and "staging".
>
> When we begin actively building and testing the branches, they will be
> renamed to "core-updates-frozen" and "staging-frozen", respectively.
>
> This will indicate that they are closed to any changes except for bug
> fixes and merges of the master branch.
>
> During those periods, new patches can be pushed to "core-updates-next"
> and "staging-next".
>
> Hopefully these changes will clarify the status of the branches and
> reduce confusion.
>
> [0] https://lists.gnu.org/archive/html/guix-devel/2021-02/msg00163.html

Re: What features does a language/runtime need to use guix as a package manager?

[Leo Prikler]

Hello Vinícius

Am Mittwoch, den 03.03.2021, 15:31 -0300 schrieb Vinícius dos Santos
Oliveira:
> Hi,
> 
> I was curious about how a language/runtime module system needs to be
> designed to use guix as a package manager. I'm developing my own
> require() function for a Lua host and I'd like to make sure it can
> use
> guix as a package manager.
As a rule of thumb, the less "smart" your language/runtime is, the
easier it gets.  Now, this is only true to a certain extent – if your
language/runtime does not understand the concept of environment
variables at all, you may need to overthink it a little, but if it
rolls out its own package manager (see also Rust, NPM, Go) things will
get needlessly complicated.

> From a tutorial by Andrew Tropin[1] I learned about
> native-search-paths. So I changed my module system to use a *_PATH
> environment variable so guix could teach the Lua-based runtime where
> to find the modules.
If you're planning to do cross-compilation, you might add a search-path 
in lieu of/addition to the native-search-path, but that sounds like a
good idea.

> Then there is the next question: how does the module know where to
> install itself to? A similar problem is found in GStreamer, so I'll
> refer to the GStreamer scenario from now on.
> 
> The guix GStreamer package defines native-search-paths to
> GST_PLUGIN_SYSTEM_PATH=lib/gstreamer-1.0. As far as I tested, if a
> package gstreamer-foobar depends on gstreamer and install files to
> ${prefix}/lib/gstreamer-1.0, then GST_PLUGIN_SYSTEM_PATH will be
> automatically updated to also contain gstreamer-foobar's
> /gnu/store/*/lib/gstreamer-1.0.
> 
> My question here is: how should the software packaged in
> gstreamer-foobar find out the proper directory to install its
> plugins?
> Should it be ${prefix}/lib/gstreamer-1.0? Should it be
> ${prefix}/lib/gstreamer-1.1? That's something that shouldn't be
> hardcoded. 
Why not?  See [1] for how GStreamer itself does this.
More generally speaking, GNU has standard for Makefiles [2], that
should probably also be followed by your build tool of choice. 

> As far as I see, it can just use the pluginsdir from
> GStreamer's pkg-config definition. Like so:
> 
> $ pkg-config --variable=pluginsdir gstreamer-1.0
> 
> However this command will print the wrong dir in guix. It'll print
> something like:
> 
> /gnu/store/9if71w58d5mkxfxyc7fpz289qssnkqsv-gstreamer-
> 1.18.2/lib/gstreamer-1.0
> 
> But that's the "namespace" for the gstreamer package, not for the
> gstreamer-foobar package. A solution would be to invoke pkg-config as
> follows:
Using pkg-config in such a manner is a hack.  Maintainers of
traditional distros know (and ignore) this.

> $ pkg-config --define-variable="prefix=${prefix}"
> --variable=pluginsdir gstreamer-1.0
> 
> This will actually print the proper ${prefix}/lib/gstreamer-1.0. But
> that's my question here. How should pkg-config be invoked properly? I
> could just as well invoke it as:
> 
> $ pkg-config --define-variable="prefix=${prefix}"
> --define-variable=libdir='${prefix}/lib' --variable=pluginsdir
> gstreamer-1.0
I think you're trying to solve a problem, that does not really need
solving here.  Instead of trying to fix your calls to pkg-config, do
not use pkg-config to find locations to write to!

On a related note, you should probably not change versions in versioned
directories, such as lib/gstreamer-1.0, unless you also break the API
in some way.  I think Python serves as a somewhat good negative
example, since most packages would support 3.x, for some values of x,
but all get shoved into whatever Python was used during build.  Emacs
has a similar issue, which did lead many people into strange situations
during the 27 upgrade or when trying out emacs-next.  This does not
mean, that you can't version stuff like Python or Emacs do, just that
you should be careful and aware of the consequences.  (Also both Python
and Emacs are somewhat justified in doing so, since their minor
versions typically add features, that weren't present prior.)

Regards,
Leo

[1] 
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/blob/552da8569bca9ea5238bcbdad7432b8f7c0837b9/meson.build#L25-27
[2] 
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html

What features does a language/runtime need to use guix as a package manager?

[Vinícius dos Santos Oliveira]

Hi,

I was curious about how a language/runtime module system needs to be
designed to use guix as a package manager. I'm developing my own
require() function for a Lua host and I'd like to make sure it can use
guix as a package manager.

>From a tutorial by Andrew Tropin[1] I learned about
native-search-paths. So I changed my module system to use a *_PATH
environment variable so guix could teach the Lua-based runtime where
to find the modules.

Then there is the next question: how does the module know where to
install itself to? A similar problem is found in GStreamer, so I'll
refer to the GStreamer scenario from now on.

The guix GStreamer package defines native-search-paths to
GST_PLUGIN_SYSTEM_PATH=lib/gstreamer-1.0. As far as I tested, if a
package gstreamer-foobar depends on gstreamer and install files to
${prefix}/lib/gstreamer-1.0, then GST_PLUGIN_SYSTEM_PATH will be
automatically updated to also contain gstreamer-foobar's
/gnu/store/*/lib/gstreamer-1.0.

My question here is: how should the software packaged in
gstreamer-foobar find out the proper directory to install its plugins?
Should it be ${prefix}/lib/gstreamer-1.0? Should it be
${prefix}/lib/gstreamer-1.1? That's something that shouldn't be
hardcoded. As far as I see, it can just use the pluginsdir from
GStreamer's pkg-config definition. Like so:

$ pkg-config --variable=pluginsdir gstreamer-1.0

However this command will print the wrong dir in guix. It'll print
something like:

/gnu/store/9if71w58d5mkxfxyc7fpz289qssnkqsv-gstreamer-1.18.2/lib/gstreamer-1.0

But that's the "namespace" for the gstreamer package, not for the
gstreamer-foobar package. A solution would be to invoke pkg-config as
follows:

$ pkg-config --define-variable="prefix=${prefix}"
--variable=pluginsdir gstreamer-1.0

This will actually print the proper ${prefix}/lib/gstreamer-1.0. But
that's my question here. How should pkg-config be invoked properly? I
could just as well invoke it as:

$ pkg-config --define-variable="prefix=${prefix}"
--define-variable=libdir='${prefix}/lib' --variable=pluginsdir
gstreamer-1.0


[1] https://youtu.be/R8DtPnP4eL8

-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: Release on April 18th?

[Leo Famulari]

On Thu, Mar 04, 2021 at 10:41:34AM +0100, zimoun wrote:
> On Wed, 03 Mar 2021 at 13:51, Leo Famulari  wrote:
> > * Update tzdata
> 
> “guix refresh tzdata -l” provides couple of dependants.  Is it
> reasonable to update it for the next release?

For me, I see 1765 dependents (a "couple" is 2). We have the capacity to
rebuild them in this timeframe.

> > * Ungraft
> 
> I am not following closely, sorry if I miss something.  The ’ungrafting’
> branch had been merged to ’staging’ right?  And what is the state of
> ’staging’?

The staging branch has been completed, along with the previous
ungrafting. But now there are new grafts and, in my opinion, we don't
have time to do another staging round before April 18.

I think it's important to release without any grafts because they do not
always work as expected. For example:

https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00292.html

> From my point of view, the whole “ungrafting” process is unclear on two
> sides: 1. how to effectively ungraft a package?  i.e., what are the
> typical steps? and 2. what is the list of packages to ungraft?

1) Move the changes of the replacement packages into the packages that
were being replaced. For example, we should move the patch
'python-2.7-CVE-2021-3177.patch' from the origin of python-2.7/fixed to
the origin of python-2.7.

2) Grep on the master branch in gnu/packages for '(replacement'. That
will show you every graft.

Does that make sense?

Re: branch master updated: website: nls: Fix URLs for Chinese translation.

[Tobias Geerinckx-Rice]

Florian Pelz 写道：

- ("zh_CN" . "zh-cn"))
+ ("zh_CN" . "zh-CN"))


Thanks, Florian!  So there was no hidden ‘reason’ for us 
preferring cn over CN, correct?  I was too scared to break 
something...


Kind regards,

T G-R


signature.asc
Description: PGP signature

Re: Release on April 18th?

[zimoun]

Hi Leo,

On Wed, 03 Mar 2021 at 13:51, Leo Famulari  wrote:

> * Fix #46871 (problems with init scripts and guix-install.sh).

I think it is (almost) done, see .


> * Update tzdata

“guix refresh tzdata -l” provides couple of dependants.  Is it
reasonable to update it for the next release?


> * Ungraft

I am not following closely, sorry if I miss something.  The ’ungrafting’
branch had been merged to ’staging’ right?  And what is the state of
’staging’?

>From my point of view, the whole “ungrafting” process is unclear on two
sides: 1. how to effectively ungraft a package?  i.e., what are the
typical steps? and 2. what is the list of packages to ungraft?

About the #1, there are some commits, for example
a210c0d13752c38a850746fd97948121046a0e58, which could help to understand
how to do.  But grepping in the Git log with ’graft’ returns only a
couple of examples.

About the #2, because the feedback loop with ci.guix is slow and because
most of the (unnecessary) grafted packages look like more annoyance than
really unusuable packages, and for me, because it is not clear where I
have to look: master, staging, core-updates or ungrafting, then all in
all laziness applies. :-) Personally, I am saying to myself: let
investigate tomorrow, and then other stuff happens this very tomorrow,
so I never investigate at the end.

Well, I propose to first collect a list about packages that “we” would
like «ungrafted» for the next release.  This makes a concrete criteria
to decide if it releasable or not yet; as “we” do for blocking bugs.
WDYT?


Cheers,
simon