Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Chris Marusich]

Chris Marusich  writes:

> Pierre Neidhardt  writes:
>
> [...]
>
>> Shouldn't it be '"$1"/etc/profile'?  And no "exec ..."?

Without the "exec", the second shell will spawn a new process.  With the
"exec", it will not spawn a new process.  It works either way.  I just
decided to use "exec" to avoid spawning a new process.

-- 
Chris


signature.asc
Description: PGP signature

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Chris Marusich]

Pierre Neidhardt  writes:

>> However, I wonder why nobody has implemented
>> "guix environment --profile=/path/to/my/profile".
>
> Note that there is the "--manifest" option to "guix environment".
>
> This brings up another "pro" for manifest: Right now it's possible to
> create environments out of manifests while it's not possible to create
> environments out of profiles without Chris' hack.

This is true.  However, in practice, I often wind up in this sort of
situation:

1.  I want an environment for hacking on a project for which there isn't
a Guix package yet.  I build the environment using a manifest, and then
I run "guix environment -m manifest.scm".  Great!
2.  Many weeks or months pass as I work on other things.
3.  I run "guix pull" at some point.
3.  Eventually, I want to work on that project again, so I run "guix
environment -m manifest.scm".  But now I have to wait hours for Guix to
build and install stuff!

In this situation, if I could just "enter the environment for a
profile", I wouldn't have to wait for Guix to recompile the world.  With
"guix environment -m manifest.scm", you have to wait for the latest
version of everything to be built.  I realize that you could roll back
your Guix installation to whatever version was being used back then (do
you even remember when it was?  I often forget...) and then "guix
environment" ought to work (unless the outputs were GC'd in the
meantime), but I don't think that's a very viable solution because it
requires the user to remember what version was in use at the time and to
take the extra steps necessary to rollback the Guix installation first.
For that reason, I like your suggestion of using "GUIX_PROFILE=profile;
. $GUIX_PROFILE/etc/profile" better.  It requires no waiting, and
because it's a profile you get easy per-profile roll-back.

Pierre Neidhardt  writes:

> A question that I believe has been brought up before: is it possible to
> specify multiple manifests from the command line, such as to provide the
> union of the manifests?

Not sure - I don't think it is possible on the CLI right now.

> Even better: what high-level functions to manipulate manifests, such as
> "manifest-union", "manifest-difference"?

I'm not sure what I would personally use "manifest-difference" for.

>> guix-profile-env()
>> {
>> if [ -z "$1" ]; then
>> echo "usage: guix-profile-env PROFILE [CMD [ARG ...]]" 2>&1
>> return 2
>> fi
>> local sh
>> sh="${SHELL:-/bin/sh}"
>> if [ -z "$2" ]; then
>> "$sh" \
>> -c \
>> 'GUIX_PROFILE="$0" && . "$0"/etc/profile && exec "$1"' \
>
> Shouldn't it be '"$1"/etc/profile'?  And no "exec ..."?

There are two shells.  One is the calling shell, the second is the "$sh"
shell.  The $0 expands to the first argument because it is invoked with
-c (see: (bash) Special Parameters), which in this case is the profile.

>> "$1" \
>> "$sh"
>> else
>> "$sh" \
>> -c \
>> 'GUIX_PROFILE="$0" && . "$0"/etc/profile && exec "$@"' \
>> "$@"
>
> Can you explain why you need to repeat $@ here?

Here, the first "$@" is expanded by the second shell ("$sh").  The
second "$@" is expanded by the calling shell.  You might think we could
write it like this:

  "$sh" \
  -c \
  "GUIX_PROFILE=\"$1\" && . \"$1\"/etc/profile && exec $@"

If we did that, then $1 and $@ are expanded by the calling shell, and
the second shell doesn't do any parameter expansion at all.  However,
this would not work as intended if the elements of $@ (i.e., the CMD and
its ARGs) ever contained whitespace, since the second shell would
perform word splitting (see: (bash) Word Splitting) on them.  To avoid
that, it is necessary to have the second shell expand $@ within
quotation marks (see: (bash) Special Parameters), which is how I
originally wrote it.

In the end, this is still just a hack, and I think it would be nicer to
have a command like "guix environment --profile=/my/profile" to make it
even simpler to use.  Perhaps I will give an implementation a try.

-- 
Chris


signature.asc
Description: PGP signature

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Ludovic Courtès]

Hi,

Pierre Neidhardt  skribis:

> Ludovic Courtès  writes:
>
>> Though maybe we don’t need to have both a blog post and a section in the
>> cookbook, at least once the cookbook is visible on-line.  :-)
>
> It would be nice to have a web feed for the cookbook then :)
>
> Is this planned?

Of course, that’s the whole point of having a Cookbook.  :-)

For that we need two things:

  1. A variant of ‘doc/build.scm’ to build the cookbook.

  2. A new ‘static-web-site-service-type’ instance in
 

 along with appropriate rules in
 

 so that the cookbook is automatically published on-line.

Nothing difficult, but as always, help welcome!  :-)

Thanks,
Ludo’.

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Konrad Hinsen]

Am 14.10.19 um 09:27 schrieb Pierre Neidhardt:


However, I wonder why nobody has implemented
"guix environment --profile=/path/to/my/profile".


Note that there is the "--manifest" option to "guix environment".

This brings up another "pro" for manifest: Right now it's possible to
create environments out of manifests while it's not possible to create
environments out of profiles without Chris' hack.


When I explain Guix, I start with environments, which I consider that 
more fundamental concept, and then introduce profiles as persistent 
environments. From that perspective, creating an environment from a 
profile makes no sense, though it may be useful as a convenience 
function for some needs (not mine). Manifests are persistent 
*specifications* of environments, which do make sense (a lot, in fact).



A question that I believe has been brought up before: is it possible to
specify multiple manifests from the command line, such as to provide the
union of the manifests?

Even better: what high-level functions to manipulate manifests, such as
"manifest-union", "manifest-difference"?


Union is OK, difference isn't. A manifest is not just any set of 
packages, it's a set of packages that has no dependencies outside of the 
set.


This sounds like the beginning of another chapter for the cookbook: 
package lists, manifests, bags, etc. What are the differences, and when 
should I use what?


Which reminds me that when working on the command line, the "manifest" 
option in practice always refers to a package list that is then 
converted to a manifest. And most people *want* to work at the package 
list level. And then differences make sense again. It's all a bit of a 
mess...


Cheers,
  Konrad.

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Pierre Neidhardt]

I'll send a draft to guix-blog just now.

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Chris Marusich]

Pierre Neidhardt  writes:

> While the documentation refers to profiles and manifests, it does not tell 
> much
> of the use cases and the practical benefits.

I just wanted to chime in and say I think a post about this, or a
cookbook section on it, would be great.  I'm sure there are many people
who love declarative manifests and isolated profiles, but they just
don't know it yet.  ;-) It would be great to convince them through clear
examples.

> - When a profile is off, it's easy to enable it for an individual shell 
> without
>   "polluting" the rest of the user session:
>
>   #+begin_src sh
>   GUIX_PROFILE="$profile" ; . "$profile"/etc/profile 
>   #+end_src

This is simple, for sure.  However, I wonder why nobody has implemented
"guix environment --profile=/path/to/my/profile".  I imagine it to
behave just like the "env" command, but it sets the environment
variables for the profile before doing so.  It would be trivial to
define a shell function like "guix-profile-env" that does this:

guix-profile-env()
{
if [ -z "$1" ]; then
echo "usage: guix-profile-env PROFILE [CMD [ARG ...]]" 2>&1
return 2
fi
local sh
sh="${SHELL:-/bin/sh}"
if [ -z "$2" ]; then
"$sh" \
-c \
'GUIX_PROFILE="$0" && . "$0"/etc/profile && exec "$1"' \
"$1" \
"$sh"
else
"$sh" \
-c \
'GUIX_PROFILE="$0" && . "$0"/etc/profile && exec "$@"' \
"$@"
fi
}

Surely it would be fairly simple to do the same in Scheme and expose it
through a friendly CLI interface.  This function just adds to the
environment, but it would be easy to make it "pure" by adding "env -i"
before "$sh" above.

Anyway, I went off on a tangent.  I think your post is a great idea!

-- 
Chris


signature.asc
Description: PGP signature

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Ludovic Courtès]

Hi,

Pierre Neidhardt  skribis:

> Org mode has a Texinfo export library, so I'm thinking I'll do as for my
> other blog posts and write them in Org, then export in
>
> - Markdown for the website;
>
> - Texinfo for the cookbook.

Though maybe we don’t need to have both a blog post and a section in the
cookbook, at least once the cookbook is visible on-line.  :-)

>> Yes, I was going to suggest ‘--list-profiles’.  Hopefully that makes a
>> centralized ~/.guix-extra-profiles less necessary, but either way, it’s
>> good in the cookbook to have a discussion of the various ways you can do
>> things and the various tradeoffs that ensue.
>>
>> Would you like to turn that into a section of the Cookbook?
>
> What do you mean?

I mean that what you wrote could become a section of the Cookbook (in
‘doc/guix-cookbook.texi’ in the repo).

HTH!

Ludo’.

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Ricardo Wurmus]

Pierre Neidhardt  writes:

> Ricardo, how did you convert the blog posts to Texinfo?

Manually.  I had to change the wording in a few places anyway, so it
wasn’t all that much more work.

-- 
Ricardo

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Ludovic Courtès]

Hi Pierre,

Pierre Neidhardt  skribis:

> So what about writing a blog article / cookbook chapter to explain why
> profiles and manifests are truly awesome indeed?  (Unless this has
> already been done and I missed it?)

Go for it!  :-)

Seriously, I’ve discussed profiles and all with several newcomers, who
hadn’t really discovered that facility initially, and the discussion
obviously talks about how to use them, when to use them, and so on.

So I think a section in the Cookbook would be very welcome!

> We can create a manifest per profile and install them this way:
>
>   guix package --manifest=/path/to/guix-dev-manifest.scm  
> --profile=$HOME/.guix-extra-profiles/dev/dev
>   
> Placing all your profiles in a single folder, with each profile getting its 
> own
> subfolder is somewhat cleaner, plus it's obvious to "loop over profiles" from
> any programming language (e.g. a shell script) by
> simply looping over the sub-directories of .guix-extra-profiles.
>
> Note that it's also possible to loop over the output of `guix package
> --list-profiles` although you'll probably have to filter out
> `~/.config/guix/current`.

Yes, I was going to suggest ‘--list-profiles’.  Hopefully that makes a
centralized ~/.guix-extra-profiles less necessary, but either way, it’s
good in the cookbook to have a discussion of the various ways you can do
things and the various tradeoffs that ensue.

Would you like to turn that into a section of the Cookbook?

Note that it’s (still!) not on-line, but I’ll try to update the
berlin.scm configuration (in maintenance.git) so that it’s published, if
nobody beats me at it.

Thanks!

Ludo’.

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Todor Kondić]

On Monday, 7 October 2019 17:37, Konrad Hinsen  
wrote:

> Pierre Neidhardt m...@ambrevar.xyz writes:
>
> > Konrad Hinsen konrad.hin...@fastmail.net writes:
> >
> > > Sounds good in principle. I just wonder how many authors (and thus
> > > different preferences and use cases) it takes to make this really
> > > useful. I note for example that my own use of profiles is rather
> > > different from yours: I have per-project profiles for long-term projects
> > > whose software I don't want to update regularly to avoid breaking stuff.
> >
> > Hmmm, this sounds like what I do too! :)
>
> The main difference is that I don't have all my profiles activated
> in normal use, only my main profile plus one project profile. Different
> project profiles can contain different versions of the same software,
> so activating them together is not a good idea.
>
> What got me started with this strategy was the need to use
> Python 2 and Python 3 in different projects. But then it turned out to
> be useful for other software as well.

Persistent environments anyone? :)



> > Anyways, I'm all for discussing multiple strategies, the more the better!
>
> Sounds like a good plan!
>
> Cheers,
> Konrad

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Konrad Hinsen]

Pierre Neidhardt  writes:

> Konrad Hinsen  writes:
>
>> Sounds good in principle. I just wonder how many authors (and thus
>> different preferences and use cases) it takes to make this really
>> useful. I note for example that my own use of profiles is rather
>> different from yours: I have per-project profiles for long-term projects
>> whose software I don't want to update regularly to avoid breaking stuff.
>
> Hmmm, this sounds like what I do too! :)

The main difference is that I don't have all my profiles activated
in normal use, only my main profile plus one project profile. Different
project profiles can contain different versions of the same software,
so activating them together is not a good idea.

What got me started with this strategy was the need to use
Python 2 and Python 3 in different projects. But then it turned out to
be useful for other software as well.

> Anyways, I'm all for discussing multiple strategies, the more the better!

Sounds like a good plan!

Cheers,
  Konrad

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Pierre Neidhardt]

Konrad Hinsen  writes:

> Sounds good in principle. I just wonder how many authors (and thus
> different preferences and use cases) it takes to make this really
> useful. I note for example that my own use of profiles is rather
> different from yours: I have per-project profiles for long-term projects
> whose software I don't want to update regularly to avoid breaking stuff.

Hmmm, this sounds like what I do too! :)
Maybe I wasn't very clear in my draft.

Anyways, I'm all for discussing multiple strategies, the more the better!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Konrad Hinsen]

Hi Pierre,

> So what about writing a blog article / cookbook chapter to explain why
> profiles and manifests are truly awesome indeed?  (Unless this has
> already been done and I missed it?)

Sounds good in principle. I just wonder how many authors (and thus
different preferences and use cases) it takes to make this really
useful. I note for example that my own use of profiles is rather
different from yours: I have per-project profiles for long-term projects
whose software I don't want to update regularly to avoid breaking stuff.

> Placing all your profiles in a single folder, with each profile getting its 
> own
> subfolder is somewhat cleaner, plus it's obvious to "loop over profiles" from
> any programming language (e.g. a shell script) by
> simply looping over the sub-directories of .guix-extra-profiles.

That's very good advice that definitely should be written down somewhere.

Konrad.

Re: [Blog/Cookbook?] On multiple Guix profiles and manifests

[Todor Kondić]

On Saturday, 5 October 2019 12:55, Pierre Neidhardt  wrote:

> Hi!
>
> While the documentation refers to profiles and manifests, it does not tell 
> much
> of the use cases and the practical benefits.
>
> For users coming from non-functional package managers, it's not really obvious
> why we would need profiles.
>
> Regarding manifest, I suppose some people might find them annoying to use
> because updating/adding just 1 program means rebuilding the whole profile, 
> which
> after a Guix pull can be somewhat lengthy.
>
> So what about writing a blog article / cookbook chapter to explain why
> profiles and manifests are truly awesome indeed? (Unless this has
> already been done and I missed it?)
>
> A quick 'n' dirty outline:
>
> A manifest can be slow to install if it's too big. But Guix supports profile,
> which are perfect to break down manifests into multiple sets of semantically
> connected packages.
>
> Example profiles:
>
> -   Emacs.
> -   TeXlive (this one can be really useful when you need to install just one
> package for the next document you've received over email).
>
> -   Your favourite programming language libraries.
> -   The dependencies of a project you are working on
> -   Games :p
>
> We can create a manifest per profile and install them this way:
>
> guix package --manifest=/path/to/guix-dev-manifest.scm 
> --profile=$HOME/.guix-extra-profiles/dev/dev
>
> Placing all your profiles in a single folder, with each profile getting 
> its own
> subfolder is somewhat cleaner, plus it's obvious to "loop over profiles" 
> from
> any programming language (e.g. a shell script) by
> simply looping over the sub-directories of .guix-extra-profiles.
>
> Note that it's also possible to loop over the output of `guix package 
> --list-profiles` although you'll probably have to filter out
> `~/.config/guix/current`.
>
> To "enable" all profiles on login, add this to your .bash_profile (or 
> .profile
> if you don't use bash):
>
> #+begin_src sh
> for i in ~/.guix-extra-profiles/*; do
> profile=$i/$(basename "$i")
> if [ -f "$profile"/etc/profile ]; then
> GUIX_PROFILE="$profile" ; . "$profile"/etc/profile
> fi
> unset profile
> done
> #+end_src
>
> I like to keep the default ~/.guix-profile manifest-less for trash-away 
> packages
> that I would just use for a couple of days.
> This way it's easy to just run
>
> guix install FOO
> guix upgrade BAR
>
> and I don't have to specify the profile.
>
> Other benefits of manifests:
>
> -   No need to generate or maintain a manifest from an ad-hoc profile.
> -   "guix package -u" will always suggest to update some packages of those 
> have
> propagated inputs. Guix manifests avoid this problem.
>
> -   "guix package -u [packages...]" may report conflicts which are annoying to
> resolve manually. Manifests avoid this problem altogether.
>
> Other benefits of multiple profiles
>
> -   It's easy to toggle a specific profile on/off.
> -   When a profile is off, it's easy to enable it for an individual shell 
> without
> "polluting" the rest of the user session:
>
> #+begin_src sh
> GUIX_PROFILE="$profile" ; . "$profile"/etc/profile
> #+end_src
>
> Happy to hear about your feedback!
>
> Cheers!
>
> --
> Pierre Neidhardt
> https://ambrevar.xyz/
>


What is the advantage of having different profiles to just defining different 
top-level packages propagating all programs and libraries one needs and placing 
them in a custom channel? My feeling is, defining a package is cleaner than 
manifests+profiles, because it does not involve creating yet another shell 
program that needs to be sourced.