Re: [Nix-dev] What is the end game?

[Tim Cuthbertson]

On Mon, Aug 31, 2015 at 1:34 AM, Daniel Peebles  wrote:
> I'm trying to avoid making this conversation about Hydra (or services)
> specifically, but agree with the points you're making.
>
> To try to steer the discussion in the direction I'm most interested in, what
> if binutils had a release.nix in its repo? How about gcc or clang? What
> would the expressions look like in nixpkgs? The options I can see:
>
> 1) nixpkgs is full of e.g., binutils = (import "${fetch* { url =
> "url/to/binutils"; sha256 = "abc"; }}/release.nix").build
> 2) Duplicate the release.nix of binutils in nixpkgs
> 3) Some combination of 1 and 2, where metadata, name, dependencies are
> expressed in nixpkgs, but build instructions are in the repo's release.nix
> 4) ???
> 5) profit
>
> Actually, I'm not sure #4 and #5 are relevant, but I couldn't think of other
> options.
>
> #1 seems impractical because a single nix-env would result in thousands of
> fetchurls
> #2 seems undesirable due to duplicated information
> #3 seems like our best bet, but I don't know what that would look like


As someone who's been using nixpkgs out-of-tree (i.e. writing
individual expressions for nix-build, but not yet integrating them
into nixpkgs) for a while, this interests me.

I thought (1) as Daniel described would have been be a good solution,
and seems to work fine for _building_ software. But I tried this out
with a new package I'm adding:
https://github.com/NixOS/nixpkgs/pull/9653

nix-env fails - and rightly so, because it can't figure out the name
(and presumably other details) of this package without downloading it,
which would indeed be silly.

The other approach (as cabal2nix is doing elsewhere in this thread) is
to have the primary expression in nixpkgs, and have the upstream
project extend it with overrideDerivation or similar. But that doesn't
make any sense for new package - there's nothing in nixpkgs yet to
extend!

So it seems like doing it this way would involve a few different stages:

 1) write a local .nix file in my package's repo
 2) once it seems to be working, move / copy it to nixpkgs
 3) once it's reached one or more channels, convert my package's .nix
file to override the "downstream" nixpkgs expression. It's not clear
when I should do this, as suddenly anyone using a "too old" nixpkgs
won't be able to build it anymore.

Further, I could imagine adding overrides in my repo for packaging
changes which should then be taken out once those same changes have
been merged into nixpkgs. But again, that's not a concrete point in
time - it depends what channel you're on, and when you last updated.

So... that's not very appealing.

Is there some variant of the "main expression in upstream repo,
metadata in nixpkgs" which would allow nix-env to get enough
information, while keeping the bulk of the logic upstream (and
non-duplicated)?

Cheers,
 - Tim.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Peter Simons]

Hi Kosyrev,

 >> The binutils release.nix file would then customize those expressions
 >> with overrideDerivation or some other appropriate mechanism to build
 >> developer versions instead, i.e. by switching the 'src' location to a
 >> Git repository etc.
 >
 > Do we have precedents for this?

https://github.com/NixOS/cabal2nix/blob/master/release.nix

Best regards,
Peter

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Colin Putney]

On Mon, Aug 31, 2015 at 1:26 AM, Peter Simons  wrote:

> Hi Colin,
>
>  > That seems backwards. The upstream project would then depend on nixpkgs.
>
> the upstream project depends on Nixpkgs because it most likely needs a
> compiler and a libc and some other core utils to build. Where are those
> expressions supposed to come from without depending on Nixpkgs?
>

Anywhere you like! It could depend on nixpkgs, sure. Or you could have your
own library of nix expressions for dependencies. Or there could be another
shared repository for dependencies, perhaps around a particular programming
language or use-case. Nix is a good way of specifying dependencies,
regardless of where the those specifications come from. An upstream project
that wants to use nix should not have to depend on nixpkgs just to build.

Colin
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Peter Simons]

Hi Colin,

 > That seems backwards. The upstream project would then depend on nixpkgs.

the upstream project depends on Nixpkgs because it most likely needs a
compiler and a libc and some other core utils to build. Where are those
expressions supposed to come from without depending on Nixpkgs?

Best regards,
Peter

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Peter Simons]

Hi Daniel,

  What if binutils had a release.nix in its repo? How about gcc or
  clang? What would the expressions look like in nixpkgs?

the expressions in Nixpkgs would look exactly like they do today, i.e.
they download a release tarball and build it. The binutils release.nix
file would then customize those expressions with overrideDerivation or
some other appropriate mechanism to build developer versions instead,
i.e. by switching the 'src' location to a Git repository etc.

Best regards,
Peter

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Anderson Torres]

My two satoshis on it.

Looking at our famous brothers as Slackware, Gentoo, Debian, Arch
etc., we can trace and forecast a situation:
* Small projects can be handled by single developers, with high
independence from upstream maintainers (as myself taking care of mpv
and xiphos);
* Medium and huge projects can be handled by sub-teams, such as GCC
NixOS team, or Haskell NixOS Team; these huge teams will be more
tightly connected to upstream, with developers in common.

In practical terms, I prefer the expressions centralized on our Nixpkgs release.

2015-08-30 13:12 GMT-03:00 Nicolas Pierron nicolas.b.pier...@gmail.com:
 I don't think the #1 is unpractical, if you consider that these
 fetches are properly mirrored, which we currently do with channels.

 On Sun, Aug 30, 2015 at 5:34 PM, Daniel Peebles pumpkin...@gmail.com wrote:
 I'm trying to avoid making this conversation about Hydra (or services)
 specifically, but agree with the points you're making.

 To try to steer the discussion in the direction I'm most interested in, what
 if binutils had a release.nix in its repo? How about gcc or clang? What
 would the expressions look like in nixpkgs? The options I can see:

 1) nixpkgs is full of e.g., binutils = (import ${fetch* { url =
 url/to/binutils; sha256 = abc; }}/release.nix).build
 2) Duplicate the release.nix of binutils in nixpkgs
 3) Some combination of 1 and 2, where metadata, name, dependencies are
 expressed in nixpkgs, but build instructions are in the repo's release.nix
 4) ???
 5) profit

 Actually, I'm not sure #4 and #5 are relevant, but I couldn't think of other
 options.

 #1 seems impractical because a single nix-env would result in thousands of
 fetchurls
 #2 seems undesirable due to duplicated information
 #3 seems like our best bet, but I don't know what that would look like



 On Sun, Aug 30, 2015 at 9:54 AM, Domen Kožar do...@dev.si wrote:

 Or we could, as any software, tag and release hydra so that users can
 fetch working versions for easier learning curve.


 On Sun, 30 Aug 2015 15:38 Nicolas Pierron nicolas.b.pier...@gmail.com
 wrote:

 Hi Daniel,

 On Tue, Aug 25, 2015 at 5:42 PM, Daniel Peebles pumpkin...@gmail.com
 wrote:
  Let's say for a moment that Nix has taken over the world, and every
  open
  source project now includes a default.nix or release.nix in its repo
  root.
 
  What does nixpkgs look like in this world? Does it duplicate the
  individual
  package .nix files in their respective repositories? Does it only
  duplicate
  minimal information (dependencies and meta) from the remote
  repositories?

 If we were in such world, then the module would probably be best
 handled by upstream maintainers.

 The way NixOS modules are working, we need all of them before
 evaluating any configuration, thus we would need to have a copy of the
 configuration file, even if we have to download it.  In such case, it
 makes sense that NixOS list of modules would be built out-of an
 aggregate of fetched resources.

 Thus, if we ever do a copy, we should do it with an url and a hash,
 and have one of the multiple output of packages be the NixOS module
 that we will aggregate, as-if it was a generic post-install script.

 Then, the problem with Hydra, is slightly different.  Currently hydra
 does not provide any stable (tagged) version.  I guess we could
 experiment the previous suggestion, but I would prefer to have
 multiple instances of this problem before attempting any generic
 solution as described above.  In the mean time, I think having your
 own copy of hydra, and using it to aggregate the module which is
 inside might be the best solution.

 --
 Nicolas Pierron
 http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev





 --
 Nicolas Pierron
 http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Nicolas Pierron]

I don't think the #1 is unpractical, if you consider that these
fetches are properly mirrored, which we currently do with channels.

On Sun, Aug 30, 2015 at 5:34 PM, Daniel Peebles pumpkin...@gmail.com wrote:
 I'm trying to avoid making this conversation about Hydra (or services)
 specifically, but agree with the points you're making.

 To try to steer the discussion in the direction I'm most interested in, what
 if binutils had a release.nix in its repo? How about gcc or clang? What
 would the expressions look like in nixpkgs? The options I can see:

 1) nixpkgs is full of e.g., binutils = (import ${fetch* { url =
 url/to/binutils; sha256 = abc; }}/release.nix).build
 2) Duplicate the release.nix of binutils in nixpkgs
 3) Some combination of 1 and 2, where metadata, name, dependencies are
 expressed in nixpkgs, but build instructions are in the repo's release.nix
 4) ???
 5) profit

 Actually, I'm not sure #4 and #5 are relevant, but I couldn't think of other
 options.

 #1 seems impractical because a single nix-env would result in thousands of
 fetchurls
 #2 seems undesirable due to duplicated information
 #3 seems like our best bet, but I don't know what that would look like



 On Sun, Aug 30, 2015 at 9:54 AM, Domen Kožar do...@dev.si wrote:

 Or we could, as any software, tag and release hydra so that users can
 fetch working versions for easier learning curve.


 On Sun, 30 Aug 2015 15:38 Nicolas Pierron nicolas.b.pier...@gmail.com
 wrote:

 Hi Daniel,

 On Tue, Aug 25, 2015 at 5:42 PM, Daniel Peebles pumpkin...@gmail.com
 wrote:
  Let's say for a moment that Nix has taken over the world, and every
  open
  source project now includes a default.nix or release.nix in its repo
  root.
 
  What does nixpkgs look like in this world? Does it duplicate the
  individual
  package .nix files in their respective repositories? Does it only
  duplicate
  minimal information (dependencies and meta) from the remote
  repositories?

 If we were in such world, then the module would probably be best
 handled by upstream maintainers.

 The way NixOS modules are working, we need all of them before
 evaluating any configuration, thus we would need to have a copy of the
 configuration file, even if we have to download it.  In such case, it
 makes sense that NixOS list of modules would be built out-of an
 aggregate of fetched resources.

 Thus, if we ever do a copy, we should do it with an url and a hash,
 and have one of the multiple output of packages be the NixOS module
 that we will aggregate, as-if it was a generic post-install script.

 Then, the problem with Hydra, is slightly different.  Currently hydra
 does not provide any stable (tagged) version.  I guess we could
 experiment the previous suggestion, but I would prefer to have
 multiple instances of this problem before attempting any generic
 solution as described above.  In the mean time, I think having your
 own copy of hydra, and using it to aggregate the module which is
 inside might be the best solution.

 --
 Nicolas Pierron
 http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev





-- 
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Colin Putney]

On Sun, Aug 30, 2015 at 11:09 AM, Peter Simons sim...@cryp.to wrote:

 Hi Daniel,

   What if binutils had a release.nix in its repo? How about gcc or
   clang? What would the expressions look like in nixpkgs?

 the expressions in Nixpkgs would look exactly like they do today, i.e.
 they download a release tarball and build it. The binutils release.nix
 file would then customize those expressions with overrideDerivation or
 some other appropriate mechanism to build developer versions instead,
 i.e. by switching the 'src' location to a Git repository etc.


That seems backwards. The upstream project would then depend on nixpkgs.

-Colin
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Daniel Peebles]

I linked to one of those in my email and was the submitter of the other, so
I've seen them :)

TBC: this is not intended as a discussion of the Hydra package/service. I
was just using Hydra as an example of a project that already fit my
description. I want to know what the plan is for other projects that fit
that description, since so far it appears that usablity (or discoverability
and unified documentation, at least) currently suffers a little in the case
where we have nix-enabled projects upstream.

On Thu, Aug 27, 2015 at 5:40 AM, Joachim Schiele j...@lastlog.de wrote:

 please see this discussions:

 https://github.com/NixOS/nixpkgs/issues/7504
 https://github.com/NixOS/hydra/issues/197

 i also would like the service in nixpkgs but eelco certainly knows what
 he is doing! on the other hand installing hydra is a real pain and many
 many will simply give up quite soonish.

 On 25.08.2015 17:42, Daniel Peebles wrote:
  Hi all,
 
  Let's say for a moment that Nix has taken over the world, and every open
  source project now includes a default.nix or release.nix in its repo
 root.
 
  What does nixpkgs look like in this world? Does it duplicate the
  individual package .nix files in their respective repositories? Does it
  only duplicate minimal information (dependencies and meta) from the
  remote repositories?
 
  We're already running into (a small amount of) pain because Hydra
  already fits the description
  above:
 https://github.com/NixOS/nixpkgs/issues/7504#issuecomment-95547176.
  Based on expectations set by most packages in nixpkgs/nixos, we expect
  to find services and packages in nixpkgs, and Hydra breaks that
  expectation. I'm not necessarily saying we should move its expressions
  into nixpkgs, but I am looking for someone to paint a picture of what
  Nix(OS) life will look like when we achieve world domination :)
 
  Thanks
  Dan
 
 
 
  ___
  nix-dev mailing list
  nix-dev@lists.science.uu.nl
  http://lists.science.uu.nl/mailman/listinfo/nix-dev
 


 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] What is the end game?

[Joachim Schiele]

please see this discussions:

https://github.com/NixOS/nixpkgs/issues/7504
https://github.com/NixOS/hydra/issues/197

i also would like the service in nixpkgs but eelco certainly knows what
he is doing! on the other hand installing hydra is a real pain and many
many will simply give up quite soonish.

On 25.08.2015 17:42, Daniel Peebles wrote:
 Hi all,
 
 Let's say for a moment that Nix has taken over the world, and every open
 source project now includes a default.nix or release.nix in its repo root.
 
 What does nixpkgs look like in this world? Does it duplicate the
 individual package .nix files in their respective repositories? Does it
 only duplicate minimal information (dependencies and meta) from the
 remote repositories?
 
 We're already running into (a small amount of) pain because Hydra
 already fits the description
 above: https://github.com/NixOS/nixpkgs/issues/7504#issuecomment-95547176.
 Based on expectations set by most packages in nixpkgs/nixos, we expect
 to find services and packages in nixpkgs, and Hydra breaks that
 expectation. I'm not necessarily saying we should move its expressions
 into nixpkgs, but I am looking for someone to paint a picture of what
 Nix(OS) life will look like when we achieve world domination :)
 
 Thanks
 Dan
 
 
 
 ___
 nix-dev mailing list
 nix-dev@lists.science.uu.nl
 http://lists.science.uu.nl/mailman/listinfo/nix-dev
 


___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

[Nix-dev] What is the end game?

[Daniel Peebles]

Hi all,

Let's say for a moment that Nix has taken over the world, and every open
source project now includes a default.nix or release.nix in its repo root.

What does nixpkgs look like in this world? Does it duplicate the individual
package .nix files in their respective repositories? Does it only duplicate
minimal information (dependencies and meta) from the remote repositories?

We're already running into (a small amount of) pain because Hydra already
fits the description above:
https://github.com/NixOS/nixpkgs/issues/7504#issuecomment-95547176. Based
on expectations set by most packages in nixpkgs/nixos, we expect to find
services and packages in nixpkgs, and Hydra breaks that expectation. I'm
not necessarily saying we should move its expressions into nixpkgs, but I
am looking for someone to paint a picture of what Nix(OS) life will look
like when we achieve world domination :)

Thanks
Dan
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev