Re: [Nix-dev] fetchgit - why sha256 protection?
Hi, On 19/11/12 07:11, Marc Weber wrote: Isn't it enough to depend on the git's hash value, No, because Nix's fixed-output derivation feature requires a md5/sha1/sha256 hash of the expected contents. -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Excerpts from Eelco Dolstra's message of Mon Nov 19 11:01:39 +0100 2012: No, because Nix's fixed-output derivation feature requires a md5/sha1/sha256 hash of the expected contents. I know what the current implementation requires. Just wondering whether this should be relaxed for git (like) VCS sources, because they naturally have a hash. I mean why run nix-prefetch git if using url and git commit hash could be enough? If you don't trust builders, fetching git sources is that common that it could even be built into the nix tool. My goal is to simplify installing packages from other sub universes such as ruby. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Hi, On 19/11/12 11:25, Marc Weber wrote: Excerpts from Eelco Dolstra's message of Mon Nov 19 11:01:39 +0100 2012: No, because Nix's fixed-output derivation feature requires a md5/sha1/sha256 hash of the expected contents. I know what the current implementation requires. Just wondering whether this should be relaxed for git (like) VCS sources, because they naturally have a hash. No. fetchgit won't work if it's not a fixed-output derivation, because it won't necessarily have network access (it might run in a chroot). -- Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/ ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Excerpts from Eelco Dolstra's message of Mon Nov 19 11:36:00 +0100 2012: No. fetchgit won't work if it's not a fixed-output derivation, because it won't necessarily have network access (it might run in a chroot). Again: I'm not talking about the current state. I'm aware about how it works. I'm talking about: Does it make sense to introduce a special fixed hash for git repos or what about implementing git checkouts natively so that passing the git's hash is enough? git sources are very common today. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
- Original message - Excerpts from Eelco Dolstra's message of Mon Nov 19 11:36:00 +0100 2012: No. fetchgit won't work if it's not a fixed-output derivation, because it won't necessarily have network access (it might run in a chroot). Again: I'm not talking about the current state. I'm aware about how it works. I'm talking about: Does it make sense to introduce a special fixed hash for git repos or what about implementing git checkouts natively so that passing the git's hash is enough? git sources are very common today. True, simplification sounds like a good idea. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Is it terribly difficult to run nix-prefetch-git? Built-in vcs-specific support doesn't strike me as simplification. On Nov 19, 2012, at 7:10 AM, Joachim Schiele j...@lastlog.de wrote: - Original message - Excerpts from Eelco Dolstra's message of Mon Nov 19 11:36:00 +0100 2012: No. fetchgit won't work if it's not a fixed-output derivation, because it won't necessarily have network access (it might run in a chroot). Again: I'm not talking about the current state. I'm aware about how it works. I'm talking about: Does it make sense to introduce a special fixed hash for git repos or what about implementing git checkouts natively so that passing the git's hash is enough? git sources are very common today. True, simplification sounds like a good idea. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Could fetchgit handle that on its own though? Also, at least for github, if you want to install a specific tag, which isn't always the case, you can link to the .zip copy of it from the github page. /M Shea Levy s...@shealevy.com writes: Is it terribly difficult to run nix-prefetch-git? Built-in vcs-specific support doesn't strike me as simplification. On Nov 19, 2012, at 7:10 AM, Joachim Schiele j...@lastlog.de wrote: - Original message - Excerpts from Eelco Dolstra's message of Mon Nov 19 11:36:00 +0100 2012: No. fetchgit won't work if it's not a fixed-output derivation, because it won't necessarily have network access (it might run in a chroot). Again: I'm not talking about the current state. I'm aware about how it works. I'm talking about: Does it make sense to introduce a special fixed hash for git repos or what about implementing git checkouts natively so that passing the git's hash is enough? git sources are very common today. True, simplification sounds like a good idea. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Excerpts from Shea Levy's message of Mon Nov 19 13:38:37 +0100 2012: Is it terribly difficult to run nix-prefetch-git? YES: I'm talking about such configurations: http://gembundler.com/ And here you have git repo and hash. Trying to semi automatically package such things requires much overhead if you have to prefetch everything to get a sha256 hash. I'm not talking about the one project you do package once in a year. I'm talking about 20 small ruby gem packages you need to get some bleeding edge code working. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Marc Weber marco-owe...@gmx.de writes: Excerpts from Shea Levy's message of Mon Nov 19 13:38:37 +0100 2012: Is it terribly difficult to run nix-prefetch-git? YES: I'm talking about such configurations: http://gembundler.com/ And here you have git repo and hash. Trying to semi automatically package such things requires much overhead if you have to prefetch everything to get a sha256 hash. I'm not talking about the one project you do package once in a year. I'm talking about 20 small ruby gem packages you need to get some bleeding edge code working. Have you looked at Shea's npm2nix utility for node.js packages? It's really not that big/scary. Just give it the name of an npm package and it outputs a nix expression (including sha256) for that package, including its dependencies. A similar solution for rubygems would probably not be too hard. As rubygems itself is written in ruby, you can probably plug in to its dependency resolution and downloading capabilities so you can focus on generating the sha256 and the nix expression. Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
A similar solution for rubygems would probably not be too hard. As rubygems itself is written in ruby, you can probably plug in to its dependency resolution and downloading capabilities so you can focus on generating the sha256 and the nix expression. If you still haven't got it: I worte nixpkgs-ruby-overlay which already does it. I also wrote hack-nix packaging all hackage - and I did so after having disregarder a 80% working attempt doing it the nodejs way. I'm looking for packaging fast changing dev versions of packages. And then I don't want to wait for any double fetches. I want to give code a try. I know what I want and why. I accept that the nix community eventually things differently about this. So this may just end up being another patch in my github repos. Maybe I have to use standard ubuntu distribution cause cause I may not have time to finish all this in time (yet) Marc Weber ___ nix-dev mailing list nix-dev@lists.science.uu.nl http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Excerpts from Eelco Dolstra's message of Mon Nov 19 16:31:26 +0100 2012:
Why would you need a double fetch? After running fetchgit, the Git tree is
in
the Nix store and shouldn't be downloaded again unless you do a garbage
collect
in between.
You're right about this.
I want to make bundler (which dynamically fetches updates for
dependencies of ruby packages) use the nix store to share git sources
and gem install results.
nixpkgs-ruby-overlay gets the job done, and I could manually package all
git sources additionally to the packages found on rubyforge. It just
takes too long.
I want to work like other ruby using people do:
bundle update (fetch all dependencies, and if this was done previously reuse
store paths)
Of course running nix-prefetch-git is an option, however checking
whether a store path representing { url = ..; hash = .. } already exists
is harder. If you run nix-prefetch-git twice it will fetch twice
(waste). I haven't looked for options.
If nix could handle this, I could just create a .nix file and I'd always
get what I want: the source - if it exists I would not have to bother at
all.
About changeroot builds: You're right. So mabye a hacky
mkDerivation {
allownetwork = true;
}
would do. It could be used for such cases. Why should it be allowed?
If a programmer wants to shoot himself into the food, you can't prevent
him doing so. Thus the goal should be making it hard to do it by
accident. And this property still holds if allownetwork = true or such
existed.
So comment on whether you see huge security risks using git url and
git's hash only.
Also mind that I don't say that sha256 checks for fetchgit should no
longer be used. I just think its not worth bothering for use cases where
other tools neither do (such as bundler for ruby) - they don't even
bother to use the full git hash length (which is bad IMHO).
Marc Weber
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
Of course running nix-prefetch-git is an option, however checking
whether a store path representing { url = ..; hash = .. } already exists
is harder. If you run nix-prefetch-git twice it will fetch twice
(waste). I haven't looked for options.
nix-store --check-validity $(nix-store -q --outputs $(nix-instantiate
expression.nix -A src))
?
Also, I do use fresh checkouts as src for various Nix expressions. I
just added a repository set to chroot-accessible locations and do what
you say (telling only git hashes to Nix).
If nix could handle this, I could just create a .nix file and I'd always
get what I want: the source - if it exists I would not have to bother at
all.
So comment on whether you see huge security risks using git url and
git's hash only.
It is not so much security risks as it is about special case being a
separate source of bugs.
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
On Sun, Nov 18, 2012 at 10:11 PM, Marc Weber marco-owe...@gmx.de wrote:
Isn't it enough to depend on the git's hash value, eg
fetchgit { git_hash = xxx; url = yyy; }
Is compromising a git repository (even using shallow clones) that much
easier than compromising a .tar.* file protected by sha256?
That would be better because there is no trivial way to check the
sha256 when making the Nix expression.
How does git distinguish a branchnamed after a revision?
--
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
Re: [Nix-dev] fetchgit - why sha256 protection?
On Sun, Nov 18, 2012 at 10:24 PM, Nicolas Pierron
nicolas.b.pier...@gmail.com wrote:
On Sun, Nov 18, 2012 at 10:11 PM, Marc Weber marco-owe...@gmx.de wrote:
Isn't it enough to depend on the git's hash value, eg
fetchgit { git_hash = xxx; url = yyy; }
Is compromising a git repository (even using shallow clones) that much
easier than compromising a .tar.* file protected by sha256?
That would be better because there is no trivial way to check the
sha256 when making the Nix expression.
How does git distinguish a branchnamed after a revision?
We should also enforce that provided hashes have all digits, to
prevent easier attack.
--
Nicolas Pierron
http://www.linkedin.com/in/nicolasbpierron - http://nbp.name/
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
