[Nix-dev] "Package archeology" in nixpkgs

2014-08-30 Thread Daniel Peebles 
Hi all,

I've had a sudden urge to do some Haskell archeology and that led me
to a question about how we feel "philosophically" about keeping
abandoned projects and old versions of live projects in nixpkgs. I
think it could be valuable to preserve important pieces of Haskell
history (and perhaps other projects) and it seems like nix is uniquely
positioned to be able to do that well. I don't propose keeping all
versions of all the compilers around, but I'd like to pick out key
points in history and preserve them.

In particular, I was thinking of attempting to get the following working:
- HBC: perhaps the original Haskell compiler. I'd probably aim for a
version that implements Haskell 1.4 and one before that standard was
even proposed. Polymorphic map and (++) in Prelude!
- NHC: can build it with HBC
- GHC: the latest version that supports linear implicit parameters,
because they're gone now and I think people should be able to tinker
with them

The nice thing about doing this sort of thing with compilers is that
they tend to not have many dependencies, but I expect I might also
need to package up an old version of yacc for HBC. If it starts
getting too messy I might abandon the project, but I think it could
work fairly nicely. This would also pave the way to exploring other
interesting abandoned projects like fudgets and such.

How do people feel about this? Is it something I should maintain
independently of nixpkgs or does it belong in the main repo?

Thanks,
Dan
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-30 Thread Nikita Karetnikov 
> - NHC: can build it with HBC

According to [1], it can be bootstrapped via C.  If I remember
correctly, I successfully built it this way in the past (outside of
NixOS).

> How do people feel about this?

I think it’s a great idea.  I always wanted to try HBI, which “supports
the full language at the command line.” [2]  But I have no idea where to
get the source.

[1] http://www.haskell.org/nhc98/download.html
[2] http://stackoverflow.com/questions/4084790/compilers-for-haskell/5918876


pgp_aVy7aiE0O.pgp
Description: PGP signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Anderson Torres 
Maybe a secondary git branch would be good to implement it.
Em 31/08/2014 00:32, "Daniel Peebles"  escreveu:

> Hi all,
>
> I've had a sudden urge to do some Haskell archeology and that led me
> to a question about how we feel "philosophically" about keeping
> abandoned projects and old versions of live projects in nixpkgs. I
> think it could be valuable to preserve important pieces of Haskell
> history (and perhaps other projects) and it seems like nix is uniquely
> positioned to be able to do that well. I don't propose keeping all
> versions of all the compilers around, but I'd like to pick out key
> points in history and preserve them.
>
> In particular, I was thinking of attempting to get the following working:
> - HBC: perhaps the original Haskell compiler. I'd probably aim for a
> version that implements Haskell 1.4 and one before that standard was
> even proposed. Polymorphic map and (++) in Prelude!
> - NHC: can build it with HBC
> - GHC: the latest version that supports linear implicit parameters,
> because they're gone now and I think people should be able to tinker
> with them
>
> The nice thing about doing this sort of thing with compilers is that
> they tend to not have many dependencies, but I expect I might also
> need to package up an old version of yacc for HBC. If it starts
> getting too messy I might abandon the project, but I think it could
> work fairly nicely. This would also pave the way to exploring other
> interesting abandoned projects like fudgets and such.
>
> How do people feel about this? Is it something I should maintain
> independently of nixpkgs or does it belong in the main repo?
>
> Thanks,
> Dan
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Ertugrul Söylemez 
On Sat, 30 Aug 2014 23:31:42 -0400
Daniel Peebles  wrote:

> I've had a sudden urge to do some Haskell archeology and that led me
> to a question about how we feel "philosophically" about keeping
> abandoned projects and old versions of live projects in nixpkgs.

I think it's a very good idea, but ...
 

> How do people feel about this? Is it something I should maintain
> independently of nixpkgs or does it belong in the main repo?

... please keep it out of the main Nixpkgs, at least for now.  Reason
is that currently Nix performance drops linearly with the number of
packages.  Until we have a caching solution of some sort, I'd prefer to
keep Nixpkgs slim.


Greets,
Ertugrul

-- 
Ertugrul Söylemez 
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Michael Raskin 
>> How do people feel about this? Is it something I should maintain
>> independently of nixpkgs or does it belong in the main repo?
>
>... please keep it out of the main Nixpkgs, at least for now.  Reason
>is that currently Nix performance drops linearly with the number of
>packages.  Until we have a caching solution of some sort, I'd prefer to
>keep Nixpkgs slim.

I'd say, though, that only name-based operations drop in performance 
that way…

Although for some strange reason we still recommend this way of using 
Nix.



___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Tobias Geerinckx-Rice 
On 31 August 2014 18:27, Michael Raskin <7c6f4...@mail.ru> wrote:
> I'd say, though, that only name-based operations drop in performance
> that way…
>
> Although for some strange reason we still recommend this way of using
> Nix.

Nix newb asks: what would be the superiour alternative, then?

Regards,

T G-R
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Michael Raskin 
>On 31 August 2014 18:27, Michael Raskin <7c6f4...@mail.ru> wrote:
>> I'd say, though, that only name-based operations drop in performance
>> that way…
>>
>> Although for some strange reason we still recommend this way of using
>> Nix.
>
>Nix newb asks: what would be the superiour alternative, then?

nix-env -iA insted of nix-env -i

The difference is that you use attribute name, not package name; lookup
by attribute name allows to read only all-packages.nix and relevant 
sources, not entire NixPkgs tree.



___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Roger Qiu 
Can there be a better database rather than a text file?

On 1/09/2014 4:04 AM, Michael Raskin wrote:
>> On 31 August 2014 18:27, Michael Raskin <7c6f4...@mail.ru> wrote:
>>> I'd say, though, that only name-based operations drop in performance
>>> that way…
>>>
>>> Although for some strange reason we still recommend this way of using
>>> Nix.
>> Nix newb asks: what would be the superiour alternative, then?
> nix-env -iA insted of nix-env -i
>
> The difference is that you use attribute name, not package name; lookup
> by attribute name allows to read only all-packages.nix and relevant
> sources, not entire NixPkgs tree.
>
>
>
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-08-31 Thread Michael Raskin 
>Can there be a better database rather than a text file?

The problem is not about a _single_ text file; the problem is that you
need to read some file specific to a package to find out its name, as 
opposed to its attribute name,

I am pretty sure we don't want to mention package versions in
all-packages.nix because the version in name will never match the actual
tarball version…

So the solution would require some way of careful caching (note that to
find updates reliably you need to stat() all the files — so caching has
to sometimes lag behind the actual files…)

>On 1/09/2014 4:04 AM, Michael Raskin wrote:
>>> On 31 August 2014 18:27, Michael Raskin <7c6f4...@mail.ru> wrote:
 I'd say, though, that only name-based operations drop in performance
 that way…

 Although for some strange reason we still recommend this way of using
 Nix.
>>> Nix newb asks: what would be the superiour alternative, then?
>> nix-env -iA insted of nix-env -i
>>
>> The difference is that you use attribute name, not package name; lookup
>> by attribute name allows to read only all-packages.nix and relevant
>> sources, not entire NixPkgs tree.



___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-01 Thread Michael Raskin 
>But with a proper database implementation, perhaps it can do full text 
>indexing on its description, or indexing on the names, and proper 
>caching mechanisms as well.
>
>Why reinvent the wheel?

What triggers reindexing?

Package name is a evaluated attribute, result of running a minimal 
expression.

Do you offer to store source code in a database? That clashes with 
versioning…

>On 1/09/2014 3:04 PM, Michael Raskin wrote:
>>> Can there be a better database rather than a text file?
>> The problem is not about a _single_ text file; the problem is that you
>> need to read some file specific to a package to find out its name, as
>> opposed to its attribute name,
>>
>> I am pretty sure we don't want to mention package versions in
>> all-packages.nix because the version in name will never match the actual
>> tarball version…
>>
>> So the solution would require some way of careful caching (note that to
>> find updates reliably you need to stat() all the files — so caching has
>> to sometimes lag behind the actual files…)
>>
>>> On 1/09/2014 4:04 AM, Michael Raskin wrote:
> On 31 August 2014 18:27, Michael Raskin <7c6f4...@mail.ru> wrote:
>> I'd say, though, that only name-based operations drop in performance
>> that way…
>>
>> Although for some strange reason we still recommend this way of using
>> Nix.
> Nix newb asks: what would be the superiour alternative, then?
 nix-env -iA insted of nix-env -i

 The difference is that you use attribute name, not package name; lookup
 by attribute name allows to read only all-packages.nix and relevant
 sources, not entire NixPkgs tree.



___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-01 Thread Vladimír Čunát 

On 09/01/2014 11:24 AM, Michael Raskin wrote:

What triggers reindexing?


I suggest we distribute the database with the channel, similarly to the 
command-not-found database. IMO it is mostly channel users that use 
installing by name, not developers who build from modified git tree.


Vladimir




smime.p7s
Description: S/MIME Cryptographic Signature
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-01 Thread Florent Becker 

On 01/09/2014 11:22, Vladimír Čunát wrote:
 > I suggest we distribute the database with the channel, similarly
 > to the command-not-found database.

Can't command-not-found suggest using nix-env -iA rather than nix-env 
-i? That would be one indirection less for that use case, and teach new 
users about nix-env -iA.

--
Florent


___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-01 Thread Wout Mertens 
Don't forget the nix-env -u case which updates based on name (in fact, that
kinda sucks for sub-attributes like python packages, there are lots of
attributes mapping to the same names).

Wout.


On Mon, Sep 1, 2014 at 10:44 PM, Florent Becker  wrote:

>
> On 01/09/2014 11:22, Vladimír Čunát wrote:
>  > I suggest we distribute the database with the channel, similarly
>  > to the command-not-found database.
>
> Can't command-not-found suggest using nix-env -iA rather than nix-env
> -i? That would be one indirection less for that use case, and teach new
> users about nix-env -iA.
>
> --
> Florent
>
>
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-01 Thread Roger Qiu 
The CI system can support this process, every time a new package is 
built, its then triggers a reindex and mutation of the database, which 
is then supplied via the channel.


On 1/09/2014 7:22 PM, Vladimír C(unát wrote:

On 09/01/2014 11:24 AM, Michael Raskin wrote:

What triggers reindexing?


I suggest we distribute the database with the channel, similarly to 
the command-not-found database. IMO it is mostly channel users that 
use installing by name, not developers who build from modified git tree.


Vladimir




___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-01 Thread Roger Qiu 
I think the name and the attribute name could be named with better 
names, to prevent confusion. Perhaps a "label" and "path"/"fully 
qualified path". The name is just really used for fuzzy search which I 
believe is there for convenience for the operator. Like the difference 
between a search engine term, and the actual domain name. If the number 
of packages (and package versions) keeps growing, full indexing will be 
needed to maintain performance in fuzzy operations.


On 2/09/2014 6:47 AM, Wout Mertens wrote:
Don't forget the nix-env -u case which updates based on name (in fact, 
that kinda sucks for sub-attributes like python packages, there are 
lots of attributes mapping to the same names).


Wout.


On Mon, Sep 1, 2014 at 10:44 PM, Florent Becker 
mailto:florent.bec...@ens-lyon.org>> wrote:



On 01/09/2014 11:22, Vladimír C(unát wrote:
 > I suggest we distribute the database with the channel, similarly
 > to the command-not-found database.

Can't command-not-found suggest using nix-env -iA rather than nix-env
-i? That would be one indirection less for that use case, and
teach new
users about nix-env -iA.

--
Florent


___
nix-dev mailing list
nix-dev@lists.science.uu.nl 
http://lists.science.uu.nl/mailman/listinfo/nix-dev




___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev


___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-02 Thread Roger Qiu 
But with a proper database implementation, perhaps it can do full text 
indexing on its description, or indexing on the names, and proper 
caching mechanisms as well.

Why reinvent the wheel?

On 1/09/2014 3:04 PM, Michael Raskin wrote:
>> Can there be a better database rather than a text file?
> The problem is not about a _single_ text file; the problem is that you
> need to read some file specific to a package to find out its name, as
> opposed to its attribute name,
>
> I am pretty sure we don't want to mention package versions in
> all-packages.nix because the version in name will never match the actual
> tarball version…
>
> So the solution would require some way of careful caching (note that to
> find updates reliably you need to stat() all the files — so caching has
> to sometimes lag behind the actual files…)
>
>> On 1/09/2014 4:04 AM, Michael Raskin wrote:
 On 31 August 2014 18:27, Michael Raskin <7c6f4...@mail.ru> wrote:
> I'd say, though, that only name-based operations drop in performance
> that way…
>
> Although for some strange reason we still recommend this way of using
> Nix.
 Nix newb asks: what would be the superiour alternative, then?
>>> nix-env -iA insted of nix-env -i
>>>
>>> The difference is that you use attribute name, not package name; lookup
>>> by attribute name allows to read only all-packages.nix and relevant
>>> sources, not entire NixPkgs tree.
>
>

-- 
Founder of Polycademy & SnapSearch
http://polycademy.com
https://snapsearch.io
+61420925975

___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-02 Thread Michael Raskin 
>But with a proper database implementation, perhaps it can do full text 
>indexing on its description, or indexing on the names, and proper 
>caching mechanisms as well.
>
>Why reinvent the wheel?

Because every package is a piece of code in Nix programming language.



___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-02 Thread Ganesh Sittampalam 
On 31/08/2014 04:31, Daniel Peebles wrote:

> I've had a sudden urge to do some Haskell archeology and that led me
> to a question about how we feel "philosophically" about keeping
> abandoned projects and old versions of live projects in nixpkgs. I
> think it could be valuable to preserve important pieces of Haskell
> history (and perhaps other projects) and it seems like nix is uniquely
> positioned to be able to do that well. I don't propose keeping all
> versions of all the compilers around, but I'd like to pick out key
> points in history and preserve them.

On a related note, I've thought that nix would be a great way of trying
to bootstrap things like GHC from the original version (or at least the
oldest version we can find).

It's also sometimes useful to have old compilers around for testing,
though nixpkgs does go back a reasonable amount itself.

Ganesh
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev

Re: [Nix-dev] "Package archeology" in nixpkgs

2014-09-03 Thread Daniel Peebles 
That was actually my original motivation for doing this. I looked for the 
oldest GHC I could find (0.26) and it still wanted to be compiled by itself, 
although it said that HBC could do it if you put in more effort. I was hoping 
to get the binary bootstrapping out of the GHC build pipeline under the 
assumption that binary caching would work in largely the same way after an 
initial slow build. Still not sure if adding several additional intermediate 
versions of GHC to nixpkgs would be the best idea though :)

> On Sep 3, 2014, at 2:31, Ganesh Sittampalam  wrote:
> 
>> On 31/08/2014 04:31, Daniel Peebles wrote:
>> 
>> I've had a sudden urge to do some Haskell archeology and that led me
>> to a question about how we feel "philosophically" about keeping
>> abandoned projects and old versions of live projects in nixpkgs. I
>> think it could be valuable to preserve important pieces of Haskell
>> history (and perhaps other projects) and it seems like nix is uniquely
>> positioned to be able to do that well. I don't propose keeping all
>> versions of all the compilers around, but I'd like to pick out key
>> points in history and preserve them.
> 
> On a related note, I've thought that nix would be a great way of trying
> to bootstrap things like GHC from the original version (or at least the
> oldest version we can find).
> 
> It's also sometimes useful to have old compilers around for testing,
> though nixpkgs does go back a reasonable amount itself.
> 
> Ganesh
> ___
> nix-dev mailing list
> nix-dev@lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
___
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev