On Mon, May 9, 2011 at 3:05 AM, Ondrej Certik <ond...@certik.cz> wrote: > On Sat, May 7, 2011 at 9:41 PM, Robert Bradshaw > <rober...@math.washington.edu> wrote: >> On Fri, May 6, 2011 at 2:21 PM, Ondrej Certik <ond...@certik.cz> wrote: >>> On Fri, May 6, 2011 at 2:19 PM, Ondrej Certik <ond...@certik.cz> wrote: >>>> On Fri, May 6, 2011 at 11:26 AM, Robert Bradshaw >>>> <rober...@math.washington.edu> wrote: >>>> [...] >>>>> Were I to design the system from scratch, I'd put >>>>> all our code (devel/scripts/...) in a single repo, along with the >>>>> top-level files, and a list of dependencies (spkgs). Building sage >>>>> would fetch (locally or remotely) the dependencies listed and build >>>>> them in such a way that changing the list of dependencies and >>>>> re-building would easily and cheaply reversible. I would probably >>>>> still build my own Python, but may require it (flexible version) as a >>>>> bootstrapping prerequisite. Whether the non-upstream parts of an spkg >>>>> belong in the spkgs or the main repo, I'm not sure, but I'd rather >>>>> *everything* be expressed as commit to a single repository (possibly >>>>> moving a pointer to some new, vanilla upstream source, rather than >>>>> putting all upstream sources in our repo). >>>>> >>>>> If others have similar views, maybe we could move in that direction. >>>> >>>> I actually developed such a system from scratch (it's called Qsnake: >>>> http://qsnake.com) and pretty much followed your paragraph above, so I >>>> pushed all the repos at github: >>>> >>>> https://github.com/qsnake >> >> Cool. >> >>>> for example the Cython repo is here: >>>> >>>> https://github.com/qsnake/cython >>>> >>>> it is actually a fork of the official Cython repo. And then Qsnake is >>>> clever enough, that when it is fetching the sources, and if there is >>>> setup.py and no spkg-install, it creates spkg-install automatically >>>> with "setup.py install" (and other default stuff), and then saves the >>>> cython.spkg package into the spkg/standard/ directory. >>>> >>>> Having all spkg-install scripts in the main repository (so in my case >>>> in this repository: https://github.com/qsnake/qsnake) is a valid idea, >>>> that I have been bouncing around too. I decided not to, to keep things >>>> localized, as sometimes nontrivial modifications are needed to the >>>> upstream packages, and the best way to do such modifications is to >>>> simply create couple git patches. Also for testing, if there is >>>> spkg-install file, committed, I can checkout the git repo for a >>>> particular package and >>>> >>>> qsnake install . >>>> >>>> and it will install the package. And I can debug it easily. >> >> Is it a reversible install? > > No, currently it's just like in Sage. For reversible install, one > would have to redo every single package to be able to take things from > SPKG_LOCAL (=SAGE_LOCAL in Sage), but install into something like > SPKG_INSTALL, so that one can package it and keep track of files.
I could see this done with paths, with every spkg installing into its one versioned directory, and the python/shell/include/library paths set up to have the list of all currently enabled spkgs. > Hand in hand with this go "binary packages". If one can do > "uninstall", then immediately one can start creating automatic binary > packages. That would be super cool. > > I am currently undecided whether to go with this or not. No doubt it > would be useful. If Sage goes this way, than surely we'll follow too. > Otherwise probably not, as compatibility is important. People who know > Sage should find it quite easy to play with Qsnake and vice versa. > Learning a completely new system and how it works is an obstacle. > > Adding uninstall and binary packages will make everything more complex > (imagine installing the wrong binary into incompatible base > install....). Right now it's all just source packages, and possibly > one binary for the whole thing (for each platform), which is > manageable. > >> What about concurrent versions? > > Only if you rename the package. In this I have the same (or similar) > vision as Sage, that is to create a well tested scientific environment > with just one tested version of each package, that works with > everything. Yes, that's what I'm thinking, but it'd be nice to be able to, e.g., try out a new spkg without hosing the entire install (in a possibly irreversible manner). > If there are two incompatible versions, then one should > change the name of the package, e.g. python -> python2 + python3. Then > it can live side by side. Or python2.6p10. >> What about >> dependencies (e.g. if the version of Cython was updated, would all the >> stuff depending on Cython get re-compiled? > > No. Only the other way round -- if you want to install "phaml", that > uses Cython to wrap Fortran, it will also pull in "cython" > automatically (as well as all the other packages). However, since the > dependency tree is known, we can add this feature as well, to > recompile all "rdepends" (reverse dependencies, to use Debian > terminology). That would be useful to get the known-stable environment, regardless of the order you install packages. That's one of the things that's so appealing about http://nixos.org/nix/ >>> Oh, and then people can simply send pull requests to the respective >>> packages directly. So I think it solves lots of the problems raised in >>> this thread. For sage, it would have to move to github though, so it >>> might not be an option. >> >> Probably not. Is it really tied to github, or could any dvcs be plugged in? > > I just made two releases last couple days, one can download it from here: > > https://github.com/qsnake/qsnake/archives/master > > and that is just one big source tarball, and installs completely > locally, no git is needed (it will actually install its own > automatically --- but it is not needed for the build system). Git is > only used to create the big source tarball using "qsnake -d". It can > be easily changed to "hg" in the build system (and then one needs to > move all the packages, which is simple but tedious). I was thinking of something even simpler, where the dependent packages would just be tarballs (at least as an option), so no revision control is needed by the build system at all. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
