Hi, After sleeping on it for quite a long time, I finally took the time to prepare a prototype tarball:
http://sage.math.washington.edu/home/burcin/sage-prefix.tar Despite the name, ATM it only installs Singular with it's dependencies. Only linux platforms that already have python > 2.6 are expected to work. This doesn't solve any of the problems mentioned before about using gentoo-prefix for the Sage build system. It was merely an exercise for me to see the scope of the problems. :) Details of how it works are at the end of this message. On Wed, 13 Apr 2011 05:34:48 -0700 (PDT) "Georg S. Weber" <georgswe...@googlemail.com> wrote: > Hi all, > > thanks for this highly motivating thread! It seems there are a handful > people interested, and I'm happy to join in. My goal might seem at > first pretty moderate: Identify the mutual benefits of Sage and Gentoo > (especially Gentoo Prefix), and write this down somewhere at an easily > accessible place (e.g. the Sage Wiki, maybe as some "SEP" --- Sage > Enhancement Proposal). We should take great care that we do write down > and communicate the discussions, results, proposals agreed (at least > among the participants) upon, sample code/installations, links, and > such. Here is a list of pros & cons from my (Sage centered) point of view: Pros: - flexible, very configurable source based package manager, extends the current system with: - uninstall packages - binary packages - separation of build time and run time dependencies - properly handles CFLAGS, etc. - separates source, metadata, and build instructions so it is easier to update packages - many packages already available - in the gentoo portage tree [http://gentoo-portage.com/sci-mathematics] - sage-on-gentoo overlay [https://github.com/cschwan/sage-on-gentoo] - gentoo-science overlay [http://git.overlays.gentoo.org/gitweb/?p=proj/sci.git;a=tree] - makes it easier to create custom distributions, which include different sets of default packages - reduces optional/experimental package maintenance - it is possible to specify what is already available on different platforms (we don't need to distribute gfortran, ATLAS, etc. if not needed) See for example: http://sage.math.washington.edu/home/burcin/portage/profiles/sage/linux/package.provided Cons: - portage has a lot of dependencies - python, sed, bash, etc. This makes bootstrapping nontrivial and makes the distribution bigger - package creation process involves learning portage conventions [http://devmanual.gentoo.org/index.html] - still a lot of open ends - easy development - relocating an install - automating maintenance of prefix portage tree > Although Gentoo Prefix is by far more mature than Sage (and so the > benefits for Sage are surely much greater, than possible synergies for > Gentoo Prefix), I think I can identify some points where Gentoo Prefix > might eventually profit, too. See e.g. the Gentoo (Alt) Prefix mailing > thread containing the following message (Fabian Groffen does know what > he talks about, but it would be good to have at least the hard facts > clearly documented, and easily accessible): > > > http://archives.gentoo.org/gentoo-alt/msg_3e0aa31ec42c917484e6b0f86ffa5f2a.xml Since we are not aiming to provide a complete system with a separate compiler tool chain and libraries, I don't think much of this applies to us. Sage already manages to relocate a binary install, so we know that it is not impossible to do this in our restricted use case. :) > or the current lack of Gentoo Prefix "binary distributions" resp. the > need of external support to do the "very first start up" (since > bootstrapping ain't really for the faint-hearted --- I did that myself > successfully on Maemo5, OS X 10.4 and OS X 10.6, but I'm a developer, > not a "mere user"), see e.g.: > > http://sourceforge.net/projects/prefix-launcher/ IMHO, bootstrapping is still the biggest problem. Though at least we don't have to worry about building a tool chain. > One last thing about "sage -clone, -b, -ba, br, ..." --- from my > memory about several discussions, the sage community regards this as a > "killer feature", i.e. that there is not some "sage" and then some > other, different "sage-dev" package/distribution, but only one. So the > hurdle for any user to gradually become a developer is made as low as > possible. I have some rough ideas how to make this possible "inside" a > standard distribution with also an eye on reliable package management, > i.e. not restricted to Gentoo/Gentoo Prefix, but also Debian and so > on. But these ideas would need thorough discussion. I agree that this is a killer feature. The prototype tarball does nothing about this problem. As I said earlier, I think we can reconfigure portage to use "devel/*" as the build directory, and hook up some internal portage commands to handle "make install" etc. Though I haven't tried any of this and I'd appreciate hearing any other ideas. ------------------------- Notes about the prefix tarball: - To work around the bootstrapping problem, I assume that python and the basic tools like bash (maybe also sed) are already installed on the system. Of course this works perfectly well on my machines which already run gentoo and portage (the package manager) is installed on the base system. :) IIRC, there were messages on this list about having python as a dependency already. What are the dependencies of Sage besides tar, gcc and gfortran? Does Sage depend on bash? Is python available on the default install of all the systems supported by Sage at the moment? Which version? Unfortunately portage expects python-2.6 or newer. Using pkgcore or paludis instead might solve this problem though. pkgcore: http://www.pkgcore.org/ paludis: http://paludis.pioto.org/ Files in the tarball: - install.sh script to handle bootstrapping (= installing portage) then call portage to install singular - local/ this is the real prefix (in the gentoo sense), the root of the install, named this way to replicate the directory structure of a Sage tree. The symlink 'usr' in this directory also serves this purporse. - dist/ directory which includes package management related stuff, modeled after the 'spkg' directory. - portage/ the ebuild database for portage, there is a version checked into mercurial in my home directory on sage.math: http://sage.math.washington.edu/home/burcin/portage/ Though I only started using mercurial at the end, after realizing this is much more complicated than I expected. :) - scripts/ contains the scripts used for the prefix tarball - install.sh is the one you see at the root of the tarball - bootstrap-prefix.sh script borrowed from gentoo-prefix, commented out most lines that actually run code, functions from this are called in bootstrap-sage.sh - bootstrap-sage.sh redefines the paths and necessary functions from bootstrap-prefix.sh to conform to our layout Two important functions: - bootstrap_tree() creates the directory tree found in the tarball - bootstrap_setup() This is called from bootstrap_portage while installing portage in the first (and only) step of bootstrapping sets a default profile for portage based on the detected platform. I kept only the linux platforms from the ones defined in bootstrap-prefix - profiles/ "profile" definitions, mostly copied from the gentoo-prefix tree The only new part is the sage/ directory: http://sage.math.washington.edu/home/burcin/portage/profiles/sage/ This adds two new profiles that inherit from the corresponding prefix ones (see the files named 'parent'), These profiles define architecture keywords. The package manager uses these to decide which packages should be considered stable for a given platform. The packages which should be considered already installed are listed in the file packages.provided: http://sage.math.washington.edu/home/burcin/portage/profiles/sage/linux/package.provided How does it work? - The install script - sets the PATH and EPREFIX variables EPREFIX is used to indicate the root directory for the prefix install in the gentoo-prefix project - installs portage in this EPREFIX "by hand" without using a package manager. This is done by the bootstrap_portage() function in bootstrap-prefix.sh. - during the portage install, it detects the "profile" needed, by looking a the CHOST variable. The profile, as explained above, lets the package manager know the architecture etc. - install.sh also puts a configuration file for portage in the prefix. gentoo-prefix distributes a modified the portage tarball with special default configuration. I didn't want to do that again, or patch portage during the install. When the package manager is installed properly below, the global configuration is patched as necessary. See: http://sage.math.washington.edu/home/burcin/portage/sys-apps/portage/files/make.globals.patch So the configuration file is overwritten after that step. - installs Volker's compiler wrapper using portage AFAICT, this wasn't absolutely necessary since I didn't try relocating the tree after the build. However, I can see that the singular built in the prefix uses the ntl libraries installed in the prefix instead of the ones on my system. This is an improvement over the current Sage build I have. - calls portage to install itself properly This pulls in: app-shells/bash sys-libs/zlib app-admin/eselect virtual/libintl virtual/libffi app-admin/eselect-python dev-lang/python sys-apps/portage app-admin/python-updater - calls portage to install baselayout-prefix This is the package that provides files for the environment variables, etc. in the prefix. this pulls in: sys-libs/readline sys-apps/baselayout-prefix I don't know why readline is built here and not as a dependency of python. Perhaps there is a missing dependency in the gentoo ebuilds. - calls portage to install singular This pulls in: dev-libs/gmp dev-libs/gf2x dev-libs/ntl sci-mathematics/singular - puts in a startprefix script when it's done This starts a new shell with the PATH, etc. set. To try it out, you just need to unpack the tar and run install.sh. Then the startprefix script will get you a shell inside the prefix, where you can run Singular. Comments? Cheers, Burcin -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
