Jordan K. Hubbard wrote:
This is what RPM does already... rpmbuild(8) builds, rpm(8) installs.
Building has BuildRequires, Installing has Requires. Otherwise
both use .rpm packages (for digests, signatures, etc). There's
even separate types of binary packages. Some packages have really
long and complex needs for building from source, but installing
the binary does not. So having "foo" and "foo-devel" helps cut
down the number of dependencies, when not always wanting to
install the developer files too (like done with ports). Libraries
can also be split off from programs, same reason.
Yep, RPM already has this whole ecosystem covered - SRPMs are their
"ports" and the RPMs their packages. However, I think we can also
all agree that the "spec" format for declaring all of this has
always been pretty ugly (I don't know what Jeff has in store for
RPM's future, but right now we have what we have) and we don't
necessarily want to target it without a good reason. Arguably, we
also have a really good build system already, so all of those
portions of RPM would be redundant extra code to carry around, in a
certain sense, and what we really need are just the package
delivery bits so RPM is probably overkill.
I don't think the yaml-based .spec format is _that_ much uglier than
the text-based .port, but that is all handled behind the scenes
anyway since the MacPorts build system stays the same. This was more
about "the package delivery bits". Splitting packages into
subpackages is optional with RPM, not required. OpenPKG doesn't do
it, for instance (they have no *-devel).
But you know where RPM is, if wanted. The previous versions are at
http://rpm4darwin.sourceforge.net/ (or just use "port install rpm")
or http://rpm5.org/files/rpm/rpm-5.2/BINARY/RPM5-20090707.dmg (port
"rpm52"), or build the new stuff for Snow Leopard from source on
http://rpm5.org/sources.php for all the new features that Jeff
mentioned... If wanting more than archives.
Where the only difference to what it does now is that the final
install step is separate from the others, so that you can start
with just "pkg(1)" and "*.xpkg" and not have to do all the rest ?
Correct! More to the point, you should also be able to do
something like: pkg -R -add someCoolPackage.xpkg in order to
connect to a remote HTTP / FTP server somewhere and just grab the
latest version of someCoolPackage for your relevant OS /
architecture, installing any dependencies that someCoolPackage
might have automatically.
This isn't just a weird idea - *BSD's pkg_add command currently
supports this with the -r flag, e.g.:
freebsd8# pkg_add -r bash
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.1-
release/Latest/bash.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.1-
release/All/libiconv-1.13.1_1.tbz... Done.
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.1-
release/All/gettext-0.18_1.tbz... Done.
I can install FreeBSD 8.1 for AMD64/SPARC/PowerPC on the relevant
hardware, for example, and then as the very first thing on the new
box "pkg_add -r bash" and it just works, automatically appending
the architecture and FreeBSD version # to a built-in search path
which is used to search around at ftp://ftp.freebsd.org/pub/FreeBSD/
packages/ until the most applicable bash package can be found (I
think it looks in about 5 places, in decreasing order of specific
relevance). I don't have to know or care which OS revision or
architecture I'm on if the nice package makers over at FreeBSD.org
have taken my needs into account with their package building fleet,
and it's all very convenient and easy to use - my internet
connection is the only bottleneck in quickly populating my system
with the extra software I need.
I don't care very much for pkg_add(1) directly, as it's a bit too
lowlevel for me. I prefer using portinstall(1) from pkgtools, with
the -PP (--use-packages-only) option if I'm looking for binaries...
But yeah it is nice, the main missing features are the package size
or the package contents. But that is easy enough to add with
additional metadata beyond the INDEX (such as FILE_LIST, or MANIFEST)
Archives are different in that they only allow you to shortcut the
"destroot" phase, but do need all of the others. Packages would
need to have all runtime information available, without peeking in
the port. I think you are missing the "activate" step above, but
then again I don't think that step is very useful.
The activate step was a wrinkle I kind of deliberately glossed
over. :-) We'd have to decide, in a package producing world, just
what to do about the different installation types, direct and
image, and perhaps make some hard decisions about perhaps just
supporting one of them. I can certainly see packages being
somewhat polymorphic in how they deal with direct/image, doing
different things at install time (I even have a pretty fair idea as
to how to do it), but it still begs the question as to whether the
additional complexity would really be worth it or if we couldn't
just say "to hell with it, packages are always direct mode
installs!". Food for thought.
That is what the current package targets (pkg, rpm) do, as they don't
have any images. Image mode is only useful if you can have multiple
versions active at the same time, and since you can't they're just
sitting there wasting space. At least the packages are archived and
compressed, but still there without rebuild if you need to reinstall.
I'm not sure why the runtime has to be in the packages themselves,
it seems less redundant to have a small "installer" program
installation contain that ? But either way, it would have been a
"MacPorts Developer" that would offer to build things from ports
and a "MacPorts User" that would simply install pre-made archives.
And for that to work better, archives would need some improvements
(extra data).
I also don't know how far the pkg tool and xpkg format have come,
and I would be the last person to argue for some crufty legacy
system in favor of something truly new and shiny, but if we're a
ways from seeing home-grown tools and people are sick and tired of
hearing me talk about packaging all the time, I would be more than
willing to port the FreeBSD package tools over since I know how
they work and what their OS dependencies are, then enhance archives
to the point to where the +CONTENTS file is actually meaningful and
relevant to the task of installing packages. At that point,
archives really would be packages since you could scrape them all
together and put them on an FTP / HTTP site for use by pkg_add -r,
as demonstrated earlier. I would probably have to contrive a new
Portfile variable for expressing some of the extra metadata which
can't be easily contrived from inspecting ${DESTROOT}, and for some
period of time this would "parallel" the post-install/post-activate
procedures in some of the ports until we could retire one in favor
of the other, but that's also the easiest transition to make which
doesn't break anything.
The pkg(1) tool was just a copy/paste of port(1) tool, with lots of
sections ripped out and a shortcut to handle file arguments (rather
than port names). This was so you could say "pkg install
foo-1.2_3.tgz" or "pkg install http://example.com/foo-1.2_3.tgz";,
rather than "pkg install foo" and use the pkgindex. But it didn't
make it as far as a separate install or anything.
The xpkg format "version 0.2" was added in MacPorts 1.8.0, it would
need similar improvements to +CONTENTS. Porting the needed fields
over from FreeBSD sounds like a good transition, but pkg would
probably still need to embed Tcl for the more complex package
scripts. And yes, it would still all be "archives". The main
difference with xpkg was the XML...
It just seems like a lot of work, making a new package manager and
reinventing digests and signatures and embedded scripts and
storage formats ?
Not if we start with something existing. See above.
Archives didn't have digests or signatures, but could use something
simplistic like "CHECKSUMS.md5.asc" or "Release.gpg". That is, just
have GNU Coreutils digest all the package containers into one index
and then sign that file with GnuPG, like Slackware or Debian does it.
Or use .rmd160 and openssl, for each archive file ? (like http://
trac.macports.org/browser/trunk/base/src/package1.0/
portarchivefetch.tcl) Depends on what new fields you end up with for
the pkgdata and pkgindex, I suppose...
Well, the increasing popularity of HomeBrew probably says something
about the popularity of Ruby vs Tcl, but I would also maintain that
most folks writing Portfiles don't even really know they're using
Tcl since the language looks so much like standard english ("frob
some arguments") in its simpler form. I don't think Tcl is the
biggest barrier to entry for most users, simply the lack of pre-
built software which mirrors their experience on other platforms
(including the iPhone :) ).
I prefer my recipes to be markup rather than scripts, but I'm OK with
either scripting language (Sh, Tcl, Ruby) for the build scripts.
It's not needed for "base" to be in Tcl too, just because the
"dports" tree is, but I think that is also being fixed now (with
Objective-C).
And I didn't think MacPorts cared about making binary packages…
I think MacPorts "cares" for some value of care, it's just waiting
for someone else to do the work. I'm even volunteering, since I'm
tired of always just ranting about it and then wandering off
again. ;-) Of course, there are always dissenting opinions:
Then again, why *should* MacPorts care about making binary packages
that can be installed without MacPorts ? That was the main argument
for going with archives, as they were "internal".
But actually building binaries, or fixing licensing issues such as
openssl, was much more important than discussing storage formats or
logging formats or bikeshed colors. Thus: archives.
On Sep 21, 2010, at 5:08 AM, Takeshi Enomoto wrote:
Those who prefer binaries could use Fink or Homebrew.
To me MacPorts is an aid to build open source software.
It is better than free format instructions and binary packages that
could be inconsistent with others.
Even with consistent binaries it would not be easy to provide
every possible combination of variants.
Which to me suggests kind of an admission of defeat. "We can't
provide packages, so why not go use Fink or Homebrew?" I think
that really under-sells what MacPorts is capable of, and one reason
I *don't* just use Fink or Homebrew has a lot to do with the
quality of the build system and my faith in its ability to create
software which has been compiled in an appropriate "jail" such that
I know it's going to work. I also have a lot more faith in
MacPorts' small army of volunteers keeping the build recipes up to
date and otherwise making sure it works on a reasonable number of
Darwin platforms, I just also want to offer that magic "pkg_add -r"
functionality such that building stuff is more of a step of last
resort.
So the first step would be finishing the MPAB project ?
As to the variants, I would agree that they're harder to create
packages for since you would have to iterate through every possible
package/variant combination to truly generate a package collection
which anyone could use, but that's perhaps where the idea of an "on
demand package builder" makes sense, since you then only have to
build and cache stuff users are actually asking for, and this would
also allow you to keep stats on the most popular packages
(globally) such that people who don't actually know what variants
to ask for can simply look at what other people are using and pick
the most popular variant configuration.
Just building the default variants sounds like a start.
On Sep 21, 2010, at 10:00 AM, Landon J Fuller wrote:
I agree with just about everything Jordan has had to say, but I
also agree that it would be a shame to put the breaks on an
emerging pragmatic solution. As long as MacPorts doesn't *require*
a development environment for end-users, then using signed
archives -- as long as they include the same metadata a port
install would -- seems like a good first step.
Thanks, Landon. I think that first step is actually fairly
straight-forward, and unless Anders would like to pursue pkg(1) +
xpkg format, which I would also enthusiastically support, I'll be
happy to do the work. All this management is making my brain turn
to sludge and I could use the distraction. :-)
Well, MacPorts *does* require Xcode now. (http://trac.macports.org/
changeset/68593) Even if all you want to do is install archives... It
also requires a full ports tree, and even does a "sync" for you
automagically. When using packages you would probably have a
(similar) "pkgindex" rather than the "portindex" (+Portfiles), once
archives have all metadata. The "selfupdate" could simply link to the
download webpage.
I don't think pursuing your xpkg format is worthwhile, IMHO it (.xar
and .xml) doesn't offer anything over the usual (.tar and .txt)
besides worse compression ratios and more complex parsing. Adding pkg
(1) as a complement to port(1) is still doable, but one can just use
"port -b" meanwhile ? Of course, the first thing required for that to
work would be to enable "archivemode" which is currently disabled by
default.
--anders
_______________________________________________
macports-dev mailing list
[email protected]
http://lists.macosforge.org/mailman/listinfo.cgi/macports-dev
