Re: new package system proposal
Sorry to jump in late.
On Saturday 04 April 2009 16:13:22 Chris Whitehouse wrote:
pkg_add somewhat addresses this but it doesn't work quite as well as
ports because of possible version mismatches.
The suggestion below is not aimed at servers because they have
completely different requirements.
My suggestion is to start with a ports tree that is fixed in time. Make
that ports tree available as part of this package system and compile a
typical desktop set of ports, particularly choosing ones which are large
or have many dependencies. When it is all complete release it and start
again. Surely quite a wide selection of desktops, wm's and apps could be
compiled in a couple of weeks?
Modify pkg_add so that it can be told to use this 'snapshot' including
downloading the fixed ports tree that was used.
I've already thought this through and you're going about it the wrong way.
Version/package mismatches are caused by two things:
1) The buildservers do not repack dependants of an updated port
2) Installed slave ports as dependency cannot be identified by a package that
has the master port built in. Ex: if installing kdelibs3-nocups through
pkg_add, anything using it will download kdelibs3, try to install and fail.
The first is easy to solve, by iterating through the built packages and
repacking them after reinstallation so that +CONTENTS is updated correctly
with the new dependency. I've written a tool pkgsync which does that, but
there's still some corner cases I need to fix and I'm not happy with the time
it takes to read deps from 800+ packages, so might consider using aio(4) or
fixing up my algorithm.
The second case is more difficult, as it requires flexible dependencies.
I've fixed this myself, by hacking port Makefiles to correctly set the
dependency origin, but this only applies to one build (meaning: my
buildserver only makes packages with a fixed set of slave ports, if I'd want
to use CUPS, I'd have to create a seperate buildserver or jail).
The most important thing is that you don't need a snapshot ports tree if 1) is
met. What you also need for binary upgrading is an index format that doesn't
need a ports tree and is consistent with what is actually available on the
given build server.
I've solved this like so:
#!/bin/sh
# vim: ts=4 sw=4 noet ai tw=78
# $Coar: pkgtools/pkgindex/pkgindex.sh,v 1.5 2008/04/17 21:43:46 mdev Exp $
AWK=${AWK:=/usr/bin/awk}
PKGDB=${PKGDB:=/var/db/pkg}
PACKAGES=${PACKAGES:=/home/packages-6}
PKGINDEX=${PKGINDEX:=${PACKAGES}/All/INDEX.bz2}
TMPINDEX=/tmp/INDEX.$$
# clean up a previous halted version that had the same pid
[ -f ${TMPINDEX} ] rm -f ${TMPINDEX}
[ -f ${TMPINDEX}.bz2 ] rm -f ${TMPINDEX}.bz2
DONE=0
echo -n Creating new in index in $PACKAGES/All
cd ${PACKAGES}/All
for pkg in *.tbz; do
origin=$(pkgorigin ${pkg})
hash=$(sha256 -q ${pkg})
size=$(ls -aLln ${pkg} | ${AWK} '{ print $5}')
echo $origin:${pkg}:$hash:$size ${TMPINDEX}
DONE=$((${DONE} + 1))
if test $((${DONE} % 10)) -eq 0; then
if test $((${DONE} % 50)) -eq 0; then
echo -n ${DONE}
else
echo -n .
fi
fi
done
echo done.
bzip2 ${TMPINDEX}
mv -v ${TMPINDEX}.bz2 ${PKGINDEX}
This gives you origin, pkgname, sha256 hash and filesize for download
verification. All you need to determine whether a package is eligible for
binary upgrade.
pkgorigin is a utility I wrote that quickly gets an origin out of a package
without the need for a staging directory. Attached as shar if it makes it to
the list.
Btw, I don't think supporting various OPTIONS is a good idea. Port maintainers
should properly use PACKAGE_BUILDING. There's no such thing as good
defaults, so try to satisfy what you think is the majority and redirect
complaints to the infamous bit bucket. Proper use also includes disabling
RESTRICTED and IS_INTERACTIVE ports.
--
Mel
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Bob Johnson wrote: - reduced energy use for everyone. I think the difference in energy use would be so small as to be pointless. If I have a system that consumes 75 kilowatt hours per month, and I spend an extra 0.05 kilowatt hour per month updating ports, is the difference (less than 1/10 of 1 percent) really meaningful? I can't even measure my power usage accurately enough to detect the difference. Convince me to use three liters less hot water per month, and you will save more energy. How do you get the figures above? I measured electricity use for a typical 2 year old computer (excluding screen) as: - computer idling - 80 watts - computer working hard - 125 watts That's a diff of 45 watts. Suppose normally you use your computer 4 hours a day and it normally takes you 20 hours to upgrade your ports. You start the upgrade while using the computer but you leave it compiling for an extra 16 hours. Thats 4 hours at 45 watts plus 16 hours at 125 watts. Thats 2180 watt hours or over 2 kWh on top of your normal use for one port upgrade. However you jig the figures there is no escaping that cpu cycles use energy. Multiply by the number of times ports are upgraded per computer per year and the number of computers being upgraded and a package system seems like a worthwhile saving. (Thanks Colin Percival et al for freebsd-update, this aspect of it hadn't occurred to me) To complete your comparison if you used electricity to heat 3 litres of water from 15 degC to 55 degC you would use 0.139kWh. Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Bob Johnson wrote: On 4/8/09, Jonathan McKeown j.mcke...@ru.ac.za wrote: On Tuesday 07 April 2009 23:35:03 Bob Johnson wrote: On 4/4/09, Chris Whitehouse cwhi...@onetel.com wrote: The drawback I can see is the disk space required to keep several generations of packages online - if the package-port bundle is rebuilt every three weeks, let's say, and you want to keep 6 months' worth of packages online, you need to keep 9 complete versions available. Is there a quick way to find out how big are the tarballs without downloading them all or adding them up one by one? My distfiles directory is 1.2gb. I guess you could allow 5gb for each cycle, that's only 45gb for 6 months. If that is realistic a 500gb hard disk would store 5 years worth. Would anyone want a five year old package? I think a bigger drawback is the security issue. As soon as any package in the collection has a significant announced security flaw, you are faced with the choice of withdrawing the entire collection, withdrawing only that package, or leaving the flawed package out there for people to use because it is more convenient for them. Yes this might be an issue How often are there serious security issues with desktop type ports? PC-BSD seems to already keep up-to-date binary packages of their applications. Do they accomplish that by only offering a small subset of the full ports collection? A big difference with PBI's is that each PBI is self contained with all the files and libraries necessary for the installed program to function (quote from the website). Upside is that it is very easy to install and avoids dependency problems. Downside is that it requires more bandwidth to download and more disk space. I did wonder if it would make sense to just use the PBI system. The number of packages depends to some extent on individuals volunteering to make and maintain them - true FreeBSD style. Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On Fri, 10 Apr 2009 15:25:13 +0100, Chris Whitehouse cwhi...@onetel.com wrote: Is there a quick way to find out how big are the tarballs without downloading them all or adding them up one by one? I think it's possible to obtain an FTP ls listing and then use awk to get the column with the size (in bytes) and add them, printing the final result and maybe converting it into MB, GB if needed. Would anyone want a five year old package? Yes, I would, because today's packages are sooo slooow. :-) Just as an unimportant sidenote: The FreeBSD OS is capable in gaining speed on the same (old) hardware with every release. So I can install it today on a 150 MHz P1 with 128 MB RAM without any problems, and it will run fast. But I cannot use today's applications on that system as I could the older ones, such as StarOffice, older Opera versions, older Mplayer versions, older X-Chat versions etc. because all of them depend on newer libraries (including, excuse me, bloat) that would render the system nearly unusable speed-wise (see the big jump in Gtk, compare usability and speed of X-Chat 1 vs. X-Chat 2, or the transition of Sylpheed from Gtk 1 toolset to Gtk 2 toolset). Disk occupation is, of course, another topic. If you've got only a 4 GB hard disk which could hold a fully functional and feature-rich system of FreeBSD 5, it's hard to achieve this with FreeBSD 7 and its set of applications because of the many dependencies (just have a look at how Gtk and Gnome stuff can fill your hard disk, maybe you want to use gmplayer only). A big difference with PBI's is that each PBI is self contained with all the files and libraries necessary for the installed program to function (quote from the website). Upside is that it is very easy to install and avoids dependency problems. Downside is that it requires more bandwidth to download and more disk space. Another problem is that if a minor (but important) library change appears that does require the update of the library, but not of the dependent applications, that new PBIs have to be built and installed, while the traditional way would suggest to update the library only. But I don't think customers of PC-BSD will be interested in such geek stuff. PC-BSD is fine for average users who install once, use then, update very few times. I did wonder if it would make sense to just use the PBI system. The number of packages depends to some extent on individuals volunteering to make and maintain them - true FreeBSD style. Allthough I prefer the traditional and well intended ways, I could live with PBI as long as there's an automated way to install them, read: command line, ability for batch processing. I simply don't want to waste time for Next, next, okay, next, next, next, reboot. :-) -- Polytropon From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Polytropon wrote: On Fri, 10 Apr 2009 15:25:13 +0100, Chris Whitehouse cwhi...@onetel.com wrote: Is there a quick way to find out how big are the tarballs without downloading them all or adding them up one by one? I think it's possible to obtain an FTP ls listing and then use awk to get the column with the size (in bytes) and add them, printing the final result and maybe converting it into MB, GB if needed. It's in port/distinfo: chr...@pcbsd% grep SIZE distinfo SIZE (vlc-0.8.6i.tar.bz2) = 11786172 Would anyone want a five year old package? Yes, I would, because today's packages are sooo slooow. :-) Well my disk space estimate was way high, I didn't allow for the fact that not all packages would be upgraded each cycle, therefore the disk space requirement would be less. So maybe 500gb would do 10 years. Do you think you will be using 10 year old packages? By then you will have freecycled a computer that will run relatively newer apps. I did wonder if it would make sense to just use the PBI system. The number of packages depends to some extent on individuals volunteering to make and maintain them - true FreeBSD style. Allthough I prefer the traditional and well intended ways, I could live with PBI as long as there's an automated way to install them, read: command line, ability for batch processing. I simply don't want to waste time for Next, next, okay, next, next, next, reboot. :-) It's very easy actually. A widget, if so configured, tells you when there is an update for a PBI. You select it and click ok and that's about it. Reboot? This is not Windoze you know! :0 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Ok here's an improved description of how it works. The key to the whole thing is the snapshot of the ports tree. Everything else follows from that. Build process: A predetermined set of packages is built from a ports tree. The most benefit comes with packages which would acceptable for use with the default config and require a lot of compile time (large distfiles and/or lots of dependencies). This probably means xorg, window managers, desktop environments and applications plus dependencies and libraries. The ports tree is not updated during this process. The ports tree and package tarballs are made available online as a ports-snapshot which is tagged to identify it. The tag could be a text file with a date in it which lives in the snapshot directory. This cycle is repeated at some suitable interval, by taking a new ports tree and building packages for any ports that have been updated in the meantime. (The actual build process could be done by one of the ports-mgmt tools which would take care of only updating changed ports or ports whose dependencies had changed). The new packages and the ports tree used to create them are tagged as a new snapshot. How to use: For a fresh install a user installs FreeBSD then downloads the tagged ports tree. S/he can then install packages from the ports-snapshot knowing that everything is in sync versionwise. At this stage the selection of packages is down to the user (plus dependencies will automatically be pulled in). Additional ports that are not in the snapshot can be installed using the normal ports mechanism and they will automatically be in sync. For an existing install, when a new ports-snapshot is available changed packages can be installed using some utility (call it ports-update) which downloads and installs the ports tree and tag file from the snapshot, checks versions of installed packages and upgrades accordingly. People can choose to upgrade frequently or infrequently. By choosing a tag users can choose to upgrade to the most recent ports-snapshot or an intermediate one. At this stage the selection of packages to upgrade is by ports-update. ports-update compares the version of each installed package with the one in the ports-snapshot. If a newer version exists ports-update installs it. Any installed packages which are not in the ports-snapshot are ignored. If a user wants to install or upgrade ports which are not in the ports-snapshot this can be done with the standard ports mechanism. As the bigger ports are already available as packages in the ports-snapshot any additional ports will be relatively quicker to install by building from source. If a port is being installed or upgraded and it has a dependency which has been installed from the snapshot the dependency port will be the right version and the new port will integrate perfectly. Thanks for reading this far Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On Wednesday 08 April 2009 21:24:00 Bob Johnson wrote: PC-BSD seems to already keep up-to-date binary packages of their applications. Do they accomplish that by only offering a small subset of the full ports collection? Yes - have a look at http://www.pbidir.com/. I installed PC-BSD on a spare machine to investigate it. The first three ports/metaports I tried to install after completing the base setup were emacs, TeTeX and the Psi Jabber/XMPP client. None of those was available, and after seeing how few prebuilt packages there were in all categories, I gave up. My personal view is that PC-BSD gives the end user an impressive and reasonably slick computer-as-appliance with some ability to customise and still stay ``on the path''. For people who need that, PC-BSD is what they need. My feeling, though, is that anyone who finds themselves wanting to install a bunch of stuff from outside the PBI system (in other words, from ports, which are still there under the hood of PC-BSD) will soon want to switch to mainstream FreeBSD. As such PC-BSD has the potential to be an effective ``gateway drug''(!) Jonathan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
I'd like to use this opportunity to generally support this and any other ideas taking direction of making binary installs and upgrades easier and more manageable. I recognize the need for people to configure custom options and compile from ports (that is why any new system *must* be compatible with ports), however, it should be noted that there's a lot of people running simple LAMP servers, almost exclusively using default options, who would greatly benefit from better binary package support. I've already ranted about this (http://lists.freebsd.org/pipermail/freebsd-questions/2008-December/188119.html) in a slightly different context (I talked about -SECURITY equivalent instead of -DESKTOP that the OP suggests) with almost the same idea - make it easy for people who are interested in running stable, secure servers do binary upgrades without the hassle of going through a major system recompile because of, for example, openldap shared library version bump. Regards, -- Nino ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On Thu, 9 Apr 2009 09:16:12 +0200, Jonathan McKeown j.mcke...@ru.ac.za wrote: Yes - have a look at http://www.pbidir.com/. I installed PC-BSD on a spare machine to investigate it. The first three ports/metaports I tried to install after completing the base setup were emacs, TeTeX and the Psi Jabber/XMPP client. None of those was available, and after seeing how few prebuilt packages there were in all categories, I gave up. The problem with PC-BSD is that it concentrates on the average desktop user, read: the usual KDE user. That's why they have lots of KDE stuff available and applications for common productivity uses, as well as multimedia. You made the mistake to choose software that is non-standard. So teTeX? What's this? Who uses teTeX? Go use KOffice, man! :-) I could say something similar about emacs and psi. On PC-BSD, you can always use pkg_add or the ports collection, but it may cause problems to do so. Allthough it's possible, it's adviced to use the PBI installer. My personal view is that PC-BSD gives the end user an impressive and reasonably slick computer-as-appliance with some ability to customise and still stay ``on the path''. For people who need that, PC-BSD is what they need. That seems to bei their goal, yes. My neighbor uses it for some years now and he's completely happy with it. In fact, he isn't interested in FreeBSD, nor does he have fundamental UNIX knowledge, but he likes KDE and the fact that he has not Windows on his machine (with all the advantages this fact implies). My feeling, though, is that anyone who finds themselves wanting to install a bunch of stuff from outside the PBI system (in other words, from ports, which are still there under the hood of PC-BSD) will soon want to switch to mainstream FreeBSD. Take a look at DesktopBSD (their tools are even in the ports collection). They stick to the ports and packages, but added some GUI stuff for installation and administration, without doing a compatibility break as PC-BSD does. As such PC-BSD has the potential to be an effective ``gateway drug''(!) In fact, it has. -- Polytropon From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: new package system proposal
I haven't worked with *nix os's as much as FreeBSD - Well, maybe different flavors of SCO but as far as installing apps and what not mostly FreeBSD. I've installed maybe half a dozen apps and NONE of them took less than 2 - 3 days. Part of that is my slow a$$ test system - and my ignorance, but it seems like a MAJOR hassle for even the simplest thing! It could just be me, but seems like many developers choose to link to other libraries/modules even if they just need one simple function they could build into their source directly. Hence a dependency is needlessly created. Multiply this 500 times and installing a simple app turns into a nightmare. I'm definitely gonna start trying to use more packages than ports, but the port system is necessary and eventually I get stuff working! G -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of n j Sent: Thursday, April 09, 2009 3:15 AM To: User Questions Subject: Re: new package system proposal I'd like to use this opportunity to generally support this and any other ideas taking direction of making binary installs and upgrades easier and more manageable. I recognize the need for people to configure custom options and compile from ports (that is why any new system *must* be compatible with ports), however, it should be noted that there's a lot of people running simple LAMP servers, almost exclusively using default options, who would greatly benefit from better binary package support. I've already ranted about this (http://lists.freebsd.org/pipermail/freebsd-questions/2008-December/1881 19.html) in a slightly different context (I talked about -SECURITY equivalent instead of -DESKTOP that the OP suggests) with almost the same idea - make it easy for people who are interested in running stable, secure servers do binary upgrades without the hassle of going through a major system recompile because of, for example, openldap shared library version bump. Regards, -- Nino ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org font size=1 div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in' /div This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system. /font ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: new package system proposal
This is the kinda B$ I'm talking about. Trying to install krb5 from ports, and after 2 hours (or more) of finding and compiling dependencies and whatever else make does - it aborts! WTF!!! I'm sure when I try to remove heimdal-1.0.1 it will cause more problems that lead to more problems === Installing for krb5-1.6.3_5 === krb5-1.6.3_5 conflicts with installed package(s): heimdal-1.0.1 They install files into the same place. Please remove them first with pkg_delete(1). *** Error code 1 Stop in /usr/ports/security/krb5. *** Error code 1 Stop in /usr/ports/security/krb5. -Original Message- From: Gary Gatten Sent: Thursday, April 09, 2009 10:46 AM To: 'n j'; User Questions Subject: RE: new package system proposal I haven't worked with *nix os's as much as FreeBSD - Well, maybe different flavors of SCO but as far as installing apps and what not mostly FreeBSD. I've installed maybe half a dozen apps and NONE of them took less than 2 - 3 days. Part of that is my slow a$$ test system - and my ignorance, but it seems like a MAJOR hassle for even the simplest thing! It could just be me, but seems like many developers choose to link to other libraries/modules even if they just need one simple function they could build into their source directly. Hence a dependency is needlessly created. Multiply this 500 times and installing a simple app turns into a nightmare. I'm definitely gonna start trying to use more packages than ports, but the port system is necessary and eventually I get stuff working! G -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of n j Sent: Thursday, April 09, 2009 3:15 AM To: User Questions Subject: Re: new package system proposal I'd like to use this opportunity to generally support this and any other ideas taking direction of making binary installs and upgrades easier and more manageable. I recognize the need for people to configure custom options and compile from ports (that is why any new system *must* be compatible with ports), however, it should be noted that there's a lot of people running simple LAMP servers, almost exclusively using default options, who would greatly benefit from better binary package support. I've already ranted about this (http://lists.freebsd.org/pipermail/freebsd-questions/2008-December/1881 19.html) in a slightly different context (I talked about -SECURITY equivalent instead of -DESKTOP that the OP suggests) with almost the same idea - make it easy for people who are interested in running stable, secure servers do binary upgrades without the hassle of going through a major system recompile because of, for example, openldap shared library version bump. Regards, -- Nino ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org font size=1 div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in' /div This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system. /font ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: new package system proposal
As expected. pkg_delete: package 'heimdal-1.0.1' is required by these other packages and may not be deinstalled: freeradius-2.1.3 FreeRADIUS is the WHOLE reason I'm trying to install SAMBA! LOVE this B$!!! -Original Message- From: Gary Gatten Sent: Thursday, April 09, 2009 10:54 AM To: 'n j'; 'User Questions' Subject: RE: new package system proposal This is the kinda B$ I'm talking about. Trying to install krb5 from ports, and after 2 hours (or more) of finding and compiling dependencies and whatever else make does - it aborts! WTF!!! I'm sure when I try to remove heimdal-1.0.1 it will cause more problems that lead to more problems === Installing for krb5-1.6.3_5 === krb5-1.6.3_5 conflicts with installed package(s): heimdal-1.0.1 They install files into the same place. Please remove them first with pkg_delete(1). *** Error code 1 Stop in /usr/ports/security/krb5. *** Error code 1 Stop in /usr/ports/security/krb5. -Original Message- From: Gary Gatten Sent: Thursday, April 09, 2009 10:46 AM To: 'n j'; User Questions Subject: RE: new package system proposal I haven't worked with *nix os's as much as FreeBSD - Well, maybe different flavors of SCO but as far as installing apps and what not mostly FreeBSD. I've installed maybe half a dozen apps and NONE of them took less than 2 - 3 days. Part of that is my slow a$$ test system - and my ignorance, but it seems like a MAJOR hassle for even the simplest thing! It could just be me, but seems like many developers choose to link to other libraries/modules even if they just need one simple function they could build into their source directly. Hence a dependency is needlessly created. Multiply this 500 times and installing a simple app turns into a nightmare. I'm definitely gonna start trying to use more packages than ports, but the port system is necessary and eventually I get stuff working! G -Original Message- From: owner-freebsd-questi...@freebsd.org [mailto:owner-freebsd-questi...@freebsd.org] On Behalf Of n j Sent: Thursday, April 09, 2009 3:15 AM To: User Questions Subject: Re: new package system proposal I'd like to use this opportunity to generally support this and any other ideas taking direction of making binary installs and upgrades easier and more manageable. I recognize the need for people to configure custom options and compile from ports (that is why any new system *must* be compatible with ports), however, it should be noted that there's a lot of people running simple LAMP servers, almost exclusively using default options, who would greatly benefit from better binary package support. I've already ranted about this (http://lists.freebsd.org/pipermail/freebsd-questions/2008-December/1881 19.html) in a slightly different context (I talked about -SECURITY equivalent instead of -DESKTOP that the OP suggests) with almost the same idea - make it easy for people who are interested in running stable, secure servers do binary upgrades without the hassle of going through a major system recompile because of, for example, openldap shared library version bump. Regards, -- Nino ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org font size=1 div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in' /div This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system. /font ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Nino wrote: I'd like to use this opportunity to generally support this and any other ideas taking direction of making binary installs and upgrades easier and more manageable. You may be interested to read http://www.lpthe.jussieu.fr/~talon/freebsdports.html and to consider playing with http://www.lpthe.jussieu.fr/~talon/pkgupgrade -- Michel TALON ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On Tuesday 07 April 2009 23:35:03 Bob Johnson wrote: On 4/4/09, Chris Whitehouse cwhi...@onetel.com wrote: Hi all [...] My suggestion is to start with a ports tree that is fixed in time. Make that ports tree available as part of this package system and compile a typical desktop set of ports, particularly choosing ones which are large or have many dependencies. When it is all complete release it and start again. Surely quite a wide selection of desktops, wm's and apps could be compiled in a couple of weeks? How is it an improvement over the existing tools? I must be missing something, because it sounds to me like you are merely asking that there be more ports made available as packages than are now offered. I think what you're missing is the suggestion to bundle a set of pre-built packages with a snapshot of the ports tree used to build them. Currently it's difficult to mix and match packages and ports because the versions of dependencies are likely to differ between the package and the local version of the ports tree. If you know you have the same ports tree your packages were built from, you can much more easily combine pre-built packages and local builds from source. This has clear advantages. At the moment, unless you're very lucky with your timing, you tend to find that as soon as you want to build one port from source (perhaps to fiddle with the configuration) you have to stop using prebuilt packages altogether. The drawback I can see is the disk space required to keep several generations of packages online - if the package-port bundle is rebuilt every three weeks, let's say, and you want to keep 6 months' worth of packages online, you need to keep 9 complete versions available. Chris's suggestion is certainly more than just a request for more packages, though. Jonathan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On 4/8/09, Jonathan McKeown j.mcke...@ru.ac.za wrote: On Tuesday 07 April 2009 23:35:03 Bob Johnson wrote: On 4/4/09, Chris Whitehouse cwhi...@onetel.com wrote: Hi all [...] My suggestion is to start with a ports tree that is fixed in time. Make that ports tree available as part of this package system and compile a typical desktop set of ports, particularly choosing ones which are large or have many dependencies. When it is all complete release it and start again. Surely quite a wide selection of desktops, wm's and apps could be compiled in a couple of weeks? How is it an improvement over the existing tools? I must be missing something, because it sounds to me like you are merely asking that there be more ports made available as packages than are now offered. I think what you're missing is the suggestion to bundle a set of pre-built packages with a snapshot of the ports tree used to build them. Currently it's difficult to mix and match packages and ports because the versions of dependencies are likely to differ between the package and the local version of the ports tree. If you know you have the same ports tree your packages were built from, you can much more easily combine pre-built packages and local builds from source. OK, I see now. This has clear advantages. At the moment, unless you're very lucky with your timing, you tend to find that as soon as you want to build one port from source (perhaps to fiddle with the configuration) you have to stop using prebuilt packages altogether. I've not really had a lot of trouble with that, although it sometimes causes problems. OK, with something big like KDE it causes problems. The drawback I can see is the disk space required to keep several generations of packages online - if the package-port bundle is rebuilt every three weeks, let's say, and you want to keep 6 months' worth of packages online, you need to keep 9 complete versions available. I think a bigger drawback is the security issue. As soon as any package in the collection has a significant announced security flaw, you are faced with the choice of withdrawing the entire collection, withdrawing only that package, or leaving the flawed package out there for people to use because it is more convenient for them. At the very least, it creates a management headache for whomever has to make the decisions. Chris's suggestion is certainly more than just a request for more packages, though. It seems to me that a great deal of what his suggestion would accomplish would be accomplished by building a very extensive set of packages once a week or so, so that it is easy to do binary updates of anything that needs updating. For many, that should solve the bulk of the problem. And because most ports don't change weekly, the week-to-week changes shouldn't be unmanageably large. That could also be a starting point for implementing his full suggestion. Keeping around week-to-week deltas rather than an entire collection would reduce the storage requirement substantially. PC-BSD seems to already keep up-to-date binary packages of their applications. Do they accomplish that by only offering a small subset of the full ports collection? -- -- Bob Johnson fbsdli...@gmail.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Chris Whitehouse wrote: You've suggested solutions to a couple of Polytropon's objections, thank you. Do you think there is anough mileage in my suggestion to make it worth putting in front of some ports people? What would have to happen to take it forward? I could rewrite the proposal more clearly. Any well-considered proposal is interesting, and suitable fodder for the freebsd-po...@... mailing list. However you must be prepared for your ideas to undergo some fairly rigourous critique by people who have spent a great deal of time in doing exactly the sort of operations you are talking about. It can be pretty daunting -- remember though that it is your /ideas/ that are being dissected: it's not a personal attack against you for having the temerity to try and suggest something. Also, as ever in the FreeBSD world, code speaks louder than words. It's easy for anyone to come up with a proposal, hard to turn that into a prototype that demonstrates the validity of your ideas. Expect skepticism until you have done that. I suspect it would be easier to implement than freebsd-update, as a good deal of the infrastructure already exists, and would have similar benefits. To start developing it would require a ports tree and a selection of packages compiled from that ports tree. 7.2 Release is coming up. Maybe the ports tree plus packages from that would be a good place to start. freebsd-update and portsnap existed only on Colin Percival's own machines for quite some time, and then they were made available through ports before being accepted into the core system. That is the usual sort of progression for any major new system modifications. The infrastructure may well exist, but don't assume that there is any spare capacity on it. Getting time on the ports build cluster for running experiments is not impossible, but it's somewhere way down the queue after the daily work of building packages for the FTP sites and testing the effects of bug fixes in the bsd.ports.mk infrastructure or important and highly interconnected groups of ports like xorg or gnome. Also, right before 7.2-RELEASE is probably not the best time as that's when things are most hectic. Right /after/ 7.2-RELEASE would be better I think your basic idea of snapshotting the ports tree at regular intervals and building a self-consistent group of desktop related applications is a pretty good one actually. You need to work a bit on the details -- for instance, is it worthwhile rebuilding (say) the X libraries if there have been no changes to them since the previous snapshot? Also, I'd take a good look at exactly how the PC-BSD and Desktop-BSD groups deal with this problem. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: new package system proposal
Matthew Seaman wrote: Chris Whitehouse wrote: You've suggested solutions to a couple of Polytropon's objections, thank you. Do you think there is anough mileage in my suggestion to make it worth putting in front of some ports people? What would have to happen to take it forward? I could rewrite the proposal more clearly. Any well-considered proposal is interesting, and suitable fodder for the freebsd-po...@... mailing list. However you must be prepared for your ideas to undergo some fairly rigourous critique by people who have spent a great deal of time in doing exactly the sort of operations you are talking about. It can be pretty daunting -- remember though that it is your /ideas/ that are being dissected: it's not a personal attack against you for having the temerity to try and suggest something. Also, as ever in the FreeBSD world, code speaks louder than words. It's easy for anyone to come up with a proposal, hard to turn that into a prototype that demonstrates the validity of your ideas. Expect skepticism until you have done that. I suspect it would be easier to implement than freebsd-update, as a good deal of the infrastructure already exists, and would have similar benefits. To start developing it would require a ports tree and a selection of packages compiled from that ports tree. 7.2 Release is coming up. Maybe the ports tree plus packages from that would be a good place to start. freebsd-update and portsnap existed only on Colin Percival's own machines for quite some time, and then they were made available through ports before being accepted into the core system. That is the usual sort of progression for any major new system modifications. The infrastructure may well exist, but don't assume that there is any spare capacity on it. Getting time on the ports build cluster for running experiments is not impossible, but it's somewhere way down the queue after the daily work of building packages for the FTP sites and testing the effects of bug fixes in the bsd.ports.mk infrastructure or important and highly interconnected groups of ports like xorg or gnome. Also, right before 7.2-RELEASE is probably not the best time as that's when things are most hectic. Right /after/ 7.2-RELEASE would be better I think your basic idea of snapshotting the ports tree at regular intervals and building a self-consistent group of desktop related applications is a pretty good one actually. You need to work a bit on the details -- for instance, is it worthwhile rebuilding (say) the X libraries if there have been no changes to them since the previous snapshot? Also, I'd take a good look at exactly how the PC-BSD and Desktop-BSD groups deal with this problem. Cheers, Matthew All your points and your encouragement taken on board, thank you. I have a bit of spare time next week so I am going to work on this then thanks Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On 4/4/09, Chris Whitehouse cwhi...@onetel.com wrote: Hi all [...] My suggestion is to start with a ports tree that is fixed in time. Make that ports tree available as part of this package system and compile a typical desktop set of ports, particularly choosing ones which are large or have many dependencies. When it is all complete release it and start again. Surely quite a wide selection of desktops, wm's and apps could be compiled in a couple of weeks? A lot of ports are already available as packages. Do you simply want more ports available as packages, or a more fundamental change? Modify pkg_add so that it can be told to use this 'snapshot' including downloading the fixed ports tree that was used. What is the benefit of this? Some benefits to this system are - much easier for lower power or laptop users to keep their desktop machine up to date A large subset of ports are already available as packages. Would making more of them available solve the problem you perceive? Who would decide what the appropriate default configuration should be for each port? - problems with particular ports can be centrally fixed by knowledgeable people, possibly reducing time on lists. Huh? Aren't they already? - reduced energy use for everyone. I think the difference in energy use would be so small as to be pointless. If I have a system that consumes 75 kilowatt hours per month, and I spend an extra 0.05 kilowatt hour per month updating ports, is the difference (less than 1/10 of 1 percent) really meaningful? I can't even measure my power usage accurately enough to detect the difference. Convince me to use three liters less hot water per month, and you will save more energy. - the ports system is still available for those who do want to change the config options - ports which are not included in the snapshot are still available - since this system has already provided the larger ports as packages, the remaining ones would be less onerous to install That's already done. - ports that are installed with make install would maintain compatibility with the other installed packages. In what way are they now not compatible? - don't need to mess with portupgrade etc. What's the significant difference between messing with pkg_add and messing with portupgrade? A large subset of ports are already available as packages, so it isn't clear to me how this proposal is significantly improved over using portupgrade -PR _portname_ when a port needs updating. Or portupgrade -PRa when you feel the urge to bring everything up to date. Installing and using portaudit is another useful step in the process. It will send you email to tell you WHEN you should use portupgrade. Of course, it uses energy to do so. - it could (I think) be fitted fairly well into the existing package building process. - it generally increases the useability of FreeBSD as a desktop system. [...] How is it an improvement over the existing tools? I must be missing something, because it sounds to me like you are merely asking that there be more ports made available as packages than are now offered. To me, the best way to improve FreeBSD as a desktop system would be to get Flash 9, 10, etc. working properly (this seems to have recently been accomplished for Flash 9), and to figure out how to get the Gnome-related stuff to update smoothly without the constant need for manual interference (I get tired of running gnomeloganalyzer). But that's a discussion for a different thread. -- -- Bob Johnson fbsdli...@gmail.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Hmm Polytropon you seem to be dismissing my idea with minor examples. I am convinced it could work, and that people would appreciate it. I've tried to answer your points, apologies if I have misunderstood any of them. Polytropon wrote: Compiling applications in general will lead you into one main problem: Many ports have different options that need to be set at compile time. For a set of n options, 2^n packages would be created, if I consider the WITH_SOMETHING options only. It's true but many ports would not be included in this desktop package set. I suspect still that plenty of people would be happy with defaults for many of the desktop apps. One example is mplayer. Its various options select which codecs to include or if / if not to build with mencoder. In regards of different national law, it may even be prohibited to include a several codec, so it needs to be installed afterwards manually. i think Matthew deals with this one in his later post. But ok maybe there are one or two ports for which you provide a binary with default config but many people recompile it anyway. They would still have all the dependencies already installed. Since we are talking about a fixed point ports tree then all the lib and dependency versions would match and - voila no problem. Another example is (you mentioned it) OpenOffice. In the past, I was happy to do # pkg_add -r de-openoffice or something similar. Today, I'm happy that someone put a precompiled package of OpenOffice online and announced it on the de- mailing list. So you would be keen to have OO available. So would a few other people judging by the openoffice topic going at the moment. The topic internationalization comes into mind here. I'm not sure how OpenOffice decides which language to use, maybe this is to be set at compile time, too. Yes this occurred to me after I made my inital post but I think Matthew deals with this one as well. (Side note: I prefer good english language in my programs instead of poor german translation which is quite bad. OpenOffice, and in the past StarOffice, is the only exception for me.) As you see, I am a big fan of pkg_add, but it doesn't work in every case. No because the packages are built on a rolling ports tree. The crucial difference is that the whole thing is a type of ports-snapshot so everything matches. On Sat, 04 Apr 2009 15:13:22 +0100, Chris Whitehouse cwhi...@onetel.com wrote: Ports is rightly a flagship element of FreeBSD. The benefit is configureability and consistency. The obvious downside is it takes so long to update a desktop machine with a normal set of ports installed, particularly lower spec hardware or laptops. pkg_add somewhat addresses this but it doesn't work quite as well as ports because of possible version mismatches. It's always good to use an integrated tool such as portupgrade or portmaster to get rid of such problems (like pkgdb -aF). It allows automating the updating process, but as you know, something can happen and the update stops during the night. yes this is a downside of upgrading by compiling from ports, regardless of whether you use portmanager portupgrade or portmaster. I'm trying to avoid the necessity of the update happening through the night at all. Modify pkg_add so that it can be told to use this 'snapshot' including downloading the fixed ports tree that was used. You can tell pkg_add to get packages from a completely differnent place, this doesn't need a modification of this system's program itself. But a kind of wrapper would help here. The modification is that pkg_add with --ports-snapshot option (or a completely new utility) would hook into this ports-snapshot which consists of a ports tree and a set of packages which are built from 'this' ports tree. Maybe the only change is that pkg_add gets the ports tree snapshot from which the ports were built. I think it is also implicit that if you download a new snapshot you get the ports tree plus all the packages installed on your computer that have been upgraded since your last snapshot. You would not use it by downloading the ports tree snapshot and choosing only to upgrade certain ports. Compare freebsd-update which I think updates everything in your base system, not by you choosing which bits to update. Some benefits to this system are [...] - don't need to mess with portupgrade etc. I always felt that tools like portupgrade make things easier, not messier, but I'm oldfashioned, so don't give anything on my very individual opinion. :-) Yes the ports-mgmt utilities are useful. Still quite a lot of list time is spent on problems around upgrading ports, regardless of the utility used to do the upgrading. A centrally managed set of packages would have access to a group of experts who would be able to fix problems quickly (using the time they didn't have to spend on answering questions on list :) ) - it generally increases the useability of FreeBSD as a
Re: new package system proposal
Matthew Seaman wrote: Polytropon wrote: Compiling applications in general will lead you into one main problem: Many ports have different options that need to be set at compile time. For a set of n options, 2^n packages would be created, if I consider the WITH_SOMETHING options only. One example is mplayer. Its various options select which codecs to include or if / if not to build with mencoder. In regards of different national law, it may even be prohibited to include a several codec, so it needs to be installed afterwards manually. Another example is (you mentioned it) OpenOffice. In the past, I was happy to do # pkg_add -r de-openoffice or something similar. Today, I'm happy that someone put a precompiled package of OpenOffice online and announced it on the de- mailing list. Hmmm... I was thinking about this the other day. There are two classes of behaviour where OPTIONS functionality could be passed down to the compiled pkg level. The first is where choosing an option /only/ affects the dependency tree for a package. The phpMyAdmin port I maintain is like this: by setting OPTIONS you can avoid installing some php modules -- the phpMyAdmin code automatically detects the presence or absence of those modules and does the right thing automatically. Adding an interactive options menu to provide the same functionality when installing from packages seems to me to be do-able, although I admit to no great expertise at C programming. However, aside from meta- ports, this sort of OPTIONS behaviour is probably fairly unusual in the ports tree. The second case is far more common and far more interesting. This is where toggling an option controls whether some sub-set of files get installed or not, without any changes to other parts of the port. Adding different localizations in many programs, or choosing which out of a set of drivers for different pieces of hardware to install (eg. in print/ghostscript8) are cases in point. Now, one answer to providing the full flexibility of such a port when installed via packages is simply to split up the port into a lot of smaller ports, which reduces the problem to the previous one of using OPTIONS to control the dependency tree. The various different php5 modules are a good example of this sort of approach in practice. The disadvantages are exploding the number of directories within the ports tree, requiring maintainers for all of the newly created tiny little ports and generally increasing the amount of work it takes to maintain everything. Now, one way of alleviating some of the the maintenance burden would be a fairly simple idea I had. At the moment, there's a one-to-one relationship between port directories in the ports tree and the packages installed from them. But that doesn't have to be so: why can't typing 'make install' in a port directory end up installing several different packages? Seems quite feasible to me to install a number of sub-ports as one operation. One final note: there's a degenerate case of this behaviour for virtually all ports in the tree. When installing from ports, you can set 'NOPORTDOCS' and 'NOPORTEXAMPLES' to avoid installing documentation or examples respectively. When installing from packages you don't get that capability. Having foo-docs-n.nn.nn and foo-examples-n.nn.nn sub-ports would give you that. cheers, Matthew You've suggested solutions to a couple of Polytropon's objections, thank you. Do you think there is anough mileage in my suggestion to make it worth putting in front of some ports people? What would have to happen to take it forward? I could rewrite the proposal more clearly. I suspect it would be easier to implement than freebsd-update, as a good deal of the infrastructure already exists, and would have similar benefits. To start developing it would require a ports tree and a selection of packages compiled from that ports tree. 7.2 Release is coming up. Maybe the ports tree plus packages from that would be a good place to start. Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
On Mon, 06 Apr 2009 22:48:01 +0100, Chris Whitehouse cwhi...@onetel.com wrote: Hmm Polytropon you seem to be dismissing my idea with minor examples. Actually not, because I'm a big fan of pkg_add -r. :-) I honestly run older machines, the oldest one is a P1 150MHz with 128 MB EDO-RAM where compiling is no fun at all, even a 5.x kernel needs 24 hours. And with the new optimizing cc, it would surely need much more time. If I could install, let's say, FreeBSD 7.1 on that system, use freebsd-update to follow the security updates, and then install software as new as possible (via a pkg_add -r like means), this machine would make a good server without any problems. Well, I even used it as a serious workstation, this should still be possible. It's true but many ports would not be included in this desktop package set. I suspect still that plenty of people would be happy with defaults for many of the desktop apps. I think so, too. My favourite example, mplayer, would be one of the few problematic points, because usually a desktop user wants all the codes, even those that are illegal in his country. i think Matthew deals with this one in his later post. But ok maybe there are one or two ports for which you provide a binary with default config but many people recompile it anyway. They would still have all the dependencies already installed. Yes, and it's mostly okay to get them through a regular pkg_add -r call. Let me give this example: When I'm about to install mplayer on an otherwise fresh system, I don't start an mplayer build in order to compile everything needed. I usually hit ^C as soon as I see a line of ... depends on ... not found and add this via pkg_add -r. If there are dependencies for such a dependency, they will be installed as well. So I finally end up compliling mplayer, and not Gnome or other heavy stuff. Since we are talking about a fixed point ports tree then all the lib and dependency versions would match and - voila no problem. Exactly. Because the sources of pkg_add -r are usually a bit older than the port mplayer itself, there may be slightly different version numbers. But in most cases, it doesn't matter because we're talking about subsubminor version numbers. Another example is (you mentioned it) OpenOffice. In the past, I was happy to do # pkg_add -r de-openoffice or something similar. Today, I'm happy that someone put a precompiled package of OpenOffice online and announced it on the de- mailing list. So you would be keen to have OO available. So would a few other people judging by the openoffice topic going at the moment. Yes, I completely agree with that. As far as I know, the correct internationalisation *requires* compiling. A precompiled OO in English cannot be made a German one. (Side note: I prefer good english language in my programs instead of poor german translation which is quite bad. OpenOffice, and in the past StarOffice, is the only exception for me.) As you see, I am a big fan of pkg_add, but it doesn't work in every case. No because the packages are built on a rolling ports tree. The crucial difference is that the whole thing is a type of ports-snapshot so everything matches. Well, the precompiled packages are somewhat -STABLE every time. There is no exactly RELEASE, except you're using the RELEASE system without updating, and then the packages from the CD. Or the packages for RELEASE from the FTP server. In every other case, the Latest packages are used which may bring up problems with a system that is not up to date. yes this is a downside of upgrading by compiling from ports, regardless of whether you use portmanager portupgrade or portmaster. I'm trying to avoid the necessity of the update happening through the night at all. That's why I do install once, use then. :-) The modification is that pkg_add with --ports-snapshot option (or a completely new utility) would hook into this ports-snapshot which consists of a ports tree and a set of packages which are built from 'this' ports tree. Maybe the only change is that pkg_add gets the ports tree snapshot from which the ports were built. That would preserve the consistency of ports and packages. While you can use the sup files for make update to specify a certain point in time of the ports tree, I think you cannot to the samt with a package, let's say pkg_add -date=2009-01-01 -r xmms to fit a requirement of a local ports tree dated at this timestamp. I think it is also implicit that if you download a new snapshot you get the ports tree plus all the packages installed on your computer that have been upgraded since your last snapshot. You would not use it by downloading the ports tree snapshot and choosing only to upgrade certain ports. Again, this is a good idea which would also preserve consistency. You could binary upgrade your whole system without getting into trouble with a certain version changing. Compare
Re: new package system proposal
per...@pluto.rain.com wrote: Chris Whitehouse cwhi...@onetel.com wrote: My suggestion is to start with a ports tree that is fixed in time. Make that ports tree available as part of this package system and compile a typical desktop set of ports ... Isn't this exactly what is currently done as part of a release? The ports tree is tagged so that a snapshot can be retrieved using csup, and packages are built for publication on (for example) ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/ It is but the difference in what I am imagining is the frequency which would be much higher than releases. And answering more than your question, the difference between this and the pkg_add system is that this is based on a fixed ports tree, whereas pkg_add is based on a rolling ports tree, so Granted this includes all package-distributable ports rather than a typical desktop subset. I know you wrote something else but I am too tired to respond or rather to read it properly tonight. Tomorrow... Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Compiling applications in general will lead you into one main problem: Many ports have different options that need to be set at compile time. For a set of n options, 2^n packages would be created, if I consider the WITH_SOMETHING options only. One example is mplayer. Its various options select which codecs to include or if / if not to build with mencoder. In regards of different national law, it may even be prohibited to include a several codec, so it needs to be installed afterwards manually. Another example is (you mentioned it) OpenOffice. In the past, I was happy to do # pkg_add -r de-openoffice or something similar. Today, I'm happy that someone put a precompiled package of OpenOffice online and announced it on the de- mailing list. The topic internationalization comes into mind here. I'm not sure how OpenOffice decides which language to use, maybe this is to be set at compile time, too. (Side note: I prefer good english language in my programs instead of poor german translation which is quite bad. OpenOffice, and in the past StarOffice, is the only exception for me.) As you see, I am a big fan of pkg_add, but it doesn't work in every case. On Sat, 04 Apr 2009 15:13:22 +0100, Chris Whitehouse cwhi...@onetel.com wrote: Ports is rightly a flagship element of FreeBSD. The benefit is configureability and consistency. The obvious downside is it takes so long to update a desktop machine with a normal set of ports installed, particularly lower spec hardware or laptops. pkg_add somewhat addresses this but it doesn't work quite as well as ports because of possible version mismatches. It's always good to use an integrated tool such as portupgrade or portmaster to get rid of such problems (like pkgdb -aF). It allows automating the updating process, but as you know, something can happen and the update stops during the night. Modify pkg_add so that it can be told to use this 'snapshot' including downloading the fixed ports tree that was used. You can tell pkg_add to get packages from a completely differnent place, this doesn't need a modification of this system's program itself. But a kind of wrapper would help here. Some benefits to this system are [...] - don't need to mess with portupgrade etc. I always felt that tools like portupgrade make things easier, not messier, but I'm oldfashioned, so don't give anything on my very individual opinion. :-) - it generally increases the useability of FreeBSD as a desktop system. Well, when we're talking about desktop systems, there are the both two big philosophies: (a) install it once, use it then (b) always upgrade There are (reasonable) needs for both concepts, and they may even mix. Thinking about the problems / difficulties that came up with the recent X.org update, my X is still in (a) state. :-) I think this could work because I think the default config options are probably suitable for most desktop users so it wouldn't be necessary to create loads of different binaries for the same port, [...] I may disagree and bring (only) one example: mplayer. Reason: codecs. Most users I know who dislike compiling always take the time to get the maximum out of mplayer, and this cannot be achieved using pkg_add. [...] and security considerations are not so important as for server admins. Don't say that. An incorrectly administrated desktop can develop into a massive threat to others on the Internet. Do you know that most spam, data espionage, malware, viruses etc. are distributed by home PCs (even if they are placed in a company's network)? It might be possible to distribute the actual compiling to users with spare cpu cycles, a bit like the s...@home projects etc. I'll reserve some spare cycles for you with my for(;;); program. :-) -- Polytropon From Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ... ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: new package system proposal
Polytropon wrote: Compiling applications in general will lead you into one main problem: Many ports have different options that need to be set at compile time. For a set of n options, 2^n packages would be created, if I consider the WITH_SOMETHING options only. One example is mplayer. Its various options select which codecs to include or if / if not to build with mencoder. In regards of different national law, it may even be prohibited to include a several codec, so it needs to be installed afterwards manually. Another example is (you mentioned it) OpenOffice. In the past, I was happy to do # pkg_add -r de-openoffice or something similar. Today, I'm happy that someone put a precompiled package of OpenOffice online and announced it on the de- mailing list. Hmmm... I was thinking about this the other day. There are two classes of behaviour where OPTIONS functionality could be passed down to the compiled pkg level. The first is where choosing an option /only/ affects the dependency tree for a package. The phpMyAdmin port I maintain is like this: by setting OPTIONS you can avoid installing some php modules -- the phpMyAdmin code automatically detects the presence or absence of those modules and does the right thing automatically. Adding an interactive options menu to provide the same functionality when installing from packages seems to me to be do-able, although I admit to no great expertise at C programming. However, aside from meta- ports, this sort of OPTIONS behaviour is probably fairly unusual in the ports tree. The second case is far more common and far more interesting. This is where toggling an option controls whether some sub-set of files get installed or not, without any changes to other parts of the port. Adding different localizations in many programs, or choosing which out of a set of drivers for different pieces of hardware to install (eg. in print/ghostscript8) are cases in point. Now, one answer to providing the full flexibility of such a port when installed via packages is simply to split up the port into a lot of smaller ports, which reduces the problem to the previous one of using OPTIONS to control the dependency tree. The various different php5 modules are a good example of this sort of approach in practice. The disadvantages are exploding the number of directories within the ports tree, requiring maintainers for all of the newly created tiny little ports and generally increasing the amount of work it takes to maintain everything. Now, one way of alleviating some of the the maintenance burden would be a fairly simple idea I had. At the moment, there's a one-to-one relationship between port directories in the ports tree and the packages installed from them. But that doesn't have to be so: why can't typing 'make install' in a port directory end up installing several different packages? Seems quite feasible to me to install a number of sub-ports as one operation. One final note: there's a degenerate case of this behaviour for virtually all ports in the tree. When installing from ports, you can set 'NOPORTDOCS' and 'NOPORTEXAMPLES' to avoid installing documentation or examples respectively. When installing from packages you don't get that capability. Having foo-docs-n.nn.nn and foo-examples-n.nn.nn sub-ports would give you that. cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW signature.asc Description: OpenPGP digital signature
Re: new package system proposal
Chris Whitehouse cwhi...@onetel.com wrote: My suggestion is to start with a ports tree that is fixed in time. Make that ports tree available as part of this package system and compile a typical desktop set of ports ... Isn't this exactly what is currently done as part of a release? The ports tree is tagged so that a snapshot can be retrieved using csup, and packages are built for publication on (for example) ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/ Granted this includes all package-distributable ports rather than a typical desktop subset. Modify pkg_add so that it can be told to use this 'snapshot' including downloading the fixed ports tree that was used. AFAIK pkg_add currently does not download the ports tree at all -- that is done using csup or portsnap -- but it can be directed to a release set or other remote package repository via the -r flag. My only issue with that approach is that, last I knew, there was no way to specify a mix of local and remote repositories (to deal with the case where I have already downloded some subset of what I end up needing, so I want pkg_add to search locally first and use the remote repository only for packages that it can't find locally). ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
