Re: Something rotten in ports (was Re: package building failure irritation)
On Sat, 13 Mar 2010 23:31, xorquewasp@ wrote: 'Lo, On 2010-03-13 20:40:32, jhell wrote: Not that this is a solution to your problem but it might be a possibility for you to consider, but a couple years back I dropped using make package(-recursive) and just scripted out making backup packages using something like the following. I particularly like this solution! I might just go ahead and implement this here, including the signing. Thanks. xw No Problem. Your welcome. -- jhell ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Something rotten in ports (was Re: package building failure irritation)
On Sat, Mar 13, 2010 at 08:40:32PM -0500, jhell wrote: Rather simple way to go about creating final packages and from some earlier emails to the list there was word of some directories not being included in final built packages due to empty directories or something like that so be careful when/if considering something like this. I believe you are talking about ports/144164 which was about RC scripts not being included in a package when using package-noinstall. This is currently in the hands of portmgr. It will have to wait until 7.3 is out, at the earliest. -- WXS ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Something rotten in ports (was Re: package building failure irritation)
On Fri, Mar 12, 2010 at 11:52 PM, xorquew...@googlemail.com wrote: Have you tried just setting PORTSDIR and letting bsd.port.mk set the rest of the paths with their defaults that are relative to PORTSDIR? If that works, then we can start hunting for places that are not handling absolute vs. relative paths correctly in bsd.port.mk. Now, with only: PORTSDIR=/var/ports/tree .. in make.conf, the error is: Creating package /var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz Registering depends: dbus-glib-0.84 gio-fam-backend-2.22.4 gamin-0.1.10_3 glib-2.22.4 gettext-0.17_1 dbus-1.2.16_1 libxml2-2.7.6_1 libiconv-1.13.1_1 libX11-1.2.1_1,1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4 libXdmcp-1.0.2_1 xproto-7.0.15 pkg-config-0.23_1 perl-5.10.1 python26-2.6.4 gnome_subr-1.0 expat-2.0.1_1 kbproto-1.0.3. Creating bzip'd tar ball in '/var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz' rmdir: /var/ports/tree/devel/eggdbus/work: Directory not empty *** Error code 1 (ignored) === Generating temporary packing list Creating package /var/ports/tree/textproc/docbook-420/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/var/ports/tree/textproc/docbook-420/docbook-4.2.tbz' tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 *** Error code 1 Note: all packages being created via package-recursive are created via package-noinstall, and because the pkg-plist is referring to files and directories which don't exist when install is run, the call will always fail: [gcoo...@bayonetta ~]$ sudo make -C /usr/ports/textproc/docbook-420/ package-noinstall Password: === Generating temporary packing list Creating package /usr/ports/packages/All/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/usr/ports/packages/All/docbook-4.2.tbz' tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 *** Error code 1 Stop in /usr/ports/textproc/docbook-420. *** Error code 1 Stop in /usr/ports/textproc/docbook-420. [gcoo...@bayonetta ~]$ cat /usr/ports/textproc/docbook-420/pkg-plist share/sgml/docbook/4.2/ChangeLog share/sgml/docbook/4.2/calstblx.dtd @unexec %%XMLCATMGR%% -sc %%CATALOG_PORTS_SGML%% remove %%DTD_NAME%%/%%DTD_VERSION%%/catalog share/sgml/docbook/4.2/catalog share/sgml/docbook/4.2/catalog.xml @exec %%XMLCATMGR%% -sc %%CATALOG_PORTS_SGML%% add CATALOG %%DTD_NAME%%/%%DTD_VERSION%%/catalog share/sgml/docbook/4.2/dbcentx.mod
Re: Something rotten in ports (was Re: package building failure irritation)
On Sat, Mar 13, 2010 at 11:51 AM, Garrett Cooper yanef...@gmail.com wrote: On Fri, Mar 12, 2010 at 11:52 PM, xorquew...@googlemail.com wrote: Have you tried just setting PORTSDIR and letting bsd.port.mk set the rest of the paths with their defaults that are relative to PORTSDIR? If that works, then we can start hunting for places that are not handling absolute vs. relative paths correctly in bsd.port.mk. Now, with only: PORTSDIR=/var/ports/tree .. in make.conf, the error is: Creating package /var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz Registering depends: dbus-glib-0.84 gio-fam-backend-2.22.4 gamin-0.1.10_3 glib-2.22.4 gettext-0.17_1 dbus-1.2.16_1 libxml2-2.7.6_1 libiconv-1.13.1_1 libX11-1.2.1_1,1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4 libXdmcp-1.0.2_1 xproto-7.0.15 pkg-config-0.23_1 perl-5.10.1 python26-2.6.4 gnome_subr-1.0 expat-2.0.1_1 kbproto-1.0.3. Creating bzip'd tar ball in '/var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz' rmdir: /var/ports/tree/devel/eggdbus/work: Directory not empty *** Error code 1 (ignored) === Generating temporary packing list Creating package /var/ports/tree/textproc/docbook-420/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/var/ports/tree/textproc/docbook-420/docbook-4.2.tbz' tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 *** Error code 1 Note: all packages being created via package-recursive are created via package-noinstall, and because the pkg-plist is referring to files and directories which don't exist when install is run, the call will always fail: Before a few folks correct me on this claim... all DEPENDENT packages are created via package-noinstall. The top-level package created with package-recursive is actually created via package. [gcoo...@bayonetta ~]$ sudo make -C /usr/ports/textproc/docbook-420/ package-noinstall Password: === Generating temporary packing list Creating package /usr/ports/packages/All/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/usr/ports/packages/All/docbook-4.2.tbz' tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 *** Error code 1 Stop in /usr/ports/textproc/docbook-420. *** Error code 1 Stop in /usr/ports/textproc/docbook-420. [gcoo...@bayonetta ~]$ cat /usr/ports/textproc/docbook-420/pkg-plist share/sgml/docbook/4.2/ChangeLog
Re: Something rotten in ports (was Re: package building failure irritation)
On Sat, 13 Mar 2010 02:52, xorquewasp@ wrote:
Have you tried just setting PORTSDIR and letting bsd.port.mk set the
rest of the paths with their defaults that are relative to PORTSDIR? If
that works, then we can start hunting for places that are not handling
absolute vs. relative paths correctly in bsd.port.mk.
Now, with only:
PORTSDIR=/var/ports/tree
.. in make.conf, the error is:
Creating package /var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz
Registering depends: dbus-glib-0.84 gio-fam-backend-2.22.4 gamin-0.1.10_3
glib-2.22.4 gettext-0.17_1 dbus-1.2.16_1 libxml2-2.7.6_1 libiconv-1.13.1_1
libX11-1.2.1_1,1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4
libXdmcp-1.0.2_1 xproto-7.0.15 pkg-config-0.23_1 perl-5.10.1 python26-2.6.4
gnome_subr-1.0 expat-2.0.1_1 kbproto-1.0.3.
Creating bzip'd tar ball in '/var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz'
rmdir: /var/ports/tree/devel/eggdbus/work: Directory not empty
*** Error code 1 (ignored)
=== Generating temporary packing list
Creating package /var/ports/tree/textproc/docbook-420/docbook-4.2.tbz
Registering depends: iso8879-1986_2 xmlcatmgr-2.2.
Creating bzip'd tar ball in
'/var/ports/tree/textproc/docbook-420/docbook-4.2.tbz'
tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors.
pkg_create: make_dist: tar command failed with code 256
*** Error code 1
Stop in /var/ports/tree/textproc/docbook-420.
*** Error code 1
Stop in /var/ports/tree/textproc/docbook-420.
*** Error code 1
Stop in /var/ports/tree/graphics/inkscape.
Regards,
xw
Not that this is a solution to your problem but it might be a possibility
for you to consider, but a couple years back I dropped using make
package(-recursive) and just scripted out making backup packages using
something like the following.
_pkg_bld(){
cd /exports/packages
for package in `ls /var/db/pkg |sed 's/pkgdb.db//'`; do
echo Building package: $package
pkg_create -v -b $package pkg_bld.log 21
done
}
Rather simple way to go about creating final packages and from some
earlier emails to the list there was word of some directories not being
included in final built packages due to empty directories or something
like that so be careful when/if considering something like this.
There are additional flags that can be passed to pkg_create to keep it
from rebuilding the package if it already exists but I have not worked
that into the above example usage.
At some point I also worked in package signing like so,
_pkg_sign(){
cd /exports/packages
for package in `ls *.tbz`; do
echo Signing package: $package
gpg -u 3588E52D -v -ba $package
done
}
Good Luck Regards,
--
jhell
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Something rotten in ports (was Re: package building failure irritation)
'Lo, On 2010-03-13 20:40:32, jhell wrote: Not that this is a solution to your problem but it might be a possibility for you to consider, but a couple years back I dropped using make package(-recursive) and just scripted out making backup packages using something like the following. I particularly like this solution! I might just go ahead and implement this here, including the signing. Thanks. xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Something rotten in ports (was Re: package building failure irritation)
This is a complete lot of how to reproduce the various errors I've
seen with 'make package-recursive'. I've checked the pointyhat logs
and there are no errors logged for the packages involved here. There
seems to be a bug somewhere in ports. I've used inkscape as a scapegoat
here but the errors occur with many, many ports.
There is no ZFS or nullfs involved here (apart from the read-only nullfs
mount that ezjail uses to share /usr/bin and the like).
I've used ezjail-admin to create jails.
$ sudo ezjail-admin create 8.0-amd64-pkg_viper-2 127.1.0.11
$ sudo ezjail-admin onestart 8.0-amd64-pkg_viper-2
$ sudo jexec `jls | grep pkg_viper-2 | awk '{print $1}'` sh
jail# mkdir /var/ports/work
jail# mkdir /var/ports/tree
jail# mkdir /var/ports/packages
jail# mkdir /var/ports/distfiles
jail# vi /etc/make.conf
DISTDIR= /var/ports/distfiles
PACKAGES= /var/ports/packages
WRKDIRPREFIX= /var/ports/work
PORTSDIR= /var/ports/tree
jail# vi /etc/profile
FTP_PASSIVE_MODE=yes
HTTP_PROXY=10.1.3.3:8080
export FTP_PASSIVE_MODE
export HTTP_PROXY
jail# . /etc/profile
jail# portsnap -p /var/ports/tree fetch extract
Looking up portsnap.FreeBSD.org mirrors... 3 mirrors found.
Fetching public key from portsnap2.FreeBSD.org... done.
Fetching snapshot tag from portsnap2.FreeBSD.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Fri Mar 12 00:22:12 UTC 2010:
d67bd7a10044c70dc705b2c5b05db32b07ab8bd2262c3e 1% of 61 MB 161 kBps
...
jail# cd /var/ports/tree/graphics/inkscape
jail# make config-recursive
jail# make fetch-recursive
jail# make package-recursive 21 | tee /tmp/inkscape.log
Of course, at the end of inkscape.log:
Creating package /var/ports/packages/All/docbook-4.1_4.tbz
Registering depends: iso8879-1986_2 xmlcatmgr-2.2.
Creating bzip'd tar ball in '/var/ports/packages/All/docbook-4.1_4.tbz'
rmdir: /var/ports/work/var/ports/tree/textproc/docbook-410/work: Directory not
empty
*** Error code 1 (ignored)
Creating package /var/ports/packages/All/eggdbus-0.6.tbz
Registering depends: dbus-glib-0.84 gio-fam-backend-2.22.4 gamin-0.1.10_3
glib-2.22.4 gettext-0.17_1 dbus-1.2.16_1 libxml2-2.7.6_1 libiconv-1.13.1_1
libX11-1.2.1_1,1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4
libXdmcp-1.0.2_1 xproto-7.0.15 pkg-config-0.23_1 perl-5.10.1 python26-2.6.4
gnome_subr-1.0 expat-2.0.1_1 kbproto-1.0.3.
Creating bzip'd tar ball in '/var/ports/packages/All/eggdbus-0.6.tbz'
rmdir: /var/ports/work/var/ports/tree/devel/eggdbus/work: Directory not empty
*** Error code 1 (ignored)
=== Generating temporary packing list
tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory
tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors.
pkg_create: make_dist: tar command failed with code 256
Creating package /var/ports/packages/All/docbook-4.2.tbz
Registering depends: iso8879-1986_2 xmlcatmgr-2.2.
Creating bzip'd tar ball in '/var/ports/packages/All/docbook-4.2.tbz'
*** Error code 1
Stop in /var/ports/tree/textproc/docbook-420.
*** Error code 1
Stop in /var/ports/tree/textproc/docbook-420.
*** Error code 1
Stop in /var/ports/tree/graphics/inkscape.
The full log is here:
http://coreland.ath.cx/tmp/inkscape.log
xw
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Something rotten in ports (was Re: package building failure irritation)
On 2010-03-12 22:36:54, Greg Larkin wrote: Hi xw, I noticed something strange here. How is WRKDIR (in this case /var/ports/work/var/ports/tree/devel/eggdbus/work) defined? It looks like bsd.port.mk combined your WRKDIRPREFIX and PORTSDIR to create that path, but skimming the code, I can't figure out how it's doing that. How many levels of that directory tree exist on your system? 'Lo. Not sure I understand what you're asking me to check, but things have basically laid themselves out like this: # ls /var/ports/work/var/ports/tree/devel/eggdbus/work/eggdbus-0.6/ AUTHORS MakefileREADMEconfig.h.in configure.ac eggdbus-1.pc.in missing COPYING Makefile.am aclocal.m4config.logconfigure.bak gtk-doc.makesrc ChangeLog Makefile.in compile config.status depcomp install-sh stamp-h1 HACKING Makefile.in.bak config.guess config.subdocs libtool INSTALL NEWSconfig.h configure eggdbus-1.pc ltmain.sh Have you tried just setting PORTSDIR and letting bsd.port.mk set the rest of the paths with their defaults that are relative to PORTSDIR? If that works, then we can start hunting for places that are not handling absolute vs. relative paths correctly in bsd.port.mk. Will try that now. In the original setup, I did specifically want to keep these directories separate (as PORTSDIR was a read-only nullfs mount), but that's obviously not the case for this example setup. xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Something rotten in ports (was Re: package building failure irritation)
Have you tried just setting PORTSDIR and letting bsd.port.mk set the rest of the paths with their defaults that are relative to PORTSDIR? If that works, then we can start hunting for places that are not handling absolute vs. relative paths correctly in bsd.port.mk. Now, with only: PORTSDIR=/var/ports/tree .. in make.conf, the error is: Creating package /var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz Registering depends: dbus-glib-0.84 gio-fam-backend-2.22.4 gamin-0.1.10_3 glib-2.22.4 gettext-0.17_1 dbus-1.2.16_1 libxml2-2.7.6_1 libiconv-1.13.1_1 libX11-1.2.1_1,1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4 libXdmcp-1.0.2_1 xproto-7.0.15 pkg-config-0.23_1 perl-5.10.1 python26-2.6.4 gnome_subr-1.0 expat-2.0.1_1 kbproto-1.0.3. Creating bzip'd tar ball in '/var/ports/tree/devel/eggdbus/eggdbus-0.6.tbz' rmdir: /var/ports/tree/devel/eggdbus/work: Directory not empty *** Error code 1 (ignored) === Generating temporary packing list Creating package /var/ports/tree/textproc/docbook-420/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/var/ports/tree/textproc/docbook-420/docbook-4.2.tbz' tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 *** Error code 1 Stop in /var/ports/tree/textproc/docbook-420. *** Error code 1 Stop in /var/ports/tree/textproc/docbook-420. *** Error code 1 Stop in /var/ports/tree/graphics/inkscape. Regards, xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
xorquew...@googlemail.com writes: Is there any resolution for this problem? I told you, zfs set mountpoint and ditch nullfs. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
On Fri, 5 Mar 2010 06:56, des@ wrote: xorquew...@googlemail.com writes: Is there any resolution for this problem? I told you, zfs set mountpoint and ditch nullfs. DES Adding on to this. There were reports in various cases dating back to ~1 year with bad results, possible data loss, hard and soft dead locks when nullfs was used with ZFS. nullfs at one point that I do remember was not recommended to be used at all with ZFS and there exist quite a bit of other functionality in ZFS cloning, snapshots, mountpoint= etc... that serve well enough to not use nullfs at all. Surely in the case above you are talking about packages and in which you really should not need to mount this multiple times as a writable FS, correct me if you feel that it does and lets see why (please provide your process if you do). This should suffice mounting a packages type collection in multiple places: # Mount one dataset wherever you need it. zfs set mountpoint=/path/to/wherever pool/packages # Create a snapshot and clone it for further mountpoints. zfs snapshot pool/packa...@20100305 zfs clone pool/packa...@20100305 pool/packages2 zfs set mountpoint=/path/to/other/dir pool/packages2 If none of this would suffice then your package management needs to be re-thought out. Central FTP, NFS, SMB, RSYNC?. You have plenty of options available but the only one that will be suitable to you your process will be the one that is planned for accordingly and thoroughly. Best of luck, -- jhell ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
'Lo, On 2010-03-05 09:47:11, jhell wrote: Adding on to this. There were reports in various cases dating back to ~1 year with bad results, possible data loss, hard and soft dead locks when nullfs was used with ZFS. nullfs at one point that I do remember was not recommended to be used at all with ZFS and there exist quite a bit of other functionality in ZFS cloning, snapshots, mountpoint= etc... that serve well enough to not use nullfs at all. Except, as far as I can see (not far), there's no way to emulate one directory/fs being mounted in multiple places. I'll come to why in a second... Surely in the case above you are talking about packages and in which you really should not need to mount this multiple times as a writable FS, correct me if you feel that it does and lets see why (please provide your process if you do). Basically, what I'm doing is trying to share a writable distfile directory across multiple package-building jails as a sort of cache. I don't have unlimited bandwidth usage per-month and would like to avoid multiple jails downloading 150mb distfiles (I've already exceeded monthly bandwidth limits due to this exact problem). I'd also like to avoid having multiple copies of distfiles laying around (as the number of jails increases, the space required becomes not insignificant). A possible solution to the above might be to use make fetch-recursive on the host and then just mount the distfile directory read-only inside the actual build jail to eliminate one of the read/write mounts. Of course, what distfiles are actually required might change based on the settings in make.conf in each jail... I'm also sharing the ports tree read-only across jails to ensure that each jail has the exact same tree. This isn't a problem as far as I know (due to the read-only part). It might be the case that the only real problem (the one that causes the package building errors) is the read/write nullfs mount of the /work directory so I'm about to build a test jail now and see if this is so (and I'll use DES' method of setting the ZFS mountpoint for this directory if it is as there's specifically no need for shared state here). This should suffice mounting a packages type collection in multiple places: # Mount one dataset wherever you need it. zfs set mountpoint=/path/to/wherever pool/packages # Create a snapshot and clone it for further mountpoints. zfs snapshot pool/packa...@20100305 zfs clone pool/packa...@20100305 pool/packages2 zfs set mountpoint=/path/to/other/dir pool/packages2 If none of this would suffice then your package management needs to be re-thought out. Central FTP, NFS, SMB, RSYNC?. In this setup, jails build packages for other jails that specifically don't have network access (see the example given before about sandboxing a pdf reader): 8.0-amd64-mistrust_pkg mounts /storage/jails/8.0/x86_64/mistrust/pkg at /pkg and writes built packages to it whilst 8.0-amd64-mistrust mounts the same directory read-only at /pkg and, of course, installs packages from it. In this case, the only sane way to do things seems to be to mount a read-only view of a directory of packages built by another jail with nullfs. I don't particularly want to be cloning and managing snapshots - I'd be exchanging the simplicity of a direct solution (a direct, read-only view of the current state of a given package directory) with a solution where I have to endlessly shuffle and clone snapshots. You have plenty of options available but the only one that will be suitable to you your process will be the one that is planned for accordingly and thoroughly. I'm surprised that something as simple as the above requires this much thought to be honest. xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
'Lo. As a test, I've now stripped nullfs out of the picture entirely. The exact same problem still occurs: Creating package /pkg/All/libgpg-error-1.7.tbz Registering depends: gettext-0.17_1 libiconv-1.13.1_1. Creating bzip'd tar ball in '/pkg/All/libgpg-error-1.7.tbz' rmdir: /work/ports/security/libgpg-error/work: Directory not empty *** Error code 1 (ignored) === Generating temporary packing list tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 Creating package /pkg/All/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/pkg/All/docbook-4.2.tbz' xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
So, anyway... Is there any resolution for this problem? I have quite a few packages I'd like to get built (just custom enough that pre-built packages from FreeBSD mirrors won't suffice). I have to admit to not being convinced that nullfs is at the root of the problem... I have vague, confused memories of this happening about a year ago before I had ZFS or knew nullfs was actually usable. I switched to using tinderbox so didn't actually resolve the problem at the time. xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
xorquew...@googlemail.com writes: Basically, I have a ton of jails and each jail mounts a shared 'tmp', That's not a good idea, there are too many opportunities for conflicts (software that creates sockets and state directories with non-randomized names in /tmp) and might even allow a compromised jail to compromise the others. and possibly unshared 'pkg' 'work' 'distfiles' and 'ports' directories: /storage/jails/8.0/x86_64/mk4/pkg /jail/8.0-amd64-mk4/pkgnullfs rw /storage/jails/8.0/x86_64/mk4/work /jail/8.0-amd64-mk4/work nullfs rw /storage/distfiles /jail/8.0-amd64-mk4/distfiles nullfs rw /storage/ports /jail/8.0-amd64-mk4/ports nullfs ro /storage/shared_tmp /jail/8.0-amd64-mk4/shared_tmp nullfs rw zfs set mountpoint=/jail/8.0-amd64-mk4 storage/jails/8.0/x86_64/mk4 Children of storage/jails/8.0/x86_64/mk4 will inherit this property, so they will automatically appear where you expect; alternatively, you can set the mountpoint property for each individual fileset. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
m...@coreland.ath.cx writes: Is it possible to define multiple mountpoints (to emulate what nullfs provides)? Sorry, no. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
On 2010-03-02 11:00:45, Dag-Erling Smørgrav wrote: xorquew...@googlemail.com writes: Basically, I have a ton of jails and each jail mounts a shared 'tmp', That's not a good idea, there are too many opportunities for conflicts (software that creates sockets and state directories with non-randomized names in /tmp) and might even allow a compromised jail to compromise the others. Don't panic. It's actually mounted at /shared_tmp as an explicit means for jails to communicate via the filesystem. In other words, it's known to be unsafe. I use it to sandbox programs to some extent (download a pdf on the host into /shared_tmp and open it in a pdf reader in a jail that has no network or other filesystem access). The jails also aren't externally accessible. zfs set mountpoint=/jail/8.0-amd64-mk4 storage/jails/8.0/x86_64/mk4 Children of storage/jails/8.0/x86_64/mk4 will inherit this property, so they will automatically appear where you expect; alternatively, you can set the mountpoint property for each individual fileset. I see. Is it possible to define multiple mountpoints (to emulate what nullfs provides)? xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
% fgrep 'Directory not empty' inkscape.txt rmdir: /work/ports/devel/boost-libs/work: Directory not empty rmdir: /work/ports/x11-toolkits/gtkmm24/work: Directory not empty rmdir: /work/ports/devel/boehm-gc/work: Directory not empty rmdir: /work/ports/math/gsl/work: Directory not empty rmdir: /work/ports/textproc/gtkspell/work: Directory not empty rmdir: /work/ports/graphics/libwpg/work: Directory not empty rmdir: /work/ports/graphics/ImageMagick/work: Directory not empty rmdir: /work/ports/textproc/p5-XML-XQL/work: Directory not empty rmdir: /work/ports/devel/boost-jam/work: Directory not empty rmdir: /work/ports/devel/icu/work: Directory not empty rmdir: /work/ports/devel/glibmm/work: Directory not empty rmdir: /work/ports/graphics/cairomm/work: Directory not empty rmdir: /work/ports/x11-toolkits/pangomm/work: Directory not empty This tells me that either there is another build running in parallel, or /work is mounted from a dodgy NFS server. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
On 2010-03-01 14:34:50, Dag-Erling Smørgrav wrote: % fgrep 'Directory not empty' inkscape.txt rmdir: /work/ports/devel/boost-libs/work: Directory not empty rmdir: /work/ports/x11-toolkits/gtkmm24/work: Directory not empty rmdir: /work/ports/devel/boehm-gc/work: Directory not empty rmdir: /work/ports/math/gsl/work: Directory not empty rmdir: /work/ports/textproc/gtkspell/work: Directory not empty rmdir: /work/ports/graphics/libwpg/work: Directory not empty rmdir: /work/ports/graphics/ImageMagick/work: Directory not empty rmdir: /work/ports/textproc/p5-XML-XQL/work: Directory not empty rmdir: /work/ports/devel/boost-jam/work: Directory not empty rmdir: /work/ports/devel/icu/work: Directory not empty rmdir: /work/ports/devel/glibmm/work: Directory not empty rmdir: /work/ports/graphics/cairomm/work: Directory not empty rmdir: /work/ports/x11-toolkits/pangomm/work: Directory not empty This tells me that either there is another build running in parallel, or /work is mounted from a dodgy NFS server. 'Lo, There's certainly no parallel building going on, but /work is nullfs mounted (from ZFS). Could this cause the above? xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
xorquew...@googlemail.com writes: There's certainly no parallel building going on, but /work is nullfs mounted (from ZFS). Could this cause the above? Not sure. Why are you using nullfs? Can you show us the contents of those directories? DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
On Mon, Mar 01, 2010 at 01:58:29PM +, xorquew...@googlemail.com wrote: On 2010-03-01 14:34:50, Dag-Erling Smørgrav wrote: % fgrep 'Directory not empty' inkscape.txt rmdir: /work/ports/devel/boost-libs/work: Directory not empty ... rmdir: /work/ports/x11-toolkits/pangomm/work: Directory not empty This tells me that either there is another build running in parallel, or /work is mounted from a dodgy NFS server. 'Lo, There's certainly no parallel building going on, but /work is nullfs mounted (from ZFS). Could this cause the above? Caveat: this may be a red herring. But please note that for stable/7 prior to r190970 (2009-04-12 10:43:41 -0700) or head prior to r189287 (2009-03-02 12:51:39 -0800; prior to stable/8 branch, so it's part of stable/8 already), there was a rather nasty (IMO, as I spent a fair amount of time trying to figure out what was going on) such that a FreeBSD NFS client would see precisely the above symptoms if: * A process on the FreEBSD NFS client performed a chdir() to a directory that was NFS-mounted, then started a recursive descent (e.g., tar c ... or rm -fr) from that directory and * Some other process on the same FreeBSD NFS client attempted to perform an unmount() of the NFS-mounted file system referenced above. Note that the unmount() is doomed, as the file system is active -- a directory in it is the $cwd for the first process, after all. This may seem an unlikely -- possibly even perverse -- combination of events. However, it is actually SOP for amd(8): the master amd process periodically forks a child to perform an attempted unmount of auto-mounted NFS file systems periodically, and the way amd realizes that the file system is not eligible for unmounting is if the attempted unmount() gets EBUSY. It is *possible* that something akin to this mechanism *might* be affecting the OP. Peace, david -- David H. Wolfskill da...@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. pgp2kxBStWDpC.pgp Description: PGP signature
Re: package building failure irritation
On 2010-03-01 15:10:16, Dag-Erling Smørgrav wrote: xorquew...@googlemail.com writes: There's certainly no parallel building going on, but /work is nullfs mounted (from ZFS). Could this cause the above? Not sure. Why are you using nullfs? Basically because I don't know any better. Can you show us the contents of those directories? I'm off out, but I'll post the contents of /work when I get back tonight. xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
xorquew...@googlemail.com writes: Dag-Erling Smørgrav d...@des.no writes: xorquew...@googlemail.com writes: There's certainly no parallel building going on, but /work is nullfs mounted (from ZFS). Could this cause the above? Not sure. Why are you using nullfs? Basically because I don't know any better. That's begging the question... Clearly you're using nullfs because you need something done that you think nullfs can do for you. I'd like to know what that is, because I suspect that you don't actually need it, and it might be the source of your problems. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
'Lo, Here's the current state of '/work': http://coreland.ath.cx/tmp/work.txt On 2010-03-01 20:08:45, Dag-Erling Smørgrav wrote: That's begging the question... Clearly you're using nullfs because you need something done that you think nullfs can do for you. I'd like to know what that is, because I suspect that you don't actually need it, and it might be the source of your problems. I agree, ZFS likely has something to replace my use of nullfs. I used it previously to do the following using only UFS filesystems: Basically, I have a ton of jails and each jail mounts a shared 'tmp', and possibly unshared 'pkg' 'work' 'distfiles' and 'ports' directories: /storage/jails/8.0/x86_64/mk4/pkg /jail/8.0-amd64-mk4/pkgnullfs rw /storage/jails/8.0/x86_64/mk4/work /jail/8.0-amd64-mk4/work nullfs rw /storage/distfiles /jail/8.0-amd64-mk4/distfiles nullfs rw /storage/ports /jail/8.0-amd64-mk4/ports nullfs ro /storage/shared_tmp /jail/8.0-amd64-mk4/shared_tmp nullfs rw xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
On 2010-02-26 17:38:00, Greg Larkin wrote: Can you post a full log file of the process somewhere? The best way to capture it is with the commands: 'Lo, I did actually send a log in response to DES' email but I sent the email from the wrong account and it still awaits list moderation. Here it is again: http://coreland.ath.cx/tmp/inkscape.txt xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
package building failure irritation
'Lo. I've come up against an infuriating problem with ports. I don't even know what causes it, let alone how to solve it. Essentially, to stop already-installed software from polluting port builds, I build ports in a jail. I put a list of ports in a file and use a trivial shell script to build them one after the other with: cd $port make clean make deinstall make package-recursive The jail is set up with: /etc/make.conf: WRKDIRPREFIX= /work DISTDIR= /distfiles PACKAGES= /pkg PORTSDIR= /ports With /ports, /distfiles and /pkg being mounted from outside the jail using nullfs (just because those directories exist on a ZFS array outside of the jail for performance reasons). Before building a set of packages, the jail is wiped clean with 'pkg_delete -a'. The problem is that it seems most ports just outright fail to produce packages. I log each build and often the last lines of the log files read something like: xchm: tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/calstblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/catalog.xml: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbcentx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbgenent.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbhierx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbnotnx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/dbpoolx.mod: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.cat: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dcl: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbook.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/docbookx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/soextblx.dtd: Cannot stat: No such file or directory tar: share/sgml/docbook/4.2/README: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 Creating package /pkg/All/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/pkg/All/docbook-4.2.tbz' *** Error code 1 Stop in /ports/textproc/docbook-420. *** Error code 1 Stop in /ports/textproc/docbook-420. *** Error code 1 Stop in /ports/deskutils/xchm. fatal: failed gimp: === Patching for libopenraw-0.0.8_2 === Applying FreeBSD patches for libopenraw-0.0.8_2 === libopenraw-0.0.8_2 depends on file: /usr/local/bin/intltool-extract - found === libopenraw-0.0.8_2 depends on executable: pkg-config - found === libopenraw-0.0.8_2 depends on shared library: boost_thread.4 - not found ===Verifying install for boost_thread.4 in /ports/devel/boost-libs === Returning to build of libopenraw-0.0.8_2 Error: shared library boost_thread.4 does not exist *** Error code 1 Stop in /ports/graphics/libopenraw. *** Error code 1 Stop in /ports/graphics/gegl. *** Error code 1 Stop in /ports/graphics/gimp-app. *** Error code 1 Stop in /ports/graphics/gimp. fatal: failed inkscape: Creating bzip'd tar ball in '/pkg/All/atk-1.28.0.tbz' === Generating temporary packing list tar: lib/gtk-2.0/2.10.0/printbackends/libprintbackend-cups.a: Cannot stat: No such file or directory tar: lib/gtk-2.0/2.10.0/printbackends/libprintbackend-cups.la: Cannot stat: No such file or directory tar: lib/gtk-2.0/2.10.0/printbackends/libprintbackend-cups.so: Cannot stat: No such file or directory tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 Creating package /pkg/All/gtk-2.18.7.tbz Registering depends: atk-1.28.0 cups-client-1.4.2_4 gnutls-2.8.3 libgcrypt-1.4.4 libgpg-error-1.7 pango-1.26.2 shared-mime-info-0.71 gio-fam-backend-2.22.4 gamin-0.1.10_3 glib-2.22.4 gettext-0.17_1 libxml2-2.7.6_1 libiconv-1.13.1_1 libXinerama-1.0.3,1 libXi-1.2.1,1 libXrandr-1.3.0 libXcomposite-0.4.0,1 libXext-1.0.5,1 libXdamage-1.1.1 libXcursor-1.1.9_1 libXfixes-4.0.3_1 libXft-2.1.14 cairo-1.8.8,1 libXrender-0.9.4_1 libX11-1.2.1_1,1 xcb-util-0.3.6_1 libxcb-1.5 libpthread-stubs-0.3_3 pcre-8.00 libXau-1.0.4 libXdmcp-1.0.2_1 xorg-fonts-truetype-7.4 font-bh-ttf-1.0.0 font-misc-meltho-1.0.0_1 font-misc-ethiopic-1.0.0 mkfontdir-1.0.4 mkfontscale-1.0.6 libfontenc-1.0.4 xproto-7.0.15 bitstream-vera-1.10_4 fontconfig-2.8.0,1 freetype2-2.3.11 pixman-0.16.6 pkg-config-0.23_1 jasper-1.900.1_9 tiff-3.9.2_1 jbigkit-1.6 jpeg-8 png-1.2.42 perl-5.10.1 python26-2.6.4 hicolor-icon-theme-0.12 expat-2.0.1_1 encodings-1.0.2,1 font-util-1.0.1 compositeproto-0.4 damageproto-1.1.0_2 fixesproto-4.0 inputproto-1.5.0 kbproto-1.0.3 randrproto-1.3.0 renderproto-0.9.3 xextproto-7.0.5 xineramaproto-1.1.2. Creating bzip'd tar ball in '/pkg/All/gtk-2.18.7.tbz' ***
Re: package building failure irritation
xorquew...@googlemail.com writes: tar: share/sgml/docbook/4.2/ChangeLog: Cannot stat: No such file or directory [...] tar: Error exit delayed from previous errors. pkg_create: make_dist: tar command failed with code 256 Creating package /pkg/All/docbook-4.2.tbz Registering depends: iso8879-1986_2 xmlcatmgr-2.2. Creating bzip'd tar ball in '/pkg/All/docbook-4.2.tbz' *** Error code 1 It looks like it's trying to create the package without first installing the port... which is weird, since package implies install. You're sure it's not a permissions issue? Is your script running as root inside the jail? Is /usr/local writable by root inside the jail? You're not trying to build multiple ports in paralell? This would be easier to diagnose if you showed us the complete log for a single port... DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 xorquew...@googlemail.com wrote: 'Lo. I've come up against an infuriating problem with ports. I don't even know what causes it, let alone how to solve it. Essentially, to stop already-installed software from polluting port builds, I build ports in a jail. I put a list of ports in a file and use a trivial shell script to build them one after the other with: cd $port make clean make deinstall make package-recursive The jail is set up with: /etc/make.conf: WRKDIRPREFIX= /work DISTDIR= /distfiles PACKAGES= /pkg PORTSDIR= /ports With /ports, /distfiles and /pkg being mounted from outside the jail using nullfs (just because those directories exist on a ZFS array outside of the jail for performance reasons). Before building a set of packages, the jail is wiped clean with 'pkg_delete -a'. The problem is that it seems most ports just outright fail to produce packages. I log each build and often the last lines of the log files read something like: xchm: [...] Note that in most cases, the compilation *will* produce a package but will fail to produce packages for dependencies, resulting in pkg_add spewing errors upon install: pkg_add: could not find package liblqr-1-0.4.1 ! pkg_add: could not find package enchant-1.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! pkg_add: could not find package libxml++-2.26.1 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! pkg_add: could not find package libxml++-2.26.1 ! pkg_add: autoload of dependency '/pkg/All/cairomm-1.8.4.tbz' failed! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! pkg_add: could not find package libxml++-2.26.1 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: could not find package libsigc++-2.2.4.2 ! pkg_add: autoload of dependency '/pkg/All/glibmm-2.22.1,1.tbz' failed! And so on and so on. What's the problem here? It surely shouldn't be this hard to use 'make package'. The system is a vanilla install of FreeBSD 8.0 AMD64 with an up-to-date ports tree as of yesterday. xw Hi xw, make package-recursive calls make package-noinstall for all of the dependent ports. I'm assuming that means that the dependent packages all have to be installed first, so I would change your script like so: cd $port make clean make deinstall make depends# just to be safe make install make package-recursive On the other hand, this is similar to what Tinderbox (http://tinderbox.marcuscom.com/README.html), so perhaps that would be a solution for you, too? Hope that helps, Greg - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. http://twitter.com/sourcehosting/ - Follow me, follow you -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLiAdO0sRouByUApARAhK0AKCjyH0T/n6lxt7w3fqaQM0dlh1CZQCgitmN 9R3WnYFpesYLW0mlZkVVzy0= =BCbS -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
'Lo, On 2010-02-26 12:39:26, Greg Larkin wrote: make package-recursive calls make package-noinstall for all of the dependent ports. I'm assuming that means that the dependent packages all have to be installed first, so I would change your script like so: cd $port make clean make deinstall make depends # just to be safe make install make package-recursive I see! I'll edit the script and re-run the build to see if this sorts the problem. On the other hand, this is similar to what Tinderbox (http://tinderbox.marcuscom.com/README.html), so perhaps that would be a solution for you, too? I used it up until about a week ago when I finally lost patience with it. Essentially, I found it too big, too slow, too opaque (too hard to work out what it was currently doing), required too much infrastructure (a working postgres setup - not good if you're trying to bootstrap packages for a new machine) and possibly most annoyingly, the distfile cache is still buggy (it redownloaded the 'qt-anywhere*' source archive about 20 times, making me exceed my monthly bandwidth limit). Oh, there were also serious problems building java packages in tinderbox (due to the 'agree to download' JDK garbage). I wish things were otherwise, but, oh well... xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
cd $port make clean make deinstall make depends # just to be safe make install make package-recursive This apparently didn't solve the problem (same errors). I'm stumped. xw ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
I also build my ports in a jail environment (only its not a really a
jail, just a chroot).
After I have built the ports I want, I create the packages using the
following little script. (My mail client will have mangled the script,
so take care when you copy and paste it, e.g. it needs to be tabs not
spaces and the beginning of lines.)
#!/usr/bin/make -f
PACKAGES?=/usr/home/stephen/packages-8-amd
PKG_DBDIR?=/var/db/pkg
PORTSDIR?=/usr/ports
PKG_LIST!=ls ${PKG_DBDIR}
all:${PKG_LIST:C+(.*)+${PACKAGES}/All/\1.tbz+}
.for target in ${PKG_LIST}
${PACKAGES}/All/${target}.tbz: ${PKG_DBDIR}/${target}/+CONTENTS
@origin=${PORTSDIR}/`sed -n 's/@comment ORIGIN://p' \
${PKG_DBDIR}/${target}/+CONTENTS`; \
if [ $$origin != ${PORTSDIR}/ ] [ -e $$origin ] (! [ -e
$$origin/work ] || [ -e $$origin/work/.install_done* ]); then \
echo creating package ${target}; \
make -C $$origin PACKAGES=${PACKAGES} package-links; \
pkg_create -GjY -b ${target} $@ || (rm -f $@ false); \
fi
.endfor
.BEGIN:
@mkdir -p ${PACKAGES}/All
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 xorquew...@googlemail.com wrote: cd $port make clean make deinstall make depends # just to be safe make install make package-recursive This apparently didn't solve the problem (same errors). I'm stumped. xw Can you post a full log file of the process somewhere? The best way to capture it is with the commands: script /tmp/build.log# Starts a new shell # Run your script here and wait until it finishes exit # Saves the log file and # returns to the original shell dos2unix /tmp/build.log # Gets rid of ^M chars Then post /tmp/build.log somewhere for me to review. Thank you, Greg - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. http://twitter.com/sourcehosting/ - Follow me, follow you -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFLiE1H0sRouByUApARAv81AJ98rkAnQsfymc/HUpg5c6pt85eLsACfaZh2 SFcLzJAJPZBsIcWhyEqif0g= =USeg -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: package building failure irritation
fwiw, the canonical way to find out if a port will package up in a clean environment (choort, with dependencies all loaded via package) is on http://pointyhat.freebsd.org/errorlogs/ , e.g., http://pointyhat.freebsd.org/errorlogs/i386-8-full/ . http://portsmon.freebsd.org gives you lots of cross-references into it and the PR database. Ignore the 'connection failed' message, it is transient and I need to fix it RSN. mcl ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
