OpenPKG CVS Repository http://cvs.openpkg.org/ ____________________________________________________________________________
Server: cvs.openpkg.org Name: Ralf S. Engelschall Root: /e/openpkg/cvs Email: [EMAIL PROTECTED] Module: openpkg-src openpkg-web Date: 18-Sep-2003 13:45:09 Branch: HEAD Handle: 2003091812450404 Added files: openpkg-src/openpkg rpmdb Modified files: openpkg-src/openpkg HISTORY openpkg.boot openpkg.spec rpm.patch.bugfix rpm.patch.feature rpm.patch.porting rpm.patch.regen rpmpopt openpkg-web news.txt Log: Completely revamp the RPM database fiddling: 1. Introduce new <prefix>/lib/openpkg/rpmdb utility for administrating the RPM database on the lower level and hook it into the <prefix>/bin/rpm command line with four options: --db-build RPM database administration: build new database (destructive operation; you have to know what you are doing) --db-rebuild RPM database administration: rebuild new from old database (upgrading operation; reasonable after upgrades or on DB corruption) --db-cleanup RPM database administration: cleanup existing database (cleaning operation; reasonable after DB out-of-sync situations) --db-fixate RPM database administration: fixate existing database (harmless operation; for fixating files only) 2. Use rpmdb from within openpkg.boot to create initial database the correct way and to query the list of filenames of all database files. 3. Use rpmdb from within openpkg.spec:%post to fixate the database on upgrades only (no more rebuilding) but drop note to RPM 4.0 to 4.2 upgraders about "rpm --db-rebuild" and "rpm --db-cleanup". 4. Patch RPM's embedded Berkeley-DB to accept zero-size DB region files. Because permissions and ownership on Berkeley-DB region files __db.XXX have to be already fixed in advance and those files cannot be created in advance on the command line with some reasonable default (as it is the case for the other files which are created with [rpm]db_load), we have to force Berkeley-DB to accept those zero-sized files and (on DB_CREATE) create/overwrite them (instead of thinking another process is currently creating them). This especially workarounds the "Resource temporarily unavailable" problems we've seen on some platforms, which were mainly caused by the pre-existing DB region files. 5. Patch RPM so it passes flag DB_CREATE to Berkeley-DB also if DB region file __db.001 exists, but is (still) zero-size. With those changes all together we are now able to perform lower-level administration of the RPM database, have moved all RPM database fiddling into a single script (rpmdb), this way removed all redundancy related to this database fiddling, and should workround the "Resource temporarily unavailable" problems we experienced. OpenPKG-CURRENT drivers are strongly advised to immediately upgrade to this "openpkg-20030918-20030918" release of the bootstrap package and especially perform an "rpm --db-rebuild" once after the installation. If you have already trouble ("Resource temporarily unavailable" or other hangs) in upgrading the previous "openpkg" package to this latest version, you can workaround it with the following more upgrade procedure (note the "private" on the __dbi_cdb macro definition): $ <prefix>/bin/rpm --rebuild openpkg-20030918-20030918.src.rpm $ rm <prefix>/RPM/DB/__db.* $ <prefix>/bin/rpm -Uvh \ --define '__dbi_cdb create cdb mpool mp_mmapsize=16Mb mp_size=1Mb private' \ <prefix>/RPM/PKG/openpkg-20030918-20030918.*.rpm $ <prefix>/bin/rpm --db-rebuild Summary: Revision Changes Path 1.54 +1 -0 openpkg-src/openpkg/HISTORY 1.34 +12 -30 openpkg-src/openpkg/openpkg.boot 1.219 +54 -56 openpkg-src/openpkg/openpkg.spec 1.21 +1 -1 openpkg-src/openpkg/rpm.patch.bugfix 1.22 +62 -1 openpkg-src/openpkg/rpm.patch.feature 1.25 +1 -1 openpkg-src/openpkg/rpm.patch.porting 1.20 +1 -1 openpkg-src/openpkg/rpm.patch.regen 1.1 +425 -0 openpkg-src/openpkg/rpmdb 1.7 +37 -0 openpkg-src/openpkg/rpmpopt 1.6611 +1 -0 openpkg-web/news.txt ____________________________________________________________________________ patch -p0 <<'@@ .' Index: openpkg-src/openpkg/HISTORY ============================================================================ $ cvs diff -u -r1.53 -r1.54 HISTORY --- openpkg-src/openpkg/HISTORY 16 Sep 2003 15:32:13 -0000 1.53 +++ openpkg-src/openpkg/HISTORY 18 Sep 2003 11:45:07 -0000 1.54 @@ -2,6 +2,7 @@ 2003 ==== +20030918 introduce new rpmdb utility for administrating the RPM database on the lower level 20030916 RPM database mutex workaround for Solaris plus the passing of option -b to patch(1) 20030915 remove the librpmmisc.a library because its contents RPM includes in librpmio.a 20030915 Berkeley-DB fixes for fcntl(2) usage and Linux O_DIRECT issue @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/openpkg.boot ============================================================================ $ cvs diff -u -r1.33 -r1.34 openpkg.boot --- openpkg-src/openpkg/openpkg.boot 27 Aug 2003 21:10:15 -0000 1.33 +++ openpkg-src/openpkg/openpkg.boot 18 Sep 2003 11:45:07 -0000 1.34 @@ -464,7 +464,12 @@ ## echo "++ initializing RPM database" -$tmpdir/rpm --initdb +$RPM_BUILD_ROOT$prefix/lib/openpkg/bash \ +$RPM_BUILD_ROOT$prefix/lib/openpkg/rpmdb \ + --prefix=$RPM_BUILD_ROOT$prefix \ + --dbpath=$RPM_BUILD_ROOT$prefix/RPM/DB \ + --rpm=$tmpdir/rpm \ + --build --quiet ## ## now turn over and re-iterate over the RPM spec, but this time @@ -492,13 +497,6 @@ $tmpdir/rpm --install --justdb --force --noscripts --notriggers --ignoresize \ $RPM_BUILD_ROOT$prefix/RPM/PKG/openpkg-*.rpm -## -## install OpenPKG OpenPGP public key into RPM database -## - -echo "++ installing OpenPKG OpenPGP public key into RPM database" -$tmpdir/rpm --import $RPM_BUILD_ROOT$prefix/etc/openpkg/openpkg.pgp - ## Puhhhh!!! what a tricky bootstrapping procedure. But now we are ## mostly finished. All we finally have to do is to roll a bootstrap ## tarball in addition to the binary RPM and save the stuff in a @@ -560,28 +558,12 @@ sed -e 's;%config(noreplace) *;;' -e 's;%config *;;' \ -e 's;%dir *;;' -e 's;%{l_prefix}/;;' -e 's;^ *;;' -e "s;%{V_rpm};${V_rpm};"` db_files="" -for db in \ - hash:Basenames hash:Conflictname hash:Depends btree:Dirnames hash:Filemd5s \ - hash:Group btree:Installtid hash:Name hash:Packages hash:Providename \ - btree:Provideversion hash:Pubkeys hash:Requirename btree:Requireversion \ - hash:Sha1header hash:Sigmd5 hash:Triggername index:__db.001 index:__db.002 \ - index:__db.003 index:__db.004 index:__db.005 index:__db.006 index:__db.007 \ - index:__db.008 index:__db.009; do - eval `echo $db | sed -e 's/^\(.*\):\(.*\)$/db_type="\1"; db_file="\2";/'` - if [ ! -f $RPM_BUILD_ROOT$prefix/RPM/DB/$db_file ]; then - if [ ".$db_type" = .hash -o ".$db_type" = .btree ]; then - ( echo "VERSION=3" - echo "format=bytevalue" - echo "type=${db_type}" - echo "db_pagesize=16384" - echo "HEADER=END" - echo "DATA=END" - ) | $RPM_BUILD_ROOT$prefix/lib/openpkg/rpmdb_load \ - $RPM_BUILD_ROOT$prefix/RPM/DB/$db_file - else - touch $RPM_BUILD_ROOT$prefix/RPM/DB/$db_file - fi - fi +for db_file in \ + `$RPM_BUILD_ROOT$prefix/lib/openpkg/bash \ + $RPM_BUILD_ROOT$prefix/lib/openpkg/rpmdb \ + --prefix=$RPM_BUILD_ROOT$prefix \ + --dbpath=$RPM_BUILD_ROOT$prefix/RPM/DB \ + --list --quiet`; do db_files="$db_files RPM/DB/$db_file" done chmod 644 $RPM_BUILD_ROOT$prefix/RPM/DB/* @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/openpkg.spec ============================================================================ $ cvs diff -u -r1.218 -r1.219 openpkg.spec --- openpkg-src/openpkg/openpkg.spec 16 Sep 2003 15:32:13 -0000 1.218 +++ openpkg-src/openpkg/openpkg.spec 18 Sep 2003 11:45:07 -0000 1.219 @@ -39,7 +39,7 @@ # o any cc(1) # the package version/release -%define V_openpkg 20030916 +%define V_openpkg 20030918 # the used software versions %define V_rpm 4.2.1 @@ -122,6 +122,7 @@ Source53: rpm-config.sh Source54: rpm-config.pod Source55: rpm-config.8 +Source56: rpmdb # build information Prefix: %{l_prefix} @@ -773,6 +774,11 @@ strip $RPM_BUILD_ROOT%{l_prefix}/lib/openpkg/bash ) || exit $? sed -e "s;@l_prefix@;%{l_prefix};g" \ + -e "s;@l_musr@;%{l_musr};g" \ + -e "s;@l_mgrp@;%{l_mgrp};g" \ + <`SOURCE rpmdb` >$RPM_BUILD_ROOT%{l_prefix}/lib/openpkg/rpmdb + chmod a+x $RPM_BUILD_ROOT%{l_prefix}/lib/openpkg/rpmdb + sed -e "s;@l_prefix@;%{l_prefix};g" \ <`SOURCE rpmx.sh` >$RPM_BUILD_ROOT%{l_prefix}/lib/openpkg/rpmx.sh chmod a+x $RPM_BUILD_ROOT%{l_prefix}/lib/openpkg/rpmx.sh sed -e "s;@l_prefix@;%{l_prefix};g" \ @@ -964,6 +970,7 @@ %{l_prefix}/lib/openpkg/rc %{l_prefix}/lib/openpkg/rpmb %{l_prefix}/lib/openpkg/rpmd + %{l_prefix}/lib/openpkg/rpmdb %{l_prefix}/lib/openpkg/rpmdb_dump %{l_prefix}/lib/openpkg/rpmdb_load %{l_prefix}/lib/openpkg/rpmdb_verify @@ -1589,62 +1596,53 @@ esac fi + # remember what OpenPKG RPM version was installed, so we can + # correctly upgrade the RPM database in the %post section (see below). + if [ -d ${prefix}/RPM/DB -a -x ${prefix}/bin/rpm ]; then + rm -f ${prefix}/RPM/DB/.version >/dev/null 2>&1 || true + ${prefix}/bin/rpm --version |\ + sed -e 's;^;X;' \ + -e 's;^X[^0-9]*\([0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\).*$;\1;' \ + -e 's;^X[^0-9]*\([0-9][0-9]*\.[0-9][0-9]*\).*$;\1;' \ + -e 's;^X.*$;4.0;' \ + >${prefix}/RPM/DB/.version + fi + %post - # Finally, rebuild the RPM database with the newly installed - # RPM program on upgrades (especially important on upgrade from - # RPM-4.0.x/DB-3.2.x to RPM-4.2.x/DB-4.1.x in order to upgrade - # from DB version 7 to DB version 8 hash format files). Because - # of RPM internal database locking we have to play a little trick - # here and delay the operation until RPM thinks this "%post" - # script has already terminated. - if [ ".$1" = .2 ]; then - ( # wait until RPM has released the database - i=0 - while [ $i -lt 10 ]; do - if %{l_prefix}/bin/rpm -q openpkg; then - break - fi - sleep 1 - i=`expr $i + 1` - done - - # rebuild the RPM database - %{l_prefix}/bin/rpm --rebuilddb || true - - # import OpenPKG OpenPGP public key into RPM database - %{l_prefix}/bin/rpm -q gpg-pubkey-63c4cb9f-3c591eda >/dev/null 2>&1 || \ - %{l_prefix}/bin/rpm --import %{l_prefix}/etc/openpkg/openpkg.pgp || true - - # make sure all RPM 4.2.x database files are present - for db in \ - hash:Basenames hash:Conflictname hash:Depends btree:Dirnames hash:Filemd5s \ - hash:Group btree:Installtid hash:Name hash:Packages hash:Providename \ - btree:Provideversion hash:Pubkeys hash:Requirename btree:Requireversion \ - hash:Sha1header hash:Sigmd5 hash:Triggername index:__db.001 index:__db.002 \ - index:__db.003 index:__db.004 index:__db.005 index:__db.006 index:__db.007 \ - index:__db.008 index:__db.009; do - eval `echo $db | sed -e 's/^\(.*\):\(.*\)$/db_type="\1"; db_file="\2";/'` - if [ ! -f %{l_prefix}/RPM/DB/$db_file ]; then - if [ ".$db_type" = .hash -o ".$db_type" = .btree ]; then - ( echo "VERSION=3" - echo "format=bytevalue" - echo "type=${db_type}" - echo "db_pagesize=16384" - echo "HEADER=END" - echo "DATA=END" - ) | %{l_prefix}/lib/openpkg/rpmdb_load \ - %{l_prefix}/RPM/DB/$db_file || true - else - touch %{l_prefix}/RPM/DB/$db_file || true - fi - fi - done - - # fix ownership and permissions of (especially newly created) - # RPM database files to make sure they are consistent - chown %{l_musr}:%{l_mgrp} %{l_prefix}/RPM/DB/* || true - chmod 644 %{l_prefix}/RPM/DB/* || true - ) </dev/null >/dev/null 2>/dev/null & + # determine new and old OpenPKG RPM version + V_rpm_new="%{V_rpm}" + if [ -f %{l_prefix}/RPM/DB/.version ]; then + V_rpm_old="`cat %{l_prefix}/RPM/DB/.version`" + rm -f %{l_prefix}/RPM/DB/.version >/dev/null 2>&1 || true + else + V_rpm_old="4.0" + fi + + # make sure all RPM database files are present and that their + # attributes are correct, etc. This is not a database rebuild + # operation, so this is fully harmless and can be done always. + %{l_prefix}/lib/openpkg/rpmdb --quiet --fixate + + # check RPM database for upgrade requirements + if [ ".$V_rpm_old" != ".$V_rpm_new" ]; then + # RPM 4.0/4.1 does not allow concurrent database access, so we + # cannot upgrade the database immediately from here (because + # we are still running within the execution context of the + # old RPM which in turn has the database locked) and instead + # have to just drop a note to the administrator about database + # upgrading. RPM 4.2 in general allows concurrent database + # access, but the whole stuff is too risky here, too. + ( echo "You are upgrading the OpenPKG bootstrap from a previous" + echo "RPM $V_rpm_old based version to this RPM $V_rpm_new based" + echo "version. The RPM database in %{l_prefix}/RPM/DB/ can still" + echo "be used, but it is strongly recommended to up upgrade it" + echo "as soon as possible to the latest Berkeley-DB format files" + echo "by executing the command:" + echo " \$ %{l_prefix}/bin/rpm --db-rebuild" + echo "In case of problems with the RPM database you can always" + echo "execute the command:" + echo " \$ %{l_prefix}/bin/rpm --db-cleanup" + ) | %{l_prefix}/lib/openpkg/rpmtool msg -b -t warn fi %preun @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/rpm.patch.bugfix ============================================================================ $ cvs diff -u -r1.20 -r1.21 rpm.patch.bugfix --- openpkg-src/openpkg/rpm.patch.bugfix 16 Sep 2003 15:32:14 -0000 1.20 +++ openpkg-src/openpkg/rpm.patch.bugfix 18 Sep 2003 11:45:08 -0000 1.21 @@ -10,7 +10,7 @@ ## 'patch' tool to upgrade those files. Each patch snippet is annotated ## with a short description. ## -## Created on: 16-Sep-2003 +## Created on: 17-Sep-2003 ## ## ATTENTION: THIS PATCH FILE WAS AUTO-GENERATED FROM AN OPENPKG ## RPM CVS REPOSITORY, HENCE DO NOT EDIT THIS FILE. @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/rpm.patch.feature ============================================================================ $ cvs diff -u -r1.21 -r1.22 rpm.patch.feature --- openpkg-src/openpkg/rpm.patch.feature 16 Sep 2003 15:32:14 -0000 1.21 +++ openpkg-src/openpkg/rpm.patch.feature 18 Sep 2003 11:45:08 -0000 1.22 @@ -10,7 +10,7 @@ ## 'patch' tool to upgrade those files. Each patch snippet is annotated ## with a short description. ## -## Created on: 16-Sep-2003 +## Created on: 17-Sep-2003 ## ## ATTENTION: THIS PATCH FILE WAS AUTO-GENERATED FROM AN OPENPKG ## RPM CVS REPOSITORY, HENCE DO NOT EDIT THIS FILE. @@ -1223,3 +1223,64 @@ /[EMAIL PROTECTED]@*/ case 'i': ba->buildAmount |= RPMBUILD_INSTALL; + ++--------------------------------------------------------------------------- +| Because permissions and ownership on Berkeley-DB region files +| __db.XXX have to be already fixed in advance and those files cannot +| be created in advance on the command line with some reasonable +| default (as it is the case for the other files which are created +| with [rpm]db_load), so force Berkeley-DB to accept zero-sized files +| and create them (instead of thinking another process is currently +| creating them). ++--------------------------------------------------------------------------- +Index: db/env/env_region.c +--- db/env/env_region.c 18 Jan 2003 14:05:01 -0000 1.1.1.5 ++++ db/env/env_region.c 17 Sep 2003 14:12:02 -0000 +@@ -184,6 +184,24 @@ + } + + /* ++ * OpenPKG hack: because permissions and ownership on Berkeley-DB ++ * region files __db.XXX have to be already fixed in advance and ++ * those files cannot be created in advance on the command line ++ * with some reasonable default (as it is the case for the other ++ * files which are created with [rpm]db_load), so force Berkeley-DB ++ * to accept zero-sized files and create them (instead of thinking ++ * another process is currently creating them). ++ */ ++ if (mbytes == 0 && bytes == 0) { ++ if (F_ISSET(dbenv, DB_ENV_CREATE)) ++ goto creation; ++ else { ++ ret = ENOENT; ++ goto err; ++ } ++ } ++ ++ /* + * !!! + * A size_t is OK -- regions get mapped into memory, and so can't + * be larger than a size_t. + ++--------------------------------------------------------------------------- +| Make sure RPM passes DB_CREATE to Berkeley-DB also if file exists, +| but is (still) zero-sized. ++--------------------------------------------------------------------------- +Index: rpmdb/db3.c +--- rpmdb/db3.c 3 Mar 2003 16:18:29 -0000 1.1.1.5 ++++ rpmdb/db3.c 17 Sep 2003 14:46:45 -0000 +@@ -1019,9 +1019,13 @@ + } else { /* dbhome is writable, check for persistent dbenv. */ + /[EMAIL PROTECTED]@*/ + const char * dbf = rpmGetPath(dbhome, "/__db.001", NULL); ++ struct stat sb; ++ long size = -1; + /[EMAIL PROTECTED]@*/ + +- if (access(dbf, F_OK) == -1) { ++ if (stat(dbf, &sb) == 0) ++ size = (long)sb.st_size; ++ if (access(dbf, F_OK) == -1 || size == 0) { + /* ... non-existent (or unwritable) DBENV, will create ... */ + dbi->dbi_oeflags |= DB_CREATE; + dbi->dbi_eflags &= ~DB_JOINENV; @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/rpm.patch.porting ============================================================================ $ cvs diff -u -r1.24 -r1.25 rpm.patch.porting --- openpkg-src/openpkg/rpm.patch.porting 16 Sep 2003 15:32:14 -0000 1.24 +++ openpkg-src/openpkg/rpm.patch.porting 18 Sep 2003 11:45:08 -0000 1.25 @@ -10,7 +10,7 @@ ## 'patch' tool to upgrade those files. Each patch snippet is annotated ## with a short description. ## -## Created on: 16-Sep-2003 +## Created on: 17-Sep-2003 ## ## ATTENTION: THIS PATCH FILE WAS AUTO-GENERATED FROM AN OPENPKG ## RPM CVS REPOSITORY, HENCE DO NOT EDIT THIS FILE. @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/rpm.patch.regen ============================================================================ $ cvs diff -u -r1.19 -r1.20 rpm.patch.regen --- openpkg-src/openpkg/rpm.patch.regen 16 Sep 2003 15:32:14 -0000 1.19 +++ openpkg-src/openpkg/rpm.patch.regen 18 Sep 2003 11:45:08 -0000 1.20 @@ -10,7 +10,7 @@ ## 'patch' tool to upgrade those files. Each patch snippet is annotated ## with a short description. ## -## Created on: 16-Sep-2003 +## Created on: 17-Sep-2003 ## ## ATTENTION: THIS PATCH FILE WAS AUTO-GENERATED FROM AN OPENPKG ## RPM CVS REPOSITORY, HENCE DO NOT EDIT THIS FILE. @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/rpmdb ============================================================================ $ cvs diff -u -r0 -r1.1 rpmdb --- /dev/null 2003-09-18 13:45:08.000000000 +0200 +++ rpmdb 2003-09-18 13:45:08.000000000 +0200 @@ -0,0 +1,425 @@ [EMAIL PROTECTED]@/lib/openpkg/bash +## +## rpmdb -- OpenPKG RPM Database Administration Utility +## Copyright (c) 2003 The OpenPKG Project <http://www.openpkg.org/> +## Copyright (c) 2003 Ralf S. Engelschall <[EMAIL PROTECTED]> +## Copyright (c) 2003 Cable & Wireless <http://www.cw.com/> +## +## Permission to use, copy, modify, and distribute this software for +## any purpose with or without fee is hereby granted, provided that +## the above copyright notice and this permission notice appear in all +## copies. +## +## THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED +## WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +## MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +## IN NO EVENT SHALL THE AUTHORS AND COPYRIGHT HOLDERS AND THEIR +## CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +## SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +## LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +## USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +## ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +## OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +## OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +## SUCH DAMAGE. +## + +# program information +progname="rpmdb" + +# configuration defaults +help="" +prefix="@l_prefix@" +dbpath="" +rpm="" +musr="@l_musr@" +mgrp="@l_mgrp@" +mode="" +force=no +verbose=2 + +## +## PARSE COMMAND LINE +## + +# iterate over argument line +for opt +do + case $opt in + -*=*) arg=`echo "$opt" | sed 's/^[-_a-zA-Z0-9]*=//'` ;; + *) arg='' ;; + esac + case $opt in + -h|--help ) help="Usage" ;; + -B|--build ) mode=build ;; + -R|--rebuild ) mode=rebuild ;; + -C|--cleanup ) mode=cleanup ;; + -F|--fixate ) mode=fixate ;; + -L|--list ) mode=list ;; + -f|--force ) force=yes ;; + -q|--quiet ) verbose=0 ;; + -v|--verbose ) verbose=`expr $verbose + 1` ;; + --prefix=* ) prefix=$arg ;; + --dbpath=* ) dbpath=$arg ;; + --rpm=* ) rpm=$arg ;; + --musr=* ) musr=$arg ;; + --mgrp=* ) mgrp=$arg ;; + * ) help="Invalid option \`$opt'"; break ;; + esac +done + +# makre sure exactly one run-time mode is specified +if [ ".$mode" = . ]; then + help="No or invalid run-time mode specified" +fi + +# error or usage message +if [ ".$help" != . ]; then + if [ ".$help" != ".Usage" ]; then + echo "$progname:ERROR: $help" 1>&2 + fi + cat 1>&2 <<EOT +$progname:USAGE: $progname [options] + + -h, --help print usage message (this one) + -B, --build build new RPM database from scratch + -R, --rebuild rebuild new from old RPM database + -C, --cleanup cleanup existing RPM database + -F, --fixate fixate existing RPM database + -L, --list list all RPM database files + -f, --force operate in force mode (no save operations) + -q, --quiet operate in quiet mode (no verbose messages at all) + -v, --verbose operate in more verbose mode (increase verbose level) + --prefix=PREFIX use OpenPKG instance under PREFIX + --dbpath=PATH use OpenPKG RPM database under PATH + --rpm=PROG use OpenPKG RPM executable PROG + --musr=USERNAME use OpenPKG management user USERNAME + --mgrp=GROUPNAME use OpenPKG management group GROUPNAME + +This is OpenPKG rpmdb, an RPM database administration utility, providing +lower-level maintainance functions for the Berkeley-DB 4.1 based RPM 4.2 +database. It allows building new RPM databases from scratch, rebuilding +a new from an old RPM database (content dumping and reloading), +cleaning up problems on an existing RPM database (removal of DB region +files, etc) and fixating the files in an existing RPM database (file +attributes). + +EOT + if [ ".$help" != ".Usage" ]; then + exit 2 + else + exit 0 + fi +fi + +# delayed determination of variables +if [ ".$dbpath" = . ]; then + dbpath="$prefix/RPM/DB" +fi +if [ ".$rpm" = . ]; then + rpm="$prefix/bin/rpm" +fi + +## +## DATABASE FILE INFORMATION +## + +dbfiles=" + hash:Basenames + hash:Conflictname + hash:Depends + btree:Dirnames + hash:Filemd5s + hash:Group + btree:Installtid + hash:Name + hash:Packages + hash:Providename + btree:Provideversion + hash:Pubkeys + hash:Requirename + btree:Requireversion + hash:Sha1header + hash:Sigmd5 + hash:Triggername + region:__db.001 + region:__db.002 + region:__db.003 + region:__db.004 + region:__db.005 + region:__db.006 + region:__db.007 + region:__db.008 + region:__db.009 +" + +## +## HELPER FUNCTIONS +## + +error () { + echo "$progname:ERROR: $*" 1>&2 + exit 1 +} + +warning () { + echo "$progname:WARNING: $*" 1>&2 +} + +verbose () { + local level=$1 + shift + if [ $level -le $verbose ]; then + local lead="" + case "$level" in + 1 ) lead="" ;; + 2 ) lead="" ;; + 3 ) lead=" " ;; + * ) lead=" " ;; + esac + echo "$progname: $lead$*" 1>&2 + fi +} + +rpm () { + local opts="--dbpath `echo $dbpath | sed -e 's;/*$;;' -e 's;$;/;'`" + if [ ".$force" = .yes ]; then + opts="$opts --define '__dbi_private yes'" + fi + verbose 3 "run: $rpm $opts $@" + eval "$rpm $opts \"[EMAIL PROTECTED]"" +} + +rpmdb_load () { + $prefix/lib/openpkg/rpmdb_load ${1+"$@"} +} + +rpmdb_dump () { + $prefix/lib/openpkg/rpmdb_dump ${1+"$@"} +} + +## +## RPM DATABASE OPERATIONS +## + +db_wait () { + # wait until RPM has released the database in case we are called + # asynchronously to RPM (especially important when upgrading from + # RPM 4.0 where concurrent access is still not possible) + verbose 2 "waiting for RPM database to be available" + local i=0 + while [ $i -lt 10 ]; do + if $prefix/bin/rpm -q openpkg >/dev/null 2>&1; then + break + fi + sleep 1 + i=`expr $i + 1` + done + if [ $i -eq 10 ]; then + exit 1 + else + exit 0 + fi +} + +db_remove () { + # remove all known files + verbose 2 "removing (possibly existing) old RPM database DB files" + for dbfile in $dbfiles; do + eval `echo $dbfile | sed -e 's/^\(.*\):\(.*\)$/dbtype="\1"; dbfile="\2";/'` + verbose 3 "removing database file: $dbpath/$dbfile ($dbtype)" + rm -f $dbpath/$dbfile + done +} + +db_init () { + # perform official "initdb" operation + # (is mostly a no-operation in RPM 4.2, but anyway) + verbose 2 "creating new RPM database (built-in RPM procedure)" + rpm --initdb + + # perform some real RPM work, so more database files + # magically spring into existence + verbose 2 "operating on new RPM database" + rpm --import $prefix/etc/openpkg/openpkg.pgp || true + rpm -e gpg-pubkey-63c4cb9f-3c591eda --allmatches || true + + # perform official "rebuilddb" operation in the hope it + # creates even more database files now that we have some content + verbose 2 "rebuilding new RPM database (built-in RPM procedure)" + rpm --rebuilddb +} + +db_unbreak () { + # cleanup DB region files + verbose 2 "cleaning up RPM database DB region files" + for dbfile in $dbfiles; do + eval `echo $dbfile | sed -e 's/^\(.*\):\(.*\)$/dbtype="\1"; dbfile="\2";/'` + if [ ".$dbtype" = .region ]; then + verbose 3 "cleaning up DB file: $dbpath/$dbfile ($dbtype)" + rm -f $dbpath/$dbfile || true + touch $dbpath/$dbfile || true + fi + done +} + +db_extend () { + # make sure all RPM database DB files are present + verbose 2 "making sure RPM database contains all possible DB files" + for dbfile in $dbfiles; do + eval `echo $dbfile | sed -e 's/^\(.*\):\(.*\)$/dbtype="\1"; dbfile="\2";/'` + if [ ! -f $dbpath/$dbfile ]; then + verbose 3 "creating DB file: $dbpath/$dbfile ($dbtype)" + if [ ".$dbtype" = .hash -o ".$dbtype" = .btree ]; then + ( echo "VERSION=3" + echo "format=bytevalue" + echo "type=$dbtype" + echo "db_pagesize=16384" + echo "HEADER=END" + echo "DATA=END" + ) | rpmdb_load $dbpath/$dbfile || true + else + touch $dbpath/$dbfile || true + fi + fi + done +} + +db_reload () { + # rebuilding new from old RPM database DB files by dumping and + # reloading their entire content + verbose 2 "dumping and reloading RPM database DB file contents" + for dbfile in $dbfiles; do + eval `echo $dbfile | sed -e 's/^\(.*\):\(.*\)$/dbtype="\1"; dbfile="\2";/'` + verbose 3 "dumping and reloading DB file: $dbpath/$dbfile ($dbtype)" + if [ -f $dbpath/$dbfile ]; then + if [ ".$dbtype" = .hash -o ".$dbtype" = .btree ]; then + rpmdb_dump $dbpath/$dbfile |\ + rpmdb_load $dbpath/$dbfile.new + rm -f $dbpath/$dbfile + mv $dbpath/$dbfile.new $dbpath/$dbfile + else + rm -f $dbpath/$dbfile || true + touch $dbpath/$dbfile || true + fi + fi + done +} + +db_rebuild () { + # perform official "rebuilddb" operation + verbose 2 "rebuilding RPM database (built-in RPM procedure)" + rpm --rebuilddb +} + +db_operate () { + # perform some read/write operation on RPM database + # (we have no package available, but removing and reimporting + # the OpenPKG OpenPGP key is a harmless thing and always possible) + verbose 2 "performing read/write operation on RPM database" + rpm -q gpg-pubkey-63c4cb9f-3c591eda >/dev/null && \ + rpm -e gpg-pubkey-63c4cb9f-3c591eda --allmatches || true + rpm -q gpg-pubkey-63c4cb9f-3c591eda >/dev/null || \ + rpm --import $prefix/etc/openpkg/openpkg.pgp || true + rpm -qa >/dev/null 2>&1 +} + +db_fixate () { + # fix ownership and permissions of (especially newly created) + # RPM database files to make sure they are consistent + verbose 2 "making sure RPM database files have consistent attributes" + for dbfile in $dbfiles; do + eval `echo $dbfile | sed -e 's/^\(.*\):\(.*\)$/dbtype="\1"; dbfile="\2";/'` + verbose 3 "fixating DB file: $dbpath/$dbfile ($dbtype)" + chown $musr:$mgrp $dbpath/$dbfile 2>/dev/null || true + chmod 644 $dbpath/$dbfile 2>/dev/null || true + done +} + +db_list () { + # list all database files + for dbfile in $dbfiles; do + eval `echo $dbfile | sed -e 's/^\(.*\):\(.*\)$/dbtype="\1"; dbfile="\2";/'` + if [ $verbose -eq 0 ]; then + echo "$dbfile" + elif [ $verbose -eq 1 ]; then + echo "$dbpath/$dbfile" + elif [ $verbose -ge 2 ]; then + echo "$dbpath/$dbfile ($dbtype)" + fi + done +} + +## +## ENVIRONMENT CONSISTENCY CHECKS +## + +# sanity check OpenPKG instance +if [ ".$prefix" = . ]; then + error "OpenPKG instance directory is empty" +fi +if [ ! -d $prefix ]; then + error "OpenPKG instance directory \"$prefix\" not found" +fi +if [ ! -x $prefix/bin/rpm ]; then + error "OpenPKG instance directory \"$prefix\" not valid" +fi + +# sanity check OpenPKG RPM database +if [ ".$dbpath" = . ]; then + error "OpenPKG RPM database directory is empty" +fi +if [ ! -d $dbpath ]; then + error "OpenPKG RPM database directory \"$dbpath\" not found" +fi +if [ ! -w $dbpath ]; then + error "OpenPKG RPM database directory \"$dbpath\" not writable" +fi + +## +## DISPATCH INTO COMMANDS +## + +case "$mode" in + build ) + verbose 1 "BUILDING NEW RPM DATABASE FROM SCRATCH ($dbpath)" + db_remove + db_init + db_extend + db_rebuild + db_operate + db_fixate + ;; + + rebuild ) + verbose 1 "REBUILDING NEW FROM OLD RPM DATABASE ($dbpath)" + db_unbreak + db_extend + db_reload + db_rebuild + db_operate + db_fixate + ;; + + cleanup ) + verbose 1 "CLEANING UP EXISTING RPM DATABASE ($dbpath)" + db_unbreak + db_extend + db_rebuild + db_operate + db_fixate + ;; + + fixate ) + verbose 1 "FIXATING EXISTING RPM DATABASE ($dbpath)" + db_extend + db_fixate + ;; + + list ) + db_list + ;; +esac + +exit 0 + @@ . patch -p0 <<'@@ .' Index: openpkg-src/openpkg/rpmpopt ============================================================================ $ cvs diff -u -r1.6 -r1.7 rpmpopt --- openpkg-src/openpkg/rpmpopt 26 Aug 2003 13:55:17 -0000 1.6 +++ openpkg-src/openpkg/rpmpopt 18 Sep 2003 11:45:08 -0000 1.7 @@ -1,3 +1,30 @@ +## +## Copyright (c) 2000-2003 The OpenPKG Project <http://www.openpkg.org/> +## Copyright (c) 2000-2003 Ralf S. Engelschall <[EMAIL PROTECTED]> +## Copyright (c) 2000-2003 Cable & Wireless <http://www.cw.com/> +## +## Permission to use, copy, modify, and distribute this software for +## any purpose with or without fee is hereby granted, provided that +## the above copyright notice and this permission notice appear in all +## copies. +## +## THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED +## WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +## MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +## IN NO EVENT SHALL THE AUTHORS AND COPYRIGHT HOLDERS AND THEIR +## CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +## SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +## LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF +## USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +## ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +## OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +## OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +## SUCH DAMAGE. +## +## The purpose of this configuration file is to extend and override +## the "rpmpopt" directives in the default RPM POPT configuration with +## OpenPKG specific extensions and adjustments. +## # RPM extensions: "rpm --stowaway" rpm exec --stowaway rpmx.sh --stowaway \ @@ -29,6 +56,16 @@ rpm alias --option --define "!#:+ !#:+" \ --POPTdesc=$"set '%option <name> <value>'" \ --POPTargs=$"<name> <value>" + +# RPM extension: "rpm --db-{build,rebuild,cleanup,fixate}" +rpm exec --db-build rpmdb --build \ + --POPTdesc=$"RPM database administration: build new database (destructive operation; you have to know what you are doing)" +rpm exec --db-rebuild rpmdb --rebuild \ + --POPTdesc=$"RPM database administration: rebuild new from old database (upgrading operation; reasonable after upgrades or on DB corruption)" +rpm exec --db-cleanup rpmdb --cleanup \ + --POPTdesc=$"RPM database administration: cleanup existing database (cleaning operation; reasonable after DB out-of-sync situations)" +rpm exec --db-fixate rpmdb --fixate \ + --POPTdesc=$"RPM database administration: fixate existing database (harmless operation; for fixating files only)" # RPM adjustment: improved and cleaned up "rpm -i/--info" rpmq alias --info --qf '\ @@ . patch -p0 <<'@@ .' Index: openpkg-web/news.txt ============================================================================ $ cvs diff -u -r1.6610 -r1.6611 news.txt --- openpkg-web/news.txt 18 Sep 2003 10:54:08 -0000 1.6610 +++ openpkg-web/news.txt 18 Sep 2003 11:45:04 -0000 1.6611 @@ -1,3 +1,4 @@ +18-Sep-2003: Upgraded package: P<openpkg-20030918-20030918> 18-Sep-2003: Upgraded package: P<cvsd-0.9.20-20030918> 17-Sep-2003: Upgraded package: P<spread-3.17.1-20030917> 17-Sep-2003: Upgraded package: P<gcc34-3.4s20030917-20030917> @@ . ______________________________________________________________________ The OpenPKG Project www.openpkg.org CVS Repository Commit List [EMAIL PROTECTED]