Re: Making rpm5 work with dnf

[Jeff Johnson]

Sent from my iPad

> On Nov 25, 2016, at 11:42 PM, Mark Hatle  wrote:
> 
>> On 11/25/16 5:07 PM, Jeffrey Johnson wrote:
>> 
>>> On Nov 24, 2016, at 8:50 AM, Alexander Kanavin
>>> >> >
>>> wrote:
>>> 
 On 11/22/2016 06:10 PM, Jeffrey Johnson wrote:
 
 Again: I cannot do much more than suggest porting approaches unless I can
 attempt reproducers.
 Please try to expose your git repositories somehow, either publicly or 
 privately.
>>> 
>>> Here it is:
>>> 
>>> https://github.com/kanavin/libhif/commits/master
>>> 
>> 
>> Good. I will take a look at your libhif efforts. Note that libhif is only
>> one of several repositories that are going to be needed.
>> 
>> FWIW, your invitation expired or is otherwise unusable (but at least
>> I can read your code, todo++).
>> 
>> (aside)
>> Before we go too much further here:
>> 
>> What is the intent of this collaboration?
> 
> The functional goal in this work is to stop using smart in the Yocto Project 
> and
> switch to DNF.  (We've found nothing else that is a reasonable alternative 
> that
> is still under active development.)
> 

This part I understand (from conversations mid-summer).

> As far as the way things are done, I'll leave that between you, Alexander and
> anyone else interested.
> 

We're getting there ... perhaps some OpenMandriva involvement after
their weekly meeting 11am EST next Wed. I believe there is PLD interest
as well, perhaps a few other efforts interested in collaborating too.

hth

73 de Jeff

> --Mark
> 
>> You have chosen to fork libhif from
>> rpm-software-management/libhif 
>> 
>> rather than a fork (of a fork) from
>> https://github.com/rpm5/libhif
>> 
>> That forces our coordination to be pulled from the only common
>> root at
>> rpm-software-management/libhif 
>> 
>> which almost certainly precludes any participation from me and rpm5.org
>> 
>> for various reasons.
>> 
>> Note that there are several other efforts attempting a dnf->…->rpm5 tool 
>> chain
>> that I
>> am aware of. Which is why I attempted RPM5 repositories to permit 
>> collaboration, and
>> am perfectly willing to give write access to anyone who wishes.
>> 
>> I am also perfectly willing to let someone other than rpm5.org 
>> 
>> administrate the mess if that
>> is what is desired. I do encourage all of you to collaborate early and work
>> forward from
>> working tools. There’s a fair amount of subtle work that will be needed imho.
>> 
>> What is your intent: collaboration with rpm5.org  or
>> collaboration with rpm-software-management?
>> 
>>> I've fixed what I could by adding rpm5 includes, but the remaining build
>>> issues are all caused by actual API incompatibilities between 4 and 5. I can
>>> file them as separate github issues if you want.
>>> 
>> 
>> Um rpm5.org  and rpm.org  have different 
>> API’s,
>> and are most definitely different
>> implementations these days. Its not just include files, and more than libhif 
>> is
>> going to
>> be needed to build a working dnf->…->rpm5 tool chain.
>> 
>> I’ll take a look at your libhif this weekend.
>> 
>> 73 de Jeff
>>> Alex
>>> 
>>> __
>>> RPM Package Managerhttp://rpm5.org
>>> User Communication List rpm-users@rpm5.org
>>> 
>> 
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm 5.1.9 issues on powerPC architecture

[Jeff Johnson]

On Apr 14, 2016, at 12:15 PM, Jeff Johnson wrote:

> 
>> >>>> rpm files are generated on x86_64 machines but querying it on ppc 
>> >>>> machines (arch within rpm is ppc). Inbuilt metadata can be queried 
>> >>>> using --qf but not custom metadata. I guessed that endianness should 
>> >>>> not have impact rpm metadata content. But, even if it has, then it 
>> >>>> should give garbled values and not "(none)" as its result. However, 
>> >>>> querying custom metadata works fine in 5.4.14.
>>  
> 
> Yes arbitrary tags in metadata should be platform independent as a goal. But 
> not in rpm-5.1.9 == a bug.
> 

I should state the problem more carefully:

The meatadata and the tags assigned to the arbitrary metadata ARE independent
of platform in the sense that that are marked with 0xc000. What is/was 
broken
is that the truncated SHA1 for the remaining 30bits of the tag assignment is 
little
endian.

The fix on PPC and big-endian machines is to swab the 30bits before generating
the tag assignment.

> The tag itself, not its content, is what is not platform independent. Hence 
> not "garbled".
> 

The fix on PPC and big-endian machines is to swab the 30bits before generating
the tag assignment.

So its a fairly simple one line patch in rpmio/tagname in _tagGenerate() in 
rpm-5.1.9
that end up looking should look like this:

if (digest && digestlen > 4) {
  /* The tag is stored in a uniform byte order for cross-endian 
compatib  ility. 
 Swap to the host uses. */
  memcpy(&tag, digest + (digestlen - 4), 4);
  tag = le32toh(tag);
  tag = (rpmTag) (tag & 0x3fff);
  tag = (rpmTag) (tag | 0x4000);
  }   

73 de Jeff

Re: rpm 5.1.9 issues on powerPC architecture

[Jeff Johnson]

On Apr 12, 2016, at 2:14 PM, Rajul Bhavsar wrote:
> 
> Hmm why rpm-5.1.9, which was released quite some time ago?
>  Yes, its older version. But that is what current version we had. Will 
>  like to move to 5.4.14 in near future and that rpm version performs all 
>  required queries perfectly fine.
>> $ rpm -qai
>> rpm: -qai: unknown option
>> $
>> 
> 
> Try a simpler query first: does rpm -qa "work"?
>  
>  Yes, -qa works, but --xml or --yaml do not work along with -qpi. Also, 
>  use of --force give same error in rpm -iv command.
>  

OK. All of -i/--info, --xml and --yaml are popt aliases.

Make sure (with strace -e open) that rpm can read /usr/lib/rpm/rpmpopt.

You can also (for --xml and --yaml at least) verify with --queryformat.

All that --yaml does is add this --queryformat option:

rpm -q --qf '[%{*:yaml}\n]' bash

> The -i option is contextually dependent with multiple meanings in rpm
> and the processing is painfully complex. Identifying what does and does not
> work will be helpful in finding a fix.
> 
>> When we try to query custom metadata (implemented using arbitrary tags) then 
>> also it is failing (giving output as (none)). We have proper macro file in 
>> /usr/lib/rpm-5.1.9/macros and rpm is referring to it.
>> 
> 
> (from memory of a recent patch from Poky/Yocto)
> 
> The arbitrary tag values are likely in native endian.
> Did you build and query both on the same machine, or are you
> trying to query a package built on x86_64 on a ppc* machine?
>  
>  rpm files are generated on x86_64 machines but querying it on ppc 
>  machines (arch within rpm is ppc). Inbuilt metadata can be queried using 
>  --qf but not custom metadata. I guessed that endianness should not have 
>  impact rpm metadata content. But, even if it has, then it should give 
>  garbled values and not "(none)" as its result. However, querying custom 
>  metadata works fine in 5.4.14.
>  

Yes arbitrary tags in metadata should be platform independent as a goal. But 
not in rpm-5.1.9 == a bug.

The tag itself, not its content, is what is not platform independent. Hence not 
"garbled".

With arbitrary tags, there are never any missing tags, any tag name will do.
RPM queries have always returned '(none)' in-band (i.e. in the actual output 
string)
for optional tags. That behavior has been extended to arbitrary tags.

hth

73 de Jeff

Re: rpm 5.1.9 issues on powerPC architecture

[Jeff Johnson]

On Apr 11, 2016, at 1:25 PM, Rajul Bhavsar wrote:

> Hi,
> 
> We are trying to use rpm 5.1.9 on powerPC architecture. But, we see that 
> there are issues with basic querying:
> 

Hmm why rpm-5.1.9, which was released quite some time ago?

> $ rpm -qai
> rpm: -qai: unknown option
> $
> 

Try a simpler query first: does rpm -qa "work"?

The -i option is contextually dependent with multiple meanings in rpm
and the processing is painfully complex. Identifying what does and does not
work will be helpful in finding a fix.

> When we try to query custom metadata (implemented using arbitrary tags) then 
> also it is failing (giving output as (none)). We have proper macro file in 
> /usr/lib/rpm-5.1.9/macros and rpm is referring to it.
> 

(from memory of a recent patch from Poky/Yocto)

The arbitrary tag values are likely in native endian.
Did you build and query both on the same machine, or are you
trying to query a package built on x86_64 on a ppc* machine?

> $ rpm --showrc | grep macros
> macrofiles: 
> /usr/lib/rpm-5.1.9/macros:/usr/lib/rpm-5.1.9/ppc-linux/macros:/etc/rpm/5.1.9/macros.*:/etc/rpm/5.1.9/macros:/etc/rpm/5.1.9/ppc-linux/macros:~/.rpmmacros-5.1.9
> $
> 
> We see that on x86_64 these things work fine. We do not see anything specific 
> to architecture in the build of rpm, but only difference is that - host's 
> arch is x86_64.
> 
> Can anybody tell what is peculiar about rpm on powerPC arch?
> 

hth

73 de Jeff
> Thanks,
> Rajul

Re: RPM sqlite3 support

[Jeff Johnson]

On Jan 13, 2016, at 6:34 AM, devzero2000 wrote:

> 
> On Tue, Jan 12, 2016 at 10:06 PM, Mark Hatle  wrote:
> On 1/12/16 2:06 PM, Tim Mooney wrote:
> > In regard to: Re: RPM sqlite3 support, Jeffrey Johnson said (at 12:46pm 
> > on...:
> >
> >> The sqlite3 code (and support) in rpm5 was abandoned in favor of
> >> Berkeley DB ACID transactional support quite some years ago
> >
> > I've been meaning to ask about this for a while, and this provides a
> > good segue...
> >
> > With Oracle's license change on BDB 6.x (or 12.x, or whatever they're
> > calling it) to AGPL, does that impact rpm5's long-term use of BDB?
> 
> I know we and many of our commercial customers has rejected BDB 6.x because of
> the change, so we've been forced to 'support' BDB 5.x.
> 
> Personally I would like to get rid of anything that has to do with BerkeleyDB,
> just to get rid of this or future license questions from customers.  (But
> unfortunately I don't know what can reasonably replace BDB.)
> 
> Hello 
> 
> FWIW, @rpm.org started a plan to replace the rpmdb format
> 
> https://fedoraproject.org/wiki/Changes/NewRpmDBFormat
> 
> Here the original announcement
> 
> http://comments.gmane.org/gmane.linux.redhat.fedora.devel/215427
> 
> Currently no details are known (to me)

There aren't many details, certainly the reasons for change are not clearly
expressed in the description:

The current implementation of the RPM Database is based on Berkeley DB. 
There are doubts about the its future
 and level of maintenance. In addition rpm's use of the database has 
multiple issues on its own. As a result RPMx
 upstream is working to replace the database format with a new 
implementation.

The implementation (assuming rpm/lib/backend/ndb is the new format) is a rather
straightforward hash based database afaict. I am not seeing the necessary
interconversion tools between the formats, not hard to write when the time 
comes.
I'm also not seeing provisions for transactions and ACID and caching and
all the other sophistication/complexity that Berkeley DB provides, but
perhaps I am not appreciating the ndb implementation from a single
pass through the code.

There is also no obvious signs of remote access and/or replication. The API is
simple enough that any form of RPC will serve for remoting. OTOH, there's
much much more to replication than RPC.

The discussions are mostly legitimate concerns about compatibility and its 
pretty
clear that the proposed solution for other projects is
Use rpmlib API's if you wish compatibility with an rpmdb.
Which isn't bad advice, just opposite from the entire intent of using a 
"standard"
API in Berkeley DB way back when so that other/better implementations than rpm
could be developed (of course the main problem there is that the Packages 
database
contains a header blob rather than simple/rich data types that 
applications/developers expect).

Most of the applications that use an rpmdb appear to be a 1-time data scrape of 
an rpmdb
(libguestfs with db_dump(1) and what has been done in yum/zypper in the past). 
A 1-time
data scrape of an entire rpmdb perhaps indicates that no database whatsoever in 
rpm
is what developers wish: there is no reason I know that a pile of *.rpm files 
cannot replace
the /var/lib/rpm/Packages store and leave indexing to applications (which are 
doing the indexing
into the data scrape already).

*shrug*

C+ would be my grade for what I am seeing implemented in rpm/lib/backend/ndb

hth

73 de Jeff
> 
> Elia 
>  
> 
> --Mark
> 
> > Tim
> >
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org
> 

Re: How to detect rpm post scriptlet failure

[Jeff Johnson]

On Oct 13, 2011, at 6:13 AM, Levon Poghosyan wrote:

> It seems to be there is no chance to do this in rpm 4.8.1, am I right ? 

rpm-4.8.1 (and later/earlier versions from @rpm.org) does not save scriptlet
exit codes persistently.

There are also side-effects introduced by ignoring scriptlet exit codes like 
here:

https://bugzilla.redhat.com/show_bug.cgi?id=569930 

If you have an application that is installing a bunch of rpm packages, and
some of those packages have a %post scriptlet failing, then the failures need
to be anaylized, classified, and remedied somehow.

If you can detect "failure", then you should be able to diagnose and fix the
package scriptlets.

Asking RPM to detect failures, when the only information
available is the scriptlet exit code, which is ignored and not saved 
persistently in
rpm-4.8.1, simply isn't a feasible approach.

73 de Jeff

> 
> On 13 October 2011 11:04, devzero2000  wrote:
> On Thu, Oct 13, 2011 at 9:15 AM, Levon Poghosyan  
> wrote:
> Thanks for the reply.
> 
> The situation is following. I've got an application which installs bunch of 
> rpm packages. 
> Some of those packages have post scriptlet failing, so I need to know/figure 
> out which are failing. 
> Those packages are many, so I think it wont be the best solution to modify 
> all off them to store their post install scriptlet execution status in /var, 
> any other solution for this case ? 
> 
> Regards,
> Levon
> 
> FWIW, there is a related bugzilla for @rpm.org 
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=569930 
> 
> (rpm exit 0 always on scriptlet execution)
> 
> In other package management system exists state as "half installed" or so. But
> here are two issue relevant i think :
> 
> - Should be failing script relevant or not (see the bugzilla for a discussion 
> on this, in
> particular for %post) ? @rpm.org choose to exit always with 0
> 
> - How should applications or users know that there is some errors ?
> 
> In any case @rpm5.org add the scriplets exit code to the header registrated 
> in a rpmdb and also to the source rpm (for detecting a wrong use of 
> --short-circuit). 
> 
> rpm -q --yaml arbitrarytags
> (snip)
> .
> 
> Scriptstates:
> - 0
> - 0
> - 0
> - 131072 < %post exit code : this number mean exit 0
> 
> 
> you can also use 
> 
> rpm -q --qf '[%{SCRIPTSTATES}]\n' arbitrarytag
> 000131072
>   
> ^
> |
> 
> Hope this help
> 
> 
> Best Regards
> 
> 
> 
> 
> 
> 
> here are two issues relevant (imho
>Should failing scripts be permitted?
> 
> and a obscurely related general design issue
> 
>How should applications and users and ... be notified of errors?
> 
> 
>   
> 
> On 12 October 2011 21:22, Jeff Johnson  wrote:
> 
> On Oct 12, 2011, at 11:38 AM, Levon Poghosyan wrote:
> 
> > Hello,
> >
> > How can I detect if the execution of post scriptlet of the rpm package 
> > failed ?
> 
> The simple answer here is:
>You don't.
> 
> All I mean by that is that the %post script needs to be written "robustly"
> so that the exit code is always 0.
> 
> If you need/want to tell whether some specific operation "worked",
> then write a test after rpm has run for that specific operation.
> 
> Another approach would be to have the %post script register its
> state somewhere in /var so that you can easily test whether the script
> "worked" or not.
> 
> > I've generated the and rpm package which has a test command "lalala" in 
> > post install section. So during the installation it prints out information 
> > that command lalala was not found but the installation is still successful.
> > How do I identify this failure from post install section.
> 
> A %post "scriptlet" (the only difference between script and scriptlet is that 
> a
> scriplet is macro expanded and may eventually have some envvar's
> prepended instead of having RPM add to the environ directly) is
> just a script.
> 
> SO use test(1) to test for existence and executability, and write that into
> the %post section directly.
> 
> Because a %post is part of a package install state machine, the script
> SHOULD return 0 for all but catasstrophic faiulures.
> 
> There are side-effects of returning failure from %post, the most important
> of which is that on an upgrade, the erase will be skipped if/when the install 
> fails.
> 
> >
> > Please note I'm not interested in failure in other places, I just ne

Re: How would one extract the trigger scripts from a spec file.

[Jeff Johnson]

On Oct 12, 2011, at 4:36 PM, James Olin Oden wrote:

> I figured out that there are a bunch of tags with TRIGGER at the
> begining of the name that form some set of parallel arrays of
> information on all the triggers:
> 
>   TRIGGERFLAGS
>   TRIGGERINDEX
>   TRIGGERNAME
>   TRIGGERSCRIPTPROG
>   TRIGGERSCRIPTS
>   TRIGGERVERSION
>   TRIGGERCONDS
>   TRIGGERTYPE
> 

You've mixed in some header tag extensions with
actual tags that are present in packages.

The extensions are there to make it easier to use
--queryformat to display trigger scripts, not anything
deep.

Triggers have:
   1) a "firing" condition
   2) a script and interpreter to run when "fired"

> In my case I just want to run the scripts through bash -c and validate
> that there syntax is good, and show the information on the script.

Likely
rpm -qp --triggers *.rpm | …
into a script is the easiest path for a bash -c syntax check.

(aside)
Note that the same syntax check is automated by
invoking bash --rpm-requires @rpm5.org (and with triggers
as well iirc). The back port likely isn't hard, and it is as easy
to use "bash -c" as it is to do "bash --rpm-requires" if desired.

The goal would be better automation: fail the build on scriptlet syntax errors.

> However I can't seem to locate the scripts, and I figure the "name"
> would be a concatenation of the trigger name an d the type, but I'm
> not sure how to interpret the type.   I can dig into the source and
> figure this out, but before that can anyone point me to a quick
> explanation of the tags involving triggers or could they explain them?
> 

The basic change for RPMTAG_TRIGGER* tag storage is merely this:
For most scriptlets packages <-> script relation is 1-to-1.
But there can be multiple triggers in a single package so the
the store is 1-to-N (i.e. one package can have multiple triggers).
The TRIGGERINDEX is "glue" to make 1-to-N possible (and also handles
the various types of triggers which are distinguished by bits in *FLAGS.

(aside)
I used to think it was a good idea to move all scriptlets into a storage
like used by triggers to reduce clutter in the RPMTAG_* name space.

Then along came LSB demanding "standard" compliance for no real purpose
and it became infeasible to change anything at all in RPM without also
attempting parallel change to LSB "standards" and politcs took over …

*shrug*

Holler if you have a specific question and I'll look at code.

hth

73 de Jeff



__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How to detect rpm post scriptlet failure

[Jeff Johnson]

On Oct 12, 2011, at 11:38 AM, Levon Poghosyan wrote:

> Hello,
> 
> How can I detect if the execution of post scriptlet of the rpm package failed 
> ? 

The simple answer here is:
You don't.

All I mean by that is that the %post script needs to be written "robustly"
so that the exit code is always 0.

If you need/want to tell whether some specific operation "worked",
then write a test after rpm has run for that specific operation.

Another approach would be to have the %post script register its
state somewhere in /var so that you can easily test whether the script
"worked" or not.

> I've generated the and rpm package which has a test command "lalala" in post 
> install section. So during the installation it prints out information that 
> command lalala was not found but the installation is still successful.
> How do I identify this failure from post install section. 

A %post "scriptlet" (the only difference between script and scriptlet is that a
scriplet is macro expanded and may eventually have some envvar's
prepended instead of having RPM add to the environ directly) is
just a script.

SO use test(1) to test for existence and executability, and write that into
the %post section directly.

Because a %post is part of a package install state machine, the script
SHOULD return 0 for all but catasstrophic faiulures.

There are side-effects of returning failure from %post, the most important
of which is that on an upgrade, the erase will be skipped if/when the install 
fails.

> 
> Please note I'm not interested in failure in other places, I just need to be 
> informed in post install scriptlet failed.
> 

Personal;ly, I'd just write the %post script to write 1 line into 
/var/lib/application/state
with the message
The %post script "succeeded"
or (on failure)
The %post script "failed".

I'd have to know more about what is implied by a %post success/failure
in order to suggest some other approach.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm install and folder creation

[Jeff Johnson]

On Sep 4, 2011, at 3:33 PM, Belal Salem wrote:

> Thanks! According to all of that, I found the best solution just for now, is 
> to rename rpm -> rpm.exec
> and make a 'rpm' script that executes rpm.exec --noparentdirs $* all the time 
> unless I don't want it, I can use rpm.exec itself.
> 

Poifect! Note that I _ALWAYS_ wrap rpm/rpmbuild in ~/bin/*rpm.

(aside)
You might check the persistence enabled by
--disable-dirname-and-symlink-deps
by doing
rpm --showrc | more
All that the build options is doing is setting/exposing 2 macros.
If the macros default value is "opt-out" rather than "opt-in",
that too would explain what you are seeing (and the AutoFu is
functional as intended: devzero2000 usually does careful AutoFu
but building-with-macro-disablers-for-parentdirs isn't
the same thing as configuring-the-macro-disablers-for-parentdirs-to-off.

That permits me to have multiple versions of RPM installed,
all configured differently, and to be able to do build-install
cycles without having to reconfigure the box itself.

hth
> On Sun, 04 Sep 2011 21:20:33 +0200, devzero2000  wrote:
> 
>> On Sun, Sep 4, 2011 at 4:47 PM, Jeff Johnson  wrote:
>> 
>>> 
>>> On Jun 4, 2010, at 2:55 AM, Belal Salem wrote:
>>> 
>>> > Hi there!
>>> > I issued the same issue before, when installing some packages, the RPM
>>> package manager doesn't create the required folders and ask for the folders
>>> as unresolved dependencies, although those folders are part of the package
>>> being installed.
>>> 
>>> Its "part of the package" which is confusing.
>>> 
>>> There are two meanings for "part of the package":
>>> 
>>> 1) directory components as part of file paths
>>> 2) directory explicitly listed in "rpm -qpl *.rpm"
>>> 
>>> If its not explicitly in the file manifest, its not "part of the package"
>>> and you *will* see what you are reporting.
>>> 
>>> > Recompiling RPM with the options: --disable-dirname-and-symlink-deps
>>> didn't solve the problem, anyway through that?!
>>> >
>>> 
>>> I'm not the person to "fix" --disable-dirname-and-symlink-deps.
>>> 
>>> My fix will be to rip out the "Have it your own way!" functionality that
>>> isn't "working"
>>> and remove the
>>>   --disable-dirname-and-symlink-deps
>>> in order to simplify RPM's build and clarify "supported" functionality. I
>>> see
>>> no future in carrying around functionality that doesn't work as it should
>>> and is "vendor supported" by others here @rpm5.org.
>>> 
>>> I will rip out the option if it isn't fixed by someone else @rpm5.org
>>> this month.
>>> 
>>> No, it is right that i rip out the option tomorrow. I have introduced it,
>> it is equivalent to "vendor supported" - i am pretty sure -  but probably
>> don't work as it should. But if so there should be a spec with
>> directory-symlink broken deps that does not work even in Mandriva for
>> example. I would love to see this spec, possibly for tomorrow. Perhaps ark
>> linux and mandriva could be find useful. Thanks. Best Regards
>> 
>> PS
>> 
>> FWIW, this is description of this disabler
>> https://blueprints.launchpad.net/rpm/+spec/rpm-split-vendor-config-in-autofu.
>> <https://blueprints.launchpad.net/rpm/+spec/rpm-split-vendor-config-in-autofu>
>> 
>> The idea was to simplify bootstrapping distributions that do
>> not use rpm5 as a package manager or that are broken from a  QA POW
>> such  as RHEL5:
>> in short for simplify rpm5 adoption in first place. I am sure someone @
>> rpm5.org had used patch - not upstream - similar for RHEL5. My memory is not
>> so bad, i have seen this in a mail some year ago. The old dog can have good
>> memory, yes.
>> 
>> The idea was
>> 
>>> 73 de Jeff
>>> 
>>> __
>>> RPM Package Managerhttp://rpm5.org
>>> User Communication List rpm-users@rpm5.org
>>> 
> 
> 
> -- 
> Using Opera's revolutionary email client: http://www.opera.com/mail/
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm install and folder creation

[Jeff Johnson]

On Jun 4, 2010, at 12:04 PM, Belal Salem wrote:

> Thanks for explaining that!
> But now, all the RPM packages that I have this issue with are installing 
> correctly with RPM-4.x as that the directory structure are listed when using 
> -qlp, but with RPM-5.3.11, the package manager complain about the directories 
> until I use the --nodeps to workaround it, hence it installs correctly and it 
> does create the needed directories.
> 

Instead of --nodeps, try --noparentdirs/--nolinktos disablers.

Yes. I've described another "recommended" procedure to track
the "orphan" (i.e. not explicitly included in any package) directories
and "dangling" (i.e. the end-point is not in any package) symlinks.

To mask a single directory during development/bootstrapping
until you can correct what is intrinsically a packaging flaw (imho):

mkdir -p /etc/rpm/sysinfo
echo "/some/orphan/directory/" >> /etc/rpm/sysinfo/Dirnames
echo "/some/dangling/symlink: >> /etc/rpm/sysinfo/Filelinktos

You -- in fact -- can satisfy any missing dependency using /etc/rpm/sysinfo:
echo "libmissing.so" >> /etc/rpm/sysinfo/Providename
echo "awol-package = 1:2.3" >> /etc/rpm/sysinfo/Providename

If you don't wish the manual one-by-one setup tedium, then
use --noparentdirs --nolinktos (or --nodeps) while installing into a chroot
(with --root /some/chroot) and then extract *all* of the
parentdir/linkto flaws like
rpm -Va --nofiles | grep '^/' | sort -u > /tmp/Dirnames.new

Compiling the disabler directly into RPM prevents usage. You can do
that as well if you wish: just its "Have it your own way!" supported
(and I am not the person to ask for support of what I believe isn't
a feature).

hth

73 de Jeff
> 
> On Sun, 04 Sep 2011 16:47:38 +0200, Jeff Johnson  wrote:
> 
>> 
>> On Jun 4, 2010, at 2:55 AM, Belal Salem wrote:
>> 
>>> Hi there!
>>> I issued the same issue before, when installing some packages, the RPM 
>>> package manager doesn't create the required folders and ask for the folders 
>>> as unresolved dependencies, although those folders are part of the package 
>>> being installed.
>> 
>> Its "part of the package" which is confusing.
>> 
>> There are two meanings for "part of the package":
>> 
>> 1) directory components as part of file paths
>> 2) directory explicitly listed in "rpm -qpl *.rpm"
>> 
>> If its not explicitly in the file manifest, its not "part of the package"
>> and you *will* see what you are reporting.
>> 
>>> Recompiling RPM with the options: --disable-dirname-and-symlink-deps didn't 
>>> solve the problem, anyway through that?!
>>> 
>> 
>> I'm not the person to "fix" --disable-dirname-and-symlink-deps.
>> 
>> My fix will be to rip out the "Have it your own way!" functionality that 
>> isn't "working"
>> and remove the
>>  --disable-dirname-and-symlink-deps
>> in order to simplify RPM's build and clarify "supported" functionality. I see
>> no future in carrying around functionality that doesn't work as it should
>> and is "vendor supported" by others here @rpm5.org.
>> 
>> I will rip out the option if it isn't fixed by someone else @rpm5.org
>> this month.
>> 
>> 73 de Jeff
>> 
>> __
>> RPM Package Managerhttp://rpm5.org
>> User Communication List rpm-users@rpm5.org
>> 
> 
> 
> -- 
> Using Opera's revolutionary email client: http://www.opera.com/mail/
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm install and folder creation

[Jeff Johnson]

On Jun 4, 2010, at 2:55 AM, Belal Salem wrote:

> Hi there!
> I issued the same issue before, when installing some packages, the RPM 
> package manager doesn't create the required folders and ask for the folders 
> as unresolved dependencies, although those folders are part of the package 
> being installed.

Its "part of the package" which is confusing.

There are two meanings for "part of the package":

1) directory components as part of file paths
2) directory explicitly listed in "rpm -qpl *.rpm"

If its not explicitly in the file manifest, its not "part of the package"
and you *will* see what you are reporting.

> Recompiling RPM with the options: --disable-dirname-and-symlink-deps didn't 
> solve the problem, anyway through that?!
> 

I'm not the person to "fix" --disable-dirname-and-symlink-deps.

My fix will be to rip out the "Have it your own way!" functionality that isn't 
"working"
and remove the
--disable-dirname-and-symlink-deps
in order to simplify RPM's build and clarify "supported" functionality. I see
no future in carrying around functionality that doesn't work as it should
and is "vendor supported" by others here @rpm5.org.

I will rip out the option if it isn't fixed by someone else @rpm5.org
this month.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Why rpm5 and smart for Belenix

[Jeff Johnson]

On Sep 3, 2011, at 3:48 PM, pinto.e...@gmail.com wrote:

> Sorry for the top posting. I think @rpm5.org is glad of this. Just a, perhaps 
> naive, question : What mean for a package manager to be more zfs-aware ? 
> Thanks and best regards.

For starters:

ZFS has snapshots for --rollback.

And Solaris has SEEK_HOLE, useful if sparse files ever need to be packaged.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm -U completes with non-zero return code but no error message

[Jeff Johnson]

On Aug 20, 2011, at 3:31 AM, akiyoshi kawamura wrote:

> Hello,
> 
> I tried to upgrage mysql_devel package by having all required rpms in
> a directory. I used the following command to do the upgrade
> 
> /bin/rpm --replacepkgs --oldpackage -Uvh /v/rpm/sw/var/petidomo/*.rpm; echo 0
> 
> Here is the output; there are many warning messages but no error
> message. The return code is 93. What does the return code imply?

The return code in rpm is the number of failed items, in this case
packages that you asked to be installed that were not.

> Non-zero return code means there are failures during the upgrade. But
> without error messages I am not sure what to look for.
> 

Because users tend to do
rpm -Uvh … *.rpm
(as you have done) rpm filters the given arguments when there are multiple 
packages
with identical names, choosing the "newest" package.

The warnings are exactly the error messages that you are seeking.

73 de Jeff


> warning: 
> /root/e-admin_5.0/rpms/cs.latest/mysql_devel_all/apr-1.3.9-3.fc13.x864.rpm:
> Header V3 RSA/SHA256 Signature, key ID 97a1071f: NOKEY
> warning: package apr = 1.3.9-3.fc13 was already added, replacing with
> apr > 1.5-1.fc14
> warning: package apr-util = 1.3.12-1.fc14 was already added, skipping
> apr-util 1.3.9-3.fc13
> warning: package apr-util-ldap = 1.3.12-1.fc14 was already added,
> skipping aprtil-ldap > 1.3.9-3.fc13
> warning: package glibc = 2.12.90-17 was already added, replacing with
> glibc >13-1
> warning: package glibc-common = 2.12.90-17 was already added,
> replacing with gbc-common > 2.13-1
> warning: package glibc-devel = 2.12.90-17 was already added, replacing
> with glc-devel > 2.13-1
> warning: package glibc-headers = 2.12.90-17 was already added, replacing with
> ibc-headers > 2.13-1
> warning: package httpd = 2.2.16-1.fc14 was already added, replacing with httpd
>  2.2.17-1.fc14
> warning: package httpd-manual = 2.2.16-1.fc14 was already added, replacing wit
> httpd-manual > 2.2.17-1.fc14
> warning: package httpd-tools = 2.2.16-1.fc14 was already added, replacing with
> ttpd-tools > 2.2.17-1.fc14
> warning: package krb5-devel = 1.8.2-10.fc14 was already added, skipping krb5-d
> el > 1.8.2-6.fc14
> warning: package krb5-libs = 1.8.2-10.fc14 was already added, skipping krb5-li
>  > 1.8.2-6.fc14
> warning: package krb5-workstation = 1.8.2-10.fc14 was already added, skipping
> b5-workstation > 1.8.2-6.fc14
> warning: package libcom_err = 1.41.12-5.fc14 was already added, replacing with
> ibcom_err > 1.41.12-6.fc14
> warning: package libcom_err-devel = 1.41.12-5.fc14 was already added, replacin
> with libcom_err-devel > 1.41.12-6.fc14
> warning: package libselinux = 2.0.96-5.fc14 was already added, replacing with
> bselinux > 2.0.96-6.fc14.1
> warning: package libselinux-devel = 2.0.96-5.fc14 was already added, replacing
> ith libselinux-devel > 2.0.96-6.fc14.1
> warning: package libselinux-python = 2.0.96-5.fc14 was already added, replacin
> with libselinux-python > 2.0.96-6.fc14.1
> warning: package libselinux-utils = 2.0.96-5.fc14 was already added, replacing
> ith libselinux-utils > 2.0.96-6.fc14.1
> warning: package mysql = 5.1.51-2.fc14 was already added, replacing with mysql
>  5.1.56-1.fc14
> warning: package mysql-libs = 5.1.51-2.fc14 was already added, replacing with
> sql-libs > 5.1.56-1.fc14
> warning: package mysql-server = 5.1.51-2.fc14 was already added, replacing wit
> mysql-server > 5.1.56-1.fc14
> warning: package ncurses = 5.7-8.20100703.fc14 was already added, replacing wi
>  ncurses > 5.7-9.20100703.fc14
> warning: package ncurses-base = 5.7-8.20100703.fc14 was already added, replaci
>  with ncurses-base > 5.7-9.20100703.fc14
> warning: package ncurses-libs = 5.7-8.20100703.fc14 was already added, replaci
>  with ncurses-libs > 5.7-9.20100703.fc14
> warning: package nss-softokn-freebl = 3.12.10-1.fc14 was already added, skippi
>  nss-softokn-freebl > 3.12.7-6.fc14
> warning: package openssl = 1.0.0a-2.fc14 was already added, replacing with ope
> sl > 1.0.0d-1.fc14
> warning: package openssl-devel = 1.0.0a-2.fc14 was already added, replacing wi
>  openssl-devel > 1.0.0d-1.fc14
> warning: package php = 5.3.3-1.fc14 was already added, replacing with php > 5.
> 6-1.fc14
> warning: package php-cli = 5.3.3-1.fc14 was already added, replacing with php-
> i > 5.3.6-1.fc14
> warning: package php-common = 5.3.3-1.fc14 was already added, replacing with p
> -common > 5.3.6-1.fc14
> warning: package php-mysql = 5.3.3-1.fc14 was already added, replacing with ph
> mysql > 5.3.6-1.fc14
> warning: package php-pdo = 5.3.3-1.fc14 was already added, replacing with php-
> o > 5.3.6-1.fc14
> Preparing...### [100%]
>1:libgcc ### [  2%]
>2:glibc-common   ### [  3%]
>3:nss-softokn-freebl ### [  5%]
>4:glibc  ###

Re: rpm-5.3.11

[Jeff Johnson]

On Aug 14, 2011, at 1:15 PM, Belal Salem wrote:

> Thank for the perfect details!
> since I'm running RHEL-5 (as my system base and based on rpm-4.4) but updated 
> from RHEL-6 (glibc and its deps and python),
> and according to your explanation, I guess I would better recompile rpm with 
> the --disable-dirname-and-symlink-deps
> specially that adding the folders to _SOME_ will not be suitable for my case 
> since I have requests for different folder whenever I'm trying to install 
> packages from RHEL-5 or RHEL-6.
> 

Actually its likely easier to use a run-time rather than a compile-time
setting. The parentdir/linto dependencies are implicit
in the sense that there's no Requires: that is explicitly needed.
The dependencies are synthesized from pre-existing information in
a package.

To configure parentdir/linkto dependencies at run-time:

1) Make sure that RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS is enabled.
That is likely already the case if you haven't disabled. Check
config.h after running ./configure (and you can always edit config.h
after running configure if all else fails).

>> 
>> The code to Rip It Out! Have it your own way! is (from lib/depends.c):
>>  #if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
>> defined(RPM_VENDOR_OPENMAMBA) || 
>> defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
>> optional-dirname-and-symlink-deps */
>>  int dirname_deps;
>>  int symlink_deps;
>>  #endif
>> …
>>  #if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
>> defined(RPM_VENDOR_OPENMAMBA) || 
>> defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
>> optional-dirname-and-symlink-deps */
>>  dirname_deps = 
>> rpmExpandNumeric("%{?_check_dirname_deps}%{?!_check_dirname_deps:1}");
>>  if (dirname_deps) {
>>  #endif
>> …
>>  #if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
>> defined(RPM_VENDOR_OPENMAMBA) || 
>> defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
>> optional-dirname-and-symlink-deps */
>>  }
>> 
>>  symlink_deps = 
>> rpmExpandNumeric("%{?_check_symlink_deps}%{?!_check_symlink_deps:1}");
>>  if (symlink_deps) {
>>  #endif
>> …
>>  }
>> 

2) Edit /etc/rpm/macros (create if not existent) to add
%_check_dirname_deps0
%_check_symlink_deps0
to disable.

3) Turn on symlink dependencies: its less annoying than parentdir deps and
is almost always a package failure when encountered (aka a dangling symlink
that points to a non-existent end-point).

4) When you have a mostly stable/unchanging set of packages installed, then
turn on parentdirectiory dependencies. Then do
mkdir -p /etc/rpm/sysinfo
rpm -Va --nofiles | grep '^/' | sort -u > /etc/rpm/sysinfo/Dirnames
(there is a --orphandirs option to rpm -Va that does the above, I'm just too 
lazy to check
how to use it: its a popt alias that does essentially the above).

5) Run
rpm -Va --nofiles
periodically, and when you see a "orphan directory", i.e. a directory that
isn't explicitly in any package file manifest, then just add that directory
to /etc/rpm/sysinfo/Dirnames.

The annoyance is mostly that RHEL5/RHEL6 never bothered to do the QA to
ensure that rpm --erase would also remove created directories.

BTW, if you are going to use rpm-5.3.11` on RHEL5/RHEL6, then you need
to convert your rpmdb so that header instance numbers are always in
network order (needed for optimal btree access).

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm-5.3.11

[Jeff Johnson]

On May 13, 2010, at 5:46 PM, Belal Salem wrote:

> So! Are there another solution by recompiling 'rpm' source code to just work 
> as we used to!
> means, are there any configure script options that I can use to override 
> that?!
> 

Sure you can recompile to change two control bits and their
default setting compiled into RPM to "work as it used to" (this change
was made 7+ years ago in like rpm-4.4.6 or something).

Here are the bits (from lib/rpmts.h):
RPMDEPS_FLAG_NOPARENTDIRS = (1 << 4), /*!< from --noparentdirs */
RPMDEPS_FLAG_NOLINKTOS = (1 << 5),/*!< from --nolinktos */ 
Here is where the "default" values for dependency flags are set (lib/poptALL.c):
/*@unchecked@*/ int global_depFlags = RPMDEPS_FLAG_ADDINDEPS; 
Add those two disabler bits as you please.

There are macros that can be set to change the default
values of those two bits so that you -- like every other vendor --
can revert to "traditional" rpm behavior from 7+ years ago without
recompiling.

The code to Rip It Out! Have it your own way! is (from lib/depends.c):
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
defined(RPM_VENDOR_OPENMAMBA) || 
defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
optional-dirname-and-symlink-deps */
int dirname_deps;
int symlink_deps;
#endif
…
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
defined(RPM_VENDOR_OPENMAMBA) || defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) 
/* optional-dirname-and-symlink-deps */
dirname_deps = 
rpmExpandNumeric("%{?_check_dirname_deps}%{?!_check_dirname_deps:1}");
if (dirname_deps) {
#endif
…
#if defined(RPM_VENDOR_MANDRIVA) || defined(RPM_VENDOR_ARK) || 
defined(RPM_VENDOR_OPENMAMBA) || 
defined(RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS) /* 
optional-dirname-and-symlink-deps */
}

symlink_deps = 
rpmExpandNumeric("%{?_check_symlink_deps}%{?!_check_symlink_deps:1}");
if (symlink_deps) {
#endif
…
}

And there is AutoFu to set the "default" value in "Have it your own way!" 
fashion
in several ways. The RPM_OPTIONAL_DIRNAME_AND_SYMLINK_DEPS functionality is 
tied to
./configure --help
…
  --disable-dirname-and-symlink-deps
  permit to disable via macro the automatic dirname
  and symlink dependecy

The other way to disable -- because vendors MUST see their "branding" 
everywhere and
can't really figure out how to build RPM unless they can see/find their name
in the code -- is tied to

  --with-vendor=VENDORbuild with a supported vendor's specific set 
of
  changes: mandriva, ark, fedora, openmamba

Note that per-vendor changes are _NOT_ supported through @rpm5.org. I can't
support code that is never enabled in my development versions (and that is
controversial for some reason, in this case because the issue was discussed
and decided 7+ years ago and no one has bothered to reopen discussions about
what SHOULD be done since).

But the better way to "fix" instead of recompiling RPM is to add
%files
…
%dir /etc/fonts
%dir /etc/fonts/conf.d

to _SOME_ package so that the driectores are actually present in _SOME_ package.
(the directories can be in multiple packages too).

Adding the directories to _SOME_ package ensures that the directories will
be removed by --erase, which is/was the reason for adding the "annoying"
parent directory dependency in the first place.

hth

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm-5.3.11

[Jeff Johnson]

On May 13, 2010, at 5:24 PM, Belal Salem wrote:

> # rpm -Uvh fontconfig-2.8.0-3.el6.i686.rpm
> 
> error: Failed dependencies:
>/etc/fonts is needed by fontconfig-2.8.0-3.el6.i686
>/etc/fonts/conf.d is needed by fontconfig-2.8.0-3.el6.i686
> 
> the folders '/etc/fonts' and '/etc/fonts/conf.d' are already installed by 
> this package!!

Yes … but …

Those directories are not explicitly in the package, but rather
implicitly created in order to install a file on a path.

That's the reason. There are two rules:
1) Every file Requires: its parent directory.
2) Every symlink requires its end-point.

You can add --noparentdirs (or --nolinktos) to disable the functionality.

Or you can do
mkdir -p /etc/rpm/sysinfo/Dirnames
echo "/etc/fonts">> /etc/rpm/sysinfo/Dirnames
echo "/etc/fonts/conf.d" >> /etc/rpm/sysinfo/Dirnames

There's also some macro (I never use it, so I've forgotten) to add 
--noparentidrs persistently.

Or you can compile rpm without the functionality.

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm-5.3.11

[Jeff Johnson]

On May 13, 2010, at 4:59 PM, Belal Salem wrote:

> I compiled and upgraded to rpm-5.3.11, and noticed that installing packages 
> are most of the times requesting dependencies that are already in the package 
> being installing, using the --nodeps flag bypass this, but this is annoying 
> and occurs with almost all packages you want to install..
> Does anybody noticed that?! or I have something wrong with my installation.

Dunno … but if you send me a copy of -Uvv output that includes
whatever issue you find annoying, I'll try to bigger
what is going on.

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 10, 2011, at 6:56 AM, Anders F Björklund wrote:

> Jeff Johnson:
> 
>>> 6. Configure RPM:
>>> -
>>> # ./configure --prefix=/usr --libdir=/usr/lib64 --with-uuid=/usr/lib64 
>>> --with-xar=/usr/lib64 \
>>>   --enable-rpm-lua-extensions-based-on-rpmlib 
>>> --enable-fast-install --enable-shared \
>>>  --enable-rpmvercmp-digits-beat-alpha --with-expat=/usr/lib64 
>>> --with-neon=/usr/lib64 \
>> 
>> Hmmm … I did not know there is a --enable-rpmvercmp-digits-beat-alpha
>> or a --enable-rpm-lua-extensions-based-on-rpmlib option.
>> 
>> I'm not at all sure what they are supposed to do or who added (its likely 
>> from RSE and OpenPKG several
>> years back). It's sure to be  some pretty arcane functionality ...
> 
> Actually it's quite new...
> http://rpm5.org/cvs/chngview?cn=15680  optional-dirname-and-symlink-deps
> http://rpm5.org/cvs/chngview?cn=15735  
> rpm-lua-extensions-based-on-rpm-lib-functionality
> http://rpm5.org/cvs/chngview?cn=16017  old-comparision-behaviour
> 
> It came from splitting vendor configuration into autotools configuration:
> https://blueprints.launchpad.net/rpm/+spec/rpm-split-vendor-config-in-autofu
> 
> Actual config is the same.
> 
> Not sure it's a good idea.
> 

Could you be a bit more specific about what isn't a good idea?

Per-vendor configuration: an excuse to not work through
the best solution. Its always based on a "Not time, we know what we are doing …"
mind-set. I don't mind the divergence if "we" did not include "me" doing
support and development.

AutoFu: I've always believed that compiled in options need to become run-time
options. The problem with compiled in options is the assumption that you
can re-compile. That is often not the case any more. And its not as simple
as adding a macro somewhere when there are deep semantic changes like version
comparison, or removing "colors" and not attaching dependencies to files etc:
configuration should be about file paths, or verbosity levels, and other UI
issues, not fundamental changes to RPM.

But somehow per-vendor configuration needs to be merged/dropped imho: blaming
RPM (and me) for bugs and lack of support on code that isn't well used/tested,
and where "vendor support" isn't an actuality, is, well, not such a good idea.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 9, 2011, at 12:26 AM, Belal Salem wrote:

> DONE IT !!! [SOLVED]
> 

Nice ;-)

> I passed out all the errors the last one was resolved by using db-5.1.19 
> instead of db-5.2.x
> 

Note that there is a data item removed from the middle
of an internal structure between 5.1.19 and 5.1.25.

So the transition 5.1.19 -> 5.1.25 forces a recompile. I.e. don't
expect db-5.1.25 to be compatible.

But either of db-5.1.19 and/or db-5.1.25 work fine, its only the transition
that will lead to surprises.

> Here is a quick reference on how I did it, may be some one stuck into it like 
> me:
> 
> 
> 
> Compiling and Installation:
> ___
> ===
> 
> 1. Configure TCL:
> -
>   # ./configure --prefix=/usr --libdir=/usr/lib64 --enable-shared 
> --enable-threads --enable-64bit \
> --enable-static
> 
> 2. Configure the db-5.1.19:
> ---
>   # cd build_unix
>   #  ../dist/configure --prefix=/usr --libdir=/usr/lib64 --enable-tcl 
> --enable-shared --enable-sql \
>  --enable-dbm --with-tcl=/usr/lib64
>   # make
>   # make install
>   # cp db.h dbsql.h /usr/include/db51/
> 
> 3. Configure ncurses:
> -
>   # ./configure --prefix=/usr --libdir=/usr/lib64 --with-shared --with-static 
> --with-libtool
> 

Hmmm … not sure there's anything in rpm that MUST have ncurses (but
it may be a secondary dependency). Certainly won't hurt anything.

> 4. Configure expat:
> ---
>   # ./configure --prefix=/usr --libdir=/usr/lib64 --with-shared --with-static
> 
> 5. Configure the rest of the 3d-parties as follows:
> ---
>   * Configure all of them as follows:
> #./configure --prefix=/usr --libdir=/usr/lib64
> # make
> # make install
> 
> Note: Some of them doesn't use a 'configure' script, so they don't have 
> an option line to install
>   libs to /usr/lib64, so,
>  after installing them, do a symlink from them to /usr/lib64
> 
> 6. Configure RPM:
> -
>   # ./configure --prefix=/usr --libdir=/usr/lib64 --with-uuid=/usr/lib64 
> --with-xar=/usr/lib64 \
> --enable-rpm-lua-extensions-based-on-rpmlib 
> --enable-fast-install --enable-shared \
>--enable-rpmvercmp-digits-beat-alpha --with-expat=/usr/lib64 
> --with-neon=/usr/lib64 \

Hmmm … I did not know there is a --enable-rpmvercmp-digits-beat-alpha
or a --enable-rpm-lua-extensions-based-on-rpmlib option.

I'm not at all sure what they are supposed to do or who added (its likely from 
RSE and OpenPKG several
years back). It's sure to be  some pretty arcane functionality ...

Congrats!

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 7:03 PM, Jeff Johnson wrote:

> 
> On Aug 9, 2011, at 12:54 AM, Belal Salem wrote:
> 
>> Thanks for the detailed explanation.
>> So, here after I satisfied all the prerequisites, when compiling the RPM 
>> source, I encounter the following compilation errors:
>> 
>> In file included from dbconfig.c:14:
>> ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'
>> ./rpmdb.h:490: error: expected specifier-qualifier-list before 'DB_LOGC'
>> ./rpmdb.h: In function 'dbiCopen':
>> ./rpmdb.h:589: error: 'struct _dbiIndex' has no member named 'dbi_vec'
>> ./rpmdb.h: In function 'dbiCclose':
>> .
>> .
>> 
>> what do you think that is?!
>> 
> 
> This is the same problem that is being reported multiple times.
> 
> In order to do a release -- using --with-db=external as everyone wishes --
> rpm has
>   #include 
> hardwired in the code.
> 
> What everyone wishes is that rpm Just Works with whatever Berkeley DB
> happens to be installed *for all possible distros and versions*.
> 
> The simplest fix is to build/install db-5.1.25 externally. The "portable"
> (in the sense you will have to edit to taste, the basic build is fine)
> that I used is available here
> 
>   cvs -d :pserver:anonym...@rpm5.org:/v/rpm/cvs get distro/db51
> 
> Add the "official" release of db-5.1.25 from Oracle, and the ancient db.1.85 
> tar ball,
> and build/install the db51 packages.
> 

Before we get too much further:

The RPM version you are building *requires* a change in the /var/lib/rpm
databsed to add transactional logs and ACID behavior for increased reliability
(and performance).

If your goal is to stick with a RHEL5 based system, building/installing
rpm-5.3.x is a one-way trip:
No compatibility with other versions of rpm is promised in rpm-5.3.x 
whatsoever.
Instead, there's a conversion, and tools to help with the conversion,
to use transactions and ACID properties in Berkeley DB.

The conversion isn't hard (32bit unsigned integers are being swapped
so that rpmdb's are *always* in network order for "portability" and
so that B-tree access is optimal on all platforms) and is invertible
by running a 5-7 step process.

But you might want to think carefully about what your goals are.

In addition to having fewer side-effects from interrupted behavior like kill -9,
the other major benefit in rpm-5.3.x is increased performance in many accesses,
in spite of that fact that transactional logging forces all I/O to be done
twice.

You may want to look at rpm-5.2 instead: that is the last release that
is "compatible" as in doing same old, same old.

Note that I've measured 80x faster installation with rpm-5.3.x compared
to rpm-4.6.1, installing ~1000 identical packages using 5.3.x and 4.6.1.

While -- like all benchmarks -- that 80x faster number is illusory when one 
digs into
the techniques used to achieve the "win", it is indeed a significant performance
improvement, and rpm-5.3.x is bot faster and more reliable because of 
transactions
and ACID behavior and other improvements too numerous to mention.

Again:
No compatibility with other versions of rpm is promised in rpm-5.3.x 
whatsoever.

hth

73 de Jeff



__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 9, 2011, at 12:54 AM, Belal Salem wrote:

> Thanks for the detailed explanation.
> So, here after I satisfied all the prerequisites, when compiling the RPM 
> source, I encounter the following compilation errors:
> 
> In file included from dbconfig.c:14:
> ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'
> ./rpmdb.h:490: error: expected specifier-qualifier-list before 'DB_LOGC'
> ./rpmdb.h: In function 'dbiCopen':
> ./rpmdb.h:589: error: 'struct _dbiIndex' has no member named 'dbi_vec'
> ./rpmdb.h: In function 'dbiCclose':
> .
> .
> 
> what do you think that is?!
> 

This is the same problem that is being reported multiple times.

In order to do a release -- using --with-db=external as everyone wishes --
rpm has
#include 
hardwired in the code.

What everyone wishes is that rpm Just Works with whatever Berkeley DB
happens to be installed *for all possible distros and versions*.

The simplest fix is to build/install db-5.1.25 externally. The "portable"
(in the sense you will have to edit to taste, the basic build is fine)
that I used is available here

cvs -d :pserver:anonym...@rpm5.org:/v/rpm/cvs get distro/db51

Add the "official" release of db-5.1.25 from Oracle, and the ancient db.1.85 
tar ball,
and build/install the db51 packages.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 1:19 PM, Belal Salem wrote:

> Actually, what I'm trying to do is the following, may be you have an advice 
> for me:
> I'm running RHEL-5, I found it the most stable and secure Distro that runs 
> smoothly almost all the high-end simulation and 3D applications that I'm 
> using intensively, the problem is that there are dozens of simple but 
> important applications/utilities that requires higher versions of glibc and 
> python, those that where supplied as rpm packages that requires newer 
> versions of the RPM to be installed!

Yes, you aren't alone here either: most "enterprise" distros have such a ;one
"supported" life that they become irrelevant.

I'm not sure that ether's any "enterprise" distro with a solid upgrade path.
Dag's repository is likeliest best both reputation ally and in in "reality".
Check there first imho.

> So, I decided to build a custom system installation that satisfies all my 
> needs.
> I manually forced upgrade glibc with a version that gives 2.6 and 2.7 without 
> breaking the system dependencies, and I compiled python-2.7.2, and now my 
> system runs really great! and I can run those new applications smoothly.
> 

Upgrading REHL5 to python-2.7.2 is non-trivial: congratulations on success.

> The only one thing that I'm suffering from is that I have to manually extract 
> and install them since they require higher signature of fileDegist, payload 
> and rpm compressing... for that I decided to upgrade to RPM-5 to be able to 
> install them regularly using the RPM Packager.
> 

Yes: distros have chosen to use SHA-256 file digests without bothering
to release a compatible version of rpm.

There is a way to disable file digests using --nofdigests (I've forgotten the
older option, it has been 6 years or so: --nomd5sums? something like that).

Sadly that *still* isn't enough because of the XZ! XZ! XZ! craziness. So there's
often nothing one can do except extract the payload
rpm2cpio.sh foo*.rpm | cpio -dim
(there's a shell script that permits inserting a unxz where needed).

There are back ports of XZ and SHA256 file digests to whatever RPM is
in RHEL5 if needed: I can dig out the necessary patches if GOOG
cannot find something that Just Works.

> Any advices about that! are there an easy way to let the RPM source compile 
> both of the i686 and x86_64 bit.
> 

All depends on what is considered "easy".

FYI: rpm-4.4.7 (on which @rpm5.org is based) treats arch simply as a string. SO
it's entirely feasible to do
rpmbuild -ba --target=hotmommacpu foo.spec
and end up producing binary
foo*.hotmammacpu.rpm
Whether that is useful or not is a far more complex answer: but RPM won't stop 
you
from attempting.

For RHEL5, its likely easiest to add (and change)
/etc/rpm/platform
which contains in 1-line the usual config.guess string like
x86_64-unknown-linux

There are ways to invoke with a setarch(1) prefix like
setarch x86_64 rpmbuild -ba yadda.spec
but I find that rather more annoying than easy: usually I'm building
a set of packages, and editing /etc/rpm/platform is persistent (and easy)
even if I usually forget to check the setting when building.

> What I'm confused about, is that the RPM has many dependencies, and I don't 
> know which one of them should be compiled for both i686 and x86_64 and which 
> needs to only compile for x86_64.
> 

All ELF dependencies have a "color" and are attached to file(s) (if the 
functionality is not disabled).
So color is 0 == "white", 1 == elf32, and 2 == elf64 and you can see the 
dependencies by doing
rpm -qp --filerequire *.rpm
to see which dependencies are associated with each file.

There is a --fileprovide as well, and you can also see what rpm is going'
to do by typing, say,
find /bin | /usr/lib/rpm/rpmdeps --requires
or even
echo "/bin/sh" | …
(there's a --provides analogue to --requires too).

Other arch-specific dependencies are usually dictated by Packaging Policy 
Police somewhere.

Note that RHEL5 is particularly strangely painful in the way it handles ix86 
<-> x86_64 "multilib".

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 12:39 PM, Belal Salem wrote:

> Yes!
> The repository you give to me is a good place to compare the versions I have 
> and to upgrade what needs upgrading.
> 
> Thanks for your time.
> 

NP. If you catch some annoyance, its not personal:

I maintain both RPM5 and POPT and yours isn't the first
report of build failures.

The problem isn't simple: RPM uses POPT in easy that no other program does.

E.g.POPT_ARGFLAG_TOGGLE adds the ability to set/clear a set of flags from
CLI options with both
--thisoption
--nothisoption
Since RPM is chock full of disablers (like --nodeps etc) which are attempting
to use POPT_ARGFLAG_TOGGLE in reasonable ways, well, the inability to upgrade
POPT by distros in 3-4 years is, well, a tad bit disturbing.

POPT is *NOT* a hard upgrade, nowhere near as complicated as upgrading RPM has 
become.

Anyways holler at me (and ignore my mutterings) if you need help. I'll try
to get you fixed up no matter what …

(aside to Robert Xu):
Getting there just buried …

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 11:37 AM, Belal Salem wrote:

> Great!
> Updating to popt-1.14 works.
> 
> Now,rpmaug.c with POPT_ARG_MAINCALL has passed correctly, but I encounter the 
> same error type different place:
> rpmzq.c:179: error: 'POPT_ARGFLAG_TOGGLE' undeclared here (not in a function)
> 
> Are there any other packages that needs updating in addition to popt?
> 

All depends on how what build options you choose for rpm: RPM is *NOT*
a build that you can just do
./configure --prefix=/usr
and hope for the best. There's simply too many build options to even
attempt the needed QA coverage explicitly.

The prerequisites for RPM are in INSTALL where it says:

POPTmandatory 1.151.16http://rpm5.org/files/popt/

INSTALL is maintained accurately for the most common build options, certainly 
the MANDATORY ones.

There's also an internal popt distributed with rpm for most of the last year.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmio not compiling

[Jeff Johnson]

On Aug 8, 2011, at 10:43 AM, Belal Salem wrote:

> Hi all,
> compiling the source for any rpm-5.x version doesn't success, I encounter the 
> following error:
> 
> rpmaug.c:650: error: 'POPT_ARG_MAINCALL' undeclared here (not in a function)

POPT_ARG_MAINCALL is in popt-1.14 and later. Upgrade your version of popt.

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Detection of "bad" packages

[Jeff Johnson]

On Aug 1, 2011, at 7:31 AM, Jeff Johnson wrote:

> 
> On Aug 1, 2011, at 4:42 AM, Michael Baudisch wrote:
> 
>> Hello,
>> i do some tests with RPM and somehow "bad" packages. When I pass a "bad" 
>> test package to RPM I observered that there's an error like "error: open of 
>> blablabla failed".  The package is a plain text file  with .rpm extension 
>> containing only "blablabla".
>> But when I put this "blablabla" in squared brackets ("[Blablabla]") there's 
>> no error.
>> So, what is the magic that does RPM not detect/report a corrupt package? Are 
>> there other circumstances under which RPM does not detect/report a "bad" 
>> package? Thanks for help in advance.
>> 
> 
> What rpm version?
> 
> All input arguments that are not *.rpm packages
> are treated as a manifest of *.rpm packages.
> 

What may not be clear here is this:
Technically there are no "bad packages": everything that isn't a package
is treated as a manifest. Manifests are checked for non-printable 
characters.

> The manifest is split on white space and each item is added
> at that position in the argument list.
> 

The items can actually be glob patterns: so adding square brackets
turns a string into a character range.

> Not reporting is not the same as not detecting.
> Did rpm actually attempt to install anything?
> 

Arguments that don't exist are silently skipped.

Consider this
touch empty
rpm -Uvh empty

This is semantically equivalent to an invocation with no arguments: the file is 
empty.

But there are differences in behavior because there is an augment: the 
mani9fest,
but the manifest is empty, and so one sees different behavior.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Detection of "bad" packages

[Jeff Johnson]

On Aug 1, 2011, at 4:42 AM, Michael Baudisch wrote:

> Hello,
> i do some tests with RPM and somehow "bad" packages. When I pass a "bad" test 
> package to RPM I observered that there's an error like "error: open of 
> blablabla failed".  The package is a plain text file  with .rpm extension 
> containing only "blablabla".
> But when I put this "blablabla" in squared brackets ("[Blablabla]") there's 
> no error.
> So, what is the magic that does RPM not detect/report a corrupt package? Are 
> there other circumstances under which RPM does not detect/report a "bad" 
> package? Thanks for help in advance.
> 

What rpm version?

All input arguments that are not *.rpm packages
are treated as a manifest of *.rpm packages.

The manifest is split on white space and each item is added
at that position in the argument list.

Not reporting is not the same as not detecting.
Did rpm actually attempt to install anything?

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: libzypp and rpm-5.3.10

[Jeff Johnson]

On Jul 30, 2011, at 1:47 PM, Eric MSP Veith wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hello,
> 
> did somebody have any luck with building libzypp against rpm-5.3.10? I've 
> weeded out most of the simple compile errors, but I'm stuck here:
> 

Poky-Yacto has succeeded using zipper -> rpm-5.4.0 (API is nearly the same,
rpm-5.4.0 was just to get out of Mandriva's way).

The patches were pretty minor iirc.

> cd /usr/src/gnyubuild/build/libzypp-9.8.9/build/zypp && /usr/bin/c++   -
> Dzypp_EXPORTS -DHAVE_PIPE2 -D_FILE_OFFSET_BITS=64 -DVERSION=\"9.9.0\" -
> D_RPM_5 -DHAVE_UDEV -DLOCALEDIR=\"/usr/share/locale\" -DTEXTDOMAIN=\"zypp\" 
> - -O2 -g -m32 -mtune=generic -march=i686 -fasynchronous-unwind-tables -
> D_RPMDB_INTERNAL -D_RPMDB_INTERNAL  -fno-strict-aliasing -fPIC -g -Wall -
> Woverloaded-virtual -Wnon-virtual-dtor -Wl,-as-needed -fPIC -
> I/usr/src/gnyubuild/build/libzypp-9.8.9 -
> I/usr/src/gnyubuild/build/libzypp-9.8.9/build -I/usr/include/rpm -
> I/usr/include/libxml2 -I/usr/src/gnyubuild/build/libzypp-9.8.9/build/zypp
> - -DZYPP_BASE_LOGGER_LOGGROUP=\"zypp\" -o 
> CMakeFiles/zypp.dir/target/rpm/BinHeader.cc.o -c 
> /usr/src/gnyubuild/build/libzypp-9.8.9/zypp/target/rpm/BinHeader.cc
> In file included from /usr/include/rpm/rpm4compat.h:60:0,
> from 

Avoid rpm*compat.h and go native rpm-5.3.x is what I would suggest (what 
Poky-Yacto did).

There is no desire to maintain a "compatible" API though what's
there mostly works. But you likely need to pick-n-choose because
there's no such thing as a rp[m4 API anymore than there is an rpm5 API.

Things change when necessary: and having an API isn't that high a priority
because most interest is in bindings, not in the API.

> /usr/src/gnyubuild/build/libzypp-9.8.9/zypp/target/rpm/librpm.h:19,
> from 
> /usr/src/gnyubuild/build/libzypp-9.8.9/zypp/target/rpm/BinHeader.cc:12:
> /usr/include/rpm/rpmdb.h: In function 'DB_TXN* dbiTxnid(_dbiIndex*)':
> /usr/include/rpm/rpmdb.h:863:50: error: invalid conversion from 'void*' to 
> 'DB_TXN*'
> /usr/include/rpm/rpmdb.h: In function 'int rpmlkId(rpmdb_s*, uint32_t*)':
> /usr/include/rpm/rpmdb.h:873:29: error: invalid conversion from 'void*' to 
> 'DB_ENV*'
> /usr/include/rpm/rpmdb.h: In function 'int rpmlkIdFree(rpmdb_s*, uint32_t)':
> /usr/include/rpm/rpmdb.h:884:29: error: invalid conversion from 'void*' to 
> 'DB_ENV*'
> /usr/include/rpm/rpmdb.h: In function 'int rpmlkGet(rpmdb_s*, DBT*, 
> uint32_t, void*)':
> /usr/include/rpm/rpmdb.h:895:29: error: invalid conversion from 'void*' to 
> 'DB_ENV*'
> /usr/include/rpm/rpmdb.h:899:69: error: invalid conversion from 'uint32_t' 
> to 'db_lockmode_t'
> /usr/include/rpm/rpmdb.h:899:69: error: invalid conversion from 'void*' to 
> 'DB_LOCK*'
> 

Theses aren't difficult issues to fix: they are all Berkeley DB typedef's 
missing,
grab whatever you need from whatever db.h is in use and bigger out how to
include when compiling.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpmbuild and automatic dependencies (symlinks)

[Jeff Johnson]

On Jul 26, 2011, at 12:15 PM, YuGiOhJCJ Mailing-List wrote:

> Hello,
> 
> I create a RPM package containing this file :
> /etc/mtab -> /proc/mounts
> 
> In fact this is a symlink to '/proc/mounts'.
> 
> So when I want to install this RPM package, there is a dependency :
> # rpm -i fhstree-20090529-1.i686.rpm 
> rpmdb: Build signature doesn't match environment
> error: Failed dependencies:
>   /proc/mounts is needed by fhstree-20090529-1.i686
> 

There are two significant differences using @rpm5.org code:

1) Every file depends on its parent directory.
2) Every symlink depends on its end-point.

The rationale for adding these "synthesized" dependencies (i.e.
you will not see them in Requires: tags) is to assist in
identifying packaging flaws that are otherwise rather difficult
to see.

E.g. "orphan" directories are directories in some path that are not
packaged. All directories in every file path are created in order to
install file content. The reverse operation while erasing isn't symmetric:

Orphan directories remain after a package is removed.

The goal for the symlink dependency (as you are seeing) is to help
detect dangling symlinks. Your usage case is a bit different because
you are using /proc/mounts which isn't typically packaged.

The recommended fix is this:
mkdir -p /etc/rpm/sysinfo
echo "/proc/mounts" > /etc/rpm/sysinfo/Providename
(I believe you can also use Filelinktos instead of Providename if
you wish to separate the types of configurable Provides that you
have, but I haven't checked. Providename will "work", the naming
convention used by RPM is pretty simple to understand:
1) Remove RPMTAG_ prefix.
2) Convert to lowercase
3) Capitalize the first letter to keep the Germans happy ;-)
and the list of available tags (not all of which are pused in 
/etc/rpm/sysinfo/*)
is available using
rpm -v --querytags
(-v must precede --querytags) which will also display the data type information.

> A working solution, for the end-user, is adding the '--nodeps' option :
> # rpm -i --nodeps fhstree-20090529-1.i686.rpm
> error: failed to open /etc/mtab: No such file or directory
> 

There are 2 more narrowlytargeted options
--noparentdirs
--nolinktos
that can be used as disablers. There is some sort of macro framework
to do this persistently (but these are per-vendor modifications, I see
no reason to add configurable ways to turn off implementations 7+ years
later, everyone has had more than enough time to either change, or argue
for feature removal, the features were discussed and the consensus was "Do it." 
years ago)

> I can't create an other package providing '/proc/mounts' file because this 
> file exists only when mounting the '/proc' special partition.
> 

There are ways to package a /proc/mounts path (but /proc has very odd behavior
with stat(2) that isn't typical for what uglix calls "files"), and so
I suggest /etc/rpm/sysinfo/Providename instead.

> So, I would like to create my RPM package containing this symlink but remove 
> the dependencies concerning symlinks.
> 
> I found this for my spec file :
> AutoReqProv: no
> 

There are far more precise solutions (as above) then just removing all 
dependencies
from a package. AutoReqProv: is as crude as a flint axe, try moving up to the
Bronze Age at least ;-)

> But it doesn't solve my problem (I think 'AutoReqProv: no' just remove the 
> automatic dependencies watching for executable scripts '#!' and binary files 
> dynamic dependencies).
> 

Yes, because the parentdir/linkto dependencies are synthesized from other tags,
not included explicitly as Requires: metadata.

> Is there a way to remove this dependency when building my RPM package?
> 

You can also build rpm without parentdir/linkto dependencies: UNSUPPORTED 
afaiac, but
you may do as you wish. This issue has been around for years and years ...

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM Stub Package

[Jeff Johnson]

On Jul 22, 2011, at 10:13 AM, Miller, Vincent (Rick) wrote:

> Hi Anders,
> 
> I installed sysutils/file and received an error message indicating that
> file 5.3 supports only version 7 magic files and that the one installed
> was version 8.  Perhaps this is the reason that RPM 5.2.1 did not install
> sysutils/file?
> 

If there is a version involved, then see if there isn't a way to rebuild the
*.mgc files for the distributed rules.

Checking … yes here's the file(1) option:

-C, --compile Write a magic.mgc output file that contains a pre-parsed 
version of the magic file or directory.

Unless something is seriously different in modern file(1), one can simply
remove/rename the *.mgc compiled version and the plain text rules file
will be used.

Again, if you have a version mismatch in *.mgc, then there are two
versions of file(1) floating about, one that has zigged, the other zagged,
and that is a distro problem and needs a bug report as Anders suggested.

73 de Jeff

> [root@bsdsandbox2 /usr/ports/sysutils/file]# find /lib |
> /usr/local/lib/rpm/bin/rpmdeps --provides
> error: magic_load(ms, /usr/local/share/file/magic) failed: File 5.3
> supports only version 7 magic files. `/usr/local/share/file/magic.mgc' is
> version 8
> Assertion failed: (mg != NULL), function rpmfcClassify, file rpmfc.c, line
> 1132.
> Abort trap: 6 (core dumped)
> 
> ==
> 
> Vincent (Rick) Miller
> Systems Engineer
> vmil...@verisign.com
> 
> t: 703-948-4395
> 21345 Ridgetop Cir Dulles, VA 20166
> 
> VerisignInc.com
> 
> 
> 
> 
> 
> On 7/22/11 7:47 AM, "Anders F Björklund"  wrote:
> 
>> Miller, Vincent (Rick) wrote:
>> 
>>> I get an error when I attempt to run rpmdeps.  It complains of a
>>> missing magic file (see error below).  I can't say this surprises me,
>>> but I am hoping there is a simple fix, instead of recompiling/installing
>>> from ports with different optionsŠ
>>> 
>>> [root@bsdsandbox2 /home/vmiller]# find /lib |
>>> /usr/local/lib/rpm/bin/rpmdeps --provides
>>> error: magic_load(ms, /usr/local/share/file/magic) failed: could not
>>> find any magic files!
>>> Assertion failed: (mg != NULL), function rpmfcClassify, file rpmfc.c,
>>> line 1132.
>>> Abort trap: 6 (core dumped)
>>> 
>>> I think the magic file it needs is /usr/share/misc/magic
>>> 
>>> Thoughts?
>> 
>> The archivers/rpm5 port was intended to link against sysutils/file,
>> rather than the system version of libmagic that you ended up with...
>> 
>> $ /usr/bin/file --version
>> file-5.03
>> magic file from /usr/share/misc/magic
>> $ /usr/local/bin/file --version
>> file-5.05
>> magic file from /usr/local/share/file/magic
>> 
>> You should file a Problem Report (PR) against the port, or perhaps
>> just install sysutils/file and rebuild. It should match the library:
>> 
>> LIB_DEPENDS=
>>  magic:${PORTSDIR}/sysutils/file \
>> CONFIGURE_ARGS+=
>>  --with-path-magic=${LOCALBASE}/share/file/magic
>> 
>> --anders
>> 
>> 
>> __
>> RPM Package Managerhttp://rpm5.org
>> User Communication List rpm-users@rpm5.org
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM Stub Package

[Jeff Johnson]

On Jul 21, 2011, at 9:25 AM, Miller, Vincent (Rick) wrote:

> Thanks again, Jeff.
> 
> I get an error when I attempt to run rpmdeps.  It complains of a missing 
> magic file (see error below).  I can't say this surprises me, but I am hoping 
> there is a simple fix, instead of recompiling/installing from ports with 
> different options…
> 

Yes: every distro in their insistence of changing /etc/magic to Something Else 
Instead
has done something different and so RPM breaks repeatedly and is forced to
some painfully complex QA on myriad platforms for no real purpose.

(aside)
RPM used to include file(1) and -lmagic and /usr/lib/rpm/magic* to avoid exactly
this failure mode. But these days "Have it your own way!" and complaints
of Bloat! Bloat! Bloat! have forced file(1) and -lmagic out of rpm. I change
rpm to attempt please the maximum no. of people even when the results
are quite predictable.

Anyways there's a macro to set the path to file magic:

$ rpm --showrc | grep magic
-14: _rpmfc_magic_path  /usr/share/misc/magic.mgc

I forget whether rpmdeps reads all of rpm's macros (it certainly
needs very little of the 500+ macrtos that vendors have chosen
to force rpm to carry around). If setting the macro doesn't "fix"
the path to magic, you will need to recompile and specify

$ ./configure --help | grep magic
  --with-file=ARG build with File (magic) library
  --with-path-magic=ARG   build with RPM magic file path (or macro)
so that the default (when the macro isn't specified) can find the needed
file.

hth

73 de Jeff

Re: RPM Stub Package

[Jeff Johnson]

On Jul 21, 2011, at 6:59 AM, Miller, Vincent (Rick) wrote:

> Hi,
> 
> I've got RPM installed via FreeBSD ports.  The next step is to create the 
> stub package which contains all the files that are already installed.  The 
> vpkg-provides.sh script does not specifically support FreeBSD.  I've 
> considered modifying it to support it.
> 

Good.

You won't need "… all the files .." generally (but you can do that if you wish).

RPM uses a database to find Provides: that match Requires:.

The stub package is a means to do a database import of certain items.

You will almost certainly need
Provides: /bin/sh
Most other dependencies that are needed are library so names.

You can see what will be needed using /usr/lib/rpm/bin/rpmdeps.

E.g. here are the provides from /lib/* libraries:

find /lib | /usr/lib/rpm/bin/rpmdeps --provides

Note that there is a --requires option, and rpmdeps can take files either
from stdin, or form the CLI argv list.

Building the stub package is little more than capturing output from rpmdeps
into a *.spec file used for markup
Provides: libc.so.6
which when installed will satisfy dependencies of other packages.

(aside)
The vpkg generated stub pkg will also generate md5 digests, and has
the ability to verify those digests using a %verifyscript, which isn't
strictly needed for stubbing out a dependencies, but may be useful.

There's other means to verify digests on a file using a probe dependency.
E.g.
Requires: digest(/path/to/file) = 123456789….
can be added to any @rpm5.org package, and will verify the content digest on
that file path.


> How much of the script will need to be modified to support FreeBSD?  
> Obviously, the case() setting OS defaults would be the first modification 
> necessary, but is there more beyond this that should be modified?
> 

Its hard to guess  before hand because the hard task isn't modifying the
script, but rather identifying all the dependencies will be needed.

Simpler than the vpkg script is to supply Provides: dependencies from a file
(1 per line please):

mkdir -p /etc/rpm/sysinfo
echo "/bin/sh" >> /etc/rpm/sysinfo/Providename

and edit in whatever dependencies you need as found.

Its more tedious than hard. But it might help to see what needs to be done
before attempting to automate with a script.

hth

73 de Jeff

> 
> ==
> Vincent (Rick) Miller
> Systems Engineer
> vmil...@verisign.com
> 
> t: 703-948-4395
> 21345 Ridgetop Cir Dulles, VA 20166
> 
> VerisignInc.com   

Re: Release in tar.gz format

[Jeff Johnson]

On Jul 14, 2011, at 1:20 PM, YuGiOhJCJ Mailing-List wrote:

> Hello,
> 
> When I go on the rpm5 website I see the release are now given in src.rpm 
> format.
> Indeed, the last rpm release using the .tar.gz format is rpm-5.3.5.

Yup.

> The tar.gz format is a common archive type easy to extract on many operating 
> systems.

Yes it is.

> Can you tell me why you chose to release into this format only?

Yes:
To illustrate the features in RPM *.src.rpm format as a distribution
format as well as de facto "dog food" testing of those same features.

I refer to something called "non-repudiable signatures" @rpm5.org in *.src.rpm.

Its basically the same as a self-signed host cert, except for a software 
distribution.

And the benefit (over detached *.asc) is that you can download _EVERYTHING_,
not just the tar-ball. In fact there's (at least) 2 tar balls necessary
to download, the 2nd tar ball has "last known good" per-platform macro 
configuration.

> Can you tell me how I can extract it?
> 

Sure. Grab scripts/rpm2cpio.sh from the tar ball you already have.
(I can post, in fact have p[osyed that script a zillion times for years)

Download the *.src.rpm and feed it to rpm2cpio.sh script like this:

mkdir /tmp/XXX
cd /tmp/XXX
./rpm2cpio.sh rpm*.src.rpm | cpio -dim

You will see the tar ball you quest magically appear in /tmp/XXX,
as well as the 2nd tar ball (that yopu might wish to examine) and
a rpm.spec that is more or less a reasonable starting point to
building rpm in *.rpm packaging (which is likely not you).

All from digitally signed container format in a single download called … drum 
roll please …

A SRPM!

Don't take my scarcasm personally … I quite well know what you and everyone
else expect: compressed tar balls. You should be able to perceive that there
are most definitely reasons and features for deliberately choosing *.src.rpm
here @rpm5.org as a distribution format.

This *IS* RPM, and its not like *.src.rpm haven't been around for more
than a decade, and SRPM's are quite commonly used instead of tar balls by
most rpm based distros.

SRPM's also avoid the need for multiply compressed (and multiply signed)
tarballs. Of course that's just smoke-and-mirrors because there is a
decompressor buried in rpm2cpio.sh and you WILL need to supply
a decompressor of the appropriate persuasion (and @rpm5.org will not have
to serve up multiple tar balls in every possible compression format
just to try to please every one).

What _IS_ Newer! Better! Bestest! @rpm5.org is the "non-repudiable signature",
based on section 13.8.2, p582 in the "Handbook of Applied Cryptography"
(and its only 2 pages, won't hurt you a bit to understand what is being
attempted because it *is* your security that is being protected even if you
wish a naked tar ball as a distribution format).

hth

73 de Jeff


> Thank you.
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: feedback : the -lexpat flag is missing in 4 Makefile

[Jeff Johnson]

On Jun 28, 2011, at 2:14 AM, YuGiOhJCJ Mailing-List wrote:

> 
> I am on Debian squeeze. I suggest you correct the 4 Makefile.am corresponding 
> to add this flag.
> 
> You can see below the 4 Makefile that I must edit :
> $ for each in `find rpm-5.3.5 -name "Makefile"`; do echo ${each} && diff 
> ${each} `echo ${each} | sed s/5.3.5/5.3.5-m/`; done
> rpm-5.3.5/syck/Makefile
> rpm-5.3.5/syck/ext/yamlbyte/Makefile
> rpm-5.3.5/syck/ext/lua/Makefile
> rpm-5.3.5/syck/tests/Makefile
> rpm-5.3.5/syck/lib/Makefile
> rpm-5.3.5/Makefile
> 250c250
> < LDFLAGS = 
> ---
>> LDFLAGS = -lexpat
> rpm-5.3.5/beecrypt/Makefile
> rpm-5.3.5/beecrypt/include/Makefile
> rpm-5.3.5/beecrypt/python/Makefile
> rpm-5.3.5/beecrypt/python/test/Makefile
> rpm-5.3.5/beecrypt/docs/Makefile
> rpm-5.3.5/beecrypt/gas/Makefile
> rpm-5.3.5/beecrypt/c++/lang/Makefile
> rpm-5.3.5/beecrypt/c++/Makefile
> rpm-5.3.5/beecrypt/c++/math/Makefile
> rpm-5.3.5/beecrypt/c++/security/Makefile
> rpm-5.3.5/beecrypt/c++/security/auth/Makefile
> rpm-5.3.5/beecrypt/c++/security/cert/Makefile
> rpm-5.3.5/beecrypt/c++/security/spec/Makefile
> rpm-5.3.5/beecrypt/c++/provider/Makefile
> rpm-5.3.5/beecrypt/c++/nio/Makefile
> rpm-5.3.5/beecrypt/c++/util/Makefile
> rpm-5.3.5/beecrypt/c++/util/concurrent/Makefile
> rpm-5.3.5/beecrypt/c++/util/concurrent/locks/Makefile
> rpm-5.3.5/beecrypt/c++/beeyond/Makefile
> rpm-5.3.5/beecrypt/c++/io/Makefile
> rpm-5.3.5/beecrypt/c++/crypto/Makefile
> rpm-5.3.5/beecrypt/c++/crypto/spec/Makefile
> rpm-5.3.5/beecrypt/java/Makefile
> rpm-5.3.5/beecrypt/masm/Makefile
> rpm-5.3.5/beecrypt/tests/Makefile
> rpm-5.3.5/rpmconstant/Makefile
> 166c166
> < LDFLAGS = 
> ---
>> LDFLAGS = -lexpat
> rpm-5.3.5/tools/Makefile
> 528c528
> < LDFLAGS = 
> ---
>> LDFLAGS = -lexpat
> rpm-5.3.5/build/Makefile
> rpm-5.3.5/build/auto/Makefile
> rpm-5.3.5/python/Makefile
> rpm-5.3.5/neon/Makefile
> rpm-5.3.5/neon/src/Makefile
> rpm-5.3.5/neon/test/Makefile
> rpm-5.3.5/rpmio/Makefile
> rpm-5.3.5/rpmio/auto/Makefile
> rpm-5.3.5/ruby/Makefile
> rpm-5.3.5/scripts/Makefile
> rpm-5.3.5/misc/Makefile
> rpm-5.3.5/po/Makefile
> rpm-5.3.5/js/Makefile
> rpm-5.3.5/rpmdb/Makefile
> rpm-5.3.5/rpmdb/auto/Makefile
> rpm-5.3.5/pcre/Makefile
> rpm-5.3.5/tests/Makefile
> rpm-5.3.5/tests/mongo/Makefile
> 173c173
> < LDFLAGS = 
> ---
>> LDFLAGS = -lexpat
> rpm-5.3.5/lib/Makefile
> rpm-5.3.5/lib/auto/Makefile
> rpm-5.3.5/doc/Makefile
> rpm-5.3.5/doc/ru/Makefile
> rpm-5.3.5/doc/ja/Makefile
> rpm-5.3.5/doc/pl/Makefile
> rpm-5.3.5/doc/manual/Makefile
> rpm-5.3.5/doc/fr/Makefile
> rpm-5.3.5/doc/ko/Makefile
> rpm-5.3.5/doc/sk/Makefile
> rpm-5.3.5/lua/Makefile
> 
> I hope this feedback can help rpm5 to be better :-)
> 

Before I forget (and your message disappears into my todo++ list) …

This is exactly what is needed to honor LDFLAGS being passed
everywhere. I'm not at all surprised at the failures, because
there's few who are attempting to use LDFLAGS, and
--with-syck=internal
is an "add-on" that most distros are either not enabling, or
using the "system" version of syck instead.

Thank you!

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'

[Jeff Johnson]

On Jun 28, 2011, at 1:16 AM, YuGiOhJCJ Mailing-List wrote:

> 
> Ok it compiles now. If I use db 5.1.19 I can compile rpm 5.3.5. If I choose 
> an other version, db 4.7.25 (previous) or db 5.2.28 (last), the rpm 5.3.5 
> compile fails.

Good.

The next step is to do
cd tests
make clean check

> I think we must use the same version mentioned in the INSTALL file in order 
> to compile rpm correctly (in the INSTALL file of rpm 5.3.5, the Berkeley-DB 
> version is 5.1.19).
> 

This is how rpm is released: targeting a single version of Berkeley DB.

Other versions (back to db-4.5.x) will "work" but won't just compile.
The changes aren't hard, just subtle. The complexity comes in getting
the AutoFu to handle all possible versions of Berkeley DB as installed
on all possible platforms. That's a huge matrix to cover and test.

> I am disappointed that Oracle now develops db. Oracle has a bad service (I 
> remember that was the same thing with MySQL, I couldn't download the 
> documentation because there was errors in the website). Now, not a surprise, 
> their website has some problems for db when you go at :
> http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html
> you can choose between some previous releases but not all :
> - 5.2.28 http://download.oracle.com/otn/berkeley-db/db-5.2.28.tar.gz
> - 5.1.25 http://download.oracle.com/berkeley-db/db-5.1.25.tar.gz
> - 5.1.19 No longer available for download!
> - …
> 

I think you should distinguish opinions about Oracle's web site from Berkeley 
DB code.

Recent changes to Berkeley DB are timely, and active, and with interesting
and useful "features". Merging sqlite3 <-> Berkeley DB is a useful and
innovative change.

Note that there *is* ABI breakage between 5.1.19 -> 5.1.25. ALl that means
is that one MUST recompile *everything* that links -ldb in order to upgrade
from 5.1.19 -> 5.1.25. The issue is that an element was removed from
within an internal structure that has some profound consequences. WHile
annoying, its not a show stopping change or anything else: someone had
a brain fart. Ooops.

> Indeed, some links requires authentication (5.2.28), so I must have an oracle 
> account, and some versions are no longer available (5.1.19).
> 

Yes I find the Oracle web site rather dreadful to navigate.

There's worse: Try finding something on HP or Cisco web sites sometime.

> But the thing which is funny is that these archives can be downloaded from 
> there web site if you find the good url :
> -http://download.oracle.com/berkeley-db/db-5.1.28.tar.gz
> -http://download.oracle.com/berkeley-db/db-5.1.25.tar.gz
> -http://download.oracle.com/berkeley-db/db-5.1.19.tar.gz
> 
> But I think this is not the place to say bad things about the Oracle services.
> 

Yes. Still: Berkeley DB is best-of-breed and the newer releases are
both active and innovative in my professional opinion.

> Anyway, thank you for rpm5 and your support :-)
> 

np.

73 de Jeff
> 
> On Mon, 27 Jun 2011 13:02:33 +
> "Miller, Vincent (Rick)"  wrote:
> 
>> 
>> I had a similar issue.  Turned out it was unable to locate the Berkley DB
>> header, db.h.  The maintainer, Jeff, says that RPM5's include looks like
>> #include .
>> 
>> Looks like you have an older version of Berkley DB.  I am unsure if it is
>> supported.  You might try checking the INSTALL file in the source
>> directory to see if older versions are supported.
>> 
>> 
>> 
>> 
>> On 6/25/11 1:17 AM, "YuGiOhJCJ Mailing-List"
>>  wrote:
>> 
>>> 
>>> Hello,
>>> 
>>> When I compile rpm-5.3.5 I got this error :
>>> ./rpmdb.h:433: error: expected specifier-qualifier-list before
>>> 'DB_SEQUENCE'
>>> You can see the full error below.
>>> 
>>> I thought that was my db version which was not compatible but this is not
>>> the case.
>>> Indeed, I try also with db-4.7.25 and I again see this error in my
>>> terminal...
>>> 
>>> Someone has already reported this error :
>>> http://rpm5.org/cvs/tktview?tn=65
>>> So I am not alone.
>>> 
>>> Could you give me a way to resolve this problem?
>>> 
>>> Thank you.
>>> 
>>> $ make
>>> make  all-recursive
>>> make[1]: Entering directory
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/po'
>>> make[2]: Nothing to be done for
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/po'
>>> Making all in pcre
>>> make[2]: Entering directory
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/pcre'
>>> make[3]: Nothing to be done for
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/pcre'
>>> make[2]: Leaving directory
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/beecrypt'
>>> make  all-recursive
>>> make[3]: Entering directory
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/beecrypt'
>>> make[4]: Nothing to be done for
>>> /home/yugiohjcj/documents/downloads/software/rpm-5.3.5/beecrypt'
>>> Making all in include
>>> make[4]: Entering directory all'.
>>> make[4]: Leaving directory
>>> /home/yugiohjcj/documents/downloads/software/rpm

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

I'd be happy to take you off the list but I do not know what address is
registered.

If you can forward me ( or ) a message --
including all the delivery headers -- then I should be able to figure
out what name needs to be unsubscribed.

If you know the name that was subscribed, then you can go to
http://rpm5.org/community and unregister as well (assuming you can
get a copy of the e-mail password from the original subscriber).

hth

73 de Jeff

On Jun 27, 2011, at 3:08 PM, Ruiz-Angel, Mariela wrote:

> Please take me off this email list. The person who this pertains to does not 
> work for hp anymore. 
> 
> m...@hp.com
> 
> 
> Mariela Ruiz-Angel
> Hewlett-Packard Company
> Commercial Sales Account Manager
> Imaging & Printing Group
> T: 1-800-277-8988 Ext: 7722825
> m...@hp.com
> 
> 
> 
> 
> 
> 
> -Original Message-
> From: rpm-users-ow...@rpm5.org [mailto:rpm-users-ow...@rpm5.org] On Behalf Of 
> Anders F Björklund
> Sent: Monday, June 27, 2011 1:06 PM
> To: rpm-users@rpm5.org
> Subject: Re: RPM5 on FreeBSD: hto* functions
> 
>>> I would be happy with the FreeBSD port of RPM.  I did do an install of it
>>> and it did not install the rpmbuild binary.  This was one of the main
>>> reasons for downloading and installing from source...
>> 
>> Unlike the Debian package, the FreeBSD port *does* install the
>> rpmbuild binary. [...]
> 
> My bad, the Debian package does include the rpmbuild binary
> even if it doesn't allow installing using rpm (by default).
> 
> --anders
> 
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

On Jun 27, 2011, at 2:39 PM, Miller, Vincent (Rick) wrote:

> On 6/27/11 2:18 PM, "Jeff Johnson"  wrote:
> 
> 
>> 
>> On Jun 27, 2011, at 1:59 PM, Miller, Vincent (Rick) wrote:
>> 
>>> On 6/25/11 4:39 AM, "Anders F Björklund"  wrote:
>>> 
>>> 
>>> 
>>>> 
>>>> The BSD and Mac ports are currently using 5.2.1, that is the pre-ACID
>>>> RPM:
>>>> http://www.freebsd.org/cgi/cvsweb.cgi/ports/archivers/rpm5/
>>>> https://trac.macports.org/browser/trunk/dports/sysutils/rpm52/
>>> 
>>> 
>>> I would be happy with the FreeBSD port of RPM.  I did do an install of
>>> it
>>> and it did not install the rpmbuild binary.  This was one of the main
>>> reasons for downloading and installing from sourceŠ
>>> 
>> 
>> Are you trying to build *.rpm's on FreeBSD? That is
>> doable, but the binary *.rpm's produced are likely
>> not going to usable anywhere except on FreeBSD (which
>> doesn't use rpmbuild) without a fair amount of work.
>> 
>> What problem are you trying to solve?
> 
> The problem being solved is not really a problem.  A decision was made
> that ops should employ a unified approach to managing software on a
> system.  By unified approach, I mean issuing the same command to install
> something, despite the platform.  The task to integrate RPM into FreeBSD
> was handed to me.
> 

Ah, k. Your easiest path by far is going to
be to use Anders build of rpm-5.2.1 in FreeBSD
ports. That SHOULD just drop-in without the
issues you are reporting, and is likely all
you need (and likely prefer) for unified management.

The harder issue with "management" will be to teach
rpm about what F*BSD provides in ports and natively.
Your "unified approach to managing software" likely
has reasonably sane exclusions for "native" software.

There are 2 approaches to teaching rpm about what rpm "provides".

1) build a stub pkg automatically that "provides" all
the usual libraries in an empty (i.e. no files) package.

2) add provides as needed to /etc/rpm/sysinfo/Providename.
E.g. you are almost certainly going to have to do
mkdir -p /etc/rpm/sysinfo
echo "/bin/sh" >> /etc/rpm/sysinfo/Providename
in order to install any package that includes a shell script let.

The approach in 1) is a little bit (but only a little bit) harder
to get into place than the drudgery of stubbing out every provides
encountered.

See scripts/vpkg_provides.sh for the script that automates
generation of the the empty package that provides all the
usual "native" FreeBSD libraries. The script will generate
a *.spec file that needs
Name:
Version:
Release:
added in order to be built and installed. You need to choose
what you want to call the package.

> 
> RPMs that get built on FreeBSD will only be installed in FreeBSD and vice
> versa.
> 

That works. If you were interested in build *.rpm packages for
linux from FreeBSD hosting, well, that too is doable just trickier.

> 
>> 
>>> Perhaps I'll setup another VM and give the FreeBSD port another tryŠ
>>> 
>> 
>> I can turn on my FreeBSD VM which worked with the "%fbsd8x"
>> build options in devtool.conf under buildbot's for several
>> weeks running "make check if there's really a need.
>> 
>> You can have the whole VM if you prepared for a 20+ GB VMFusion download.
>> 
>> I likely can reinstall and redo the entire VM and build RPM from
>> a cvs checkout in less time than the download takes.
> 
> Don't worry about your FreeBSD VM at the moment.  I'm just interested in
> the simplest way to get rpm and rpmbuild to function on FreeBSD.  If I can
> accomplish this with the BSD port, that will likely be the method I use to
> integrate.
> 

Again rpm-5.2.1 from ports is likely the minimal effort path.

rpm-5.3.x is better code and higher performing, but trickier to get built.

I'd choose the path of least resistance initially to achieve your goal.

I can/will deliver you a rpm-5.3.x package built on FreeBSD if you can
state how you want that package built. It is in fact harder to walk
through the entire set of build options than it is to do the build.

See the %fbsd8x stanza in devtool.conf for what I last used. Note
that as a developer I'm forced to maximally featured (like 4-5 crypto
stacks and 4-5 embeddings and 3+ compressions all on "best effort"
in order to achieve code coverage and to find flaws. A "production"
choice of build options could/should be simpler than what is in %fbsd8x.

> I must admit, I am unfamiliar with devtool and devtool.conf.

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

On Jun 27, 2011, at 1:59 PM, Miller, Vincent (Rick) wrote:

> On 6/25/11 4:39 AM, "Anders F Björklund"  wrote:
> 
> 
> 
>> 
>> The BSD and Mac ports are currently using 5.2.1, that is the pre-ACID RPM:
>> http://www.freebsd.org/cgi/cvsweb.cgi/ports/archivers/rpm5/
>> https://trac.macports.org/browser/trunk/dports/sysutils/rpm52/
> 
> 
> I would be happy with the FreeBSD port of RPM.  I did do an install of it
> and it did not install the rpmbuild binary.  This was one of the main
> reasons for downloading and installing from source…
> 

Are you trying to build *.rpm's on FreeBSD? That is
doable, but the binary *.rpm's produced are likely
not going to usable anywhere except on FreeBSD (which
doesn't use rpmbuild) without a fair amount of work.

What problem are you trying to solve?

> Perhaps I'll setup another VM and give the FreeBSD port another try…
> 

I can turn on my FreeBSD VM which worked with the "%fbsd8x"
build options in devtool.conf under buildbot's for several
weeks running "make check if there's really a need.

You can have the whole VM if you prepared for a 20+ GB VMFusion download.

I likely can reinstall and redo the entire VM and build RPM from
a cvs checkout in less time than the download takes.

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'

[Jeff Johnson]

On Jun 27, 2011, at 10:11 AM, Sriram Narayanan wrote:

> Hi Jeff:
> 
> Some questions below:
> 
> On Mon, Jun 27, 2011 at 7:30 PM, Jeff Johnson  wrote:
> 
> 
>> There is no simple solution that I see to the above: either more resources 
>> @rpm5.org
>> are needed (which is unlikely to happen), or users are going to have
>> to patch the released version to accommodate their own specific build needs.
>> 
> 
> What are these resources ? People or computers ?
> 

Interest mostly. Its utterly trivial to find hardware assets,
just the sysadmin maintenance and QA is already eating up
2-3 weeks per month of my time doing tedious "due diligence".

I can't justify that time expenditure any further.

>> The necessary changes are not all that hard:
>> 
>>There are two places that
>>#include 
>>is used that need to be changed. Its the "db51/" portion
>>of the include that causes build failures.
>> 
> 
> You've answered this before, but could you please tell us once again
> what would be the repercussions of changing this to
> 
> #include 
> 

That *is* what should be done. Adding any sort of hierarchical
structure to #include's causes issues. E.g. building rpm from
a cvs checkout with
#include 
is impossible because of the name collision on "rpm". Sure
the problem could be "fixed" by rearranging how directories
in RPM sources (in CVS) are created. But the general fix is
Never do
#include 
Always add ""-I/prefix/some/where/include/foo" to CFLAGS and
#include 

Anyways I'm well aware of the problem, there's a snarl of build
issues conflicting with features related to
Which sqlite should be used: from db-5.x.y or external?
that instantly appear if you try
#include 
Been there, done that, wrestled for a month last September with
the issues that cannot be resolved until
Should rpm embed sqlite? How?
have clearer answers.

>> And there are these 3 approaches to building RPM:
>> 
>>1) install db-5.1.x into
>>/usr/include/db51/db.h
>>/usr/lib/libdb-5.1.*
> 
> The default make install for bdb creates some other folder structures.
> I've had to symlink from those locations to the above.
> 

Yup. Everyone does things differently.

FYI: The *.spec I use on other platforms that do not have
Berkeley DB db-5.1.x is available by checking out the top
level "distro" module from @rpm5.org cvs. Download the necessary
"official" release tar ball's from Oracle and build. WORKSFORME
almost everywhere (though I somethings have to adjust paths to
accomodate uglix vendor divergence: *BSD tends to choose different
paths than linux does hysterically while installing Berkeley DB).

>> 
>>2) patch RPM to use a different version of Berkeley DB
>> 
>>3) re-add Berkeley DB db-5.1.x internal to RPM.
>> 
> 
> What is the reason that you'd removed this ? How about making this the
> default again, with us distro folk having to explicitly specify
> --with-db=external ?
> 

The positive justification for removal is
The released tar ball is >50% smaller without Berkeley DB included.

Sure I can flip … flop … flip … flop … forever. I have gone along
with the de facto chosen conventions of
Always use system Berkeley DB.
largely because its a waste of time to continue the discussion.

If the choice was up to me, I'd _ALWAYS_ embed Berkeley DB everywhere
needed. Meanwhile I don't care to endlessly restate my engineering reasons to
linux "community"advocates  that just want to chant
Bloat! Bloat! Bloat! Bloat! Bloat! Bloat!
in spite of what is recommended by Oracle (nee Sleepycat) for >10 years.

>> @rpm5.org cannot dictate which of those approaches end-users wish to choose 
>> in RPM.
>> 
> 
> I feel that getting rpm5 to build easily is a good goal to have. I've
> learned that if I read your INSTALL document, and then follow
> devtool.conf, then life becomes easier.
> 

The flaw here is that RPM has a whopping amount of
Have it your own way!
flexibility. SO there's insufficient context to define "easily"
meaningfully, there's already a combinatorial failure of the
dozens of build options that cannot ever be properly tested
(even if there was a desire to do so).

E.g. its quite possible to build RPM "easily" with
no database support
no compression support
no crypto digest support
and have an utterly useless executable that "builds easily".

Please note that I'm _NOT_ disagreeing with the goal. Just that
how rpm builds is way way way too complex, and its the
Have it your own way!
complexity that needs to be ripped out imho, not made Yet More Adaptable!

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'

[Jeff Johnson]

On Jun 27, 2011, at 9:02 AM, Miller, Vincent (Rick) wrote:

> I had a similar issue.  Turned out it was unable to locate the Berkley DB
> header, db.h.  The maintainer, Jeff, says that RPM5's include looks like
> #include .
> 

I should explain the reasoning here:

Berekely DB has _ALWAYS_ had issues when upgrading. In fact
embedding internally, not attempting to use a system version
of Berkeley DB, has been recommended for more than a decade.

RPM embedded Berkeley DB for many years for exactly that reason.

What happened instead is that distros ignored what was in
the released RPM tar ball, and continue to attempt to use
an external version of Berkeley DB.

SO in spite of reservations (I have been releasing RPM for 13 years now),
I have gone along with the de facto distro policy of using the
system Berkeley DB and stopped distributing a "know good"
version of Berkeley DB embedded in RPM.

Not distributing Berkeley DB with RPM opens a whopping amount
of build breakage, and also adds a combinatorial failure
to QA testing during releases that @rpm5.org does not have
the resources to address.

So @rpm5.org releases target a single (usually the latest, RPM + db-5.2.x
is high on my todo++ stack currently) version of Berkeley DB.

Meanwhile what RPM needs from Berkeley DB is rather modest:
Any version of Berkeley DB back to db-4.5.x will Just Work.
if you succeed in building.

There is no simple solution that I see to the above: either more resources 
@rpm5.org
are needed (which is unlikely to happen), or users are going to have
to patch the released version to accommodate their own specific build needs.

The necessary changes are not all that hard:

There are two places that
#include 
is used that need to be changed. Its the "db51/" portion
of the include that causes build failures.

And there are these 3 approaches to building RPM:

1) install db-5.1.x into
/usr/include/db51/db.h
/usr/lib/libdb-5.1.*

2) patch RPM to use a different version of Berkeley DB

3) re-add Berkeley DB db-5.1.x internal to RPM.

@rpm5.org cannot dictate which of those approaches end-users wish to choose in 
RPM.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Compile Problems on FreeBSD 8.2

[Jeff Johnson]

On Jun 25, 2011, at 4:13 AM, Anders F Björklund wrote:

> vrwmil...@gmail.com wrote:
> 
>> I am compiling RPM 5.3.4 on FreeBSD 8.2-RELEASE and have encountered a 
>> compile time error. I am hoping someone might be able to help resolve the 
>> error condition. The error is below. Is the error a result of missing or 
>> incorrect versions or libraries?
> 
> The easiest way to build RPM on FreeBSD is to use the port*, but if you want 
> a more recent version than 5.2.1 the second easiest is to use "devtool". If 
> you follow the %ports comments in devtool.conf, running ./devtool freebsd 
> should configure and make it for you...
> 
> * http://www.freebsd.org/cgi/cvsweb.cgi/ports/archivers/rpm5/
> 

Actually the standalone "%freebsd" target that re-invokes devtool
is more likely to have bit rot than building from a cvs checkout and using
./devtool checkout
./devtool fbsd8x
which was running under buildbot's more recently than whatever
is in FreeBSD ports.

And the "%fbsd8x" target removes a level of recursion that is
quite tricky to explain and "support" imho.

No matter what:
Have it your won way!
has made RPM nearly impossible to build reliably anywhere, not otherwise.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: rpm errors upon running

[Jeff Johnson]

On Jun 23, 2011, at 1:04 PM, Miller, Vincent (Rick) wrote:

> I had not loaded berkely-db because I thought I had read somewhere that there 
> was a bundled berkley-db with RPM5.  I have since loaded Berkley-DB 5.1.x 
> from FreeBSD ports and was able to get the compile to move along further.
> 

Good.

> It failed later when attempting to link against libgomp which is not 
> installed on the system.  I am planning to reconfigure disabling OpenMP, 
> unless it provides a feature we might be able to utilize.  Can you clarify 
> the features OpenMP provides to RPM5?
> 

RPM is headed towards multi-threaded installs.

One way to achieve parallelism would be to use OpenMP.

When looking at feasibility, I parallelized the loop over multiple digests
(not the digest algorithms per se which are nearly impossible to parallelize)
used by RPM and benchmarked rpm digest (which can run >100 digests
in parallel directly off of input/output buffers) and measured a 7x speed-up
on a 4-way.

Since a 7x speedup from a single #pragma is a quite remarkable gain,
I parallelized --verify to see how much gain might be achieved in a
non-artificial "real world" case. The results there were less impressive
(~1.5x faster from a primitive implementation, since improved to ~2x
back in Jabuary in rpm-5.4.0).

The refactoring cost to use OpenMP #pragma in RPM code would
be considerable, and so likely OpenMP will NOT be used when
RPM becomes multi-threaded.

So the "feature" provided by using OpenMP in 5.3.1 is that --verify
would run ~1.6x faster on SMP.

I personally wouldn't bother too much with OpenMP on FreeBSD based
on my experiences compiling and using RPM on Mac OS X, where attempting
to use GOMP and GCC is a rather rugged wilderness trek, paddling upstream and 
swatting
lots and lots of mosquitoes while trying to avoid more dangerous fauna.

YMMV of course ;-)

hth

73 de Jeff

Re: RPM5 on FreeBSD: hto* functions

[Jeff Johnson]

On Jun 23, 2011, at 2:24 PM, Miller, Vincent (Rick) wrote:

> Hi,
> 
> I'm compiling RPM 5.3.11 on FreeBSD 8.2-RELEASE and encountered an error that 
> is more fundamental than previous errors.  dbconvert.c  makes calls to 
> various hto*() functions, which are not defined in FreeBSD's libc.
> 

Replace dbconvert.c with dbconvert.sh.

The program is NOT essential to RPM, nor is the executable used anywhere.

The sources are included for reference.



> When compiling dbconvert.c, the compiler throws an error indicating that 
> htobe32 and htole32 are undefined symbols (see below).
> 
> dbconvert.o(.text+0x894):/home/vmiller/rpm-5.3.11/tools/dbconvert.c:184: 
> undefined reference to `htobe32'
> dbconvert.o(.text+0x8a9):/home/vmiller/rpm-5.3.11/tools/dbconvert.c:176: 
> undefined reference to `htole32'
> dbconvert.o(.text+0x8cb):/home/vmiller/rpm-5.3.11/tools/dbconvert.c:176: 
> undefined reference to `htole32'
> 
> A site (http://www.man-online.org/page/3-htobe32)  explained that including 
> sys/endian.h would be necessary.  I modified the tools/dbconvert.c to add 
> #include .  It resolved the initial errors, but produced 
> another indicating bswap32 was an undefined symbol.  However, sys/endian.h 
> does define bswap32.
> 

The dbconver.c code is pretty recent and hasn't been compiled much on
non-linux. So I'm not too surprised at portability issue with endian issues.

> Why is the compiler unable to reference the bswap32 definition in 
> sys/endian.h?  Is there a way to force this or is this a sign that installing 
> RPM5 onto FreeBSD will be more of a pain than I had initially anticipated?
> 

I can't easily answer programming questions about platforms that I
do not currently have development access to.

Don't build dbconvert.c would be my suggestion.

And reading signs instead of try-and-see can easily trick you imho.

When you finish building, do
cd tests
make check
and report what works and what doesn't.

>From the output of "make check", I will be able to tell you how
much pain you are in for, and whether you have built successfully.

hth

73 de Jeff

Re: rpm errors upon running

[Jeff Johnson]

(sorry for the empty msg)

On Jun 20, 2011, at 2:23 PM, Miller, Vincent (Rick) wrote:

> Great…thanks.  Got a RPM 5.3.11 tarball.
> 
> I configure as followed and it exited normally:
> 
> ./configure —with-expat —with-path-magic=/usr/share/misc/magic 
> CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" LIBS="-lgss"
> 
> I then ran gmake and it errored during the compile in rpmdb with:
> 
> gmake[4]: Entering directory `/usr/home/vmiller/rpm-5.3.11/rpmdb'
> /usr/local/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H 
> -I. -I..  -I. -I.. -I../build -I../lib -I../lib -I../rpmdb -I../rpmio 
> -I../misc  -I../db -I../db -I../db/src -I../db/src -I../scripts -I../scripts 
> -I/usr/local/include -DRPM_OS_FREEBSD=080200 -I/usr/local/include   
> -I/usr/local/include   -I/usr/local/include -DRPM_OS_FREEBSD=080200 
> -I/usr/local/include   -I/usr/local/include   -fopenmp -g -O2 
> -fno-delete-null-pointer-checks -D_FORTIFY_SOURCE=2 -fstack-protector 
> -fexceptions -fexceptions -Wno-missing-field-initializers -D_GNU_SOURCE 
> -D_REENTRANT -MT librpmdb_la-dbconfig.lo -MD -MP -MF 
> .deps/librpmdb_la-dbconfig.Tpo -c -o librpmdb_la-dbconfig.lo `test -f 
> 'dbconfig.c' || echo './'`dbconfig.c
> libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I. -I.. -I../build -I../lib 
> -I../lib -I../rpmdb -I../rpmio -I../misc -I../db -I../db -I../db/src 
> -I../db/src -I../scripts -I../scripts -I/usr/local/include 
> -DRPM_OS_FREEBSD=080200 -I/usr/local/include -I/usr/local/include 
> -I/usr/local/include -DRPM_OS_FREEBSD=080200 -I/usr/local/include 
> -I/usr/local/include -fopenmp -g -O2 -fno-delete-null-pointer-checks 
> -D_FORTIFY_SOURCE=2 -fstack-protector -fexceptions -fexceptions 
> -Wno-missing-field-initializers -D_GNU_SOURCE -D_REENTRANT -MT 
> librpmdb_la-dbconfig.lo -MD -MP -MF .deps/librpmdb_la-dbconfig.Tpo -c 
> dbconfig.c  -fPIC -DPIC -o .libs/librpmdb_la-dbconfig.o
> In file included from dbconfig.c:14:
> ./rpmdb.h:433: error: expected specifier-qualifier-list before 'DB_SEQUENCE'
> ./rpmdb.h:490: error: expected specifier-qualifier-list before 'DB_LOGC'

You are not finding db.h for some reason.

What  version of Berkeley DB?

Where is it? More specifically, RPM expects
#include 
to find the correct db.h to include.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: rpm errors upon running

[Jeff Johnson]

ror: 'DB_ENV' has no member named 'lock_id_free'
> ./rpmdb.h: In function 'rpmlkGet':
> ./rpmdb.h:898: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:899: error: 'DB_ENV' has no member named 'lock_get'
> ./rpmdb.h:899: error: 'db_lockmode_t' undeclared (first use in this function)
> ./rpmdb.h:899: error: (Each undeclared identifier is reported only once
> ./rpmdb.h:899: error: for each function it appears in.)
> ./rpmdb.h:899: error: expected ')' before '_lockmode'
> ./rpmdb.h: In function 'rpmlkPut':
> ./rpmdb.h:910: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:911: error: 'DB_ENV' has no member named 'lock_put'
> ./rpmdb.h:911: error: 'DB_LOCK' undeclared (first use in this function)
> ./rpmdb.h:911: error: expected expression before ')' token
> ./rpmdb.h: In function 'rpmlgcOpen':
> ./rpmdb.h:922: error: 'DB_LOGC' undeclared (first use in this function)
> ./rpmdb.h:922: error: '_logc' undeclared (first use in this function)
> ./rpmdb.h:924: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:925: error: 'DB_ENV' has no member named 'log_cursor'
> ./rpmdb.h:926: error: 'struct rpmdb_s' has no member named 'db_logc'
> ./rpmdb.h: At top level:
> ./rpmdb.h:933: error: expected declaration specifiers or '...' before 'DB_LSN'
> ./rpmdb.h: In function 'rpmlgcGet':
> ./rpmdb.h:935: error: 'DB_LOGC' undeclared (first use in this function)
> ./rpmdb.h:935: error: '_logc' undeclared (first use in this function)
> ./rpmdb.h:935: error: 'struct rpmdb_s' has no member named 'db_logc'
> ./rpmdb.h:936: error: '_lsn' undeclared (first use in this function)
> ./rpmdb.h: In function 'rpmlgcClose':
> ./rpmdb.h:945: error: 'DB_LOGC' undeclared (first use in this function)
> ./rpmdb.h:945: error: '_logc' undeclared (first use in this function)
> ./rpmdb.h:945: error: 'struct rpmdb_s' has no member named 'db_logc'
> ./rpmdb.h:948: error: 'struct rpmdb_s' has no member named 'db_logc'
> ./rpmdb.h: At top level:
> ./rpmdb.h:955: error: expected ';', ',' or ')' before '*' token
> ./rpmdb.h:965: error: expected ';', ',' or ')' before '*' token
> ./rpmdb.h: In function 'rpmlgcPrintf':
> ./rpmdb.h:979: error: 'DB_ENV' has no member named 'log_printf'
> ./rpmdb.h: At top level:
> ./rpmdb.h:986: error: expected declaration specifiers or '...' before 'DB_LSN'
> ./rpmdb.h: In function 'rpmlgcPut':
> ./rpmdb.h:989: error: 'DB_ENV' has no member named 'log_put'
> ./rpmdb.h:989: error: '_lsn' undeclared (first use in this function)
> ./rpmdb.h: In function 'rpmmpfSyncAll':
> ./rpmdb.h:999: error: 'DB_LSN' undeclared (first use in this function)
> ./rpmdb.h:999: error: '_lsn' undeclared (first use in this function)
> ./rpmdb.h:1000: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:1001: error: 'DB_ENV' has no member named 'memp_sync'
> ./rpmdb.h: In function 'rpmmpfTrickle':
> ./rpmdb.h:1013: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:1014: error: 'DB_ENV' has no member named 'memp_trickle'
> ./rpmdb.h: In function 'rpmmpfClose':
> ./rpmdb.h:1023: error: 'DB_MPOOLFILE' undeclared (first use in this function)
> ./rpmdb.h:1023: error: 'mpf' undeclared (first use in this function)
> ./rpmdb.h:1023: error: 'struct rpmdb_s' has no member named 'db_mpf'
> ./rpmdb.h:1026: error: 'struct rpmdb_s' has no member named 'db_mpf'
> ./rpmdb.h: In function 'rpmmpfGet':
> ./rpmdb.h:1035: error: 'DB_MPOOLFILE' undeclared (first use in this function)
> ./rpmdb.h:1035: error: 'mpf' undeclared (first use in this function)
> ./rpmdb.h:1035: error: 'struct rpmdb_s' has no member named 'db_mpf'
> ./rpmdb.h: In function 'rpmmpfOpen':
> ./rpmdb.h:1047: error: 'DB_MPOOLFILE' undeclared (first use in this function)
> ./rpmdb.h:1047: error: 'mpf' undeclared (first use in this function)
> ./rpmdb.h:1048: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:1051: error: 'struct rpmdb_s' has no member named '_dbi'
> ./rpmdb.h:10

Re: rpm errors upon running

[Jeff Johnson]

On Jun 20, 2011, at 1:48 PM, Miller, Vincent (Rick) wrote:

> Ok.  Based on your input with regards to the error I am seeing with the 
> rpmbuild issues.  I'll need to rebuild RPM5.  So, I was going to download it, 
> but there are no source tarballs (just source rpms) for newer versions and I 
> cannot download newer sources from CVS due to a firewall.
> 
> How do you recommend I download a newer version?

Find scripts/rpm2cpio.sh (its also on the web site).

Download the *.src.rpm and run

rpm2cpio.sh rpm-*.src.rpm | cpio -dim

You will end up w 2 tar balls and one spec file jun the current directory.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: rpm errors upon running

[Jeff Johnson]

On Jun 20, 2011, at 10:57 AM, Miller, Vincent (Rick) wrote:

> Hi Jeff…thanks for the reply.  See my replies inline.  Also, I have an 
> rpmbuild error at the tail of this message.
> 
> 
> From: Jeff Johnson 
> Reply-To: 
> Date: Fri, 17 Jun 2011 19:10:33 -0400
> To: 
> Subject: Re: rpm errors upon running
> 
> 
> On Jun 17, 2011, at 1:43 PM, Miller, Vincent (Rick) wrote:
> 
>> Hi,
>> 
>> I have installed RPM5 on FreeBSD 8.2-RELEASE and am now unable to do 
>> anything useful with it due to errors.  I tried searching the mail archives 
>> for similar issues, but only found a reference to DB_CONFIG in reference to 
>> initializing a the rpm db.  Since this is a brand new install and there were 
>> no db files on the host, I figured it might apply.  It mentions mmapsize, 
>> which is definitely set, but offered no recommendations into calculating 
>> what to set it to.
>> 
> 
> I don't know how you installed, but
> make install
> from a build SHOULD copy a /var/lib/rpm/DB_CONFIG into place.
> 
> The DB_CONFIG file is MANDATORY with @rpm5.org.
> 
> The DB_CONFIG file was put into place and the truss output shows DB_CONFIG 
> being read.
> 
> 
> The contents (from the rpm-5_3 "production" branch) that pertain to locking 
> tunables are
> 
> #  Locking
> set_lk_max_locks16384
> set_lk_max_lockers  16384
> set_lk_max_objects  16384
> mutex_set_max   163840
> 
> The heuristic (that I use) is that the mutex_set_max should be ~10x larger
> than the other values, which is about the right scaling.
> 
> The size of the others depends on how many packages are managed,
> which is an extrinsic quantity that isn't guessable a priori. 16K -> 64K are
> about the right range, and conservative. There's likely something else
> wrong if you appear to need >64K.
> 
> How did you build rpm? From CVS? rpm-5.4.1 or rpm-5.3.11?
> 
> I downloaded a source tarball for 5.3.4 and installed with the commands below 
> after installing various ports  that appeared to be required.  The ports I 
> installed were popt, beecrypt, pcre, gss, and expat.  Some of these ports had 
> their own dependencies  they installed.
> 

You likely want to use the latest "production" rpm-5.3.11 release instead of 
rpm-5.3.4.
The @rpm5.org project does tick,tock monthly releases.

> ./configure —with-expat CPPFLAGS="-I/usr/local/include" 
> LDFLAGS="-L/usr/local/lib" LIBS="-lgss"
> gmake
> gmake install
> 
> 
> How many packages do you have?
> 
> 
> I currently do not have any RPM packages.  There are a couple hundred ports 
> installed, but this should have no bearing on RPM.
> 

OK. In order to use RPM on FreeBSD as an "add-on" you will almost certainly
need to supply some dependencies.

There are two ways to do this:

1) capture the usual provides (like /bin/sh) in a virtual package and
install that first.

2) stub-in dependencies as needed. E.g. you will almost certainly have
to provide "/bin/sh" somehow, which can be done like this
mkdir -p /etc/rpm/sysinfo
echo "/bin/sh" >> /etc/rpm/sysinfo/Providename

You are likely to find that the two rules
1) Every file requires its parent directory
2) Every symlink requires its endpoint.
to be a bit bothersome as well. Stub those out in /etc/rpm/sysinfo
(or what ever path you have chosen) using
echo "/usr/bin" >> /etc/rpm/sysinfo/Dirnames
echo "/path/to/endpoint" >> /etc/rpm/sysinfo/Filelinktos


> 
> 
>> I'll include as much info as I can here, but it'll mean the length of this 
>> mail may be substantial.
>> 
>> The error I receive when I attempt to rpm –qa is:
>> 
> 
> 
> 
>> 
>> #  Locking
>> set_lk_max_locks  8000
>> set_lk_max_lockers  8000
>> set_lk_max_objects  8000
>> mutex_set_max 8000
>> 
> 
> Try setting "mutex_set_max" to 8 not 8000.
> 
> I changed mutex_set_max to 8 and ran rpm –qa and it did not produce the 
> original error.  Additionally, it appears to have created the __db files in 
> /var/local/lib/rpm/.  This is a good start.
> 
> My next test is to attempt to build an RPM.  I am going with a very simple 
> test, initially, which is to create a new /etc/motd.  When I run rpmbuild on 
> the spec file, I get the following error.  Do you have any thoughts on this?
> 
> [root@freebsd /usr/local/src/rpm/SPECS]# rpmbuild -bb 
> vrsnconfig-FreeBSD-amd64-1.0.spec 
> Executing(%prep): /bin/sh -e /var/local/tmp/rpm-tmp.447
> + umask 022
> + cd /usr/local/src/rpm/

Re: rpm errors upon running

[Jeff Johnson]

On Jun 17, 2011, at 1:43 PM, Miller, Vincent (Rick) wrote:

> Hi,
> 
> I have installed RPM5 on FreeBSD 8.2-RELEASE and am now unable to do anything 
> useful with it due to errors.  I tried searching the mail archives for 
> similar issues, but only found a reference to DB_CONFIG in reference to 
> initializing a the rpm db.  Since this is a brand new install and there were 
> no db files on the host, I figured it might apply.  It mentions mmapsize, 
> which is definitely set, but offered no recommendations into calculating what 
> to set it to.
> 

I don't know how you installed, but
make install
from a build SHOULD copy a /var/lib/rpm/DB_CONFIG into place.

The DB_CONFIG file is MANDATORY with @rpm5.org.

The contents (from the rpm-5_3 "production" branch) that pertain to locking 
tunables are

#  Locking
set_lk_max_locks16384
set_lk_max_lockers  16384
set_lk_max_objects  16384
mutex_set_max   163840

The heuristic (that I use) is that the mutex_set_max should be ~10x larger
than the other values, which is about the right scaling.

The size of the others depends on how many packages are managed,
which is an extrinsic quantity that isn't guessable a priori. 16K -> 64K are
about the right range, and conservative. There's likely something else
wrong if you appear to need >64K.

How did you build rpm? From CVS? rpm-5.4.1 or rpm-5.3.11?

How many packages do you have?


> I'll include as much info as I can here, but it'll mean the length of this 
> mail may be substantial.
> 
> The error I receive when I attempt to rpm –qa is:
> 



> 
> #  Locking
> set_lk_max_locks  8000
> set_lk_max_lockers  8000
> set_lk_max_objects  8000
> mutex_set_max 8000
> 

Try setting "mutex_set_max" to 8 not 8000.

> 
73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: I killed my RPMDB --- any rescue possible?

[Jeff Johnson]

On Jun 13, 2011, at 11:05 AM, Eric MSP Veith wrote:

> 
> What's the preferred way to initially populate a chroot? 
> 

The only difference in rpm-5.3 related ro your issue
is running dbconvert.sh in %posttrans.

The whole idea of converting an *active* rpmdb *automagically*
is _INSANE_. There are far far too many error cases for any
dbconvert.sh script to _GUARANTEE_ success _ALWAYS_.

Everything is _EXACTLY_ as has always been implemented with
rpm and chroot(2). There are no changes:
A script/executable invoked while installing with rpm -r
can escape a chroot and do whatever it wishes.

73 de Jeff




smime.p7s
Description: S/MIME cryptographic signature

Re: Compile Problems on FreeBSD 8.2

[Jeff Johnson]

On Jun 13, 2011, at 10:56 AM, Eric MSP Veith wrote:

>> 
>> />/usr/home/vmiller/rpm-5.3.4/misc/.libs/librpmmi= sc.so: undefined
>> reference to GSS_C_NT_HOST=
>> BASED_SERVICE'*** Error code 1Stop in
>> /usr/home/vmill= er/rpm-5.3.4/tools.*** Error code 1> />Stop in /usr/home/vmil= ler/rpm-5.3.4/tools.*** Error code 1> />Stop in /usr/home/vmi= ller/rpm-5.3.4.*** Error code 1> />Stop in /usr/home/vmiller/= rpm-5.3.4.
>> 

This one bites me all the time …

… neon can be compiled with or without GSSAPI and every vendor/distro handles
slightly differently.

There's 3 approaches to getting a build to work:

1) add the libraries that internal neon needs using LIBS= or LDFLAGS=
before invoking ./configure

2) add the neon flag that explicitly disables GSSAPI support

3) build against external system neon, not with the internal neon

Both 1) and 2) assume --with-neon=internal, 3) passes --with-neon=external.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: I killed my RPMDB --- any rescue possible?

[Jeff Johnson]

On Jun 13, 2011, at 9:58 AM, Eric MSP Veith wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> On Sunday 12 June 2011, Jeff Johnson  wrote:
>>> However, after using "rpm -r /my/chroot -Uvh ...", the RPMDB *outside*
>>> the chroot is corrupted. When running "rpm -qa", this is what I get:
>> 
>> So which rpmdb needs fixing? The rpm inside or outside the chroot?
> 
> Outside. The host system's RPMDB is broken (after rpm -r ... -Uvh ...). The 
> chroot's RPMDB is intact, but the package I wanted to install is missing, 
> not so surprisingly.  :-)
> 

OK.

>> And what versions of rpm are in use inside and outside the chroot?
> 
> 5.3.10 in both cases.
> 

OK.

>> Underlying what?
> 
> Sorry, wrong wording: The bdb version linked with the current rpm-5.3.10 
> install is 5.1.25. The RPMDB was originally created using bdb 4.7.something, 
> and after installing rpm-5.3.10 (with bdb-5.1.25), I immediateley ran 
> dbconvert.sh.
> 

OK. There are no significant format changes in Berkeley DB
that are NOT handled by db-utils, specifically db_dump … | db_load.

What is subtle is that one MUST choose the correct version specific
db_dump/db_load to upgrade (and downgrade). I'm not at all
sure that dbconvert.sh is properly implemented to Just Work with
1) a outer <-> inner chroot "system"
2) multiple rpmdb paths
3) multiple database versions

The "safest" way to avoid problems is to do
chroot /your/chroot /bin/bash
and just run dbconvert.sh within.


>> Clean-up and save what you have in /var/lib/rpm (...)
>> Create a new directory and copy what you have (...)
>> Check (as root) that Packages is at least mostly intact:
>>  rpm -qavv
> 
> I did as advised, and rpm -qavv outputs this:
> 
> - ---%<---
> D: pool fd: created size 208 limit -1 flags 0
> D: pool iob:created size 20 limit -1 flags 0
> D: pool mire:   created size 84 limit -1 flags 0
> D: pool lua:created size 32 limit -1 flags 0
> D: pool ts: created size 720 limit -1 flags 0
> D: pool gi: created size 92 limit -1 flags 0
> D: pool db: created size 184 limit -1 flags 0
> D: pool dbi:created size 284 limit -1 flags 0
> D: opening  db index   /var/lib/rpm/Packages thread:rdonly mode=0x0
> D: pool mi: created size 88 limit -1 flags 0
> D: pool h:  created size 216 limit -1 flags 0
> libXfixes-4.0.3-1ev.i686
> error: rpmdb: header #33554432 cannot be loaded -- skipping.
> (none)-(none)-(none)
> ==> munmap(0xb65d[180811702]) error(22): Invalid argument
> error: rpmdb: header #67108864 cannot be loaded -- skipping.
> error: rpmdb: header #83886080 cannot be loaded -- skipping.
> error: rpmdb: header #100663296 cannot be loaded -- skipping.
> error: rpmdb: header #117440512 cannot be loaded -- skipping.
> error: rpmdb: header #134217728 cannot be loaded -- skipping.
> error: rpmdb: header #150994944 cannot be loaded -- skipping.
> error: rpmdb: header #167772160 cannot be loaded -- skipping.
> error: rpmdb: header #201326592 cannot be loaded -- skipping.
> error: rpmdb: header #218103808 cannot be loaded -- skipping.
> error: rpmdb: header #234881024 cannot be loaded -- skipping.
> error: rpmdb: header #268435456 cannot be loaded -- skipping.
> error: rpmdb: header #285212672 cannot be loaded -- skipping.
> error: rpmdb: header #301989888 cannot be loaded -- skipping.
> (none)-(none)-(none)
> (none)-(none)-(none)
> (none)-(none)-(none)
> error: rpmdb: header #385875968 cannot be loaded -- skipping.
> (none)-(none)-(none)
> Segmentation fault
> - --->%---
> 

OK. You have a damaged database for whatever reason. All of those
"header #12345678" are too big so you likely have NOT succeeded
in byte swabbing the header instances (i.e. what dbconvert.sh is doing).

(alternatively) You have run dbconvert.sh an even number of times
and so two applications of dbconvert.sh has undone the byte swabbing
originally intended.

You can try reading the rpmdb with pre-rpm-5.3 to assess how much data
is savable.

Doing rpm --rebuilddb (possibly with --nosignature --nodigest --nohdrchk 
disablers
to get past the segfault) WILL discard damaged headers that you can then 
replace later.

While doing --rebuilddb, make sure you do either
cd /var/lib/rpm
rm -f __db*
or
cd /var/lib/rpm
db_recover -v
(note no -e passed to db_recover to remove the __db cache)

A damaged cache in __db* files can/will screw a --rebuilddb worser.


>> 
>> Re-create the indices (and check that dbconvert.sh swapped the bytes):
>>  rpm --rebuilddb --vv
> 
> - ---%<---
> D: pool fd: created size 208 limit -1 flags 0
> D: pool iob:

Re: Some questions on directions ahead

[Jeff Johnson]

On Jun 11, 2011, at 12:45 PM, Sriram Narayanan wrote:

> Hi all:
> 
> I've got some questions:
> 1. Which rpm5 should we consider ?
> I see that the website still mentioned 5.3.6 as stable, and 5.3.7 as
> testing, though 5.3.11 was released a few days ago.
> 

Releases are time based, there's a release at the beginning of each month.

The benefit there is that a "fix" opportunity is never more than 30 days away.
The cost of monthly releases means that there's less time to detect and
report issues (but "continuous integration" builds under buildbots
detect most obvious failures).

The rpm-5_3 branch is the "production" branch. So there's no
new "features" to speak of and no reason not to prefer the latest
afaik. Any issues reported will be dealt with as rapidly as possible.

> This is just a case of the website not being updated ?
> 

Yes.

> 2. What are the plans for rpm 5.4 ?

ATM, rpm-5.4 is essentially the same as rpm-5.3, a place for
fixes to be devised for Mandriva.

Attempting multi-threaded installs is likely in rpm-5.4, as
is switching from C -> C++ for building. (Note that all that is
anticpated is a build with CC=g++ not a rewrite into C++).

> Is this what will be the next development branch ? Would it be fine

Yes.

> for the Belenix team to stick to rpm 5.3.11 for the next three months
> while we get other stuff sorted out and to then move to 5.4 (or
> newer), or should we look at that right now ?
> 

rpm 5.3.x are the best "production" bits available atm.

> 3. What is the reality about rpm6 ?

The reality is that I unplugged a DNS record for 10 days and then
plugged it back in.

> I was trying to figure out how to initialize an rpmdb and I searched
> for "mandriva rpm5" and I ended up at some pages that are talking
> about rpm6.org.
> I checked and saw that rpm6.org gets us back to rpm5.org.

Yep. That will likely change this summer. I haven't a clue how or why yet …

> What's the real news, and what are the community directions ? I've not
> seen any discussion around this on the rpm5 mailing lists.
> 

Not much to discuss. It's nobody's "business" but mine, no "community" there at 
all,
just a bunch of gawking gossiping phear mongering and FUD sucking paparazzi ...


73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: I killed my RPMDB --- any rescue possible?

[Jeff Johnson]

On Jun 12, 2011, at 4:35 PM, Eric MSP Veith wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hello list,
> 
> I killed my RPMDB, propably by misusage. This is how I did it: I upgraded 
> from RPM-5.1.9 to 5.3.10. I ran dbconvert.sh (which told me that the 
> conversion failed), but repeated usage of db_recover -ev and rpm --rebuilddb 
> finally got things in order. I'm using a chroot, which went the same path.
> 

Well … mebbe.

> However, after using "rpm -r /my/chroot -Uvh ...", the RPMDB *outside* the 
> chroot is corrupted. When running "rpm -qa", this is what I get:
> 

So which rpmdb needs fixing? The rpm inside or outside the chroot?

And what versions of rpm are in use inside and outside the chroot?


> - ---%<---
> Freeing read locks for locker 0xa: 23945/3062028096
> Freeing read locks for locker 0xb: 23945/3062028096
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> Freeing mutex for process: 23945/0
> libXfixes-4.0.3-1ev.i686
> error: rpmdb: header #33554432 cannot be loaded -- skipping.
> (none)-(none)-(none)
> Segmentation fault

Every time you see "Segmentation fault" you will see all the noise
on the next invocation of rpm as root.

> - --->%---
> 
> How, db_convert and rpm --rebuilddb don't help anymore. The latter one 
> prints:
> 
> - ---%<---
> Freeing read locks for locker 0xe: 25167/3062372160
> Freeing read locks for locker 0xf: 25167/3062372160
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> Freeing mutex for process: 25167/0
> error: db3: header #33554432 cannot be loaded -- skipping.
> error: db3: header #67108864 cannot be loaded -- skipping.
> error: db3: header #83886080 cannot be loaded -- skipping.
> error: db3: header #100663296 cannot be loaded -- skipping.
> error: db3: header #117440512 cannot be loaded -- skipping.
> error: db3: header #134217728 cannot be loaded -- skipping.
> error: db3: header #150994944 cannot be loaded -- skipping.
> error: db3: header #167772160 cannot be loaded -- skipping.
> error: db3: header #201326592 cannot be loaded -- skipping.
> error: db3: header #218103808 cannot be loaded -- skipping.
> error: db3: header #234881024 cannot be loaded -- skipping.
> error: db3: header #268435456 cannot be loaded -- skipping.
> error: db3: header #285212672 cannot be loaded -- skipping.
> error: db3: header #301989888 cannot be loaded -- skipping.
> error: db3: header #385875968 cannot be loaded -- skipping.
> error: db3: header #486539264 cannot be loaded -- skipping.
> rpm: header.c:1050: headerLoad: Assertion `(rpmint32_t)rdl >= 0' failed.
> Aborted

This is a sainity check abort consistent with other failures. The
sanity check isn't strong enough to catch all failures because
(except for signature/digest checks which are often routinely disabled
by applications using an rpmdb) there's insufficient structural info
in the header blob to test that the data within is well-formed:
bits == bits

> - --->%---
> 
> The underlying BDB version is 5.1.25.
> 

Underlying what?

> I'd be grateful for any hint, and also some explainations about why I did it 
> to kill the system rpmdb using "rpm -r".
> 

Clean-up and save what you have in /var/lib/rpm:
cd /var/lib/rpm
db51_recover -ev
db51_load -r lsn Packages
cd ..
mv rpm rpm-SAVE

Create a new directory and copy what you have:
cd /var/lib
mkdir -p rpm/log rpm/tmp
cp rpm-SAVE/DB_CONFIG rpm/
cp rpm-SAVE/Packages rpm/

Check (as root) that Packages is at least mostly intact:
rpm -qavv

Re-create the indices (and check that dbconvert.sh swapped the bytes):
rpm --rebuilddb --vv

The check is that the "Max header instance" after the rebuild isn't
~4Gb (it should be a small positive integer of order the #pkgs you have 
installed).

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: yum support in rpm5 ?

[Jeff Johnson]

On Feb 23, 2011, at 2:05 PM, Tobias Gerschner wrote:

> Ah could've saved me quite some writing there. The question remains. Any 
> interest to review the current repo-md format and improve it or start over?
> 

These days I'm pursuing pogoplugs running rsync
to get LAN repositories on gigabit ethernet.

If interested, I have 2 (of 3 soon to become 5) pogoplugs
wired up at
http://eastham.rpm5.org
for ssh access on NAT'ed ports. Send mail privately if you
want to hack a bit (plugbox linux <- from archlinux currently.
I'd like to see an RPM based ARM distro somehow instead).

The whole idea of mirrored *.rpm and repodata/* is last century now
that NoSQL! databases can be set up in a weekend. The major
problem is the format of the markup is really really hard
to change in intelligent ways. Any change to the markup creates
instant incompatibility.

The other flaw with most depsolvers is trying to do *everything*
while some developer with a short attention span and an itchy ^C
finger is watching the progress bars. Wotta waste of talent imho.

Instead the store-and-forward piece of depsolver's needs to
be split from the depsolving piece. rsync is far more efficient
than the complexity of deltarpm's etc ...

But the simple answer to your question is in
/usr/lib/rpm/qf/yum_primary_sqlite
on boxes running @rpm5.org code.

The contents of that file is know as a "queryformat template"
and that's all it is, a --queryformat (from Hell ;-) that can be
adjusted to taste to create whatever repository metadata spewage
one wishes to invent.

You can see several (perhaphs Frankensteinian abominations ;-) work-in-progress
creations of my own in /usr/lib/rpm/qf/*. I need server side infrastructure
like mongo.rpm5.org ro exist before I get to better markup spewage.

hth

73 de Jeff







smime.p7s
Description: S/MIME cryptographic signature

Re: What's the createrepo equivalent in rpm5 ?

[Jeff Johnson]

On Feb 23, 2011, at 1:30 PM, Matthew Dawkins wrote:

> 
> We tried using rpmrepo, but it doesn't support some time saving features like 
> createrepo --update the last time I we checked. Yum is only needed by 
> createrepo, otherwise we use smart for our package manager.
> 

Quoting myself:

The output of rpmrepo was identical to what was in use by yum when
implemented bck in 2008. Since then yum has changed some minor file naming
conventions and otherwise altered the "standard" (but actually de facto)
rpm-metadata format slightly.

Wouldn't be hard to fix, is on my todo++ list, but there's been zero
interest in using rpmrepo so far, hence its a low priority.

You know where http://launchpad.net/rpm blueprints are.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: yum support in rpm5 ?

[Jeff Johnson]

On Feb 23, 2011, at 1:07 PM, Jeff Johnson wrote:

> 
> Note carefully:
>   Yum will NEVER accept any patches from @rpm5.org.
>   Clearly and publically and repeatedly stated by yum developers.
> 

I should be more positive:

YUM was forked in order to support @rpm5.org users the day
the statement of "... NEVER ..." was made by yum developers.

So there is a clone of the git repository as well as a cvs repository
of yum code, and you can expect "support" for yum + @rpm5.org
going forward, whatever that might mean.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: What's the createrepo equivalent in rpm5 ?

[Jeff Johnson]

On Feb 23, 2011, at 12:37 PM, Sriram Narayanan wrote:

> Should we consider createrepo or does rpm5 provide something that's 
> equivalent ?
> 

Depends on what you want.

There is a rpmrepo client-side tool that will create both rpm-metadata
XML used by many depsolvers as well as the sqlite schema in the database
format currently used by yum.

> Would this be usable by yum ?
> 

The output of rpmrepo was identical to what was in use by yum when
implemented bck in 2008. Since then yum has changed some minor file naming
conventions and otherwise altered the "standard" (but actually de facto)
rpm-metadata format slightly.

Wouldn't be hard to fix, is on my todo++ list, but there's been zero
interest in using rpmrepo so far, hence its a low priority.

The major features to rpmrepo are

1) client-side rather than server-side. This basically means
that rpmrepo can generate markup on the local client in any
format needed, thereby uncoupling from whatever incompaitibilities
are introduced on servers by "feeping creaturism" with later
yum versions.

2) faster (~20%) than createrepo when I last checked, largely because
in C not in python.

3) integrated support on the rpm CLI. E.g
rpm -qp --yum:primary.xml *.rpm
can be done any time one wishes, and generates rpm-metadata
in XML for whatever "*.rpm" constitutes a package repository collection.

Note carefully that rpm-metadata markup is poorly designed and includes
compute intensive elements like package file digests that largely make
benchmark (or other comparisons) irrelevant. E.g. a very minor change
to the markup to use pre-computed header+payload or header-only digests
would speed up createrepo/rpmrepo by 1-2 orders of magnitude no matter
what language was chosen for an implementation.

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: yum support in rpm5 ?

[Jeff Johnson]

On Feb 23, 2011, at 12:39 PM, Sriram Narayanan wrote:

> Is there some form of yum support in rpm5 ?
> 

Not in, but yum works fine with rpm-python from @rpm5.org.

(aside)
There's one minor issue with a StopIteration exception
from a python iterator for which both a fix and a 3 line
change to yum are known. There's no place to put the "fix".

There are "features" in @rpm5.org that will never be supported
by yum. You can either disable the feature, or otherwise
work around yum developers recalcitrance.

Note carefully:
Yum will NEVER accept any patches from @rpm5.org.
Clearly and publically and repeatedly stated by yum developers.

But I use yum with @rpm5.org all the time, and IDMS linux ships
with y...@rpm5.org as its depsolver. I.e. "works".

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Info needed - Packages vs compiling on production

[Jeff Johnson]

On Feb 21, 2011, at 1:16 PM, Sriram Narayanan wrote:

> Hi everyone:
> 
> I'm asking this question here since there are many distribution
> builders on this list.
> 
> I'm having a discussion with some colleagues about the merits of
> installing native libraries via a package manager than by compiling
> source on production.
> 
> Compiling source on production is how most Ruby application developers
> deploy rubygems having native bindings onto production servers.
> 

SHort answer: both get the job done.

Package management scales, rebiuilding does not.

And package management tracks software over its entire life-cycle,
its not just "installing".

(aside)
In fact, the poorly understood reason for RPM's success -- everyone
is so so busy chanting "dependency hell" instead -- is that rpm
has flawless erases, cleans up everything, never misses.

> My reasoning for using package managers includes the following:
> - the various benefits listed at wikipedia
> (http://en.wikipedia.org/wiki/Package_management_system)
> - dependency resolution
> - file verification (e.g. rpm -V)
> - roll forward - roll back - to the extent possible by various package 
> managers
> 

Well --rollback is (mostly) not part of package management. There are only
a few package managers (nix and conary come to mind) that can really do
--rollback without caveats like "... to the extent possible ...".

The killer for --rollback in package management is the instantaneous
(and quite predictable) RFE from users:
How do I rollback everything except blah?
and the short answer
You don't, that's not how --rollback "works".
pleases noone.


> My reasoning against compiling on production:
> - doesn't help resolve dependency hell when you start to upgrade
> libraries partly (when A depends on J1 and B1, while B1 depends on
> J2). A package manager would tell you very quickly when you run into
> such situations.
> - There's some message somewhere about installing only the bare
> minimum files onto a production server. Compilers and Devel files are
> not required on production.
> 
> Have such topics been discussed somewhere ? Have you faced such
> questions from your end users or in life ?
> 

The archetype for your question goes way way way back to *BSD
make world
as opposed to binary package management (there are source (my words) package 
managers
like MacPorts too).

The best answer I (I'm an old school *BSD developer) know of to
"make world" <-> rpm
I got from Erik Troan a long time ago and illustrates the "support" scaling
that comes from doing binary package management:

Let's say you build & test a package and it "works".

Now put that package on a DVD, wait a year or two.

Go find the DVD, install the package, and horrors! the package DOES NOT 
WORK.

The trick question is this
Q: Where do you look to fix the problem?
A: Not the binary package: it was "known good" and verifiably hasn't 
changed.

Its the long shelf life, and minimizing "support" necessary for "make world" 
distros
that scales and makes RPM worth the pain of wrestling with "dependency hell".

Caveat:
My opinions aren't typical of most RPM users.

73 de jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: RPM Installation status

[Jeff Johnson]

On Feb 18, 2011, at 4:14 PM, pinto.e...@gmail.com wrote:

> In Some rpm version it is possible that a post fail could return always a 
> zero status.there is bugzilla for this. Regards-Original 

TIts possible to mask the exit status, and its also possible for a script to
return 0 and do nothing useful.

But logic and convention sez' that:
All scripts have an exit status in uglix.
An exit status of 0 == success, else failure.

You know all this ... or perhaps you are referring to the daft (imho)
@rpm.org decision to _IGNORE_ scriptt return codes.

Might as well just add
%post -p /dev/random
and start the betting pol to see how long it takes for the
first successful return code from /dev/random.

73 de Jeff  my opinions are all mine



smime.p7s
Description: S/MIME cryptographic signature

Re: RPM Installation status

[Jeff Johnson]

On Feb 18, 2011, at 3:59 PM, Mircea Lemnaru wrote:

> Hi,
> 
> First of all tanks for the quick answer
> 
> You are saying that: "All that RPM has is a non-zero exit code to indicate 
> "failed to install". And the most useful

Exactly is the short answer.

The other longer/harder answer called "RPM ACID" or 
Transactionally Protected Package Management
TPPM is much harder. You can find details at
http://mancoosi.org
look for WP3 which is largely about modeling scriptlet side
effects in a DSL to ensure that there's more than an exit code
that both apt <-> rpm distros can use for scalable/reliable/automated
installs.

Please note its all "research" still. My role (as part of Mancoosi WP3)
is to try to get as much as possible of the "research" into "production"
over the next/last 3-4 months of the Mancoosi project. I'm just starting now ...

> behavior that most users expect is "best effort" which is proceeding with all 
> the
> rest of the installation as far as possible. Manual intervention is always 
> needed
> for "failed to install"." -- this failed to install , non-zero exit code ... 
> is it kept anywhere in the RPMdb ... can I retrieve it after the rpm command 
> finished ?
> 

Persistent scriptlet exit codes are tracked into SRPM headers
(for build scriptlets) and into an rpmdb for install scriptlets
in @rpm5.org iirc. I've forgot what I implemented, its
kinda unnecessary because ...

... knowing the exit code from a package scritlet persistently just reminds you 
of what
you likely already know:
You lose because some package failed to install and you have to clean 
up the mess.

> I mean I have these 2 packages, say pkg1 and pkg2. pkg1 fails to install 
> because a wrong script but pkg2 is just fine. Say I am executing "rpm -i pkg1 
> pkg2". It fails with an error that %post script from pkg1 failed but pkg2 has 
> been succesfully installed. At this point I am looking for a way to know 
> which one / if any of the two packages were successfully installed ...
> 
> rpm -qa | grep pkg1 shows all OK ...  so I cannot determine from that output 
> what I need ...
> 
> Could there be a way for me to parse the output of the rpm command to 
> determine from that that the failure is comming from a script , and also that 
> it only affects one package so I can assume pk2 was successfully installed ?
> 

The very best approach (and what was expected of RPM originally) is
to do the QA carefully and appropriately. RPM makes it really easy
to set up chroot's and test! test! test! your packages.

It's a bit harder than it appears because there's all sorts of
silly failure cases like
A meteor crashed through the CPU while RPM was installing.
but all it takes is some sane judgement calls on what the obvious
failure modes that a typical user will experience.

There's less than 10 types of install/upgrade/erase failures (9 iirc, but I've 
forgotten)
that need to be tested. Once you get those in place, stare at your scriptlets
looking for possible failure cases.

That's basically "RPM package management" in a nutshell. Never been any 
different.

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: RPM Installation status

[Jeff Johnson]

On Feb 18, 2011, at 10:28 AM, Mircea Lemnaru wrote:

> Hi all,
> 
>  
> I having some issues with a project that I am working on, in short , the user 
> installs 2 rpm’s , using the command say: “rpm –I pkg1 pkg2” . The first 
> package contains a post install script that fails , this results in an 
> general error / non zero return code so my wrapper over the rpm command 
> assumes the command failed and the rpms did not get successfully installed, 
> but it turns out that if I query the database after the command ended , the 
> packages show up as installed , no mention of the failed post install script.
> 
> 
> 

What version of RPM? This sounds like its _NOT_ the ACID behavior
that is implemented in rpm-5.3.x, where the package header registered
in an rpmdb is part of a transaction, and will be backed out, not left
in an rpmdb, if/when a package scriptlet fails.

(aside)
Please note that there's 2 other parts of what is known as
Transactionally Protected Package Management
that are _NOT_ turned on yet, and so the files installed are the files installed
by the package with the failing %post script. But the database registration 
SHOULD
be fully functional now.

But otherwise the behavior is as expected.

 package with a failing %post will have a header in an rpmdb, and content
on the file system. All that RPM does get right -- mostly -- is that if a %post 
fails,
then the other half of an "upgrade" aka the erasure of the old files will
be skipped if the install fails for any reasons.

This behavior prevents, say, RPM from accidentally rendering a machine
unbbotable if, say, a glibc upgrade fails for whatever reason. But
its purely "disaster avoidance", not anything else. The behavior
you are reporting is exactly what RPM has been doing for years.
> So I need a way to query the database , or look at the output of the initial 
> RPM command to determine which packages failed, and which actually got 
> installed properly - and by properly I mean a package that got installed AND 
> all the script got executed without any error.
> 
> 

The best answer that RPM has atm is this:
Do proper QA on the packaging to make sure scriptlets don't fail.
Write the scriptlets to take into account every failure mode.

Yes this is hard Q and hard to get prefect scripting.

Meanwhile, there's
rpm -qa --last
that will sort by installation time, and
rpm -qa --dupes
each of which helps find the duplicate partially installed/registered
packages from a failed "best effort" attempt to "upgrade".

> Does anyone know of any way for me do determine if the rpm’s got successfully 
> installed (that includes also the script phase) ? Shouldn’t the rpm database 
> contain also that info ? I mean it seems to me that’s a pretty important 
> information , to know if a post install script executed successfully for a 
> certain installation …
> 

The problem here is that "successfully installed" does not have any semantic 
meaning for
RPM. RPM has only the exit code from a scriptlet, cannot undertake any 
meaningful --rollback
action based purely on an exit code. E.g. package scriptlets can have huge 
side-effects that
are opaquely implemented in (possibly buggy) scriptlets.

All that RPM has is a non-zero exit code to indicate "failed to install". And 
the most useful
behavior that most users expect is "best effort" which is proceeding with all 
the
rest of the installation as far as possible. Manual intervention is always 
needed
for "failed to install".

hth

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: Question about "rpmbuild --root"

[Jeff Johnson]

On Feb 14, 2011, at 9:39 AM, Eric MSP Veith wrote:

> Am Montag, 14. Februar 2011, 14:41:42 schrieb Jeff Johnson:
>>> The latter one I solved by calling "sudo rpmbuild".
>> 
>> So no figger where rpmbuild is supposed to read its configuration from.
>> That's a bit harder than using sudo to get uid == 0.
> 
> I don't understand. :-/ There are hardcoded macro file paths like 
> /usr/lib/rpm/macros:~/.rpmmacros. "~" resolves to ~root or ~myuser based on 
> what variables sudo sets. I don't expect RPM to add some extra magic 
> dowhatIwant(). Or should I?
> 

The necessary PATH's change when chroot(2) is done. All depends on
what is being attempted. Macros are lazily evaluated and many
will be evaluated after rpmbuild starts while the *.spec file is being parsed.

The *.spec itself iscoming from the CLI before chroot(2) is
called, but will be parsed after chroot(2) is called.

>> Depends on context.
>> 
>> Traditionally, RPM implicitly added --root to --dbpath when opening
>> an rpmdb and expected the user to "know" when to add the chroot
>> prefix to the --dbpath argument for non-root execution.
>> 
>> But PLD wanted --rebuilddb --root instead of --rebuildb --dbpath
>> (where one had to manually insert the root prefix in the --dbpath
>> argument).
>> 
>> So --root and --dbpath behavior was changed.
> 
> I don't quite get the implications. But I guess that when I encounter a 
> problem related to this, I'll remember this part and hopefully be able to use 
> this particular information. :-)
> 

SHort answer: --root doesn't behave (in all contexts) correctly with --dbpath
and other configuration. And --root was originally specific to chroot(2)
while --dbpath was a path to an rpmdb. These are semantically different data
values that are now all mixed together.

>>> Ok, off trying... :-)
>> 
>> Good luck!
> 
> TA! I hoped that I could do the config file reading and loading .so files 
> into 
> memory and do the chroot() just before doing the actual call to build(). So 
> everything should already reside in memory and the only thing that needs to 
> be 
> in the chroot is the rpmdb. Well, I'll see.
> 

Add --nodeps to rpmbuild args and don't bother with build dependencies.
Chasing an rpmdb into a chroot(2) (or maintaining 2 copies in sync) is better 
avoided
initially imho.

>> FYI: --root takes a /string which is used to construct a command that looks
>> like rsh -c "chroot /string ..."
>> that isn't what anyone is expecting but is exactly what was intended.
> 
> You mean via %_remchroot, %_remroot & Co? These are never filled here. I 
> already tried to manually add -D '%_remroot /my/chroot' and so on, but that 
> didn't work, too. What am I missing?
> 

Yes those macros.

Not missing anything. Noone has tried/used those macros since forever.
I likely last looked ~2002, but got tired of the endless "Have it your own way!"
discussions.

If I get anywhere with pogoplugs and qemu for ARM builds, I may resurrect,
or (equivalently) link to a libssh.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: Question about "rpmbuild --root"

[Jeff Johnson]

On Feb 14, 2011, at 8:31 AM, Eric MSP Veith wrote:

> Am Montag, 14. Februar 2011, 13:26:32 schrieb Jeff Johnson:
>> Yes but ... rpmbuild --root is not doing chroot(2), never has done
>> chroot(2), and cannot do chroot(2) unless uid == 0.
> 
> The latter one I solved by calling "sudo rpmbuild". 
> 

So no figger where rpmbuild is supposed to read its configuration from.
That's a bit harder than using sudo to get uid == 0.

>> It in the man page because someone insisted:
>>  Its a CLI option so it MUST be in the man page.
>> and it was easier to oblige than explain.
> 
> What's the difference between '--root' and '--dbpath' then?
> 

Depends on context.

Traditionally, RPM implicitly added --root to --dbpath when opening
an rpmdb and expected the user to "know" when to add the chroot
prefix to the --dbpath argument for non-root execution.

But PLD wanted --rebuilddb --root instead of --rebuildb --dbpath
(where one had to manually insert the root prefix in the --dbpath argument).

So --root and --dbpath behavior was changed.

The change has broken 4-5 times over the years and is way too complex
and undocumented and more.

>> File path's get implicitly prefixed when chroot(2) is called.
>> 
>> There's harder issues, like readong configuration (from inside or outside
>> the chroot?) and finding the rpmdb (inside or outside?) and writing *.rpm
>> output (inside or outside), but otherwise yes.
> 
> I can answer these questions in case of what I'm planning to do. I mainly 
> don't want to populate the buildroot with all dependencies of RPM, that's 
> all. 
> But I can live with copying macro files and other config data.
> 
> Ok, off trying... :-)
> 

Good luck!

FYI: --root takes a /string which is used to construct a command that looks like
rsh -c "chroot /string ..."
that isn't what anyone is expecting but is exactly what was intended.

73 de jeff



smime.p7s
Description: S/MIME cryptographic signature

Re: Question about "rpmbuild --root"

[Jeff Johnson]

On Feb 14, 2011, at 3:43 AM, Eric MSP Veith wrote:

> Am Sonntag, 13. Februar 2011, 21:07:45 schrieb Jeff Johnson:
>> Depends. --root (as wired in rpmbuild) isn't doing what you think
>> its doing. What rpmbuild --root is actually attempting is considerably
>> harder to explain, dates back to 2000 or so ...
> 
> Hm, well, the manpage says it'll chroot() before running any scriptlets. But 
> skimming through the sources, I found no call to chroot(2). Hum. Manpage 
> "bug"?
> 

Yes but ... rpmbuild --root is not doing chroot(2), never has done chroot(2),
and cannot do chroot(2) unless uid == 0.

It in the man page because someone insisted:
Its a CLI option so it MUST be in the man page.
and it was easier to oblige than explain.

>>  chroot /wherever rpmbuild ...
>> if you want/need single invocation functionality. But you
>> likely just want to enter the chroot, and use rpmbuild inside.
>>  chroot /wherever
>>  rpmbuild ...
> 
> Which, however, will force me to install RPM with all deps inside the chroot, 
> right? 
> 

Essentially yes.

> And without having tried it I guess that doing it programatically I could 
> call 
> chroot() before buildSpec() and it *should* work? If so, I'm back working on 
> my small little build system... :-)
> 

File path's get implicitly prefixed when chroot(2) is called.

There's harder issues, like readong configuration (from inside or outside the 
chroot?)
and finding the rpmdb (inside or outside?) and writing *.rpm output (inside or 
outside),
but otherwise yes.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: Question about "rpmbuild --root"

[Jeff Johnson]

On Feb 13, 2011, at 2:23 PM, Eric MSP Veith wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Hello list,
> 
> I try to build in a chroot with an old RPM-5.1.9 using 
> 
>   sudo rpmbuild --root=/var/tmp/eglibc-2.13.buildroot -ba eglibc.spec
> 
> Basically, what I want to know, is: Am I doing something completely wrong 
> besides using an old and dusty RPM version? ;-)

Depends. --root (as wired in rpmbuild) isn't doing what you think
its doing. What rpmbuild --root is actually attempting is considerably
harder to explain, dates back to 2000 or so ...


Try
chroot /wherever rpmbuild ...
if you want/need single invocation functionality. But you
likely just want to enter the chroot, and use rpmbuild inside.
chroot /wherever
rpmbuild ...

hth

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature

Re: What am I doing wrong ?

[Jeff Johnson]

On Jan 2, 2011, at 2:12 PM, Sriram Narayanan wrote:

> On Mon, Jan 3, 2011 at 12:22 AM, Jeff Johnson  wrote:
> 
> 
>> What I recommend instead of disabling or changing default behavior is to
>> configure orphan directory/linkto paths instead:
>>mkdir -p /etc/rpm/sysinfo
>>echo "/usr/local/bin" >> /etc/rpm/sysinfo/Dirnames
>> This is what I do; the drudgery isn't _THAT_ painful imho, ymmv.
>> 
> 
> Thanks, this helped.
> 
> I was just wondering, wouldn't we need to do this for every forseeable
> folder structure then ?
> 

Depends on your packaging policy.

The goal(s) with parentdir/linkto dependencies are:

1) stricter/mandatory packaging rules. E.g. even though
rpm WILL create every component in the path in order
to install a file, the "orphandirs" will not be removed
by rpm --erase. There's other issues with metadata like
mode/owner/group (and SELinux tags) with "orphandirs"
because RPM has to guess what is intended for those values.

2) using parentidr dependencies to supply additional ordering hints
Ordering packages is based on a "partial order" from the
dependency assertions. So the ordering is as good/bad/ugly
as the package metadata is (which is pretty bad/ugly in many cases).
By using parentdir depdnencies, additional (and quite reliable
in the case of parentdir dependencies) relations reduce
"partial ordering" indedeterminancy, thereby increasing
install/upgrade/erase ordering reliability.

But while parentdir dependencies are being phased in, there are additional
topological sort relations being introduced, that leads to MORE frustration
dealing with package dependency LOOP's etc etc. Can't be helped imho, I'm
quite sure that parentdir dependencies are more reliable than the random
bad/ugly dependencies being added that are creating dependency LOOP's
all over the place.

(aside)
It's mathematically impossible to order with LOOP's. Yes LOOP's can be removed
by conventional rule's: that's a different issue (imho) than whether additional 
relations
using parentdir dependencies create Yet More LOOP's.

> Assume that some package needs /var/local/lib and we don't have it in
> /etc/rpm/sysinfo/Dirnames. What is the recommended way of dealing with
> such a situation ?
> 

Usually there's a filesystem package carrying a directory skeleton.

Hint: What is annoying is the .../LC_MESSAGES directories from other packages
that create new parentdir dependencies. Easily 50% of the issues with
parentdir dependencies are from parents of new locales with LC_MESSAGES.

One solution used in RHEL6 (which I don't recommend because there's gazillions
of directories in the package) is to add ever possible locale directory.
But "works" w/o question. I'd suggest a separate filesystem-i18n package
instead because filesystem has _THE_ directory structure for a distro,
and the separate filesystem-i18n package is easier to maintain and distribute.

>> There is also (as you proceed, you likely don't have many
>> packages installed) this POPT alias to list all missing parentdirs
>>mkdir -p /etc/rpm/sysinfo
>>rpm -Va --nofiles --orphandirs | sort -u >> /etc/rpm/sysinfo/Dirnames
>> 
> 
> I'm getting the output of the "set" command when I run
> rpm -Va --nofiles --orphandirs
> 
> What's happening here ?
> 

Just I cannot remember/type all possible RPM options any more, sigh.

All that the --orphandirs POPT alias does is this (from memory)
rpm -Va --nofiles | grep '^/'
See /usr/lib/rpm/rpmpopt for the ctual POPT alias.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: What am I doing wrong ?

[Jeff Johnson]

On Jan 2, 2011, at 1:11 PM, Sriram Narayanan wrote:

> All:
> 
> I'm trying to get a basic RPM to build, and am facing the following
> errors during install:
> 
> error: Failed dependencies:
>   /usr/local/bin is needed by bash-4.1-1.i86pc
>   /usr/local/share/info is needed by bash-4.1-1.i86pc
>   /usr/local/share/locale/af/LC_MESSAGES is needed by bash-4.1-1.i86pc
>   /usr/local/share/locale/bg/LC_MESSAGES is needed by bash-4.1-1.i86pc
>   /usr/local/share/locale/ca/LC_MESSAGES is needed by bash-4.1-1.i86pc
> 
> I've attached a test spec file. This contains basic stuff and is
> intended to only check if I'm doing the right things.
> 

You aren't doing anything wrong afaict.

These are "parentdir" dependencies:
Every file implicitly requires its parent directory.

There's also "linkto" dependencies:
Every symlink requires its target end-point.

In both cases a Requires: dependency is implictly synthesized
from other tags, RPMTAG_DIRNAMES and RPMTAG_FILELINKTOS resp.

There are --noparentdirs/--nolinktos disablers, and there are 2 bits
that need to be flipped compiled in to change the default behavior.
There's also a per-vendor peculier macro (iirc) to do same at runtime,
but you will need to add -DRPM_VENDOR_BELLENIX to CFLAGS and change
#ifdef's if you want.

What I recommend instead of disabling or changing default behavior is to
configure orphan directory/linkto paths instead:
mkdir -p /etc/rpm/sysinfo
echo "/usr/local/bin" >> /etc/rpm/sysinfo/Dirnames
This is what I do; the drudgery isn't _THAT_ painful imho, ymmv.

There is also (as you proceed, you likely don't have many
packages installed) this POPT alias to list all missing parentdirs
mkdir -p /etc/rpm/sysinfo
rpm -Va --nofiles --orphandirs | sort -u >> /etc/rpm/sysinfo/Dirnames

hth

73 de Jeff
> -- 
> Belenix: www.belenix.org
> 

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Recommendations on initializing the system rpmdb ?

[Jeff Johnson]

On Jan 2, 2011, at 12:48 AM, Sriram Narayanan wrote:

> Hello:
> 
> 1. What are the recommendations on initializing the rpmdb ?
> 

No initialization per se is needed:

[...@rhel6 wdj]$ rpm -qa --dbpath /tmp/blah
[...@rhel6 wdj]$ ls -al /tmp/blah
total 13720
drwxr-xr-x   2 jbj  jbj  4096 Jan  2 02:45 .
drwxrwxrwt. 19 root root20480 Jan  2 02:45 ..
-rw-r--r--   1 jbj  jbj 24576 Jan  2 02:45 __db.001
-rw-r--r--   1 jbj  jbj  10493952 Jan  2 02:45 __db.002
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.003
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.004
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.005
-rw-r--r--   1 jbj  jbj  41951232 Jan  2 02:45 __db.006
-rw-r--r--   1 jbj  jbj163840 Jan  2 02:45 __db.007
-rw-r--r--   1 jbj  jbj   4554752 Jan  2 02:45 __db.008
-rw-r--r--   1 jbj  jbj 40960 Jan  2 02:45 __db.009
-rw-r--r--   1 jbj  jbj  10485760 Jan  2 02:45 log.01
-rw-r--r--   1 jbj  jbj  8192 Jan  2 02:45 Packages

You WILL need to configure cache/mmap sizes and particularly
the number of locks by adding a DB_CONFIG file.

> 2. I've tried to use /usr/local/lib/rpm/vpkg-provides.sh
> This asks for --spec_header '/path/to/os-base-header.spec'
> 

vpkg-provides.sh is for a installation model where a traditional software vendor
(like in Apple Mac OS X) provides a "native" OS and *.rpm packages are added
on top of the "native" os.

You will need to add a few tags like Name:, Version:, and Release:
(i.e. that's what is in os-base-header.spec if you wish to use
vpkg-provides.sh. The rest of the script should generate a virtual
package with library sonames and a %verifyscript to check library
file digests.

> a. What are the recommended contents of this spec file ?
> 

Essentially just the tags that prevent the vpkg-provides.sh
output spec file from building.

> b. What do other distros do ?
> 

Most distros use RPM for everything, in which case vpkg-provides.sh
is unnecessary. I've done packaging on Mac OS X, using vpkg-provides.sh
to jump start a spec file.

Note that Provides: can be added (1-per-line) to a file. E.g.
mkdir -p /etc/rpm/sysinfo
echo "/bin/sh" >> /etc/rpm/sysinfo/Providename
supplies the essential pre-requsite to run *.rpm scriptlets.

The ability to configure Provides: in a flat-file was re-added because
there were too many details to explain about how to use RPM "packages"
such as what vpkg-provides.sh does.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How do I set the _topdir ?

[Jeff Johnson]

On Jan 1, 2011, at 1:10 PM, Sriram Narayanan wrote:
> 
> Thanks Jeff, that worked !
> 

Instead of the usual {RPMS,SRPMS,BUILD,SOURCES,SPECS}
hierarchy, you might try building like I do.

I tend to have multiple versions of both rpm and distro
packaging lying around always, and need to flip between
versions/distros at a dead-run.

So I work in tree's like /X, /Y, /Z.

I have private configuration as (say) /X/macros:

%_topdir/X
%_ntopdir   %{_topdir}/%{name}
%_builddir  %{_ntopdir}
%_sourcedir %{_ntopdir}
%_specdir   %{_ntopdir}
%_rpmdir%{_ntopdir}
%_srcrpmdir %{_ntopdir}
%_rpmfilename   %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm

%_tmppath   %{_topdir}/tmp


I have a wrapper in ~/bin/xxxrpm to reconfigure:

#!/bin/sh

macros="/usr/lib/rpm/macros:/usr/lib/rpm/%{_target}/macros:/etc/rpm/macros.specspo:/etc/rpm/macros.sqlite:/etc/rpm/macros.cdb:/etc/rpm/macros:/etc/rpm/%{_target}/macros:~/.rpmmacros:/X/macros"

case $1 in
-b*)rpm=/usr/bin/rpmbuild;;
*)  rpm=/usr/bin/rpm;;
esac

exec $vg $rpm --macros ${macros} $*


When I need to use a different version of rpm, or
add valgrind/time/strace/whatever, I edit the ~/bin/xxxrpm wrapper.
Similarly if/when I need to adjust to different macros to
be loaded, I edit the macros path.

In each tree, packages are installed/built into /X/%{name}.

So my packaging workflow goes something like:

xxxrpm -i foo*.src.rpm
cd /X/foo
xxxrpm -ba foo.spec
rpm -Uvh *.i386.rpm
cd ..
rm -rf foo

I find having everything in one directory far more
convenient than having to chdir all over the place
just to build a package.

(aside)
FYI: I'm very likely to change to the above as default
hierarchy in rpm-5.4.x, largely because @rpm.org changes with
~/rpmbuild/* already have changed RPM's default behavior,
and I see no reason to maintain consistency with, say,
RPM documentation going "forward". I have in fact been
building as above for years but haven't changed the RPM default
because of a "Principle of Least Surprise" and doco consistency
that simply does not matter any more.

YMMV.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How do I set the _topdir ?

[Jeff Johnson]

On Jan 1, 2011, at 12:46 PM, Sriram Narayanan wrote:


> /usr/local/lib/rpm/%{_target}/rpmpopt

Note the unexpanded macro here. That's "Have it your own way!"
flexibility as compiled.



> 
>  0   2237 open64:entry rpmbuild /home/sriram/.rpmmacros
> 

This SHOULD have read wyat is in ~/.rpmmacros.

You can verify with
rpm --showrc | grep _topdir

> If I want to set /workspace/rpmwork as the topdir (which would contain
> SPECS, SOURCES, RPMS, etc), what should the contents of .rpmmacros be
> ?
> 
> I've tried the following:
> topdir=/workspace/rpmwork
> _topdir=/workspace/rpmwork
> topdir:/workspace/rpmwork
> _topdir:/workspace/rpmwork
> 

All macros in configuration file loads have to start with '%' in
1st column.

So you need this

%_topdir/workspace/rpmwork

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How do I set the _topdir ?

[Jeff Johnson]

On Jan 1, 2011, at 11:11 AM, Sriram Narayanan wrote:

> Hi:
> 
> This is a newbie question. I presently have ~/.rpmrc containing
> _topdir:/workspace
> 

You want ~/.rpmmacros.

Check with
strace -e open rpmbuild ...
if you run into issues figgering out what paths are open'ed.

rpmbuild is _WAY_ too "Have it your own way!" flexible imho.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Wondering status of rpm5 in Mandriva?

[Jeff Johnson]

On Dec 17, 2010, at 10:29 AM, Dwight Paige wrote:

> On 12/16/2010 11:28 PM, Jeff Johnson wrote:
> 
>> There's a few issues that have been fixed this week:
>>  /etc/passwd (and %config handling) was screwed
>>  parentdir dependencies used for ordering were screwed
>>  file triggers and/or %posttrans wasn't functional w --root
>> Reported by IDMS but confirmed in Mandriva.
>> 
>> The recent segfaults I've heard (but not personally seen) in Mandriva were
>> with DistEpoch: which is a bit tricky to deploy but since
>> every package will use, issues are easy to see and fix.
>> 
>> Per Oyvind knows details better than I. I just sweep up the bugs ;-)
>> 
>> hth
>> 
>> 73 de Jeff
>> 
>> __
>> RPM Package Managerhttp://rpm5.org
>> User Communication List rpm-users@rpm5.org
>> 
> Thanks. As I understand we are basically awaiting perl-URPM 4.3. Per Oyvind 
> has reported difficulties getting it through build system. Most recent is:
> 
> perl-URPM-4.2-1mdv2011.0.x86_64.rpm 03-Dec-2010
> 
> and there doesn't appear to be anything here:
> 
> http://kenobi.mandriva.com/queue/queue/cooker/main/testing/
> 
> So I'll just wait and see what happens.

Careful: Some of the rpm-5.3.6+ changes -- particularly %config handling -- WILL
affect your Mandriva upgrades. All I'm saying is there is a rpm-5.3.7 package
from the distribution mirrors that is needed in addition to perl-URPM-4.3.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Wondering status of rpm5 in Mandriva?

[Jeff Johnson]

On Dec 16, 2010, at 9:14 PM, Dwight Paige wrote:

> I'm wondering what's going on with rpm5 in Mandriva after the recent server 
> issues? Or are there still issues? Maybe people are busy catching Up? And it 
> isn't just rpm5. Silence is deafening at Cooker ML.
> 

There's a few issues that have been fixed this week:
/etc/passwd (and %config handling) was screwed
parentdir dependencies used for ordering were screwed
file triggers and/or %posttrans wasn't functional w --root
Reported by IDMS but confirmed in Mandriva.

The recent segfaults I've heard (but not personally seen) in Mandriva were
with DistEpoch: which is a bit tricky to deploy but since
every package will use, issues are easy to see and fix.

Per Oyvind knows details better than I. I just sweep up the bugs ;-)

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: replacing a rpm which has a preventing measure in its preun section

[Jeff Johnson]

On Nov 13, 2010, at 3:45 AM, Oguz Yilmaz wrote:

> Hi,
> 
> I have a rpm named deployment which has a preun section like:
> 
> preuninstall scriptlet (using /bin/sh):
> echo "You should not uninstall this package"
> exit 1;
> 
> I have another rpm named "installrpm" which should replace "deployment"
> 
> A- I have added 
> 
> Provides: deployment
> Obsoletes: deployment
> 
> linesi n "installrpm" spec file.
> 
> However, because of the preun lines, installrpm can not replace deployment.
>  

Yes, that is the expected behavior.

(aside)
Technically, the "installrpm" package _HAS_ replaced the "deployment"
package at least for the common subset of files in each package.

What hasn't been done is to unregister the "deployment" package
and remove the files unique ti the "deployment" package.


> 
> B- I have tried to remove deplyment by hand in the post section of 
> "installrpm". 
> 
> %post
> rpm -q deployment 2>&1 > /dev/null
> if [ $? -eq 0 ]; then
> rpm -ev deployment --nodeps --noscripts
> fi
> 
> However this time, rpm db lock can not be obtained by inner "rpm -ev 
> deployment --nodeps --noscripts" command.
> 

Running rpm in %post is one approach to trying to automate an upgrade.

Whether the approach will "work" I cannot say.

But you can avoid the deadlock by renaming the file that
has the fcntl lock before running rpm in %post, and then renaming
the file back after running rpm.

E.g. (assuming the transaction lock is /var/lib/rpm/__db.000,
LOCK envvar used solely for clarity)

%post
LOCK=/var/lib/rpm/__db.000
rpm -q deployment 2>&1 > /dev/null
if [ $? -eq 0 ]; then
mv ${LOCK} ${LOCK}-SAVE
rpm -ev deployment --nodeps --noscripts
mv ${LOCK}-SAVE ${LOCK}
fi

> I have to put "installrpm" into a repository. So it should be working without 
> manual intervention.
> 
> Do you have any recommendation?
> 

You can also add --nopreun (implied by --noscripts) and manually erase the 
package.

There is not automated way to erase packages with failing scripts
except by not running the script or by ignoring the exit code (but
that's a change needed in rpm itself).

Write the %preun script differently next time, and don't
prevent the packe from being removed.

The resaon fro preventing removal isn't clear; but changing the
packaging can likely add a dependency that effectively
stops erasing --nodeps is used.

But without knowing details of why %preun was written
to prevent the :deployment" package from being erased, I can only guess.

hth

73 de jeff

> Regards,
> 
> --
> Oguz YILMAZ

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: trigger ordering

[Jeff Johnson]

On Nov 10, 2010, at 10:58 PM, Bela Lubkin wrote:

> [ This was originally composed as mail directly to Jeff Johnson, then I
> came to my senses and decided a mailing list is more appropriate.  So
> "you" that I am speaking to is Jeff or suitable proxy ;-} ]
> 
> I have a question and I'm 99.5% sure I have the right answer.  I'm
> writing you because although I have the right answer, I feel the RPM
> docs are quite unclear about this and should be fixed.  [ Below I find
> the necessary doc; however *this* doc is still unclear and should be
> improved... ]
> 
> Refer to the discussion <http://rpm5.org/community/rpm-users/0416.html>
> and the referenced doc <http://rpm5.org/docs/api/triggers.html>.  This
> section is what is unclear to me:
> 
>  any-%triggerin (%triggerin from other packages set off by new install)
>  new-%triggerin
>  old-%triggerun
>  any-%triggerun (%triggerun from other packages set off by old uninstall)
> 

(aside)
Hehe, ya gotta remember that I wrote this document on May 2nd, 1998,
my 2nd day working at Red Hat. Alan Cox's 2nd day too ... but I digress ...

Basically what I attempted to capture with "new-%triggerin" and "old-%triggerun"
is the concept of "this package" as opposed to "every pother package".

There's the other complexity, of "new" vs "old" instances of "this package"
there too.

> That is: the first and last lines are adequately clear, and I don't have
> any real issues with "old-%triggerun", but "new-%triggerin" could be
> taken in at least two radically different ways (maybe more, but these
> are the ones that get me).  I'm sure this seems perfectly clear to you,
> who have been steeped in the sequencing of RPM installs for who knows
> how many years, but for those of us who just occasionally use this
> stuff, it's a bit boggling.
> 
> The two interpretations are:
> 
>  new-%triggerin might mean:
> 
> 1. "fire any %triggerin scripts of new' which pertain to packages
> BEING INSTALLED RIGHT NOW BY THE SAME  COMMAND INVOCATION"
> 
> Like I said, I'm 99.5% certain that I know the right answer (1), but I'm
> still worried about the 0.5% chance it's (2).  And more to the point, I
> definitely understand why other people might have the same doubt, but
> not have the grounding to (fairly) firmly believe it's (1).
> 

This is the sense: a trigger from the incoming "new" package is fired at
that point in time.

> Please change the doc to read:
> 
>  any-%triggerin (%triggerin from other packages set off by new install)
>  new-%triggerin (set off by packages already installed + being installed now)

At least to my dialect of English, that's the wrong way to express.

The actual implemented semantics are most easily (imho) read directly
from the code (typically in lib/psm.c).

At the point triggers are fired "this" (aka the "new" package) is examined
for a %triggerin script. If it exists, the trigger (i.e. the %triggerin) script
is run.

What doesn't fit into what I tried to describe (and the code is quite clear 
if/when you
find it), is only "this" (aka the "new" header) is queried for a %triggerin 
script which
(if found) is run. The problem I have with your suggested wording change is the 
"already installed"
portion. The "already installed" is (at least in what I tried to write) already 
covered by the set implied
by
any-triggerin
as in the complement of "this" package, aka "already installed" packages.

I hope that is clear.


>  old-%triggerun
>  any-%triggerun (%triggerun from other packages set off by old uninstall)
> 
> Actually, wait a sec.  Why would old-%triggerun be fired at all?  I
> suppose it *could* list a trigger for *itself* being uninstalled, but if
> so, I would consider that merely a case of "any-%triggerun".  Likewise,
> if multiple packages are being removed at once, this package's
> "old-%triggerun" might fire in response to some other package also
> exiting the building, but again, that's an instance of "any-%triggerun".
> So I *think* that line of the doc is just wrong.

Again the concepts are quite difficult to describe in words. Examine
lib/psm.c (PSM == Package State Machine, simple linear state machine of


%init -> %pre -> %process -> %post -> %fini
with the added "new" vs. "old" (and "install" vs. "erase") steps merged.

> 
> It also begs a further clarification.  Suppose that packages "foo" and
> "bar" are both already installed on the system, and foo has:
> 
&

Re: Is there spec file syntax to make (rpm -e) ignore errors and continue uninstall?

[Jeff Johnson]

On Nov 11, 2010, at 11:47 AM, Joe Flowers wrote:

> Hello Everyone,
> 
> I have created a spec file and an RPM file from it but I bungled a command 
> which uncovered an interesting problem.
> 
> 
> In the %preun section of the spec file, I mistakenly put
> 
> /etc/ini.d/mydaemon/stop
> 
> instead of
> 
> /etc/ini.d/mydaemon stop
> 
> 
> When I went to uninstall the RPM, like
> 
> rpm -e mydaemon-1.00-1
> 
> It failed on the bad (/etc/ini.d/mydaemon/stop) line and would not complete 
> the uninstall.
> 
> So, I'm wondering if there is something that I can add to the bad
> /etc/ini.d/mydaemon/stop line in my spec file that will allow "rpm -e ..." to 
> complete the uninstall.
> 

There's no automated insturmented solution in *.rpm that Just Works in
spite of packaging flaws ... there are engineering flaws with
permitting random packages to start turning what RPM does on and off,
consider malware e.g.

But there is a manual disabler, in this case --nopreun, that
can be added to the command line (or set by depsolvers if/when
they ever choose to deal with the "real world" of packaging flaws),
that will skip running a buggy scriptlet when erasing.

The better fix is to test your packages more carefully ...
... but you are older & wiser now ;-)

> 
> Analogously, I know that you can add a "-" in front of commands in a Makefile 
> so that make will not fatally exit. I'm looking for the same functionality in 
> the spec file.
> 

The analogue in scriptlet's of the '-' in Makefiles is

some command || :

which ignores the exit code of a command that might fail.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Adding rpm software to platform - cpio: MD5 sum mismatch when installing rpm's

[Jeff Johnson]

On Nov 9, 2010, at 1:57 PM, Andy Furtado wrote:

> Hello, 
> 
> I'm adding the RPM software to our linux platform. It builds and almost 
> appears to work but I'm seeing the following.
> 
> I'm having a problem with installing rpm's. More details are below but the 
> most recent thing I've discovered is the call to NSS_NoDB_Init fails with 
> -5925 during RPMBUILD and RPM installs, so I'm wondering if this could be my 
> problem. 
> 

Hmmm ... what version of RPM? All I can tell is _NOT_ rpm-5.3.6,
which is opening the NSS database to test the FIP 140-2 bit.

(i.e. rpm-5.3.4 (iirc) and later is _NOT_ calling NSS_NoDB_Init() but rather 
NSS_Init() ...)

> I've also tried installing a fairly simple rpm ...
> rpm -ivv --nodeps nano-2.0.6-4.fc9.i386.rpm
> But have the same checksum problem. 
> 

Checksim problems ... hmmm ...

Is the RPM package just re-bundling already compiled libraries?

If do, you need to make sure the libraries are _NOT_ prelinked
when re-bundling.


All you really need to do is run
prelink --undo
on any library (or copy) that you choose to rebundle.

> Any help, or info on other resources would be greatly appreciated. 
> 
> Here are more details...
> I'm having a problem installing an RPM that I created that only contains 
> %files section to be installled on a i386 platform. 
> The rpm database is currently empty so this is the first thing going in.
> 
> The problem is ...
>   $ rpm -i /rpm/RPMS/i386/axis-4.0-1.i386.rpm 
> error: unpacking of archive failed on file 
> /device_handlers/axis/axis.so;4cd1a2a6: cpio: MD5 sum mismatch
> 

In addition to the (quite subtle) prelinking issue I just mentiopned,
ther's also the switch from MD5 -> SHA256 in Fedora (and RHEL6) that
has legacy compatibility issues.

> What is actually being checksum'd during an install?
> 


RPM does a checksum on file content while installing, that's the failure you 
are seeing.

Adding --nofidigest (the oleder name is --nomd5sums) to disable the content 
digest
can be done: no guarantees whatsoever what happens if you disable with 
--nofdigests.

> The target platform is based on FC10 but not a complete distribution.

FC10 is _BEFORE the switch from MD5 to SHA256.

> I'm adding RPM to our platform, and the rpm source from 
> rpm-4.6.1-3.fc10.src.rpm is built in an isolated build area using chroot. 

FYI: You're on the wrong list for rpm-4.6.1 issues.

> I'm also running rpm and rpmbuild from the same isolated build environment. 
> My spec file is included below.
> 
> I'd like to perform an rpm install in this same isolated area, but having the 
> problem above. 
> I believe I've satisfied all of the dependencies required, and manually 
> verifying the md5sums have proven successful. 
> 
> My searches on the internet have found instances about incompatible libs 
> needed for the packaged file but I don't think that is the case since I'm 
> building everything against the same set of libs. I've also tried the prelink 
> suggestions that I've found. 
> I feel like maybe I'm missing something (executable) that rpm needs for the 
> checksum.
> 
> RPM VERIFY ...
> $ rpm -Kvv /rpm/RPMS/i386/axis-4.0-1.i386.rpm
> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
> D: loading keyring from rpmdb
> D: opening  db environment /var/lib/rpm/Packages cdb:mpool:joinenv
> D: opening  db index   /var/lib/rpm/Packages rdonly mode=0x0
> D: locked   db index   /var/lib/rpm/Packages
> D: opening  db index   /var/lib/rpm/Name rdonly mode=0x0
> D: Expected size:   458122 = lead(96)+sigs(120)+pad(0)+data(457906)
> D:   Actual size:   458122
> /rpm/RPMS/i386/axis-4.0-1.i386.rpm:
> MD5 digest: NOKEY
> D: closed   db index   /var/lib/rpm/Name
> D: closed   db index   /var/lib/rpm/Packages
> D: closed   db environment /var/lib/rpm/Packages
> 
> Any information would be greatly appreciated.
> 
> Best Regards, 
> Andy
> 
> //THE SPEC FILE
> Summary: Device Handler
> Name: axis
> Version: 4.0
> Release: 1
> License: GPL
> BuildRoot: /device_handlers/%{name}
> BuildArch: i386
> 
> %define buildroot_dest %buildroot/device_handlers/%name
> 
> %prep
> 
> %description
> RPM to install handler.
> 
> %install
> mkdir -p %buildroot_dest
> cp -rv %_builddir/%name/* %buildroot_dest
> 
> %files
> %defattr(-,root,root)
> /device_handlers/%{name}/axis.so
> /device_handlers/%{name}/lib/libdhimpl.so.1
> 

These are binary libraries. Make sure that prelinking is removed by running
prelink --undo
on these images. There's details in "man prelink", and you might
want to just blacklist (i.e. disable prelinking on those specific libraries)
globally (its in /etc/prelink.conf, I fergit, see the man page).

hth

73 de Jeff
> 
> 

Re: Proposal: use rpm5 package format and the smart package manager

[Jeff Johnson]

On Nov 9, 2010, at 1:09 AM, pinto.e...@gmail.com wrote:

> Jbj, which problem for popt ? Last time have i checked the new autofu of popt 
> HEAD work fine on opensolaris. 

If I change the %indiana stanza to use
--with-popt=internal
this is the error that I see:

gcc -std=gnu99 -shared -Wl,-h -Wl,libpopt.so.0 -o .libs/libpopt.so.0.0.0  
.libs/popt.o .libs/poptparse.o .libs/poptconfig.o .libs/popthelp.o 
.libs/poptint.o  -lc 
ld: fatal: relocation error: R_386_GOTOFF: file .libs/popt.o: symbol 
_poptArgMask: a GOT relative relocation must reference a local symbol
ld: fatal: relocation error: R_386_GOTOFF: file .libs/popt.o: symbol 
_poptArgMask: a GOT relative relocation must reference a local symbol
ld: fatal: relocation error: R_386_GOTOFF: file .libs/popt.o: symbol 
_poptArgMask: a GOT relative relocation must reference a local symbol

Feel free to check the change in on -r rpm-5_3 and debug through the
eastham:8010 buildbot if you want.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Proposal: use rpm5 package format and the smart package manager

[Jeff Johnson]

On Nov 9, 2010, at 1:09 AM, pinto.e...@gmail.com wrote:

> Jbj, which problem for popt ? Last time have i checked the new autofu of popt 
> HEAD work fine on opensolaris. 

Its not a popt problem, but rather a linker problem.

The root cause might be -fpie ... but the fix needs to be in the
linker imho. The -fpie is easily disabled when necessary already.

73 de Jeff
> -Original Message-
> From: Jeff Johnson
> Sent:  08/11/2010, 16:03 
> To: rpm-users@rpm5.org
> Subject: Re: Proposal: use rpm5 package format and the smart package manager
> 
> 
> 
> On Nov 5, 2010, at 3:30 PM, Sriram Narayanan wrote:
> 
>> On Sat, Nov 6, 2010 at 12:45 AM, Jeff Johnson  wrote:
>>>> We're ready to move forward :)
>>> 
>>> OK. I'll start-up the OpenIndiana VM buildbot again. Note that
>>> the issue with running the OpenIndiana VM is largely
>>> that its running on a devel box here at home, and
>>> I'm a bit short on memory and cycles to run the number of VM's
>>> that I'm attempting.
>>> 
>> 
>> Ok, I'll see if we can set up a buildbot at one of our user's
>> computers, then. All Belenix work happens on our home desktop
>> computers :)
>> 
> 
> I sorted through the CVS issues on the OpenIndiana VM I was seeing before.
> 
> Reverting from cvs-1.12.x to cvs-1.11.23 (@rpm5.org uses cvs-1.11.22) fixed
> the issue for me.
> 
> OpenIndiana now has a "green waterfall" and all build logs attached at
>   http://eastham.rpm5.org:8010
> The tests are still a bit flimsy, yes, but there's likely coverage at ~60%
> of all RPM code, and closer to ~80% on the primary build/install/query/erase
> paths. I can regenerate LCOV/GCOV logs for rpm-5.3.6 if really interested;
> last I looked (fair number of changes since rpm-5.3.1) is available here
>   http://eastham.rpm5.org/files/rpm/rpm-5.3/lcov-5.3.1/
> 
> (aside)
> There are a few issues with linker relocations seen while building
> popt, but you are almost certain to encounter those same issues,
> I had no problem finding other instances of the same problem(s)
> searching with Google.
> 
> I still need to build/install/link elfutils for RPM, but most everything
> else Just Works afaict. I've already typed
>   make install
> and I could easily use rpm-5.3.6 to build packages. Again
> getting ELF internal to RPM does a better job than external
> helpers for library sonames. And its not too hard, just I
> haven't yet.
> 
> If you want to try a buildbot, there's a "INDIANA_rpm_release"
> slot that you could plug into on
>   http://buildbot.rpm5.org:8010
> anytime you want (I'll send a password privately if/when you
> E-mail troncato a 2,000 caratteri
> :::0:b3f4e920240d0929606abe71b3114050:7d0
> 
> Il messaggio originale si trova sul server
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How not to use rpm owner/group info on unpack?

[Jeff Johnson]

On Nov 8, 2010, at 8:49 PM, Marc MERLIN wrote:
> 
> What is the proper way to unpack an rpm without losing the user/group info 
> (i.e. without installing it and without relying on an rpmdb, just unpacking
> its files with proper perms).
> 

The likely flaw you've experienced is in cpio,
which for POSIX cpio reasons, chooses to change permissions
to 700 on directories when run as root.

rpm2cpio is most definitely doing nothing other than
seeking to the beginning of the payload, and uncompressing
everything to EOF. The result is a cpio ball written to stdout.

There is no way to "unpack" without "installing" in rpm. The
two operations are largely identical for a "package manager"

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How not to use rpm owner/group info on unpack?

[Jeff Johnson]

On Nov 8, 2010, at 11:08 PM, Marc MERLIN wrote:

> On Mon, Nov 08, 2010 at 09:12:00PM -0500, Jeff Johnson wrote:
>> 
>> On Nov 8, 2010, at 8:49 PM, Marc MERLIN wrote:
>> 
>>> I was debugging an rpm problem and pulled my hair on this:
>>> 
>>> As root, both:
>>> rpm2cpio file.rpm | cpio -idv
>>> and
>>> alien --to-tgz file.rpm ; tar xvzf file.rpm
>>> 
>>> unpack the package but lose the owner/group info for files.
>> 
>> Ick.
>> 
>> Um, and this is an rpm problem ... how? You're not even invoking rpm.
> 
> It's a problem if the package cannot be opened along with proper perms with
> any standard tool, including rpm2cpio which is part of rpm, is it not? :)
> (as far as I can tell, it's rpm2cpio that is losing the file owner info, not
> cpio, so that made it an rpm problem for me).
> 

rpm2cpio.c was one (of several) programs written
to illustrate how to program against an rpm-2.x API
that went obsolete in rpm-3.0. In 1999.

There's a rpm2cpio.sh, and a rpm2cpio.pl and likely every
other widdle language.

No rpm2cpio splits the cpio payload out of a package. Period.

> In other words, I haven't found a single way to verify ownership information
> of files inside an rpm package without actually installing the rpm.
> 

You want to verify what?

If you want to verify what is installed against what is in a *.rpm:

rpm -Vp foo*.rpm

If you want to verify what cpio -itv would display against package metadata,
it is very close (identical at one point, but cpio output changes too), then

rpm -qlvp foo*.rpm


> What if you need to validate an rpm on a server which certainly should not
> install said rpm before it's pushed to a bunch of machines?
> 

What is validate? There's file MD5 sums, there's signatures/digest/crc's on 
*.rpm,
there's installs into a chroot, or on a test machine, or any number of other 
meanings
for "validate".

Does tar/cpio/alien "validate" an archive on a server?

> I thought there would even be a way to do some rpm -qlp foo.rpm to list
> files and their rpms (à la ls -l), but didn't find a way to do that.
> 

There is, add -v.

> I'll look at the solution you posted, it looks pretty involved :)
> 

A script that containing chown/chgrp to set user/group on uglix, exactly what 
you asked for,
is "involved"?

Generated from a package without using an rpmdb? Or using alien or tar or 
anything but
a single rpm command?

What are saying _REALLY_?

73 de Jeff__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How not to use rpm owner/group info on unpack?

[Jeff Johnson]

On Nov 8, 2010, at 9:12 PM, Jeff Johnson wrote:

>> 
> 
> See the popt aliases that queries user/group/mode, pipes to awk,

Correction #1: s/pipes to awk/writes shell commands/. It's --last
that uses awk.

> and write the commands that do what you want:
>   rpm -p --setugids foo*.rpm
>   rpm -p --setperms foo*.rpm
> in spite of what cpio/alien/tar choose to do.
> 
> When you discover that you need "./" prefixed to

Correction #2: no specific need to prefix if you chdir ...

> that path in the --queryformat, then look at the popt alias
> (usually in /usr/lib/rpm/rpmpopt):
> 
> rpm alias --setugids -q --qf \
>'[ch %{FILEUSERNAME:shescape} %{FILEGROUPNAME:shescape} 
> %{FILENAMES:shes
> cape}\n]' \

.^ but added 
prefix goes here.

>--pipe "(echo 'ch() { chown -h -- \"$1\" \"$3\";chgrp -h -- \"$2\" 
> \"$3\
> "; }';grep -v \(none\))|sh" \
>--POPTdesc=$"set user/group ownership of files in a package"
> 

You can extend RPM queries in any number of ways

-- including invoking rpm2cpio/cpio/tar/alien/whatever --

if you think a bit. See "man 3 popt".

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: How not to use rpm owner/group info on unpack?

[Jeff Johnson]

On Nov 8, 2010, at 8:49 PM, Marc MERLIN wrote:

> I was debugging an rpm problem and pulled my hair on this:
> 
> As root, both:
> rpm2cpio file.rpm | cpio -idv
> and
> alien --to-tgz file.rpm ; tar xvzf file.rpm
> 
> unpack the package but lose the owner/group info for files.
> 

Ick.

Um, and this is an rpm problem ... how? You're not even invoking rpm.

> Yet, installing the rpm on a system does use the right user/group info.
> 
> Argh, lost half a day on this :(
> 
> What is the proper way to unpack an rpm without losing the user/group info 
> (i.e. without installing it and without relying on an rpmdb, just unpacking
> its files with proper perms).
> 

See the popt aliases that queries user/group/mode, pipes to awk,
and write the commands that do what you want:
rpm -p --setugids foo*.rpm
rpm -p --setperms foo*.rpm
in spite of what cpio/alien/tar choose to do.

When you discover that you need "./" prefixed to
that path in the --queryformat, then look at the popt alias
(usually in /usr/lib/rpm/rpmpopt):

rpm alias --setugids -q --qf \
'[ch %{FILEUSERNAME:shescape} %{FILEGROUPNAME:shescape} %{FILENAMES:shes
cape}\n]' \
--pipe "(echo 'ch() { chown -h -- \"$1\" \"$3\";chgrp -h -- \"$2\" \"$3\
"; }';grep -v \(none\))|sh" \
--POPTdesc=$"set user/group ownership of files in a package"

and figger out where to add the "./" prefix.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Authors and/or Developers tags in .spec file?

[Jeff Johnson]

On Nov 8, 2010, at 9:31 AM, Joe Flowers wrote:

> Hello Everyone,
> 
> I notice in the Software Management tool in Yast on SUSE (SLES) that
> there is spot for "Authors" that is blank and I wanted to fill it out.
> But, apparently there is no "Authors" tag that I can put in the .spec file.
> 

IIRC, the "Authors" info is derived from conventionally formatted
text in %description ... run rpm -qp --xml against some package
and see if that's what Yast and SuSE are doing.

> I have seen in SUSE SLED or OpenSUSE on the gimp package (for example)
> that there is an Authors spot in Software Management that says
> "Packaged by" and "Developed by" and gives a list for each, packagers
> and developers.
> 

There are tags for Packager: and Vendor: that were intended for
what you want. The tags have hardly been used because it's
a huge pain to fill in those details per-recipe in automated
build farm's.

> Any ideas how I should accomplish this in my .spec file?
> 

There's also a means to use macros to set packager/vendor,
so don't try in the *.spec if interested in seeing YOUR name
in all YOUR software. Its easier to configure the build system
than the *.spec as soon as you have more than one package,
or one release, of whatever software you want to add yourself.

It also would not be too complicated to attach external data
in a file to each package in an automated build system if that
is your goal. Again through macro configuration, not in-line
in the *.spec recipe is the better approach imho.

I can dig out details for any version of rpm whenever. Just
I have the opposite problem -- more than 1 message/minute of SPAM --
because my e-mail address was in a lot of packages. SO think
carefully about what you _REALLY_ want to do ...

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Proposal: use rpm5 package format and the smart package manager

[Jeff Johnson]

On Nov 5, 2010, at 3:30 PM, Sriram Narayanan wrote:

> On Sat, Nov 6, 2010 at 12:45 AM, Jeff Johnson  wrote:
>>> We're ready to move forward :)
>> 
>> OK. I'll start-up the OpenIndiana VM buildbot again. Note that
>> the issue with running the OpenIndiana VM is largely
>> that its running on a devel box here at home, and
>> I'm a bit short on memory and cycles to run the number of VM's
>> that I'm attempting.
>> 
> 
> Ok, I'll see if we can set up a buildbot at one of our user's
> computers, then. All Belenix work happens on our home desktop
> computers :)
> 

I sorted through the CVS issues on the OpenIndiana VM I was seeing before.

Reverting from cvs-1.12.x to cvs-1.11.23 (@rpm5.org uses cvs-1.11.22) fixed
the issue for me.

OpenIndiana now has a "green waterfall" and all build logs attached at
http://eastham.rpm5.org:8010
The tests are still a bit flimsy, yes, but there's likely coverage at ~60%
of all RPM code, and closer to ~80% on the primary build/install/query/erase
paths. I can regenerate LCOV/GCOV logs for rpm-5.3.6 if really interested;
last I looked (fair number of changes since rpm-5.3.1) is available here
http://eastham.rpm5.org/files/rpm/rpm-5.3/lcov-5.3.1/

(aside)
There are a few issues with linker relocations seen while building
popt, but you are almost certain to encounter those same issues,
I had no problem finding other instances of the same problem(s)
searching with Google.

I still need to build/install/link elfutils for RPM, but most everything
else Just Works afaict. I've already typed
make install
and I could easily use rpm-5.3.6 to build packages. Again
getting ELF internal to RPM does a better job than external
helpers for library sonames. And its not too hard, just I
haven't yet.

If you want to try a buildbot, there's a "INDIANA_rpm_release"
slot that you could plug into on
http://buildbot.rpm5.org:8010
anytime you want (I'll send a password privately if/when you are ready).

I'd like to see a SPARC buildbot somehow: the SPARC architecture
exposes certain bugs (like alignment) faster/better than
any other architecture.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Does %files support -foo ?

[Jeff Johnson]

(replied privately already, duped here too)

On Nov 5, 2010, at 5:37 PM, Marc MERLIN wrote:

> I couldn't find this in the docs.
> 
> If I have 
> %files
> /*
> 
> Can I do something like:
> %files
>   
> /*
> %dir 
> -/etc/init.d
> 
> What I'm trying to do is to package all the file and directories, including 
> /etc/init.d/foo
> but I do not want the RPM to contain /etc/init.d since the target system has
> /etc/init.d as a symlink and trying to create a directory on top of it would
> be bad.
> 
> Is that possible without listing all the directories separately and not
> using
> %files
> /* 
> anymore?
> 

There is a %exclude option that does "everything but this ..." filtering.

Caveat:
   %exclude has some minor issues in various versions of RPM.

In my experience, its just as simple to switch to a manifest using
%files -f manifest

Generate the manifest at end of %install, and filter paths however you
wish with sed/grep/cut/awk/whatever.

hth

73 de Jeff
> Thanks,
> Marc
> -- 
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems & security 
>   what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/  
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Proposal: use rpm5 package format and the smart package manager

[Jeff Johnson]

On Nov 5, 2010, at 3:30 PM, Sriram Narayanan wrote:

> 
> Ok, I'll test with these too. And I'll make sure I keep you updated as
> we go ahead.


If you want to be wired up for CVS write access @rpm5.org, send
an ssh pubkey to .

Otherwise just drop patches on  and
I'll try to get them integrated.

There should not be too much needed to build on OpenIndiana.

Getting attached to some -lelf is likely the most critical
path issue. I'd recommend using elfutils, but I can/will dig
out the same functionality using the OpenIndiana -lelf along
the way.

The -lelf issue is largely a few useful methods that are
extensions in elfutils, bit not present in the Sun derived ELF
library.

Otherwise its just ELF == ELF and RPM uses the "standard"
gelf* API.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Proposal: use rpm5 package format and the smart package manager

[Jeff Johnson]

On Nov 5, 2010, at 2:50 PM, Sriram Narayanan wrote:

> On Fri, Nov 5, 2010 at 11:56 PM, Jeff Johnson  wrote:
>> 
>> On Nov 5, 2010, at 2:05 PM, Sriram Narayanan wrote:
>> 
>>> Folks:
>>> 
>>> Here's the latest on Belenix using rpm5 :)
>>> 
>> 
>> Nice!
>> 
>> Note that I can/will re-enable the OpenIndiana buildbot on
>>http://eastham.rpm5.org:8010
>> or (this is the "production" @rpm5.org buildbot under construction)
>>http://buildbot.rpm5.org:8010
>> ifever/whenever you are ready to move forward.
> 
> We're ready to move forward :)

OK. I'll start-up the OpenIndiana VM buildbot again. Note that
the issue with running the OpenIndiana VM is largely
that its running on a devel box here at home, and
I'm a bit short on memory and cycles to run the number of VM's
that I'm attempting.

> 
> Does the latest build now, Jeff ? I had faced some errors when
> building trunk, I'll discuss that on -dev.
> 

(from private e-mail) I got all the changes that were needed (for me)
to build rpm-5.3.5 on OpenIndiana checked in last month. And
make check
was likely mostly functional (or I'd remember the flaws. I fergit ...).

The more complex issue is setting up a Bellenix (not an OpenIndiana)
build system.

I'm not sure what tool chain components to use, will adapt to your
packaging as forward progress becomes visible.

For rpm-5.3.6 release this month, I'm using
autoconf-2.68
automake-1.11.1
libtool-2.4
gettext-0.18.1
as well as (externally installed)
db-5.1.19
pcre-8.10
and (internal to RPM)
beecrypt-4.1.2
neon-0.29.5
popt-1.17
There's lua & syck internal to RPM same as always too.
which more or less is "working" on all platforms afaict.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Proposal: use rpm5 package format and the smart package manager

[Jeff Johnson]

On Nov 5, 2010, at 2:05 PM, Sriram Narayanan wrote:

> Folks:
> 
> Here's the latest on Belenix using rpm5 :)
> 

Nice!

Note that I can/will re-enable the OpenIndiana buildbot on
http://eastham.rpm5.org:8010
or (this is the "production" @rpm5.org buildbot under construction)
http://buildbot.rpm5.org:8010
ifever/whenever you are ready to move forward.

73 de Jeff


__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Building a 32-bit RPM on a 64-bit machine?

[Jeff Johnson]

On Oct 29, 2010, at 10:53 AM, devzero2000 wrote:

> 
> Sure. It was only an little observation. Sorry if  not strictly related.
> 
> 

NP.

Getting pseudo wired into rpmio, and/or using mount -bind, or
just handling the +/- chroot prefix directly in RPM (all the wrapped
syscall intercepts exist for "RPM ACID" transactional logging) and
otherwise permitting RPM to be fully featured for non-root oddly
skew RPM development.

E.g. last night the bug in rpm-4.4.2.3 preserving CWD across
internal lua scripts (in the RHEL6 setup package) when chroot(2)
is performed bit me Yet Again.

I really wish these bugs could just be "fixed", not "faked". There are
design flaws in using chroot(2) (which requires root to use) with
tools to be used by non-root. Sure there's endless hack-o-rounds
to solve the issues.

73 de Jeff

Re: Building a 32-bit RPM on a 64-bit machine?

[Jeff Johnson]

On Oct 29, 2010, at 10:11 AM, devzero2000 wrote:

> On Fri, Oct 29, 2010 at 4:03 PM, Jeff Johnson  wrote:
> That does need root access to execute chrott(2).
> Or you can use pam_chroot 
> http://www.debian.org/doc/manuals/securing-debian-howto/ap-chroot-ssh-env.en.html
>  without being a root user.
> 

Um, I'm not sure that pam_chroot is appropriate: the URL describes
how to run sshd with privilege sepeartion.

There are fakeroot (which is buggy) and pseudo syscall intercepters
that +/- the prefix implied by chroot(2).

There are other means to avoid needing root as well. But needing root
and building 32 bit on 64 bit are only incidentally related.

73 de Jeff

Re: Building a 32-bit RPM on a 64-bit machine?

[Jeff Johnson]

On Oct 29, 2010, at 9:07 AM, Joe Flowers wrote:

> Hello again,
> 
> I would like to be able to create a 32-bit RPM (with no 64-bit library 
> dependencies) on a 64-bit machine.
> 
> I have a 64-bit, SuSE development machine, and I would like to create an RPM 
> on this 64-bit machine that will install 32-bit software on a 32-bit SuSE 
> machine.
> 

SuSE does "multilib" differently than RedHat afaik. IIRC, there are 3 products
1) ELF32 only
2) ELF64 only
3) multilib.
What's missing is all the needed "other persuasion" packages on 1) and 2).

So what you want is no different than setting up a cross-compiler, or
other wise setting up host <-> target infrastucture.

But its rather easy to set up whatever you need with an install into a 
chroot(2).
That does need root access to execute chrott(2).

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Creating an RPM to install a daemon

[Jeff Johnson]

On Oct 28, 2010, at 11:10 AM, Joe Flowers wrote:

> Hello Everyone,
> 
> I am trying to create an RPM that will install a daemon "correctly", but I'm 
> not sure if or where I should put the command:
> 
> chkconfig --level 345 /etc/rc.d/mydaemon on
> 

(aside)
Well first of all, chkconfig doesn't do the right
thing in many cases like installing into a chroot,
or with SELinux enabled, or if networking isn't
configured, etc etc etc.

So one can make a strong argument (I just tried, YMMV) for _NOT_
doing chkconfig in package scriptlets whatsoever, but rather
configure daemon startup (if changed) in some other way,

The underlying design principle that is violated with chkconfig in packaging
scriptlets is that
package management != configuration management
the important difference is the persistence involved:
The time scale and state for package management is rather different
than that for configuration management.
And in many cases :One size does not fit all usage cases".

But the specific answer to your question is usually to add to %post/%preun.

You likely have examples on your system using chkconfig already installed:

rpm -qa --scripts > /tmp/scripts

and then look for an example of what is being attempted with chkconfig.

Do the same thing.

> 
> Should this line go somewhere in the Makefile (like the "install" section), 
> or should it go in the RPM .spec file somewhere?
> 

Not the Makefile, but in the %post or %preun scriptlet in the spec file.

> Is there some other command I should use rather than this external 
> "chkconfig" program?
> 

For configuring a daemon to start at a run level, chkconfig is what you want.

hth

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: What's the use of rpmmtree ?

[Jeff Johnson]

On Oct 24, 2010, at 12:25 PM, Sriram Narayanan wrote:

> Hi:
> 
> This is a newbie question. I'm facing an error about rpmmtree while
> trying to build rpm5, and I'll ask that question on the devel list.
> 
> But as a newbie, I wanted to know what rpmmtree is for. I'm unable to
> find anything meaningful on the web.
> 

Look for mtree(8), which is what is used on *BSD (but not linux oddly)
to verify file digests.

You don't need rpmmtree to use RPM: its included with RPM because
1) its a useful utility that linux has chosen to ignore
2) mtree uses fts(3) (which is used several places in RPM code)
and provides a simple standalone test framework.

hth

73 de Jeff
> Thanks.
> 
> -- Sriram
> Belenix: www.belenix.org
> __
> RPM Package Managerhttp://rpm5.org
> User Communication List rpm-users@rpm5.org

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: A question about bundling rpm5 with our distribution

[Jeff Johnson]

On Sep 6, 2010, at 3:10 PM, Sriram Narayanan wrote:

> Hello:
> 
> I'm one of the team members of the Belenix Distro (www.belenix.org).
> We are presently using the SVR4 package format, and have wanted to
> move to a more modern day package format.
> 
> We are very likely to use rpm5 as our package system :) We intend to
> bundle both yum and smart, and will leave it to our users as to which
> tool they'd like to use.
> 
> Our question:
> Would you be OK with us using the rpm5 package manager given that most
> of the system libraries on our distro are CDDL licensed ?
> 

All @rpm5.org code is under LGPLv2.1 which should be compatible
with anything.

If there are ANY issues, speak up and I'll try to "fix" whatever
licensing issue there is.

(aside)
If you want someone to represent belenix @rpm5.org, with perhaps cvs
write access, jut ask whenever.

I'd rather hear about issues sooner rather than later; there's always something
that can be done.

hth

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Regarding reliability of

[Jeff Johnson]

On Aug 24, 2010, at 5:20 AM, manjunathan.pa...@wipro.com wrote:

> Dear  Jeff
> Any update on the TPPM  support availability in the latest release of RPM5 ?
>  

Sorry I missed this, my SPAM filters are quite aggressive.

I talked about "Transactionally Protected Package Management" at FOSDEM 2010.

The first release of the rpm-5.3.0 code was May 1st.

Lease note that rpm-5.3.0 is not a "drop-in" replacement for previous
releases. There are too many changes tied to TPPM and "RPM ACID"
for "drop-in" compatibility. Meanwhile there is a great deal of
compatibility still present in rpm-5.3.x and converting from previous
releases is quite feasible. Just that rpm-5.3.x is NOT "drop-in" and
I have no intent or desire for "compatibility" -- "conversion" not 
"compatibility"
was the chosen upgrade path.

Since May I've been doing essentially monthly releases (I missed August
because busy). And I am hoping to continue monthly time-based releases.

So rpm-5.3.3 is at http://rpm5.org/rpm/rpm-5.3/SNAPSHOT now and
will be "officially" releasd on 9/1/2010.

A rpm-5.3.4 release will occur on 10/1/2010. I'm perhaps 20% of the way
through the target feature set already.

See CHANGES in some rpm-X.Y tarball for what is implemented in each release,.

Ask if you need a more complete description.

73 de Jeff

Re: Issues with rpm-5.0.3 - overwrites other RPM's

[Jeff Johnson]

On Jul 23, 2010, at 6:23 PM, Jeff Johnson wrote:

> 
> I prototyped 5-10 "stores" using sqlite3 virtual tables (like 
> /etc/{passwd,group})
> which ends up with an architectural model or a loadable sqlite3 module
> for the "virtual table" with RPM API's (I also prototyped SQL access onto
> RPM header metadata) that _ALSO_ embed sqlite3 so its quite twisty.
> 

I should perhaps describe what I mean by "embedded" wrto
sqlite3 more carefully because (if you are still targeting
multiple platforms) you might find the functionality useful.

Anything and everything that one can do with /usr/bin/sqlite3
can now be done with a macro.

The macro embedded syntax looks like this

%{foo OPTIONS ARGS:BODY}

while the scriptlet embedded syntax looks like

%post -p ""
BODY

To illustrate, this snippet of shell (for a sqlite3 rpmdb)
/usr/bin/sqlite3 /var/lib/rpm/Packages << GO_SYSIN_DD
select * from Packages;
GO_SYSIN_DD
can also be done like
%{sql /var/lib/rpm/Packages:select * from Packages;}
as well as (but you would likelier do INSERT INTO ...)
%post -p ""
select * from Packages;

i.e. its quite possible to update sqlite3 databases using *.rpm
scriptlets.

Of course you could do the same with /usr/bin/sqlite3 run as a script
too, but then you get into dependencies and ordering and all the "traditional"
packaging baggage. The embedded syntaxes can just be used.

IIRC, you had a very methodical approach to paths etc used in your
pacakages. With emebedded sqlite3 you might even be able to move that
into a sqlite3 database instead of using different macro files as a store.

OTOH, no reason to change what isn't broken whatsoever.

73 de Jeff
__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org

Re: Issues with rpm-5.0.3 - overwrites other RPM's

[Jeff Johnson]

On Jul 23, 2010, at 5:43 PM, Tim Mooney wrote:

> 
>> Note that the transition for rpm-5.1.9 -> 5.2.1 is pretty easy, but
>> that is very much not true moving to "RPM ACID" as in rpm-5.3.x,
>> where no "compatibility" is promised, and the conversion issues
>> aren't yet sufficiently well defined to attemp a shrink wrapped
>> rpmdb transition.
> 
> :-)  Based on my upgrade patterns for rpm itself, it will probably be
> very safe by the time I'm contemplating that change.
> 
> Of course, with the final displacement of Solaris in my environment,
> there's a good chance I won't even be running Solaris when I make my next
> workstation change.  If that happens, I probably won't be building my own
> RPM anymore.
> 

(aside)
Your post led me to revisiting pkg(5) on OpenSolaris. I sure
don't understand Sun's engineering methodology, with webrev's
and flag days and gating and more. One of these days I'll
have to see what all the fuss is about ...

>> But rpm-5.3.2 with a clean install is fine, its only the schema changes,
>> and the transition that is rugged. The performance is considerably
>> better, an Berkeley DB ACID eliminates most side-effects from interrupted
>> installs, etc, etc.
> 
> How's RPM's support for Berkeley DB 11.x, or whatever Oracle is calling it
> now?  I'm surprised I haven't seen any posts on the mailing list about it;
> with all the people that have asked about SQLite support over the years,
> I would think that the SQLite compatibility in 11.x would have some people
> chomping at the bit to try it.
> 

I was quite excited when I saw that db-5.0.21 (or whatever its called ;-)
included sqlite3, and instantly leaped to db-5.0.21 and (in rpm-5.3.x)
sqlite3 is embedded (actually multiply embedded I think there's 3
different versions that RPM interfaces with these days).

I was hoping to support both a SQL and (their term, not mine) CRUD
access to an rpmdb.

But because of the manner Oracle chose to layer sqlite3 on top
of CRUD, it would take Yet Another schema change to achieve full
transparent dual SQL <-> CRUD access to an rpmdb.

That sent me looking for alternative approaches and sqlite3 has
has something called "virtual tables" for quite some years now that
could be used for transparent/dual access with SQL <-> CRUD.

I prototyped 5-10 "stores" using sqlite3 virtual tables (like 
/etc/{passwd,group})
which ends up with an architectural model or a loadable sqlite3 module
for the "virtual table" with RPM API's (I also prototyped SQL access onto
RPM header metadata) that _ALSO_ embed sqlite3 so its quite twisty.

In order to attempt a somewhat simpler "production" implementation, I
set off to use OpenPGP keys which have fewer secondary indices to
maintain, and the primary pubkey "blobs" are smaller (~1Kb) than RPM headers
(~36Kb).

That got me side-tracked into adding keypair generate and sign methods,
and automated hkp:// retrieval and validation from "keys.rpm5.org",
an SKS keyserver that is "burning-in" db-5.0.21 with DB_INIT_TXN
as used by "RPM ACID".

Then I got interested in ECDSA which led to adding LibtomCrypt and
Apple CDSA crypto @rpm5.org and ... (today's active task) a TPM crypto
layer *RPM's 7th crypto stack).

But I hope to get out of these crypto black holes and
get back to looking at dual SQL <-> CRUD access of an
rpmdb one of these weeks, sigh.

Just a long circuitous path ...


>>> I had issues with the dependency generator not finding ELF library
>>> provides correctly after the update, but discovered that was because
>>> 5.1.9 doesn't build with libelf support by default (you have to request
>>> it).  Once I did that and then hacked out tools/debugedit so that wasn't
>>> built, everything worked very well.
>>> 
>> 
>> SIs you use the Sun ELF or something else? It likely would not be
>> hard to add a test for Sun ELF implementation to rpm's AutoFu if useful.
> 
> I'm using Sun's libelf.  The tests are already there (some from me, based
> heavily on work from Frank Cusack) in RPM's configure, they're just not
> enabled on solaris unless you ask for them.  It's no problem, it just
> surprised me.
> 

Good. DId you have to manually add a #define to enable Sun -lelf?

If so, that #define could likely be added easily to RPM's AutoFu.

(aside)
BTW, you might look at Ralf's RPM_CHECK_LIB which is really really spiffy.
I can hit a library API with like 5-6 lines in configure.ac

E.g. here's all that was needed to hit the IBM TPM 1.2 software
emulator checked in earlier today:

dnl # IBM TPM 1.2 software emulation
RPM_CHECK_LIB(
[IBM TPM], [tpm],
[tpm], [TPM_Init], [tpm/tpm.h],
[no,external:none], [],
[ AC_DEFINE(WITH_TPM, 1, [Define if building with IBM TPM 1.2 emulator])
], [])

You might want to swipe RPM_CHECK_LIB and add to your toolbox if still
packaging as you did before.

>> 
 Even the build process for rpm5  seemed dodgy, as I had to do things like 
 copy in find-provides, find-requires, and configure them into the macros 
 file by hand.
>>>

Re: Issues with rpm-5.0.3 - overwrites other RPM's

[Jeff Johnson]

Hi Tim!

On Jul 23, 2010, at 3:12 PM, Tim Mooney wrote:

> In regard to: Issues with rpm-5.0.3 - overwrites other RPM's, Robert Cohen...:
> 
>> We've built, and had been using for years, our own instance of rpm 4.0.3, 
>> and it worked a treat. Our platform was Solaris 8/9/10, on sparc.
> 
> I did the same with rpm 2.5.6 and then 4.4.[679].
> 

;-) Now there's a whole lotta history in that transition ;-)

>> Recently, we started using solaris10 on x86, and figured it was a good time 
>> to upgrade our rpm environment.
> 
> I do a lot of building on x86_64-pc-solaris2.10.  I upgraded to rpm 5.1.9
> about 4 months ago.  Joshua Burns' post from December 4th, 2009 had some
> useful tips on build issues with 5.x, so I as prepared for most of what
> I ran into.
> 

Yes. I'd forgotten that post about RPM + Solaris.

My own most recent experience with Solaris was back in March,
when I tried to do "make check" in a Solaris VM.

The paucity of OSS tools to set up a build environment was
a bit depressing. But I still have the VM (and Nexenta seems
like a reasonable alternative approach) both lurk on my todo
list somewhere because of
1) ZFS snapshotting as a form of --rollback
2) SEEK_HOLE/SEEK_DATA used to install files sparsely
I can revisit RPM -> Solaris anytime there is interest.

>> I got 5.2.1 built, but something weird is happening. When we install rpm's 
>> that we produce within the rpm environment, the rpm installation silently 
>> overwrites files belonging to other rpms rather than detecting
>> conflicts.  rpm seems to pay attention to prerequisites, and conflicts
>> specified in the SPEC, but rpm --install/--upgrades will
>> delete/overwrite files that were installed via rpm.
> 
> Hmmm, I've not seen that with 5.1.9.  With one very minor exception, 5.1.9
> built pretty easily and the upgrade process from 4.4.9 to 5.1.9 worked
> without a hitch, which I wasn't really expecting.
> 

Most of that portability from 4.4.9 -> 5.1.9 came from OpenPKG methodology
and Ralf Eneglschall's attention to details.

Note that the transition for rpm-5.1.9 -> 5.2.1 is pretty easy, but
that is very much not true moving to "RPM ACID" as in rpm-5.3.x,
where no "compatibility" is promised, and the conversion issues
aren't yet sufficiently well defined to attemp a shrink wrapped
rpmdb transition.

But rpm-5.3.2 with a clean install is fine, its only the schema changes,
and the transition that is rugged. The performance is considerably
better, an Berkeley DB ACID eliminates most side-effects from interrupted
installs, etc, etc.

> I had issues with the dependency generator not finding ELF library
> provides correctly after the update, but discovered that was because
> 5.1.9 doesn't build with libelf support by default (you have to request
> it).  Once I did that and then hacked out tools/debugedit so that wasn't
> built, everything worked very well.
> 

SIs you use the Sun ELF or something else? It likely would not be
hard to add a test for Sun ELF implementation to rpm's AutoFu if useful.

>> Even the build process for rpm5  seemed dodgy, as I had to do things like 
>> copy in find-provides, find-requires, and configure them into the macros 
>> file by hand.
> 
> I don't remember having to do any of that.  I certainly customized a macro
> or two, but nothing major.
> 
>> The build of 5.0.3/5.2.1 was against gcc 3.4.3, using GNU make, and involved 
>> the  installlation of beecrypt, zlib, and popt from 3rd party sources, into 
>> /usr/local/rpm, before attempting to build rpm as I have been unable to get 
>> the copies distributed with RPM to build.
> 
> I built my own copies of stuff beforehand in many cases.  I'm using
> the Sun Workshop 12u1 compilers, so if anything your builds should have
> been easier.  I probably had the most trouble getting rpm to use my
> copy of neon, but that's because neon builds static libraries only by
> default and there's no good way for RPM's configure to discover that neon
> depends on things like libssl and libkrb5.
> 

For reference: What version of neon?

Also for reference: If you send along the configure options you used
(like in config.log) then I'll add to devtool.conf as a "reference"
Solaris configure options.

(aside)
I'm still not happy with the myriad configure options needed
to build RPM these days, but devtool (also from Ralf Engelschall)
does make the complexity almost painless. Its the iteration
deciding what to turn on and what not that is hardest when
building RPM from scratch. With >30 configure options that
is a rugged decision path.

Glad to here you're still using RPM ;-)

73 de Jeff

__
RPM Package Managerhttp://rpm5.org
User Communication List rpm-users@rpm5.org