Re: libexec and multi-arch

[Jeff Johnson]

> On Feb 5, 2018, at 5:46 PM, Tomasz Pala <go...@polanet.pl> wrote:
> 
> On Mon, Feb 05, 2018 at 15:20:55 -0500, Jeff Johnson wrote:
> 
>>> No, it only hidden the problem behind getconf binary being handled
>>> _somehow_. I once even wondered how this is done, apparently rpm is
>>> trying to be way too smart.
>> 
>> RPM implements arch specific content generally as:
>>  Libraries on different paths.
>>  Executables on same path.
> 
> libexec directory should be treated as binaries, not libraries (even if
> technically some of the contents are libraries, like plugins) nor data,
> as the purpose of this is to carry private helpers of the binaries, not
> any shared code.
> 
>> This requires a resolution to a preferred arch (usually x86_64) when 
>> installing executables onto identical paths.
> 
> How is rpm recognizing files that are subject to this special-handling
> as same-path-executables?

RPM reads elf headers and sets a per file color flag in metadata:
0==no arch or not elf
1==elf32
2==elf64
4== mostly unused, except on big endian mips
Bits were used instead of an enum in case some idiot decided to put both elf32 
and elf64 in the same static archive.

Choosing between multiple files with different flags but identical paths is 
computed during install, and the preferred arch is used while other files are 
marked "wrong color" and not installed

> How to extend this to all the files inside libexecdir?

There's nothing to extend: the confusion comes from mixtures of 
libraries/modules/executables in libexecdir, not anything else.

> How to extend this to all the files in mandir maybe?
> 

There are no elf files in mandir, so an extension of what rpm does makes no 
sense.

Meanwhile there are many conventions to extend man pages, in subdirectories, 
and with suffixes, that can be used if necessary.

>> Whether RPM is too smart or the requested implementation is insufficiently 
>> general is arguable. For example, one might desire the ability for 
>> per-file-path configurable choice of architecture rather than per-system 
>> confIguration.
> 
> Or - to always use preferred arch, for ANY file that is overlapping in
> packages that differ only in arch, e.g. /etc or anything inside datadir.
> 

That is essentially what is implemented in rpm for files from different arch 
(technically "color" as in elf32 or elf64) that are on the same path.

> Currently, if such files exist, they need to have identical content for
> the packages to be installed simultaneously, or be separated into
> respective subpackages, which is unnecessary maintenance burden.
> 

Um, I'm not sure how rpm is responsible for packaging policies (like separated 
into sub packages) or maintenance burden.

hth

73 de Jeff
> -- 
> Tomasz Pala <go...@pld-linux.org>
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

[Jeff Johnson]

> On Feb 5, 2018, at 3:23 PM, Neal Gompa  wrote:
> 
> I'd probably argue that libexec should have the same multi-arch
> handling that (s)bin does (primary arch "wins" and secondary arches
> are ignored), though last I checked this isn't specific to paths and
> should Just Work(TM).
> 
> 

And then there is the /usr/lib/rpm path to "probably argue" about.

Truly the best way out of all these discussions is to complete the 10+ year 
transition from 32bit to 64bit and stop the endless discussions and fiddle up 
retrofits.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: libexec and multi-arch

[Jeff Johnson]

> On Feb 5, 2018, at 10:13 AM, Tomasz Pala  wrote:
> 
>> On Mon, Feb 05, 2018 at 10:47:32 +0100, Jacek Konieczny wrote:
>> 
>> Using lib/lib64/libx32 instead of libexec was much smarter in this case.
> 
> No, it only hidden the problem behind getconf binary being handled
> _somehow_. I once even wondered how this is done, apparently rpm is
> trying to be way too smart.
> 

RPM implements arch specific content generally as:
  Libraries on different paths.
  Executables on same path.
This requires a resolution to a preferred arch (usually x86_64) when installing 
executables onto identical paths.

Whether RPM is too smart or the requested implementation is insufficiently 
general is arguable. For example, one might desire the ability for 
per-file-path configurable choice of architecture rather than per-system 
confIguration.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

RPM+MQTT in rpm-5.4.18 ...

[Jeff Johnson]

rpm-5.4.18 (soon) will embed the paho-mqtt client to do MQTT messaging.

I will likely do a snapshot of 5.4.18 as soon as I haul out some
MQTT debugging trash and repair portability damage.

TL;DR
Don't install paho-mqtt-1.1.0 and/or build rpm-5.4.18 --without-mqtt
and ignore the rest of this message.

Read further about RPM+MQTT in the attachment.

Discussions on  please.

73 de Jeff


MQTT.msg
Description: Binary data
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python packaging

[Jeff Johnson]

On Jun 20, 2016, at 10:11 AM, Jacek Konieczny wrote:

> 
>> reality is that we have no consistency,
> 
> Some people seem to don't care at all. :-(
> I am afraid, that whatever scheme we decide on, people will still commit 
> crap. That is minority, but quite annoying.
> 

Distro package naming is up to distros ... BUT ...

If you wish naming consistency, then you will likely need detection and 
enforcement.
rpmbuild implementations is one mechanism.

rpmbuild already runs *RE's against package name  (and more) to prevent
(essentially) cross-scripting attacks on contexts where NVR are used in 
scripting.

The *RE's are doing nothing more than limiting the character set being used 
currently.
It would not be hard to apply additional *RE's against, say, python package 
naming.

There is also RPM+SED (i.e. a PCRE sed-like embedding) intended as a better
way to do %{?dist} branding in build releases. Basically the equivalent of 
running

rpm -qp --qf '%{release}'  *.rpm | sed -e 's/$/%{?dist}/'

while building, without adding dependencies on external sed(1). There are
many other usage cases for an internal/embedded find-and-replace in RPM.

hth

73 de Jeff


___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm -Uhv --oldpackage loses configs

[Jeff Johnson]

On Jun 7, 2016, at 10:45 AM, Elan Ruusamäe wrote:

> On 07.06.2016 17:39, Jeff Johnson wrote:
>> Were the files present on the file system (presumably not, but I'd like 
>> confirmation)?
> yes of course they were present. otherwise there would not be messages from 
> rpm like:
> 
> warning: /etc/security/namespace.init saved as 
> /etc/security/namespace.init.rpmsave
> 

Of course the files were present when renamed.

I am asking for confirmation whether the files were present _AFTER_ the 
upgrade/downgrade
completed trying to diagnose your problem.

73 de Jeff
> 
> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm -Uhv --oldpackage loses configs

[Jeff Johnson]

On Jun 7, 2016, at 10:44 AM, Elan Ruusamäe wrote:

> 
>> Do the repackaged rpm’s contain those files.?
> 
> the config files are present in .rpm file and in rpmdb where i installed the 
> .rpm file:
> 
> # rpm -qplvc 1465233457/pam-1.1.8-8.x86_64.rpm|grep auth
> -rw-r--r--1 rootroot 1016 veebr 25  2015 
> /etc/pam.d/system-auth
> 
> # rpm -qc pam|grep /etc/pam.d/system-auth
> /etc/pam.d/system-auth
> 
> and still marked as %config

Sorry for not being precise.

Yes the metadata header contains those files.

Are the files in the payload?
rpm2cpio *.rpm | cpio -itv

If the files  are not in the payload, then they will not be installed.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm -Uhv --oldpackage loses configs

[Jeff Johnson]

On Jun 7, 2016, at 10:37 AM, Elan Ruusamäe wrote:

> On 07.06.2016 17:31, Jeffrey Johnson wrote:
>> Its still not clear to me whether the new package files were installed or 
>> not.
> they were not
> 
> mv -i will ask to overwrite files, it never asked ... so...
> 

Yes I read the man page.

Were the files present on the file system (presumably not, but I'd like 
confirmation)?

Do the repackaged rpm’s contain those files.?

When/how were the repackaged packages created?

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

rpm needs db-6.1.26+ patch (was Re: python3 optional deps)

[Jeff Johnson]

On Jun 1, 2016, at 6:29 PM, Elan Ruusamäe wrote:

> 
> 1. were in middle of 5.4.15 -> 5.4.17 change

The attached patch (for db-6.1.26/db-6.2.23) is necessary with rpm-5.4.x).

Earlier versions of db-x.y.z need no modification.

hth

73 de Jeff
==


jbj13.patch
Description: Binary data
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: ldconfig forgotten

[Jeff Johnson]

On May 31, 2016, at 6:30 PM, Elan Ruusamäe wrote:

> On 01.06.2016 00:47, Jeff Johnson wrote:
>> Verified ... how?
> in first post i said it's 100% reproducible,
> so,
> 
> i applied your patch to rpm,
> rebuilt rpm, installed
> 
> and retried the 100% reproducible testcase -- ps command was usable after 
> package was just installed
> 

Good, fixed.

There's likely a slightly better fix, by limiting the skipping of 
/sbin/ldconfig to only removed
packages on upgrade, if anyone is interested. At worst, there will be dangling 
unused
symlinks that the skipped /sbin/ldconfig would have removed.

*shrug*

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: ldconfig forgotten

[Jeff Johnson]

On May 31, 2016, at 4:47 PM, Elan Ruusamäe wrote:

> On 31.05.2016 23:37, Jeff Johnson wrote:
>> On May 31, 2016, at 4:33 PM, Elan Ruusamäe wrote:
>>> i prefer reliability over broken optimization.
>>> 
>> So disable the optimization. I'm quite sure I have described
>> how to disable the optimization on this (and other) mailing lists
>> in the past 10 years.
> 
> i said in previous mail:
> >i'll apply your patch and push after some testing
> >however, i'd like to comment few parts
> 
> 
> your patch doesn't disable?
> 

Verified ... how?

Install with -vv, see if this message appears:

D:   install: %post(procps-3.3.11-1.x86_64) skipping redundant "/sbin/ldconfig".

Every usage of ldconfig_done tests ldconfig_path for NULL first.

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: ldconfig forgotten

[Jeff Johnson]

On May 31, 2016, at 4:33 PM, Elan Ruusamäe wrote:
> 
> i prefer reliability over broken optimization.
> 

So disable the optimization. I'm quite sure I have described
how to disable the optimization on this (and other) mailing lists
in the past 10 years.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: ldconfig forgotten

[Jeff Johnson]

On May 31, 2016, at 2:52 AM, Elan Ruusamäe wrote:

> On 31.05.2016 09:45, Elan Ruusamäe wrote:
>> [root@57c8cccdc671 bin]# poldek -u procps 
> oh, and it's 100% reproducible:
> 

Yes. And it has been discussed many times over the last 15y,
several times on this list.

Short answer:
Disable the mechanism with attached patch to lib/psm.c
- OR --
Don't use
%scriptlet -p /sbin/ldconfig
in packages that are known to invoke an executable that depends on
a new library.

Longer answer:
Libraries often have scriptlets like this:
%post -p /sbin/ldconfig
%postun -p /sbin/ldconfig
(or the equivalent within a scriptlet body).

On upgrade, with install before erase, this results in 2 executions of 
/sbin/ldconfig.

One of those invocations is redundant when a library is being replaced, 
the
install %post is sufficient to make the library avaialble for loading.

The redundant invocation of /sbin/ldconfig  was deemed a performance 
flaw @redhat,
where I was asked to implement the current optimization.

So the RPM optimization detects when /sbin/ldconfig has just been run, 
and skips the next
%scriptlet -p /sbin/ldconfig
(when written exactly like that) if /sbin/ldconfig has been executed.

Issues are only seen when a scriptlet contains an executable that needs
a changed API: this is usually a very rare occurrence

There are other flaws with the optimization when dependency loops break 
the sorted order
(and/or --noorder is used). , and the fix of running /sbin/ldconfig 
manually is rather simple
(even if the problem is obscure).

There is another flaw (as in your report), when the very last package
has a pending skipped /sbin/ldconfig invocation.

The fix of running /sbin/ldconfig manually is rather simple
(even if the problem is obscure).

hth

73 de Jeff
===
RCS file: /v/rpm/cvs/rpm/lib/psm.c,v
retrieving revision 2.399.2.20
diff -p -u -w -r2.399.2.20 psm.c
--- psm.c   11 May 2015 21:10:16 -  2.399.2.20
+++ psm.c   31 May 2016 15:19:20 -
@@ -800,7 +800,7 @@ static rpmRC runEmbeddedScript(rpmpsm ps
 static int ldconfig_done = 0;
 
 /*@unchecked@*/ /*@observer@*/ /*@null@*/
-static const char * ldconfig_path = "/sbin/ldconfig";
+static const char * ldconfig_path = NULL;
 
 /**
  * Run scriptlet with args.



> $ docker run --rm -it glen/pld bash
> [root@412a6cf4756e /]# poldek -u procps
> [root@412a6cf4756e /]# ps
> ps: error while loading shared libraries: libprocps.so.5: cannot open shared 
> object file: No such file or directory
> 
> 
> and here's appropriate rpm -vvv run:
> 
> [root@93a685e61c8b /]# poldek -u procps --pmopt=verbose --pmopt=verbose 
> --pmopt=verbose
> Loading [pndir]th...
> Loading [pndir]th...
> 25794 packages read
> Processing dependencies...
> procps-3.3.11-1.x86_64 marks systemd-libs-221-13.x86_64 (cap 
> libsystemd.so.0()(64bit))
> systemd-libs-221-13.x86_64 marks libgcrypt-1.7.0-1.x86_64 (cap libgcrypt >= 
> 1.4.5)
>  libgcrypt-1.7.0-1.x86_64 marks libgpg-error-1.22-1.x86_64 (cap libgpg-error 
> >= 1.13)
> systemd-libs-221-13.x86_64 marks lz4-libs-r131-4.x86_64 (cap 
> liblz4.so.1()(64bit))
> There are 5 packages to install (4 marked by dependencies):
> I procps-3.3.11-1.x86_64
> D libgcrypt-1.7.0-1.x86_64  libgpg-error-1.22-1.x86_64 lz4-libs-r131-4.x86_64 
>  systemd-libs-221-13.x86_64
> This operation will use 4.2MB of disk space.
> Need to get 1.4MB of archives.
> Executing pm-command.sh --upgrade -vh --root / --define _check_dirname_deps 1 
> --verbose --verbose --verbose...
> D: pool ts:created size 1200 limit -1 flags 0
> D: pool gi:created size 176 limit -1 flags 0
> D: pool dig:created size 424 limit -1 flags 0
> D: pool h:created size 360 limit -1 flags 0
> D: Expected size:   126857 = lead(96)+sigs(268)+pad(4)+data(126489)
> D:   Actual size:   126857
> D: pool ds:created size 232 limit -1 flags 0
> D: pool tsi:created size 48 limit -1 flags 0
> D: pool te:created size 368 limit -1 flags 0
> D: pool fi:created size 560 limit -1 flags 0
> D: pool al:created size 64 limit -1 flags 0
> D: pool bf:created size 56 limit -1 flags 0
> D: pool db:created size 328 limit -1 flags 0
> D: pool dbi:created size 472 limit -1 flags 0
> D: rpmdb: cpus 8 physmem 7985Mb
> D: opening  db environment /var/lib/rpm/Packages thread:lock:log:mpool:txn
> D: opening  db index   /var/lib/rpm/Packages thread:rdonly:auto_commit 
> mode=0x0
> D: opening  db index   /var/lib/rpm/Name thread:rdonly:auto_commit 
> mode=0x0
> D: pool mi:created size 152 limit -1 flags 0
> D: Expected size:   471376 = lead(96)+sigs(268)+pad(4)+data(471008)
> D:   Actual size:   471376
> D: Expected size:39670 = 

Re: man-pages-4.05-1.noarch conflicts with lirc-0.9.3a-1.x86_64

[Jeff Johnson]

On May 10, 2016, at 6:30 AM, Elan Ruusamäe wrote:

> On 10.05.2016 09:12, Jeffrey Johnson wrote:
>>> Fixed in lirc-0.9.3a-2 / man-pages-4.05-2.
>>> >
>> Is file conflict detection an RPM misfeature?
> no. it's packaging mistake (two packages installing same file path)
> 

Of course it is a packaging mistake: I have fixed dozens of file conflcts
from duplicate man pages

The question was about detecting and failing an rpm upgrade
when there is duplicate content. EVen when content is identical,
there is a timestamp in some compression formats (yes there is an
option to remove the timestamp), and/or the buildtime on the file can/will 
differ.

There are alternative automated resolutions other than human intervention
to rebuild the package that might be attempted.

>> It would not be impossibly hard to resolve a file conflict automatically,
>> perhaps with an alternatives-like symlink farm analogue mechanism
>> to permit multiple copies to coexist.
> there's no alternatives, two packages should not provide same file.
> 

Alternatives (fundamentally) is just a symlink to a unique file path. RPM
could easily install the file on a unique path and install a symlink, and
even add the configuration so that alternatives(8) (literally) could be
used to resolve the file conflict on the machine being installed.

All of the above has to do with automating (at least temporarily) the
install instead of just failing.

> lirc one was the outdated one, according to commits made.
> 

There are other relatively simple automations:
1) FIFO - first version is installed, all other file versions are 
skipped.
2) LIFO - last version clobbers all other file versions.
3) largest - the bigger file is installed
4) newest - the newest modify time is installed
5) masked - a file conflict automated resolution is read from 
configuration.
The majority of manual file conflict resolutions are one of the above.

There's also a means to detect file conflicts at buildtime and fail the build,
not the install later, if rpmbuild attempts to maintain a collection of all 
current
headers in a database, and checks for file conflicts at the end of successful 
builds.

73 de Jeff
> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3 optional deps

[Jeff Johnson]

On May 6, 2016, at 3:41 AM, Elan Ruusamäe wrote:

> On 06.05.2016 10:11, Jacek Konieczny wrote:
>> 
>> I guess the python dependency generator should be fixed in our RPM, to 
>> ignore the 'extras' dependencies (or to make them 'suggest', skipping the 
>> ':python_version' ones.
> is there python module to parse the egg info? or just manual parsing each time
> 
> btw, maybe it's better to extract the python dependency generators as 
> separate package, out of rpm sourcetree?
> it would be a lot simplier to maintain, than patching patch patching the 
> previous patch. our patches will likely never end up upstream.
> 

Submit any patches for rpm you wish.

Meanwhile, rpm cannot carry extractors for every possible dependency forever,
that simply does not scale.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: how rpm destroyed my private key

[Jeff Johnson]

On Apr 28, 2016, at 4:43 AM, Elan Ruusamäe wrote:

> On 27.04.2016 21:09, Jeff Johnson wrote:
>> What I am suggesting is using --justdb --noscripts --notriggers on the v2 
>> package before attempting an upgrade.
> 
> you're solving the problem in wrong order. suggesting to use --justdb 
> --notriggers --notriggers means i already know that specific package upgrade 
> path is broken.
> but the time i found that out it's already late, file is destroyed and 
> there's no recovery path from rpm (even repackage did not "save" me a copy).
> 

I'm not solving any problem whatsoever. I was asking
Does preinstalling with --justdb "work" as you expected?

> and the solution should be independant who runs the upgrade. should behave 
> similarily to all users who upgrade the package.
> 

Again, no "solution" with %config(noreplace) was suggested or intended.

I did suggest a %noclobber attribute which you have chosen to ignore.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: how rpm destroyed my private key

[Jeff Johnson]

On Apr 28, 2016, at 4:41 AM, Elan Ruusamäe wrote:

> 
> but rpm i use to manage configuration and expect %noreplace to mean "do not 
> replace the file". that one technical detail makes it behave differently on 
> high level does not change my expectation. i'd rather consider it flaw in 
> implementation. and that it has been so in last two decades, doesn't mean it 
> has to stay so. there are VENDOR_PLD conditions if that rpm5.org maintainers 
> do not consider usable for everybody.
> 

Yes, "noreplace" has an ambiguous meaning that is non-intuitive.

The intuitive meaning of "noreplace" is
Never replace

Meanwhile -- as I have pointed out -- %config(noreplace) was designed to replace
unmodified existing config files with different content, and -- as you have 
pointed out --
will replace pre-existing files.

Both behaviors are intentional in %config handling when implemented and
cannot easily be changed without triggering a couple years of confusion.

Meanwhile, the whole %config renaming needs to be abandoned in favor
of an etckeeper approach to do checkins instead of renaming.

RPM has embedded libgit2 sufficiently well enough to do simple checkins.
A more complete solution awaits some distro willing to work through the 
remaining
details for an entire distribution.

73 de Jeff


___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: how rpm destroyed my private key

[Jeff Johnson]

On Apr 27, 2016, at 1:08 PM, Jeff Johnson wrote:

> 
> So why are you claiming RPM "destroyed" your secret_key.php file when you are 
> misusing
>   %config(noreplace)
> with expectations of behavior that have never been implemented, and cannot be 
> implemented
> because there is no reference copy of metadata information needed to compute 
> "modified", and
> are in general confusing %config handling with --nooverwrite behavior?
> 

Let's try and make this positive, shall we?

I suspect that if you registered a closer (including secret_key.php)  
approximation to the existing
installed (but unmanaged) files, that some of your expectations with 
%config(noreplace) renaming
might Just Work.

What I am suggesting is using --justdb --noscripts --notriggers on the v2 
package before attempting an upgrade.

That might (entirely untested) provide a reference point to compute "modified" 
that will trigger
%config renaming when you reinstall (or erasing with --repackage) the v2 
package.

Expecting --repackage of the v1 package to somehow preserve secret_key.php 
(which isn't
included in the v1 package) to preserve the file does not make any sense 
whatsoever.

It would not be too hard to add a %nooverwrite (or %noclobber) attribute to 
spec files
and packages (but I suspect that there is a vanishingly small usage case).

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: how rpm destroyed my private key

[Jeff Johnson]

On Apr 27, 2016, at 10:12 AM, Elan Ruusamäe wrote:

> On 27.04.2016 16:55, Jeff Johnson wrote:
>> On Apr 27, 2016, at 3:13 AM, Elan Ruusamäe wrote:
>> 
>>> and it did it again!
>>> 
>> You are smart enough to identify a better solution for your problems than 
>> blaming RPM
>> for destroying your files ... again ... and again.
> 
> meh?
> 
> i described scenario that is complete for interested party to be able to 
> reproduce and witness the destruction.
> 
> please explain how is that my problem that rpm replaced file marked as 
> %config(noreplace) on package installation not rpm's? i'm not convinced.
> 

Sorry to be snarky ... we have gone through %config handling many many times.

In the v1 package, secret_key.php is _NOT_ in a package.
Which means:
1) there is no way to tell if the file is modified, owned, timestamped, 
should be present, etc.

In the v2 package, secret_key.php is present and marked with %config(noreplace).
You are expecting rpm to rename secret_key.php before installing the v2 file to 
preserve
the contents because you have marked the file with
%config(noreplace)

The %config(noreplace) handling in RPM is a decision tree based on being able 
to detect
whether a file is "modified". RPM will happily replace unmodified config files 
with altered
content even when the previous file differs. This is exactly the behavior one 
expects
when the newer config file has been changed, and that other executables (like 
daemons)
which will be restarted, are implicitly dependent on the new configuration.

RPM also goes to lengths to save "modified" %config files when they are 
included in packages,
as we all know.

The unmanaged version of secret_key.php that is installed does not have a record
in the rpmdb that permits detecting whether a file is "modified". So RPM 
replaces
(not "destroys") the file just like any archiver would do.

This is exactly the behavior one expects in the following circumstances
1) converting an unmanaged -> managed package where "modified" can be 
computed
2) replacing a misconfigured manually installed set of files with a 
package

Your expectations have confused the existing %config(noreplace) behavior with 
other
archiver behaviors. E.g. tar(1) has a number of "overwrite controi" options
--keep-old-files
--keep-newer-files
--overwrite-dir
--preserve-permissions
I doubt you would be surprised if you extracted a tar archive that "destroyed" 
secret_key.php,
particularly if you had forgotten to add options that might have prevented 
overwriting. Similarly,
if you added tar options wrongly and then found that, say, a daemon would not 
restart because
some config file was _NOT_ replaced as expected, I doubt that you would be 
surprised.

So why are you claiming RPM "destroyed" your secret_key.php file when you are 
misusing
%config(noreplace)
with expectations of behavior that have never been implemented, and cannot be 
implemented
because there is no reference copy of metadata information needed to compute 
"modified", and
are in general confusing %config handling with --nooverwrite behavior?

73 de Jeff




___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: how rpm destroyed my private key

[Jeff Johnson]

On Apr 27, 2016, at 3:13 AM, Elan Ruusamäe wrote:

> and it did it again!
> 

You are smart enough to identify a better solution for your problems than 
blaming RPM
for destroying your files ... again ... and again.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16-0.20160321.src.rpm, take 3

[Jeff Johnson]

On Apr 24, 2016, at 3:29 PM, Elan Ruusamäe wrote:

> On 24.04.2016 22:23, Jeff Johnson wrote:
>> When was the last time a distro rebuilt a package to correct a spelling 
>> error typo in, say,
>> the Swahili translation?
> 
> fedora rebuilds *all* packages for each release, so each release will have 
> whatever translations were added to .spec. altho i don't even know do they 
> translate or where they store the translations.
> 

Fedora (and RHEL) are likely not bothering with anything but the C locale
for package description/summary/group, largely because of the comp-lexity
of the task, and perhaps because of (from PLD years ago)

Kurwa, durni ameryka?ce sobe zawsze my?l?, ?e ca?y ?wiat mówi po 
angielsku...

> in pld, there's only -pl translation that can be considered near complete.
> 

Well there is the C locale so actually there are two copies of I18N tags
if pl is "near complete".

For a national distro with a dominant locale, its also possible to
reconfigure to eliminate the C locale, and put the pl translation
into header content (as if it were the "C" locale)

See RPMBUILD_DEFAULT_LANG (currently "C", but  trivial to turn into a macro).

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 24, 2016, at 3:20 PM, Elan Ruusamäe wrote:

> On 24.04.2016 21:08, Jeff Johnson wrote:
>> The fix is --without-sasl2 (not --without-sasl). My brain fart, now verified.
> 
> thanks. indeed that's the option.
> 
> seems libacl is also new library dependency. is there appropriate 
> functionality or it also slipped in somehow?
> 

There is a --without-acl build option that should do what you wish.

The lossage will be in in tools/mtree (swiped from F*BSD). RPM itself
doesn't give a hoot about ACL's.

Meanwhile there are already RFE's to handle ACL's and capabilities and xattr's
and worser through *.rpm packaging which are the reason to link -lacl.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16-0.20160321.src.rpm, take 3

[Jeff Johnson]

On Apr 24, 2016, at 10:25 AM, Jeff Johnson wrote:

...

> 
> I think that I18N could (and should) be done better in RPM. Meanwhile 
> devising a transparent
> "legacy compatible" retrofit for RPM_I18NSTRING_TYPE is/was the necessary 
> first step to attempting
> a better implementation.
> 

I perhaps should describe some modest extensions that improve
on the storage of I18N in headers.

RPM has something called "arbitrary tags" that are essentially an associative 
array.
The tag name string is digested with SHA1 and the least significant 30bits of 
the digest
have 0x400 (i.e. the 2 most significant bits are 01..), so there are 
potentially
1G of tagno's in the arbitrary tag space that can be used as an associative 
array.

E.g. any tag name string can be used as an arbitrary tag like
Foo:
can be added to package headers by adding to a macro in configuration, and
just adding to *.spec files.

One header tag storage improvement would be to permit tag names like (literally)
Description(en_US.utf8)
which the *.spec parser can easily generate and implicitly predefine tags of 
that form
(instead of the rather clunky %_arbitrary_tags macro configuration).

The retrieval of Summary/Group/Description would have to sequentially try
Description($LANG)
and then (if that tag doesn't exist)
Description
exactly mimic'ing existing behavior, where the locale is stored in 
RPMTAG_HEADERI18NTABLE
and a sequential comparison is undertaken before using the C locale.

(aside)
There are other compatibility issues with specspo (and a lookaside cache) that 
most
distros are using most of this century that can be retrofitted as necessary.

I personally believe that automatic translation through yandex.com, storing the 
returned
string in a local cache database, with provisions to prime the lookaside cache 
by
installing a package, or by doing a git checkout, or by using the embedded 
sqlite,
or mongo, or ... are also needed for "building" and "translation" processes to
become indpendent. 

The current means of distributing all possible locales directly in header 
metadata
simply does not scale.

When was the last time a distro rebuilt a package to correct a spelling error 
typo in, say,
the Swahili translation?

Getting rid of RPM_I18NSTRING_TYPE is (imho) a necessary first step in 
attempting
a better implementation.

YMMV.

73 de Jeff


___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 24, 2016, at 12:49 PM, Jeff Johnson wrote:

> 
> On Apr 24, 2016, at 12:32 PM, Elan Ruusamäe wrote:
> 
>> On 24.04.2016 18:28, Jeff Johnson wrote:
>>> So specifiying --without-sasl when building is the only (well you could 
>>> configure/install sasl
>>> correctly before executing rpm too) way to remove the syslog'd message.
>> 
>> added, did not help. did not investigate further, but build log is available:
>> 
>> 
>> http://carme.pld-linux.org/~glen/rpm-5.4.16-0.2.i686-linux.2016-04-24_17-50-47.log
>> 
> 
> I'll have a fix shortly.
> 

The fix is --without-sasl2 (not --without-sasl). My brain fart, now verified.

I'll also change configure.ac to be "opt-in" rather than "opt-out" in 
rpm-5.4.17.

Two syslog msgs for every rpm exec (or library load) is too much info for my 
taste. ymmv.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 24, 2016, at 12:49 PM, Jeff Johnson wrote:

> 
> (aside)
> BTW, there is one other 1-line patch to rpmio/pkgio.c needed to plug a 16b 
> memory leak
> in rdSignature(). The final code should look like
> 
> ...
>/* All packages should have RPMSIGTAG_MD5. */ 
>he->tag = (rpmTag) RPMSIGTAG_MD5; 
>xx = headerGet(sigh, he, HEADERGET_SIGHEADER); 
>he->p.ptr = _free(he->p.ptr);  /* <== THIS LINE */
>if (!xx) { 
> ...
> 

And there is another 1 liner found by fuzzing with american fuzzy-lop
after 1.1B execs that I just checked in:

Summary stats
=

   Fuzzers alive : 6
  Total run time : 37 days, 14 hours
 Total execs : 1135 million
Cumulative speed : 2097 execs/sec
   Pending paths : 0 faves, 1 total
  Pending per fuzzer : 0 faves, 0 total (on average)
   Crashes found : 38 locally unique

FWIW, I don't consider either the 16b memory leak or the header read hardening
(that affects 3 unique "hangs" found in 1.1B execs) to be worth re-rolling
(and re-testing) rpm-5.4.16.

RPM is about installing *.rpm packages, not in reading randomized inputs.

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 24, 2016, at 12:32 PM, Elan Ruusamäe wrote:

> On 24.04.2016 18:28, Jeff Johnson wrote:
>> So specifiying --without-sasl when building is the only (well you could 
>> configure/install sasl
>> correctly before executing rpm too) way to remove the syslog'd message.
> 
> added, did not help. did not investigate further, but build log is available:
> 
> 
> http://carme.pld-linux.org/~glen/rpm-5.4.16-0.2.i686-linux.2016-04-24_17-50-47.log
> 

I'll have a fix shortly.

The syslog messages from linking SASL2 are very annoying. There's
another msg that happens on exit:
Apr 24 12:31:21 hi.jbj.org rpm[5634]: DIGEST-MD5 common mech free

I'll either find a way to get rid of the messages or make --with-sasl "opt-in"
rather than "opt-out".

(aside)
BTW, there is one other 1-line patch to rpmio/pkgio.c needed to plug a 16b 
memory leak
in rdSignature(). The final code should look like

...
/* All packages should have RPMSIGTAG_MD5. */ 
he->tag = (rpmTag) RPMSIGTAG_MD5; 
xx = headerGet(sigh, he, HEADERGET_SIGHEADER); 
he->p.ptr = _free(he->p.ptr);   /* <== THIS LINE */
if (!xx) { 
...

73 de Jeff.

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 24, 2016, at 10:41 AM, Jeff Johnson wrote:

>> 
>> something logs to syslog at this time:
>> Apr 24 12:48:27 new-server rpm: looking for plugins in '/usr/lib/sasl2', 
>> failed to open directory, error: No such file or directory
>> 
> 
> That message isn't coming from RPM afaik.
> 

The message is likely coming from the .init section in libsasl.so.3.0.0

$ readelf -a /usr/lib64/libsasl2.so.3.0.0
...
  [ 9] .init PROGBITS 4268  4268
   001a    AX   0 0 4
  [10] .plt  PROGBITS 4290  4290
   08d0  0010  AX   0 0 16
  [11] .text PROGBITS 4b60  4b60
   00011193    AX   0 0 16

So specifiying --without-sasl when building is the only (well you could 
configure/install sasl
correctly before executing rpm too) way to remove the syslog'd message.

Note that adding --with-java when building RPM  links to a library with 
.init/.fini sections.

hth

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 24, 2016, at 6:03 AM, Elan Ruusamäe wrote:

> On 24.04.2016 12:51, Elan Ruusamäe wrote:
>> 
>> anyway, i finally installed the built version. the sasl2 error is probably 
>> from somewhere else.
>> 
>> 12:48:25 root[load: 0.00]@new-server /# rpm -q rpm
>> rpm-5.4.16-0.1.i686 
> There are 1 package to install, 1 to remove:
> I filesystem-4.0-46.i686
> R filesystem-4.0-42.i686
> Need to get 19.7KB of archives (19.7KB to download).
> 
> Retrieving th::filesystem-4.0-46.i686.rpm...
> .. 100.0% [19.7K (19.7K/s)]
> warn: tag Name(1000) type(0x4) != expected type(0x10006)
> warn: tag Summary(1004) type(0x7) != expected type(0x10006)
> warn: tag Buildhost(1007) type(0x4) != expected type(0x10006)

No, the above has nothing to do with I18N.

rpm-5.4.16 explicitly checks the data type on all metdata header tags.

Unfortunately, there are ancient tag number collisions between signature
and metadata headers between
RPMSIGTAG_SIZE  RPMTAG_NAME
RPMSIGTAG_MD5   RPMTAG_SUMMARY
RPMSIGTAG_PAYLOADSIZE   RPMTAG_BUILDHOST
that need to be handled by marking the headerGet()/headerNext() access with
the flag HEADERGET_SIGHEADER.

The warning is there to ensure that the HEADERGET_SIGHEADER has been specified.

Presumably those messages are triggered by poldek access, not rpm.

You can also patch tagValidate() in rpmdb/tagname.c to avoid the warning message
(which is what recent RPM versions was doing) if you cannot find the place
where HEADERGET_SIGHEADER needs to be added.

> Executing pm-command.sh --upgrade -vh --root / --define _check_dirname_deps 
> 1...
> Preparing... ### [100%]
> Repackaging...
>   1:filesystem ### [100%]
> Upgrading...
>   1:filesystem ### [100%]
> Installing set #2
> 
> probably related to RPM_I18NSTRING_TYPE what jbj talked and i don't 
> understand.
> 

Again, nothing to do with I18N, but rather disambiguating signature from 
metadata
header tagno's.

hth

73 de Jeff

> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Fwd: rpm-5.4.16 snapshot

[Jeff Johnson]

Begin forwarded message:

> From: Elan Ruusamäe <g...@pld-linux.org>
> Subject: Re: rpm-5.4.16 snapshot
> Date: April 24, 2016 5:51:58 AM EDT
> To: "PLD: Developers list (English)" <pld-devel-en@lists.pld-linux.org>
> Reply-To: "PLD: Developers list (English)" <pld-devel-en@lists.pld-linux.org>
> 
> On 20.04.2016 15:38, Jeff Johnson wrote:
>> There is a final snapshot release of rpm-5.4.16 now available at
>> 
>>  http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160420.src.rpm
> 
> baggins, could you take care of the last remaining patches on dev-5.4.16. 
> those are both yours and i don't know how to test are those still needed.
> 
> 
> anyway, i finally installed the built version. the sasl2 error is probably 
> from somewhere else.
> 

The SASL error is coming from integration with the mongo-c-driver. Adding 
--without-sasl
SHOULD just work (from a patch coming from Poky/Yocto, only lightly tested by 
me).

> 12:48:25 root[load: 0.00]@new-server /# rpm -q rpm
> rpm-5.4.16-0.1.i686
> 
> something logs to syslog at this time:
> Apr 24 12:48:27 new-server rpm: looking for plugins in '/usr/lib/sasl2', 
> failed to open directory, error: No such file or directory
> 

That message isn't coming from RPM afaik.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16-0.20160321.src.rpm, take 3

[Jeff Johnson]

On Apr 24, 2016, at 6:29 AM, Jan Rękorajski wrote:

> On Mon, 21 Mar 2016, Jeff Johnson wrote:
> 
>> There is a 3rd snapshot release of rpm-5.4.16 now available at
>> 
>>  
>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT//rpm-5.4.16-0.20160321.src.rpm
>> 
>> This addresses the following issues:
>>1) mongoc.h needed #include  to rebuild with PLD configuration
>>2) build with or without in "system.h"
>>#define SUPPORT_I18NSTRING_TYPE 1
>> 
>> The tarball has
>>#undef SUPPORT_I18NSTRING
>> which will display the Summary/Description/Group tags ifrom the header n the 
>> C locale aways,
>> and otherwise eliminates the RPM_I18NSTRING_TYPE interface.
> 
> Digging this one up.
> 
> Why are you so set on eliminating RPM_I18NSTRING_TYPE? I don't see any
> explanation anywhere and it will kill a lot of work we put in
> translating spec Summary/Descriptions.
> 

Short answer:
Patch the define and be happy. Everything "works" as before, none of 
your "work"
in *.spec files needs to be changed or is lost.

Longer answer:

I'm interested in removing (by abandoning) a problematic (sometimes 
RPM_I18NSTRING_TYPE
is a string, sometimes its an array, dependent on context) data type in RPM 
headers.

There are deeper/harder issues with specifying an encoding for 
RPM_I18NSTRING_TYPE in a spec file
as well. Already there are some deep hacks in RPM to translate what is usually 
(PLD is the only distro
I know of that specifies encoding explicitly) implicitly dependent on a 
LANG/LC_ALL envvar.

The processes of "building" and "translating" are rather different as well. 
Most distros (unlike PLD)
have some awkward "string freezes" and "string merges" as part of their release 
cycle that synchronize
the steps needed to hand data to translation teams, and to put translated 
strings into packages. I
am interested in making these processes more independent of each other, with an 
alternative
means of distribution and associating Group/Summary/Description with packages. 
That includes
handling RPMTAG_FILELANGS differently as well.

I think that I18N could (and should) be done better in RPM. Meanwhile devising 
a transparent
"legacy compatible" retrofit for RPM_I18NSTRING_TYPE is/was the necessary first 
step to attempting
a better implementation.

73 de Jeff

> Jan Rękorajski| PLD/Linux
> SysAdm | bagginspld-linux.org | http://www.pld-linux.org/
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Apr 20, 2016, at 11:05 AM, Elan Ruusamäe wrote:

> On 20.04.2016 18:01, Elan Ruusamäe wrote:
>> On 20.04.2016 15:38, Jeff Johnson wrote:
>>> There is a final snapshot release of rpm-5.4.16 now available at
>>> 
>>>http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160420.src.rpm 
>> 
>> so this did got finally addressed?
>> 
>> http://lists.pld-linux.org/mailman/pipermail/pld-devel-en/2015-November/thread.html#24537
>>  
>> 
> sorry, the patch that pld has applied now is here:
> 
> https://github.com/pld-linux/rpm/blob/ff1a99fc4/do_not_write_before_macro_buffer.patch
> 

Yes fixed.

73 de jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

There is a final snapshot release of rpm-5.4.16 now available at

http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160420.src.rpm

Unless I screwed something at the last minute, this SRPM will be released as 
rpm-5.4.16
later this week.

Since the last snapshot, there are several important changes

1) colorized rpm error messages.

2) *.rpm package reading has been hardened using american fuzzy lop on the 
command
rpm -qp --nomanifest minimal*.rpm
and all issues have been fixed. The current run has survived 400M forks.


3) configure options to enable link time optimization (-flto) and run-time
checking (-fsanitize=address et al) have been added.

73 de Jeff

On Mar 15, 2016, at 4:50 PM, Jeff Johnson wrote:

> There is a snapshot release of rpm-5.4.16 now available at
> 
>
> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm
> 
> This is the first SRPM built by itself that is headed for release
> in the next few weeks that is being provided as a public reference
> point for integration and portability testing.
> 
> See the included INSTALL document for the build pre-requisite versions used.
> 
> From a distro POV, please note the following changes that are included
> in the snapshot that will (at least) need to be considered when upgrading:
> 
>1) (recommended) rpm-5.4.16 uses BLAKE2bp for file digests.
>  BLAKe2bp is a 256bit digest that is faster than SHA256 (and MD5)
>  that will improve installation speeds.
> 
>  Details are here:
>https://blake2.net
> 
>2) (recommended) rpm-5.4.16 uses libtomcrypt (rather than BeeCrypt).
>  LibTomCrypt has support for ECDSA and is used by recent python and
>  the linux kernel (iirc).
> 
>  Details are here:
>https://github.com/libtom/libtomcrypt
> 
>3) (recommended) rpm-5.4.16 uses db-6.1.23 (not 6.1.26) with
>  DB_MULTIVERSION and DB_TXN_SNAPSHOT.
>  DB_TXN_SNAPSHOT avoids deadlocks with copy-on-write rather than
>  locking semantics.
> 
>  The change is necessary to support nested transactional commits
>  in rpm like
>command transaction
>   package transaction
>  install transaction
>  erase transaction
>  without deadlocking on trigger lookups.
> 
>  Details about DB_MULTIVERSION and DB_TXN_SNAPSHOT can be found
>  in the Oracle Berkeley DB documentation here:
>http://docs.oracle.com/cd/E17076_04/html/index.html
> 
> As always, rpm can be configured to use any of ~120 digests, any of
>BeeCrypt
>NSS
>Openssl
>Libgcrypt
>LibTomCrypt
> and (most likely, unchecked) any version of Berkeley DB back to db-4.6.x.
> 
> Bug reports are requested at
>https://launchpad.net/rpm
> 
> Patches and discussion are requested at
><rpm-de...@rpm5.org>
> 
> Enjoy!
> 
> 73 de Jeff
> 

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm loses %configs

[Jeff Johnson]

On Apr 19, 2016, at 4:59 PM, Elan Ruusamäe wrote:

> i'm pretty sure this has happened before, just did not payed attention
> 

I believe you. Meanwhile ...

> so, if i use --nodeps rpm forgets it has to put .rpmnew?
> and instead no config files left on the original path (mv -i would had said 
> that file is there)
> 

RPM verifies dependency assertions and then installs files from packages.

Adding --nodeps disables dependency assertion verification, while %config file 
resolution
(like renaming to *.rpmnew) is computed while installing.

These actions are largely indpendent of each other, based on different
package data.

Something else is going on imho.

> there are no %file %config changes between the versions in the transaction
> 

And the %config  file resolution in rpm has not changed in many years, like
all of this century.

> don't want to re-test this, but seems like it
> 

I cannot do engineering based on anecdotal inattention to details.

The minimum necessary information necessary for a rpm bug report about %config 
needs
a reproducer with rpm, not poldek, using -vv to output the %config file 
renaming computed by
rpmlib and a clear statement of what specific behavior you are claiming that is
broken.

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: [packages/file] - this version breaks rpmbuild, good testcase is gnome-initial-setup package - rel 1.1

[Jeff Johnson]

On Apr 18, 2016, at 4:52 AM, Elan Ruusamäe wrote:

> 
> pld rpm links with system libmagic, not using bundled version. i think this 
> is something broken on library level.
> 

Perhaps the libmagic code is broken, yes. Usually the issue is new definitions, 
which
are increasingly using *RE patterns. If so, then using a known good version of
/usr/share/misc/../file/magic
might help.

Meanwhile, rpm has the undocumented --rpmmgdebug which should
display how rpm rpmbuild is using libmagic.

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: [packages/file] - this version breaks rpmbuild, good testcase is gnome-initial-setup package - rel 1.1

[Jeff Johnson]

On Apr 17, 2016, at 9:44 AM, Jan Rękorajski wrote:

> I removed it from ftp and downgraded on builders, building
> gnome-initial-setup or jbig2dec causes rpm to hang indefinitely on
> "processing files" step.
> 

The coupling between rpmbuild <-> file is through /etc/magic rules
(whatever path is used these days) and libmagic.

One way to break the coupling is to put a minimalistic/deterministic
copy of magic in /usr/lib/rpm/magic and reconfigure the macro used
to find the magic file path.

Since identifier strings from /etc/magic end up in *.rpm headers,
and invokes helper scripts based on magic keywords,
there are other "reproducible build" reasons to internalize a
minimalistic/deterministic portable version of /etc/magic.

Should rpm catty a copy of magic in /usr/lib/rpm/magic?

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16-0.20160321.src.rpm, take 3

[Jeff Johnson]

On Mar 23, 2016, at 5:37 PM, Elan Ruusamäe wrote:

> On 22.03.2016 01:44, Jeff Johnson wrote:
>> There is a 3rd snapshot release of rpm-5.4.16 now available at
>> 
>>  
>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT//rpm-5.4.16-0.20160321.src.rpm
> 
> it compiled, so committed [1]
> 

Good.

Is there any reason I shouldn't release rpm-5.4.16 this week?

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

rpm-5.4.16-0.20160321.src.rpm, take 3

[Jeff Johnson]

There is a 3rd snapshot release of rpm-5.4.16 now available at


http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT//rpm-5.4.16-0.20160321.src.rpm

This addresses the following issues:
1) mongoc.h needed #include  to rebuild with PLD configuration
2) build with or without in "system.h"
#define SUPPORT_I18NSTRING_TYPE 1

The tarball has
#undef SUPPORT_I18NSTRING
which will display the Summary/Description/Group tags ifrom the header n the C 
locale aways,
and otherwise eliminates the RPM_I18NSTRING_TYPE interface.

The tarball has been checked by building with a lightly modified PLD rpm.spec 
file from here:


http://git.pld-linux.org/gitweb.cgi?p=packages/rpm.git;a=tree;h=dev-5.4.16;hb=dev-5.4.16

and examining the built package metadata content using

rpm -qip *.rpm | more

and

 rpm -qp --yaml *.rpm | more

to ensure that Summary/Description/Group display (and were as expected) with or 
without
SUPPORT_I18NSTRING_TYPE

For PLD, you will need to
1) patch system.h to add
#define SUPPORT_I18NSTRING_TYPE 1
2) don't apply the patch
rpm-dont-copy-descriptive-tags.patch

hth

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot, take 2

[Jeff Johnson]

(aside)
I've added Reply-To: <rpm-de...@rpm5.org> which is likely the better forum,
and because I did not receive this msg for whatever reason:
rpm-5.4.16 snapshot, take 2

> i wish it would be possible to disable building cruft that is not used. 
> that is tests (as we do not run tests) and some weird and perhaps 
> experimental tools like b2sum. pld intention is not to do some build 
> coverage, but just get /bin/rpm compiled and packaged
Somehow I have to release one rpm-5.4.16 tarball, not one for every possible 
distro
purpose. The issue with missing va_start has nothing to do with "cruft", but 
rather
that librpmio is misbuilt on PLD somehow: ensuring that
#include 
happens will surely be the fix. Can you find which .o is not including 
?
Or make the actual failed build available so that I can diagnose?

> and the text fields still contain garbage:
As I tried to say below, PLD and rpm5.org are likely going to differ on 
removing the
RPM_I18NSTRING_TYPE =  9
data type from rpm headers.
I explicitly verified that Summary/Group/Description could be displayed
using both rpm-4.13.0 (as in Fedora 23), and rpm-5.4.16, in both
LANG=en_US.UTF-8
and
LANG=fr_FR
with the underlying switch disabling the change in system.h:
/**
 * Eliminate RPM_I18NSTRING_TYPE.
 */
#define SUPPORT_I18NSTRING_TYPE 1
(which is what PLD is likely going to use).
I will check the other value of SUPPORT_I18NSTRING_TYPE next.

hth

73 de Jeff



On Mar 18, 2016, at 3:49 AM, Jeff Johnson wrote:

> I have uploaded another snapshot release that addresses
> all the issues you have reported:
> 
>   rpm-5.4.16-0.20160318.src.rpm
> 
> The issue of garbled text is both locale and rpm version dependent
> and is _NOT_ reproducible with rpm-5.4.16, but is reproducible
> with rpm-4.13.0 when a locale is specified.
> 
> PLD and rpm5.org are likely to disagree on the issue of including
> I18N in spec files and in headers no matter what.
> 
> I have checked that rpm can be built using --without-openssl
> and/or --without-acl.
> 
> The linkage problem with rpmio/tgfs which uses shm_open(3) has to do
> with a missing -lrt which may be implicitly specified by some other
> configure option, perhaps OPENMP.
> 
> Meanwhile I have hardwired -lrt for tgfs ldflags. The other issues I had with
> tgfs (a test program for GridFS using MongoDB) had to do with 
> misconfiguration,
> not anything else. There is a need for an error message which I will add 
> shortly.
> 
> hth
> 
> 73 de Jeff
> 
> On Mar 15, 2016, at 6:14 PM, Elan Ruusamäe wrote:
> 
>> On 15.03.2016 22:27, Jeff Johnson wrote:
>>> There is a snapshot release of rpm-5.4.16 now available at
>>> 
>>>
>>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm
>> it's text fields seem to contain garbage:
>> 
>> [~/rpm/packages/rpm(5.4.15) (dev-5.4.16)⚡] ➔ less 
>> rpm-5.4.16-0.20160315.src.rpm
>> Name: rpm  Relocations: (not relocatable)
>> Version : 5.4.16Vendor: (none)
>> Release : 0.20160315Build Date: T 15 märts 2016 
>> 21:13:03 EET
>> Install Date: (not installed)Build Host: hi.jbj.org
>> Group   : (^E^B  Source RPM: (none)
>> Size: 20861253     License: LGPLv2
>> Signature   : RSA/SHA1, T 15 märts 2016 21:13:03 EET, Key ID fd73f8a36edb4910
>> URL : http://rpm5.org
>> Summary : <88>^F^B
>> Architecture: x86_64
>> Description :
>> ^F^B
>> * K märts 16 2016 Jeff Johnson <j...@rpm5.org> -5.4.16
>> - improve uncoupling from rpmbuild configuration.
>> 
>> 
>> anyway, will push soon dev-5.4.16 branch in pld repo [1] for anyone wishing 
>> to test it
>> 
>> ps: tarball size increased 5mb:
>> 
>> [~/rpm/packages/rpm(5.4.16) (dev-5.4.16)⚡] ➔ l rpm-5.4.*gz
>> -rw-r- 1 glen users 15M 25. okt2014 rpm-5.4.15.tar.gz
>> -rw-r--r-- 1 glen users 20M 15. märts 21:04 rpm-5.4.16.tar.gz
>> 
>> 
>> [1] http://git.pld-linux.org/packages/rpm.git & 
>> https://github.com/pld-linux/rpm
>> 
>> -- 
>> glen
>> 
>> ___
>> pld-devel-en mailing list
>> pld-devel-en@lists.pld-linux.org
>> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Mar 16, 2016, at 12:46 PM, Elan Ruusamäe wrote:

> On 16.03.2016 18:28, Jeff Johnson wrote:
>> There is a recent API change in libtomcrypt (like last few months)
> 
> ah, tomcrypt,
> 
> i think will switch to previous method in pld for now for maximum 
> compatibility,
> release manager (baggins) decides if pld follows different defaults.
> 

Yes you can switch back to BeeCrypt rather than libtomcrypt: change the
order that crypto will be used at the top of rpmio/rpmpgp.c.

You will lose ECDSA support with BeeCrypt only rpm.

73 de Jeff

> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Mar 16, 2016, at 12:12 PM, Elan Ruusamäe wrote:

> On 15.03.2016 22:27, Jeff Johnson wrote:
>> There is a snapshot release of rpm-5.4.16 now available at
>> 
>> 
>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm
> 
> have not yet researched from what this error comes, maybe quicklier just to 
> ask:
> 

Yes.

There is a recent API change in libtomcrypt (like last few months)

The older name is LTC_LTC_PKCS_1_V1_5

There is an attempt to "fix" the issue like this
#if defined(LTC_LTC_PKCS_1_V1_5)
int _padding = LTC_LTC_PKCS_1_V1_5; /* XXX RSA */
#else
int _padding = LTC_PKCS_1_V1_5; /* XXX RSA */
#endif

but LTC_LTC_PKCS_1_V1_5 may not be defined (its an enum)

I'll devise a better test for portability ... meanwhile use LTC_LTC_PKCS_1_V1_5 
insead

Thanks for the report.

> 
> libtool: compile:  ccache gcc -DHAVE_CONFIG_H -I. -I.. -I. -I.. -I../build 
> -I../lib -I../lib -I../rpmdb -I../rpmio -I../misc -I../lua/local 
> -I../lua/local -I../lua -I../lua -DRPM_OS_LINUX=040116 -DLTM_DESC -DUSE_LTM 
> -I/usr/include/db5.2 -I/usr/include/ossp-uuid -I/usr/include/ossp-uuid 
> -fopenmp -O2 -fwrapv -pipe -Wformat -Werror=format-security -gdwarf-4 
> -fno-debug-types-section -fvar-tracking-assignments -g2 
> -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4 
> -fPIC -march=x86-64 -gdwarf-4 -fno-debug-types-section 
> -fvar-tracking-assignments -g2 -D_GNU_SOURCE -D_REENTRANT -MT rpmltc.lo -MD 
> -MP -MF .deps/rpmltc.Tpo -c rpmltc.c  -fPIC -DPIC -o .libs/rpmltc.o
> rpmltc.c: In function 'rpmltcVerify':
> rpmltc.c:385:20: error: 'LTC_PKCS_1_V1_5' undeclared (first use in this 
> function)
> int _padding = LTC_PKCS_1_V1_5; /* XXX RSA */
>^
> rpmltc.c:385:20: note: each undeclared identifier is reported only once for 
> each function it appears in
> rpmltc.c: In function 'rpmltcSign':
> rpmltc.c:454:20: error: 'LTC_PKCS_1_V1_5' undeclared (first use in this 
> function)
> int _padding = LTC_PKCS_1_V1_5; /* XXX RSA */
>^
> Makefile:2155: recipe for target 'rpmltc.lo' failed
> make[4]: *** [rpmltc.lo] Error 1
> 
> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Mar 16, 2016, at 1:09 PM, Elan Ruusamäe wrote:

> On 15.03.2016 22:27, Jeff Johnson wrote:
>> 3) (recommended) rpm-5.4.16 uses db-6.1.23 (not 6.1.26) with
>>   DB_MULTIVERSION and DB_TXN_SNAPSHOT.
>>   DB_TXN_SNAPSHOT avoids deadlocks with copy-on-write rather than
>>   locking semantics.
> 
> so, if i use db-5.2, these changes be reverted in DB_CONFIG ?
> 
> 
> --- ../rpm-5.4.15/rpmdb/DB_CONFIG.in2016-03-16 17:36:37.569145832 +0200
> +++ ../rpm-5.4.16/rpmdb/DB_CONFIG.in2016-03-16 18:48:10.648775378 +0200
> @@ -1,10 +1,12 @@
> #  Environment
> -#add_data_dir .
> set_data_dir .
> -set_create_dir .
> set_lg_dir ./log
> set_tmp_dir ./tmp
> 
> +set_flags DB_MULTIVERSION on
> +set_flags DB_AUTO_COMMIT on
> +set_flags DB_TXN_SNAPSHOT on
> +
> 

Depends on whether db-5.2.x supports configuring those flags in DB_CONFIG.

(aside)
There are 3 different ways to configure BerkeleyDB
1) hard-wired in C
2) using /usr/lib/rpm/macros to pass in the appropriate info
3) using DB_CONFIG which is the "official" (and documented) method to 
configure

I have done both 2) and 3) and there are almost no instances of hard-wired in 
rpm

Using DB_CONFIG to configure an rpmdb is the preferred approach because 
"ofiical" and documented.

Meanwhile try-and-see on db-5.2.x ... if you run into issues, holler and I'll 
figure a solution for you.

I'd suggest trying db-6.1.23 if you want exactly the same behavior as 
rpm-5.4.16 is being testesd with

73 de Jeff
> 
> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot, take 2

[Jeff Johnson]

I have uploaded another snapshot release that addresses
all the issues you have reported:

rpm-5.4.16-0.20160318.src.rpm

The issue of garbled text is both locale and rpm version dependent
and is _NOT_ reproducible with rpm-5.4.16, but is reproducible
with rpm-4.13.0 when a locale is specified.

PLD and rpm5.org are likely to disagree on the issue of including
I18N in spec files and in headers no matter what.

I have checked that rpm can be built using --without-openssl
and/or --without-acl.

The linkage problem with rpmio/tgfs which uses shm_open(3) has to do
with a missing -lrt which may be implicitly specified by some other
configure option, perhaps OPENMP.

Meanwhile I have hardwired -lrt for tgfs ldflags. The other issues I had with
tgfs (a test program for GridFS using MongoDB) had to do with misconfiguration,
not anything else. There is a need for an error message which I will add 
shortly.

hth

73 de Jeff

On Mar 15, 2016, at 6:14 PM, Elan Ruusamäe wrote:

> On 15.03.2016 22:27, Jeff Johnson wrote:
>> There is a snapshot release of rpm-5.4.16 now available at
>> 
>> 
>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm
> it's text fields seem to contain garbage:
> 
> [~/rpm/packages/rpm(5.4.15) (dev-5.4.16)⚡] ➔ less 
> rpm-5.4.16-0.20160315.src.rpm
> Name: rpm  Relocations: (not relocatable)
> Version : 5.4.16Vendor: (none)
> Release : 0.20160315Build Date: T 15 märts 2016 
> 21:13:03 EET
> Install Date: (not installed)Build Host: hi.jbj.org
> Group   : (^E^B  Source RPM: (none)
> Size: 20861253 License: LGPLv2
> Signature   : RSA/SHA1, T 15 märts 2016 21:13:03 EET, Key ID fd73f8a36edb4910
> URL : http://rpm5.org
> Summary : <88>^F^B
> Architecture: x86_64
> Description :
> ^F^B
> * K märts 16 2016 Jeff Johnson <j...@rpm5.org> -5.4.16
> - improve uncoupling from rpmbuild configuration.
> 
> 
> anyway, will push soon dev-5.4.16 branch in pld repo [1] for anyone wishing 
> to test it
> 
> ps: tarball size increased 5mb:
> 
> [~/rpm/packages/rpm(5.4.16) (dev-5.4.16)⚡] ➔ l rpm-5.4.*gz
> -rw-r- 1 glen users 15M 25. okt2014 rpm-5.4.15.tar.gz
> -rw-r--r-- 1 glen users 20M 15. märts 21:04 rpm-5.4.16.tar.gz
> 
> 
> [1] http://git.pld-linux.org/packages/rpm.git & 
> https://github.com/pld-linux/rpm
> 
> -- 
> glen
> 
> ___
> pld-devel-en mailing list
> pld-devel-en@lists.pld-linux.org
> http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm-5.4.16 snapshot

[Jeff Johnson]

On Mar 16, 2016, at 3:20 PM, Elan Ruusamäe wrote:

> On 15.03.2016 22:27, Jeff Johnson wrote:
>> There is a snapshot release of rpm-5.4.16 now available at
>> 
>> 
>> http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm
> 
> some rpmio/* tools link errors. hack-fixed by adding -lssl -lacl to 
> RPMIO_LDADD_COMMON

Not the best hack ;-)

Meanwhile now that I can see the configure options from the PLD rpm.spec,
I have tried rebuilding with this (fairly close) approximation to what you want
on Fedora 23 (with all build deps already installed):

./configure \
  --verbose \
  --prefix=/usr \
  --infodir='${prefix}%{__share}/info' \
  --libdir='%{_libdir}' \
  --localstatedir=/var \
  --mandir='${prefix}%{__share}/man' \
  --sysconfdir=/etc \
  --enable-shared \
  --enable-static \
  --without-apidocs \
  --with-beecrypt=external \
  --with-bugreport="http://bugs.pld-linux.org/; \
  --with-bzip2=external \
  --with-db=external \
  --with-dbapi=db \
  --with-file=external \
  --with-keyutils=external \
  --with-libelf \
  --with-lua=internal \
  --with-lzma=external \
  --with-neon=external \
  --without-path-versioned \
  --with-pcre=external \
  --with-popt=external \
  --with-python=2.7 \
  --with-selinux=no \
  --with-semanage=no \
  --with-sepol=no \
  --with-sqlite=no \
  --with-uuid=system \
  --with-vendor=pld \
  --with-xz=external \
  --with-zlib=external \
  --with-pthreads \
  --with-openssl \
  --with-acl \
  --enable-build-pic \
  --enable-build-versionscript \
  --enable-build-warnings \
  --enable-build-debug \
  --enable-maintainer-mode

This configuration almost builds: there is an error linking tgfs.c (which is 
already on my fixit list)
with this error
/usr/bin/ld: tgfs.o: undefined reference to symbol 
'shm_open@@GLIBC_2.2.5' /usr/lib64/librt.so.1: 
error adding   symbols: DSO missing from command line collect2: 
error: ld returned 1 exit status

The easiest hack to build from the SNAPSHOT tarball is to stub out tgfs.c by 
copying this onto rpmio/tgfs.c

   int main() { return 0; }

after unpacking.

I will have a better fix for tgfs.c soonishly.

Meanwhile the above configuration SHOULD build in PLD, modulo packaging and 
bcond's.

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

rpm-5.4.16 snapshot

[Jeff Johnson]

There is a snapshot release of rpm-5.4.16 now available at

http://rpm5.org/files/rpm/rpm-5.4/SNAPSHOT/rpm-5.4.16-0.20160315.src.rpm

This is the first SRPM built by itself that is headed for release
in the next few weeks that is being provided as a public reference
point for integration and portability testing.

See the included INSTALL document for the build pre-requisite versions used.

>From a distro POV, please note the following changes that are included
in the snapshot that will (at least) need to be considered when upgrading:

1) (recommended) rpm-5.4.16 uses BLAKE2bp for file digests.
  BLAKe2bp is a 256bit digest that is faster than SHA256 (and MD5)
  that will improve installation speeds.

  Details are here:
https://blake2.net

2) (recommended) rpm-5.4.16 uses libtomcrypt (rather than BeeCrypt).
  LibTomCrypt has support for ECDSA and is used by recent python and
  the linux kernel (iirc).

  Details are here:
https://github.com/libtom/libtomcrypt

3) (recommended) rpm-5.4.16 uses db-6.1.23 (not 6.1.26) with
  DB_MULTIVERSION and DB_TXN_SNAPSHOT.
  DB_TXN_SNAPSHOT avoids deadlocks with copy-on-write rather than
  locking semantics.

  The change is necessary to support nested transactional commits
  in rpm like
command transaction
   package transaction
  install transaction
  erase transaction
  without deadlocking on trigger lookups.

  Details about DB_MULTIVERSION and DB_TXN_SNAPSHOT can be found
  in the Oracle Berkeley DB documentation here:
http://docs.oracle.com/cd/E17076_04/html/index.html

As always, rpm can be configured to use any of ~120 digests, any of
BeeCrypt
NSS
Openssl
Libgcrypt
LibTomCrypt
and (most likely, unchecked) any version of Berkeley DB back to db-4.6.x.

Bug reports are requested at
https://launchpad.net/rpm

Patches and discussion are requested at


Enjoy!

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: popt warning

[Jeff Johnson]

On Feb 23, 2016, at 2:21 PM, Elan Ruusamäe wrote:

> 
> i first noticed it in a code that was invoked from in bash prompt:
> 
> rpm --define "_specdir $PWD" --specfile $specfile -q --qf '%{VERSION}\n' | 
> head -n1
> 
> but later i noticed rpm/rpmbuild (invoked from pld builder script) the same 
> message was displayed.
> 
> 
> i don't  think anything else happened on system when i ran into this, but i 
> can't be sure and it's not happening right now to debug anything further.
> 

Try running (in a loop if non-reproducible) with strace is about all I can 
suggest.

The behavior of rpm+popt will be dependent on other coincident system actions.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: popt warning

[Jeff Johnson]

On Feb 22, 2016, at 5:23 PM, Elan Ruusamäe wrote:

> sometimes i see such message when building packages:
> 
> warning: existing POPT configuration file 
> "/usr/lib/rpm/rpmpopt:/usr/lib/rpm/%{_target}/rpmpopt:/etc/rpm/rpmpopt.*:/etc/rpm/rpm
> popt:/etc/rpm/%{_target}/rpmpopt:~/.rpmpopt" considered INSECURE -- not loaded
> 
> not always, i.e building same package i sometimes see this and sometimes do 
> not.
> rpm-5.4.15-31.x86_64
> 
> any ideas what this is about?

The intent is/was to have poptSaneFile() check permissions on files being read
(non-zero is an error). The mechanism is dependent on adding a leading '@'
to a file path.

int poptSaneFile(const char * fn)
{
struct stat sb;
uid_t uid = getuid();

if (stat(fn, ) == -1)
return 1;
if ((uid_t)sb.st_uid != uid)
return 0;
if (!S_ISREG(sb.st_mode))
return 0;
if (sb.st_mode & (S_IWGRP|S_IWOTH))
return 0;
return 1;
}

There may be other returns (from failure to read a file) in poptReadFile() that 
may trigger the message.


> and it appeared rather recently...
> 

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: packages: etckeeper/etckeeper.spec - Up to 0.56

[Jeff Johnson]

On Aug 24, 2011, at 8:26 AM, Pawel Golaszewski wrote:

 
 Maybe each file in filesystem? :P
 

That won't work: one needs to put every byte in a virtualized
environment with full hardware enforced isolation to please
everyone.

And RPM will *still* have extraordinary abilities to be able
to continue same old, same old, installations.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 3:44 AM, Jacek Konieczny wrote:

 
 Ok, you may be right. But what than? We don't even have an idea how
 should the proper rpm solution work and no volunteers to design and code
 it. At the same time Python 3.2 is waiting for being included in PLD,

Um, that isn't true: I did just ask for a statement of
what is desired with __pycache__ handling for the purpose
of attempting to design/code a solution in RPM.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 4:35 AM, Patryk Zawadzki wrote:
 
 In case some automated mechanism is provided, in the future, for keeping
 the compiled python code files in the RPM database without listing them
 in the spec files, we loose nothing by having the files there already
 (by explicitly listing them in the specs).
 
 Can we plug into cpython's/pypy's module loader to detect when it
 compiles stuff? If so, can we then launch a suid helper that injects
 the newly created files into the rpm package that contained the
 original .py?
 

This is essentially a push strategy, where python compilation
is forced to undertake package management.

The design problem is that compilation is per-file, while
registered packages are containers of files. You will
end up with a package-per-file, and/or some extension
to a package container to attach compiled files.

You also need to decide whether all compilations are on
the build system or after installation: there are different
implementations.

I'd suggest that the problem needs to be solved on the install
not the build system as has been traditionally attempted.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 7:17 AM, Tomasz Pala wrote:

 On Thu, Jul 14, 2011 at 10:35:17 +0200, Patryk Zawadzki wrote:
 
 compiles stuff? If so, can we then launch a suid helper that injects
 the newly created files into the rpm package that contained the
 original .py?
 
 OMG...
 Please don't mess with rpm database in indeterministic time (like some
 runtime) and do NOT use SUID.
 
 This is excelent example of what I've called 'overcomplicated'.
 

And you comments (which I agree with) are negatively phrased.

State what you want to see positively. Or prepare yourself for
the SUID audit that will inevitably be attempted.


73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 8:04 AM, Tomasz Pala wrote:

 On Thu, Jul 14, 2011 at 07:21:35 -0400, Jeff Johnson wrote:
 
 compiles stuff? If so, can we then launch a suid helper that injects
 the newly created files into the rpm package that contained the
 original .py?
 
 OMG...
 Please don't mess with rpm database in indeterministic time (like some
 runtime) and do NOT use SUID.
 
 This is excelent example of what I've called 'overcomplicated'.
 
 
 And you comments (which I agree with) are negatively phrased.
 
 State what you want to see positively.
 
 1. I don't know python internals, but if these cache files are not
 strictly machine-dependant (i.e. they don't differ on machines having
 the same arch) they should be shipped in compiled form (otherwise we
 might as well end up with %_make; %configure; ... in %build of every
 package) - this is not gentoo.
 

Historically, the *.py[co] files have been packaged in *.rpm as if the
content is both per-arch and per-python-version. I don't know what
the implementation reality actually is, or whether the choices are unnecessarily
restrictive and overly paranoid future proofing or …

.. but if the *.py[co] are portable (whatever that means), then
the portability should be reflected in the design of the packaging.

There are aspects of eggs, where eggs can load other eggs dynamically,
that are very different from other static content delivered in *.rpm,
and the dynamic side effects are purely on the install system and
are not easily captured on the build system. E.g. what other eggs
might be installed as a side effect likely has dynamic state associated
that cannot be reproduced within a build for all usage cases.


 2. if anything wants to alter rpm database, it must be done during
 transaction or shortly after it (i.e. within installation process); it's
 not acceptable to do changes there during regular system use (on
 unprivilidged account) - after all it's MD5 repo and might be
 hard-locked by admin.
 

Well that isn't necessarily true or even needed (depending on what
is attempted).

Headers can be digitally signed, and mostly do not contain any critically
important information that can't be found in the original *.rpm file.

There are a few (mostly timestamps) pieces of information that can
be assigned only during a transaction, but that does NOT mean
within installation process necessarily.

It would mean that any attempt to register a package in an rpmdb
SHOULD
1) carry digests or attempt signing
2) faithfully include a transaction id and a installation time
just like an RPM installation does.

And yes -- if you choose to treat an rpmdb with concepts like
locked and privileged -- then other installers will be
constrained in what what is permitted.

Both locked and privileged can be avoided if an rpm tool
which is already privileged pulls information rather than having
the python JIT'er push directly into an rpmdb.

This introduces a TOCTTOU window
http://en.wikipedia.org/wiki/Time-of-check-to-time-of-use
between when an egg is installed (and might queue a request for rpmdb 
registration)
and when the registration is actually handled. There are several approaches
to avoiding TOCTTOU queueing latency if you think a bit.

 3. install - compile - save to rpmdb is pointless - saved MD5 might be 
 already
 compromised by some malicious software and thus undetectable even via
 verification run from outer source (i.e. rescuecd). Checksums must be
 calculated in clean environment and be comparable between different
 systems (so we're back to point 1).
 

And the TOCTTOU window can be eliminated by
1) including the digests in the queued request
2) signing the queued request
and having RPM undertake verification before servicing the rpmdb
registration request.

Other issues like clean and safe can only be determined by
examining explicit thread models or auditing an actual implementation.

 
 Otherwise we might just put some
 rm -f /usr/share/python/**/__pycache__/* to cron or %post of python
 pkgs to ensure that no malicious/leftovers will harm our system. If the
 files are to be regenerated by regular user, why not remove them once a
 week to save space?
 

That's one approach to caching:
Don't do it! Disk space is expensive!

 Or prepare yourself for
 the SUID audit that will inevitably be attempted.
 
 SUID is broken by design and obsoleted (and disabled entirely on some
 machines). Are we talking about CAP_DAC_OVERRIDE?
 

CAP_DAC_OVERRIDE isn't _THAT_ much better than SUID imho. But
you can advocate whatever security scheme you wish: CAP_DAC_OVERRIDE
is certainly finer grained privilege access than SUID is.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 8:05 AM, Tomasz Pala wrote:

 On Thu, Jul 14, 2011 at 07:19:34 -0400, Jeff Johnson wrote:
 
 You also need to decide whether all compilations are on
 the build system or after installation: there are different
 implementations.
 
 I'd suggest that the problem needs to be solved on the install
 not the build system as has been traditionally attempted.
 
 Do you want to skip them during repackage?
 

(aside)
Do you actually use repackaged packages? For what purpose?

There are solutions here as well, depending on what is desired.

One can view side-effects like JIT'ed cached files as
either disposable and easily regenerated (in which case
the content should not be repackaged) or you can view
the JIT'ed cached files as stageful and persistent
(in which case repackaging should include the content).

All depends on POV and expectations.

And since its highly likely that noone can state
the desired POV precisely, then
Have it your own way!
as an engineering solution would need to be attempted.

The two obvious resolutions to cached files are
Include them in repackaging.
and
Don't include in repackaging.
and that's essentially an if statement somewhere.

*shrug*

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 9:23 AM, Tomasz Pala wrote:

 On Thu, Jul 14, 2011 at 08:57:40 -0400, Jeff Johnson wrote:
 
 (aside)
 Do you actually use repackaged packages? For what purpose?
 
 Unfortunately yes, for restoring working set - too often...
 And during tests of course (various, like my private builds or bug
 hunting) [*].
 

Well its *IS* you unfortunate choice to use --repackage (or not).
You have none but yourself to blame for your misfortunes.

Hint: Repackage packages were _DELIBERATELY_ poisoned by
adding RPMTAG_REMOVETID in order to prevent morons from
install - modify - re-package - re-publish - blame @redhat
a *.rpm as if it were built reproducibly and official.

Your unfortunately is likely mostly alleviated by simply
Don't poison repackaged *.rpm by adding RPMTAG_REMOVETID.
I.e -- for most packages that are never ever modified --
you will find that repackaged *.rpm packages start to pass
rpm -Kvv repackaged*.rpm
tests instantly.

There's little that can be done about modifications to %config
and the fundamental design flaw
You can't repackage a file after rm /path/to/file has been done.

But that lack of functionality WILL give you years of RFE's and complaints:

RPM sux because …

 There are solutions here as well, depending on what is desired.
 
 One can view side-effects like JIT'ed cached files as
 either disposable and easily regenerated (in which case
 the content should not be repackaged) or you can view
 the JIT'ed cached files as stageful and persistent
 (in which case repackaging should include the content).
 
 I ask because I see that rpm stores original file list (with sizes and
 timestamps) in it's headers, regardless of actual content of cpio. It
 includes both modified and nonexistent files (especially %doc when excludedocs
 is set and ommitted %langs).

Not quite, --relocate stores both the original and the relocated paths.

And RPM uses cpio headers for exactly one purpose:

The file path in the cpio header is used to locate the
metadata in the RPM header.

 In this case cpio would contain more files than headers know about. What

And adding more files will fail to find _ALL_ information about
the added file, and the file will be skipped (iirc, been years, I fergit)

This is/was by intent: the advantage to RPM packaging is that _EVERYTHING_
necessary to perform an install is encapsulated in a digitally signed
blob that is essentially immutable. Any/all modifications to that blob
are detectable.

 might happen with recompilation on different timestamps during downgrade or 
 some other
 weird operations like --noscripts?
 

recompilation != immutable

And only immutable will do: there's lots of TOCTTOU issues with
rebuilding as well, honking about time stamps is a mere fig leaf.

 
 *  I had an idea once upon a time to verify content of repackaged files
   against original digest, I really miss this feature in rpm (rpm -Vp
   verifies package against filesystem not internal cpio).
 

Remove the RPMTAG_REMOVETID poisoning and repackaging (for most packages)
becomes an exact inverse and file digests can then be verified just
like any other package.

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 11:10 AM, Tomasz Pala wrote:

 On Thu, Jul 14, 2011 at 17:04:02 +0200, Tomasz Pala wrote:
 
 *  I had an idea once upon a time to verify content of repackaged files
  against original digest, I really miss this feature in rpm (rpm -Vp
  verifies package against filesystem not internal cpio).
 
 Remove the RPMTAG_REMOVETID poisoning and repackaging (for most packages)
 becomes an exact inverse and file digests can then be verified just
 like any other package.
 
 By rpm binary? How?
 
 For example, I got original file (currently installed):
 
 ~:  rpm -Kvv iceweasel-3.6.17-1.i686.rpm 
 D: Expected size:  6683934 = lead(96)+sigs(920)+pad(0)+data(6682918)
 D:   Actual size:  6683934
 iceweasel-3.6.17-1.i686.rpm:
Header SHA1 digest: OK (b0509c227544d3d4860b484e8afde6eec28a051b)
MD5 digest: OK (a92e0a2dc52504d42959bb67741596f7)
 
 ~:  rpm -Vp iceweasel-3.6.17-1.i686.rpm 
 Unsatisfied dependencies for iceweasel-3.6.17-1.i686:
Requires: libhunspell-1.2.so.0
 
 S.5T   /usr/bin/iceweasel
 S.5T   /usr/lib/iceweasel/components/nsLoginManager.js
 S.5T   /usr/lib/iceweasel/components/storage-Legacy.js
 S.5T   /usr/lib/iceweasel/components/storage-mozStorage.js
 
 
 And I got some previous, repackaged file:
 
 /var/spool/repackage/1309337365#  rpm -Kvv iceweasel-3.6.13-2.i686.rpm
 D: Expected size:  7452100 = lead(96)+sigs(180)+pad(4)+data(7451820)
 D:   Actual size:  7506959
 iceweasel-3.6.13-2.i686.rpm:
Header SHA1 digest: OK (9ad4a3fb6c194ba14c5b85f20e8aa567bbce3764)

This indicates that someone has already removed the poisoning:
if RPMTAG_REMOVETID were present, then the header SHA1 would fail

MD5 digest: BAD Expected(51df700b3bfa11a73df0d1219eae9f97) != 
 (43b884021c55da3d87f9e76f83f5ceb4)
 

So the operation that needs doing is essentially this

mkdir -p XXX
cd XXX
rpm2cpio iceweasel-3.6.13-2.i686.rpm | cpio -dim
rpm -Vp --root `pwd` iceweasel-3.6.13-2.i686.rpm

which RPM will not do for you.

Personally I'd rather see RPM chill the shot glasses, pour the vodka, and
serve up peanuts and chips than bother chasing down which file in a repackaged
payload is actually modified. YMMV.

 
 How can I list files within the latter archive, that were modified
 before repackage (in this case I know they are exactly the same 3 as above).
 

See above:

Kazdy wypity kieliszek – to gwózdz do naszej trumny… Pijmy wiec tak by trumna 
sie nie rozpadla.

73 de Jeff
 -- 
 Tomasz Pala go...@pld-linux.org
 ___
 pld-devel-en mailing list
 pld-devel-en@lists.pld-linux.org
 http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 2:31 PM, Artur Wroblewski wrote:

 On Thu, Jul 14, 2011 at 8:44 AM, Jacek Konieczny jaj...@jajcus.net wrote:
 [...]
 We lose:
 - a little bit of the disk space
 - some 'purity' some people see in not distributing 'sources'
 
 IMHO, it was not about purity but quite practical aspect of
 file distribution (src vs. bin) - we treated byte compiled files
 for Java and Python the same way, no src allowed.
 

All depends on what you want to emphasize in the comparison.

E.g. man pages are generated and cached and none is
asking for support from RPM for handling ownership
and integrity of the generated man pages. The ached aspects
of __pycache__ aren't that different (but are executable so
the audit is more complex than for data files).

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 14, 2011, at 3:52 PM, Artur Wroblewski wrote:

 On Thu, Jul 14, 2011 at 8:26 PM, Jeff Johnson n3...@mac.com wrote:
 
 On Jul 14, 2011, at 2:31 PM, Artur Wroblewski wrote:
 
 On Thu, Jul 14, 2011 at 8:44 AM, Jacek Konieczny jaj...@jajcus.net wrote:
 [...]
 We lose:
 - a little bit of the disk space
 - some 'purity' some people see in not distributing 'sources'
 
 IMHO, it was not about purity but quite practical aspect of
 file distribution (src vs. bin) - we treated byte compiled files
 for Java and Python the same way, no src allowed.
 
 
 All depends on what you want to emphasize in the comparison.
 
 E.g. man pages are generated and cached and none is
 asking for support from RPM for handling ownership
 and integrity of the generated man pages. The ached aspects
 of __pycache__ aren't that different (but are executable so
 the audit is more complex than for data files).
 
 Sure. I just re-stated our approach we took in the past.
 

And no offense intended: truly there's a balance point to
handle __pycache__ registration here somehow/somwhere.

I merely wished to point out one of the extrema constraints:

Don't bother handling __pycache__ at all, treat like
cached man pages.

I don't know what the right/best answer is. I do trust PLD will
find a sane answer first. There's no other distro around attempting
python-3.x packaging that I've heard/seen yet.

73 de Jeff
 Best regards,
 
 w
 ___
 pld-devel-en mailing list
 pld-devel-en@lists.pld-linux.org
 http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.py packaging, again

[Jeff Johnson]

On Jul 13, 2011, at 9:07 AM, Jakub Bogusz wrote:

 
 I don't like the idea of __pycache__ not managed by rpm (or not

We likely disagree on the details, perhaps sharply, of
__pycache__ managed by rpm
but not on the general goal.

 maintained with rpm database in case of more robust post scripts)
 because of at least:
 - possible inconsistences (like leftovers in case of upgrade failures;
  in such case it would be even possible that after installing another
  version of package with *.py files having older date than pycache,
  pycache won't be rebuilt)
 - more work needed to find package owning __pycache__/* files
 - one more place to hide some malicious code not detectable by rpm -Va
 

If you can state positively (the above comments are all negative problem 
avoidance,
not positive feature seeking) what you would like to see for RPM management
of __pycache__, I'll try to get some implementation together in RPM.

If the chosen implementation is %post scrip tie, well, so be it. Some
attempt to manage __pycache__ is better than none. I do think much
better than %post scripting needs to be attempted: the edit to
add functionality in 10's to 100's of *.spec files converges
to a solution rather slowly.

hth

73 de Jeff


___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: Deps in TH

[Jeff Johnson]

On Jun 4, 2011, at 9:41 AM, Elan Ruusamäe wrote:

 On Fri, 03 Jun 2011 12:17:20 -0400, Jeff Johnson wrote:
 On Jun 3, 2011, at 11:55 AM, Arkadiusz Miskiewicz wrote:
 
 We don't have such thing. Feel free to implement (cvs/pld-ftp-admin) but 
 note
 that in needs to be fast (and speed is the major problem).
 
 
 Its rather easy to do
  rpm -ivh --dbpath /somewhere/rpmdb --justdb --nonothing *.rpm
 and then do
  rpm -Va --nofiles --dbpath /somewhere/rpmdb
 mechanically. Quite fast too.
 
 All that's hard there is getting the *.rpm to be _EXACTLY_ what you want.
 
 getting *.rpm to be all packages from distro, is trivial using existing 
 scripts, but as the packages conflict with each other (providing same 
 functionality) can't be installed at the same time, and figuring out such 
 conflicts is manual work
 

Note that Conflicts: with missing values are broken in RPM.

Note also that
rpm -Vp --nofiles --noscripts *.rpm
is an even simpler detection scheme.

So don't. Missing dependency detction is rather easy to automate, and is
the common complaint for aggregations of *.rpm packages. If you need
every detail perfectly verified instantly, then note that the assertion
... speed is the major problem.
isn't as big a problem as apathy is.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: Deps in TH

[Jeff Johnson]

On Jun 3, 2011, at 11:55 AM, Arkadiusz Miskiewicz wrote:
 
 We don't have such thing. Feel free to implement (cvs/pld-ftp-admin) but note 
 that in needs to be fast (and speed is the major problem).
 

Its rather easy to do
rpm -ivh --dbpath /somewhere/rpmdb --justdb --nonothing *.rpm
and then do
rpm -Va --nofiles --dbpath /somewhere/rpmdb
mechanically. Quite fast too.

All that's hard there is getting the *.rpm to be _EXACTLY_ what you want.

Yes there isn't a --nonothing option in rpm (but could be added with a popt 
alias).

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.la files in xorg* packages

[Jeff Johnson]

On May 30, 2011, at 1:45 PM, Arkadiusz Miskiewicz wrote:

 On Monday 30 of May 2011, Tomasz Pala wrote:
 While updating my system I've noticed that xorg* packages still got
 *.la files, while they all got *.pc as well.
 Are there any other (not discussed before) reasons not to remove them?
 Having *.pc was the main rule allowing to do this.
 
 No one was brave enough to rebuild all dependencies on ftp.pld- which would 
 be 
 required if la were dropped.
 

Is it merely a matter of bravery or is there still a need for *.la?

I'm asking the engineering, not the advocacy, question here:
Are *.la files useful?

I already know my non-linux answer: Yes.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: *.la files in xorg* packages

[Jeff Johnson]

On May 30, 2011, at 2:05 PM, Arkadiusz Miskiewicz wrote:

 On Monday 30 of May 2011, Jeff Johnson wrote:
 On May 30, 2011, at 1:45 PM, Arkadiusz Miskiewicz wrote:
 On Monday 30 of May 2011, Tomasz Pala wrote:
 While updating my system I've noticed that xorg* packages still got
 *.la files, while they all got *.pc as well.
 Are there any other (not discussed before) reasons not to remove them?
 Having *.pc was the main rule allowing to do this.
 
 No one was brave enough to rebuild all dependencies on ftp.pld- which
 would be required if la were dropped.
 
 Is it merely a matter of bravery or is there still a need for *.la?
 
 I'm asking the engineering, not the advocacy, question here:
  Are *.la files useful?
 
 Not useful for dynamic linking, only for static linking sometimes (when pc 
 files are not used).
 

Thanks (ant to Thomasz) for sane opinions. Re-vositing statically linked RPM
is most definitely one of my todo++ items (and means *.la for all rpm prereq's).

Does PLD attempt to reduce build/install prereq's by doing static linking?

E.g some real world examples I've had to deal w in last 10 days:

I'm working from serentos (RHEL6 clone) bolting on Fedorable devel
packaging, to establish a sane development platform.

There are two easy to express dependency hell snarls that might
be solved (there are other solutions) by static linking:

1) ossp-uuid - PHP - a_whole_lotta_stuff (this started as build prereq and
quickly becomes a hugely painful wrestling match)

2) bitbake - help2man,tex2html - a_whole_lotta_ancient_tetex
bitbake is the OpenEmbedded and Poky/Yacto rpmbuild without using rpmbuild
that I'm trying to cross-compile for arm5tel images.

(aside)
The ulterior motive in asking is that I'm considering writing the
dependency analysis tool(s) that try to identify the point at which
the transition ... - a_whole_lotta_stuff might be recognized
automatically. I'm quite sure there's enough sanity in PLD that the
tool(s) aren't critically needed. But _SOME_ objective measurement
is most definitely needed in Fedorable and Mandriva to help unsnarl
prerequsites and identify objectively where a_whole_lotta_stuff
starts to become annoyingly painful with *.rpm packaging.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 9, 2011, at 2:13 PM, Patryk Zawadzki wrote:

 2011/4/9 Elan Ruusamäe g...@pld-linux.org:
 so my vote goes for keeping the old .py[co] method, and perhaps change to
 make is package only .pyc, it is rather ratre somebody invokes python with
 -O option so the .pyo is to be needed. or why it was packaged in first
 place? to get files owned in case somebody invokes as root and .pyo gets
 created? perhaps package .pyc as regular files and .pyo as ghosted then.
 
 Not sure about PLD but I suppose we just followed what the others were
 doing. Other distros did it this way so they could set proper selinux
 attributes.
 

Yes. Attaching SElinux xattr's to 1+ M paths forced *.pyo to be included
everywhere/always in *.rpm.

That *was* 5+ years ago. There's no known reason why xattr's can't be done
in other ways. In fact SELinux does relabeling outside of rpm these
days, there's literally no reason that *.pyo MUST be included in *.rpm
5+ years later.

And 5+ years later SELinux hasn't really achieved the wide deployment desired,
packaging *.pyc and adapting to changing conventions in pyton  3.2 wasn't
even conceivable 5+ years ago.

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 9, 2011, at 3:29 PM, Tomasz Pala wrote:

 On Sat, Apr 09, 2011 at 15:26:46 -0400, Jeff Johnson wrote:
 
 There's no known reason why xattr's can't be done in other ways.
 
 Like what?
 

Like not having RPM attach xattr's. 5+ years ago, I had
to swipe a hunk of code from libselinux and get things done
because the SELinux trolls hadn't written reliable tools yet.

Like patching python to attach xattr's dynamically. The ruke
5+ years ago was
The application that creates the files needs to ensure
that the xattr's are attached.
The trolls wussed out with python+selinux patches.

And note python eggs dynamically downloaded. Why oh why
should a sensible and intuitive and rapid upgrade be
forced back into *.rpm static content for no other reason
than the SELinux trolls wussed out on a patch, thereby
avoiding their own rather Draconian implementation rule,
5+ years ago?!?

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 9, 2011, at 3:50 PM, Tomasz Pala wrote:

 On Sat, Apr 09, 2011 at 15:34:55 -0400, Jeff Johnson wrote:
 
 There's no known reason why xattr's can't be done in other ways.
 
 Like what?
 
 Like not having RPM attach xattr's.
 
 Please tell me how to do root-free (capabilities-based) system without
 xattrs in rpm - doing this outside upgrade procedure leaves window for
 making system unusable in cases like power failure.
 

You asked for me to explain other ways. I am not obligated
nor inclined to argue security packaging with anyone in public.
I quite well know what *I* would do instead; but the issue here is
what *you* want to do in PLD.

 Now we're using some dumb solutions like 'admin' group for SUID ICMP ping
 instead attaching proper file capabilities. In long term we should
 remove ALL SUID binaries from distribution, as this approach is broken
 by design and should be obsoleted 10 years ago.
 

That is your right and privilege to do whatever you wish to do.

But unlike other dstros, PLD usually does sensible engineering.

The only reason I replied is because Patryk said:

 Not sure about PLD but I suppose we just followed what the others were
 doing. Other distros did it this way so they could set proper selinux
 attributes.

basically arguing Do what everyone else is doing. when the
reality is actually that SELinux wussed out on proper engineering
5+ years ago (and is considerably improved since).

73 de Jeff



smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 9, 2011, at 4:52 PM, Tomasz Pala wrote:

 On Sat, Apr 09, 2011 at 16:32:01 -0400, Jeff Johnson wrote:
 
 And I've shown that this way is wrong - having xattrs outside package
 manager is bad design per se.
 
 You WILL have users using python eggs, and there WILL be a window
 installing content outside of package management.
 
 And how is this related to storing xattrs?

Its related to SELinux security labeling. To
ensure consistency, a label MUST be attached everywhere.

This was initially done by RPM. But package management does
not include home directories and other places where windows
lurk.

 
 [...]
 None of the above involves rpm package management installing xattr's.
 
 Exactly - so don't throw me with arguments I do not care at all. I say
 rpm should handle xattrs, nothing more.
 

And I'm not disagreeing. How rpm should handle xattr's like
that capabilities you want is a whole different matter.

Attaching Yet Another per-file tag everywhere just to set
a capaibility for, say, ping and perhaps 100-300 other
files (there's often  1M, try rpm -qal | wc -l)
is a fairly expensive undertaking.

And its quite silly to have _EVERY_ file have an attached (and
usually empty/missing) capability when the right approach
is to run a short list of paths that *do* need a capability attached.

(the above is wrto what is implemented @rpm.org)

 And none of the above precludes rpm from attaching xattr's where it makes 
 sense.
 This thread started -- not about secuirity -- but how to handle
 *.pyo side effects.
 
 I have no idea and I really do not care a bit. What I do know is that
 application level is less important than system level, and the latter
 requires xattrs in modern package management system.
 How do we use them THEN, it's another issue.
 

Well if you don't care abt *.pyo side effects, you're in the wrong thread.

 SO get rid of SUID.
 
 I can't - rpm doesn't support xattrs (or it's so top secret you can't
 tell me how to do this).
 

Bullshit: rpm.org supports capabilities, and I just outlined what @rpm5.org
is going to do. What PLD does with RPM is entirely up to PLD, rpm-4.5 has
no upgrade path, discussed at length repeatedly with both arekm and glenn.

 What SUID has to do with package management of
 *.pyo side-effects isn't clear.
 
 Did I mention some pyo somewhere?
 

You're in the wrong thread. It really doesn't matter what you mentioned.

 So how can I store these caps in rpm? Or force rpm not to overwrite
 these set by me in filesystem (manually or by other tool)?
 
 How does rpm store data in an rpmdb? You look at the schema,
 you create records conistent with the schema, and you write
 tools to make that happen for other data.
 
 Thanks, that really helps. You could just write 'create your own package
 manager'.
 

What do you think python eggs and ruby gems are if not packages?

 Reasoning from RPM has a database - All content MUST be delivered
 in *.rpm packages - I want to remove SUID's! is rather muddled.
 
 No. RPM delivers content with SUIDs instead of ACLs/caps - RPM
 should be fixed to handle xattrs.
 

At this point, all I can say is
Patches cheerfully accepted.
I personally can't justify adding Yet Another per-file tag, but
if that's what you want, I can/will add *exactly* what is at
rpm.org under a vendor-peculier #ifdef.

 And this thread started (see Subject:) with
 
  What should be done with python's Newer! Better! Bestest!
  convention for storing compiled *.pyo files?
 
 not anything else.
 
 It started like this, I've added something else. That is how discussions
 work, subject doesn't have to be fixed as nobody has to stick the first mail
 only.
 

What do you think about radiation leakage in JA? Does that concern you or not?

I kinda prefer Hillary over Obama: chicks in charge! Are there any females in
positions of power in Poland? I just heard about MAM in France, she's cool!

Yep, its all discussion, isn't it?


 And all I wished to point out is the (rather flawed imho) reasoning
 that led to putting *.pyo files into *.rpm packages so that
 SELinux trolls could pretend to a solution based on security tags
 instantiated in xattr's.
 
 Maybe, don't know, don't care, won't argue.
 

I've tried repeatedly to avoid argument:
Patches cheerfully accepted.
if you want to remove SUID's and use capabilities instead.

hth

73 de Jeff
 -- 
 Tomasz Pala go...@pld-linux.org
 ___
 pld-devel-en mailing list
 pld-devel-en@lists.pld-linux.org
 http://lists.pld-linux.org/mailman/listinfo/pld-devel-en



smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 9, 2011, at 6:49 PM, Tomasz Pala wrote:

 On Sat, Apr 09, 2011 at 17:09:54 -0400, Jeff Johnson wrote:
 
 And I'm not disagreeing. How rpm should handle xattr's like
 that capabilities you want is a whole different matter.
 
 Attaching Yet Another per-file tag everywhere just to set
 a capaibility for, say, ping and perhaps 100-300 other
 files (there's often  1M, try rpm -qal | wc -l)
 is a fairly expensive undertaking.
 
 And its quite silly to have _EVERY_ file have an attached (and
 usually empty/missing) capability when the right approach
 is to run a short list of paths that *do* need a capability attached.
 
 Still, this is an implementation detail I won't meddle. The other time
 you've mentioned that this could be accomplished by %post(un) and
 %verify, so as far as I'm concerned %files section could have %acl or
 %caps tags which would be converted to appropriate functions during spec
 parse or something, you might be right that it doesn't make much sense to
 attach them to every single file.
 
 (the above is wrto what is implemented @rpm.org)
 [...]
 I can't - rpm doesn't support xattrs (or it's so top secret you can't
 tell me how to do this).
 
 Bullshit: rpm.org supports capabilities,
 
 And %caps() only - without ACLs, we could have use for, e.g.
 default:group:logs:r on /var/log as currently some logs are NOT readable
 by user in logs group and this is beyond caps scope (DAC_OVERRIDE would
 be per entire user x app set).
 
 I personally can't justify adding Yet Another per-file tag, but
 if that's what you want, I can/will add *exactly* what is at
 rpm.org under a vendor-peculier #ifdef.
 
 What exactly is the overhead of empty tag?

FOr a distro containing 1M files, empty string tags require (at least) '\0'.

But ~1Mb (for a 1M file distro) additional info is moderately significant. Go 
look at
ls -al /var/lib/rpm/Packages
run
rpm -qal | wc -l
and then add that number of bytes for an empty tag. Then also
add that to download times, and space on RO media.

Already all strings are saved in rpmdb indices without trailing '\0'
in order to save several megabytes of space.

 
 What do you think about radiation leakage in JA? Does that concern you or 
 not?
 
 It doesn't.
 
 I kinda prefer Hillary over Obama: chicks in charge! Are there any females in
 positions of power in Poland? I just heard about MAM in France, she's cool!
 
 Sorry, don't care either.
 
 I've tried repeatedly to avoid argument:
  Patches cheerfully accepted.
 if you want to remove SUID's and use capabilities instead.
 
 There are patches - in rpm.org as you know.
 

I'm well aware of what is @rpm.org even if you are not. You claimed
that rpm was preventing you from remoing SUID.

I'm also aware that Fedora tried removing SUID, and has packaged
with capabilities. *shrug*

There's lots of ways to run a command against 100-300 files to
add capabilities securely, without raciness, and without
getting package management involved. SInce you are adding a capabity,
it just means that the file will be installed in unprivlieged
capabilty-free mode until the capability is added, just like SUID's.

And ultimately, neither @rpm5.org or @rpm.org code is in use by PLD,
so it matters about as muct as radiation damage and chicks in charge
discussions.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 9, 2011, at 7:38 PM, Tomasz Pala wrote:

 On Sat, Apr 09, 2011 at 19:05:33 -0400, Jeff Johnson wrote:
 
 FOr a distro containing 1M files, empty string tags require (at least) '\0'.
 
 But ~1Mb (for a 1M file distro) additional info is moderately significant. 
 Go look at
  ls -al /var/lib/rpm/Packages
 run
  rpm -qal | wc -l
 and then add that number of bytes for an empty tag.
 
 It's 0.3% in my case.
 

SO add the tag. I can't justify the implementation for lots of reasons,
not only space. I answered what the overhead cost was, not what the
complexity cost is/was.


 Then also add that to download times,
 
 You must be kidding - downloading 1 MB more considering entire distro is
 nothing.
 

I son't personally think download is worth worrying about.
Meanwhile -- for lots of reasons -- other do,
and blame RPM for bloat.

 and space on RO media.
 
 Already all strings are saved in rpmdb indices without trailing '\0'
 in order to save several megabytes of space.
 
 Looking for a real savings?
 
 ~:  grep install_langs /etc/rpm/*
 /etc/rpm/macros:%_install_langs pl_PL
 ~:  LANG=pt_BR rpm -qi alsa-lib
 Bibliotecas para o ALSA. Esse pacote é necessário para rodar programas
 Linux queusam o driver de som ALSA.
 
 It would be sufficient to have some rpmdb postprocessing tools to remove
 descriptions or changelogs from database, especially for RO medias where
 no rollbacks would ever be required.
 

I'm well aware of the cost of RPM_I18NSTRING_TYPE. There are 2 distros
left using %description -l XY in *.spec recipes.

Mandriva is the other distro ... all @redhat derived distros stopped
using locales in *.spec a decade ago.

 There are patches - in rpm.org as you know.
 
 I'm well aware of what is @rpm.org even if you are not.
 
 Maybe I'm not - how does it even matter?
 

It doesn't matter -- we;re having a dscussion abt *.pyo files in python 3.2, 
aren't we?

 You claimed that rpm was preventing you from remoing SUID.
 
 Yeah - %caps() was added in 4.6 and we got 4.5 here in PLD.
 

You're foolish if you think 4.6  4.5 and hence newer.

 I'm also aware that Fedora tried removing SUID, and has packaged
 with capabilities. *shrug*
 
 Are you aware that SUID (and superuser at all) could be disabled on
 destination host? Thus it's not required to actually remove this bit in
 package, The Point is to supply proper xattrs and leave user a choice.
 

SO remobve SUID's or disable SUID's or write the script that attaches
capabilities to 100-300 files (a generous over estimate considering
how few setuid programs ther actually are), and do whatever you please.

But it simply isn't true that RPM package mangement is stopping you
from implementing SUID removal or me from discussing chicks-in-charge.

A chick would be busily scrubbing out SUID's already ...

 There's lots of ways to run a command against 100-300 files to
 add capabilities securely, without raciness, and without
 getting package management involved.
 
 Assuming this way can undo transaction (full rollback) in context of
 privilidged process. How to setcap setcap binary after upgrade?
 

Re-run the script and write it with +capability/-capability and --rollback if 
you ish.

 SInce you are adding a capabity,
 it just means that the file will be installed in unprivlieged
 capabilty-free mode until the capability is added, just like SUID's.
 
 Unless one wokes up with no privilidged apps in system.
 

Too bad for you.

 And ultimately, neither @rpm5.org or @rpm.org code is in use by PLD,
 so it matters about as muct as radiation damage and chicks in charge
 discussions.
 
 Eventually one of these will be used, but I doubt PLD is used in nuclear
 installations or by chicks in charge.
 

Wanna bet? AFAIK, PLD is happier with its own fork of RPM. I have offered
to assist with upgrade multiple times, gave up caring a few years back
because what PLD does is up to PLD.

But yes, there's a horrendous amount of bloat associated with
%description -l XY

WHich is why RPM_I18NSTRING_TYPE was abandoned (as in not used, yes the
implementation still exists, as b0rken as ever, in RPM) a decade ago.

But Not yet. for PLD and Mandriva.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: RPM vs xattrs [was: python3.2+ compiled files]

[Jeff Johnson]

On Apr 9, 2011, at 8:36 PM, Tomasz Pala wrote:

 BTW it's you who wrote we've got good engineering practices often.
 

Let's just leave it here:

PLD is my favorite distro, solves problems (like *.pyo files
changing in python 3.2+), devises fixes, and moves on
months and years before other distros even begin to say
Huh?

Now if PLD had a chick-in-charge, well, the engineering would be better yet.

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 3, 2011, at 8:38 AM, Patryk Zawadzki wrote:
 
 and sources are often the only source of (up to date)
 documentation.
 So the source code is the documentation. What else is new? Are you
 simply trolling, or just are too ignorant to be aware that's also what
 happens with all other programming languages?
 
 Except in Python you can execute/import .py files just fine. If the
 program is not closed source, .pyc/.pyo/__pycache__ are just an
 optimization detail. We could very well create them in %post.
 

Not %post please ;-)

But I'd generally agree that the compilation might
need to be done on the install, not during the build, given
just the few hints of how python 3.2 is changing.

Another approach would be to not bother with static *.py* content
packaged in *.rpm at all, but rather let python establish its own
format/implementation for install/upgrade/erase of python code.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 3, 2011, at 9:00 AM, an...@green.mif.pg.gda.pl wrote:

 Jeff Johnson wrote:
 Another approach would be to not bother with static *.py* content
 packaged in *.rpm at all, but rather let python establish its own
 format/implementation for install/upgrade/erase of python code.
 
 This approach would break the main PLD rule: that all binaries present in
 the system are registered in the rpm database.
 

rules == rules

So teach python to do a registration. That's already being
done with pubkeys and rpm --import: create a header, call rpmdbAdd().

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 3, 2011, at 9:15 AM, Patryk Zawadzki wrote:

 On Sun, Apr 3, 2011 at 3:04 PM, Jacek Konieczny jaj...@jajcus.net wrote:
 On Sun, Apr 03, 2011 at 02:38:49PM +0200, Patryk Zawadzki wrote:
 Except in Python you can execute/import .py files just fine. If the
 program is not closed source, .pyc/.pyo/__pycache__ are just an
 optimization detail. We could very well create them in %post.
 That is a very bad idea. Bad things will happen if the package is
 removed and *.py[o] stays (the module will still be visible to Python).
 Best way to make sure 'compiled' files are gone when the package is gone
 is to include them in the package.
 
 We have %ghost and we can remove such files on uninstallation. My
 point is that .py[co] is how even more of an optimization detail than
 it was before. Can any other implementation of python handle
 __pycache__? What if two packages install a file to somedir/? Which
 gets to own __pycache__? Can python itself remove obsolete caches when
 politely asked? Can it keep the __pycache__ tree in a separate prefix
 instead of along the files?
 

And not %ghost please: st-st_mode typing and other useful info is
lacking for paths marked %ghost.

(aside)
There's no reason whatsoever that %ghost-like cannot be extended
to include other info that is necessary.

I see a problem with /usr/bin/__pycache__/* and the rest of
the python litter on a file system. While __pycache__ subdirs works in 
python's
module trees, there will be a litter of __pycache__ subdirs on other
paths.

But can't a shadow tree (perhaps /var/cache/python/usr/bin/__pycache__/*?) be 
done?

Another alternative might be a python modules whose sole purpose
is/was to create a shadow tree within /usr/lib/python/X.Y/... for
use by python executables installed in /usr/bin etc ...

hth

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: python3.2+ compiled files

[Jeff Johnson]

On Apr 3, 2011, at 4:01 PM, Patryk Zawadzki wrote:

 On Sun, Apr 3, 2011 at 3:27 PM, Jeff Johnson n3...@mac.com wrote:
 I see a problem with /usr/bin/__pycache__/* and the rest of
 the python litter on a file system. While __pycache__ subdirs works in 
 python's
 module trees, there will be a litter of __pycache__ subdirs on other
 paths.
 
 But can't a shadow tree (perhaps /var/cache/python/usr/bin/__pycache__/*?) 
 be done?
 
 That's exactly what I mean, ideally we'd have cached pre-compiled
 files built somewhere in /var/cache for root and ~/.local/cache for
 local files.
 

Please note that there's _NEVER_ been any real world
reason to package up pure side-effects for any of
/sbin/ldconfig symlinks
sockets   (why bother with a path that is always re-created?!?
log files (%ghost was a hack hob to get uucp/hylafax/amanda perms in 
place)
*.pyc/*.pyo (these were added to give SELinux a stable/static namespace 
to attach xattr's)

All of the above should be handled automagically, with explicit
mechanisms, not just by hack-o-round in *.spec endlessly.

Its not like any of the above side-effects isn't very very well understood
and predictable and easily automated in RPM itself, not through endless 
mind-numbing
*.spec fiddle-ups churn-and-burn. The bikeshed discussions have all been 
rehashed
so many times that there isn't anything left to talk about, just late arrivals
who haven't quite figgered out what to do.

JMNSHO

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: packages: dovecot/dovecot.spec - TODO was accidentally removed (sorry), but...

[Jeff Johnson]

On Oct 30, 2010, at 8:08 PM, Paweł Zuzelski wrote:

 On Thu, 28 Oct 2010, caleb wrote:
 
 Author: calebDate: Thu Oct 28 09:53:02 2010 GMT
 Module: packages  Tag: HEAD
  Log message:
 - TODO was accidentally removed (sorry), but having [percent]service in the 
 spec even on a commented line breaks the build. Bug in builder?
 
 No, it is not a bug.
 
 '#' is not a comment in spec files. To be precise, it does not
 prevent rpmbuild from expanding %macros. The problem is that
 %service macro is multiline macro, so
 

Yup: macros are context free, expanded everywhere they are
encountered, within quuoted strings, in comments, everywhere

(aside)
They used to be expanded in the false branch of %if but that was a bit painful 
..

MEANWHILE ...

... there's actually a feature hidden ther too.

Let's say you have two sets of macros you need to define, like
Fedora - Mandriva have decided to rearrange all the paths
in a package for some reason (I use vendor only for clarity, there's lots of
times that you want two rather different sets of values defined in packaging).

If you capture, say, the Fedora != Mandriva differing in what to call mysql

%fedora \\\
Requires: MySqL \\\
#

%mandriva \\\
Requires: mYsQl \\\
#

Then in the spec file you would put

# %{?fedora} %{?mandriva}

Which (assuming no typoes from me) one ends up with multiline expansions
that drops down one or the other (or both if both %fedora/%mandriva are 
defined)
exposing the Requires: (or whatever text one wanted) into the spec file.

I've always wanted to see cleaner packaging using the fact that macros
_ARE_ expanded in comments, rather than the unexpected behavior bug report.

hth

73 de Jeff





___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm.groups - Libraries/Ruby

[Jeff Johnson]

On Jun 24, 2010, at 7:31 AM, Bartosz Taudul wrote:

 2010/6/24 Paweł Zuzelski z...@xatka.net:
 Any comments?
 Why do we care about RPM groups?
 

What else would we discuss if RPMTAG_GROUP did not exist?

Did Poland do well in South Africa?

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm.groups - Libraries/Ruby

[Jeff Johnson]

On Jun 24, 2010, at 9:36 AM, Paweł Zuzelski wrote:

 On Thu, 24 Jun 2010, Jeff Johnson wrote:
 What else would we discuss if RPMTAG_GROUP did not exist?
 
 Sorry, I don't get it.
 

Translation from obscure jbj-speak:
RPMTAG_GROUP is utterly broken as a package metadata
item because there is No One True Taxonomy!
Nor can One True Taxonomy be devised manually
(but debtag facets are the most interesting approach)

 Did Poland do well in South Africa?
 
 Not so bad, we have not lost any game yet.
 

Lots of ties eh? ;-)

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm.groups - Libraries/Ruby

[Jeff Johnson]

On Jun 24, 2010, at 9:42 AM, Bartosz Taudul wrote:

 2010/6/24 Jeff Johnson n3...@mac.com:
 Why do we care about RPM groups?
 What else would we discuss if RPMTAG_GROUP did not exist?
 I was referring to the general shit state of the group hierarchy in
 PLD. Basically 90% of the stuff is in Applications or
 X11/Applications, which makes the groups completly useless. There was
 some movement to make them more useful, but that was in 2004 and
 hadn't been talked about since then.
 
 New group hierarchy proposition can be found at
 http://cvs.pld-linux.org/cgi-bin/cvsweb/PLD-doc/nowe-grupy?rev=HEAD.
 Jeff, it would be interesting to hear what do you think about
 replacement of groups with tags, as that might make more sense.

Well all I've ever been able to figure (the proposals for Newer!
Better! Bestest! Group: values come out approx. monthly regular
as clockwork) is to get RPMTAG_GROUP out of *.rpm metadata
entirely (which was achieved years ago with specspo just noone uses).

Even if you stabilize the msgid's like Library/Ruby, you've
got the msgstr's to deal with for i18n encoding display purposes.

No way that all of that process can ever succeed if/when compiled
into *.rpm metadata. As soon as you achieve anything, the entire process
restarts itself again and again and again ...

But detaching RPMTAG_GROUP from *.rpm packages is quite doable (see
specspo for doable even if you think spescpo is crap -- specspo is crap imho),
in order to support multiple hierarchies with well defined markup
(hopefully _NOT_ PO as in specspo), with encodings and ...
 Prototype graphical package manager which sorts packages by groups can
 be downloaded from http://team.pld-linux.org/~wolf/pacman/, but it's
 not all that useful.

... per-application sorts and criteria should be attempted.

(aside re useful)
Typically RPMTAG_GROUP is implicitly used as a hierarchical attribute name
space. The hierarchical permits sorting, and the attribute permits
tagging of subsets of a very large universe of packaging.

What's tricky is that the values in the name space aren't reliable.
There's _LOTS_ that could be implemented in installers like RPM et al
with a set membership attribute if the data values in the name space
were reliable. For one slightly different, but related to RPMTAG_GROUP
as an attribute attached to packages, see the recent addition of
RPMTAG_COLLECTION on the rpm-ma...@rm.org mailing list

(Note: I don't believe the Tresys patchset as posted/adopted will ever work
in any version of RPM, but the RPMTAG_COLLECTION attribute framework for
set membership is savable if the data is populated with methodolgy and 
discipline.)

 Prototype tool for managing package groups can be downloaded from
 http://team.pld-linux.org/~wolf/rgmt.7z, but it works only with the
 old SPECS/SOURCES structure and will break on some spec files. But,
 since it displays the groups from all the spec files, it can show how
 much chaos and typos is there, since nobody really cares.
 
 tl;dr: RPM groups in PLD suck.
 

GROUPS in RPM suck, that's no PLD fault.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm.groups - Libraries/Ruby

[Jeff Johnson]

On Jun 24, 2010, at 10:11 AM, Artur Wroblewski wrote:

 2010/6/24 Jeff Johnson n3...@mac.com:
 
 On Jun 24, 2010, at 9:36 AM, Paweł Zuzelski wrote:
 
 On Thu, 24 Jun 2010, Jeff Johnson wrote:
 [...]
 Did Poland do well in South Africa?
 
 Not so bad, we have not lost any game yet.
 
 
 Lots of ties eh? ;-)
 
 No ties, it is just impossible for us to loose. :)
 

Send the Polish team down to the Gulf of Mexico when they're through
playing around with the football in South Africa please. I hear
there's an oily mess to clean up ...

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm.groups - Libraries/Ruby

[Jeff Johnson]

On Jun 24, 2010, at 9:42 AM, Bartosz Taudul wrote:

 2010/6/24 Jeff Johnson n3...@mac.com:
 Why do we care about RPM groups?
 What else would we discuss if RPMTAG_GROUP did not exist?
 I was referring to the general shit state of the group hierarchy in
 PLD. Basically 90% of the stuff is in Applications or
 X11/Applications, which makes the groups completly useless. There was
 some movement to make them more useful, but that was in 2004 and
 hadn't been talked about since then.
 
 New group hierarchy proposition can be found at
 http://cvs.pld-linux.org/cgi-bin/cvsweb/PLD-doc/nowe-grupy?rev=HEAD.

This tree is as good as any other I've seen, certainly far far
better than synaptic/aptitude choices.

Have you considered:

Setting up a process to map specific packages into your taxonomy

Something like a de.licio.us tagging framework to do the mapping
subjectively with community (whatever that means) involvement
would be one relatively painless process. Any voting metric
(with all the usual voting fraud control issues) would work
as well as de.licio.us

The other approach used in debtags is to attempt a RDF semantic
abstraction in order to truly (and objectively) hammer out
a usable taxonomy. Its objectively that is admirable (and
interesting to me wrto RPMTAG_GROUP), all the other issues of
RDF and smantic and classification and ... that are part of the
methodolgy make my head hurt *a lot*.

Setting up representations of the tree markup in JSON/XML/YAML/HTML/DocBook/...

The package taxonomy that easily and automagically transforms to the largest
number of output usage cases is going to win in the end. Its not
just installers that need to present package data in useful forms.

And -- if you show me some reasonable markup for your Group: hierarchical tree 
--
I'll happily patch up RPM to use your markup rather than specspo with
rpm -qa --qf '%{GROUP}\n'
The patch for other markup isn't anywhere near as hard as getting consensus 
regarding
how a package hierarchical namespace SHOULD look.

If your markup includes some additional means for general attribute
tagging of packages, all the better. I have several usage cases in RPM
that are blocked solely by lack of consensus on how attributes
should be attached to packages (the RPMTAG_COLLECTION patch on
rpm-ma...@rpm.org is just one of many usage cases).

hth

73 de Jeff


___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm.groups - Libraries/Ruby

[Jeff Johnson]

On Jun 24, 2010, at 11:22 AM, Bartosz Taudul wrote:

 New group hierarchy proposition can be found at
 http://cvs.pld-linux.org/cgi-bin/cvsweb/PLD-doc/nowe-grupy?rev=HEAD.
 This tree is as good as any other I've seen, certainly far far
 better than synaptic/aptitude choices.
 It was based on sourceforge trove software map.
 
 Have you considered:
 
 Setting up a process to map specific packages into your taxonomy
 
Something like a de.licio.us tagging framework to do the mapping
subjectively with community (whatever that means) involvement
would be one relatively painless process.
 Translation: noone would do it. Not worth the effort.

If you're saying
Noone has time for bookmarking at http://de.licio.us any more.
I absolutely agree.

(aside)
In fact bookmarking (and Google keyword searches)
Just Don't Work Well, but that's a whole different and quite complex
discussion. I kinda like de.licio.us because I can track through
a bookmark tag to see what _ELSE_ a person interested enough
to add a de.licio.us bookmark that blipped my radar might
be interested in. I can't do that with Google keyword searches.

But de.licio.us (and more generally Ajax) are a pathway to automated collection
of voting metrics for attributes attached to packages indepnedently
(and persistently) after a package has been built. The ultimate
design flaw with RPMTAG_GROUP (and other tags in RPM metadata) is
that static content delivery isn't very useful in 2010, the
process to embed tags in packaging reliably is just too cumbersome these days.

The code for de.licio.us (and Ajax and REST-ful in general) is
there for the stealing is all I meant to point out.

But go look at any site that is advertising RPM packages. Here's
are several URL's (from a thread on centos-devel mailing list) I had to visit
this week:

http://packages.debian.org/lenny/php5
http://www.freshports.org/

All of those package presentations are ugly, useless, uninformative
and otherwise as pointless as RPMTAG_GROUP is in RPM installers.

This URL I particularly have issue with (not with the site per se)
http://pkg.org.ua/
Note the organization using vendor as a toplevel attribute for package
choosing.

If vendor is the only attribute that is presented to lusers, well,
FL/OSS software is just doomed.

And it wouldn't be that hard to change the package presentation using, say, a
more reasonable and intelligent hierarchical tree like what you pointed me to.

hth

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: poldek -r broken again?

[Jeff Johnson]

On May 2, 2010, at 3:14 PM, Przemyslaw Iskra wrote:

 On Sun, May 02, 2010 at 02:57:39PM -0400, Jeff Johnson wrote:
 
 chroot(.) = 0
 -- chroots back to /tmp !
This way /tmp becomes new root.
 
 
 ... which re-establishes the cwd before embedded lua was run.
 
 It also establishes /tmp as new /, so now when you use path:
 /tmp/rpm/something it points to /tmp/tmp/rpm/something in real root.
 

Send along a patch if you want a fix. The code is in lib/psm.c,
and the patch is likely less than 5 lines (which is 2 orders of magnitude
fewer lines than have already been written about Glen's %pretrans script)

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: rpm: POSIX capabilities/ACLs?

[Jeff Johnson]

On Feb 6, 2010, at 11:56 AM, Tomasz Pala wrote:

 On Sat, Feb 06, 2010 at 12:04:07 +0100, Zbyniu Krzystolik wrote:
 
 Anyone knows if it is or is going to be possible in rpm to store xattrs?
 
 Not possible now.
 
 And how about The Other RPM? This is a must-be feature and sooner or
 later we must get rid of broken by design SUID/SGID...
 

You must mean rpm-5.0 as the other rpm ;-)

Yes. rpm.org has a defined tag for capabilities, and perhaps for
ACL's (of the linux persuasuoin, how to package ACl'l portably
for *BSD and MacOSX is a nastier but solvable issue).

When I looked at porting support for capabilities  ACL's, this
reasoning mad me reluctant:

There are  300K files in a typical rpm distro.

Out of that 300K files, perhaps 100-500 files would
benefit (afaik) from adding support for capabilities/ACL's.

Adding an additional per-file tag to benefit 500 of 300,000
files, with the additional download bandwidth needed to
represent missing/unused info doesn't make much sense.

Making the tag optional, present iff explicitly added,
while doable, creates a different sort of missing or optional
problem.

But if you want capabilities/ACL's ported to rpm-4.5, I can do that in
an afternoon if you wish.


 My note may be interested for you (pl); libcap-ng utils can simplify it.
 http://zz.iapt.pl/bez_root2.txt
 
 That's similar to thing I want to do. The difference is you drop
 capabilities, and I want to set some for regular users (either
 designated - for daemons having it's own files and secrets, or nobody
 for anything else, using start-stop-daemon --chuid). Like this:
 
 setcap cap_net_bind_service=ei =nc
 execcap cap_net_bind_service=i su - gotar -c 'nc -l -p 34'
 
 but this obviously requires tagging binaries. The problem is tracking
 all the xattrs (caps and ACLs).
 

Yes, tracking *all* the file paths is exactly the same as SElinux
xattr's. Note that SELinux currently doesn't trust its means to track
the xattrs across *all* file paths suufficiently that they have chosen
to package SELinuc modular policy with

Any SElinux attr that is installed is never removed.

Similar issues will be seen with capabilities/ACL's tracked across
*all* file paths in addition to the bloat I mentioned.

No matter what:
There's nothing stopping you from the applying capabilities/ACL's
in %post, and removing same (if necessary) in %postun and verifying
that indeed the correct capabilities/ACL's are applied using %verifyscript.

hth

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: RFC: Convert bconds to autoconf-like options.

[Jeff Johnson]

Before you get into a litter of additional macros for
bconds, please note that when originally implemented
(by PLD, always first ;-) bcond's were intended as booleans.

When bconds were picked up by RPM using --with/--without popt aliases
(that are essentially just --define wrappings to conventionally
set macro strings), there was already an attempt to try to
conventionalize the string values to be passed straight to
./configure.

So there is a previous attempt to do what you seem to be attempting now.

Whether the existing mapping is flexible enough (or not) is an entirely
different issue than attempting a common conventional approach to
setting build options through RPM.

Personally, I don't believe the existing macro infrastructure in RPM
is robust enough to support full blown build option configuiration.

What should be done instead is to add additional macro primitives,
with a reliably defined registry, not yet more gunky and obscure lazy
string expansions using the existing RPM macro implementation.

I can help with implementing macro primitives (and adding a registry)
if you wish. YAML would likely be a representation markup that
isn't too painful, other approaches, using embedded lua symbol
tables instead of the existing macro store (which is quite feeble imho).

But you can do whatever you please too ...

73 de Jeff

On Nov 18, 2009, at 11:07 AM, Przemyslaw Iskra wrote:

 
 Because I don't expect any constructive feedback just yet I'm going to
 introduce the changes right away and we will discuss and later fix them
 eventually.
 
 
 I'm proposing few macros to easily convert bconds to configure options.
 Implementation looks like this:
 
 
 # expand bconds to --enable-something and --disable-something
 %__enable() %{expand:%%{?with_%{1}:--enable-%{?2}%{!?2:%{1
 %__disable() %{expand:%%{!?with_%{1}:--disable-%{?2}%{!?2:%{1
 %__enable_disable() 
 --%{expand:%%{?with_%{1}:en}%%{!?with_%{1}:dis}}able-%{?2}%{!?2:%{1}}
 
 # same as above, but condition inverted
 %__enable_unless() %{expand:%%{!?with_%{1}:--enable-%{?2}%{!?2:%{1
 %__disable_if() %{expand:%%{?with_%{1}:--disable-%{?2}%{!?2:%{1
 %__enable_disable_not() 
 --%{expand:%%{!?with_%{1}:en}%%{?with_%{1}:dis}}able-%{?2}%{!?2:%{1}}
 
 # expand bconds to --with-something and --without-something
 %__with() %{expand:%%{?with_%{1}:--with-%{?2}%{!?2:%{1}}%{?3:=%{3
 %__without() %{expand:%%{!?with_%{1}:--without-%{?2}%{!?2:%{1
 %__with_without() 
 %{expand:%%{?with_%{1}:--with-%{?2}%{!?2:%{1}}%{?3:=%{3}}}%%{!?with_%{1}:--without-%{?2}%{!?2:%{1
 
 # same as above, but condition inverted
 %__with_unless() %{expand:%%{!?with_%{1}:--with-%{?2}%{!?2:%{1}}%{?3:=%{3
 %__without_if() %{expand:%%{?with_%{1}:--without-%{?2}%{!?2:%{1
 %__with_without_not() 
 %{expand:%%{!?with_%{1}:--with-%{?2}%{!?2:%{1}}%{?3:=%{3}}}%%{?with_%{1}:--without-%{?2}%{!?2:%{1
 
 
 
 Each macro requires bcond name and accepts optional option name, if option
 isn't specified bcond name is used as option name. Options returning
 --with-something also allow third argument which will be added as value
 in case or positive condition, but won't be added otherwise.
 
 I was thinking using another prefix, but __ best mimics the two dashes in
 front of options.
 
 Examples:
 * Basic enable and disable options: 
 - %{__enable bcond}
   returns --enable-bcond if build condition bcond is set,
   returns nothing otherwise
 - %{__enable bcond option}
   returns --enable-option if build condition bcond is set,
   returns nothing otherwise
 - %{__disable bcond}
   returns --disable-bcond if build condition bcond is not set,
   returns nothing otherwise
 - %{__enable_disable bcond}
   returns --enable-bcond if build condition bcond is set,
   returns --disable-bcond if build condition bcond is not set
 
 * Negated enable and disable options, probably useful only with second 
 argument:
 - %{__enable_unless bcond option}
   returns --enable-option if build condition bcond is not set,
   returns nothing otherwise
 - %{__disable_if bcond option}
   returns --disable-option if build condition bcond is set,
   returns nothing otherwise
 - %{__enable_disable_not bcond option}
   returns --enable-option if build condition bcond is not set,
   returns --disable-option if build condition bcond is set
 
 * Basic with and without options: 
 - %{__with bcond}
   returns --with-bcond if build condition bcond is set,
   returns nothing otherwise
 - %{__with bcond option}
   returns --with-option if build condition bcond is set,
   returns nothing otherwise
 - %{__with bcond option value}
   returns --with-option=value if build condition bcond is set,
   returns nothing otherwise
 - %{__without bcond}
   returns --without-bcond if build condition bcond is not set,
   returns nothing otherwise
 - %{__with_without bcond}
   returns --with-bcond if build condition bcond is set,
   returns --without-bcond if build condition bcond is not set
 - %{__with_without bcond option value}
   

Re: RFC: Convert bconds to autoconf-like options.

[Jeff Johnson]

On Nov 18, 2009, at 11:49 AM, Przemyslaw Iskra wrote:
 
 I only really care about %__enable_disable macro which replaces such
 obscure PLD idiom:
   --%{?with_runtime:en}%{!?with_runtime:dis}able-runtime-cpudetection
 with much more readable:
   %{__enable_disable runtime runtime-cpudetection}
 

Certainly macros are eye scratching pugliness, you will get no argument
from me.

 All the other macros are to complete the implementation.
 
 
 Your registry idea sounds very interesting, but PLD doesn't even have
 standarized bcond names (not often, but they differ between spec files
 sometimes). So implementing it would require a lot of effort from us.
 

If there's some fundamental design, particularly with naming conventions
(hint: just steal AutoFu conventions and put a _ in front of the names),
then a registry and inheiritance and YAML/XML/JSON/WHATEVER markup
and more could (and should) be attempted. Without strong design
conventions, well, PLD has strong methodology even if most distros
don't wrto RPM macros.

But certainly RPM macro eye scratchiness can always be attempted
by adding Yet Another Layer of abstraction.

At some point its quite hard to tell how a build should be done ...


73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: configure --host/build/target

[Jeff Johnson]

On Oct 11, 2009, at 8:01 AM, Arkadiusz Miskiewicz wrote:


 Does anyone remember why we use such configure macro:

 ./configure \
--host=%{_target_platform} \
--build=%{_target_platform} \

 instead of

 ./configure \
--host=%{_host_platform} \
--build=%{_build_platform} \
--target=%{_target_platform}

 (where rpmbuild --target xyz changes only target_macro)


Likely because if what was commonly used in autoconf and RPM way back  
when.

What additional actions RPM should undertake with %{_host_platform}
and %{_build_platform} have never been clear, nor does RPM do anything
special with those two macros, other than to permit the values to be
configured with macros and given to autoconf configure.

Sure --target changes only target macros. Well duh.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: verify rpm package contents

[Jeff Johnson]

On May 13, 2009, at 1:41 PM, Tomasz Pala wrote:

 How to verify digest of files in rpm package (like when repackaged
 modified files)? For example I've got:

 ~: rpm -qplv xorg-proto-xproto-devel-7.0.14-1.i586.rpm
 -rw-r--r--1 rootroot   167477 Oct 28  2008 /usr/ 
 include/X11/keysymdef.h
 but after un-cpio there is:167401 May 22  2008

 rpm --verify -p file.rpm

 verifies against filesystem contents not files within.


Repackaged files have no digest verification. The digest
carried in repackaged packages is the original digest;
but the file in the payload may have been modified or
even deleted and not present in te repackaged package payload.

You can work around by using a transaction probe dependency.

E.g.

 mkdir -p /etc/rpm/sysinfo
 md5sum /etc/passwd | sed -e 's/\([^ ]*\) *\(.*\)/digest(\2) =  
\1/'  /etc/rpm/sysinfo/Requirename

verifies the md5 of /etc/passwd every time rpm -Uvh is run.

hth

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: topdir macro

[Jeff Johnson]

On May 3, 2009, at 6:26 AM, Elan Ruusamäe wrote:

 why not have such default macros:

 %_topdir%{expand:%%global _topdir %(d=$([ -d ../../ 
 packages ]  (cd ../..  pwd)); d=${d:-$([ -d ../packages ]   
 (cd ..; pwd))}; echo ${d:-$HOME/rpm})}%_topdir
 %_specdir   %{_topdir}/packages/%{name}
 %_sourcedir %{_specdir}

 the %{name} seems to work too:

 $ /usr/bin/rpmbuild -bp jalbum.spec
 Executing(%prep):  env -i PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/ 
 X11R6/bin:/usr/local/bin:/home/users/glen/bin:/usr/games:/home/users/ 
 glen/okas HOME=/home/users/glen
 TMP=/home/users/glen/tmp TMPDIR=/home/users/glen/tmp  /bin/sh -e / 
 home/users/glen/tmp/rpm-tmp.82586
 + umask 022
 + cd /home/users/glen/rpm/BUILD.i686-linux

 actually i'd see topdir = packages/ dir, i.e default ~/rpm/packages


The issue is gonna be side effects, like lazy mkdir's, when CWD
is incorrect for some reason. Symlinks to directories can/will
also break ../ lookups.

If you want rpm to handle a remote build tree, or with a VPATH
search, independent of CWD, then it should likely be done
by fundamentally changing the 10 or so expansions where build
paths are instantiated, not with clever and fragile hacks to
%_topdir and CWD.

It wouldn't be hard to add a VPATH hierarchy creating build paths to  
rpm.

73 de Jeff
 -- 
 glen
 ___
 pld-devel-en mailing list
 pld-devel-en@lists.pld-linux.org
 http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: INFO: cvs downtime

[Jeff Johnson]

On May 2, 2009, at 5:25 AM, Elan Ruusamäe wrote:


 I don't want any sources to be fetched.

 those are patches not sources

 indeed --fetch-only-spec option would be nice


Likely achievable with L and R macro overrides, and
a popt alias to hide the --define.

(3+ years since fetch being implemented dims my memory)

73 de Jeff


___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: Merging PLD patches for RPM @rpm5.org

[Jeff Johnson]

On Jan 18, 2009, at 10:56 AM, Radoslaw Zielinski wrote:

 Jeff Johnson n3...@mac.com [18-01-2009 16:31]:
 [...]
 Is the AUTODEP_PKGNAMES portion of the rpm-pld-autodep.patch, which  
 maps
 dependencies back to package names, actually useful/used by PLD? The

 No.

 It has been turned off for Th, as it's an endless source of annoyances
 for cases where multiple packages can satisfy a given dependency.


ok, thanks.

The case of multiple Provides: could be detected during the mapping
and the mapping could be disabled (or build failed) when multiple  
Provides:
are encountered.

Does that permit AUTODEP_PKGNAMES to be useful/used by Th?

Note that my personal POV is that the AUTODEP_PKGNAMES mapping is  
insufficiently
general because the mapping applies only to Requires:, ignoring all  
other dependencies.

But the mapping to package names could be attempted for other  
dependencies too.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: TeXLive

[Jeff Johnson]

On Dec 7, 2008, at 4:29 AM, Łukasz Jernaś wrote:


2008/12/7 Jeff Johnson [EMAIL PROTECTED]:


Heck even gasoline is now US$1.60/per gallon now ...


Well, where most PLD devs/users live it's about 4.61498478 USD per
gallon of super...



The more interesting questions are
What's the $/Gb and $/Mbps costs?

Computers don't (yet) run on super or beer.

I'm sure a Polish vodka powered opteron would scream ... ;-)

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: TeXLive

[Jeff Johnson]

On Dec 6, 2008, at 5:35 PM, wrobell wrote:





But points noted fer sure, TeX is not easy packaging, never has been.


indeed :)



Note I dinna say either of
(Diskspace|bandwidth) is cheap.

Being stoopid is way more costly than commodity items. Ugly and evil  
have the same flaws.


Heck even gasoline is now US$1.60/per gallon now ...

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: TeXLive

[Jeff Johnson]

On Dec 6, 2008, at 6:37 PM, wrobell wrote:



[1] my desktop machine is so huuug, baby


Well I dunno nothin' about your desktop.

But my desktop has accumulated 1,337,924.20 cobblestones
patiently grinding away, mostly [EMAIL PROTECTED], through BOINC.

I hope to contribute 0.005% to the USA cobblestone total sometime in
the next 12 months. I am currently at the 99.529% percintile ...

It's __ALL__ about what you do with your desktop, not otherwise ;-)

73 de Jeff

smime.p7s
Description: S/MIME cryptographic signature
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: CVSROOT: users - make AIDA Th aliases - sorted

[Jeff Johnson]

On Nov 10, 2008, at 4:39 PM, Arkadiusz Miskiewicz wrote:

 What's AIDA?


The wife of Rademes? ;-) ;-)

 Please fix sorting, you broke it.


Those high notes are ever so hard to reach if you are merely a  
contralto ...

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: rpm.spec - 5.1.6 - updated TODO (will glob patch finally go away?)

[Jeff Johnson]

On Oct 28, 2008, at 6:24 PM, Jan Rekorajski wrote:

 On Tue, 28 Oct 2008, Arkadiusz Miskiewicz wrote:

 On Tuesday 28 of October 2008, baggins wrote:
 Author: baggins  Date: Tue Oct 28 22:11:56  
 2008 GMT
 Module: SPECS Tag: HEAD
  Log message:
 - 5.1.6


 - updated TODO (will glob patch finally go away?)

 Only if current upstream rpm is smart enough to always use own glob()
 implementation (which differently handles symlinks pointing to  
 nowhere than
 current glibc implementation).

 rpm's glob() has been moved from misc to rpmio, and I don't know how  
 to
 test if it works properly :(


If you find glob tests, I'll wire into rpm's make check.

Otherwise, glob() failure is usually not quiet. De facto
testing using rpmbuild is more than sufficient.

The reason for the glob patch (which ultimately has led
to rpm internalizing glob) had to do
with dangling symlinks.

If you can package a dangling symlink, then the internal
rpmio glob is likely as good as glibc's (upon which its based).

 http://rpm5.org/cvs/tktview?tn=38_submit=Show really got fixed?

 I don't think so. The only that got fixed was
 http://rpm5.org/cvs/tktview?tn=39_submit=Show

 I just removed duplicate URLs in spec.

 BTW Why aren't we using rpm 5.1.x in Th?


rpm-5.0 changed the rules for memory allocation. The older rule
was replaced with
All memory return'ed from headerGet() must be free'd.

Reworking the changed malloc rule into poldek is likely the rate
limiting flaw.

But rpm-4.4.9 (and rpm-4.5 and ..) are all pretty similar in usefully
used functionality.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: rpm.spec - 5.1.6 - updated TODO (will glob patch finally go away?)

[Jeff Johnson]

On Oct 28, 2008, at 6:39 PM, Arkadiusz Miskiewicz wrote:

 On Tuesday 28 of October 2008, Jeff Johnson wrote:
 On Oct 28, 2008, at 6:20 PM, Arkadiusz Miskiewicz wrote:
 http://rpm5.org/cvs/tktview?tn=38_submit=Show really got fixed?

 If not fixed, I need a more precise reproducer.

 The problem that I saw before, multiple RPMTAG_DESCRIPTION
 tags ending up in packages using rpm-5.1, I tried (and could not)
 reproduce
 with the case you originally gave me.

 Next time please file proper comment into rpm5.org bug system.


I tried (see the url), but the @rpm5.org bug system is rather awkward  
and fragile.

What is a proper comment anyways?

 Will retest again at some point.


Please do. and if you send me a toy srpm reproducer, then I
will wire up into rpm's make check. You can easily add too,
just check in a tests/foo*.srpm, that will automatically be rebuilt
and installed, which should exercise obvious flaws.

No matter what: RPM_I18NSTRING_TYPE needs to Die! Die! Die!

A data type that sometimes is a scalar, and sometimes is an array,
is no useful data type imho.

I know several ways to eliminate RPM_I18NSTRING_TYPE with
no loss of functionality if I extend arbitrary tag permitted
characters. But I cannot make a PLD change for you.

73 de Jeff

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: rpm.spec - 5.1.6 - updated TODO (will glob patch finally go away?)

[Jeff Johnson]

On Oct 28, 2008, at 6:50 PM, Jakub Bogusz wrote:


 That's nothing to do with glob(). Didn't you mean:

 %install
 # (same as above)

 %files
 /*


Yes.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: rpm.spec - 5.1.6 - updated TODO (will glob patch finally go away?)

[Jeff Johnson]

On Oct 28, 2008, at 7:50 PM, Jeff Johnson wrote:


 On Oct 28, 2008, at 6:50 PM, Jakub Bogusz wrote:


 That's nothing to do with glob(). Didn't you mean:

 %install
 # (same as above)

 %files
 /*


 Yes.


BTW, there's another approach to solving the glibc glob()
problem pursued by SuSE if you wish.

The GNU glob() has (*stat) and (*lstat) vectors that
are loaded with Stat() and Lstat() pointers when used
through rpmio.

All that is needed (iirc) is to load the (*lstat) vector with Stat()
instead of Lstat() (but I could have the issue backwards, this
is like 3 year ole memories).

See OpenSUSE patches to rpm if you want to use glibc glob() instead.

I wish distingushed Stat() and Lstat() in rpmio for potential use
with remote (i.e. through http/ftp) globbing.

There are also portability issues on non-GNU systems that do not support
the GNU extensions.

Those are the 2 rationales for internalizing GNU glob into rpmio. But  
rpmio glob also
fixed some symbol collision which arekm reported. However, reverting
to glibc glob(3) would have fixed that issue as well.

73 de Jeff

 73 de Jeff
 ___
 pld-devel-en mailing list
 pld-devel-en@lists.pld-linux.org
 http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: glibc.spec S: tzdata, too then

[Jeff Johnson]

On Oct 27, 2008, at 4:09 AM, Elan Ruusamäe wrote:

 On Monday 27 October 2008 00:06, Tomasz Pala wrote:
 On Sun, Oct 26, 2008 at 16:32:51 -0400, Jeff Johnson wrote:
 Hint: read up on using %_dependency_whiteout to snip loops
 deterministically.
 The better fix, changing the packaging to not have any loops, is a
 much harder issue to solve.

 That's why I wanted not to have S for packages with the same %{name}
 prefix, as they often require base package (xyz-plugin R: xyz, so  
 don't
 S: xyz-plugin from xyz).

 the deps place is perhaps basesystem?

 %description
 While this package does not contain any files, it does perform an
 important function. It defines the components of a basic PLD
 distribution, providing packages install in right order.


Perhaps.

But basesystem exists solely because of the way that RedHat used to  
cookie cutter
distros before sending a tree for the next batch of cdroms' to be  
burned.

Maybe PLD is planning on burning cdroms for distribution too?

Seriously, the basesystem package is as useless as toe nails and an  
appendix.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: glibc.spec S: tzdata, too then

[Jeff Johnson]

On Oct 26, 2008, at 4:08 PM, Tomasz Pala wrote:

 On Sun, Oct 26, 2008 at 21:13:28 +0300, Elan Ruusamäe wrote:

 +S: tzdata, too then

 LOOP now?

 Is is tight enough to cause any problems?


The issue is rather different than described.

Suggests: added using RPMSENSE_MISSINGOK (as PLD does) will
never cause a dependency loop while ordering.

Howvere, Suggests: are promoted to Requires: when the suggestion
is accepted. That's the issue that glen is worrying abt.

Meanwhile, I just gave glen a patch that will display LOOP:
messages always, not just when -vv is specified.

The noise from displaying LOOP: whenever a dependency loop is found
is surely gonna drown out the rather minor and technical detail
of whether a promoted Suggests: - Requires: causes an additional loop.

Hint: read up on using %_dependency_whiteout to snip loops  
deterministically.
The better fix, changing the packaging to not have any loops, is a
much harder issue to solve.

hth

73 de Jeff


 -- 
 Tomasz Pala [EMAIL PROTECTED]
 ___
 pld-devel-en mailing list
 pld-devel-en@lists.pld-linux.org
 http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS: ant.spec - unify java package (template-java.spec)

[Jeff Johnson]

On Oct 21, 2008, at 5:20 PM, Paweł Zuzelski wrote:

 On Tuesday 21 of October 2008 22:32:03 Elan Ruusamäe wrote:
 If I understand what does %ghost mean, you should either revert my
 commit and then mark this file as %ghost or revert your commit.

 perhaps you should first understand how to package a %ghost-ed file?

 Perhaps. That is why I asked you.

 According to rpm documentation:

 'A %ghost tag on a file indicates that this file is not to be included
 in the package.  It is typically used when the attributes of the file
 are important while the contents is not (e.g. a log file).'

 This file *is* going to be included in the package. It is created in  
 %install
 section not in %post, so I do not understand why it should be  
 %ghosted.


(aside) That should likely be %ghost attribute instead of tag. and
%exclude, not %ghost, is closer to the not to be included semantic.
oh well ...

The difference is that %install is run on the build machine,
while %post is run on the install machine.

%ghost was added to accomodate files created by scripts like %post run
during install, as well as to handle permisions on log files that
cannot be packaged on the build machine.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en

Re: SPECS (rpm-4_5): rpm.spec - deem. need different sources for db due md5

[Jeff Johnson]

On Sep 3, 2008, at 11:15 AM, Jakub Bogusz wrote:


 What are profits?

The profit is avoiding Berekely DB symbol clash by using --with- 
uniquename
for rpm with internal db.

Statically linking rpm will have larger bloatiness cost than what you  
mention.

Whether you want that profit is a whole different question I'll  
leave to glen.

73 de Jeff
___
pld-devel-en mailing list
pld-devel-en@lists.pld-linux.org
http://lists.pld-linux.org/mailman/listinfo/pld-devel-en