Re: [Rpm-maint] [rpm-software-management/rpm] RFE: add an option to park the rpmdb (Issue #2219)

[Colin Walters]

I came across 
https://github.com/microsoft/rpmoci/blob/47443c4312505222460dfcd185308db7153897bb/src/lockfile/build.rs#L173

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2219#issuecomment-2303007254
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: add an option to park the rpmdb (Issue #2219)

[Colin Walters]

> so I think this should be either a default thing in rebuilddb 

On this topic, it would be a nontrivial hit to the ergonomics of default 
container builds if we really start telling people to do:

`RUN dnf -y install foo && dnf clean all && rpm --fix-database`

So probably what should happen is dnf should detect it's in a container and do 
this by default as part of `dnf clean all` most likely.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2219#issuecomment-2171410119
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: add an option to park the rpmdb (Issue #2219)

[Colin Walters]

@ffesti the issue I mentioned at Devconf.cz

That said...now that I'm digging in again to reproducible builds, 
https://github.com/containers/buildah/issues/5592 is higher on my radar, but 
this is probably in the top 5.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2219#issuecomment-2171290047
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Return to Tralla La or: RPM in C++ (Discussion #2983)

[Colin Walters]

> As such, moving to C++ now will probably make it harder to move to Rust later.

Well, maybe.  My original comment here remember was about how we very 
intentionally moved rpm-ostree to "C compiling as C++" explicitly to bridge 
with cxx.rs.  This...kind of worked in some ways, and definitely didn't in 
others - although I'd say at least 50% of that is just not executing well 
enough probably.

Using some C++ definitely doesn't exclude Rust in the future either.



-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/discussions/2983#discussioncomment-8946642
You are receiving this because you are subscribed to this thread.

Message ID: 
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Return to Tralla La or: RPM in C++ (Discussion #2983)

[Colin Walters]

> For starters, it's a show-stopper as a bootstrapping dependency for something 
> as early in that chain as rpm.

(Threading this) do you have a link to these discussions?  There's a *ton* of 
work on bootstrapping Rust (and systems in general) on self-hosting 
OSes/distributions.  The [GUIX 
one](https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/)
 is pretty cool.  And for sure Rust gets handled as part of that, it's well 
documented and understood.  I get that people who want to do something similar 
for self-hosting rpm-based systems would want to build rpm pretty early and 
switch over a bootstrap process to using it, but I'd imagine there's quite a 
bit of stuff that needs to be done using not-RPM before that that maintaining 
such a system would be quite close to needing to maintain two different build 
systems anyways, and having Rust in that set in addition to other compilers and 
toolchains that are already needed seems like not a large addition.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/discussions/2983#discussioncomment-8877438
You are receiving this because you are subscribed to this thread.

Message ID: 
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Return to Tralla La or: RPM in C++ (Discussion #2983)

[Colin Walters]

I have some experience with this; we did a similar thing in rpm-ostree starting 
around https://github.com/coreos/rpm-ostree/pull/2336#issuecomment-739556744
The rationale there was actually to aid porting to Rust, because we could use 
https://cxx.rs/

There were a lot of things there, see various PRs like 
https://github.com/coreos/rpm-ostree/issues?q=label%3Aproject%2Fc%2B%2B+is%3Aclosed
 around that time (I didn't label all of them, there were a lot).

Definitely dealing with NUL terminated strings versus C++ strings *and* Rust 
strings was and is a notable pain.

Probably the biggest ergonomic problem was exceptions...Rust and C align there, 
and "standard" C++ doesn't.  Of course it's possible to use C++ without 
exceptions...but, it's an ecosystem bifurcation.

> It is a ridiculously large and complicated language

Yes.  Not to be That Person but...I dunno, I have a pretty strong opinion that 
Rust is the right choice for systems software, and for those bits you are 
rewriting, it can pay itself back.  Rust is also very complex and was a huge 
learning curve for me (and others).  The trap with C++ in a nutshell IMO is 
that it can feel very high level, yet low-level things like string_view 
use-after-frees will just come and bite you.



-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/discussions/2983#discussioncomment-8869019
You are receiving this because you are subscribed to this thread.

Message ID: 
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] SOURCE_DATE_EPOCH=0 not clamping file mtime (Issue #2679)

[Colin Walters]

ostree always uses zero for mtime of files it writes because there are no 
timestamps in the file format at all.  And in order to have sharing via 
hardlinks, there's then the question of what time to apply to that inode.

If there was a way in a Unix filesystem to have no timestamp at all, we'd 
definitely do that!  In Rust terms, ideally `stat()` would return 
`Option`.

Longer term though, we're moving towards 
https://github.com/containers/composefs which will give us in-memory/on-disk 
sharing even if we just let file timestamps float to "whatever", which will be 
helpful here.  Also, it's likely in that world that we could consider switching 
to canonicalizing the on-disk (in EROFS) timestamp to e.g. something like the 
overall build's SOURCE_DATE_EPOCH, which could definitely be non-zero.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2679#issuecomment-1739079658
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: allow overriding buildtime and hostname via environment variable (Issue #2603)

[Colin Walters]

> But people consider both BUILDTIME and BUILDHOST very useful for figuring out 
> where/when/who exactly build a package.

For Fedora using Koji, there is always exactly one Koji build for a given 
NEVRA, and the server side metadata contains the build host.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2603#issuecomment-1714171760
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: detect filesystem capabilities before starting a transaction (Issue #2637)

[Colin Walters]

xref https://github.com/containers/storage/pull/1608

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2637#issuecomment-1699140774
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Replace fakechroot with proper container technology (PR #2559)

[Colin Walters]

It's worth noting that rpm-ostree has been isolating individual scripts (e.g. 
`%post`) with bwrap for a long time now.  That's distinct from the test suite 
only usage here, but just FYI.


-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/2559#issuecomment-1621273057
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] docs/users_and_groups: Mention DynamicUser (PR #2558)

[Colin Walters]

Well, the doc is talking all about sysusers.d, which is (today) only 
implemented by systemd...

I coincidentally recently did https://github.com/ostreedev/ostree/pull/2914 
which updates the ostree equivalent of this section, and intentionally talked 
about `DynamicUser=yes` there because it entirely removes a big problem domain.

It's IMO quite relevant for RPM because a packager or upstream developer may 
very likely be referred to this page (by another packager e.g.) and very 
commonly the best change to adopt `DynamicUser=yes` upstream, and *not* to port 
from running `useradd` in `%post` to `sysusers.d`.



-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/2558#issuecomment-1614787917
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] docs/users_and_groups: Mention DynamicUser (PR #2558)

[Colin Walters]

Because it really is just better (where it's possible to use).
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/2558

-- Commit Summary --

  * docs/users_and_groups: Mention DynamicUser

-- File Changes --

M docs/manual/users_and_groups.md (7)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/2558.patch
https://github.com/rpm-software-management/rpm/pull/2558.diff

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/2558
You are receiving this because you are subscribed to this thread.

Message ID: 
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RPM with Copy on Write (PR #2378)

[Colin Walters]

I referenced this effort in 
https://marc.info/?l=linux-fsdevel&m=167794198604510&w=2 - my PoV is that the 
rpm-cow effort makes some sense.  I thought a lot though about hard requiring 
reflinks for ostree though and determined it was not viable.  There are too 
many people that use e.g. ext4.  And even if we stopped supporting new installs 
with that, IMO we need to support in-place upgrades for rpm systems deployed 
w/ext4 (or xfs without `-m reflink=1`) for a (really) long time.  

On the ostree side I thought about hard requiring reflinks and just concluded 
it was not viable to have two different code paths.

What ultimately I think will work better here is using overlay-style 
filesystems.  From my PoV composefs or something similar is the successor to 
ostree.  And that type of overlayfs style approach Just Works even on 
non-reflink filesystems (albeit less efficiently sometimes).  So there's no 
concerns with in-place upgrades, and that's ultimately why I think it's a more 
viable approach for both bootable host systems and containers.  (Now, how 
exactly one would try to tie something composefs-like into something that feels 
more traditional-rpm like is a bit more of an open question; would a higher 
level tool like yum/zypper end up making composefs snapshots locally, etc?  
We'll be doing this for rpm-ostree at least, but it was designed from the start 
for that)

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/2378#issuecomment-1458653493
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] unreproducible `rpmdb.sqlite-shm` (Issue #2219)

[Colin Walters]

> The existence of .sqlite-shm is required for read-only WAL mode to work at 
> all (a very important use-case being queries by regular users), see 
> https://www.sqlite.org/wal.html#read_only_database

I find this weird - because unprivileged code can't write directly to the 
database, what practical use is the shared memory?

Hmm, I just found https://www.sqlite.org/uri.html#uriimmutable - seems like rpm 
may be able to use that when it detects it can't write?

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2219#issuecomment-1292136069
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] unreproducible `rpmdb.sqlite-shm` (Issue #2219)

[Colin Walters]

This is a followup to 
https://github.com/rpm-software-management/rpm/commit/71456f2fc09900a027a33dc3d6d75c69a9b39488
which is about generating bit-for-bit reproducible images (container/disk) that 
include an RPM database.

At the time, the person working on that PR was looking at RHEL8 (BDB format rpm 
database).  We've since moved on to sqlite, which is nicer.

Except in my testing, this `rpmdb.sqlite-shm` is the very last thing blocking 
reproducible rpm-ostree container images.  I hacked up 
https://github.com/coreos/rpm-ostree/pull/4074
but this issue is to track fixing this more properly inside librpm.

I think what would be the best is if this file could be deleted entirely at the 
end of a build.  It seems like rpm (or sqlite) will auto-recreate it in that 
case, but what blocks us from doing that is that things fail if the mount is 
read-only.



-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2219
You are receiving this because you are subscribed to this thread.

Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: Relocate RPM and DNF databases to /usr

[Colin Walters]

On Mon, Jan 3, 2022, at 2:44 AM, Panu Matilainen wrote:
> On 12/16/21 16:41, Colin Walters wrote:
>
>> I didn't wake up one day and say "hey you know what, today I'm going to move 
>> the rpm database just for fun".  Neither, for that matter did the OpenSUSE 
>> folks.  We haven't had this conversation over and over throughout the years 
>> just because it was some minor thing.
>> 
>> What I *did* wake up one day and say I'm going to do is make upgrades 
>> transactional and offline by default and hence safe.  No one should ever 
>> fear starting an operating system upgrade while their laptop is at 30% 
>> battery.  Administrators running important servers must be able to easily 
>> roll back when the kernel *or* systemd (or something) else regresses, 
>> because it's software, it regresses all the time despite our best efforts.
>> 
>> So yes again, this does matter.  And it matters because whether you're doing 
>> "image based upgrades" like ostree or just "client side offline updates" 
>> like the 
>> https://kubic.opensuse.org/documentation/man-pages/transactional-update.8.html
>>  thing - it's very important *what data specifically* is 
>> versioned/snapshotted and what isn't.  On an ostree system for example, it's 
>> completely normal that there are *two* rpm databases (one you're running, 
>> one that's pending in the new root).
>> 
>> All the data in the rpmdb is about content that's in `/usr`.
>
> But that's just plain, utter bollocks.
>
> On modern distros, MOST of rpmdb may be about /usr content but most is 
> not all, not by a long shot. If that's an assumption this change is 
> based on then it needs to stop right there.

There is indeed the `/opt` issue that Florian raised.

But is there other data you're thinking of?
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: Relocate RPM and DNF databases to /usr

[Colin Walters]

On Wed, Dec 15, 2021, at 5:34 PM, Florian Weimer wrote:
> * Chris Murphy:
>
>> Fedora 36 seems like a good time to do this. What do you think?
>
> It's a bit odd to locate a database under /usr that isn't pre-built and
> installed.  

Why is that odd?  

> I guess in theory there could be systems with a read-only
> /usr out there that still allow installation of packages into /opt.
>
> Does it really help to improve the snapshot situation? 

Yes.  

I didn't wake up one day and say "hey you know what, today I'm going to move 
the rpm database just for fun".  Neither, for that matter did the OpenSUSE 
folks.  We haven't had this conversation over and over throughout the years 
just because it was some minor thing.

What I *did* wake up one day and say I'm going to do is make upgrades 
transactional and offline by default and hence safe.  No one should ever fear 
starting an operating system upgrade while their laptop is at 30% battery.  
Administrators running important servers must be able to easily roll back when 
the kernel *or* systemd (or something) else regresses, because it's software, 
it regresses all the time despite our best efforts.

So yes again, this does matter.  And it matters because whether you're doing 
"image based upgrades" like ostree or just "client side offline updates" like 
the 
https://kubic.opensuse.org/documentation/man-pages/transactional-update.8.html 
thing - it's very important *what data specifically* is versioned/snapshotted 
and what isn't.  On an ostree system for example, it's completely normal that 
there are *two* rpm databases (one you're running, one that's pending in the 
new root).  

All the data in the rpmdb is about content that's in `/usr`.


> What about
> software under /opt? 

https://github.com/coreos/rpm-ostree/issues/233
Today indeed, rpm-ostree explicitly denies that.  Which, note we can do because 
we basically take over chunks of what librpm is doing on non-ostree systems.  
But, it would be nice to drive some of this support into librpm too so it 
applies to non-ostree-but-snapshottable systems.


>  Maybe this needs multiple RPM databases
> eventually, roughly aligned along file system boundaries.

Yeah, that's one approach, though I would scope it really to just "/usr rpmdb" 
and "/usr/local" rpmdb - i.e. `/opt` and `/usr/local` are basically the same 
thing with different names.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: Relocate RPM and DNF databases to /usr

[Colin Walters]

On Thu, Dec 9, 2021, at 10:11 AM, Chris Murphy wrote:

>> The change is not so simple. It is not only the movement of files from one 
>> location to another one. We store more types of data in that location - 
>> history database (sqlite), module failsafe data (yamls). In future we will 
>> store system state data (toml). Data is not only modified after RPM 
>> transactions but also by module and mark commands. What I want to say is 
>> that the change will be painful but in the proposal there are limited 
>> benefits.

When the rpmdb was bdb and not extensible, I understand the idea of dnf having 
its own separate database.  But I don't understand why there are two separate 
sqlite databases now.

Anyways, a lot going on here and rather than point by point I'll say basically:

It'd be great from the rpm-ostree PoV to have a shared consistent place for the 
rpmdb.  Last time this came up I think Panu really wanted to get the sqlite 
change out first, which - fair.  But now that's done, so are there any other 
blockers?

We could try simply not changing existing systems in-place at first, but just 
do it for new installs, add the backcompat symlink, ensure the stack looks in 
both places (preferring new); that's actually mostly done AFAIK.

Today rpm-ostree has its own state management that doesn't use the libdnf 
history stuff.  Longer term it would be good to have a unified vision there - 
most of this is touched on in https://github.com/coreos/rpm-ostree/issues/2326

But I think I'm arguing to decouple the rpmdb move from the dnf move.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Document that using assert() is frowned upon (PR #1807)

[Colin Walters]

I think this is a nuanced topic.  Very recently, I started regretting an 
assertion we added in rpm-ostree (not a library, but a daemon providing a 
service, so acts like a library in some ways).  It's tangentially related to 
rpm actually, over here: https://github.com/coreos/rpm-ostree/pull/3183 around 
dealing with the bdb/sqlite transition.
To remove that assertion I need to go through and change various internal APIs, 
which is definitely the right thing here.

But then in ostree more recently, I started adding [more assertions to help out 
gcc -fanalyzer](https://github.com/ostreedev/ostree/pull/2463).  I tried 
building rpm with `-fanalyzer` and it looks like there might be some similar 
needs (or potentially real bugs).

See also [this article](https://lwn.net/Articles/342330/) about how missing 
these types of assertions can lead to the gcc optimizer doing something you 
really don't want.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1807#issuecomment-951045368___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Allow the transaction ID and installation time to be overridden (PR #1803)

[Colin Walters]

@cgwalters commented on this pull request.

Thank you so much for working on this!

> @@ -1296,3 +1296,30 @@ rpmtxn rpmtxnEnd(rpmtxn txn)
 }
 return NULL;
 }
+
+#define FAKE_CLOCK_INITIALIZED (1<<0)
+#define FAKE_CLOCK_ACTIVE  (1<<1)
+
+static int _fakeClockState = 0;
+static time_t _fakeClock = -1;

(Not an rpm maintainer myself but)

The term "fake" has negative connotations, but this is actually going to (IMO) 
be quite a normal thing to do.  Alternative terms: "synthetic", "external", 
"override" or so?



> @@ -1296,3 +1296,30 @@ rpmtxn rpmtxnEnd(rpmtxn txn)
 }
 return NULL;
 }
+
+#define FAKE_CLOCK_INITIALIZED (1<<0)
+#define FAKE_CLOCK_ACTIVE  (1<<1)
+
+static int _fakeClockState = 0;
+static time_t _fakeClock = -1;
+
+rpm_time_t rpmtsGetTime(time_t step)

It would seem cleaner to me to have `rpmtsInitTime()` and `rpmtNextTime()` or 
so, which correspond to `rpmtsGetTime(0)` and `rpmtsGetTime(1)`.

> @@ -1296,3 +1296,30 @@ rpmtxn rpmtxnEnd(rpmtxn txn)
 }
 return NULL;
 }
+
+#define FAKE_CLOCK_INITIALIZED (1<<0)
+#define FAKE_CLOCK_ACTIVE  (1<<1)
+
+static int _fakeClockState = 0;

It'd be nice to avoid more global mutable state in RPM.  Can we actually hook 
this clock state inside the `ts` structure?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1803#pullrequestreview-788150043___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Drag our CI to this decade (#1771)

[Colin Walters]

> OK, so this is strange. The dnf update command in the image fails with 
> getaddrinfo() thread failed to start, see the CI logs below. I can reproduce 
> this locally, but with Docker only.

https://bugzilla.redhat.com/show_bug.cgi?id=1958474 probably.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1771#issuecomment-914358704___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Installation / verification should not pass if the (sub)key(s) has been revoked or expired (#1598)

[Colin Walters]

Related discussion of this over here 
https://github.com/ostreedev/ostree/pull/2260


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1598#issuecomment-872509233___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RPM with Copy on Write (#1470)

[Colin Walters]

Big picture I'd summarize this as: You're proposing a big alternative way to 
handle RPMs that only works on reflink-enabled filesystems.

But the basic fact is that there's widespread use of non-reflink filesystems 
(ext4, older xfs) on Linux in general.  Even if this code was 100% complete and 
tested, we'd still have to support the non-reflink case for *years* to come for 
in-place upgrade support at the least.  

Don't get me wrong, I'd love to push towards a state were we can try to hard 
require reflinks in e.g. Fedora derivatives and make system software that just 
fails without it.   And require people using older versions to reprovision with 
a newer filesystem.  But it's hard to underestimate the disruption from that - 
the benefit has to be really large.

As I've said before in rpm-ostree we already carry a re-implementation of 
various chunks of librpm as part of the transactional/anti-hysteresis model.  
The concept of carrying "pristine" RPMs has been implemented from the start by 
importing RPMs into ostree commits, then using plain old hardlinks.  For 
`apply-live` updates, that also inherently implements the "unpack all files, 
rename() into place" model.

This would be a 3rd new thing with a new mix of tradeoffs.   Which isn't 
inherently bad, just noting.

Certainly for example, we'd need a way to *force off* reflink support so that 
the "traditional" path could be tested too.  If this ends up as an optional 
plugin, then that simplifes that aspect because then e.g. Anaconda could only 
install it if it detected the target filesystem chosen by the user supported 
reflinks.

Anyways beyond that high level, on the details of the code I don't have much to 
add beyond what the librpm maintainers have.  I hope we can figure out how to 
share more code between these 3 approaches (traditional librpm path, 
rpm-ostree, RPMCoW).  


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1470#issuecomment-856839607___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Rip out the atexit handler (#1667)

[Colin Walters]

This seems related to 
https://github.com/rpm-software-management/rpm/commit/56f49d7f5af7c1c8a3eb478431356195adbfdd25
 - in rpm-ostree for example the atomicity of the rpmdb is also done alongside 
the filesystem too - if we don't have a `SIGINT` handler then we shouldn't have 
an `atexit` handler either.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1667#issuecomment-849015032___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add --nocompression option to rpm2archive (#1657)

[Colin Walters]

@cgwalters approved this pull request.

Gave this a quick test and it works for me.Thanks!



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1657#pullrequestreview-645896828___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] rpm2archive: Support uncompressed output (#1530)

[Colin Walters]

See https://github.com/coreos/rpm-ostree/issues/2458
Basically it'd be inefficient for rpm-ostree to use rpm2archive today because 
it always compresses the output which we'd need to immediately uncompress again.

I looked at patch for this but got slightly lost in learning the ins&outs of 
popt arg parsing, but if this sounds ok I can try to finish that.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1530___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Document thread-safety of librpm (#1483)

[Colin Walters]

FWIW in rpm-ostree we reimplement most of the RPM install path for multiple 
reasons (among them we "snapshot" multiple RPM versions into ostree commits as 
part of implementing transactional updates, we want to sandbox scripts etc.) 
but another big reason is that we simply cannot have librpm call `chroot()` - 
that has process-global effects.  

(Also on this topic btw rpm-ostree is oxidizing fast now, see e.g. 
https://github.com/coreos/rpm-ostree/pull/2502 which I just submitted - so if 
there's RPM+Rust topics it might come up there too)



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1483#issuecomment-768456090___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] option to disable fsync (#1401)

[Colin Walters]

There's a recent [change to add volatile to 
overlayfs](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c86243b090bc)
 which will mostly mitigate this, although overlayfs isn't available to 
unprivileged code paths right now.

I guess in the end it is easier to just change the kernel rather than trying to 
patch various userspace projects.

See also [this twitter 
subthread](https://twitter.com/ibuildthecloud/status/1318928003422826497).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1401#issuecomment-718757869___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] option to disable fsync (#1401)

[Colin Walters]

> Are you aware of https://github.com/kjn/nosync

Looks like the same as what I mentioned above originally:

> I know about things like libeatmydata but the problem of course is that 
> librpm is linked into the same process (rpm-ostree) as the code that is doing 
> the ostree commit.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1401#issuecomment-711066926___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] option to disable fsync (#1401)

[Colin Walters]

For rpm-ostree, we rely on ostree for transactionality - ostree does fsync (or 
not) per its configuration.

As best I can tell, rpm doesn't offer an API to disable its use of fsync (for 
the database or for writing files, though I only really care about the former). 
 There are references to `nofsync` which I *think* is only parsed by the db3 
backend?

I know about things like 
[libeatmydata](https://github.com/stewartsmith/libeatmydata) but the problem of 
course is that librpm is linked into the same process (rpm-ostree) as the code 
that is doing the ostree commit.

We already use FUSE for scripts, so I could probably intercept the writes with 
FUSE but I'd imagine adding an API for this would be fairly straighforward, 
right?



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1401___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add dbus-announce plugin (#1255)

[Colin Walters]

> imposes its own signal handling on users when the db is open

Can turn that off since 56f49d7f5af7c1c8a3eb478431356195adbfdd25 right?

> and our ABI hasn't been particularly stable historically

Maintaining C shared libraries is indeed extremely hard.  I've messed up in the 
past and only just barely caught it before releases etc.  That said there are 
nice tools now like [libabigail](https://pagure.io/libabigail) - should be easy 
to set that up on CI here right?
(I still need to do that for ostree)

> dbus library itself is much more stable AIUI.

Yeah, libdbus as a C shared library has been fine and will continue to be so, 
but exposing an API over DBus is a separate thing with long term commitments.

If we're effectively saying most projects shouldn't be using librpm to read the 
rpm database effectively (that seems like what you're implying) that's a rather 
radical thing right?  Maybe I am misunderstanding though.  



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1255#issuecomment-649535595___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: add database change notification API (#1124)

[Colin Walters]

Just to echo 
https://github.com/rpm-software-management/rpm/pull/1255#issuecomment-645103007
here - inotify is used today for e.g. `/usr/share/applications` - when you e.g. 
`zypper/apt/yum/whatever install firefox` that's how the desktops pick up the 
change.

As far as portability, there are already a *lot* of cross platform wrappers for 
inotify.  For example, GLib has one.

So I think the simple solution is:

- Add an API to librpm to get a directory file descriptor (or path) to the 
rpmdb directory
- Change the code to whever something is written call `touch()` on that 
directory
- Suggest that higher level libraries use inotify (not rpm itself)


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1124#issuecomment-645350098___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add dbus-announce plugin (#1255)

[Colin Walters]

> This isn't just for dnf's benefit. It's a mechanism that anybody who needs to 
> do so is free to listen, and everybody else can ignore.

Speaking as a former DBus maintainer upstream here and I have written many DBus 
APIs...

It seems dramatically simpler to me to go with the inotify approach, the 
requirement is basically for librpm to `touch` the toplevel directory of the 
database path.  It's near zero cost if there is nothing listening.

And most crucially it avoids creating a new API for things to try to 
query/monitor the rpm database state.  There's already an API for that via 
librpm.  This signal isn't extensible - if e.g. a new field (say something 
related to modules) becomes interesting then we'd have to define a new one.  
Handling that requires going down to a "bag of properties" i.e. `a{sv}` in DBus 
type code model.  (We do this in many places in rpm-ostree) - but it loses some 
of the elegance.

A DBus API is a serious commitment, particularly once you have things listening 
to it.

That said a particular strength of DBus (really the reason it was created) is 
to be able to to cross privilege boundaries by having unprivileged processes 
(originally the desktop) sanely monitor and interact with system services.  But 
in this case since the consumer is (theoretically) dnf they're in the same 
privilege level (and in fact the same process!), so that doesn't apply.

FWIW here's the libostree code to bump mtime: 
https://github.com/ostreedev/ostree/blob/62632511f149adfc186da7f0e976006845591c8f/src/libostree/ostree-sysroot.c#L357

And here's where rpm-ostree uses it: 
https://github.com/coreos/rpm-ostree/blob/e6907d209b258388339b26e58a6da8cd022d1081/src/daemon/rpmostreed-sysroot.c#L788

That's how rpm-ostreed gets notified if e.g. an admin directly runs `ostree 
admin `, which is a lot like the situation where someone is using a 
traditional rpm system with a DBus daemon like dnfdragora/PackageKit or 
whatever and an admin uses `rpm -ivh` or so.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1255#issuecomment-645103007___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add dbus-announce plugin (#1255)

[Colin Walters]

> Well, we add this on request of the DNF team that need to be notified if 
> something else (rpm cli) changes the rpmdb underneath their daemon. 

Ah, I see.  Well, nothing else can/should be changing the rpmdb on rpm-ostree 
based systems (see also https://github.com/coreos/rpm-ostree/pull/1789 where 
rpm-ostree will start wrapping `/usr/bin/rpm`).

> So this is going to be a plugin that can be disabled or just plainly not be 
> installed in the first place. We will probably not integrate this into the 
> systemd-inhibit plugin but have it as a separate plugin that is very similar 
> and comes in it's own sub package.

OK that seems fine, then we can not install it in the container image and 
rpm-ostree based systems.

> Can you please elaborate on why sending some DBus signals should be 
> problematic?

A few things.  First, rpm-ostree updates are *offline* (generating a new root) 
by default - so anything listening and responding to these DBus signals would 
be confused because it's not affecting the *current* rpm database but a new one 
it can't see.

Second, and related to that - which exact component would be listening to these 
signals?  Would it be `dnf` or `libdnf`?  I really hope it's not the library 
because then rpm-ostree would be talking to *itself* over DBus which...yeah I 
would like not to have that happen :smile: 

Do we really need DBus for this versus just e.g. having interested processes 
use `inotify()` on the path to the rpmdb's directory, and ensuring that RPM 
does `touch(/path/to/rpmdb)` whenever it makes a change?  This is basically 
what we do for ostree changes.  (And yes in some cases rpm-ostreed does catch 
inotify updates from its own changes but that's kind of intentional and much 
less heavy-weight than talking to oneself over dbus)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1255#issuecomment-642022674___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add dbus-announce plugin (#1255)

[Colin Walters]

@cgwalters commented on this pull request.



> +state->logging = 1;
+
+/* ...don't log test transactions */
+if (rpmtsFlags(ts) & (RPMTRANS_FLAG_TEST|RPMTRANS_FLAG_BUILD_PROBS))
+   state->logging = 0;
+
+/* ...don't log chroot transactions */
+if (!rstreq(rpmtsRootDir(ts), "/"))
+   state->logging = 0;
+
+/* Don't open */
+if (!state->logging || state->bus)
+   return RPMRC_OK;
+
+if (lstat("/run/systemd/system/", &st) == 0) {
+if (S_ISDIR(st.st_mode)) {

DBus the protocol doesn't need systemd (it predates it) but lately integrates 
with it well.  The reason I think we do this check is it's a proxy for "is this 
system booted".

For example, there's no dbus running in a default podman/docker style container 
(neither is there systemd, or in fact usually there isn't any other processes 
at all except e.g. yum/zypper running).  So there's no reason to try announcing.

Related discussion and code: https://github.com/systemd/systemd/pull/7631


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1255#discussion_r436890836___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add dbus-announce plugin (#1255)

[Colin Walters]

Is there any coordination between this and the work to add dbus to libdnf in 
https://github.com/rpm-software-management/libdnf/pull/941 for example?

For rpm-ostree we are already a DBus daemon, and having multiple other 
libraries in the stack also going out and talking to DBus is going to be a bit 
problematic.  

Binding this to the systemd-inhibit plugin makes sense because we already turn 
that off for rpm-ostree (because it's transactional, there's no reason to 
inhibit).


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1255#issuecomment-640780302___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Support ed25519 signatures (#1202)

[Colin Walters]

Just to xref, in ostree we recently merged an ed25519 signing system too: see 
https://github.com/ostreedev/ostree/issues/1233
and https://github.com/ostreedev/ostree/pull/1878

The main motivation apparently is that GPG being lgplv3 and carrying the patent 
clauses is problematic for some people making embedded systems.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1202#issuecomment-634008594___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] WIP/RFE: Hint to users to use ostree/rpm-ostree if we get EROFS (#320)

[Colin Walters]

Just to xref, since this one was closed the current plan is for rpm-ostree to 
wrap rpm, instead of having rpm wrap rpm-ostree: 
https://github.com/coreos/rpm-ostree/pull/1789


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/320#issuecomment-614072259___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] 4.16: no longer possible to package invalid symlink (#1159)

[Colin Walters]

> This is, however, a regression and places us in a difficult situation wrt 
> packaging our tests.

You can just ship a tarball of this stuff instead right?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/1159#issuecomment-607461205___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Make database backends self-aware (#910)

[Colin Walters]

> Make database backends self-aware

:cold_sweat: 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/910#issuecomment-544540645___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

[Colin Walters]

Maybe key off `rpmsqSetInterruptSafety()`?  I don't think mock calls that yet 
but it probably should.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/900#issuecomment-542695270___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

[Colin Walters]

Oh wow.  Is there a way for me to unconditionally turn this off in rpm-ostree?  
Since our upgrades are always offline+transactional we never need it.

More generally, it seems like this plugin should be a no-op if we're installing 
into a root != `/` for e.g. the mock case right?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/900#issuecomment-542693704___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Remove "support" for loading keyring from filesystem (#857)

[Colin Walters]

(Except I think that code is still not used by dnf because it dates to the 
PackageKit C vs yum Python days?)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/857#issuecomment-535674087___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Remove "support" for loading keyring from filesystem (#857)

[Colin Walters]

Also worth noting https://github.com/rpm-software-management/libdnf/issues/43
(libdnf auto-injects `/etc/pki/rpm-gpg`)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/857#issuecomment-535673882___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Fix excessive use of thread local storage (RhBug:1722181) (#759)

[Colin Walters]

LGTM.   Regarding

> The __thread uses in rpmsq don't make sense at all in reality, so better 
> revert and rethink from scratch.

I don't know what all is writing/reading to those variables but, global mutable 
statics are basically always wrong.  In Rust for example a common pattern is 
`lazy_static!` + `Mutex`, like [this bit of code in 
rpm-ostree](https://github.com/projectatomic/rpm-ostree/blob/c94bd08b02499a3f2a5804f5c2ba207a0afbda18/rust/src/progress.rs#L50).

(Or of course for integer variables like thise atomics can be used)


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/759#issuecomment-504019053___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Compress annobin notes (#751)

[Colin Walters]

Can you keep the commit message title to < 80 chars?  And move the text from 
the PR description into the commit message as well.

Seems OK to me but: why don't the compilers do this by default?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/751#issuecomment-502178666___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RPM must kill all childs on exit (from section?) (#134)

[Colin Walters]

Keep in mind that if rpmbuild itself used container features it'd heavily 
overlap with tools like `mock` - and while recursive containerization does 
work, it really requires both ends to be ready for it.

See also https://github.com/projectatomic/bubblewrap/issues/284

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/134#issuecomment-422080378___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] luaext/Pexec: optimize setting CLOEXEC (#444)

[Colin Walters]

At a quick glance anyways, your code looks fine to me.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/444#issuecomment-389980176___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] luaext/Pexec: optimize setting CLOEXEC (#444)

[Colin Walters]

glib code here: 
https://gitlab.gnome.org/GNOME/glib/blob/487b1fd20c5e494366a82ddc0fa6b53b8bd779ad/glib/gspawn.c#L1207

Also worth looking at what systemd does.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/444#issuecomment-389979013___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Make sure all dependencies for scriptlet are installed before executing them (#436)

[Colin Walters]

FWIW, in rpm-ostree today we lay out all files before executing scripts.  Then 
we execute all `%pre`, then `%post` and `%posttrans`.   Becuase of the `%pre` 
-> passwd dependency, we then do chowns after `%pre`.  Also, [speaking of 
dependency loops and 
scripts](https://github.com/projectatomic/rpm-ostree/blob/e34f17fc3ee9b29f05d9910e9dff52a260964cfa/src/libpriv/rpmostree-core.c#L3813)...

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/436#issuecomment-384287110___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

More followup, just a FYI.  Note that in this PR the whole of
/usr/lib/sysimage is just usr_t:

https://github.com/fedora-selinux/selinux-policy-contrib/pull/43
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] COPYING: Minor grammar fixes (#380)

[Colin Walters]

Updated :arrow_up: to use your suggestion - though can you help me understand: 
what's the difference between "RPM" and "its source code"?  Documentation? 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/380#issuecomment-35552___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] COPYING: Minor grammar fixes (#380)

[Colin Walters]

I was looking at this file in the context of dnf/rpm-ostree integration:
https://github.com/rpm-software-management/dnf/pull/991#issuecomment-355587679

The incorrect use of "it's" was distracting; I ended up rewording the initial
sentence to be more direct.

The second hunk is another grammar fix.
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/380

-- Commit Summary --

  * COPYING: Minor grammar fixes

-- File Changes --

M COPYING (4)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/380.patch
https://github.com/rpm-software-management/rpm/pull/380.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/380
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

FYI:

https://src.fedoraproject.org/rpms/filesystem/pull-request/3
and
https://github.com/projectatomic/rpm-ostree/pull/1142
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

[/me moves to center of thread, pulls out defibrillation machine, *zzzt* *zzzt*]

On Thu, Oct 26, 2017, at 04:59 AM, Richard Brown wrote:
>
> As we're pressed on time, openSUSE will be going right ahead and moving it's 
> rpmdb to /usr/lib/sysimage/rpm as
> fast as possible

I assume this was done?

> Meanwhile I plan to take all of the points that have bubbled up as part of 
> this conversation and present them
> to Debian/yum/dnf/PackageKit/Arch to try and get them onboard with the idea 
> in the hope that we can start
> coalescing a cohesive solution around this idea

Before we move on to other packaging ecosystems, a useful next step to me seems 
to
be teaching various libs (librpm, libsolv, libdnf) to look in this location 
first.   Maybe
something like supporting %dbpath as a list in librpm?  Or maybe have it be
empty by default, and if it's unset that means "find automatically"?
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

On Wed, Oct 18, 2017, at 06:56 AM, Panu Matilainen wrote:

> I don't see /usr/lib as a *good* place for this, but if a new top level 
> directory in /usr seems too much then maybe /usr/lib/sysimage or such 

I'm OK with /usr/lib/sysimage.

> As Neal pointed out, there's dpkg and the other package management 
> systems that put their stuff in /var too. And you don't need to go even 
> outside the rpm land for those other things, there's yum/dnf/PackageKit 
> (and who knows what else) that keep their own databases on 
> install-related information in /var.

Yes; those are generally not designed with snapshots in mind either.
For rpm-ostree, we import each RPM into an ostree repo
(https://github.com/projectatomic/rpm-ostree/pull/1055 for some
 current work in this area)
and we put metadata (e.g. the source repo ID) in the ostree commit:
https://github.com/projectatomic/rpm-ostree/blob/c107a05b8e26ecfe30c679dd98570bfc4cf38005/src/libpriv/rpmostree-unpacker.c#L432
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

On Thu, Oct 12, 2017, at 07:18 AM, Panu Matilainen wrote:
> 
> Rpm is not the only data of this kind, I can think of at least one other 
> similar need (SWID) and almost certainly there are more. Why not give 
> this data a place of its own? Something like
> 
> /usr/sysimage

Hm...creating a new subdirectory of /usr isn't something to do lightly;
for example in the FHS /usr/X11 is explicitly defined, but that's just
a legacy nowadays -  xorg has moved into /usr/lib and /usr/share just
like everything else.

Creating a new subdirectory of /usr seems to imply it's *really* special,
but...take the example of systemd unit files, which live in 
/usr/lib/systemd/system.
I could certainly imagine the systemd developers saying systemd unit
files are so critical to system operation that the deserve higher visibility, in
say /usr/systemd.
But they choose (IMO rightly) not to do that.   Don't get me wrong, I find
putting things like this in /usr/lib a little bit weird.  But it's not clear to 
me that
rpm is quite special enough to just do this right now.   Particularly not
without e.g. getting buy-in from say the dpkg/gentoo/arch/whatever maintainers
too right?
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

On Tue, Oct 10, 2017, at 03:41 PM, Richard Brown wrote:
> On 2017-10-10 20:05, Colin Walters wrote:
> 
> > My opinion here boils down to: if rpm upstream is happy with 
> > /usr/lib/rpmdb,
> > I'm happy to do the work of changing rpm-ostree to use that.
> 
> That's great news, I'm happy to help if I can

Just landed some prep:
https://github.com/projectatomic/rpm-ostree/pull/1048

> When I started out looking at this, I agreed with you.
> I started the discussion within the openSUSE community advocating for 
> following in rpm-ostrees footsteps with /usr/share.
> But the arguments, not just theoretical, but real world examples where 
> /usr/share would be problematic but /usr/lib not were compelling.
> Giving up on the idea of /usr/share led to plenty of pitchforks and 
> torches being put away, especially once we added the "db" suffix also :)

Heh, yeah understood.

> We're planning on making this change for all of our distributions via 
> all of their distribution methods, for consistencies sake.
> Given how we build our containers, we'd actually have to do extra work 
> to exclude my patch and restore the old location once my submit-request 
> is accepted in OBS :)

A single patch?  Can you link to it?  I looked through the thread some
but couldn't find it.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RFC: SUSE/openSUSE's proposed relocation of /var/lib/rpm

[Colin Walters]

On Mon, Oct 9, 2017, at 11:25 AM, Richard Brown wrote:

> The discussion currently boils down to either copying rpm-ostree and placing 
> our rpmdb in /usr/share/rpm, or
> locating it in /usr/lib/rpmdb

I definitely like the "db" suffix, makes it less of a potential naming clash if
librpm decided to have arch-independent data.

> Product timetables are likely to force SUSE/openSUSE to adopt one of the 
> above options relatively soon, so any
> quick feedback and thoughts will be greatly appreciated. 

My opinion here boils down to: if rpm upstream is happy with /usr/lib/rpmdb,
I'm happy to do the work of changing rpm-ostree to use that.

(Longer version: I don't find the "sharable architecture-independent data" 
argument to
 be a very strong one today; I'm sure one could find people doing that but
 I suspect most of those people are doing full NFS root or PXE-live + 
NFS-for-/usr
 etc.  But OTOH, I don't think it'll be difficult for rpm-ostree to make the 
move,
 and I definitely like the "db" suffix, so I'm ending up as a weak-ish +1)

Random question: Is OpenSUSE planning to make this change for their
container/Docker images too, or just host systems managed via snapper?

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] WIP/RFE: Hint to users to use ostree/rpm-ostree if we get EROFS (#320)

[Colin Walters]

> @cgwalters: This patch looks very specific to RPM-OSTree. Is there not a 
> better, more general way to do this?

More general to...other image systems that happen to use rpm?  Possibly.  As 
far as I'm aware though rpm-ostree is fairly unique in the way it's a hybrid 
image/package system.   I guess the old oVirt Node "classic" model might apply, 
AFAIK they shipped rpm but had no unlock functionality even.  (slapping an 
overlayfs on /usr is really handy!).  But they switched to 
https://github.com/fabiand/imgbased with everything writable so `yum` works.  
(But `yum` is totally unaware of the underlying image system)

> if ostree permits live update even with -EROFS, then your patch should teach 
> rpm to do similar live-update, not spew nagware adverts.

Well, it's not an advertisement - we don't and will never support librpm doing 
(persistent¹) writes.  This isn't like how `dnf` (used to thankfully) print a 
message and continue for people who type `yum`.  This case is a hard error. 

So...a path we could pursue instead of this would be having `/usr/bin/rpm` be a 
symlink → `/usr/bin/rpm-ostree`.  There's a lot of advantages to that, but it'd 
also be obviously a large maintenance overhead of detecting operations we want 
to intercept (basically writes like `-i`, `-U`, but not `-q`).  If consensus 
favors that I'd be OK with doing that instead.

¹ `rpm -Uvh` etc work fine on top of `ostree admin unlock`, which is 
intentional and very handy (and in fact how I tested this patch, by building a 
new rpm of rpm and installing it that way).  But the way persistent changes 
work is totally different.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/320#issuecomment-328296999___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] WIP/RFE: Hint to users to use ostree/rpm-ostree if we get EROFS (#320)

[Colin Walters]

BTW, down the line I'd like to extend this ostree-detection functionality a bit 
more so that e.g. `rpm -V` understands that timestamps are different, and 
that's OK.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/320#issuecomment-328109875___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Sync disks at the end of transactions (RhBug:1461765)" (#318)

[Colin Walters]

In active use?  I don't think there's that many.   Certainly the original bug 
report was from someone using dnf.

I just don't understand the logic for pushing this directly into librpm without 
even making it configurable after all of the feedback from people using librpm 
for higher level tools on the original commit.

The simplest would seem to be making `rpmtsSync()` public API.  Then higher 
level tools could call it as desired.  Having direct `rpm -i` calls invoke it 
and whether or not to do it by default seems like something that could be 
addressed later.

And to repeat, I definitely don't want this for rpm-ostree; we already have our 
own `syncfs()` calls and there we actually check the return codes for example; 
and make it so that if anything failed, the system is left unchanged.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/318#issuecomment-328069038___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] WIP/RFE: Hint to users to use ostree/rpm-ostree if we get EROFS (#320)

[Colin Walters]

Hi @herrold - I read your comment a few times and I'm confused...isn't that 
what my patch is doing?

BTW, it's easy to try an rpm-ostree based system, see 
https://getfedora.org/atomic/ and https://pagure.io/workstation-ostree-config 
for example.  One thing I'd note though is that the rpm database is read-only 
from the point of view of everything *except* rpm-ostree - we support both 
[package 
layering](https://www.projectatomic.io/blog/2016/07/hacking-and-extending-atomic-host/)
 and [live package 
layering](http://www.projectatomic.io/blog/2017/06/rpm-ostree-v2017.6-released/).
  But even when doing live changes, the system still appears read-only to 
everything else due to some underlying ostree magic.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/320#issuecomment-327962015___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] WIP/RFE: Hint to users to use ostree/rpm-ostree if we get EROFS (#320)

[Colin Walters]

Down the line, it's likely that for rpm-ostree based systems we'll
install an interceptor for `/usr/bin/rpm` to redirect at least things
like local package installs.  There's a lot of work to do for
that, so in the meantime let's help admins out by giving them a
helpful error message.
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/320

-- Commit Summary --

  * Hint to users to use ostree/rpm-ostree if we get EROFS
  * lib/transaction: Don't SEGV in sync if a txn was failed

-- File Changes --

M lib/rpmlock.c (24)
M lib/transaction.c (6)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/320.patch
https://github.com/rpm-software-management/rpm/pull/320.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/320
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] WIP/RFE: Hint to users to use ostree/rpm-ostree if we get EROFS (#320)

[Colin Walters]

This PR rolls in https://github.com/rpm-software-management/rpm/pull/319

After:
```
# rpm -e atomic
error: This system is managed by rpm-ostree
error: can't create transaction lock on /var/lib/rpm/.rpm.lock (Read-only file 
system)
```

This is still a WIP/RFE - the code here isn't pretty obviously, happy to have 
feedback about a better way to do it.  (Also honestly I'm struggling to 
convince Emacs to do this mix of tabs/spaces in the rpm codebase)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/320#issuecomment-327870580___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] lib/transaction: Don't SEGV in sync if a txn was failed (#319)

[Colin Walters]

I was actually testing a change to better handle `rpm -e atomic` on Fedora
Atomic Host, wondering why my patch was crashing, but in fact it was the
recently added sync code in master.
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/319

-- Commit Summary --

  * lib/transaction: Don't SEGV in sync if a txn was failed

-- File Changes --

M lib/transaction.c (6)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/319.patch
https://github.com/rpm-software-management/rpm/pull/319.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/319
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Sync disks at the end of transactions (RhBug:1461765)" (#318)

[Colin Walters]

Anyone (including you) could trivially submit a patch to dnf or whatever to 
call `sync` after transactions and have that ship to users pretty quickly if 
that's what you want.  That'd address the original bug report, no?

Why should this be hardcoded in librpm?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/318#issuecomment-327793752___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Sync disks at the end of transactions (RhBug:1461765)" (#318)

[Colin Walters]

Or stated another way, if you change dnf it'd feel for the vast majority of 
people that it defaults to on, and do so pretty quickly.  Right?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/318#issuecomment-327793958___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Sync disks at the end of transactions (RhBug:1461765) (b7a869f)

[Colin Walters]

https://github.com/rpm-software-management/rpm/pull/318

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/commit/b7a869f0f322cbe428e78150f2c175abea4c8c4b#commitcomment-24120847___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm-software-management/rpm] Revert "Sync disks at the end of transactions (RhBug:1461765)" (#318)

[Colin Walters]

This reverts commit b7a869f0f322cbe428e78150f2c175abea4c8c4b.

This would need to be at a minimum configurable, and default to off for
compatibility with software like rpm-ostree that is already taking care of all
of this.
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/318

-- Commit Summary --

  * Revert "Sync disks at the end of transactions (RhBug:1461765)"

-- File Changes --

M lib/transaction.c (4)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/318.patch
https://github.com/rpm-software-management/rpm/pull/318.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/318
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Sync disks at the end of transactions (RhBug:1461765) (b7a869f)

[Colin Walters]

BTW, see also https://github.com/ostreedev/ostree/pull/1049

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/commit/b7a869f0f322cbe428e78150f2c175abea4c8c4b#commitcomment-24120807___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Sync disks at the end of transactions (RhBug:1461765) (b7a869f)

[Colin Walters]

rpm-ostree also has its own `syncfs()` calls and we definitely don't want 
librpm doing it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/commit/b7a869f0f322cbe428e78150f2c175abea4c8c4b#commitcomment-24120791___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Adding an event loop in RPM to run asynchronous operations. (#258)

[Colin Walters]

> all file operations are durable (in the sense that all installed files are 
> known written to disk).

Right now many real world systems rely on files written by scripts (for 
example, the bootloader configuration, `/etc/ld.so.cache`, etc.); librpm as 
designed today doesn't have visibility into those.  This is a problem that 
rpm-ostree fixes, because we generate a new root, and commit the entire thing 
(including scripts) into ostree, which itself handles `syncfs()` for objects.

> all dirty pages in the buffer cache are invalidated (so there is no I/O cache 
> blowout from performing an RPM upgrade).

I still think a chunk of this problem is in the kernel 
 - but userspace should probably allow some 
configuration here too...there is a `RWF_HIPRI` flag to `pwritev2`, maybe we 
could add a `RWF_LOWPRI` flag that was the opposite.  There is of course also 
the [block IO 
controller](https://www.kernel.org/doc/Documentation/cgroup-v1/blkio-controller.txt)
 - today since `rpm-ostree` always operates as a daemon, an admin could 
conveniently place it into a different cgroup.  But one could of course do 
something equivalent with [systemd 
run](https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdForMemoryLimiting) 
wrapping `yum` or whatever.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/258#issuecomment-320752370___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Semaphore CI (#266)

[Colin Walters]

cgwalters commented on this pull request.



> @@ -0,0 +1,49 @@
+FROM fedora
+MAINTAINER Igor Gnatenko 
+
+WORKDIR /opt/rpm
+COPY . .
+
+RUN echo -e "deltarpm=0\ninstall_weak_deps=0\ntsflags=nodocs" >> 
/etc/dnf/dnf.conf
+RUN dnf -y update
+RUN dnf -y install \

Which calls into this: 
https://github.com/projectatomic/rpm-ostree/blob/f25444554dac95c5ff469f64192bad17a2916c3a/ci/libbuild.sh#L18

(Which is a good example of a minor annoyance from the yum → dnf rename)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/266#discussion_r128541216___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Semaphore CI (#266)

[Colin Walters]

cgwalters commented on this pull request.



> @@ -0,0 +1,49 @@
+FROM fedora
+MAINTAINER Igor Gnatenko 
+
+WORKDIR /opt/rpm
+COPY . .
+
+RUN echo -e "deltarpm=0\ninstall_weak_deps=0\ntsflags=nodocs" >> 
/etc/dnf/dnf.conf
+RUN dnf -y update
+RUN dnf -y install \

In some [PAPR](https://github.com/projectatomic/papr) projects like rpm-ostree 
we just pull the builddeps from the distro, I think this is a lot saner than 
repeating them here again.  See [this 
example](https://github.com/projectatomic/rpm-ostree/blob/f25444554dac95c5ff469f64192bad17a2916c3a/ci/build.sh#L20).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/266#pullrequestreview-51231058___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Don't modify interupts if db is read only (#251)

[Colin Walters]

Just a note to remember that for generating new roots (i.e. not inplace 
upgrades), we can just let SIGINT kill us.  For rpm-ostree, every operation is 
a new root, so we [turn off rpm's SIGINT 
completely](https://github.com/projectatomic/rpm-ostree/commit/ddf2227ae9a7d91b8e7e580dbe8c5d21120d24d6).
 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/251#issuecomment-315350162___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add macro to force fsync() on close() (#187)

[Colin Walters]

@n3npq Regarding full-system durability - I'm the maintainer of 
https://github.com/projectatomic/rpm-ostree/ which uses 
https://github.com/ostreedev/ostree/ to implement full-system transactional 
updates.  Every change (package install, update) is atomic.  libostree 
currently [stages objects per boot ID, then uses 
syncfs()](https://mail.gnome.org/archives/ostree-list/2017-January/msg0.html).



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/187#issuecomment-294845668___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add macro to force fsync() on close() (#187)

[Colin Walters]

@jaymzh But are your tests with a kernel with the writeback changes or not?  It 
looks like this commit? 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e34cbd307477ae07c5d8a8d0bd15e65a9ddaba5c
Which offhand appears to be in v4.10.   Ah yes, [kernelnewbies has a 
link](https://kernelnewbies.org/LinuxChanges#head-aa8362f44837f5f05342ca34683ccd63642466b0).

I'm not saying it's a bad idea to patch RPM - there will be people using older 
kernels for a while.  But it is an argument against extensively polishing the 
patch and testing alternative changes, and possibly dropping the functionality 
after some time.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/187#issuecomment-294288704___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add macro to force fsync() on close() (#187)

[Colin Walters]

Isn't this https://lwn.net/Articles/682582/ ?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/187#issuecomment-294231945___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: add a digest on the compressed payload content (#163)

[Colin Walters]

Okay, but that'd also be caught by MD5, right?  So...do we expect every package 
system to verify *both* the rpm-md checksum and this one?  Running SHA256 or 
whatever *is* pretty cheap, I know.

Perhaps enough people rely on "untrusted rpm-md fetched over http + GPG signed 
RPMs" that we have to fix this.  But I think greater security comes from 
pushing everyone to do [cert pinned 
rpm-md](https://pagure.io/fedora-infrastructure/issue/5372).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/163#issuecomment-283363152___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: add a digest on the compressed payload content (#163)

[Colin Walters]

In practice though, people shouldn't be using raw `rpm` to install RPMs.  They 
should (and 90% of the time are) using a higher level system like zypper, yum, 
or rpm-ostree.  

These systems all consume "rpm-md/yum" metadata, which obviously today has a 
checksum over the content, which can be verified without opening the RPM.

I know they're not the same - having a checksum just over the content as 
opposed to header+content should (AIUI) allow us to GPG sign without 
invalidating the content checksum (right?).

But it's surprising to me that we'd do something here without (apparently) 
considering how it interacts with rpm-md.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/163#issuecomment-283343716___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] heap out of bounds read in copyTdEntry() (#133)

[Colin Walters]

Need to check if this happens before GPG verification.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/133#issuecomment-275155371___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Check return value of uname() (#132)

[Colin Walters]

Under what conditions?  An aggressive seccomp policy?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/132#issuecomment-274226643___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: BuildRequires generator (#104)

[Colin Walters]

I think the solution is to not try to improve RPM here, just do what everyone 
in the world that hit this has done: move to generating specs via a better 
build system.

Examples:

 - [Yocto/OpenEmbeded](https://www.yoctoproject.org/)
 - [Clear Linux autospec](https://github.com/clearlinux/autospec)

And there's the vast existing set of dynamic language -> spec tools, like the 
venerable [cpan2rpm](http://search.cpan.org/dist/cpan2rpm/cpan2rpm)
and this [gofed](https://github.com/gofed/gofed) one for golang in fedora.

There are obviously challenges to this route too, like if you support 
*multiple* generators then they need to learn how to interact for dependencies. 
 However, I think a lot of those issues can be solved by having everything use 
provides rather than package names for example.  (And this would require 
generating provides for everything)



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/104#issuecomment-264534317___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] testing GHPRB (#67)

[Colin Walters]

Any chance you could set up Jenkins against e.g. your personal fork of the repo 
and test things there first?

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/67#issuecomment-213441008___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] te: Don't reload header for JUSTDB transactions (#52)

[Colin Walters]

Note I don't care about transaction locking - everything rpm-ostree does is 
always atomic.  We create a new (hardlinked) tree, then a new rpmdb from 
scratch every time.

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/52#issuecomment-174513906___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] te: Don't reload header for JUSTDB transactions (#52)

[Colin Walters]

I ended up storing lead/sig/header in the ostree commit, then checking it out 
as a tempfile and passing it back to librpm.  So we can close this.

Long term, I think what I really want is to make `rpmdbOpen` and `rpmdbAdd` 
public.  Then I can stay out of the business of writing to the db and dealing 
with berkeleydb, but know I have total control over everything else.

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/52#issuecomment-174322177___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] te: Don't reload header for JUSTDB transactions (#52)

[Colin Walters]

Closed #52.

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/52#event-524294958___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] te: Don't reload header for JUSTDB transactions (#52)

[Colin Walters]

Ideally, we just let `mmap()` handle that.  That's what OSTree does for 
metadata by default.  It'd probably work for packed RPMs as well...except in 
practice we byteswap in `regionSwab()` so require a writable mapping =/  For 
OSTree, GVariant handles a lot of this automatically for interior data, for 
"exterior" integers we just byteswap on access.

Anyways, one option I looked at would be for OSTree to store *everything* from 
the original RPM (lead, signature, header), then attempt to re-synthesize it 
from the ostree commit when RPM asks for it.  

Downsides:
  - A pipe-to-self is just ugly and inefficient
  - I think I'd have to parse the RPM myself to accurately extract the start of 
the cpio stream

Positives:
 - We'd also store the original gpg sigs

I guess regardless, it'd be nice if RPM supported (somehow) a mode where the 
calling application could optionally provide just a `Header` reference, not a 
full fd to an RPM.


---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/52#issuecomment-173925491___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [rpm] te: Don't reload header for JUSTDB transactions (#52)

[Colin Walters]

I'm working on some custom rpm-ostree code to "install" rpms by
unpacking them with libarchive (since this can be a lot safer than
invoking librpm - my process has total control over writes), then just
using librpm to update the database via _JUSTDB.

However, it appears that for some reason that if the transaction is
not _TEST, we attempt to reload the header from the original package
here.  I don't have the original package, just the header, since they
get unpacked into OSTree commits (with the header as ostree commit
metadata).

Now, I don't understand why we re-parse the RPM - what's wrong with
the original header we loaded?, but it fixes the problem for me to
also skip with _JUSTDB, and should be a very low risk change.
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/52

-- Commit Summary --

  * te: Don't reload header for JUSTDB transactions

-- File Changes --

M lib/rpmte.c (62)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/52.patch
https://github.com/rpm-software-management/rpm/pull/52.diff

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/52
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] Ft2 (#43)

[Colin Walters]

Is there a reason for `runImmedFileTriggers` not to chroot?


---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/43#issuecomment-170393488___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm] Rename expandMacrosU to rpmExpandMacros (#34)

[Colin Walters]

Looks sane to me, FWIW.

---
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/34#issuecomment-158985853___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [RFC v5 00/11] RPM: Include and install file signatures

[Colin Walters]

On Tue, Jan 27, 2015, at 10:04 AM, f...@linux.vnet.ibm.com wrote:
> IMA-appraisal, upstreamed in linux-3.7, enforces local file integrity based on
> a known 'good' value stored as an extended attribute 'security.ima'. Labeling
> the filesystem is currently done post install using a local private key.
> Including file signatures in the package provides not only file integrity, but
> file provenance.

This is interesting stuff, and it's a topic I've been thinking about quite a 
bit lately.
I definitely like the idea of being able to verify that the on-disk software 
state
matches what was signed.

The more I think about IMA though, it would seem to me to be significantly
cleaner if the system (really kernel) just disallowed modification of system 
files in the first
place, rather than asserting it externally.

OSTree tries to do this now with a read-only bind mount over /usr as well as
the immutable bit on /.  A step further is:
http://www.spinics.net/lists/linux-fsdevel/msg75085.html

Are you doing anything special with respect to RPM %config() files?
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Add API to completely disable librpm's use of Unix signal handlers

[Colin Walters]

On Tue, Feb 17, 2015, at 07:07 AM, Florian Festi wrote:
> Sorry, for the last response. DevConf takes its toll...
> 
> On 01/23/2015 04:07 AM, Colin Walters wrote:
> > Numerous consumers of librpm use it in a pattern where they're
> > constructing fresh chroots.  For example, rpm-ostree operates this
> > way, and is used to provide atomic upgrades in concert with rpm.
> > 
> > If the process dies due to SIGINT or another signal, the root can
> > simply be discarded.
> > 
> > Currently today, rpm-ostree undoes the signal handlers after loading
> > librpm so that Control-C does what I want, but there's still a race
> > condition where the interrupt can be lost.
> > 
> > Add an API so callers can disable the behavior.
> 
> Is there any chance someone would want to switch them back on? 

I can't think of one offhand...tools that interact with a live root
should be happy with what RPM does today, right?

> My gut
> feeling tells me this should rather be rpmsqSetInterruptSafety(int on);

But here's a patch which does it, in case you prefer it.  I did write
a better API doc this time.
From ae6d2de85b7b81cf91318183ba253402ac538785 Mon Sep 17 00:00:00 2001
From: Colin Walters 
Date: Thu, 22 Jan 2015 17:57:14 -0500
Subject: [PATCH] Add API to disable librpm's use of Unix signal handlers

Numerous consumers of librpm use it in a pattern where they're
constructing fresh chroots.  For example, rpm-ostree operates this
way, and is used to provide atomic upgrades in concert with rpm.

If the process dies due to SIGINT or another signal, the root can
simply be discarded.

Currently today, rpm-ostree undoes the signal handlers after loading
librpm so that Control-C does what I want, but there's still a race
condition where the interrupt can be lost.

Add an API so callers can disable the behavior.
---
 rpmio/rpmsq.c | 26 ++
 rpmio/rpmsq.h |  2 ++
 2 files changed, 28 insertions(+)

diff --git a/rpmio/rpmsq.c b/rpmio/rpmsq.c
index bfd4db0..8515a0e 100644
--- a/rpmio/rpmsq.c
+++ b/rpmio/rpmsq.c
@@ -16,6 +16,7 @@
 
 #include "debug.h"
 
+static int disableInterruptSafety;
 static sigset_t rpmsqCaught;
 
 typedef struct rpmsig_s * rpmsig;
@@ -70,6 +71,9 @@ int rpmsqEnable(int signum, rpmsqAction_t handler)
 rpmsig tbl;
 int ret = -1;
 
+if (disableInterruptSafety)
+  return 0;
+
 for (tbl = rpmsigTbl; tbl->signum >= 0; tbl++) {
 	if (tblsignum != tbl->signum)
 	continue;
@@ -112,3 +116,25 @@ int rpmsqEnable(int signum, rpmsqAction_t handler)
 return ret;
 }
 
+
+/** \ingroup rpmio
+ * 
+ * By default, librpm will trap various unix signals such as SIGINT and SIGTERM,
+ * in order to avoid process exit while locks are held or a transaction is being
+ * performed.  However, there exist tools that operate on non-running roots (traditionally
+ * build systems such as mock), as well as deployment tools such as rpm-ostree.
+ *
+ * These tools are more robust against interruption - typically they
+ * will just throw away the partially constructed root.  This function
+ * is designed for use by those tools, so an operator can happily
+ * press Control-C.
+ *
+ * It's recommended to call this once only at process startup if this
+ * behavior is desired (and to then avoid using librpm against "live"
+ * databases), because currently signal handlers will not be retroactively
+ * applied if a database is open.
+ */
+void rpmsqSetInterruptSafety(int on)
+{
+  disableInterruptSafety = !on;
+}
diff --git a/rpmio/rpmsq.h b/rpmio/rpmsq.h
index 00a1a72..22bfcc5 100644
--- a/rpmio/rpmsq.h
+++ b/rpmio/rpmsq.h
@@ -52,6 +52,8 @@ void rpmsqAction(int signum);
  */
 int rpmsqEnable(int signum, rpmsqAction_t handler);
 
+void rpmsqSetInterruptSafety(int on);
+
 #ifdef __cplusplus
 }
 #endif
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH 2/2] set mtime of all .py files to date of last Changelog

[Colin Walters]

On Wed, Aug 13, 2014, at 07:12 AM, bmorb...@redhat.com wrote:

> This ensures that rebuilds of an srpm result in the same .py{c,o} files,
> even if the source files were modified during the build.
> This should make the builds of most python packages reproducible.

I like the idea here.  Timestamps on .pyc files have caused me
extreme pain in the past wrt multilib ( 
https://bugzilla.redhat.com/show_bug.cgi?id=718404 )

> -depth=`(find "$RPM_BUILD_ROOT" -type f -name "*.py" -print0 ; echo /) | \
> +depth=`(find "$RPM_BUILD_ROOT" -type f -name "*.py" -exec touch 
> "$RPM_CHANGELOG_DATE" {} \+ -print0 ; echo /) | \
> xargs -0 -n 1 dirname | sed 's,[^/],,g' | sort -u | tail -n 1 | wc -c`
>  if [ -z "$depth" -o "$depth" -le "1" ]; then
>   exit 0

At some point, it becomes a lot saner to use an actual programming 
language...this might be the time?  Can we write brp-python-bytecompile in 
python?

I'm guessing the reason this patch broke is it looks like you're missing the 
";" to close the -exec ?
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] whitelisting/wrapping script execution

[Colin Walters]

10 months later, I finally made some progress on this. See:

https://github.com/hughsie/libhif/pull/39

Which is used by: https://github.com/projectatomic/rpm-ostree/pull/107

However, to make this more useful, I'd like to move forward on having
"known OSTree safe" scriptlets (e.g. ldconfig) in known to RPM. Any
thoughts on that?

By far the simplest approach would be moving as much as possible to
%posttrans. For things like ldconfig, it *almost* works except for the
case of %post which executes a binary that uses a new shared library
introduced by a previous package in the transaction. We could change
librpm to automatically delete /etc/ld.so.cache and force the
search-all-the-paths mode of the dynamic linker until the end. I doubt
anyone would notice the difference.

On Wed, Apr 23, 2014, at 08:13 AM, Colin Walters wrote:
> Hi,
>
> For the rpm-ostree project, I need the ability to whitelist
> scriptlets. There are a few reasons for this, but the most critical is
> that due to the way OSTree uses hardlinks, and because I can't assume
> I have copy-on-write links, I need to know that scriptlets will
> cleanly break hardlinks instead of edit-in-place.
>
> For example, suppose I'm layering a package on top of a base tree that
> contains a shared library. We need to update /etc/ld.so.cache. In the
> traditional RPM model, that package will contain a scriptlet to run
> /sbin/ldconfig.
>
> I happen to know that /sbin/ldconfig is safe - it creates
> /etc/ld.so.cache~ and rename()s into place, so I can just run it. But
> if the package happens to do something I haven't whitelisted as safe,
> I need to error out and refuse to install it.
>
> (Note an important side benefit here of moving away from the "whatever
> arbitrary code you want that runs as root" aspect of scripts)
>
> Now related to this, I'd like to have rpm-ostree take care of
> executing any subprocesses (and even Lua code potentially). The
> reason for this is because *every* rpm-ostree install will be in a
> chroot. In the OSTree model, you running system is immutable - so to
> "upgrade" a package, we make a hardlink farm of the current tree, and
> re-unpack the new version of of a package on top, then set it up for
> the next boot.
>
> If you look at software like mock, it does a complex dance to set up
> things like /proc in the target root and then run yum --installroot.
> But if we were in control over all subprocesses launched, we could use
> systemd to put each scriptlet inside a container, with its own private
> /proc and such, inside its own mount namespace. There are a huge
> number of advantages to this - systemd is very good at isolation,
> logging, etc. For example, we could cut off scriptlets from access to
> physical devices using
> https://fedoraproject.org/wiki/Changes/PrivateDevicesAndPrivateNetwork
>
> So, a few ways to accomplish this:
>
> 1) Add rpmtsSetScriptCallback() - This would allow me to set a
>callback which would be used instead of the fork()/exec() bits in
>rpmScriptRun. I could analyze the script content, and exec it
>myself using something like "systemd-run".
>
> 2) Have rpm-ostree extract the scripts manually *before* the
>transaction, analyze them, unpack all of the rpms, then run all of
>the scripts as if they were %posttrans anyways
>
> I need to do a bit more evaluation, but the more I think about it the
> more I like #2 as it maximizes control. If we go that route though
> there's a detail - Lua-based scripts. In order to implement these it's
> a bit harder as I'd need to exec a new RPM process inside the root and
> run the script inside that process. Maybe I could get away with just
> ignoring Lua scripts, I suspect many of them are there to work around
> things like "can't replace directory with symlink" problem that OSTree
> itself fixes.
>
> Thoughts?
> _
> Rpm-maint mailing list Rpm-maint@lists.rpm.org
> http://lists.rpm.org/mailman/listinfo/rpm-maint

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Add API to completely disable librpm's use of Unix signal handlers

[Colin Walters]

Numerous consumers of librpm use it in a pattern where they're
constructing fresh chroots.  For example, rpm-ostree operates this
way, and is used to provide atomic upgrades in concert with rpm.

If the process dies due to SIGINT or another signal, the root can
simply be discarded.

Currently today, rpm-ostree undoes the signal handlers after loading
librpm so that Control-C does what I want, but there's still a race
condition where the interrupt can be lost.

Add an API so callers can disable the behavior.
---
 rpmio/rpmsq.c | 16 
 rpmio/rpmsq.h |  2 ++
 2 files changed, 18 insertions(+)

From cb6f9a97baa4022789b469881bbc87aa1fb86697 Mon Sep 17 00:00:00 2001
From: Colin Walters 
Date: Thu, 22 Jan 2015 17:57:14 -0500
Subject: [PATCH] Add API to completely disable librpm's use of Unix signal
 handlers

Numerous consumers of librpm use it in a pattern where they're
constructing fresh chroots.  For example, rpm-ostree operates this
way, and is used to provide atomic upgrades in concert with rpm.

If the process dies due to SIGINT or another signal, the root can
simply be discarded.

Currently today, rpm-ostree undoes the signal handlers after loading
librpm so that Control-C does what I want, but there's still a race
condition where the interrupt can be lost.

Add an API so callers can disable the behavior.
---
 rpmio/rpmsq.c | 16 
 rpmio/rpmsq.h |  2 ++
 2 files changed, 18 insertions(+)

diff --git a/rpmio/rpmsq.c b/rpmio/rpmsq.c
index bfd4db0..533ac14 100644
--- a/rpmio/rpmsq.c
+++ b/rpmio/rpmsq.c
@@ -16,6 +16,7 @@
 
 #include "debug.h"
 
+static int disableInterruptSafety;
 static sigset_t rpmsqCaught;
 
 typedef struct rpmsig_s * rpmsig;
@@ -70,6 +71,9 @@ int rpmsqEnable(int signum, rpmsqAction_t handler)
 rpmsig tbl;
 int ret = -1;
 
+if (disableInterruptSafety)
+  return 0;
+
 for (tbl = rpmsigTbl; tbl->signum >= 0; tbl++) {
 	if (tblsignum != tbl->signum)
 	continue;
@@ -112,3 +116,15 @@ int rpmsqEnable(int signum, rpmsqAction_t handler)
 return ret;
 }
 
+
+/** \ingroup rpmio
+ * 
+ * Completely disable all Unix signal handling inside librpm.  This
+ * function is useful to call for build tools and the like which are
+ * operating on chroots, and if interrupted, simply delete them and
+ * start over.
+ */
+void rpmsqDisableInterruptSafety(void)
+{
+  disableInterruptSafety = 1;
+}
diff --git a/rpmio/rpmsq.h b/rpmio/rpmsq.h
index 00a1a72..d38d70a 100644
--- a/rpmio/rpmsq.h
+++ b/rpmio/rpmsq.h
@@ -52,6 +52,8 @@ void rpmsqAction(int signum);
  */
 int rpmsqEnable(int signum, rpmsqAction_t handler);
 
+void rpmsqDisableInterruptSafety(void);
+
 #ifdef __cplusplus
 }
 #endif
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] fsync and package systems

[Colin Walters]

Hi,

See: https://bugs.freedesktop.org/show_bug.cgi?id=70366

TL;DR: Should triggers like ldconfig, gtk-update-icon-cache, and 
update-mime-database:


1) Not fsync, and honor an environment variable like 
PKGSYSTEM_ENABLE_FSYNC

2) Do fsync, and honor  PKGSYSTEM_DISABLE_FSYNC
3) Something else?




___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] whitelisting/wrapping script execution

[Colin Walters]

Hi,

For the rpm-ostree project, I need the ability to whitelist scriptlets. 
There are a few reasons for this, but the most critical is that due to 
the way OSTree uses hardlinks, and because I can't assume I have 
copy-on-write links, I need to know that scriptlets will cleanly break 
hardlinks instead of edit-in-place.


For example, suppose I'm layering a package on top of a base tree that 
contains a shared library.  We need to update /etc/ld.so.cache.  In the 
traditional RPM model, that package will contain a scriptlet to run 
/sbin/ldconfig.


I happen to know that /sbin/ldconfig is safe - it creates 
/etc/ld.so.cache~ and rename()s into place, so I can just run it.  But 
if the package happens to do something I haven't whitelisted as safe, I 
need to error out and refuse to install it.


(Note an important side benefit here of moving away from the "whatever 
arbitrary code you want that runs as root" aspect of scripts)


Now related to this, I'd like to have rpm-ostree take care of executing 
any subprocesses (and even Lua code potentially).  The reason for this 
is because *every* rpm-ostree install will be in a chroot.  In the 
OSTree model, you running system is immutable - so to "upgrade" a 
package, we make a hardlink farm of the current tree, and re-unpack the 
new version of of a package on top, then set it up for the next boot.


If you look at software like mock, it does a complex dance to set up 
things like /proc in the target root and then run yum --installroot.  
But if we were in control over all subprocesses launched, we could use 
systemd to put each scriptlet inside a container, with its own private 
/proc and such, inside its own mount namespace.  There are a huge 
number of advantages to this - systemd is very good at isolation, 
logging, etc.  For example, we could cut off scriptlets from access to 
physical devices using 
https://fedoraproject.org/wiki/Changes/PrivateDevicesAndPrivateNetwork


So, a few ways to accomplish this:

1) Add rpmtsSetScriptCallback() - This would allow me to set a callback 
which would be used instead of the fork()/exec() bits in rpmScriptRun.  
I could analyze the script content, and exec it myself using something 
like "systemd-run".


2) Have rpm-ostree extract the scripts manually *before* the 
transaction, analyze them, unpack all of the rpms, then run all of the 
scripts as if they were %posttrans anyways


I need to do a bit more evaluation, but the more I think about it the 
more I like #2 as it maximizes control.  If we go that route though 
there's a detail - Lua-based scripts.  In order to implement these it's 
a bit harder as I'd need to exec a new RPM process inside the root and 
run the script inside that process.  Maybe I could get away with just 
ignoring Lua scripts, I suspect many of them are there to work around 
things like "can't replace directory with symlink" problem that OSTree 
itself fixes.


Thoughts?
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] installroot and NSS modules

[Colin Walters]

On Tue, Jan 28, 2014 at 4:16 AM, Panu Matilainen 
 wrote:


Rpm 4.6 - 4.11.0 are buggy on non-trivial nsswitch setups due to my 
less-than-brilliant idea (wish I knew what I was thinking at the 
time...) See 
http://rpm.org/gitweb?p=rpm.git;a=commitdiff;h=abbf4897db217b4977b4c21eb09929c797ee015d. 

So now we consistently use the nss configuration from the "host" side?  
If so I think that makes sense.  What I ended up doing is just 
installing nss-altfiles on the host side - it's harmless there if 
/usr/lib/passwd doesn't exist.


I think this OK, but long term, I'd like to move towards something more 
declarative, like:


AddSystemUser: polkit

That could then be processed by the host system however it wants, 
rather than
relying on running /usr/sbin/useradd inside the chroot which then the 
host system

picks up.


___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint