without SystemLock being held works fine:
root@xx:~# strace -o $PWD/log -e openat,open,close,fcntl python3 $PWD/foo.py
(Reading database ... 25835 files and directories currently installed.)
Removing aptitude-common (0.7.4-2ubuntu2) ...
COMMAND PID TYPE SIZE MODE M START END PATH
iscsid 432 POSIX 4B WRITE 0 0 0 /run/iscsid.pid
atd 421 POSIX 4B WRITE 0 0 0 /run/atd.pid
cron 418 FLOCK 4B WRITE 0 0 0 /run/crond.pid
python3 1207 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock-frontend
python3 1207 POSIX 0B WRITE 0 0 0 /var/cache/apt/archives/lock
dpkg 1215 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock
Processing triggers for man-db (2.7.5-1) ...
with SystemLock held works fine:
# strace -o $PWD/log.locked -e openat,open,close,fcntl python3 $PWD/foo.py
(Reading database ... 25835 files and directories currently installed.)
Removing aptitude-common (0.7.4-2ubuntu2) ...
COMMAND PID TYPE SIZE MODE M START END PATH
iscsid 432 POSIX 4B WRITE 0 0 0 /run/iscsid.pid
atd 421 POSIX 4B WRITE 0 0 0 /run/atd.pid
cron 418 FLOCK 4B WRITE 0 0 0 /run/crond.pid
dpkg 2285 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock
python3 2277 POSIX 0B WRITE 0 0 0 /var/lib/dpkg/lock-frontend
python3 2277 POSIX 0B WRITE 0 0 0 /var/cache/apt/archives/lock
Processing triggers for man-db (2.7.5-1) ...
In the strace log, we can see that the frontend lock is closed after the
dpkg runs (after we have relocked the lock).
** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to python-apt in Ubuntu.
https://bugs.launchpad.net/bugs/1795407
Title:
python-apt frontend locking
Status in python-apt package in Ubuntu:
Fix Released
Status in python-apt source package in Xenial:
Fix Committed
Status in python-apt source package in Bionic:
Fix Released
Bug description:
[Impact]
Add support to python-apt for frontend locking. This is a bit more
complicated, and also requires some other restructuring:
(1) The archives lock was only taken for a short time, when it should
have been kept for the duration of an installation, as otherwise debs
could disappear from archives/ while the install is running.
(2) There was no lock handling at all. Tools essentially acquire the
lock, and then release it before commit().
The fix is based on the apt fix, which makes apt_pkg.SystemLock()
manage both the frontend and the normal lock, and allows code to call
apt_pkg.pkgsystem_unlock_inner() and apt_pkg.pkgsystem_lock_inner()
around dpkg invocations to release the inner lock.
Cache.commit() takes care of locking itself now. It releases the inner
lock as needed for dpkg invocations. It also now requires
apt_pkg.SystemLock to be hold - if it is not, it will acquire it
itself.
[Test case]
1. Mark a package in the cache for install/removal and commit() - FE lock
should be taken while dpkg is running.
2. Mark a package in the cache for install/removal and commit() while holding
the lock. Releasing the lock afterwards should still work correctly, and the FE
lock should never be released
3. Observe that the archive lock is not released until the last dpkg run
The locking behavior can be observed via strace.
[Regression potential]
The dpkg lock changed the most: Some lock counting might go wrong and
programs might fail as a result. Any problems will be isolated, though - dpkg
still acquires its locks, and if a lock miscount happens, apt would not tell
dpkg to skip acquiring the frontend lock, so even if we were losing the lock
anywhere, dpkg would still try to acquire it and not run unlocked.
The archive lock done by commit()/fetch_archives() in apt.Cache() is
now hold as long as the fetcher object exists though (as it uses the
fetcher's get_lock method), which might cause unexpected behavior in
apps not expecting that.
The lists/ lock is unaffected by the changes, so there should not be
any regressions when it comes to updating index files.
[Other info (1)]
CI changes: There were some improvements to type hints for mypy needed to
make type checks pass, and changes to the CI's Dockerfile to pull a new apt
from the apt stable PPA (as a new enough apt is only in unapproved atm).
Neither of those files are used anywhere outside of the package, so they should
not cause any problems.
[Other info (2)]
This is part of a wider series of SRUs for frontend locking
- dpkg (bug 1796081)
- apt (bug 1781169)
- python-apt (bug 1795407)
- packagekit (bug 1795614)
- unattended-upgrades (bug 1789637)
- aptdaemon (no bug filed yet)
Further details about frontend locking can be found in
https://lists.debian.org/debian-dpkg/2017/01/msg00044.html
- This SRU depends on the apt SRU above, and breaks unattended-
upgrades in bionic without its SRU due to a bug in the u-u code in
handling the new API.
- xenial depends on unattended-upgrades SRU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/1795407/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
