** Changed in: snapd
Status: New => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage notifications about
The latest lxd rootfs have been tested and daily images have been
released.
** Changed in: cloud-images
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
** Changed in: ubuntu-z-systems
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
This bug was fixed in the package snapd - 2.53+21.10ubuntu1
---
snapd (2.53+21.10ubuntu1) impish; urgency=medium
* Cherry pick https://github.com/snapcore/snapd/pull/10898 to
fix apparmor profile for snap-confine on s390x,ppc64el,risvc64
This fixes the snapd.seeded.service
I've tested the proposed upload on a pple64 and s390x systems. I used
the following procedure
1. lxc launch ubuntu-daily:impish test-i-update
2. lxc shell test-i-update
3. enabled proposed in /etc/apt/sources.list
4. apt update
5. apt install snapd
Get:1 http://ports.ubuntu.com/ubuntu-ports
** Changed in: snapd (Ubuntu)
Status: In Progress => Fix Committed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
** Changed in: ubuntu-z-systems
Status: New => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
** Also affects: snapd
Importance: Undecided
Status: New
** Also affects: ubuntu-z-systems
Importance: Undecided
Status: New
** Tags added: openstack-ibm s390x uosci
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to
I distro patched the impish version of snapd with the fix from Ian and uploaded
a new version:
https://launchpad.net/ubuntu/impish/+queue?queue_state=1_text=snapd
with the fix as 2.53+21.10ubuntu1.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
Okay, so after too much time trying to test this, I confirmed that this
change will fix it for both powerpc and for s390x:
https://github.com/snapcore/snapd/pull/10898
That PR should be merged to snapd master, then cherry-picked to
release/2.53 branch, then we can upload a 2.53.1 deb to
ls -lah /lib64/ld64.so.2 on ppcle64 impish, hirsute
On Impish
ls -lah /lib64/ld64.so.2
lrwxrwxrwx 1 root root 36 Sep 2 21:26 /lib64/ld64.so.2 ->
/lib/powerpc64le-linux-gnu/ld64.so.2
Hirsute:
ls -lah /lib64/ld64.so.2
lrwxrwxrwx 1 root root 37 Mar 31 2021 /lib64/ld64.so.2 ->
ppcle64 hirsute lxd journalctl
** Attachment added: "ppc_hirsute_lxd_journalctl.txt"
https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1944004/+attachment/5531325/+files/ppc_hirsute_lxd_journalctl.txt
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
ppcle64 outputs:
lxc ubuntu-daily:impish
ls -lah /usr/lib/ld64.so.1
ls: cannot access '/usr/lib/ld64.so.1': No such file or directory
ldd /usr/lib/snapd/snap-confine
linux-vdso64.so.1 (0x7fd86005)
libudev.so.1 => /lib/powerpc64le-linux-gnu/libudev.so.1 (0x7fd85ff7)
ppcle64 impish lxd journalctl
** Attachment added: "ppc_impish_lxd_journalctl.txt"
https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1944004/+attachment/5531324/+files/ppc_impish_lxd_journalctl.txt
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
There don't appear to be any AppArmor denials in the logs provided. From
snap debug seeding output, looks like the command `udevadm trigger
--subsystem-nomatch=input` failed when invoked during installation of
the snapd, which was done outside of confinement.
--
You received this bug
lxc container snap changes
root@test-i:~# snap changes
ID Status Spawn Ready Summary
1Error yesterday at 19:26 UTC today at 13:12 UTC Initialize system state
2Donetoday at 13:12 UTC today at 13:12 UTC Initialize device
3Error today at
debugging info from a s390x VM
test:
1. lxd init (all defaults)
2. lxc launch ubuntu-daily:impish test-i
3. lxc shell test-i
4. cat /etc/cloud/build.info
build_name: server
serial: 20211006
confirms latest
5. snap debug changes (See attachment)
I'm seeing a ton of "permission denied" all
images from 20211006 have been posted. These can be pulled with
lxc launch ubuntu-daily:impish
The build date can be verified at
/etc/cloud/build.info
On our nightly test of lxd with ubuntu-daily:impish I'm still seeing
powerpcel64 and s390x fail. I'll attempt to get VMs up for these for
a new lxd rootfs and lxd metadata tar have been published to cloud-
images.ubuntu.com/impish/20211004/ I confirmed with a local check of
lxc launch ubuntu-daily:impish
that lxd is picking up the daily produced on 20211004 from streams.
Anyone affected, if you could test the latest impish daily
TY anonymouse67, that was my assumption as well, and acceptable from my
POV for the image. I do think it's worth putting in our release notes,
so I shall :)
I'm still working on getting something out. We're getting held up by
riscv64 builders just not wanting to play nicely. and those builds take
John, the difference between those two that I can see is that now the
image was creating with cgroupsv2 present when preseeding (which was the
expected change), but the runtime image does not use cgroupsv2 and thus
things need to be regenerated.
This is sort of expected since the runtime kernel
latest daily was built yesterday. Testing, however, is inconclusive.
Even though I tested livecd-rootfs in the Launchpad environment, it
seems like something still isn't jiving, as the system restart key is
still present. Test case below:
1. downloaded arm64 daily lxc rootfs from Launchpad
2.
** Changed in: cloud-images
Status: Confirmed => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
This is now released but for some reason the bug task didn't get
autoclosed. https://launchpad.net/ubuntu/+source/livecd-rootfs/2.741 is
in impish release:
livecd-rootfs (2.741) impish; urgency=medium
[ John Chittum ]
* mount cgroup2 type for snapd LP:#1944004
-- Steve Langasek Wed, 29
this is not released, the package is waiting in the -unapproved queue
and has definitely not reached the release pocket.
** Changed in: livecd-rootfs (Ubuntu)
Status: Fix Released => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is
** Changed in: livecd-rootfs (Ubuntu)
Status: New => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
** Changed in: livecd-rootfs (Ubuntu)
Assignee: (unassigned) => John Chittum (jchittum)
** Changed in: cloud-images
Assignee: (unassigned) => John Chittum (jchittum)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
I've added `livecd-rootfs` as affected after discussing the issue with
the snapd team. I'll be introducing a change to `livecd-rootfs` to
address how snapd expects cgroup2 with a mount at /sys/fs/cgroup
** Also affects: livecd-rootfs (Ubuntu)
Importance: Undecided
Status: New
--
You
Full output from `snap debug seeding`
{'image-preseeding': '7.949s',
'preseed-system-key': {'apparmor-features': ['caps', 'dbus', 'domain', 'file',
'mount', 'namespaces', 'network',
'network_v8',
after the upload of `snapd (2.53~pre1.git19b68f708)`, I see that snap
pre-seed optimizations aren't completing properly. the `seed-restart-
system-key` is now a call seen in `snap debug seeding`
'seed-restart-system-key': {'apparmor-features': ['caps', 'dbus',
'domain', 'file', 'mount',
i've tested manually using the following:
1. created an arm64 server instance on AWS
2. uploaded the most recent rootfs from CPC's internal build system
3. created lxd metadata tool using CPC's metadata script (sorry, this and above
aren't public)
4. uploaded lxd metadata tar.xz
5. lxd init on
@sergiodj : Build happened early this AM. I see in the image manifest
snapd 2.53~pre1.git19b68f708
I'll do manual verification this morning, and push an image if it passes
manual testing.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to
This is great, thanks for the fix.
@jchittum, do you think you guys at CPC could respin the Ubuntu Impish
images with this new snapd? Thanks!
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
This bug was fixed in the package snapd - 2.53~pre1.git19b68f708
---
snapd (2.53~pre1.git19b68f708) impish; urgency=medium
* New git snapshot with fix for snapd.seeded.service in lxd
(LP: #1944004)
* New git snapshot that supports cgroups v2 (LP: #1850667)
-- Michael Vogt
** Changed in: docker.io (Ubuntu)
Status: New => Invalid
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
** Changed in: cloud-images
Status: New => Confirmed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage notifications
** Tags added: rls-ii-incoming
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage notifications about this bug go to:
This is now uploaded to impish as
snapd_2.53~pre1.git19b68f708_source.changes
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To
** Changed in: snapd (Ubuntu)
Status: In Progress => Fix Committed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
The PR is up: https://github.com/snapcore/snapd/pull/10839 also needs +1
from security
** Changed in: snapd (Ubuntu)
Status: New => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
Thanks to @mwhudson for arranging access to the test host. Per his
comments I launched both 20210903 and 20210904 snapshots of impish.
Debugging, I noticed that there was an apparmor denial logged when snap
(the snap binary from snapd) was transitioning to snap-confine. While
snap-confine runs
SNAP_CONFINE_DEBUG=1 doesn't seem to make any difference. also disabling
clone3 doesn't seem to help :(
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never
Adding SNAP_CONFINE_DEBUG=1 in the env will produce more debugging
output.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
A few more observations:
1) The output shown for the hook is just snapd debugging goo. It's not
meaningful.
2) It is libc6 :(
You can demonstrate 2) by: starting a container with a known good image,
removing lxd, upgrading libc6 & co and then running "snap install
@mwhudson I'm still not convinced that the udev noise is not the
problem, but another data point I just went ahead and got to prove this
is that when trying to seed another snap which uses interfaces and has
an install hook and is available on arm64, the docker snap, it fails the
"same" way with
So one thing that I think is going on here is that the udev noise is
caused by trying to _undo_ the "Setup snap "snapd" (12886) security
profiles" task. In the state.json of the image, that task is done:
"7": {
"id": "7",
"kind": "setup-profiles",
"summary": "Setup snap
So some thoughts:
1. The root failure of snapd failing to finish seeding is that it is
trying to run `udevadm trigger --subsystem-nomatch=input` which ends up
dying on every write like so:
openat(AT_FDCWD, "/sys/devices/virtual/workqueue/writeback/uevent",
We have reproducers on arm64 and s390x now. I'm seeing the same things
as the snapd.log in
https://bugs.launchpad.net/cloud-images/+bug/1944004/comments/1
any attempts to restart the service do the same thing. So something has
set up snapd with incorrect permissions. At this point, I'm going to
available at: http://cloud-
images.ubuntu.com/daily/server/impish/20210904/impish-server-cloudimg-
arm64-root.manifest
** Attachment added: "20210904-impish-server-cloudimg-arm64-root.manifest"
Apologies, i had some file confusion locally in creating the diff of
packages, which led to confusion. Attached is the proper diff. I'll also
attached the manifest used to generate this diff
From the list of package changes here, I'm not seeing a smoking gun.
`cloud-init` upreved, as well as a
available at: http://cloud-
images.ubuntu.com/daily/server/impish/20210903/impish-server-cloudimg-
arm64-root.manifest
** Attachment added: "20210903-impish-server-cloudimg-arm64-root.manifest"
** Attachment removed: "diff-2021093-to-20210904.txt"
https://bugs.launchpad.net/cloud-images/+bug/1944004/+attachment/5527398/+files/diff-2021093-to-20210904.txt
** Attachment added: "20210903-to-20210904-rootfs-diff-unified.txt"
livecd-rootfs, used for building this, had a release on 20210830 and on
20210914. This feels like it may be a dead end, as I should have been
able to reproduce the error on the 20210903 image built with the
20210830 release of livecd-rootfs if there was an issue in the base
build scripts.
for cpc
Manifest Diff from 20210903 to 20210904 is now attached
20210903 rootfs is working fine. I did the following:
1. started an ARM image on Oracle
2. downloaded
http://cloud-images.ubuntu.com/impish/20210903/impish-server-cloudimg-arm64-root.tar.xz
and
ACK. I've started looking into it. initial things I can see:
1. there have been no code changes to how the image is being made. code
is
https://git.launchpad.net/livecd-rootfs/tree/live-build/ubuntu-
cpc/hooks.d/base/create-root-dir.binary
and
Hi CPC folks,
It looks like there's something wrong with the non-amd64 LXD images for
Ubuntu Impish. stgraber kindly helped me diagnose the issue and noticed
that the problem is likely in the images themselves. Please refer to
this issue:
https://github.com/lxc/lxd/issues/9263
or, more
** Also affects: cloud-images
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
To manage
Correct, this happens with normal containers. FWIW, I can provide
access to the s390x machine I'm using (from canonistack) if needed.
Thanks.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
To be clear you see this behavior with normal containers and not with
nested containers? Snapd doesn't support nested containers unfortunately
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
A few details/clarifications:
1) I can confirm that stopping the snapd.{service,socket} unblocks the
other services and allows the boot to finish (cloud-init finally reports
"done").
2) The problem happens when one launches an lxd container from a non-
amd64 architecture. For example, I have an
Adding a docker.io to reflect that this bug is blocking the package on
impish-proposed.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1944004
Title:
snapd.seeded.service never finishes on non-amd64
61 matches
Mail list logo
