Hello Zachary, or anyone else affected, Accepted openafs into resolute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openafs/1.8.14-1ubuntu2.1 in a few hours, and then in the -proposed repository.
Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- resolute to verification-done-resolute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-resolute. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Description changed: [ Impact ] Linux 6.17+ changed the way that the d_entry operations are set for a super block in 05fb0e666495c. - In particular, this means openafs 1.18.14 silently fails to register its own d_entry operations, leading + In particular, this means openafs 1.8.14 silently fails to register its own d_entry operations, leading to a stale d_entry cache in many situations. This only impacts resolute as stonking and tenacious already package - openafs 1.18.16 which contains the upstream fix and noble and earlier + openafs 1.8.16 which contains the upstream fix and noble and earlier kernels predate the d_entry operations change. [ Test Plan ] The cache revalidation issues occur when changes are made on a remote client, so the test setup needs two machines: an openafs server which will make all the modifications, and an openafs client which will only observe the changes. In order to accurately reproduce the issues it is important to only run commands on the client when the testplan says as any read to the network filesystem can populate the cache. Launch two resolute VMs: ```bash lxc launch ubuntu:resolute afs-server --vm -c limits.memory=4GiB -c limits.cpu=4 \ -c boot.mode=uefi-nosecureboot lxc launch ubuntu:resolute afs-client --vm -c limits.memory=2GiB -c limits.cpu=4 \ -c boot.mode=uefi-nosecureboot ``` Secure Boot must be off: the openafs module is built by DKMS and signed with a key that is not enrolled. NOTE: Run every command below as root. On the server, configure the cell and install the server packages. This sets up the server without needing a KDC and creates a cell at /afs/localcell: ```bash debconf-set-selections <<EOF openafs-client openafs-client/thiscell string localcell openafs-client openafs-client/cachesize string 100000 openafs-client openafs-client/dynroot select Yes openafs-client openafs-client/fakestat boolean true openafs-fileserver openafs-fileserver/thiscell string localcell EOF apt update apt install linux-headers-$(uname -r) openafs-modules-dkms \ openafs-client openafs-fileserver openafs-dbserver openafs-krb5 krb5-user ``` Create the random cell key and the realm to match: ```bash printf '[libdefaults]\n default_realm = LOCALCELL\n dns_lookup_realm = false\n dns_lookup_kdc = false\n' > /etc/krb5.conf printf 'addent -password -p afs/localcell -k 1 -e aes256-cts\n%s\nwkt /etc/openafs/server/rxkad.keytab\nquit\n' \ "$(head -c 24 /dev/urandom | base64)" | ktutil akeyconvert ``` Point the server at its own cell: ```bash for db in /etc/openafs/CellServDB /etc/openafs/server/CellServDB; do printf '>localcell\n%s # afs-server\n' "$(hostname -I | awk '{print $1}')" > $db done ``` Give the file server a partition and start the servers: ```bash mkdir -p /vicepa && touch /vicepa/AlwaysAttach systemctl start openafs-fileserver until bos status localhost -localauth; do sleep 2; done bos create localhost ptserver simple /usr/lib/openafs/ptserver -localauth bos create localhost vlserver simple /usr/lib/openafs/vlserver -localauth bos create localhost dafs dafs -cmd /usr/lib/openafs/dafileserver -cmd /usr/lib/openafs/davolserver \ -cmd /usr/lib/openafs/salvageserver -cmd /usr/lib/openafs/dasalvager -localauth until vos partinfo localhost a -localauth; do sleep 5; done # about 30 s ``` Create the volume and the admin user: ```bash vos create localhost a root.cell -localauth pts createuser admin -localauth pts adduser admin system:administrators -localauth ``` Restart the client and get an admin token: ```bash systemctl restart openafs-client aklog -c localcell -keytab /etc/openafs/server/rxkad.keytab -principal admin ``` Let any user read the volume and create the test directory: ```bash fs setacl -dir /afs/localcell -acl system:anyuser rl mkdir /afs/localcell/sru ``` NOTE: Run all later afs-server commands in this root shell, so that they use the admin token. If the token expires, run the aklog command again. On the client, configure the cell and install the client packages: ```bash echo "openafs-client openafs-client/thiscell string localcell" | debconf-set-selections apt update apt install linux-headers-$(uname -r) openafs-modules-dkms openafs-client dkms status openafs ``` Point the client at afs-server and restart: ```bash echo ">localcell" > /etc/openafs/CellServDB echo "$(getent ahostsv4 afs-server.lxd | awk 'NR==1{print $1}') # afs-server" >> /etc/openafs/CellServDB echo localcell > /etc/openafs/ThisCell systemctl restart openafs-client ``` With both VMs set up we can demonstrate the stale dentry cache in a few ways. On the server, create a file: ```bash echo hello > /afs/localcell/sru/repro ``` On the client, read it once to cache it: ```bash cat /afs/localcell/sru/repro ``` Expected: `hello`. The patch should not change this. On the server, delete and recreate the file: ```bash rm -f /afs/localcell/sru/repro && echo newdata-1 > /afs/localcell/sru/repro ``` On the client, read it once: ```bash cat /afs/localcell/sru/repro ``` Expected Output: ```text newdata-1 ``` Unpatched output: ```text hello ``` On the client, flush the AFS cache only for that name, then read it: ```bash fs flush -path /afs/localcell/sru/repro cat /afs/localcell/sru/repro ``` Expected Output: ```text newdata-1 ``` Unpatched output: ```text cat: /afs/localcell/sru/repro: Input/output error ``` On the client, flush both caches: ```bash fs flush -path /afs/localcell/sru/repro >/dev/null 2>&1 fs flush -path /afs/localcell/sru sync echo 2 > /proc/sys/vm/drop_caches cat /afs/localcell/sru/repro ``` Expected, with and without the patch: ```text newdata-1 ``` On the server, delete the file: ```bash rm -f /afs/localcell/sru/repro ``` On the client, read it once: ```bash cat /afs/localcell/sru/repro ``` Expected Output: ```text cat: /afs/localcell/sru/repro: No such file or directory ``` Unpatched output: ```text newdata-1 ``` On the client, flush both caches: ```bash fs flush -path /afs/localcell/sru/repro >/dev/null 2>&1 fs flush -path /afs/localcell/sru sync echo 2 > /proc/sys/vm/drop_caches cat /afs/localcell/sru/repro ``` Expected, with and without the patch: ```text cat: /afs/localcell/sru/repro: No such file or directory ``` On the server, recreate the file: ```bash echo hello > /afs/localcell/sru/repro ``` On the client, read it once: ```bash cat /afs/localcell/sru/repro ``` Expected Output: ```text hello ``` Unpatched output: ```text cat: /afs/localcell/sru/repro: No such file or directory ``` To run the test again, use a new directory instead of /afs/localcell/sru, so that no cached entries from an earlier run can change the result. [ Where problems could occur ] * This will trigger a rebuild of the openafs DKMS module. Which could put an unexepected load on systems using openafs when they upgrade. * We're changing the dentry operations so, we could introduce other staleness issues, false cache misses. In principle, the patch should simply restore the pre 6.17 behavior, but it's possible that there could be some hidden kernel regression behind the move from s_d_op to set_default_d_op. Nonetheless, this would be a transition from a currently broken state to a different broken state. * I also rebased one existing patch (see below). The only non-cosmetic change was removing the `.parameters = NULL` from the initialization of the afs_fs_type struct. If anything in the kernel explicitly relies on the parameter being set to NULL this could potentially cause regressions. [ Other Info ] * A patch for the issue landed upstream in 1.8.16 <https://gerrit.openafs.org/16700> * The existing patch 0016-6.18-mount_nodev-removed-use-new-mount- API.patch was taken from an older version of a gerrit changeset than was committed upstream. I updated the patch to the version that actually landed in order to reduce backporting noises in this SRU and any subsequent cherry-picks we have to do. The difference was between patchset 2 and patchset 9 <https://gerrit.openafs.org/#/c/16646/2..9/src/afs/LINUX/osi_vfsops.c> ** Also affects: openafs (Ubuntu Noble) Importance: Undecided Status: New ** Also affects: openafs (Ubuntu Jammy) Importance: Undecided Status: New ** Changed in: openafs (Ubuntu Resolute) Status: In Progress => Fix Committed ** Tags added: verification-needed-resolute -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2173580 Title: [SRU] dentry cache becomes stale on kernel >= 6.17 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openafs/+bug/2173580/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
