Hello Zachary, or anyone else affected,

Accepted openafs into resolute-proposed. The package will build now and
be available at
https://launchpad.net/ubuntu/+source/openafs/1.8.14-1ubuntu2.1 in a few
hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
resolute to verification-done-resolute. If it does not fix the bug for
you, please add a comment stating that, and change the tag to
verification-failed-resolute. In either case, without details of your
testing we will not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Description changed:

  [ Impact ]
  
  Linux 6.17+ changed the way that the d_entry operations are set for a super 
block in 05fb0e666495c.
- In particular, this means openafs 1.18.14 silently fails to register its own 
d_entry operations, leading
+ In particular, this means openafs 1.8.14 silently fails to register its own 
d_entry operations, leading
  to a stale d_entry cache in many situations.
  
  This only impacts resolute as stonking and tenacious already package
- openafs 1.18.16 which contains the upstream fix and noble and earlier
+ openafs 1.8.16 which contains the upstream fix and noble and earlier
  kernels predate the d_entry operations change.
  
  [ Test Plan ]
  
  The cache revalidation issues occur when changes are made on a remote
  client, so the test setup needs two machines: an openafs server which
  will make all the modifications, and an openafs client which will only
  observe the changes.
  
  In order to accurately reproduce the issues it is important to only run
  commands on the client when the testplan says as any read to the network
  filesystem can populate the cache.
  
  Launch two resolute VMs:
  
  ```bash
  lxc launch ubuntu:resolute afs-server --vm -c limits.memory=4GiB -c 
limits.cpu=4 \
      -c boot.mode=uefi-nosecureboot
  lxc launch ubuntu:resolute afs-client --vm -c limits.memory=2GiB -c 
limits.cpu=4 \
      -c boot.mode=uefi-nosecureboot
  ```
  
  Secure Boot must be off: the openafs module is built by DKMS and signed
  with a key that is not enrolled.
  
  NOTE: Run every command below as root.
  
  On the server, configure the cell and install the server packages. This
  sets up the server without needing a KDC and creates a cell at
  /afs/localcell:
  
  ```bash
  debconf-set-selections <<EOF
  openafs-client openafs-client/thiscell string localcell
  openafs-client openafs-client/cachesize string 100000
  openafs-client openafs-client/dynroot select Yes
  openafs-client openafs-client/fakestat boolean true
  openafs-fileserver openafs-fileserver/thiscell string localcell
  EOF
  apt update
  apt install linux-headers-$(uname -r) openafs-modules-dkms \
      openafs-client openafs-fileserver openafs-dbserver openafs-krb5 krb5-user
  ```
  
  Create the random cell key and the realm to match:
  
  ```bash
  printf '[libdefaults]\n default_realm = LOCALCELL\n dns_lookup_realm = 
false\n dns_lookup_kdc = false\n' > /etc/krb5.conf
  printf 'addent -password -p afs/localcell -k 1 -e aes256-cts\n%s\nwkt 
/etc/openafs/server/rxkad.keytab\nquit\n' \
      "$(head -c 24 /dev/urandom | base64)" | ktutil
  akeyconvert
  ```
  
  Point the server at its own cell:
  
  ```bash
  for db in /etc/openafs/CellServDB /etc/openafs/server/CellServDB; do
      printf '>localcell\n%s  # afs-server\n' "$(hostname -I | awk '{print 
$1}')" > $db
  done
  ```
  
  Give the file server a partition and start the servers:
  
  ```bash
  mkdir -p /vicepa && touch /vicepa/AlwaysAttach
  systemctl start openafs-fileserver
  until bos status localhost -localauth; do sleep 2; done
  bos create localhost ptserver simple /usr/lib/openafs/ptserver -localauth
  bos create localhost vlserver simple /usr/lib/openafs/vlserver -localauth
  bos create localhost dafs dafs -cmd /usr/lib/openafs/dafileserver -cmd 
/usr/lib/openafs/davolserver \
      -cmd /usr/lib/openafs/salvageserver -cmd /usr/lib/openafs/dasalvager 
-localauth
  until vos partinfo localhost a -localauth; do sleep 5; done   # about 30 s
  ```
  
  Create the volume and the admin user:
  
  ```bash
  vos create localhost a root.cell -localauth
  pts createuser admin -localauth
  pts adduser admin system:administrators -localauth
  ```
  
  Restart the client and get an admin token:
  
  ```bash
  systemctl restart openafs-client
  aklog -c localcell -keytab /etc/openafs/server/rxkad.keytab -principal admin
  ```
  
  Let any user read the volume and create the test directory:
  
  ```bash
  fs setacl -dir /afs/localcell -acl system:anyuser rl
  mkdir /afs/localcell/sru
  ```
  
  NOTE: Run all later afs-server commands in this root shell, so that they
  use the admin token. If the token expires, run the aklog command again.
  
  On the client, configure the cell and install the client packages:
  
  ```bash
  echo "openafs-client openafs-client/thiscell string localcell" | 
debconf-set-selections
  apt update
  apt install linux-headers-$(uname -r) openafs-modules-dkms openafs-client
  dkms status openafs
  ```
  
  Point the client at afs-server and restart:
  
  ```bash
  echo ">localcell" > /etc/openafs/CellServDB
  echo "$(getent ahostsv4 afs-server.lxd | awk 'NR==1{print $1}')  # 
afs-server" >> /etc/openafs/CellServDB
  echo localcell > /etc/openafs/ThisCell
  systemctl restart openafs-client
  ```
  
  With both VMs set up we can demonstrate the stale dentry cache in a few
  ways.
  
  On the server, create a file:
  
  ```bash
  echo hello > /afs/localcell/sru/repro
  ```
  
  On the client, read it once to cache it:
  
  ```bash
  cat /afs/localcell/sru/repro
  ```
  
  Expected: `hello`. The patch should not change this.
  
  On the server, delete and recreate the file:
  
  ```bash
  rm -f /afs/localcell/sru/repro && echo newdata-1 > /afs/localcell/sru/repro
  ```
  
  On the client, read it once:
  
  ```bash
  cat /afs/localcell/sru/repro
  ```
  
  Expected Output:
  
  ```text
  newdata-1
  ```
  
  Unpatched output:
  
  ```text
  hello
  ```
  
  On the client, flush the AFS cache only for that name, then read it:
  
  ```bash
  fs flush -path /afs/localcell/sru/repro
  cat /afs/localcell/sru/repro
  ```
  
  Expected Output:
  
  ```text
  newdata-1
  ```
  
  Unpatched output:
  
  ```text
  cat: /afs/localcell/sru/repro: Input/output error
  ```
  
  On the client, flush both caches:
  
  ```bash
  fs flush -path /afs/localcell/sru/repro >/dev/null 2>&1
  fs flush -path /afs/localcell/sru
  sync
  echo 2 > /proc/sys/vm/drop_caches
  cat /afs/localcell/sru/repro
  ```
  
  Expected, with and without the patch:
  
  ```text
  newdata-1
  ```
  
  On the server, delete the file:
  
  ```bash
  rm -f /afs/localcell/sru/repro
  ```
  
  On the client, read it once:
  
  ```bash
  cat /afs/localcell/sru/repro
  ```
  
  Expected Output:
  
  ```text
  cat: /afs/localcell/sru/repro: No such file or directory
  ```
  
  Unpatched output:
  
  ```text
  newdata-1
  ```
  
  On the client, flush both caches:
  
  ```bash
  fs flush -path /afs/localcell/sru/repro >/dev/null 2>&1
  fs flush -path /afs/localcell/sru
  sync
  echo 2 > /proc/sys/vm/drop_caches
  cat /afs/localcell/sru/repro
  ```
  
  Expected, with and without the patch:
  
  ```text
  cat: /afs/localcell/sru/repro: No such file or directory
  ```
  
  On the server, recreate the file:
  
  ```bash
  echo hello > /afs/localcell/sru/repro
  ```
  
  On the client, read it once:
  
  ```bash
  cat /afs/localcell/sru/repro
  ```
  
  Expected Output:
  
  ```text
  hello
  ```
  
  Unpatched output:
  
  ```text
  cat: /afs/localcell/sru/repro: No such file or directory
  ```
  
  To run the test again, use a new directory instead of
  /afs/localcell/sru, so that no cached entries from an earlier run can
  change the result.
  
  [ Where problems could occur ]
  
  * This will trigger a rebuild of the openafs DKMS module. Which could put an 
unexepected load on systems using openafs when they upgrade.
  * We're changing the dentry operations so, we could introduce other staleness 
issues, false cache misses. In principle, the patch should simply restore the 
pre 6.17 behavior, but it's possible that there could be some hidden kernel 
regression behind the move from s_d_op to set_default_d_op. Nonetheless, this 
would be a transition from a currently broken state to a different broken state.
  
  * I also rebased one existing patch (see below). The only non-cosmetic
  change was removing the `.parameters = NULL` from the initialization of
  the afs_fs_type struct. If anything in the kernel explicitly relies on
  the parameter being set to NULL this could potentially cause
  regressions.
  
  [ Other Info ]
  
   * A patch for the issue landed upstream in 1.8.16
  <https://gerrit.openafs.org/16700>
  
  * The existing patch 0016-6.18-mount_nodev-removed-use-new-mount-
  API.patch was taken from an older version of a gerrit changeset than was
  committed upstream. I updated the patch to the version that actually
  landed in order to reduce backporting noises in this SRU and any
  subsequent cherry-picks we have to do. The difference was between
  patchset 2 and patchset 9
  <https://gerrit.openafs.org/#/c/16646/2..9/src/afs/LINUX/osi_vfsops.c>

** Also affects: openafs (Ubuntu Noble)
   Importance: Undecided
       Status: New

** Also affects: openafs (Ubuntu Jammy)
   Importance: Undecided
       Status: New

** Changed in: openafs (Ubuntu Resolute)
       Status: In Progress => Fix Committed

** Tags added: verification-needed-resolute

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2173580

Title:
  [SRU] dentry cache becomes stale on kernel >= 6.17

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openafs/+bug/2173580/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to