[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
** Changed in: linux (Ubuntu Noble) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: Fix Released Status in linux source package in Mantic: Fix Released Status in linux source package in Noble: Fix Released Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing list discussion: https://lore.kernel.org/linux- cifs/6a65b2d1-7596-438a-8ade-2f7526b15...@rd10.de/T/#m22cd9b7289f87cd945978bd7995bcaf1beebfe67 To manage notifications about this bug go to:
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Hi R. Diez, The 6.5.0-27-generic kernel just got released, so it should make its way to an archive mirror near you in the next couple of hours, and should solve your SMB 1 issue. The 6.8 kernel should be good to go when you get it in the next couple of months. Let us know if you encounter any more SMB 1 issues in the future. Thanks, Matthew -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: Fix Committed Status in linux source package in Mantic: Fix Released Status in linux source package in Noble: Fix Committed Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
This bug is awaiting verification that the linux-raspi/6.5.0-1014.17 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-mantic-linux-raspi' to 'verification-done- mantic-linux-raspi'. If the problem still exists, change the tag 'verification-needed-mantic-linux-raspi' to 'verification-failed-mantic- linux-raspi'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: kernel-spammed-mantic-linux-raspi-v2 verification-needed-mantic-linux-raspi -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: Fix Committed Status in linux source package in Mantic: Fix Released Status in linux source package in Noble: Fix Committed Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
This bug was fixed in the package linux - 6.5.0-27.28 --- linux (6.5.0-27.28) mantic; urgency=medium * mantic/linux: 6.5.0-27.28 -proposed tracker (LP: #2055584) * Packaging resync (LP: #1786013) - [Packaging] drop ABI data - [Packaging] update annotations scripts - debian.master/dkms-versions -- update from kernel-versions (main/2024.03.04) * CVE-2024-26597 - net: qualcomm: rmnet: fix global oob in rmnet_policy * CVE-2024-26599 - pwm: Fix out-of-bounds access in of_pwm_single_xlate() * Drop ABI checks from kernel build (LP: #2055686) - [Packaging] Remove in-tree abi checks * Cranky update-dkms-versions rollout (LP: #2055685) - [Packaging] remove update-dkms-versions - Move debian/dkms-versions to debian.master/dkms-versions - [Packaging] Replace debian/dkms-versions with $(DEBIAN)/dkms-versions * linux: please move erofs.ko (CONFIG_EROFS for EROFS support) from linux- modules-extra to linux-modules (LP: #2054809) - UBUNTU [Packaging]: Include erofs in linux-modules instead of linux-modules- extra * performance: Scheduler: ratelimit updating of load_avg (LP: #2053251) - sched/fair: Ratelimit update to tg->load_avg * IB peer memory feature regressed in 6.5 (LP: #2055082) - SAUCE: RDMA/core: Introduce peer memory interface * linux-tools-common: man page of usbip[d] is misplaced (LP: #2054094) - [Packaging] rules: Put usbip manpages in the correct directory * CVE-2024-23851 - dm: limit the number of targets and parameter size area * CVE-2024-23850 - btrfs: do not ASSERT() if the newly created subvolume already got read * x86: performance: tsc: Extend watchdog check exemption to 4-Sockets platform (LP: #2054699) - x86/tsc: Extend watchdog check exemption to 4-Sockets platform * linux: please move dmi-sysfs.ko (CONFIG_DMI_SYSFS for SMBIOS support) from linux-modules-extra to linux-modules (LP: #2045561) - [Packaging] Move dmi-sysfs.ko into linux-modules * Fix AMD brightness issue on AUO panel (LP: #2054773) - drm/amdgpu: make damage clips support configurable * Mantic update: upstream stable patchset 2024-02-28 (LP: #2055199) - f2fs: explicitly null-terminate the xattr list - pinctrl: lochnagar: Don't build on MIPS - ALSA: hda - Fix speaker and headset mic pin config for CHUWI CoreBook XPro - mptcp: fix uninit-value in mptcp_incoming_options - wifi: cfg80211: lock wiphy mutex for rfkill poll - wifi: avoid offset calculation on NULL pointer - wifi: mac80211: handle 320 MHz in ieee80211_ht_cap_ie_to_sta_ht_cap - debugfs: fix automount d_fsdata usage - nvme-core: fix a memory leak in nvme_ns_info_from_identify() - drm/amd/display: update dcn315 lpddr pstate latency - drm/amdgpu: Fix cat debugfs amdgpu_regs_didt causes kernel null pointer - smb: client, common: fix fortify warnings - blk-mq: don't count completed flush data request as inflight in case of quiesce - nvme-core: check for too small lba shift - hwtracing: hisi_ptt: Handle the interrupt in hardirq context - hwtracing: hisi_ptt: Don't try to attach a task - ASoC: wm8974: Correct boost mixer inputs - arm64: dts: rockchip: fix rk356x pcie msg interrupt name - ASoC: Intel: Skylake: Fix mem leak in few functions - ASoC: nau8822: Fix incorrect type in assignment and cast to restricted __be16 - ASoC: Intel: Skylake: mem leak in skl register function - ASoC: cs43130: Fix the position of const qualifier - ASoC: cs43130: Fix incorrect frame delay configuration - ASoC: rt5650: add mutex to avoid the jack detection failure - ASoC: Intel: skl_hda_dsp_generic: Drop HDMI routes when HDMI is not available - nouveau/tu102: flush all pdbs on vmm flush - ASoC: amd: yc: Add DMI entry to support System76 Pangolin 13 - ASoC: hdac_hda: Conditionally register dais for HDMI and Analog - net/tg3: fix race condition in tg3_reset_task() - ASoC: da7219: Support low DC impedance headset - nvme: introduce helper function to get ctrl state - nvme: prevent potential spectre v1 gadget - arm64: dts: rockchip: Fix PCI node addresses on rk3399-gru - drm/amdgpu: Add NULL checks for function pointers - drm/exynos: fix a potential error pointer dereference - drm/exynos: fix a wrong error checking - hwmon: (corsair-psu) Fix probe when built-in - LoongArch: Preserve syscall nr across execve() - clk: rockchip: rk3568: Add PLL rate for 292.5MHz - clk: rockchip: rk3128: Fix HCLK_OTG gate register - jbd2: correct the printing of write_flags in jbd2_write_superblock() - jbd2: increase the journal IO's priority - drm/crtc: Fix uninit-value bug in drm_mode_setcrtc - neighbour: Don't let neigh_forced_gc() disable preemption for long - platform/x86: intel-vbtn: Fix missing tablet-mode-switch events - jbd2: fix soft lockup in
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Performing verification for mantic. I started two VMs. A jammy VM for the cifs server, and a mantic VM, for the client. I set the jammy VM up as per the testcase. I set the mantic VM up as per the testcase. The mantic VM uses kernel 6.5.0-25-generic from -updates. $ uname -rv 6.5.0-25-generic #25-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 7 14:58:39 UTC 2024 I created the testdata, took the SHA256 hash, and copied it over: $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt $ cp testdata.txt share/ On the jammy VM, I confirmed the hash has changed: $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt $ less sambashare/testdata.txt Looking at the file with less, I see the large blocks of zeros. I then enabled -proposed, and installed 6.5.0-27-generic: $ uname -rv 6.5.0-27-generic #28-Ubuntu SMP PREEMPT_DYNAMIC Thu Mar 7 18:21:00 UTC 2024 I mounted the share: $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.65/sambashare ~/share Password for ubuntu@//192.168.122.65/sambashare: Looking in dmesg, I see: [ 57.280020] Key type cifs.spnego registered [ 57.280035] Key type cifs.idmap registered [ 57.280345] Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers [ 57.280861] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers [ 57.281307] CIFS: VFS: wsize rounded down to 16384 to multiple of PAGE_SIZE 4096 [ 57.281599] CIFS: Attempting to mount //192.168.122.65/sambashare the line of interest is "CIFS: VFS: wsize rounded down to 16384 to multiple of PAGE_SIZE 4096" showing we now have a wsize of 16384, and we can see this with mount -l: $ mount -l //192.168.122.65/sambashare on /home/ubuntu/share type cifs (rw,relatime,vers=1.0,cache=strict,username=ubuntu,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.122.65,soft,unix,posixpaths,serverino,mapposix,acl,rsize=1048576,wsize=16384,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=1) I then copied the file over: $ cp testdata.txt share/ Looking at the SHA256 sum on the Jammy server: $ sha256sum sambashare/testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 sambashare/testdata.txt When looking at the file with less, there are no more binary zeros. The kernel in -proposed fixes the issue, happy to mark verified for mantic. ** Tags removed: verification-needed-mantic-linux ** Tags added: verification-done-mantic-linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: Fix Committed Status in linux source package in Mantic: Fix Committed Status in linux source package in Noble: Fix Committed Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
The fix for noble should be present in 6.8.0-16-generic and later. Marking as Fix committed for Noble. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: Fix Committed Status in linux source package in Mantic: Fix Committed Status in linux source package in Noble: Fix Committed Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing list discussion: https://lore.kernel.org/linux- cifs/6a65b2d1-7596-438a-8ade-2f7526b15...@rd10.de/T/#m22cd9b7289f87cd945978bd7995bcaf1beebfe67 To manage notifications about this bug go to:
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
** Changed in: linux (Ubuntu Noble) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: Fix Committed Status in linux source package in Mantic: Fix Committed Status in linux source package in Noble: Fix Committed Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing list discussion: https://lore.kernel.org/linux- cifs/6a65b2d1-7596-438a-8ade-2f7526b15...@rd10.de/T/#m22cd9b7289f87cd945978bd7995bcaf1beebfe67 To manage notifications about this bug go to:
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
This bug is awaiting verification that the linux/6.5.0-27.28 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-mantic-linux' to 'verification-done-mantic-linux'. If the problem still exists, change the tag 'verification-needed-mantic- linux' to 'verification-failed-mantic-linux'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: kernel-spammed-mantic-linux-v2 verification-needed-mantic-linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: Fix Committed Status in linux source package in Noble: In Progress Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Reviews from Senior Kernel Team Members: ACK 1 https://lists.ubuntu.com/archives/kernel-team/2024-February/149051.html ACK 2 https://lists.ubuntu.com/archives/kernel-team/2024-February/149061.html Applied https://lists.ubuntu.com/archives/kernel-team/2024-February/149119.html Will let you know when this gets tagged to a kernel release. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: Fix Committed Status in linux source package in Noble: In Progress Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing list
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
** Changed in: linux (Ubuntu Mantic) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: Fix Committed Status in linux source package in Noble: In Progress Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing list discussion: https://lore.kernel.org/linux- cifs/6a65b2d1-7596-438a-8ade-2f7526b15...@rd10.de/T/#m22cd9b7289f87cd945978bd7995bcaf1beebfe67 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2049634/+subscriptions
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Submitted the patch as SRU to mantic. Noble will pick it up when the kernel team pulls in 6.8-rc5. Cover letter: https://lists.ubuntu.com/archives/kernel-team/2024-February/149042.html Patch: https://lists.ubuntu.com/archives/kernel-team/2024-February/149043.html -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: BugLink: https://bugs.launchpad.net/bugs/2049634 [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. This destroys what you are copying, and the data corruption is completely silent. A workaround is to set wsize to a multiple of PAGE_SIZE. [Fix] This was fixed upstream in 6.8-rc5 by the following: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 This patch has been selected for upstream stable. The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] We are changing the wsize that the SMB server negotiates or the user explicitly sets on the mount command line to a safe value, if the asked for value was not a multiple of PAGE_SIZE. It is unlikely that any users will notice any difference. If the wsize happens to be rounded down to a significantly smaller number, there may be a small performance impact, e.g. you set wsize=8094 for some reason, it would round down to wsize=4096, and lead to double the writes. At least you have data integrity though. Most users default to wsize=65535, and will have no impact at all. Only those with very old smb 1.0 servers that negotiate down to 16850 will see any difference. If a regression were to occur, it could result in user's wsize being incorrectly set, leading to the underlying netfs/folios data corruption being triggered and causing data corruption. [Other info] The issue was introduced in 6.3-rc1 by: commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Author: David Howells Date: Mon Jan 24 21:13:24 2022 + Subject: cifs: Change the I/O paths to use an iterator rather than a page list Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 Upstream mailing list discussion: https://lore.kernel.org/linux-
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
** Description changed: + BugLink: https://bugs.launchpad.net/bugs/2049634 + [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto - a cifs smb 1.0 mount. + a cifs smb mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more - binary zeros. + binary zeros. This destroys what you are copying, and the data + corruption is completely silent. - A workaround is to increase wsize, but this only works for small files, - as any files larger than wsize will see the bug. + A workaround is to set wsize to a multiple of PAGE_SIZE. - Most users will want to use the 6.2 HWE kernel until this is fixed. + [Fix] + + This was fixed upstream in 6.8-rc5 by the following: + + commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 + Author: Steve French + Date: Tue Feb 6 16:34:22 2024 -0600 + Subject: smb: Fix regression in writes when non-standard maximum write size negotiated + Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 + + This patch has been selected for upstream stable. + + The patch looks at wsize negotiation from the server, and also what is set on the mount command line, and if not a multiple of PAGE_SIZE, rounds it down, taking care to not be 0. + The real corruption bug still exists in netfs/folios subsystems and we are looking for it still, but this solves the immediate data corruption issues on smb. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] + We are changing the wsize that the SMB server negotiates or the user + explicitly sets on the mount command line to a safe value, if the asked + for value was not a multiple of PAGE_SIZE. + + It is unlikely that any users will notice any difference. If the wsize + happens to be rounded down to a significantly smaller number, there may + be a small performance impact, e.g. you set wsize=8094 for some reason, + it would round down to wsize=4096, and lead to double the writes. At + least you have data integrity though. + + Most users default to wsize=65535, and will have no impact at all. Only + those with very old smb 1.0 servers that negotiate down to 16850 will + see any difference. + + If a regression were to occur, it could result in user's wsize being + incorrectly set, leading to the underlying netfs/folios data corruption + being triggered and causing data corruption. + [Other info] - Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline - 6.8-rc3. + The issue was introduced in 6.3-rc1 by: + + commit d08089f649a0cfb2099c8551ac47eef0cc23fdf2 + Author: David Howells + Date: Mon Jan 24 21:13:24 2022 + + Subject: cifs: Change the I/O paths to use an iterator rather than a page list + Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d08089f649a0cfb2099c8551ac47eef0cc23fdf2 + + Upstream mailing list discussion: + + https://lore.kernel.org/linux- + cifs/6a65b2d1-7596-438a-8ade-2f7526b15...@rd10.de/T/#m22cd9b7289f87cd945978bd7995bcaf1beebfe67 ** Tags added: mantic noble -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Hi R. Diez, The patch was merged upstream in 6.8-rc5: commit 4860abb91f3d7fbaf8147d54782149bb1fc45892 Author: Steve French Date: Tue Feb 6 16:34:22 2024 -0600 Subject: smb: Fix regression in writes when non-standard maximum write size negotiated Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4860abb91f3d7fbaf8147d54782149bb1fc45892 I'll work on getting this SRU'd into Ubuntu kernels. Thanks, Matthew -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] [Other info] Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline 6.8-rc3. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2049634/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
The patch does seem like a step forwards, but the warning messages are still easy to miss and misleading. I would rather abort the connection than risking a non-functional connection (in the case of the round-up I described with PAGE_SIZE = 64 KiB). I do not think it is a good idea either to modify the user's wsize value (if specified) and carry on: if the system cannot fulfill the user's request due to a temporary software bug, I would rather abort and state the truth. In order to test the new patch, I would have to devote more of my private time again, and this feels like an uphill battle with diminishing returns. You (or the cifs developers) should be able to automate tests simulating an old Windows Server with smbd by setting "max xmit" in smb.conf, for example, to an unaligned value just below the current PAGE_SIZE. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] [Other info] Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline 6.8-rc3. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2049634/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Hi R. Diez, Steve posted a new patch [1] that also rounds the wsize down when specified on the mount command line, just as we wanted: [1] https://lore.kernel.org/linux- cifs/CAH2r5mvPz2CUyKDZv_9fYGu=9L=3uime7xajgbbu+if8ch8...@mail.gmail.com/ I tested it, and it works great. $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ mount -l //192.168.122.172/sambashare on /home/ubuntu/share type cifs (rw,relatime,vers=1.0,cache=strict,username=ubuntu,uid=0,noforceuid,gid=0,noforcegid, addr=192.168.122.172,soft,unix,posixpaths,serverino,mapposix,acl,rsize=1048576,wsize=16384,bsize=1048576,retrans=1,echo_interval=60,actimeo=1,closetimeo=1) $ sudo dmesg | tail [ 48.767560] Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers [ 48.768399] CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers [ 48.769427] CIFS: VFS: wsize rounded down to 16384 to multiple of PAGE_SIZE 4096 [ 48.770069] CIFS: Attempting to mount //192.168.122.172/sambashare I no longer get data corruption when specifying wsize on the mount command line. I built the new patch ontop of 6.5.0-15-generic again for you to try, if you want. Please note this package is NOT SUPPORTED by Canonical, and is for TESTING PURPOSES ONLY. ONLY Install in a dedicated test environment. You probably have my ppa already in place, you can just do a: $ sudo apt update $ sudo apt install linux-image-unsigned-6.5.0-15-generic linux-modules-6.5.0-15-generic linux-modules-extra-6.5.0-15-generic linux-headers-6.5.0-15-generic $ sudo reboot $ uname -rv 6.5.0-15-generic #15~22.04.1+TEST2049634v20240216b1-Ubuntu SMP PREEMPT_DYNAMIC Fr Note the date change to 20240216. I kept using 6.5.0-15-generic over -17 due to some other bugs I have been debugging on -17 that I am looking into. Anyway, please test the new kernel, and try mount your smb1 share, and paste back the last lines of dmesg with the CIFS: strings, so we can see the round down take place. Feel free to write back to Steve on linux-cifs too. Hopefully we get this patch merged into upstream, and then I'll make sure we get this into the next Ubuntu SRU cycle. Thanks, Matthew -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] [Other info] Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline 6.8-rc3. To manage notifications about this bug go
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
I had some difficulty with the test kernel, because the installation command wanted to remove the running kernel, so I got a warning about it. I think there was some kind of conflict with the normal 6.5.0-15 kernel. In the meantime, the kernel has been upgraded to 6.5.0-17, so I removed all older versions and tried again, and I think I managed to do it in the end: $ uname -a Linux rdiez-L2017 6.5.0-15-generic #15~22.04.1+TEST2049634v20240208b1-Ubuntu SMP PREEMPT_DYNAMIC Th x86_64 x86_64 x86_64 GNU/Linux This is the output from "journalctl --dmesg --follow": Feb 11 12:31:25 rdiez-L2017 kernel: FS-Cache: Loaded Feb 11 12:31:25 rdiez-L2017 kernel: Key type cifs.spnego registered Feb 11 12:31:25 rdiez-L2017 kernel: Key type cifs.idmap registered Feb 11 12:31:25 rdiez-L2017 kernel: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers Feb 11 12:31:25 rdiez-L2017 kernel: CIFS: VFS: Use of the less secure dialect vers=1.0 is not recommended unless required for access to very old servers Feb 11 12:31:25 rdiez-L2017 kernel: CIFS: Attempting to mount // This is the output from mount -l: // on /home/rdiez/MountPoints/ type cifs (rw,noexec,relatime,vers=1.0,cache=strict,username=,domain=,uid=1000,noforceuid,gid=0,noforcegid,addr=,file_mode=0666,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=61440,wsize=16384,bsize=1048576,echo_interval=4,actimeo=1,closetimeo=1,user=) I didn't specify "wsize" (I never did), so it looks like it has automatically negotiated the wsize down to 16 KiB for that SMB 1.0 connection. I then copied my test text file to and from the server, and there was no data corruption this time. So the patch is a step forwards. I am still concerned though that the work-around the patch implements is needlessly unreliable, misleading and risks data corruption at the smallest user's mistake, despite repeated reasoning in the mailing list. I hope the kernel cifs guys find the real bug and fix it properly, so that this patch can be reverted soon. By the way, during data transfer, I also get many errors like this: Feb 11 12:41:01 rdiez-L2017 kernel: CIFS: VFS: SMB signature verification returned error = -13 Feb 11 12:41:11 rdiez-L2017 kernel: CIFS: VFS: SMB signature verification returned error = -13 ... I looked more carefully this time, and it happens when I read data back from the server. It looks like there is one such warning per read block, according to the negotiated rsize. But these errors are "normal" over the years. I am guessing that the SMB 1.0 support is not very well polished. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Thanks, I'll test the new kernel over the weekend. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] [Other info] Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline 6.8-rc3. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2049634/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
Hi R. Diez, You have probably been following the chatter on the upstream mailing list discussion. I initially thought the patch didn't fix the issue, as when I mount with wsize=16850, the issue still occurs [1], but it seems that the intent of the patch is to only correct when the server specifies an incorrect wsize, and to correct that instead [2]. [1] https://lore.kernel.org/linux-cifs/CAKAwkKuJvFDFG7=bcymj0jdmmhytlunygdueaubtoctbnqt...@mail.gmail.com/ [2] https://lore.kernel.org/linux-cifs/CAH2r5mt9gPhUSka56yk28+nksw7=LPuS4VAMzGQyJEOfcpOc=g...@mail.gmail.com/ Anyway, I have built you a test kernel that includes the patch, not from today, but yesterday's corrected one [3] ontop of 6.5.0-15-generic for Jammy HWE. [3] https://lore.kernel.org/linux- cifs/cah2r5mvzyxp7vhqvct6iep4nmxdaz2uqtt7g4yrxcvobkv_...@mail.gmail.com/ If you want to try it out on your system before it gets merged upstream, please do, since your SMB server sends a incorrect wsize. Please note this package is NOT SUPPORTED by Canonical, and is for TESTING PURPOSES ONLY. ONLY Install in a dedicated test environment. Instructions to install (On a Jammy system): 1) sudo add-apt-repository ppa:mruffell/lp2049634-test 2) sudo apt update 3) sudo apt install linux-image-unsigned-6.5.0-15-generic linux-modules-6.5.0-15-generic linux-modules-extra-6.5.0-15-generic linux-headers-6.5.0-15-generic 4) sudo reboot 5) uname -rv 6.5.0-15-generic #15~22.04.1+TEST2049634v20240208b1-Ubuntu SMP PREEMPT_DYNAMIC Th Perform your smb1 mount but without any wsize set, and then run: $ sudo dmesg | tail or just look at the end of dmesg in journalctl, and could you please 1) send the output of the lines with "CIFS:" at the start. I just want to see if the strings "CIFS: VFS: wsize should be a multiple of 4096 (PAGE_SIZE)" and "CIFS: VFS: wsize too small, reset to minimum ie PAGE_SIZE, usually 4096" are there. and: 2) see if the issue is actually fixed by doing a test copy and looking with less. In the meantime I will ask Steve if its worth just setting all wsize to multiples of PAGE_SIZE regardless how it is set, since a partial one on the mount command line will destroy your data anyway, and thats not useful. Thanks! Matthew -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] [Other info] Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline 6.8-rc3. To manage notifications about this bug go to:
[Kernel-packages] [Bug 2049634] Re: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data
** Summary changed: - smb1: wsize blocks of bytes followed with binary zeros on copy, destroying data + smb: wsize blocks of bytes followed with binary zeros on copy, destroying data -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2049634 Title: smb: wsize blocks of bytes followed with binary zeros on copy, destroying data Status in linux package in Ubuntu: In Progress Status in linux source package in Mantic: In Progress Status in linux source package in Noble: In Progress Bug description: [Impact] Upon installing the 6.5 HWE kernel on Jammy, users with a custom wsize set will see data destruction when copying files from their systems onto a cifs smb 1.0 mount. wsize defaults to 65535 bytes, but when set to smaller values, like 16850, users will see blocks of 16850 bytes copied over, followed by 3900 binary zeros, followed by the next block of data followed by more binary zeros. A workaround is to increase wsize, but this only works for small files, as any files larger than wsize will see the bug. Most users will want to use the 6.2 HWE kernel until this is fixed. [Testcase] Start two VMs, one for the server, and the other, the client. Server -- $ sudo apt update $ sudo apt upgrade $ sudo apt install samba $ sudo vim /etc/samba/smb.conf server min protocol = NT1 [sambashare] comment = Samba on Ubuntu path = /home/ubuntu/sambashare read only = no browsable = yes $ mkdir ~/sambashare $ sudo smbpasswd -a ubuntu Client -- $ sudo apt update $ sudo apt install cifs-utils $ mkdir ~/share $ sudo mount -t cifs -o username=ubuntu,vers=1.0,wsize=16850 //192.168.122.172/sambashare ~/share $ ( set -o pipefail && head --bytes=$(( 55 * 1000 )) /dev/zero | openssl enc -aes-128-ctr -nosalt -pass "pass:my-seed" -iter 1 | hexdump --no-squeezing --format '40/1 "%02x"' --format '"\n"' >"testdata.txt" ) $ sha256sum testdata.txt 9ec09af020dce3270ea76531141940106f173c7243b7193a553480fb8500b3f2 testdata.txt Now copy the file to the share. Client -- $ cp testdata.txt share/ Server -- $ sha256sum sambashare/testdata.txt 9e573a0aa795f9cd4de4ac684a1c056dbc7d2ba5494d02e71b6225ff5f0fd866 sambashare/testdata.txt The SHA256 hash is different. If you view the file with less, you will find a block of wsize=16850 bytes, then 3900 bytes of binary zeros, followed by another wsize=16850 bytes, then 3900 bytes of binary zeros, etc. An example of a broken file is: https://launchpadlibrarian.net/712573213/testdata-back-from-server.txt [Where problems could occur] [Other info] Currently bisecting. Introduced in 6.3-rc1. Currently broken on mainline 6.8-rc3. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2049634/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
