[Bug 2071474] Re: fsck.xfs doesn't call xfs_repair in force mode from systemd unit during the system startup
** Description changed: Ubuntu 20.04.6 LTS xfsprogs 5.3.0-1ubuntu2 According to the fsck.xfs(8) man page [1] an administrator can force xfs_repair on every boot by adding the "fsck.mode=force" to the kernel boot options. But it doesn't work. The culprit is this check in the xfs_fsck.sh file [2]. This is one of the checks that is supposed to prevent running the fsck.xfs tool in the force mode from the interactive session but apparently it also blocks it's execution from the systemd unit. According to the systemd-fsck@.service man page [3] a separate service is instantiated for each device that is configured for file system checking. And that service calls /lib/systemd/systemd-fsck (/usr/lib/systemd/systemd-fsck) which is a part of the systemd package. This in turn is built from the fsck/fsck.c source [4]. The tool calls fsck.xfs with -f if the kernel command line fsck.mode=force is present [5][6]. Steps to reproduce: 1. install Ubuntu Focal with two disks (one for the system, the other for test xfs partition) 2. create partition and xfs file system on the second drive (in my case it is /dev/vdb (the disk), /dev/vdb1 (the xfs partition)) 3. create mount point dir (ie, /data) and add appropriate line to the /etc/fstab, ie: /dev/vdb1 /data xfs defaults 0 2 - 4. sudo mount --all to test the above, next sudo umount /data + 4. sudo mount --all to test the above, next: sudo umount /data 5. break the xfs fs on /dev/vdb1: sudo xfs_db -x -c blockget -c "blocktrash -s 512109 -n 1000" /dev/vdb1 6. verify it's broken: sudo mount --all 7. add the "fsck.mode=force" to the kernel boot option in grub (/etc/default/grub): GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0 fsck.mode=force" 8. sudo update-grub 9. sudo reboot; # watch console during the startup for messages similar to the following: - Starting File System Check on /dev/vdb1... + Starting File System Check on /dev/vdb1... [ OK ] Started File System Check Daemon to report status. [ OK ] Finished File System Check on /dev/vdb1. - Mounting /data... + Mounting /data... [2.592015] XFS (vdb1): Metadata CRC error detected at xfs_inobt_read_verify+0x1a/0xc0 [xfs], xfs_inobt block 0x8 [2.593023] XFS (vdb1): Unmount and run xfs_repair [2.593579] XFS (vdb1): First 128 bytes of corrupted metadata buffer: [2.594195] : b9 be bd cc 0f 00 00 01 ff ff ff ff ff ff ff ff [2.596159] 0010: 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00 00 [2.596929] 0020: 70 ef 5b b8 a3 e5 4e 12 a0 f3 45 c9 0a 20 11 3f p.[...N...E.. .? [2.597807] 0030: 00 00 00 00 28 a7 30 1d 00 00 00 80 00 00 40 3d (.0...@= [2.598661] 0040: ff ff ff ff ff ff ff f8 00 00 00 00 00 00 00 00 [2.599451] 0050: 00 00 00 00 00 00 00 00 00 c0 ff 07 00 00 00 00 [2.600244] 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [2.601034] 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [2.601877] XFS (vdb1): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x8 len 8 error 74 [FAILED] Failed to mount /data. 10. comment out lines 61-63 in the /sbin/fsck.xfs: diff /sbin/fsck.xfs ~/fsck.xfs 61,63c61,63 < #if [ -n "$PS1" -o -t 0 ]; then < # FORCE=false < #fi --- > if [ -n "$PS1" -o -t 0 ]; then > FORCE=false > fi 11. repeat step 9. (reboot). The system will repair /dev/vdb1 and mount it automatically: - Starting File System Check on /dev/vdb1... + Starting File System Check on /dev/vdb1... [ OK ] Started File System Check Daemon to report status. [ OK ] Mounted Mount unit for core20, revision 2318. [ OK ] Mounted Mount unit for lxd, revision 24061. [ OK ] Mounted Mount unit for snapd, revision 21759. [ OK ] Finished File System Check on /dev/vdb1. [ OK ] Reached target Mounted snaps. - Mounting /data... + Mounting /data... [ OK ] Mounted /data. journalctl proves that xfs_repair run on the disk: date; journalctl --no-pager -u systemd-fsck@dev-vdb1.service --since "5 minutes ago" Fri Jun 28 14:37:19 UTC 2024 -- Logs begin at Fri 2024-06-28 07:53:49 UTC, end at Fri 2024-06-28 14:36:24 UTC. -- Jun 28 14:34:12 xfs systemd[1]: systemd-fsck@dev-vdb1.service: Succeeded. Jun 28 14:34:12 xfs systemd[1]: Stopped File System Check on /dev/vdb1. -- Reboot -- Jun 28 14:34:28 xfs systemd[1]: Starting File System Check on /dev/vdb1... Jun 28 14:34:28 xfs systemd-fsck[564]: Phase 1 - find and verify superblock... Jun 28 14:34:28 xfs systemd-fsck[564]: Phase 2 - using internal log Jun 28 14:34:28 xfs systemd-fsck[564]: - zero log... Jun 28 14:34:28 xfs systemd-fsck[564]: - scan filesystem freespace and inode maps... Jun 28 14:34:28 xfs
[Bug 2071474] [NEW] fsck.xfs doesn't call xfs_repair in force mode from systemd unit during the system startup
Public bug reported: Ubuntu 20.04.6 LTS xfsprogs 5.3.0-1ubuntu2 According to the fsck.xfs(8) man page [1] an administrator can force xfs_repair on every boot by adding the "fsck.mode=force" to the kernel boot options. But it doesn't work. The culprit is this check in the xfs_fsck.sh file [2]. This is one of the checks that is supposed to prevent running the fsck.xfs tool in the force mode from the interactive session but apparently it also blocks it's execution from the systemd unit. According to the systemd-fsck@.service man page [3] a separate service is instantiated for each device that is configured for file system checking. And that service calls /lib/systemd/systemd-fsck (/usr/lib/systemd/systemd-fsck) which is a part of the systemd package. This in turn is built from the fsck/fsck.c source [4]. The tool calls fsck.xfs with -f if the kernel command line fsck.mode=force is present [5][6]. Steps to reproduce: 1. install Ubuntu Focal with two disks (one for the system, the other for test xfs partition) 2. create partition and xfs file system on the second drive (in my case it is /dev/vdb (the disk), /dev/vdb1 (the xfs partition)) 3. create mount point dir (ie, /data) and add appropriate line to the /etc/fstab, ie: /dev/vdb1 /data xfs defaults 0 2 4. sudo mount --all to test the above, next sudo umount /data 5. break the xfs fs on /dev/vdb1: sudo xfs_db -x -c blockget -c "blocktrash -s 512109 -n 1000" /dev/vdb1 6. verify it's broken: sudo mount --all 7. add the "fsck.mode=force" to the kernel boot option in grub (/etc/default/grub): GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0 fsck.mode=force" 8. sudo update-grub 9. sudo reboot; # watch console during the startup for messages similar to the following: Starting File System Check on /dev/vdb1... [ OK ] Started File System Check Daemon to report status. [ OK ] Finished File System Check on /dev/vdb1. Mounting /data... [2.592015] XFS (vdb1): Metadata CRC error detected at xfs_inobt_read_verify+0x1a/0xc0 [xfs], xfs_inobt block 0x8 [2.593023] XFS (vdb1): Unmount and run xfs_repair [2.593579] XFS (vdb1): First 128 bytes of corrupted metadata buffer: [2.594195] : b9 be bd cc 0f 00 00 01 ff ff ff ff ff ff ff ff [2.596159] 0010: 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00 00 [2.596929] 0020: 70 ef 5b b8 a3 e5 4e 12 a0 f3 45 c9 0a 20 11 3f p.[...N...E.. .? [2.597807] 0030: 00 00 00 00 28 a7 30 1d 00 00 00 80 00 00 40 3d (.0...@= [2.598661] 0040: ff ff ff ff ff ff ff f8 00 00 00 00 00 00 00 00 [2.599451] 0050: 00 00 00 00 00 00 00 00 00 c0 ff 07 00 00 00 00 [2.600244] 0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [2.601034] 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [2.601877] XFS (vdb1): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x8 len 8 error 74 [FAILED] Failed to mount /data. 10. comment out lines 61-63 in the /sbin/fsck.xfs: diff /sbin/fsck.xfs ~/fsck.xfs 61,63c61,63 < #if [ -n "$PS1" -o -t 0 ]; then < # FORCE=false < #fi --- > if [ -n "$PS1" -o -t 0 ]; then > FORCE=false > fi 11. repeat step 9. (reboot). The system will repair /dev/vdb1 and mount it automatically: Starting File System Check on /dev/vdb1... [ OK ] Started File System Check Daemon to report status. [ OK ] Mounted Mount unit for core20, revision 2318. [ OK ] Mounted Mount unit for lxd, revision 24061. [ OK ] Mounted Mount unit for snapd, revision 21759. [ OK ] Finished File System Check on /dev/vdb1. [ OK ] Reached target Mounted snaps. Mounting /data... [ OK ] Mounted /data. journalctl proves that xfs_repair run on the disk: date; journalctl --no-pager -u systemd-fsck@dev-vdb1.service --since "5 minutes ago" Fri Jun 28 14:37:19 UTC 2024 -- Logs begin at Fri 2024-06-28 07:53:49 UTC, end at Fri 2024-06-28 14:36:24 UTC. -- Jun 28 14:34:12 xfs systemd[1]: systemd-fsck@dev-vdb1.service: Succeeded. Jun 28 14:34:12 xfs systemd[1]: Stopped File System Check on /dev/vdb1. -- Reboot -- Jun 28 14:34:28 xfs systemd[1]: Starting File System Check on /dev/vdb1... Jun 28 14:34:28 xfs systemd-fsck[564]: Phase 1 - find and verify superblock... Jun 28 14:34:28 xfs systemd-fsck[564]: Phase 2 - using internal log Jun 28 14:34:28 xfs systemd-fsck[564]: - zero log... Jun 28 14:34:28 xfs systemd-fsck[564]: - scan filesystem freespace and inode maps... Jun 28 14:34:28 xfs systemd-fsck[564]: Metadata CRC error detected at 0x564c48e69d0d, xfs_bnobt block 0x20/0x1000 Jun 28 14:34:28 xfs systemd-fsck[564]: btree block 0/4 is suspect, error -74 . Jun 28 14:34:28 xfs systemd-fsck[564]: UUID mismatch on inode 191 Jun 28 14:34:28 xfs systemd-fsck[564]: cleared inode 191 Jun 28 14:34:28 xfs systemd-fsck[564]: - agno = 1 Jun 28 14:34:28 xfs systemd-fsck[564]: -
[Bug 820896] Re: freshclam ERROR: Database load killed by signal 11
*** This bug is a duplicate of bug 766519 *** https://bugs.launchpad.net/bugs/766519 Same problem here: # catchsegv freshclam ClamAV update process started at Sat Mar 10 21:59:36 2012 WARNING: Your ClamAV installation is OUTDATED! WARNING: Local version: 0.96.5 Recommended version: 0.97.3 DON'T PANIC! Read http://www.clamav.net/support/faq main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven) daily.cld is up to date (version: 14627, sigs: 114302, f-level: 63, builder: guitar) connect_error: getsockopt(SO_ERROR): fd=4 error=110: Connection timed out Can't connect to port 80 of host db.local.clamav.net (IP: 188.40.42.237) Downloading bytecode.cvd [100%] [LibClamAV] *** [LibClamAV] *** This version of the ClamAV engine is outdated. *** [LibClamAV] *** DON'T PANIC! Read http://www.clamav.net/support/faq *** [LibClamAV] *** ERROR: During database load : *** Segmentation fault [...] ff60-ff601000 r-xp 00:00 0 [vsyscall] ERROR: Database load killed by signal 11 ERROR: Failed to load new database: No viruses detected cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Xeon(R) CPU E31245 @ 3.30GHz stepping: 7 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings: 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid bogomips: 6600.23 clflush size: 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to clamav in Ubuntu. https://bugs.launchpad.net/bugs/820896 Title: freshclam ERROR: Database load killed by signal 11 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/clamav/+bug/820896/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 820896] Re: freshclam ERROR: Database load killed by signal 11
*** This bug is a duplicate of bug 766519 *** https://bugs.launchpad.net/bugs/766519 Same problem here: # catchsegv freshclam ClamAV update process started at Sat Mar 10 21:59:36 2012 WARNING: Your ClamAV installation is OUTDATED! WARNING: Local version: 0.96.5 Recommended version: 0.97.3 DON'T PANIC! Read http://www.clamav.net/support/faq main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven) daily.cld is up to date (version: 14627, sigs: 114302, f-level: 63, builder: guitar) connect_error: getsockopt(SO_ERROR): fd=4 error=110: Connection timed out Can't connect to port 80 of host db.local.clamav.net (IP: 188.40.42.237) Downloading bytecode.cvd [100%] [LibClamAV] *** [LibClamAV] *** This version of the ClamAV engine is outdated. *** [LibClamAV] *** DON'T PANIC! Read http://www.clamav.net/support/faq *** [LibClamAV] *** ERROR: During database load : *** Segmentation fault [...] ff60-ff601000 r-xp 00:00 0 [vsyscall] ERROR: Database load killed by signal 11 ERROR: Failed to load new database: No viruses detected cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Xeon(R) CPU E31245 @ 3.30GHz stepping: 7 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings: 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid bogomips: 6600.23 clflush size: 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/820896 Title: freshclam ERROR: Database load killed by signal 11 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/clamav/+bug/820896/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
